We often conflate being data-driven with being dashboard-driven, a distinction that is rarely trivial. Modern strategy has become a performance of accountability, where we prioritize the metrics that are convenient to audit over the signals that are actually revealing. Impressions and conversion rates offer a seduction of control; they satisfy the demand for certainty even when they fail to explain the outcome.
I felt this friction most acutely while working on a series of campaigns for a Black hair care line at Pantene. My mandate was clear: use the quantitative data to drive growth. And on paper, the strategy worked. The dashboard showed the kind of “market penetration” and high-velocity spend that makes for a comfortable quarterly review. But as I executed the work, the disconnect became impossible to ignore. We weren’t actually engaging with these women; we were just catering to a version of them that fit into a global supply chain.
The data was warped by its own narrowness. By chasing a mass-market “average,” the campaigns ended up flattening the specific, textured rituals of Black hair care just to hit a defensible number. It was a success of marketing, but a failure of resonance. Because the brand was built on those surface-level metrics rather than actual conversations about need, it lacked any real connective tissue with the community. It worked for a minute, and the launch looked like a breakthrough, but it wasn’t sustainable. Eventually, that lack of depth caught up with the line, and it struggled to survive long-term. You can’t build a lasting brand on a dashboard; you build it on trust, and the numbers alone couldn’t do the work to earn it.
We sideline qualitative insight not because it’s “soft,” but because it’s inconvenient. It resists the kind of clean packaging that corporate reporting demands. You can’t fit a customer’s hesitation into a cell on a spreadsheet, and there is no automated dashboard for nuance.
In most organizations, the systems are literally built to ignore the human signal. Reporting structures prioritize what can be audited, meaning open-ended feedback is usually skimmed for “sentiment” and then discarded, while interview transcripts sit in folders, unread and un-synthesized. We treat these insights as anecdotes: unofficial, unactionable, and essentially invisible to leadership.
This is a structural failure, not a lack of time. Most decision-making environments were designed without a formal path for ambiguity. Without a rigorous process to turn a “good story” into a repeatable decision, even the most vital human context gets sidelined. We aren’t just choosing the wrong data; we’re intentionally building shallower strategies because they’re easier to explain in a performance review.
Qualitative insight isn’t just a supplement; it’s the only way to find the friction that a dashboard is designed to smooth over. It’s where you find the gap between what you intended to build and how it’s actually being received.
I saw this clearly in my recent work with Chinatown Power, a clean energy initiative in Boston. Initially, the quantitative data suggested a standard outreach problem: low visibility. The numbers showed awareness was clustered among a few community leaders, while the rest of the neighborhood was essentially a blank slate. Naturally, my first instinct was a polished marketing website to “raise recognition.” It was a textbook response to a textbook metric.
But when I actually got into the field, the “visibility” problem evaporated. The residents didn’t need a marketing campaign; they needed an interpreter. During interviews, people admitted they couldn’t even read their own utility bills, and they were essentially being locked out of their own energy decisions by complex jargon. I talked to program managers who spent their entire days walking residents line-by-line through those bills just to establish a baseline of trust.
That insight killed the marketing showcase. We pivoted the entire website strategy to a community knowledge hub with plain-language guides and resources designed to help people navigate a system that felt like a labyrinth. Rather than a “tactical adjustment”, it was a total reframing of what the organization actually did for the community.
These moments don’t happen on a dashboard. They happen when you stop looking at numerical trends as the final answer and start using them as a reason to ask better questions. Without a rigorous way to capture these field insights, you’re just guessing with high-resolution data. You don’t need a “good story”, you need a process for turning human confusion into a design requirement.
If you want to turn qualitative notes into something a board will actually respect, you have to stop treating them as “stories” and start treating them as a dataset. It’s about rigor, not just listening.
I usually start by identifying the “knowledge gaps” where the dashboard is silent. If I have the time, I’ll run semi-structured Interviews; the key isn’t a rigid script, but having a few core questions and the discipline to let the participant wander. If you’re stuck on phrasing, I usually point teams toward IDEO’s Design Kit (page 39 is a great starting point for open-ended questions). And when I’m in a rush, like during the Pantene campaigns, I’ll use intercept interviews for a five-minute gut check in a lobby or at an event. CustomerIQ’s guide is a solid reference for how to recruit people without being intrusive.
When I need to see how a community talks to each other, rather than just to me, I’ll use focus groups. They are great for uncovering shared language or seeing where points of disagreement actually live in a social context. But they are also a minefield, so you need a facilitator who can shut down the one person trying to dominate the room and make space for the quieter, more honest perspectives. If you’re running these for the first time, The Nielsen Norman Group’s focus groups 101 is a good checklist for avoiding the “groupthink” trap.
Once you’ve gathered the raw data, you have to resist the urge to only pick the “best” quotes. The unglamorous part is coding: tagging recurring emotional cues across your transcripts using tools like Dedoose or NVivo. This turns a mess of anecdotes into a defensible dataset where you can prove to a data-driven team that “confusion” isn’t just a story, it’s a statistically significant pattern.
But the biggest mistake is stopping at the “insight.” If your research says people are confused, your next task is to find out what actually provides clarity. I use rapid prototyping as a way to put a low-fidelity solution, like a rewritten FAQ or a one-page Figma mockup, back in front of the community.
This isn’t about aesthetic design, but evaluative research. By letting a participant interact with a potential fix, I can run comprehension checks in real-time. I don’t ask if they “like” the new version; I use Lyssna or Typeform to see if they can explain the concept back to me. If they can’t, the strategy has failed, and it doesn’t matter how “clean” the quantitative data looks. It’s the only way to move from “I think this is the problem” to “I know this is the solution”.
The real challenge isn’t the research itself, but stopping it from evaporating the second you walk out of the room. In a culture built on quarterly targets, qualitative work is often treated like a “special project”: it’s interesting for a week, then it gets buried the moment a deadline hits.
To make this stick, you have to anchor it into the actual bureaucracy of the company. It means moving beyond “nice-to-have” and making human context a mandatory part of the conversation. I’ve found that the only way to do this is to change the requirements of the meeting.
If a new proposal doesn’t include a specific user friction point alongside the spreadsheet data, it should be sent back for more work. We shouldn’t be accepting “clean” numbers that don’t have a human rationale attached to them. It also means carving out a “repeatable pulse”: dedicating five minutes in every weekly standup to share one raw moment of user confusion. This keeps the human layer from feeling like a special event and turns it into a constant baseline.
Ultimately, culture follows the budget. If you aren’t explicitly allocating resources for participant incentives and research tools, your team will default to the metrics they can get for free, even if those metrics are hollow. Leaders have to be okay with the fact that qualitative data is “inconvenient” and slow. If you only push for immediate, automated fixes, you’re just managing a version of the customer that fits your dashboard. True impact happens when you have the courage to discuss the nuances that a spreadsheet can’t capture.
We settle for “surface-level” strategy because it’s a lot easier to defend in a meeting. A dashboard is a shield; a story about a frustrated customer is a responsibility. But as I saw at Pantene, a brand built on defensible numbers alone has a shelf life. It might hit its launch targets, but it will eventually lose its “connective tissue” and fail.
This isn’t a luxury. Whether you’re trying to understand the energy needs of a neighborhood like Chinatown or the hair care rituals of a community you don’t belong to, the goal is the same: stop catering to a spreadsheet. Start engaging with the person who actually exists. The data will tell you if you’re moving, but only the people will tell you if you’re actually going anywhere worth the trip.
Substack Newsletter
On Substack, I write about design as a tool for change – from participatory methods and civic tech to alternative economies and community storytelling. Each piece explores how we move beyond profit-driven systems toward clarity, equity, and impact.
