Progressive Sample Selection (PSS): A Methodology for Optimizing Qualitative Insights

Identifying challenges for ethnographic and qualitative approaches

At Northstar research partners, we are committed to the relentless pursuit of progress in the way we approach unearthing Insights that Inspire. With regard to human and cultural insights generation, approaches and methodologies are typically ethnographic in nature. These projects are typically rooted in an anthropological ethos that seeks to explore the rich, cultural context that shapes and influences consumer behavior. They also look at sociological and psychological factors that have an impact on consumerism.

With such a robust and potentially infinite set of variables, sometimes a major challenge is in discerning how to approach data collection so you can both define what constitutes data as well as how to generate it. Another key issue is also related to identifying patterns in the data.

In the majority of ethnographic projects, although a good amount of time might be spent observing context and interacting with participants, often pattern identification is a more intuitive exercise, as participant samples are typically smaller and it’s difficult to quantify context unless significant rigor exits around data classification and data entry / analysis.

And if we look at purely qualitative projects, which typically utilize a more question-response based approach to collecting and contextualizing data, in absence of spending more time with research participants, much richness can be left unearthed under the surface.

And this is not to say that Insights aren’t meaningful if they don’t have large sample sizes and statistical significance. Nor is it to say that there isn’t significant value in understanding patterns from the lowest hanging fruit on the surface.

But, in the space of consumer research, one can observe a fair amount of anxiety on the client side in determining the best type of research for guiding and backing up strategic decisions. Often budgets are stretched and the myth of economies of scale leads to use of a qualitative or quantitative methodologies in a “silo”. Also, while human and cultural insights are absolutely critical for driving business growth that will be sustainable over time, much of the time, the C Suite decision makers need to feel that the research results are driven not just by gut instinct and keen interpretation of surface patterns, but by analytical rigor as well. It is an easy but often-erroneous presumption to make that qualitative analysis is not “rigorous”

Introducing PSS: Progressive Sample Selection
Progressive Sample Selection is a Northstar process that enables us to efficiently select the “best” respondents through a multi-stage qualitative exploration

This qualitative approach begins with tasks assigned to a large sample of candidates.

These tasks completed by this larger pool of candidates produce a broad range of data outputs from which we identify patterns and themes. Next, we query those candidates we deem the best equipped to provide further depth and insight into these identifies phenomena.

While qualitative research typically seeks to uncover in-depth commentary from a specified sample and that is where it ends, Progressive Sample Selection achieves depth and fine-tunes focus throughout the process by both deepening the conversation, but also by progressively selecting those candidates best able to shed light on emerging findings and issues as they arise in the overall investigation.

How does PSS optimize qualitative and ethnographic research?

Progressive Sample selection serves a couple several “masters”.

First, It elevates qualitative research to a broader, ethnographic purview by allowing-in data that is ethnographic in nature, generated by participants.

Second, in the space of qualitative and ethnographic research, it also elevates the quality of the research participant sample in the more in-depth sample…by seeding thoughtfulness / preparation in participants, alleviating peer-generated bias that may happen in a focus group setting by more outspoken participants. It also allows for more in depth screening on creativity and articulateness so the right respondents can be selected for either qualitative or further in-context ethnographic deep dives.

Third, it supports efficient rigor in collection and analysis of qualitative and ethnographic data: providing a structure for data collection and analysis through coding of pre-task responses and outlining of specific data collection-points against each strategic research objective. The rationale: in order to properly scope out a PSS project, you must first identify which data source each specific objective will be covered by and how (pre-task, focus group discussion capture, ethnographic photo capture, etc.)…Allowing for checkpoints for coding data throughout the qualitative process. Also, by letting your research participants be informants from the outset, they are doing the bulk of the initial data generation in the pre-task assignment, which is (by design) the most robust data set. It also, again, allows the research team leader extra exposure to qualifying characteristics of the respondents in order to select the best participants for subsequent stages.

Finally, if the initial sample is large enough, some identified patterns can even be quantified. I like to call this “quantilative” research. With an initial sample of 100 to 200 respondents, one can identify legitimately quantifiable patterns from the pre-tasks, depending on the sample distribution.

A fictional example of PSS in practice

Lets say ACME widgets wants to identify the value and purchase drivers of their long-standing line of wonder-widgets among a couple of consumer segments (from an existing segmentation): one that is a sizeable and loyal customer base and one that is a psychographic fit but somehow not adopting the franchise in the volume ACME hypothesizes they ought to be. The objective of the project is to prioritize product line enhancements, changes and price-tiers that will exceed the expectations of their loyal consumers and get them to buy higher-priced wonder-widgets, as well as to seed relevance and encourage purchase consideration among the prospect group.

The PSS sample might start with 30 participants in each target segment group, in each of 3 priority ACME markets. Each of those 180 participants will be given a week to complete an ethnographic homework assignment that might involve a photo journal of their widget use, a widget-brand collage with accompanying narrative to explain the collage content, etc. From there, the research team would enter data and do an initial analysis to identify patterns, then find a selection of participants who represent those patterns most accurately and / or are the most articulate, thoughtful or diligent in completing their assignment…giving them an elevated right to a point of view on the topic.

The participants identified as “optimal” would then be invited to participate in a focus group discussion…perhaps two groups of 8 participants for each target in each market: just over half of the initial sample. An additional 4 participants (mutually exclusive from the focus group sample) who seem like they would thrive in a more personal deep-dive setting might also be selected to conduct ethnographic immersions in the days following the groups to dig in on identified patterns and bring some more context to the initial findings…perhaps adding a deeper level of findings and insights in the process.

The insights that results from the exploration for ACME widgets would identify both a breadth and depth of insights that can identify meaningful similarities and differences in wonder-widget value drivers and purchase consideration factors. It will not only identify the patterns, but also unearth the meaning behind those patterns that in turn lead to actionable product implications…with brand implications likely to surface as well.

The process can minimally add the project timeline (perhaps a few weeks to a month) but also serves to add significant incremental value to the results and implications.

Want to know more?  Would you like .pdf copy of the “official” whitepaper? Send me an email:


3 responses to “Progressive Sample Selection (PSS): A Methodology for Optimizing Qualitative Insights

  1. Jamie, interesting post! I like the idea of a second screening of research participants to identify those with the most insight for the next phase. More insightful participants are bound to deliver more insightful research. Thanks for telling me about this group on LinkedIn.

  2. igwe, chukwuma


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s