Skip to content Skip to search

Insight Center Resources

Industry knowledge to help you grow your business

Why Won’t The Sample Sizing Debate Ever Stop?

As I have mentioned in earlier blogs, 400 returned surveys are enough to adequately represent virtually any large population. Logically you’d think that the required sample would increase proportionately with the population – but this is incorrect..explaining the math behind random sampling is an advanced statistical course. For many years in my Survey Essentials course, I used a “soup” analogy to explain this principal. It’s a practical way to look at sampling.

Let’s say you have one gallon of soup in the pot, and you want to sample it for taste, temperature and ingredients. Intuitively, you’d give the pot a good stir to ensure it’s all mixed together, and take a tablespoon as a sample. Based on that ‘representative sample’ , you’re can decide whether to declare your soup ready for consumption or not. One tablespoon constitutes your ‘representative sample’ and based on the results, further testing is determined to be necessary or unnecessary.

Now suppose you’re making 50 gallons of soup in an extremely huge pot. That’s 50 times more soup than in the previous example. Ready to take the sample? Do you think the sample also needs to be 50 times bigger? After giving the huge pot a stir to make sure it’s all mixed together, what would you grab to take the sample? A tablespoon, a ¼ cup measure, or perhaps a gallon jug? Intuition would tell you that maybe you’d better drink a gallon of soup to be sure it’s ready, since 50 gallons is a big pot of soup. But, this isn’t the case, Just a big tablespoon will do. Sure, for 50 gallons you may use a couple of tablespoons full, but you certainly don’t need a full gallon. The most important detail is that the sample is completely random….like making sure the soup is all stirred up.

Still don’t believe me? You have the choice to take that college course, or you can use any number of on-line sample calculators to do some fact checking. Just Google ‘sample calculators’. When using any of the online calculators, here are the populations and required samples:

Original Population Required Sample Size
500 217
1,000 278
10,000 370
100,000 383
1 million 383
1 billion 383

You’ll see that the required sample size does not exceed 384, even for populations of 1 billion. We round up to 400 just for simplicity. This is based on a 95% confidence level and a confidence interval of 5%, both of which are industry standards. It’s also for a generally homogenous population. If there are specialized strata, then that must be taken into consideration

So, let’s stop the debate over the required sample size once and for all, the required sample pool never exceeds 384 (rounded up to 400). Now, we can focus on creating valid surveys, conducting appropriate data analysis and effective survey follow-up.