InsightFab
Knowledge Base/Sample Size Calculation: Formulas for t-test, ANOVA, and Proportion Test
Statistical Analysis6 min read

Sample Size Calculation: Formulas for t-test, ANOVA, and Proportion Test

The author shares a practical lesson learned from an experience where insufficient sample size led to unreliable Cpk reports, causing dismay to their boss. The article vividly explains that relying on intuition or outdated SOPs for sample size makes data interpretation a gamble, even for seemingly good Cpk values. It guides readers on how to determine reliable sample sizes based on specific statistical test objectives, emphasizing the importance of rigorous data for risk management.

That day the Cpk report came out, the whole room was silent for three seconds, and the boss's face turned green

That afternoon, at the production line yield meeting, the supervisor reported the previous week's data as usual. When reporting the Cpk for the new machine, a figure of 1.08 prominently appeared on the screen. The air in the room instantly froze, and the boss's face turned green in an instant. He said, "Cpk 1.08? How many samples did you submit to claim this machine is fine?" My inner monologue was: "Isn't the SOP just 30 pieces?" As a result, I got an earful from the boss. From that day on, I truly began to understand "sample size"; it's not just about following what the SOP says, there's real science behind it.

Where's the problem? Your "feelings" are unreliable

To put it plainly, if your sample size is insufficient, your conclusions are a gamble. If you sample 5 wafers today, measure a critical dimension, and then declare that the entire batch meets specifications, how is that different from rolling dice? How do you know those 5 pieces didn't just happen to fall into the "sweet spot" of the distribution? When the sample size is too small, your data variability will be very large, and even if you calculate a Cpk of 1.33, it might just be good luck. In other words, using "feelings" or "habit" to determine sample size is like placing a bomb on the production line, liable to explode at any moment. This is not just a statistical issue; it's also risk management.

How to actually do it? It depends on your "testing objective"

Frankly, there isn't a single formula for determining sample size; it depends on the "statistical test" you are using. The most common ones are probably these:

  1. t-test (comparing two group means):

Suppose you have a new batch of chemical reagents and want to see if it truly increases the etch rate compared to the old one. You need to know the desired "effect size" (the difference in etch rates between new and old reagents), the "risk" you are willing to take (Type I error, usually 0.05), and your "power" to detect a difference (Power, usually 0.8 or 0.9).

For example, if the old reagent's average etch rate is 100 nm/min, and the standard deviation is 5 nm/min. If you want the new reagent to improve by at least 3 nm/min, and you have 90% confidence in detecting this difference, then running it through software might show that each group needs 45 wafers. Honestly, manual calculation formulas are complex; nowadays, there are online calculators or JMP software that can help.

  1. ANOVA (comparing three or more group means):

If you have three different PVD machines and want to compare their film thickness uniformity. The concept is similar to the t-test; you still need to set the effect size, risk, and power. It's just that with more groups to compare, the calculation becomes more complex.

Suppose you have three machines A, B, and C. Old machine A's film thickness uniformity standard deviation is 0.5%, and you want new machines B and C's uniformity to be 0.2% better than A. Similarly, set your alpha (0.05) and power (0.8), and the calculation might show that each machine needs at least 60 wafers to be measured.

  1. Proportion Test (comparing yield or defect rates):

The most common is yield comparison. For example, you developed a new process and want to see if the defect rate truly decreased from 6210 DPMO (0.621%) to 3000 DPMO (0.3%).

In this case, you will need to know the current proportion (P1), the proportion you expect to achieve (P2), as well as the same risk (alpha) and power (power).

Suppose your current yield is 99.379%, and you expect the new process to improve it to 99.7%. If you want to state with 95% confidence that the new process truly shows improvement, and have 80% power to detect this improvement, you might need to sample 2500 wafers for verification. This number sounds astonishing, but if you want to push a high-yield process even further, the sample size will indeed be this large.

So the key is that you must first clearly think about "what you want to prove," and then select the corresponding test method based on that objective to calculate a reasonable sample size.

The most common pitfall: blindly applying historical data

The most common mistake I used to make was directly using the sample size specified in the SOP, or simply following what seniors did. The result was that sometimes the sample size was too small to capture subtle process changes; other times, it was too large, wasting time, manpower, and materials.

Once, we tested a new wafer cleaning machine, sampling 20 wafers according to the SOP for before-and-after comparison. The report showed that the yield "appeared" to improve, but it was not statistically significant. The boss then asked, "Is your sample size insufficient, which is why you can't see it?" After re-evaluation, if we wanted to detect a 0.5% yield improvement, 20 wafers were nowhere near enough; at least 500 were needed! The lesson from that time was: don't treat the SOP as gospel. SOPs might be based on past empirical values, but your process goals and variability might have changed.

One thing you can do today

Open Excel or JMP, and start practicing calculating sample sizes for different scenarios using the software.

Want to try it yourself?

Every tool mentioned in this article is available on InsightFab — just upload a CSV to analyze.

Go to Tools

More in this category

Correlation Coefficient: Pearson vs Spearman – Linear vs Monotonic Relationships

This article sheds light on a common pitfall in engineering data analysis: the misapplication of correlation coefficients. It explains why relying solely on Pearson correlation, which primarily detects linear relationships, can lead to incorrect conclusions when data exhibits non-linear, monotonic trends, especially when addressing issues like unexpected product yield drops.

Post-hoc Test: Choosing Between Tukey vs Bonferroni

This article addresses a common production line challenge: identifying the specific source of a problem when overall metrics like CPK fall short. It introduces Post-hoc Tests as a crucial tool to pinpoint significant differences between individual machines or groups, enabling precise problem-solving and immediate action.

Statistical Power: How to Ensure Your Conclusions Are Meaningful

This article addresses a common challenge for engineers: drawing conclusions solely based on observed data changes without accounting for statistical significance. It emphasizes the importance of using 'Statistical Power' to validate findings, ensuring that conclusions are robust and well-supported, particularly when presenting to management.