InsightFab
Knowledge Base/DOE Confirmation Experiment: Why the Last Step is the Most Important
DOE6 min read

DOE Confirmation Experiment: Why the Last Step is the Most Important

This article details a common pitfall where initial DOE predictions for a new process show high Cpk, but confirmation experiments reveal significantly lower values in a real production environment. It underscores the critical importance of conducting confirmation experiments under realistic conditions to bridge the gap between ideal lab data and actual manufacturing performance.

That Day the Cpk Report Came Out, the Entire Room Was Silent for Three Seconds

I still remember several years ago, a rookie engineer in our department was responsible for introducing a new process. He had done an excellent job with the DOE beforehand, completing factor analysis and response surface methodology. The data looked exceptionally good, predicting a Cpk of over 1.6 under optimal conditions. What happened then? After the last batch of the DOE confirmation experiment ran, the Cpk report came out, and the entire room was indeed silent for three seconds. The Cpk value was only 1.08, and DPMO soared to 6210. The boss's face instantly turned paler than a cleanroom whiteboard, and the new engineer was on the verge of tears.

Where Was the Problem?

To be honest, the problem lay in the "confirmation experiment" step. Many times, when we conduct DOE, the preceding experimental design, factor screening, and optimization are performed beautifully, and even the data obtained in the lab is extremely perfect. But did you know that these experiments are usually conducted under "ideal conditions"? Perhaps the equipment you chose was in the best condition, the materials used were the most stable, and even the operators were exceptionally careful. In other words, the data you saw might just be the tip of the iceberg, or rather, it holds true under "the best-case scenario."

So what's the point? A confirmation experiment, to put it plainly, is taking the "best" conditions you identified earlier and testing them in a "real production environment." It's not about verifying the precision of your mathematical model, but about verifying whether your "optimal conditions" can withstand the demands of an actual production line.

How Is It Done in Practice?

The method is actually simple, but there are many subtle points:

  1. Scale Up: In your previous DOE, you might have experimented with only a few wafers or dozens of products. For the confirmation experiment, it is crucial to significantly increase the quantity. For example, if you ran 25 wafers before, the confirmation experiment should run at least 200 wafers. Insufficient wafer count means insufficient statistical representativeness, rendering the data meaningless.
  2. Simulate Reality: Don't just use the equipment you "think" is best. Conduct the experiment across multiple machines, with multiple batches of materials, and even under different shifts. Your goal isn't to prove your conditions are great in the lab, but to prove they are also great on the production line.
  3. Look at Cpk and DPMO: These are the real-world indicators. A Cpk of 1.08 and DPMO of 6210 indicate excessive process variation and unstable yield. If your target is 1.33 or higher, the data from the confirmation experiment is your most truthful report card.

Therefore, the key is that you must validate using conditions as close to mass production as possible, rather than just being content with laboratory results.

The Most Common Pitfalls

The most common pitfall I've seen is engineers being overconfident, believing their previous models are accurate enough, and that the confirmation experiment is just "going through the motions." What was the result?

  • Running only once: Thinking that running it once would be sufficient, but that day the machine happened to be in exceptionally good condition, producing beautiful data. Upon introduction to mass production, the results were disastrous.
  • Selecting specific equipment and materials: To make the data look good, deliberately choosing the best quality equipment and the most stable materials, which ultimately amounts to self-deception.
  • Ignoring variation: Only looking at the average value and not the standard deviation. A DPMO blowout usually indicates excessive variation. In a process with a Cpk of only 1.08, even if the average value is very close to the target, the yield will immediately drop significantly with any slight disturbance.

Frankly, a confirmation experiment is like taking a newly designed product and letting consumers try it out. No matter how impressive the lab data is, if consumers don't buy it, everything is in vain. You must let it be tested in the real world.

One Thing You Can Do Today

Next time you conduct a DOE, remind yourself: A confirmation experiment is not for the boss; it's for the yield.

Want to try it yourself?

Every tool mentioned in this article is available on InsightFab — just upload a CSV to analyze.

Go to Tools

More in this category

DOE: Not Just Running Experiments, But Hitting the Most Targets with the Fewest Bullets

A full factorial experiment with four factors and three levels each requires 81 runs. Using a Taguchi L9 orthogonal array, only 9 runs are needed, saving 89%, while simultaneously estimating the main effects of all four factors without mutual contamination.

DOE Taguchi Method: Finding Optimal Parameters in 9 Experiments, Not by Luck

A full factorial design for 4 factors at 3 levels requires 81 runs, while a one-factor-at-a-time approach, though shorter at 36 runs, misses crucial factor interactions. The Taguchi L9 method offers a highly efficient alternative, completing the analysis in just 9 experiments and achieving an 89% reduction in experimental effort.

Response Surface Method: Finding the Optimal Process Sweet Spot

After DOE identifies the significant factors, RSM (Response Surface Methodology) helps pinpoint the precise parameter combination that optimizes the outcome. This marks a crucial transition from understanding 'which factors are important' to determining 'what settings are best'.