InsightFab
Knowledge Base/The Correct Interpretation of Confidence Intervals: 95% CI Is Not What You Think
Statistical Analysis6 min read

The Correct Interpretation of Confidence Intervals: 95% CI Is Not What You Think

This highly practical article addresses a common meeting scenario: a PM reviewing a CPK report showing "CPK: 1.08 (95% CI: 0.95-1.21)" and questioning its implications, leading to an awkward silence. It explains why the popular belief that there's a 95% probability of the value falling within the confidence interval is incorrect, providing the accurate way to interpret confidence intervals to enhance report comprehension.

That day, the CPK report came out, and the whole room fell silent for three seconds.

I still remember several years ago, when a new machine was introduced to our production line. Everyone cautiously completed the validation runs, and the data was in. The PM, with a serious expression, looked at the CPK report on the screen, which read "CPK: 1.08 (95% CI: 0.95-1.21)". He frowned and asked, "Is this CPK 1.08 good or bad? And what does that interval next to it mean?" The entire conference room instantly went quiet, everyone exchanged glances, and no one dared to answer directly. Isn't this a situation you and I encounter daily? Do you truly understand the confidence intervals on your reports?

Where the Problem Lies

Do you also think that the 95% confidence interval "0.95-1.21" means there's a 95% probability that our CPK 1.08 falls between 0.95 and 1.21? Honestly, this is a misunderstanding almost all new engineers make, and even some more experienced ones often get confused. Frankly, this interpretation is wrong! The correct interpretation of a confidence interval is actually about "our estimation method," not about the probability of "a single value."

In other words, if we repeatedly sample one hundred times, and each time use the same method to calculate a 95% confidence interval, then approximately 95 of those confidence intervals would contain the "true" CPK value. So, the key point is that this 95% probability refers to the "interval" itself, not to the "single point estimate" CPK 1.08. It's like throwing darts: each time you throw a dart, you define a range, and there's a 95% chance that your range will encompass the bullseye, not that your dart has a 95% chance of landing within that specific range.

What to Actually Do

So how should we look at this CPK 1.08 (95% CI: 0.95-1.21)? First, your point estimate (here, CPK 1.08) is your "best guess" of the true situation. However, because we cannot measure all products, this "guess" will inevitably have errors. The confidence interval is used to tell you the possible range of this "guess."

  1. Look at the width of the interval: The narrower the interval, the more precise your estimate. For example, with 0.95-1.21, the width is 0.26. If your interval was 0.8-1.4, it would indicate a potentially larger estimation error.
  2. Check if the interval crosses a critical value: Suppose your customer requires a CPK of at least 1.0. Your point estimate is 1.08, which seems to pass. But the confidence interval is 0.95-1.21, with a lower bound of 0.95. This means that the "true" CPK value could very likely be below 1.0! In this situation, you cannot confidently declare, "Our CPK definitely passes 1.0." You might need to increase the sample size to narrow the interval and make a more confident judgment.

The Most Common Pitfall

I once encountered a situation where we were evaluating the defect rate after a process change, and DPMO dropped from 7500 to 6210. The report showed the new process DPMO (95% CI: 5800-6600) and the old process DPMO (95% CI: 7100-7900). The PM, seeing the lower DPMO figure for the new process, happily exclaimed, "Excellent, significant improvement!"

But upon closer inspection, do the confidence intervals for the old and new processes overlap? 5800-6600 and 7100-7900 – the two intervals do not overlap at all! This indicates that the new process indeed shows a statistically significant improvement. If the two intervals overlapped, for example, if the new process was (95% CI: 6000-7000), then you couldn't jump to conclusions so quickly, because the true value might fall within the overlapping region, suggesting there might be no significant difference. In essence, non-overlapping intervals truly prove a difference.

One Thing You Can Do Today

Next time you see a confidence interval, don't just look at the point estimate; remember to check if the interval crosses your standard!

Want to try it yourself?

Every tool mentioned in this article is available on InsightFab — just upload a CSV to analyze.

Go to Tools

More in this category

Correlation Coefficient: Pearson vs Spearman – Linear vs Monotonic Relationships

This article sheds light on a common pitfall in engineering data analysis: the misapplication of correlation coefficients. It explains why relying solely on Pearson correlation, which primarily detects linear relationships, can lead to incorrect conclusions when data exhibits non-linear, monotonic trends, especially when addressing issues like unexpected product yield drops.

Post-hoc Test: Choosing Between Tukey vs Bonferroni

This article addresses a common production line challenge: identifying the specific source of a problem when overall metrics like CPK fall short. It introduces Post-hoc Tests as a crucial tool to pinpoint significant differences between individual machines or groups, enabling precise problem-solving and immediate action.

Statistical Power: How to Ensure Your Conclusions Are Meaningful

This article addresses a common challenge for engineers: drawing conclusions solely based on observed data changes without accounting for statistical significance. It emphasizes the importance of using 'Statistical Power' to validate findings, ensuring that conclusions are robust and well-supported, particularly when presenting to management.