InsightFab
Knowledge Base/Principal Component Analysis (PCA): Dimensionality Reduction Visualization for High-Dimensional Data
Statistical Analysis6 min read

Principal Component Analysis (PCA): Dimensionality Reduction Visualization for High-Dimensional Data

This article offers a highly practical solution for situations where product yield is low and CPK plummets, despite numerous parameters appearing normal. The author recounts a new product launch where DPMO soared, highlighting the challenge of data overload. By reading this, engineers will learn how to leverage Principal Component Analysis (PCA) to efficiently identify critical variables impacting quality, enabling precise problem-solving and preventing unproductive efforts.

That day, when the CPK report came out, the whole room fell silent for three seconds before I realized what it was.

I remember once, our newly developed product was about to go into mass production. We had adjusted the process parameters through several rounds and thought everything was stable. However, when the CPK report for the first batch came out, it had directly dropped to 1.08, and DPMO surged to 6210! The conference room instantly went quiet, and the boss's face turned green. Everyone exchanged glances; dozens of parameters – voltage, current, temperature, time – everyone claimed theirs was fine, but the yield just wouldn't improve. At that moment, I thought, with so many variables, who exactly was the real culprit? Was I supposed to track them down one by one? Wouldn't that be making things impossible for myself?

Where's the Problem? You Have Too Much Data!

To put it bluntly, many times it's not that we lack data, but that we have an overwhelming amount of it, making it impossible to know where the focus should be. Imagine having a hundred sensors, each recording different information during the wafer fabrication process. These variables might be correlated with each other, some might have little to no impact, while others are critically important. But due to their sheer number, when yield issues arise, it's hard to spot the clues at a glance from dense reports.

So, the key is that we need a method to simplify these "intertwined" variables into a few "truly representative" indicators. To put it simply, it's about extracting the most important elements from a lot of seemingly complex information, making it clear at a glance. This method is Principal Component Analysis (PCA).

In other words, PCA is like a smart data compressor. It analyzes the relationships between all your data and then identifies a few "new axes that best represent" the variations in this data. These new axes are what we call "principal components." It can condense dozens or even hundreds of original parameters into two or three core variables, while still retaining most of the original information.

How is it Actually Done? Understanding the 'Explained Variance Ratio' is Key.

When practically performing PCA, you don't need to manually calculate those complex matrices; modern statistical software or Python packages handle it with one click. You just input your process data, and it will output several principal components.

  1. Look at the "Explained Variance Ratio": This is the most critical indicator in PCA results. It tells you what proportion of the variance in the original data each principal component can explain.

* For example, the first principal component might explain 70% of the variance, and the second explains 15%.

* This means by only looking at these two principal components, you can explain 85% of the variation in the original data. Isn't that amazing? You've condensed dozens of variables into two and still captured 80% of the key information.

  1. Decide how many principal components to keep: Usually, we select the number of principal components whose cumulative explained variance ratio reaches 80% or 90%. If you can reach this threshold with just two or three principal components, that would be perfect, because then you can plot the results on a 2D or 3D graph and visually inspect them directly.

That time when the CPK dropped to 1.08, I used PCA to reduce over 30 process parameters into 3 principal components. The results showed that the batches with low yield were clearly separated into two distinct clusters from the batches with good yield on the graph of Principal Component 1 and Principal Component 2. Further analyzing the relationship between these two principal components and the original parameters, it was then discovered that the stability of a certain heating voltage and the flow rate of another reaction gas were the real culprits causing the trouble.

The Most Common Pitfall: Jumping to Conclusions Just by Looking at Graphs.

Frankly speaking, many people, after running PCA, get excited as soon as they see those colorful dimensionality reduction plots, and then point at a certain dot saying, "Look! There's a problem here!" But if you haven't carefully examined what original variables each principal component is composed of, then your conclusion might just be a wild guess.

I once encountered a rookie engineer who, after seeing the PCA results, noticed that the data points of a certain batch were separated from other batches, and immediately told the boss, "This batch has process anomalies!" However, upon closer inspection later, it was discovered that that batch had simply switched to a different material supplier, so the distribution of process parameters should naturally be different. He failed to understand the meaning of the principal components, merely looked at the "clustering," and almost led to a misleading judgment. Therefore, it is crucial to delve into each principal component and understand what important original variables it actually represents.

One Thing You Can Do Today

Go back and open your most recent process data, try running PCA on it, and see if you can find those truly critical parameters from a mass of them.

Article Category: Statistical Analysis

Want to try it yourself?

Every tool mentioned in this article is available on InsightFab — just upload a CSV to analyze.

Go to Tools

More in this category

Correlation Coefficient: Pearson vs Spearman – Linear vs Monotonic Relationships

This article sheds light on a common pitfall in engineering data analysis: the misapplication of correlation coefficients. It explains why relying solely on Pearson correlation, which primarily detects linear relationships, can lead to incorrect conclusions when data exhibits non-linear, monotonic trends, especially when addressing issues like unexpected product yield drops.

Post-hoc Test: Choosing Between Tukey vs Bonferroni

This article addresses a common production line challenge: identifying the specific source of a problem when overall metrics like CPK fall short. It introduces Post-hoc Tests as a crucial tool to pinpoint significant differences between individual machines or groups, enabling precise problem-solving and immediate action.

Statistical Power: How to Ensure Your Conclusions Are Meaningful

This article addresses a common challenge for engineers: drawing conclusions solely based on observed data changes without accounting for statistical significance. It emphasizes the importance of using 'Statistical Power' to validate findings, ensuring that conclusions are robust and well-supported, particularly when presenting to management.