That day the CPK report came out, the room fell silent for three seconds, and then I saw the section chief's face turn green.
Have you ever encountered such a mess? The boss was rushing to ship, and the client was hounding us relentlessly. The testing department finally finished testing a batch of goods, but when the RAT report came out, there were red flags everywhere! The air in the meeting room instantly solidified; I glanced over and saw the section chief's face had turned green. The boss asked, "Did it pass or not?" Everyone exchanged bewildered glances, because some items were just a hair's breadth away, but that 'just a hair's breadth' made it hard to decide. At this point, if you haven't clearly understood the pass/fail criteria for "Reliability Acceptance Testing (RAT)", you'll truly be put in a difficult spot.
Where's the problem? It's not just about testing; it's about "judgment".
To put it plainly, RAT acceptance testing is when we subject a batch of products to rigorous conditions to see if they are durable and if they will fail before the warranty period. It's different from general product functional testing; it's not just about whether a function turns on. We test the product's limits, such as high temperature, high humidity, voltage surges, etc., simulating various conditions it might encounter during actual use by the customer.
But after testing, with a lot of data, the problem arises: what exactly constitutes "passing"? Many times, the data falls into an awkward range, not entirely good, nor entirely bad. If you lack clear judgment standards, decisions can easily become "subjective" or "based on who shouts loudest". The worst part is, if problematic products are released and subsequently fail at the customer's end, the resulting customer complaints and compensation will leave you heartbroken.
How to do it in practice? Let numbers speak, avoid ambiguity.
To be frank, judgment criteria usually fall into two types, and both must be met:
- Limit on Failures: This is the most straightforward. We usually set an upper limit, for example: "Among 1000 units under test, the number of failures cannot exceed 3." If it exceeds, it's directly deemed a Fail. This number is usually defined with the customer or internal quality department and cannot be changed arbitrarily. For example, we previously had a new product where the customer required DPMO (Defects Per Million Opportunities) of 6210. This translates to no more than 6.21 defective units per 1000 units. If, after testing, 8 units are found to be bad, then sorry, it's an immediate Fail.
- Conformance to Specification Values: This is a bit more technical and usually judged using Cpk or PpK. Cpk is an indicator of process capability; it tells you how close the product's characteristic values are to the specification centerline and the degree of variation. For instance, if a customer requires a certain voltage output to be between 3.2V ± 0.1V, and we tested a batch with an average of 3.22V, which looks good. However, if the spread is wide, sometimes reaching 3.1V and sometimes 3.3V, then the Cpk value will be low. Suppose the customer requires a Cpk of 1.33 or higher, but we calculate it to be 1.08. This indicates insufficient process capability and raises doubts about product stability. Even if the current number of failures hasn't exceeded the limit, there's a high probability of issues at the customer's end. Therefore, even if the number of failures is within limits, but Cpk doesn't meet the target, it could still be judged as a Fail.
So the key is that you must satisfy both of these conditions simultaneously. Neither can be missing.
The most common pitfall: numbers can lie, but you cannot.
The most common pitfall I've encountered is people trying to "negotiate" or "bend the rules". For instance, if the number of failures is 4, but the standard is 3, they might think: "Hey, it's just one extra, can we let it pass?" Or if Cpk is stuck at 1.30, just shy of 1.33. At such times, many people try to find excuses, such as "This failure is just an isolated incident, let's retest it," or "The measurement instrument might be drifting a bit," and so on.
Honestly, this mindset is very dangerous. Because one concession can lead to increasingly vague standards in the future. Moreover, if truly problematic products enter the market, the resulting losses will undoubtedly be far greater than the time you might spend re-validating or improving the process. Previously, we had a new hire who, in a rush to ship, insisted on submitting a report with a Cpk of 1.25. As a result, the products frequently malfunctioned shortly after reaching the customer. Eventually, the entire batch was returned, we incurred significant losses, and the new hire almost got fired.
Therefore, the data is there; if it passes, it passes, if it fails, it fails. Don't think about taking shortcuts.
One thing you can do today
Go back and check your RAT specifications, clearly defining the number of Failures and the Cpk/PpK thresholds.