As a Quality Engineer working in the Medical Device field, there is nothing more frustrating than a “no-fault-found” condition on a product complaint. The product is returned by the customer due to a problem while in use, and the manufacturer cannot replicate the problem. This is commonly referred to as no-fault-found (NFF). I could not find a definite rate on NFF for medical devices. However, I did find that for the avionics industry it is 40-60% of all the complaints.
The NFF can be also described as “cannot duplicate”, “trouble not identified”, “met all specifications”, “no trouble found”, or “retest ok”. This menacing condition can be quite bothersome for the customer as well as the manufacturer. In this post, I will try to define some red flags that one should watch out for, and a list of root causes that might explain the reasons behind the NFF condition. I will finish off with a great story from the field.
The following list contains some of the red flags that one should watch out for, if no-fault was found with the product that was returned. This list is of course by no means meant to be an exhaustive list, but might provide some guidance.
- Major incident associated with the complaint – If the return was associated with a major incident such as a serious injury or even worse, death, one should test the unit exhaustively to identify the root cause.
- Unit was returned more than once – If the unit was returned for the same problem, it is an indicator of an inherent root cause creating the problem. Sometimes, an existing condition can act as an enabling condition and can create more than one effect. In this case, the problem may not be the same for the second or third return. Alternatively, the enabling condition can be present at the customer’s site.
- Nonstandard Rework(s) performed on the unit during production – I am a skeptic of reworks. A rework is deviation from the normal production. And sometimes, fixing one thing can cause another thing to fail.
- The product is part of the first lots produced after a major design change – If the product validation process is not adequate or if proper stress tests were not performed, the unit can be produced with latent issues/bugs.
- More than one customer reporting the same problem – If there is more than one source reporting the problem, it is a clear indication of an inherent issue.
Potential root causes for NFF condition:
The following list contains some of the root causes that might be associated with a no-fault condition. This list is of course by no means meant to be an exhaustive list.
- Adequacy of test methods – If the test method is susceptible to variations, it may not catch failures. This cause is self-explanatory.
- Excess stress during use – Reliability Engineering will tell you that if the stress during use exceeds the inherent strength of the product, the product will fail. This stress can be environmental or can be due to use beyond the intended use of the product. An example is if the product is used at a wrong voltage.
- New user or lack of training – If the end user is not familiar with the product, he/she can induce the failure that might not occur otherwise. This is not an easy root cause to figure out. Sometimes this is evident by the appearance of the product in the form of visible damages (dents, burn marks etc.)
- High number of features – Sometimes, the higher the number of features, the more the complexity of the product and worse the ease of use of the product. If the product is not easy to use, it can create double or triple fault conditions more easily. A double or triple fault condition occurs when two or three conditions are met for the fault to happen. This is considered to be isolated in nature.
- Latent bugs/issues – No matter how much a new product design is tested, all the issues cannot be identified. Some of the issues are left unidentified and thus unknown. These are referred to as latent issues/bugs. This is the reason why your mobile phone or your computer requires updates or why some cars are recalled. These bugs will result in failures that are truly random and not easy to replicate.
- Problem caused by an external accessory or another component – The product is sometimes used as part of a system of devices. Sometimes, the fault may lie with another component, and when the product is returned, it may not accompany all the accessories, and it will be quite hard to replicate the complaint.
- Lack of proper validation methods – Not all of the problems may be caught if the validation methods are not adequate. This cause is similar but not the same as latent bugs/issues. Here, if there was no stress testing performed like transportation or environmental, obvious failures may not be caught.
- Customer performed repairs – Sometimes, the failure was induced by something that the customer did on the product. This may not always be evident unless revealed by the customer.
- Customer bias – This is most likely the hardest cause to identify on this list. Sometimes, the customer may “feel” that the product is not functioning as intended. This could be because they experienced a failure of the same brand at another time, and the customer feels that the entire product brand is defective.
- Other unknown isolated event – Murphy’s Law states that “whatever can go wrong will go wrong.” Law of Large Numbers loosely states that “with enough number of samples, even the most isolated events can happen.” Combined together, you can have an isolated incident that happened at the customer site and may never happen at the manufacturer site.
The mystery of diathermy burns:
I got this story from the great book “Medical Devices: Use and Safety” by Bertil Jacobson MD PhD and Alan Murray PhD. Sometimes, a surgery that uses a device like an RF Generator can cause burns on the patient from the heat induced by the device. This is referred to as “diathermy burns”.
A famous neurosurgeon retired and started working at a private hospital. Curiously, after a certain date, five of his patients reported that they have contracted ugly, non-healing ulcers. These were interpreted as diathermy burns. These burns were present on the cheek bones of the patients who were placed face-down for the surgery and on the neck region of the patient who were operated in the supine position (face-upward). The surgeon has had a long uneventful and successful career with no similar injuries ever reported.
No issues were found with the generator used for the surgery. A new generator was purchased, and the problem persisted. The manufacturer of the generator advised replacing the wall outlet. The problem still persisted. The surgery routines were updated and rigorous routines involving specific placement of electrodes were put in place. The problem still persisted.
A clinical engineer was consulted. He also could not find any fault with any of the equipment. At that point he requested witnessing the next operation. During this, it was discovered that the new assistant surgeon was placing his hands heavily on the patient’s head during the operation. Thus, the diathermy burns were actually pressure necroses caused by the assistant surgeon. These apparently can be misinterpreted as diathermy burns!
This story, in a curious way, implies the need to go to the gemba as well! Always keep on learning…