Abstract
Deviations occur across the pharmaceutical and device industries nearly every day. Companies have spent hundreds of hours and thousands of dollars to investigate deviations, identify their causes and implement corrective actions to prevent their recurrence. Yet with all of the investment we continue to frustrate both our personnel and our management as the same problems continue to recur.
FDA and other regulators routinely request to see lists of deviations, non-conformances, CAPAs, complaints, and out of specification investigations during inspections. For nearly 20 years, failure in these systems has been in the top 5 of all observations issued. So what are we doing wrong? This article attempts to explore the issues and provide some possible solutions and paths forward to eliminate the problems routinely documented as deviations within our manufacturing and testing environments. Although the examples focus on U.S. FDA, the problems and solutions are applicable worldwide.
I. Introduction
Numerous articles, seminars, regulations, and guidance documents have been generated in an attempt to address the problems observed by regulators, auditors and quality assurance departments. Despite all the attention and the abundance of available resources to address problems, the FDA and other regulators continue to cite failures with deviations and non-conformances, CAPAs, complaints, and out of specifications. These failures to adequately address management of deviations to ensure they are appropriately investigated, analyzed to determine the reasons for their occurrence and actions taken to eliminate their causes have become a bane to the regulated industry. We continue to spend extensive time in dealing with backlogs, recurring problems, audit findings and regulatory citations.
Too often we in the industry have a variety of definitions, concepts and tools that we frequently use interchangeably without ensuring that we are talking about the same or different things. One of my colleagues frequently stated that we often are in violent agreement when we thought we were on opposite sides of an argument.
In an attempt to avoid confusion, this article will provide a brief list of definitions of common deviation terms, followed by some tools repeatedly found in use in typical deviation management systems, some frequently cited observations from regulators, characteristics of common deviation systems and this author’s suggestions for why the citations continue and finally some ideas for eliminating these problems in the future.
II. Definitions
Correction/containment: steps taken as soon as possible to correct - stop the deviation from continuing and contain the potential impact. Examples: Fixing the specific instance of a record error, turning off the water after a pipe burst, quarantine of suspected product.
Corrective Action: steps taken to address the underlying cause of a deviation to ensure the deviation does not recur in any part of operation. Note the usage – to correct and prevent Recurrence of a problem.
Deviation: Departures from laws, standards, manufacturer’s instructions, policies, procedures, specifications, or accepted practices that are not discovered and corrected prior to, or by, the next critical control point in the system. Note: this is a somewhat more narrow view as items discovered and corrected prior to a critical control point are also deviations, but in this author’s opinion do not make sense to manage within the deviation system).
Out of Specification (OOS): A test result that is a variance from an established material or product specification.
Out of Trend/Tolerance (OOT): A test result that is atypical from established historical norms for this equipment or material.
Preventive Action: steps taken to ensure a deviation never occurs. These actions are typically part of a continuous process improvement process. Note the usage – to prevent Occurrence of a problem.
Root Cause: The basic reason(s) a deviation has occurred (why a deviation occurred).
III. Typical Root Cause Analysis tools
There are a myriad of tools in use throughout the industry but some of the most common are listed below with a brief description.
A. Retesting Although not thought of as a Root Cause Analysis tool, this process is typically used to identify whether an initial OOS/OOT result was valid and if so, identify a cause for its occurrence.
B. Fishbone /Ishikawa Diagram/5 M’s There are a variety of variations on Kaoru Ishikawa’s Fishbone diagram in use to document cause and effect of problems. Although the labels vary, in principle they fall into evaluating the following categories for their effect on a problem: personnel, procedures, materials, environment, machines, and measurements. The principal goal of this approach is to expand the scope of thinking.
C. 5 Why’s The basis for the 5 Why’s approach is to understand the cause of actions by delving deeper below the symptoms. This process helps teams recognize a broad network of problem causes and their relationships and drives problem solvers to successively pursue causes for additional causes until underlying “root” causes become evident.
D. Fault Tree Analysis This process is designed to show linkages between issues and to explore each linkage individually. It helps teams get to solutions to problems that may have more than one cause contribution.
E. Affinity Diagrams This process organizes large numbers of ideas into their natural relationships. This method taps the team’s creativity and intuition and frequently is used to narrow solutions.
IV. Examples of objections (from Turbo EIR, FDA.gov)
Next we are going to look at common examples of findings / objections from regulators. Although this presentation focuses on FDA data which points out common reoccurring issues, it is also applicable to other regulators findings across the world.
There are no written procedures for production and process controls designed to assure that the drug products have the identity, strength, quality, and purity they purport or are represented to possess. Cited 21 CFR 211.100(a) Written Procedures; deviations.
There is a failure to thoroughly review [any unexplained discrepancy] [the failure of a batch or any of its components to meet any of its specifications] whether or not the batch has been already distributed. Cited 21 CFR 211.192 Production Record Review
The following although admittedly somewhat difficult to read is a consolidation of findings from FDA.gov’s turbo EIR program that are commonly linked to deviation management failures: Deviations from written [specifications] [standards] [sampling plans] [test procedures] [laboratory mechanisms] are not [recorded] [justified]; Established [specifications] [standards] [sampling plans] [test procedures] [laboratory control mechanisms] are not [followed] [documented at the time of performance]; The establishment of [specifications] [standards] [sampling plans] [test procedures] [laboratory control mechanisms] including any changes thereto, are not [drafted by the appropriate organizational unit][reviewed and approved by the quality control unit].
Procedures for corrective and preventive action have not been adequately established. Cited 21 CFR 820.100(a) Corrective and Preventive action
Corrective and preventive action activities and/or results have not been adequately documented. Cited 21 CFR 820.100(b) Corrective and Preventive action
V. Costs
Here we will explore the costs associated with management of deviations. While we all recognize that cost should not be a primary driver for quality decisions, in practice all too often companies fail to recognize all of the costs involved with investigation of deviations and how some of these costs could be eliminated with a different approach to deviation management. The primary costs include routine Investigation and Investment in people and tools to better manage deviation investigations and records.
A. Routine Investigation Investigation costs arise from a variety of sources including but not limited to:
Time spent investigating, analyzing, developing CAPA and implementing actions to eliminate or prevent a problem
Time spent reviewing and approving deviation reports
Costs of disposition of non-conforming materials
Costs related to implementing CAPA
Costs for remediation of deviation systems including additional training
Lost opportunities (loss of reputation, product shortages, impacts on production schedules, etc.)
The costs for investigation typically can run between several hundred to several thousand dollars per occurrence of a deviation. A simple calculation often used (# of hours x # of people x $ loaded employment rate (hourly salary + benefits) addresses just the cost of management of the deviation investigation and ensuring records are appropriately reviewed and closed. It should be noted however that there are also the containment costs related to the management of the non-conforming material (quarantine/storage), the management of impacted equipment, utilities and facilities (ensuring they are offline/not in use until issues resolved) management of impacted personnel, and additional management of production schedules. Additionally, there are the costs related to remediation e.g. creation of new procedures, replacement/repair of equipment, utilities and facilities, validation/revalidation of equipment, methods and production processes, additional testing, drug/device shortages, recalls, lost reputation, etc. As you can quickly see the costs of even one deviation can quickly add up into the tens of thousands.
B. Investment
Technology The industry has invested many tens of thousands of dollars in electronic tools including but not limited to: Sparta Systems – Trackwise®, Master Control-Quality Management System (QMS) and Pilgrim Quality – SmartSolve®. Each of these tools provides data platforms for capturing, tracking and trending deviations and the actions resulting from them. Typically these systems are designed with various forms for collecting information in searchable fields and allowing multiple attachments. Although excellent tools, each of these systems has been used across the regulated industry to varying degrees of success. The mentality here seems to be if I just automate the deviation management process, the problems will go away.
Personnel The industry has spent many tens of thousands of dollars in providing training and other support to personnel in an attempt to address the continuing issues of recurring deviations. These include but are not limited to: In-house training, attendance at internal and external seminars and extensive use of consultants. The mentality seems to be let’s train / re-train / hire consultants to make the problem go away.
VI. So why is it not working?
With all of the investment in time, money and energy, why do we continue to receive citations, not resolve our problems, and eliminate the recurrence of deviations? Below are some ideas for some of the fundamental causes that lead to our recurring issues.
A. Thinking our job is to manage deviations/CAPAs
Many companies believe that it is their job to create a deviation system that manages problems encountered at their companies. They miss the concept that their real job is to eliminate the problems, create documentation that demonstrates they eliminated the problems and prevented their recurrence.
FDA reports that more than 95% of all deviations they encounter during inspections are associated with recurring issues and are corrective in nature with less than 5% being preventive in nature. Numerous FDA officials and consultants have stated in multiple conference presentations that the goal is 95% of deviation investigations should be preventive with only 5% being corrective in nature.
B. Failure to tell a good story
At the end of the day, if we do not tell a story that is clear, concrete (factually based), concise, complete, shows that the problem was corrected, and all documentation closed we get no credit for any of the work and effort put worth in our deviation management documentation. It is a well-known axiom that if it isn’t written down, it didn’t happen. But if others can’t understand what you did, you really are just wasting your time. Our goal should not be to write long epistles but rather to clearly demonstrate both to ourselves and to others that we identified a problem and promptly took actions to investigate, understand and resolve it.
C. Confusing and conflicting responsibilities
Most Deviation Management systems are owned and operated by the Quality organization. Deviations however, are not routinely originating in the Quality organization but rather in other parts of the operations. The challenge immediately then arises – who is responsible for evaluating and resolving the deviations – the owner of the deviation system, the originator of the deviation or someone else? Most systems in the industry are set up with an immediate notification of the quality organization but from that point the variety and effectiveness vary. Some Quality organizations believe it is their responsibility to determine if an occurrence is a deviation and if so, what should be done, and frequently who is assigned. Others see their role as micromanaging the investigations, still others as wordsmiths of the documentation, and still others as final arbiter of the conclusions. However, too frequently, these approaches result in Quality being the policeman and owner of the problems. As the owner, the quality organization is then accountable for ensuring they are fixed while the creator of the problem is exonerated.
Other Quality organizations see their role as primarily or even exclusively oversight and frequently distance themselves from the day to day execution of the management of the individual deviation. Operational personnel frequently execute the investigation and analysis of the issues always looking over their shoulder for what is Quality going to think about what they did. When documentation is provided to the Quality organization there is frequent back and forth (with the concomitant delays) until the Quality group agrees with the final document.
D. Lack of Knowledge
Failure to understand the systems being evaluated Too often while there is a recognition that a process does or doesn’t work, there is typically a general lack of knowledge for how the process is supposed to work. That is, how the equipment, personnel, materials, methods, environment and measurements come together to create a process that ensures it works consistently. Without this understanding, when a process fails, staff more often than not proceed to implement solutions in an attempt to get the process working without truly understanding whether they have returned the process to its original designed state. The end result more often than not is that the corrective actions are more Band-Aids that cause other problems rather than cures.
Failure to Gather Facts of the incident The basic facts of the incident help us to fully understand what happened on this particular instance of the execution of a process and allow us to compare and contrast these facts with other executions of the process where there were no incidents. Too often, our deviation process is 1) notify Quality, 2) issue a form, 3) assign a risk classification, 4) decide what the root cause is and 5) come up with some CAPAs in an attempt to fix the issue. We skip gathering facts about what was actually going on at the time the problem occurred. More often than not, this results in problems being open for extended periods of time while staff determine what to do. When specific facts including but not limited to time of occurrence, personnel present (did we speak to them), location of occurrence, what was going on before during and after the occurrence, how long did it take to gather the information, can we repeat the error, was anything moved before you evaluated it, etc. result in lost opportunities for understanding, and all lead to incomplete and more often than not incorrect conclusions. Additionally failure to gather facts often results in incorrect or incomplete assessment of the risks the issue posed and directly leads to either too few resources assigned or too many to evaluate the issue.
Failure to link problems to the systems responsible for making sure they didn’t occur in the first place Too often we look at deviations as individual ocurrences or random acts of problems and more often than not concluding one or more individuals caused the problem. What we frequently miss in our evaluations is that we designed systems (e.g. production management, materials management, equipment management, analytical testing, etc.) to execute our processes that took into account that individuals were involved when they were designed. If a problem occurred, we typically never consider whether we didn’t design the system correctly to ensure it was capable of overcoming or preventing errors when personnel engaged them.
E. Using right tools in wrong ways
Below are three common tools used in the deviation management process, but in this authors opinion are used the wrong way.
Expecting Automated Systems to Define the Deviation Management Process As noted above, Sparta Systems – Trackwise®, Master Control-Quality Management System (QMS) or Pilgrim Quality – SmartSolve® and others are frequently used to manage deviation data. As data management tools they are very effective in capturing problems, tracking the closure times, and who provided the closure. But as in any automated systems, there are examples where these tools have been employed effectively and where these tools have been employed to the detriment of companies as they document repeated deviations and the failures to eliminate them. Pharmaceutical and Medical Device companies frequently have failed to have well defined deviation management systems that they are /were attempting to automate. Too often, organizations stop short by allowing the tool to define their deviation/CAPA process rather than procedurally defining their own process and implementing a tool configured to meet the needs and requirements of their process. While commercially available automated systems do a good job in capturing data and allowing for tracking and trending deviations, their out of the box systems are more effective at defining processes that manage data than they are at defining processes for investigating, analyzing and correcting problems. The tools typically provide locations for the outputs of these processes without defining the execution of these processes. An additional problem routinely identified with this approach is that user requirements have not been adequately defined for either the tool or the process and therefore the resulting validation efforts are inadequate/incomplete. Regulators have pointed out that companies are using the tools in the wrong way by the observations they write. Several User Requirement failures routinely encountered by auditors and regulators, demonstrate how using the right tool in the wrong way will not. Each of these are described below:
Absence of or incompletely defined outputs, e.g.
Failure to have defined a user requirement for formatted outputs of the electronic records. Companies are frequently unable to generate complete and legible paper representations of the electronic documentation or to provide true and accurate copies of the electronic documentation.
2. Insufficient size and number of electronic fields
Limiting field size and number without ensuring that necessary information that clearly describes a step often leads to either missing information or ‘see attached’ statements. These both limit the usefulness of these fields as searchable tools during trend analyses.
3. Undefined and unlinked attachments
Attachments should assist in providing facts and telling the story about the issue, how it was investigated and resolved. Where these documents are not coherently linked to the deviation, or where their organization is unclear, their purpose is no longer evident.
4. Poorly defined trending categories
The key to a good deviation management program is the ability to trend issues to ensure that they have been eliminated. The use of too broad or too narrow of categories and tracking by root causes often obscures issues that are repetitive or suggests linkage of issues that have nothing to do with each other.
2. Fishbone /Ishikawa Diagram/5 M’s
While Fishbone /Ishikawa Diagram/5 M tools are important analytical tools for ensuring investigations consider all aspects of issues that could be involved in a problem, too often this tool is used more in the investigation process as an elimination tool. As such the investigations typically assume a defined set of causes and proceed to gather information that eliminates those causes from consideration, often without thoroughly evaluating each of the items. More often than not, these investigations eliminate all ‘causes’ except for personnel and then attribute the cause of the problem to be human error.
The intent of the tool is to appropriately and thoroughly evaluate each item for its contribution to a problem to avoid oversimplifying issues. Some examples of over simplification:
When evaluating procedures/methods, the focus is more on does the procedure and or steps exist rather than are there specific steps that clearly provideinstruction for the consistent execution of processes and are they effective?
When evaluating equipment, the focus is on whether there were any objections or notifications documented in logs rather than do all parameters (including but not limited to speed, pressure, alarms, sound during operation, etc) demonstrate the equipment was performing as designed?
When evaluating measurements the focus is on whether values are within or outside of specification rather than does the variability of the measurement impact the conclusions?
3. 5 Why’s
As noted above, the purpose of the 5 why approach is to delve deeper into a problem to determine if there are multiple actionable causes supported by the data. However, our focus often results in trying to get to the ‘one right answer’ or the easiest solution to fix ending in a lack of depth in our evaluations. Conversely, the process results in a never ending theoretical evaluation of all possible events without considering whether they are probable or whether the current situation conditions support the identified cause.
VII. So What Can We do differently?
A. Proactively evaluate systems and remediate where necessary As noted above, the expectation of regulators is that the vast majority of our efforts should be in ensuring problems never occur in the first place rather than in dealing with problems after they arise. Systems such as calibration, maintenance, training, qualification, standard operating procedures, test methods, etc. should be designed to ensure activities are suitable for their intended use and operate consistently. Where we have a lack of detailed knowledge and understanding, efforts should be undertaken to obtain this knowledge. Where necessary addition of steps to error proof processes should be undertaken. Frequently these error proofing involve more specific details in set up, take down and cleaning of equipment. As noted above, equipment, personnel, materials, methods, environment and measurements play important parts in consistent process execution and therefore are all potentially contributing parts of a problem. When a problem occurs, it is critical to evaluate the various systems we have in place for ensuring these problems do not occur. At a minimum we want to capture the occurrence so that trend analysis can be used to recognize whether one or more systems is having multiple failures. For example, when an equipment failure occurs, we should evaluate the occurrence and timing of preventive maintenance, calibration, cleaning, lubrication, qualification and measurements to ensure those systems were functioning appropriately. That is, not just is it within the designated windows, but did the activities occur recently or are they near deadlines and are those windows appropriate.
B. Establish/Reestablish the appropriate roles and responsibilities Investigations are best executed with teams made up of persons knowledgeable about the issue as well as with persons that provide the independent eye. The Quality organization has specifically required roles of oversight and must ensure that it maintains this role. Oversight should ensure the process executes as designed and documentation is generated to reflect that activity. However, Quality organizations must also have staff that partners with the process owners to ensure that investigations remain objective, fact based, and do not devolve into opinions. At the end of the day the company as a whole must be proud of the documentation that is created, irrespective of the underlying issue that was encountered.
C. Use tools to document facts and to evaluate the gathered facts for what are they telling you. Determine real root causes.
Facts are the most critical factor in successful investigations. Timely gathering of facts at the location where the problem occurred ensures that they represent what was actually going on at the time this issue arose. Delays in fact gathering and attempting to create the investigation from behind a desk frequently lead to gaps in our information (don’t remember, disassembled equipment, missing records, moved items, etc.) about the issue at hand. Answering the basics of who, what, when, and how an issue arose allows us to determine why it occurred. Once we know why it occurred, we can determine what actions are required to ensure it does not recur.
Well-designed tools both prompt the investigator to document gathered facts as well as facilitate capturing how the conclusions were reached.
D. Link Problems to systems and remediate systems as necessary
For more than 10 years FDA and other regulators have been talking about quality systems in various regulations and guidance documents. Many of these internationally recognized systems are described in ICH Q7-Q10 for pharmaceutical systems and in various ISO guidances, such as ISO 13485 for device systems. Well-functioning systems ensure problems do not occur or when the do the systems are in place to minimize their impact on patients. When problems occur, it is critical to successful companies that they also evaluate their systems to ensure they are or continue to function to eliminate problems and minimize any problems that occur. Where problems point to one or more systems not performing as required, these systems must be remediated.
E. Tell a good story
Well written deviation/CAPA documents coherently describe the sequence of events that occurred, and both the conclusions that were drawn as well as how those conclusions were reached. All conclusions should be supported by facts.
Your goal is always that any reviewer reaches the exact same conclusion as you do. A well written story that is coherent, comprehensive yet concise and clearly shows that issues were identified, they were promptly investigated, that appropriate actions were taken to prevent impact to patients and that the issue was resolved in a timely manner is fundamental in eliminating citations from auditors and regulators.
VIII. Conclusion
While the industry has spent much time and effort in dealing with deviations, it has very little to show for the investment. This article attempted to point out some of the common mistakes being made across the industry. By eliminating these issues and focusing more on the investigation process and capturing a coherent story, we will be better able to demonstrate that we thoroughly understand what happened and thus the actions we have taken to eliminate the problems are both appropriate and effective.
David W. Husman, Ph.D., RAC, CPGP, has over 30 years of diverse international experience in Regulatory Affairs, Quality Assurance, and Quality Control for the pharmaceutical, healthcare, and medical device industries. Dr. Husman has experience and expertise in compliance, Quality System, process mapping, design and implementation, auditing, aseptic processing, data integrity, validation, and information systems. He has worked with products that include both human and animal parenterals, medical devices, human tissues, blood products, biotech products, solid dosage, and vaccines. Dr. Husman can be reached at raqaprf@yahoo.com
