Back to Quality Matters Home

Published in Volume 37.1 of Quality Matters, Spotlight Article

Addressing Bias in the Quality World

SQA Communications and History Committee
See Author List

Download this article as PDF

 

Introduction

As Quality Assurance professionals, we are aware of the impact bias can have on the quality and integrity of research and manufacturing.  However, we may not recognize the extent or impact that the different types of biases, including our own, have in our jobs and how they may impact our decision-making or ability to be objective. Bias is defined by various sources as unfairness or prejudice towards or against something, someone, or a group. In research, this is often called systemic bias or distortion.  So how does bias come into play in the role as a Quality Assurance professional?  

We are all influenced by our experiences which can sometimes result in unconscious bias in certain areas.  For illustrative purposes, here is an example:  A father and young son are in a car accident.  The father dies at the scene of the accident.  The son, who has survived the accident, is rushed to the closest emergency department.  Upon evaluation, the emergency department sends him to a hospital surgeon for surgery.  The surgeon states “I can’t operate on this patient; he is my son!”  How can this be?  The father died in the car accident.  Perhaps you are puzzled by the example or perhaps you immediately realized that the surgeon was the patient’s mother.  

In this article we will begin to explore different types of bias, how they impact research, manufacturing, and our ability to be independent auditors, and finally some ways to avoid bias and be a critical thinker!

Does Bias Play a Role in Science?

Astrophysicist Dr. Neil deGrasse Tyson described the role of bias in science during a Distinguished Speakers Series at the University of Buffalo on 31 March 2010.  He explains:
‘Scientists are human like everybody else.  We are trained to minimize our bias in our experiments and interpretations but that is sometimes hard if not impossible to do.…. The enterprise of science has built in error checking mechanisms.  Do you know what that is? Someone else who is checking you who has a different set of bias or no bias at all.  If they get the same results as you, that adds confidence that your result is right.  If they get a different result, either they are wrong, or you are wrong or both of them are wrong.  That is the interesting thing about science, if you are both saying different things, you both can not be right.  The enterprise of science sorts that out.  Eventually over time, biases are revealed.  If it is later shown that your bias interfered with your result, your next study will be significantly discounted in peoples trust.  It would interfere with your career as a scientist.  This is the value of science as an enterprise.  Because we know as humans we are susceptible to bias of all kinds." ¹

Conscious and Unconscious Bias in Science

There are many defined and measurable types of biases in research and they often are given different names based on the discipline of study. Readers may be familiar with the same ‘type’ of bias but use different terminology depending upon the context, discipline or activity.  We have provided a list of common biases in the glossary and will be using these terms throughout this document.  But remember other terms may apply in your field or discipline.

Bias may be conscious or unconscious.  Conscious biases, sometimes known as explicit biases, are those that a person is aware of and attempts to control or mitigate.  Such biases may be real or sometimes are just perceived.  Conscious biases are often controlled through steps, such as Conflict of Interest (COI) declarations and COI management plans, or through education.  

Unconscious biases, sometimes called implicit biases, are much more of a challenge in that the affected person is unaware of their biases, whether favorable or unfavorable, and they are without the person’s awareness or intentional control.  The affect is therefore often unknown and uncontrolled. Unconscious biases are individualistic (we all have them) and form as a result of education, experience and exposure to social and cultural norms, and often result from an individual’s perception and categorization of the world.  It can result in unsubstantiated underlying attitudes, beliefs, stereotypes and awareness that can affect behavior and understanding in all aspects of an individual’s life.  Unconscious biases are more common than explicit biases.  In fact, they are pervasive and do not necessarily align with our conscious beliefs.  Implicit bias usually favors an individual’s in-group or discipline norms.  In fact, it was the acknowledgement of unconscious bias that lead Sir Francis Bacon to propose the ‘scientific method’.

Bias in Research

In the next several sections we will take a specific look at biases in research that we, as QA professionals, may have assessed or experienced or that we should consider in our role as quality auditors and managers.  We will use our common regulatory disciplines to highlight and provide examples of some common biases but many of these biases are cross-disciplinary.  Bias cuts across all scientific disciplines.

Bias in Good Laboratory Practice Studies

Non-clinical research is subject to bias in design, conduct, interpretation, and reporting like any other research endeavor.  The Good Laboratory Practice (GLP) for non-clinical laboratory studies regulations (e.g., 21 CFR 58 and 40 CFR 160)² require research protocols to not only provide a description of experimental design but also requires that it include the methods for the control of bias.  In some protocols we see this as a distinct section of the protocol listing all bias controls but in other protocols, we see bias controls described throughout the protocol.  Some examples might include:³

1. Standardization
2. Calibration
3. Randomization
4. Blinding/Masking
5. Peer Review
6. Report Review

Standardization: As a quality management system, the GLPs rely heavily on standardization to control bias.  The GLPs require both study protocols and Standard Operating Procedures (SOPs) be written and authorized prior to study conduct.  Both assist in ensuring that study methods are conducted consistently to reduce the introduction of individual or procedural bias by research personnel.  Protocols need to be approved by both the Study Director and the Sponsor prior to study start.  SOPs need to be set forth in writing and authorized by Test Facility Management in order to ensure quality and integrity of the data in the course of a study.  Personnel are required to be trained in SOPs that govern or impact their study participation.  SOPs reduce bias by ensuring methods are consistently done in the same manner which limits or prevents errors or individual differences in procedure.  Deviations from SOPs are required to be documented in study records, acknowledged by the Study Director, and any impacts on study integrity documented in the deviation documentation and/or the final report. SOPs also delineate acceptance and rejection criteria and the use of controls; all are methods designed to prevent procedural bias. 

Calibration:  Equipment calibration, testing and standardization operations are required by the GLPs and must be in written procedures, including the methods, materials and schedules to be followed and documented.  Such activities assist in assuring that measurements are accurate, and that measurement bias is limited and that bias in data is not introduced by lack of equipment maintenance or use; or that any inherent measurement bias is limited or of known value.  

Randomization: Randomization is often used as a bias-free process by which test systems (e.g. study animals, cell tissue tube, aquaria, human subjects (found in clinical research)) in a study are assigned by chance either to the study, to the treatment group, or to different interventions.  Randomization is often an inherent basic assumption of the experimental design and statistical analysis because it minimizes the differences between groups by equally distributing systemic differences in the population among the research groups.

Blinding/Masking:  Also frequently used in clinical research, blinding/masking is a technique used in GLP studies to prevent performance bias by keeping the test, control, and reference group assignments hidden from study personnel, particularly those whose activities can be influenced by knowing the treatment groups and making inferences about those groups.  Measurement of outcomes, in particular subjective ones such as health effects or clinical system observations, can be easily influenced when treatment is known and not knowing treatment groups can eliminate such bias.  A blinding/masking plan either as a standalone document or as part of the study protocol, is advised, listing who is blinded and who is not and describing the conditions of unblinding, especially in the event of adverse responses.   How blinding is to occur to ensure study integrity is critical and measures to control information should be carefully considered including test/control article preparation and labeling, cage card information, data sheet information, randomization impacts, specimen labels etc.  Coding schemes and a separation of duties (e.g., test article administrator and observational staff) are often needed and should be carefully implemented to ensure study integrity.

Peer Review:  Peer Review, such as during pathology, but also valuable in other subjective measures, can reduce bias by identifying where individual bias may occur due interpretation of scoring standards, previous experience, and training that is not internally consistent, and other sources of systemic error in observation or measurement.  Having two or more individuals independently measure or score during the study can identify sources of bias and can be statistically evaluated to provide a measure of potential internal bias.  Note that the GLPs require the signed and dated reports of each individual scientist or other profession be provided in the Final Report, so that a receiving Authority can easily confirm if the Study Director fails to represent the information accurately or has a biased interpretation.
Report Review:  The GLP Final Reports in addition to including all peer review reports, must also include a description of all circumstances that may have affected the quality or integrity of the data per regulation.  Therefore, an assessment of known biases must be included in all GLP Final Reports. 

 

Bias in Good Clinical Practice Studies 

In the world of quality assurance, bias is something that we strive to avoid.  We design clinical trials with the aim of eliminating bias by evaluating and reducing potential conflicts of those who contribute to clinical trials whether it be representation of the Institutional Review Board, the clinical investigators managing the study or the clinical trial participants. In Good Clinical Practice, we often find these types of bias:

1. Population bias
2. Financial bias
3. Attribution Bias

Population bias: Ethics committees (e.g., Institutional Review Boards/IRBs) are required to be diverse: "Every nondiscriminatory effort will be made to ensure that no IRB consists of entirely men or entirely of women." (21 Code of Federal Register (CFR) 56.107)⁴.  These committees should comprise diverse professions, scientific and nonscientific, and at least one person who is not affiliated with the institution in order to not introduce bias during the trial review and approval process.  When auditing at a clinical site, we check that subjects who are screened but not selected to participate have adequate documentation about why they failed screening to ensure selection is unbiased as another check on population bias.  Based on study design, care is taken not to favor an age group, ethnicity, or gender with the aim of not biasing the results when selecting subjects for a study.  Cherry picking (e.g. the act of selecting based on specific characteristics) subjects to participate in a clinical research study, based on whether they have a better chance of responding to study, is not allowed.

Financial bias: “One potential source of bias in clinical studies is a financial interest of the clinical investigator in the outcome of the study because of the way payment is arranged or because the investigator has an equity interest in the sponsor of the covered trial.” (21 CFR 54.1)⁵.  We review financial disclosures of clinical trial investigators to ensure that they will not be biased toward a successful clinical outcome. 

Attribution bias: There are other situations where bias may be obvious but ignored.  Many of us have been part of auditing teams where we have seen principal investigator or study staff bias towards an auditor based on the auditor’s race, gender, and age.  This bias can affect the audit conduct and unless it includes sexual harassment or making someone uncomfortable, there is usually not much that can be done except recognize it and try to minimize the effect.  For example, an older and more experienced auditor went to an investigator site with a younger auditor who was leading the audit (her first as lead).  The principal investigator would acknowledge her questions but focused on the older auditor when answering.  And if he had a question, he directed his question to the older auditor.  This awkward situation was managed by the older auditor looking to the lead auditor to confirm that the older auditor’s response was correct, acting as if she was the one being trained.    The principal investigator adjusted his perception and the rest of the interview went well.  The lead auditor was grateful for this approach and both the younger and older auditors laughed about the situation later. It could have been detrimental to her career development if the older auditor had taken over the interview and lost the training opportunity.  It provided an opportunity to discuss how to handle this type of bias when a young auditor goes solo.  How would you have responded?

Our powers of observation are what make us good auditors, but sometimes we can miss bias, discrimination, and unfair treatment.  It’s not always about numbers or data.  We need to be aware of our bias; we need to address bias as best we can in a situation and call it out when warranted.  We need to learn how to address it, cope with it if we are the target, and learn from it.
  

Bias in Good Manufacturing Practice

The manufacturing process and associated analytical work evaluating the finished product are validated, but may yet contain bias. Bias in Good Manufacturing Practices exists in several forms including:

1. Test bias
2. Data evaluation bias

Test bias:  exists where a test must be conducted by a particular laboratory analyst or using a specific piece of instrumentation in order to achieve a result that is within specification. The analytical test may not be robust enough to support testing by multiple individuals, multiple facilities, using multiple pieces of instrumentation, etc. This bias can be counteracted by robust validation testing that includes multiple analysts over several days using varied pieces of equipment to ensure there is no bias to a particular factor within the test method or procedure.  If this level of testing is not available or practical, then principles of incurred sample reanalysis, internal controls and control charting and other quality control methods should be applied.

Data evaluation bias:  occurs when an inconsistent number of significant figures is used throughout a calculation or due to a rounding issue of the final reported value. This may be a result of the programs used to calculate the data versus a manual calculation or an alteration of when the data is set to report the minimum number of significant figures. One example is the program Microsoft Excel will round a number down when the digit after the number of assigned significant figures is a five (5), while other programs will round this number up. This bias may be eliminated through validation of the formulas used during the calculations, verification of the calculations to reproduce the result through a secondary method, and validation of the calculation spreadsheet to ensure the formulas are not corrupted through repeated use.

Any sources of bias within the GMP world should be evaluated as part of the robust validation criteria used during manufacturing and method validation. The strict validation criteria of the regulation are in place to minimize sources of bias, and ensure good quality of the resulting product and data. The quality checks conducted by Quality Control and Quality Assurance review provide additional assurance of limited bias within the final product and data supporting release of that product.

Machine Bias

Anyone who recently watched the Netflix docu-drama "The Social Dilemma" ⁶ has seen a dramatization of how algorithms and artificial intelligence can perpetuate human biases once they are introduced into the system.   Surely you have seen that early Google search results are there because of paid advertising algorithms.  Perhaps you have fallen victim to the autocorrect programs on your smartphone or email program that thinks it knows better than you what you really meant to type. 

In "The Social Dilemma" example, algorithms that discover preferences of individuals are designed to lean into user history and continue to present similar content to hold your interest on social media platforms. When taken to an extreme, this limits exposure to contrasting views over time as the algorithms gather more data and presents only content the user wants to see. Without an individual’s efforts to seek a variety of sources and examine contrary views, they will find themselves within an echo chamber where they only see things they already agree with (whether or not this information is representative of the dialogue and information available to the whole of society.)

The same tendencies could play out within the context of the life sciences. Computer algorithms and artificial intelligence are key to making intelligent sense of the “big data” available. Future scientific advances in the areas of Quantitative Structure Activity Relationship (QSAR) and in silico predictions of toxicity would be nearly impossible without the aid of algorithms and machine learning. Many hours of work, hundreds if not thousands of animals, and a great deal of money can be saved through embracing these technologies. To minimize the introduction and perpetuation of bias, they need to be used in a thoughtful, understanding way. 

The central problem with algorithms is to determine if the information used to set up the structure is free of bias itself.   A simple example is that inappropriate rejection of data outliners can easily bias the mean of a dataset towards an investigator’s targeted outcome.   Deriving algorithms from improper datasets was summed up nicely in an episode of the podcast You Are Not So Smart (highly recommend if you are interested in this article) by philosopher Shannon Vallor at Santa Clara University.  “I want a machine-learning algorithm to learn what tumors looked like in the past, and I want it to become biased toward selecting those kind of tumors in the future. But I don’t want a machine-learning algorithm to learn what successful engineers and doctors looked like in the past and then become biased toward selecting those kinds of people when sorting and ranking resumes.” ⁷   

Machine learning can also be problematic when the algorithm is only computational and lacks bias for human ethics and values.  In Shannon Vallor’s example, the computer chooses appearances as valid criteria for employment decisions.  When a computer might someday weigh a “Trolley Problem”⁸ in my child’s autopiloted car, it should not calculate that the lives of three deer matter more than the risk of sending the vehicle off a bridge.  In the sci-fi movie “I Robot,”⁹ the computer VIKI concluded Isaac Asimov's "Three Laws of Robotics" can justify murder: “To protect Humanity, some humans must be sacrificed. To ensure your freedom, some freedoms must be surrendered. We robots will ensure mankind's continued existence. You are so like children. We must save you from yourselves.”

Understanding the potential for human biases to enter into artificial intelligence (AI) and create machine bias is the first step to preventing it from happening. Future data inputs and their effect on the direction of the algorithms should be manually reviewed through a change control mechanism or periodically assessed against “gold standard” inputs for fully autonomous AI to prevent undesirable drift over time. As auditors, we should be critical of the assumptions made at the start of algorithm creation, the data determined acceptable for use in building databases, and the inclusion or exclusion criteria for future data to be entered into the system in order to protect against unintentional machine bias.    

 

Auditing Bias

Auditing is an activity that has a great potential to be biased. There is often the tendency when auditing to search for, or interpret, new information during an audit in a way that confirms our preconceptions, both good and bad.  At the same time we may unconsciously avoid information and interpretations that contradict those preconceptions (Confirmation bias).   Similarly, we are taught to come prepared for each audit and this process can result in having a significant amount of pre-existing information that can determine the whole tone and effort of the audit, again either good or bad, without seeing all the information or having a full understanding of the system or context of the pre-reviewed information (Anchoring bias).  This type of bias can also occur when an unconscious perception of the quality system is made based on just the first information reviewed or a previous inspection result.

Another potential audit bias that can occur early in the audit is based on positive attributes of the person, facility, data organization or other preliminary attributes (Halo Effect).  For example, the facility is nice and clean, therefore the data may be unconsciously assumed to be nice and clean and the audit may not be as thorough as needed or the conclusions may be less objective.  The opposite of the Halo Effect is the Horn Effect, where a single negative finding or trait may lead to a biased harsh conclusion and influence the entire inspectional process.

Biases may be more prevalent when multi-tasking or working under time pressure.  For example, when deadlines are tight it is easier to rush an audit to its conclusion by not having the time to get to objective information (Availability bias), review difficult or detailed documents such SOPs or maintenance log books, or to ask for additional information that will take a lot of time to review or sort. When multitasking, it is easy to forget to follow-up on a line of concern of inquiry when pursuing another thought or task and so judgments are made on unconscious biases.

These unconscious biases may be the familiarity you feel toward the individuals who represent the auditee (Affinity bias).

So as an auditing team how can you adjust, reduce or prevent audit bias?

1. Auditor rotation, that is rotating auditors can reduce biases based on familiarity, past experience or preconceived assessments.
2. Develop individual and team awareness and understanding of the presence and effect of bias in general in auditing and their own particular work and personal life.  As QA professionals we need to acknowledge that no matter how professional we believe we are, we are all susceptible to the effects of unconscious influences. However, through self-reflection, training, education, and networking, we take the journey needed to limit these unintended and sometimes destructive tendencies from our professional and personal lives.
3. Focus writing audit reports around objective evidence. Recognize the inherent nature of bias in auditing and develop team awareness and a bias-free culture in all aspects of work life.
4. Develop a personal culture, free of preconceived notions with a well-balanced professional skepticism, based on evidence and explanation.
   
   

Reporting Bias

Bias is often based on feelings and opinions rather than on facts. As QA professionals we often review reports to ensure they reflect the data.  It is critical that our review confirms that the report is an unbiased representation of data.  Two types of bias in reporting are:  

1. Systemic bias (also known as institutional bias)
2. Statistical bias

Systemic bias: also known as institutional bias, is the inherent tendency of processes to result in particular outcomes. For example, this often occurs when the majority of people adhere to existing rules or norms implemented at an institution without questioning the rules or norms.       

Statistical bias:  occurs when a calculation is made in a way that is systematically different from the population parameter being estimated.  Basically, this occurs   when there is a tendency to overestimate or underestimate a parameter.  For example, this can occur when a subset of dataset is selected for analysis rather than the complete dataset for a particular parameter. 
 
Many of us have seen where a report author expects the product(s) to perform the same way as it did in the past.  It is critical for QA to evaluate reports without bias to ensure the report reflects the raw data.

Considerations for Managing Our Own Bias

Remember from above, we are all inherently biased but we can put into place practices that will reduce our own bias and those inherent in the work we do.  As we proceed from the realm of scientific and research bias, where does societal bias step in?  And, when we look at societal bias, we may unmask unconscious and implicit bias that affects our job.  It is worth reviewing the unconscious bias from above in the context of managing bias.  

What is unconscious bias?

• Implicit attitudes, actions or judgments that are controlled by automatic evaluations without a person’s awareness
• We often make instinctive decisions/preferences about other people
• It plays a significant part in the way we engage people and the decisions we make about them
• It is a form of social cognition, an implicit bias that refers to the attitudes or stereotypes that affect our understanding, actions, and decisions in an unconscious manner.

Some interesting attributes of unconscious bias are:

• People will often not have insight into their own biases
• People under emotional or cognitive load are more likely to invoke bias behavior
• Unconscious bias affects our behavior in subtle and unintentional ways

When attempting to manage unconscious bias, there are many considerations.  Howard J. Ross, a lifelong social justice advocate and founder of diversity consulting company Cook Ross, Inc, wrote Everyday Bias: Identifying and Navigating Unconscious Judgements in Our Daily Lives¹⁰.  Ross is often considered one of the world’s thought leaders on unconscious bias.  Ross provides steps you can take to manage unconscious bias. Ross also provides questions and comments for individual exploration.  

1. Tell the truth to yourself.  Remember we all have unconscious biases.  Examples of exploration questions to ask yourself include:
   
   • What groups make you most uncomfortable (appearance/racial/cultural/religious, etc.)?
   • Notice the situations you are in when you feel most uncomfortable.
   • What kinds of people make you most nervous? Are these personal traits or group traits?
   • What kinds of people do you try the hardest to please?  What characteristics do they share?
2. Notice what influences your decisions.  Consider unconscious biases present in your decision-making.
   
   • How might my perceptions or bias be influencing my decisions?
   • Consider several of the last major decisions.  How were they influenced by your feelings about others?
   • Do you take extra time during times of stress to ensure you make a fair decision?
   • If you are feeling nervous in a social situation do you ask yourself, “What am I reacting to?”
3. Gather data about yourself.  
   
   • What has most influenced your thinking and feelings about other groups?
   • What results seem most accurate or off the mark?
4. Stretch your comfort zone.
   
   • Do your hobbies or home activities regularly expose you to diverse people and groups?
   • Do you attend local cultural festivals, eat at ethnic restaurants, and go online to research the origins of interesting new cultures?
5. Be open, seek feedback.
   
   • How do you respond to feedback from particular individuals?
   • Do you notice that you ask one kind of person for feedback, and not others?
   • What can you learn about yourself from this?
6. Leading with Authenticity
   
   • Be aware of own unconscious biases
   • Avoid supporting or strengthening your unconscious bias
   • Expose yourself to positive images or stories prior to key decisions
   • Recognize and be particularly vigilant in stressful situations or when you have incomplete information
   • Recognize and be particularly vigilant where your own biases are likely to be active
   • Proactively identify and implement strategies for promoting a culture of inclusion

 

Conclusion

As we have seen from the above descriptions, there are many types of bias within our personal and professional lives.  The initial step of changing our biases is to recognize them.  Howard Ross’ book provides us excellence guidance on how to identify our individual bias and outlines path to make changes.  A recent example of bias in healthcare is referenced in a warning letter issued jointly by the Food & Drug Administration (FDA) and the Federal Trade Commission (FTC).  On 21 July 2020, the FDA and FTC issued a warning letter to 21st Century Laser Med Pain Institute / Create Wellness Clinics for their claim “I’m going to give you the most unbiased, updated information on how to protect yourself from this supposed coronavirus, COVID-19 pandemic …”¹¹ As Quality Assurance professionals it is imperative that we strive to be as unbiased as possible to ensure objectivity and the safety of our patients using our products.

In conclusion, Dr Richard Feynman once stated with regard to bias “The first principle is that you must not fool yourself – and you are the easiest person to fool.”  As humans we are programmed to pay more attention to evidence that agrees with our preconceptions and to reject evidence that doesn’t.   And so there is much more to cover on this topic.  The authors look forward to future opportunities to develop additional articles and collaborations and understanding on this topic.  We have started to collect a list of articles and resources that we have found to be helpful (see Appendix 2).  We hope to keep expanding this resource list for all SQA members and we will post it and any update in the SQA toolbox.  Please send us any resources or articles on bias that you come across and would like to add to the resource list. 

So, what exactly is ‘bias’?  What do you think?  Let’s have the discussion!