Principles for Assuring Audit Effectiveness

Auditors must be impartial and free from influences that could affect their judgement during an audit. If an auditee uses an internal auditor, the auditor must be independent of the systems or processes they are auditing. To eliminate bias, the auditor should not audit the systems or processes that they perform. For example, a company’s internal auditor should not audit the audit process. To avoid this conflict, many organizations hire an external auditing organization to perform their audits. When selecting an external auditor, it is important to ensure that the auditor is impartial. They should not be an authorized representative of the manufacturer, should have no financial interest in the company being audited, and should not be auditing processes that they were involved in developing or remediating in the past.  In addition to impartiality, it is imperative to ensure that auditors are competent to perform an audit. This competency includes education, skills, and experience relative to auditing, regulatory requirements, and the devices and technologies being manufactured. Auditing organizations should take the appropriate measures to ensure that the auditors they select are impartial and have the appropriate skills and experience to perform the audit they are assigned to.  External or 3^rd party auditor’s qualifications need to be maintained as part of the supplier qualification files.

The auditing organization and auditee need to agree on the scope of the audit in advance.  This is not to imply that the auditee determines the specific aspects of a process, documents, or records to be audited.  Rather there should be an understanding of which QMS processes are to be audited, relevant products, applicable regulations and standards, and retrospective date range to be covered when records and data are sampled.  The auditing organization has the responsibility for developing the audit plan and schedule in accordance with the agreed-upon scope of the audit.  The audit plan should be provided to the auditee such that there is adequate time for review and discussion before the initiation of the audit.   Getting feedback from the auditee helps ensure the audit scope and process meets everyone’s expectations.  In addition to the objectives and scope, the roles, responsibilities, and authorities of all parties involved in the audit process should be defined and documented within the audit plan. This will clarify expectations and create accountabilities for all parties involved. 

To ensure the audit runs smoothly and efficiently, adequate resources should be allocated by the auditee to support the audit. These resources should be planned and allocated in advance of the audit. Resources to consider include:

• Assignment of appropriate employees to provide administrative support for document delivery, notetaking, and logistical oversight,
• Assignment of Subject Matter Experts (SMEs) in areas subject to audit,
• Arrangement for IT support services,
• Prioritizing audit support over routine daily tasks for those involved in the audit,
• Provision of adequate space for conducting the audit,
• Translators, as needed to support international audits,
• Provision of technical tools and resources (projectors, copiers, office supplies, etc.), and
• Ready access to document management systems, data, records, and the personnel needed to navigate these systems.

When planning the audit, the auditee should work with the auditor to develop a schedule for the audit to ensure the appropriate subject matter experts (SMEs) are available for interviews.

Most companies will set up a “back room” to support the audit taking place in the “front room.” A back room coordinates timely retrieval and organization of documents and records for delivery to the auditor, coordinates and prepares SMEs for their interviews, coordinates plant walk-throughs and area inspections, coordinates logistics for cleanroom access, communicates audit progress to internal stakeholders, and oversees hospitality services such as translation, catering, and transportation. Ensuring the appropriate resources are available will maximize the effectiveness of the audit and communication during the audit, and ensure that audit results and conclusions accurately represent the state of the systems audited.

There should be persons assigned responsibility for the overall management of the front room and the back room. The front room is typically managed by the audit host who should have relevant experience in conducting and managing audits and inspections. The back room is often managed by a QA or document control professional with intimate knowledge of the company’s QMS, operations, and product lines.

Notetakers and “runners” are an important part of the administrative resources to support the audit. For audits that are planned for 8-hour days, it is important to have multiple notetakers so these can be switched out to prevent fatigue. Notetakers are especially important during inspections by regulatory authorities where formal responses to the audit will need to be generated. Having an accurate transcript of the audit proceedings can help those preparing responses to understand the context of inspection findings and the expectations of the personnel performing the audit or inspection. Document runners relay document and interview requests from the front room to the back room and documents, records, and data from the back room to the front room. Runners need to be stealthy and able to enter and exit the front room unobtrusively without interrupting the audit in progress.

Audits should be conducted in accordance with defined procedures and the approach of audits of similar type and scope should be consistent. Procedures for auditing should include details on managing audits and should include expectations regarding technical and administrative audit support. The procedures must consider relevant regulatory requirements. It is helpful for both the auditee and the auditing organization to have procedures detailing their audit processes. Documentation from each audit must be maintained according to regulatory requirements and internal procedures for document control, CAPA, etc. It should be traceable to the audit plan and different audits of the same system should be continuous (i.e., consider previous audit observations.)

The auditing organization should compile audit findings into a final report, which provides the auditee with the opportunity to review the auditor’s findings and implement corrective actions and quality improvements to address those findings. Typically audit findings are weighted based on risk to product, process, and compliance.  The audit plan should define the terminology and definitions to be applied when assigning risk levels to findings (e.g., critical, major, minor).

It is important for auditing organizations and their auditors to maintain confidentiality and professionalism and ensure all work is performed ethically. Any documents, records, data, and information associated with an audit or obtained during an audit are considered confidential. Disclosure of information and documents to a third party is prohibited, except when approved by the auditee or in the case of a regulatory requirement. It is important for auditors to practice professional care, use good judgement, and ensure their behavior is ethical when performing an audit. This is important during both the management and planning of the audit and the execution of the audit.   Cybersecurity measures must be adequate to protect audit information that is being transferred electronically or shared electronically between the auditee and the auditor.  Audit procedures should define measures to be taken to protect electronic and paper document security and confidentiality during audits.  

To provide the auditee with confidence in the output of the audit, results should be consistent and accurate. Auditors need to provide an adequate opportunity for the auditee to provide objective evidence in response to audit questions.  Auditors need to review objective evidence and ask questions to ensure they are understanding not only what they are reviewing but whether or not there is alternative evidence that might better address the question or issue at hand.   Sometimes it is helpful for the auditor to word questions in a few different ways to ensure that the request is being understood by the auditee.   Effective communication and confirmation of facts presented help ensure audit accuracy.   This is important as audit results and conclusions are inherently based on a limited sample of objective evidence (records, data, etc.).   

Auditing organizations should implement a quality management system to govern their audit work. Creating and maintaining a quality system will help the auditing organization to ensure that their audits are performed according to applicable regulations and standards, such as 21 CFR Part 820 Quality System Regulation and ISO 19011:2018 Guidelines for Auditing Management Systems, and guidance documents. Auditing within the construct of a quality management system will support consistency between auditors, maintain quality of deliverables, and encourage the auditing organization to implement continuous improvements to their audit processes. Implementing and maintaining a QMS within auditing organizations helps drive audit effectiveness and helps ensure that the auditees realize the greatest benefits from the audit process.