When Audit Shows No Improvement: Understanding and Responding to Stalled Progress

Introduction

Clinical audit is a fundamental activity within the NHS, aiming to improve patient care and outcomes by comparing current practice against agreed standards. Typically, after an initial audit identifies areas for improvement and interventions are implemented, a re-audit is conducted to measure the impact of those changes. But what happens when the re-audit shows no significant improvement, or even a deterioration?

This can be a disheartening experience for clinicians who have invested time and effort. This resource explores common reasons for stalled progress, provides practical steps to diagnose the underlying issues, and outlines strategies for moving forward, ensuring your audit efforts contribute meaningfully to clinical quality.

Why This Topic Matters

A re-audit showing no improvement is not a failure of the audit process itself, but rather an indicator that the interventions were either ineffective, not fully implemented, or that the underlying problem is more complex than initially understood. Ignoring such a result can lead to:

• Continued suboptimal patient care: The issues identified in the initial audit persist, potentially impacting patient safety and outcomes.
• Wasted resources: Time, effort, and possibly financial resources invested in interventions that didn't work effectively.
• Clinician demoralisation: A feeling of futility can set in if improvement efforts repeatedly yield no results.
• Missed learning opportunities: Each audit cycle, especially one showing no improvement, offers valuable insights into system complexities and barriers to change. Understanding these is crucial for effective improvement work within the NHS.

Responding constructively to this situation is vital for maintaining the integrity of the audit cycle and driving sustained improvements in care.

Practical Explanation: Diagnosing Stalled Progress

When a re-audit shows no improvement, it's essential to adopt a diagnostic approach rather than immediately trying new interventions. Consider the following categories:

1. Issues with the Initial Audit or Standard

• Flawed data collection: Was the initial or re-audit data collected reliably? Were there inconsistencies in interpretation or methodology? This is particularly important if different people collected data for the baseline and re-audit.
• Incorrect baseline: Was the initial baseline data representative of usual practice, or was it an outlier? A 'Hawthorne effect' (where practice improves just because it's being watched) might have skewed the baseline.
• Unrealistic standard: Was the chosen standard aspirational rather than achievable in the current clinical context? While striving for excellence is good, an unachievable target can lead to continuous 'failure' even if some improvement occurs.
• Poorly defined measure: Was the audit criterion clear, unambiguous, and directly measurable? Vague criteria can lead to variations in interpretation.

2. Issues with the Intervention(s)

• Intervention design: Was the intervention (e.g., new protocol, education session, prompt in notes) well-designed and evidence-based? Did it target the root causes of the problem identified in the initial audit, or was it a superficial fix?
• Relevance to root cause: Did the intervention truly address why the initial standard wasn't being met? A common pitfall is treating symptoms rather than causes.
• Complexity: Was the intervention too complex for staff to adopt easily? Simple, clear interventions are often more successful.
• Lack of piloting: Was the intervention rolled out widely without a small-scale pilot to test its feasibility and effectiveness?

3. Issues with Implementation

• Poor communication: Were staff aware of the intervention and why it was being introduced? Was the purpose clearly articulated?
• Inadequate training/education: Was sufficient training provided? Was it accessible, engaging, and reinforced?
• Lack of ownership/engagement: Did the staff involved feel a sense of ownership over the problem and the solution? Was there adequate buy-in from key stakeholders, including senior staff?
• Resource limitations: Were there insufficient staff, time, or equipment to implement the change effectively?
• System barriers: Were there wider organisational or systemic barriers that prevented the intervention from being embedded (e.g., conflicting priorities, IT system limitations, inadequate supervision)?
• Sustainability: Were there mechanisms in place to sustain the change beyond the initial enthusiasm? Did the change become 'the way we do things here' or was it short-lived?
• Intervening factors: Were there other changes in the clinical environment (e.g., staff shortages, new policy, increased workload) that negatively impacted the ability to implement or maintain the intervention?

4. Flawed Re-audit Timing or Scope

• Too soon: Was the re-audit conducted too quickly after the intervention, not allowing enough time for the changes to become embedded and show an effect?
• Too broad/narrow: Did the re-audit measure the right things? Was the sample size sufficient and representative?

Common Pitfalls

• Blaming individuals: It's rare for audit failure to be solely due to individual negligence. Most issues stem from system problems. Focus on the system, not individuals.
• Jumping to new interventions: Without understanding why the previous attempt failed, a new intervention is unlikely to succeed and might repeat the same mistakes.
• Ignoring the result: Filing the report and moving on is a missed opportunity for learning and improvement.
• Lack of stakeholder involvement: Trying to drive change in isolation without engaging the staff who do the work, managers, or senior clinicians is a common reason for failure.
• Poor documentation: If the initial audit, interventions, and re-audit processes aren't well documented, it's hard to learn from them.

Step-by-Step Approach: Responding to No Improvement

1. Re-examine the Data: Scrutinise both the initial audit and re-audit data. Were there any anomalies? Was data collection consistent? Did the sample sizes have sufficient power to detect a change?
2. Review the Initial Audit Parameters: Re-investigate the standard, criteria, and methodology against the categories above (Issues with the Initial Audit or Standard).
3. Conduct a Root Cause Analysis (RCA) of the Intervention Failure: Utilise QI tools like a '5 Whys' analysis or a Fishbone (Ishikawa) diagram to explore why the chosen interventions didn't work. Focus on the categories: 'Intervention Design', 'Implementation', and 'System Barriers'. Involve front-line staff who were meant to implement the changes.
4. Engage Stakeholders: Present the findings to relevant staff, managers, and clinical leads. Facilitate an open discussion about barriers encountered and potential solutions. Their insights are invaluable.
5. Refine the Problem and Interventions: Based on your RCA, redefine the core problem (it may be different from your initial assumption). Develop new, tailored interventions. Consider using a 'Plan-Do-Study-Act' (PDSA) cycle approach for testing interventions on a small scale.
6. Plan for Robust Implementation: Think proactively about how to overcome previously identified implementation barriers. This might include: clearer communication, better training, designated champions, integrating changes into existing workflows, and ensuring senior buy-in.
7. Establish a Clear Re-audit Strategy: Plan when and how the next re-audit will be conducted, ensuring sufficient time for changes to embed. Consider using a different data collection method if the previous one was flawed. Ensure the new measure is sensitive enough to detect anticipated improvement.
8. Document and Communicate: Maintain comprehensive records of your findings, RCA, revised plan, and subsequent actions. Communicate progress and setbacks transparently to all involved parties.

Example in Clinical Practice: Venous Thromboembolism (VTE) Prophylaxis Documentation

A junior doctor team conducted an audit on appropriate VTE prophylaxis documentation for medical inpatients, finding only 60% compliance against a 95% standard. They implemented an intervention: a re-designed prescription chart section and a brief teaching session to medical staff.

Six months later, a re-audit showed only 62% compliance – no meaningful improvement.

Applying the Step-by-Step Approach:

1. Re-examine Data: Data collection was consistent, but initial audit showed variation among wards. Re-audit sample size was adequate.
2. Review Initial Audit Parameters: Standard (95%) was considered achievable. Criteria were clear.
3. Root Cause Analysis: The team facilitated a discussion with ward doctors, nurses, and pharmacists. Key findings:
   • Intervention Design: The new chart section was clearer, but junior doctors often completed it under pressure during ward rounds and didn't always remember the teaching.
   • Implementation: The teaching session was a one-off, and new junior doctors joining the rotation missed it. There was no reinforcement. Nurses felt it was a doctor's responsibility, and pharmacists only flagged major omissions, not minor documentation issues.
   • System Barriers: No 'hard stop' in the electronic prescribing system if VTE assessment wasn't completed. Competing demands during ward rounds meant documentation often rushed.
4. Engage Stakeholders: They presented findings to the VTE committee, ward managers, and the medical director.
5. Refine Problem & Interventions: The problem was refined: Lack of consistent, reinforced VTE documentation prompt at point of prescribing, alongside inadequate system support. New interventions formulated:
   • Automated prompt: Work with the digital team to incorporate an unavoidable VTE assessment prompt into the electronic prescribing system for all new admissions.
   • Ward-based VTE champions: Identify enthusiastic junior doctors on each ward to act as local champions, providing informal reminders and 'micro-teaching'.
   • Integrated into induction: Include VTE documentation as a mandatory component of junior doctor induction and regular grand rounds.
6. Plan for Robust Implementation: Digital team committed to rolling out the prompt. Ward champions identified. Induction updated. Monitoring by ward managers and pharmacy.
7. Establish Re-audit Strategy: Plan a re-audit in 3 months once the electronic prompt is live and embedded, assessing both documentation and actual prescription of appropriate VTE prophylaxis.
8. Document & Communicate: All actions documented and shared with the VTE committee and medical workforce team.

This systematic approach transformed a 'failed' audit into a deeper understanding of system issues, leading to more targeted and potentially effective interventions.

How Lazomis Can Help

Lazomis provides tools that can streamline the audit process, even when faced with no improvement:

• Audit Setup & Data Collection: Use Lazomis's structured audit templates to ensure consistent data collection for both initial and re-audits, reducing the risk of methodological flaws. The platform helps standardise criteria and includes built-in validation checks.
• Action Planning & Tracking: Document your interventions, link them to specific audit findings, and track their implementation progress. This helps you monitor whether your planned changes are actually being put into practice.
• Reporting & Visualisation: Generate clear reports and visualisations that highlight trends and differences between audit cycles. This makes it easier to spot areas of no improvement and present complex data to stakeholders during your root cause analysis discussions.
• Resource Library Integration: Seamlessly access relevant resources within Lazomis that can guide your root cause analysis or help you design more effective interventions, drawing on best practice and improvement methodologies.

This resource supports, but does not replace, clinical judgement. Local policy, formulary and specialist advice should be followed.

Key Takeaways