Demonstrating Improvement After Clinical Audit

Introduction

Clinical audit is a cycle of measurement, improvement, and re-measurement. The initial audit identifies where practice deviates from standards; interventions are then designed and implemented to address these gaps. The critical final step, often overlooked or inadequately addressed, is the re-audit and the demonstration of sustained improvement.

Successfully showing improvement not only validates the effort invested but also provides essential evidence for governance, CQC inspections, and professional development portfolios, such as for ARCP.

Why this topic matters

Closing the audit loop

A well-conducted re-audit closes the audit loop, confirming whether implemented changes have effectively improved practice. Without this step, the audit process remains incomplete, and the impact of improvement efforts cannot be reliably assessed.

Demonstrating impact and value

In an NHS environment focused on outcomes and efficiency, demonstrating positive change is crucial. Showing improvement provides concrete evidence of better patient care, enhanced safety, or more efficient processes. This evidence can be vital for securing resources, influencing policy, and acknowledging the hard work of clinical teams.

Professional development and accountability

For junior doctors and trainees, actively participating in and completing the audit cycle, including demonstrating improvement, is a core requirement for training progression. For QI leads and clinical managers, it’s a key aspect of their role in ensuring high-quality, safe care.

Practical explanation: Re-audit and demonstrating improvement

When to re-audit

The timing of a re-audit is critical. It needs to allow sufficient time for the implemented changes to embed and demonstrate their effect. Factors to consider include:

• Nature of the intervention: Simple changes might show results quickly, while complex changes (e.g., new pathways, cultural shifts) require more time.
• Frequency of the audited event: If the event occurs infrequently, a longer period may be needed to collect a representative sample for re-audit.
• Local context: Discuss with your supervisor or audit lead to agree on an appropriate timeframe, typically 6-12 months after implementation of interventions.

Data collection for re-audit

The re-audit should ideally use the exact same methodology as the initial audit to ensure comparability. This includes:

• Same standards/criteria: Do not change the criteria or standards you are auditing against.
• Same patient population/sampling method: Ensure the re-audit sample is drawn from the same population and using the same method (e.g., consecutive patients, random sampling) as the initial audit.
• Same data points: Collect the same specific data items for each criterion.
• Same data collection tools: Use the original data collection form or digital tool if available.
• Comparable sample size: Aim for a similar sample size to the initial audit to allow robust comparisons, especially if the initial sample was statistically powered.

Deviations from the original methodology should be clearly documented and justified, as they may impact the validity of your comparison.

Analysing and comparing data

Once re-audit data is collected, the analysis involves a direct comparison with your initial audit findings:

1. Calculate adherence rates: For each standard/criterion, calculate the percentage of cases meeting the standard in the re-audit. Do the same for the initial audit data.
2. Quantify the change: Determine the absolute percentage change for each criterion (Re-audit % - Initial Audit %). This clearly shows the degree of improvement or, conversely, areas where further work is needed.
3. Identify trends: Look for patterns across multiple criteria. Has there been a general improvement, or are there specific areas that have responded well to interventions, while others have not?
4. Statistical considerations (optional but useful): For larger datasets, consider consulting a statistician or QI expert to assess if the observed improvements are statistically significant. This adds rigour to your findings, though it's not always required for simple audits.

Common pitfalls

• Changing audit criteria: If the standards or criteria change between audits, comparing the data becomes difficult or impossible.
• Inconsistent data collection: Using different methods or data points in the re-audit compromises comparability.
• Insufficient time for change: Re-auditing too soon after implementing interventions may not capture the full effect of the changes.
• Lack of clear interventions: If interventions were not well-defined or implemented effectively, re-audit results are unlikely to show improvement.
• "Audit fatigue": Teams can become disengaged if the audit cycle doesn't demonstrate positive impact or if the process feels burdensome without clear benefits.
• Failure to report back: Not sharing re-audit results with the team, particularly if they show improvement, can demotivate staff and undermine future engagement.

Step-by-step approach: Demonstrating sustained improvement

Step 1: Review initial audit findings and interventions

• Revisit your initial audit report, focusing on the identified gaps in practice and the specific improvement actions planned.
• Confirm that these actions were indeed implemented. Document evidence of implementation (e.g., training attendance, new policies, updated forms). This forms the 'story' of your improvement journey.

Step 2: Plan the re-audit

• Agree timing: As discussed above, allow reasonable time for interventions to embed.
• Confirm methodology: Ensure the re-audit will precisely replicate the initial audit's methodology (criteria, population, data items, sample size).
• Secure resources: Allocate time for data collection and analysis. Engage your team and supervisor early.

Step 3: Collect re-audit data

• Systematically collect data using the same tools and methods as before.
• Maintain clear records, noting any deviations or challenges during collection.

Step 4: Analyse and compare data

• Follow the analysis steps outlined above: calculate adherence rates, quantify change, and identify trends.
• Present the 'before' and 'after' data clearly, often using tables or graphs.

Step 5: Interpret the results

• Did you meet the standard? If so, congratulations! Document the sustained improvement.
• Was there improvement, but not yet to standard? Acknowledge progress, identify remaining gaps, and plan further interventions.
• Was there no improvement, or even deterioration? This is a learning opportunity. Critically review the implemented interventions. Were they appropriate? Were they implemented effectively? Was the problem misidentified? Plan new interventions.

Step 6: Present and disseminate findings

Effective presentation is key to demonstrating value and ensuring lessons are learned. Consider:

• Summary report: A concise document outlining initial findings, interventions, re-audit results, and future plans.
• Visualisations: Use clear graphs (e.g., bar charts showing before/after percentages) to highlight improvement.
• Meetings: Present findings at team meetings, departmental governance meetings, or local audit days.
• Key messages: Distil the main points: what was the problem, what did you do, and what was the impact?
• Acknowledgements: Thank everyone involved in the audit and improvement efforts.
• Documentation: Ensure the full audit cycle, including the re-audit and improvement, is documented for governance and CQC.

Example in clinical practice: Reducing hospital-acquired pressure ulcers

Initial Audit (January 2023):

• Standard: 95% of patients with limited mobility should have a documented pressure ulcer risk assessment (e.g., Waterlow/Braden) on admission.
• Findings: Only 60% compliance (N=100 patients).
• Problem: Inconsistent nursing documentation, staff unaware of updated trust policy.

Interventions (February-April 2023):

• Mandatory online training on pressure ulcer prevention and documentation.
• New ward-specific 'aide-memoire' pocket cards distributed to nursing staff.
• Weekly ward huddles reinforcing documentation importance.
• Designated 'pressure ulcer champion' on each ward.

Re-audit (October 2023):

• Methodology: Replicated initial audit – 100 consecutive patients with limited mobility admitted in September-October 2023.
• Findings: 88% compliance with documented pressure ulcer risk assessment.

Demonstrating Improvement:

• Quantified Change: +28% improvement (from 60% to 88%) in documentation compliance.
• Visualisation: A simple bar chart showing 'Initial Audit 60%' vs 'Re-audit 88%'.
• Interpretation: Significant improvement, nearing the target standard. Further work needed for the remaining 7% gap, perhaps exploring barriers to compliance in the remaining cases (e.g., night shift, new staff).
• Dissemination: Presented at the ward meeting, departmental governance meeting, and submitted to the Trust's Clinical Audit Department. This demonstrates effective audit closure and continuous quality improvement.

How Lazomis can help

Lazomis provides a structured framework to manage your entire clinical audit cycle, from initial planning to demonstrating re-audit outcomes. Our tools facilitate:

• Project Setup: Clearly define your audit criteria and data points, ensuring consistency for re-audits.
• Data Collection: Standardised forms and templates streamline data capture, making comparisons easier.
• Analysis Tools: Quickly generate comparative reports showing 'before' and 'after' data, highlighting improvement trends.
• Reporting and Dissemination: Easily create clear, visual presentations of your re-audit findings for meetings, reports, and portfolio requirements.
• Progress Tracking: Monitor the implementation of interventions and schedule re-audits, helping you close the loop effectively.

This resource supports, but does not replace, clinical judgement. Local policy, formulary and specialist advice should be followed.

Key takeaways