The 2002 Data Quality Audit (DQA) for Madagascar was conducted between July 28 and August 11, 2003. Led by PricewaterhouseCoopers in partnership with the Ministry of Health and international agencies including WHO, UNICEF, and USAID, the audit aimed to assess the accuracy and reliability of immunization data used for GAVI funding decisions.
A total of four districts, Antananarivo Renivohitra, Arivonimamo, Ambositra, and Ambatondrazaka, were selected through random sampling. In each district, six Centres de Santé de Base (CSBs) were visited, resulting in 24 health centers audited nationwide. The process included data verification, system evaluation, and documentation reviews at all levels of the health infrastructure.
Audit Scope and Methodology
The audit focused on evaluating how immunization data, particularly DTP3 coverage for children under one year—was recorded, reported, and managed at national, district, and health center levels. The findings informed both the validation of Madagascar’s health data system and its eligibility for GAVI-related funding.
The approach followed WHO’s standardized DQA methodology. Four districts were randomly selected, and within each, six CSBs were audited. The evaluation included a thorough review of vaccine registers, stock records, cold chain documentation, and monthly activity reports. Data from health facilities were compared against district and national-level aggregations to assess consistency and identify points of data distortion. Technical interviews and supervision assessments also formed part of the methodology.
Key Statistics
The audit compiled a set of critical indicators to assess system performance across reporting accuracy, coverage, and operational consistency. These figures provide a quantitative snapshot of Madagascar’s immunization data landscape in 2002.
- Verification Factor: 58% (GAVI threshold: 80%)
- System Quality Index: 49% (national average)
- District-Level System Quality Index: 53.5%
- CSB-Level System Quality Index: 45.2%
- Reported DTP3 Coverage in 2002: 58.4%
- Reported DTP3 Coverage in 2001: 84.1%
- Drop in Vaccinated Children (DTP3): 117,355 fewer than the previous year
- Completeness of Reports (2002): 90.2% (down from 97.4% in 2001)
- Districts with DTP3 Coverage ≥ 80%: 10.8%
- Districts with Measles Coverage ≥ 90%: 6.3%
- Districts with Dropout Rate Below 10%: 8.2%
- Vaccine Loss Rates at CSBs: Ranged from 7.5% to 90.6%, based on available records
These statistics highlight both the scale of disruption caused by the political crisis and the systemic issues within data handling and reporting processes.
Common Challenges Identified
The audit highlighted a range of issues that compromised the integrity and usability of immunization data. These challenges were consistent across national, district, and CSB levels and point to structural and operational weaknesses in the health information system.
1. Documentation and Recordkeeping
Many health centers lacked consistent access to vaccination cards, stock registers, and tally sheets during the 2002 reporting period. In several sites, forms had been erased and reused, eliminating previous entries and leaving gaps in historical records.
This lack of reliable documentation made it difficult to verify reported figures or identify children who missed scheduled doses. The absence of a standardized recordkeeping system significantly reduced traceability and data confidence at all levels.
2. Inconsistent Reporting Formats
Monthly reports did not consistently separate doses by age or vaccination stage, leading to overcounting and data overlap. Some facilities reported combined totals for children under and over one year of age without clear labeling.
This created inconsistencies between local and national datasets, especially in calculating coverage for DTP3. Without clear guidance on how to complete forms, staff often made assumptions or copied previous formats that were not aligned with audit standards.
3. Population Denominator Issues
Districts applied different population estimates and growth rates, some using local Fokontany data while others followed outdated national census projections. This inconsistency led to wide variations in denominator figures used in coverage calculations.
As a result, some districts reported coverage rates exceeding 100%, which could not be verified. The absence of a unified standard for estimating target populations undermined data comparability and weakened decision-making based on these figures.
4. Limited Staff Training and Turnover
Personnel responsible for reporting were often unfamiliar with the proper classification of vaccine doses, including DTP1, DTP2, and DTP3. In many cases, they lacked the training to correctly document or interpret the required indicators.
Frequent staff turnover added to the problem, as new staff rarely received adequate handover or mentoring. This contributed to a loss of institutional knowledge and reinforced poor practices that had not been corrected or standardized.
5. Systemic IT and Supervision Gaps
Several districts experienced data loss due to hardware failures and a lack of regular backups. Some relied entirely on single machines without external storage, making the system highly vulnerable to crashes or file corruption.
Supervisory visits were sporadic and poorly documented, with no standard tools used to track performance or provide feedback. Without structured oversight, many data quality issues went unnoticed or unaddressed at the facility level.
Recommendations
To strengthen the immunization data system and address the core issues identified during the audit, several targeted actions were proposed. These recommendations aim to improve data accuracy, consistency, and overall reporting reliability across all levels:
- Revise reporting templates to include distinct fields for each vaccine dose and age group. This will eliminate confusion between DTP1, DTP2, and DTP3 entries and ensure children under one year are accurately accounted for in coverage statistics.
- Standardize population estimates by using a uniform growth rate and the same census baseline across all districts. This step will prevent conflicting coverage rates and improve comparability of data nationally.
- Implement formal training programs for CSB and district staff on data recording, stock tracking, dropout rate analysis, and proper use of immunization tools like the 3A manual. Training should be mandatory and refreshed regularly.
- Strengthen supervisory systems by conducting scheduled field visits focused on coaching and feedback, rather than only error correction. Documentation of visits should be required at all sites for follow-up purposes.
- Introduce consistent data backup procedures and enforce digital safeguards such as updated antivirus software. All facilities using computers should also maintain printed records and off-site backups to reduce the risk of total data loss.
These recommendations were validated by stakeholders during the audit’s closing meeting and are expected to guide improvements ahead of the next proposed DQA in 2005.
Next Steps
Following the 2002 audit, the evaluation team advised delaying the next Data Quality Audit until 2005 to allow sufficient time for implementing corrective measures. Given the extent of system-level adjustments required, particularly in reporting formats, staff training, and population estimation, a full year of changes would be necessary to yield measurable improvements.
Key stakeholders, including the Ministry of Health, WHO, and UNICEF, agreed to incorporate the audit recommendations into national planning. Updates to the Monthly Activity Report, training for district and CSB staff, and improved supervision structures were prioritized for early rollout. A stronger emphasis was also placed on monitoring vaccine loss rates, documenting adverse events, and aligning population denominators at all levels.
By targeting these reforms ahead of the next audit, Madagascar aims to meet the 80% verification threshold set by GAVI and improve the quality of immunization data used for funding decisions and national program performance tracking.
Conclusion
The 2002 Data Quality Audit provided a detailed assessment of Madagascar’s immunization reporting system, revealing both technical gaps and operational constraints. Despite challenges such as vaccine shortages, inconsistent data entry, and outdated population figures, the audit confirmed a strong willingness among national and local health teams to improve performance.
With a verification factor of 58% and a system quality index of 49%, the audit underscored the urgent need for system-wide improvements. Key recommendations—including standardized reporting formats, unified denominator usage, targeted training, and enhanced supervision—are already being incorporated into planning efforts.
The next audit, scheduled for 2005, will serve as a benchmark to assess the progress made in response to these findings. Strengthening the accuracy and reliability of immunization data will remain essential to ensuring equitable vaccine access, effective funding allocation, and sustained public health outcomes in Madagascar.