Redefining Vaccination Coverage and Timeliness Measures Using Electronic Immunization Registry Data in Low- and Middle-income Countries

For many national immunisation programmes, there are challenges with accurately estimating vaccination coverage based on available data sources. However, an increasing number of low- and middle-income countries (LMICs) have begun implementing electronic immunisation registries (EIRs) to replace health facilities' paper-based tools and aggregate reporting systems. These systems, forming part of an immunisation information system (IIS), allow for more efficient capture and use of routinely reported individual-level data that can enhance programmatic decision-making at all levels of the health system. This commentary discusses how measures for assessing vaccination status and programme performance can be redefined and recalculated using these data when generated at the health facility level and examines the implications of the use and availability of electronic individual-level data.

As the authors explain, historically, common administrative data sources in LMICs have been paper-based immunisation registries, aggregate monthly summaries, and home-based records. These data sources often lack at least one of the following characteristics: timeliness, accuracy, and particularity. The impracticality of using paper-based records to calculate performance measures causes programmes to rely on unspecific aggregate measures that are often inaccurate due to recording errors and that suffer from poor completeness and timeliness of reporting.

Next, the authors define and describe commonly used measures amongst LMICs to provide background on how these measures can be improved using individual-level data from EIRs. The measures include: vaccination coverage, drop-out and lost to follow-up, and dose validity and timeliness. They summarise the strengths and weaknesses of EIR data compared to aggregate administrative routine health information systems (RHIS) and survey data. They also illustrate how improved definitions can be used by considering how measures would change amongst a small cohort of children. For example, measuring the number of children lost to follow-up, or those children not returning to any facility for vaccination, can be defined using multiple criteria stemming from individual-level data. Some EIRs may allow for reasons for children not receiving a vaccine to be captured. In the case that a caregiver refuses for their child to be vaccinated, that reason can be tracked alongside the number of children not returning for a scheduled vaccination so immunisation programmes can better target their activities. The definition of a child lost to follow-up can be honed and made relevant to a given population based on observed trends in service delivery and healthcare seeking behaviour for that area. With more accessible data, missed opportunities for vaccination can be assessed at sub-national levels; these occurrences are not often tracked due to the limitations of available data.

The authors note that, despite the potential benefits of using data generated by EIRs, challenges and limitations with use of the data persist. For instance, if a child has a documented third dose in a vaccination series recorded in their home-based record, should a nurse assume they received the first and second dose as well? If legacy data is not back-entered consistently for all children, vaccination coverage estimates would be inaccurate for all doses administered prior to the introduction of the EIR. In 2018, the American Immunization Registry Association (AIRA) published guidance on data quality practices for use of EIR data to tackle these potential limitations. As countries transition from paper to electronic data systems, they will have to decide when the EIR data is considered accurate enough to replace paper-based data and satisfy reporting requirements of the World Health Organization (WHO) and Gavi, the Vaccine Alliance.

Although EIR data has been most commonly used to evaluate vaccination coverage, the data could be used by programmes and activities beyond routine immunisation. For example, individual RHIS data can be used for programme evaluations and research studies using quasi-experimental designs. The concept is that fostering better understanding vaccination coverage using redefined definitions can improve countries' push towards healthcare equity and universal health coverage by helping to identify children left out of healthcare services.