A shared vision for improving vaccine uptake through data-driven decision-making
The Vaccine Data CoLab, together with partners and stakeholders across Nigeria and Uganda, has learnt much about the Better Future that we collectively hope to see for the vaccine data health ecosystem, and what we need to do in order to get there.
Introduction
This page consolidates some of the evidence and insights that we have gathered and generated during the Vaccine Data CoLab, with the first phase running for just over a year (Sept 2022 - Oct 2023). In a short span of time, we have been able to generate rich insights from: our initial landscape mapping, implementation of our portfolio of grants, and our perspective on opportunities to strengthen the vaccine data system going forward.
We collected the insights through initial landscape reviews, interviews, and in-person and virtual workshops, from implementing interventions to improve vaccine data systems, and from subsequent reflections on learning.
Underpinning our learning is a shared vision for how data can enable better decision-making to improve vaccine uptake in each country. We will dive deeper into our systems lens for learning, and will share the systems frameworks that helped us frame the insights.
We expect that our learning will be useful for stakeholders across the vaccine data ecosystem:
for funders to align upcoming projects with country priorities
for government decision and policymakers to understand more about unmet needs and opportunities domestically
to increase the visibility that data experts have of the breadth of datasets and tools currently used
to clarify the upstream use cases of data for vaccine and healthcare professionals.
Better Futures CoLab uses its own methodology and approach, and is guided by high-level principles that shape the ways in which we work. We work collectively towards long-term goals, we’re not afraid to fail and evolve, we have a bias for the applied end of evidence, we speed things up by cutting through red tape and working on issues concurrently instead of sequentially, and we care deeply about raising unheard voices to imagine new futures.
How the CoLab learnt
In a CoLab, we:
Start from a desired future and work backwards from there: rather than starting from a problem and moving incrementally forward, we all align around a shared vision.
Take a systems view: understand the barriers and enablers at a systemic level, and what array of changes could make this future possible. We think about the policy environment, how resources are allocated, and common perceptions and practices, as well as people’s needs, incentives and behaviours. You will find insights from our initial mapping below.
Collaborate with others: by creating a movement of government, experts, innovators and funders alongside communities to imagine and work towards a radically better future, we spread the responsibility and expand what’s possible to do.
Raise unheard voices: involve the people who will feel the impact of the work, and continually experiment with different ways of putting more power into their hands.
Experiment and do: we see action and experimentation as a way to learn throughout our involvement, and create portfolios of complementary initiatives that are more than the sum of their parts.
This page synthesises learning from all stages of implementation. In the table below you can see a high level overview of the number of stakeholders that we engaged as part of our work in Nigeria and Uganda. These include government and non-government stakeholders, international organisations, as well as relevant volunteer organisations involved in the wider vaccine data ecosystem (e.g. religious groups and community leaders).
-
Nigeria
Interviews with individuals from 11 organisations
Uganda
Interviews with individuals from 9 organisations
-
Nigeria
Workshop with virtual and in-person attendees from 11 organisations.
Uganda
Workshop with in-person attendees from 13 organisations.
-
Nigeria
Observations from meetings, workshops and trainings; surveys of people attending workshops and trainings; policy reviews; interviews with stakeholders.
Uganda
Observations from meetings, workshops and trainings; surveys of people attending workshops and trainings; policy reviews; interviews with stakeholders.
-
Nigeria
Workshop with virtual and in-person attendees from 10 organisations.
Uganda
Workshop with in-person attendees from 8 organisations.
Workshops in Nigeria and Uganda, 2023
Shaping a desired Better Future
We stand at a pivotal moment in global health. The COVID-19 pandemic exposed the fragility of our global systems and the deep-seated inequities that exist in healthcare access worldwide. Despite significant strides in vaccine development and distribution, the pandemic also brought to light the importance of data and actionable insights to address challenges such as vaccine hesitancy, disinformation, and systemic barriers to access.
As we move forward, we have a unique opportunity not only to catch up on progress lost in routine immunisation but also to accelerate progress and make transformative strides towards a more equitable future.
The Vaccine Data CoLab exists to strengthen data systems and co-create change with local actors to enable data based decision making that improves immunisation programming. In our work, we have explored different aspects of the vaccine data system in Uganda, Nigeria and Indonesia, as well as working at a global level.
One of the first things we did together with partners was to craft Better Future statements for both Nigeria and Uganda. We think it’s important to focus not only on where the respective vaccine data ecosystems in each country have come from or where they are today, but also where they have ambitions to go.
Nigeria’s Better Future vision:
In 2030, the vaccine data health system in Nigeria is a one-stop shop for data and insights that can be relied upon to be used for effective planning and decision-making.
Uganda’s Better Future vision:
In 2040, the vaccine data health system in Uganda is known for having actionable, high quality and targeted data at all levels of the system that is widely available and accessible for decision makers.
Read more on our blog
+
Read more on our blog +
Taking a systems view: the vaccine data system in Nigeria and Uganda
This page consolidates some of the evidence and insights that we gathered and generated during the Vaccine Data CoLab between November 2022 and February 2023. We collected the insights through initial literature reviews and interviews, and in-person and virtual workshops.
The 7 Strategic Pathways Framework
Throughout our discovery, we have framed our thinking and learning through a systems lens created by the WHO called the 7 Strategic Pathways Framework.
This has enabled us to constantly remember that vaccine data exists within an interconnected system of pathways that, in combination and when ideally functioning, ensure that data can be and is used effectively. Our systems lens moves us beyond seeing data as a “shiny tool” and instead as 7 interconnected pathways that work together to ensure data is used for decision making.
You can explore the 7 Strategic Pathways Framework below.
Enabling systems change
WHO 7 Strategic Pathways to support implementation and sustainability at country level
Partnership & Collaboration
How are we building a culture of inclusion, partnerships and strategic alliances to build health data systems?
Advocacy & Communication
How are we raising awareness and engagement to ensure participation and contribution from all stakeholders at all levels?
Capacity
What are the gaps in knowledge, skills and competencies that we need to strengthen?
Interoperability & Data Management
What is the data supply chain and the process for establishing, maintaining and updating data?
Financial Sustainability
How are interventions being funded and supported?
Innovation & Technology
How are new technologies tested, documented and utilised?
Governance & Policies
What is the government stance, political will and policy framework?
Early insights into the vaccine data system Nigeria and Uganda
Our initial landscaping showed that we need hyperlocal data systems that empower decision-makers to deliver targeted vaccine campaigns.
-
In both Nigeria and Uganda, government actors have the formal mandate to oversee and manage the delivery of primary healthcare, including vaccine services. These Government actors tend to have established long-standing relationships and partnerships with data actors and often facilitate the introduction and adoption of intervention programs which the government mostly controls.
In Nigeria, The Nigeria Primary Health Care Development Agency (NPHCDA), with its formal mandate across primary health care including immunisation services, is at the centre of the vaccine data system.
In Uganda, although Government lead in improving health data systems, implementing partners and collaborators ensure optimal coverage and reach.
-
In Nigeria, delays in conducting the national census have hampered the provision of accurate population denominators and data for healthcare. For example, estimates of population figures are based on assumptions around fertility, mortality, and migration figures. This has been challenging from an accuracy perspective, with these estimates and assumptions informing the design of vaccination interventions.
In Uganda, government agencies including the Uganda Bureau of Statistics (UBOS) and the National Identification and Registration Authority (NIRA) are central to efforts to improve demographic data through efforts such as the national population registry. In both contexts, it’s important to work with and build on these types of wider national datasets whilst also recognising their shortfalls, limitations, and gaps.
-
International partners have been an impactful way to supplement data blindspots where in-country capacity is either lacking in general or lacking access to opportunities. For instance in Nigeria, where hand-drawn maps had been used to collect data on eligible vaccine recipients at the settlement, community, and household levels, specialist geo-spatial data partners have provided printed maps of health facilities, and improved micro-planning by removing bottlenecks and issues associated with the use of hand-drawn maps.
In Uganda implementing partners and collaborators have helped Government actors ensure optimal coverage and reach. For example, by supplementing the government's efforts and extending its reach to less connected communities that are further from key service points through access to vehicles, public address systems, and more.
Partnerships and Collaboration
-
The design and implementation of shared data systems requires political, financial, and logistical support in particular during the pre-implementation phase, including advocacy visits to state-level authorities. In both Nigeria and Uganda, the implementation of the Routine Immunisation Module, which is part of the web-based software District Health Information System V.2 (DHIS2), required lobbying of government officials - particularly state executive secretaries - to ensure that they received funding and resources in state budgets.
-
There is no consensus on what data is needed or which formats will allow interoperability, with different stakeholders having different approaches. The demand for vaccine data comes from private, international, and donor organisations such as WHO and UNICEF, alongside public health authorities, and each has its own priorities and requirements. While some are advocating for the need to better identify and understand sociocultural norms within communities regarding vaccine hesitancy, others are working to better capture demographic distribution data for improved geolocation of populations eligible for vaccination.
In both Nigeria and Uganda, and indeed beyond, there is a need for a level of alignment, standardisation, and interoperability to allow data sets to speak to each other.
-
Where data exists, the outputs from data analysis are often not translating into useful insights for effective decision-making. This is driven by multiple factors, including: a lack of data coordination which limits valuable comparative analysis; low funding across all levels in the data value chain, including for data visualization; and a lack of a standardized or harmonized approach to data outputs. Inconsistencies in how data is shared is challenging for time-poor healthcare officials and frontline workers, where too much data without insight becomes confusing instead of actionable.
In Uganda, we learnt that the volume of data and numbers can be overwhelming at the local level, and that strict rules around how to communicate and write data can be challenging for those tasked with doing so in more rural or local contexts.
-
At the community level, local influential figureheads such as religious leaders at different places of worship and political leaders, have supported the dissemination of vaccination information to sometimes harder-to-reach individuals and communities.
In both Uganda and Nigeria, various types of community hubs, coupled with more targeted outreach such as door-to-door campaigns by local leaders, were important parts of broader efforts to engage and inform communities in national vaccination messaging and services.
Advocacy and Communication
Workshop in Nigeria, 2023
-
Across all levels of government, training is often provided by third-party organisations who organise workshops for staff. Most state and national agencies can only expand capacity through external support. International partners have helped to support skills- and knowledge-related capacity gaps through training and workshops to support various parts of the health data ecosystem. For example, data quality is often negatively affected by technical or human error when data is copied from paper-based forms into digital platforms.
In Nigeria, The National Stop Transmission of Polio (NSTOP) program, delivered under the African Field Epidemiology Network (AFENET), organises supervisory visits and delivers refresher training which has been improving data quality. However, global partners are spotting and responding to gaps in silos across organisations and across different health projects, and this lack of coordination can result in the replication of training and/or delivering unsuited training to various groups. There are opportunities for a more collaborative and coordinated approach.
-
Stakeholders have suggested that healthcare personnel are not motivated to develop or learn new skills and expertise in data collection methods and analysis. In addition, for time-poor frontline workers whose main focus is not explicitly data-related, ‘motivation’ or ‘incentive’ for upskilling compete with the realities of limited time and bandwidth for what can feel like additional work. In Uganda, we learnt that many workers lack the capacity and bandwidth to attend training and upskilling about using new systems.
Participation in training is being incentivized in Nigeria, for example, through the use of daily subsistence allowances and per diems for workshops and training, as well as an Estacode Supplementation Allowance which allows staff to expense overseas travel costs in some instances. In general, better articulating the usefulness of data to different people’s jobs, and in so doing changing people’s perspectives and behaviour, is an important aspect of the wider upskilling enabling environment.
-
There is a real need for sustained investment in personnel and a growing workforce, as well as an ongoing need to augment and expand delivery capacity by continuing to work with volunteers in some cases. There are too few personnel to support the design, collection, and monitoring of data.
For example, In Nigeria, the Department of Advocacy and Communication at the NPHCDA does not have enough Monitoring and Evaluation staff, and the only available Monitoring and Evaluation officer is overwhelmed with the level of sole responsibility and workload. The department has in the past outsourced aspects of this role to try and alleviate the situation, for instance, it has recruited volunteers from the Nigerian Red Cross Society to support with data collection.
In Uganda, a general lack of human resources staff and skills lead to operational and organisational challenges around the appropriate collection of data.
In general, there needs to be a growing focus on developing people as well as tools across the vaccine data ecosystem.
Capacity
A data system that is hard to access or deemed non-robust can incentivise people to invest effort in collecting ever more primary data as a workaround, rather than contributing to and expanding an existing data system.
Interoperability and Data Management
-
Standardised tools and platforms help create the template for consistency and interoperability but are only as useful as the quality of data they hold. In both Uganda and Nigeria, the DHIS2 is used as a public health information management system, tracking aggregate data by health facility on key public health indicators, such as routine immunisation. Since its implementation, DHIS2 has received strong state-level support.
For example in Nigeria, out of its 36 states, 22 upload monthly data on Routine Immunisation, 33 conduct supportive supervision and on-the-job training on a monthly basis, and 7 consistently provide funds to support DHIS2. However, there are no data transfer mechanisms between government, privately owned, or non-government dashboards, and vaccine data is transferred by submission of paper-based forms despite the associated risks.
-
A Routine Immunisation module was developed on the DHIS2 platform to provide government officials with high-quality data for monitoring performance and improving vaccination coverage. This data covers administrative, operational, and cold chain aspects of immunisation service provision, and is intended to correct and improve Routine Immunisation performance.
In Nigeria, data is gathered at the health facility level on paper-based collection forms and submitted to the Local Government Authority on a monthly basis. These types of shared platforms and processes help, but there is more to be done. For example in Uganda, whilst the introduction of Smart Paper Technology as part of this data collection ecosystem has been valuable, there are also shortfalls in terms of the provision of paper, scanners, proper storage of paper, and so on.
-
Vaccination data are collected from either primary or secondary data sources. Whilst secondary data is theoretically the most convenient form of data to collect and apply, issues regarding the robustness of vaccination metadata result in different stakeholders collecting their own primary data to deliver their programmes.
This means that some organisations across the ecosystem are deploying their own data collectors at local levels, and creating their own platforms for them to disseminate the data that is collected. This may suggest that a data system that is hard to access or deemed non-robust can incentivise people to invest effort in collecting ever more primary data as a workaround, rather than contributing to and expanding an existing data system.
-
Stakeholders who collect data for vaccine deployment highlighted that questionnaires used in the field for community polling are in general not standardised, resulting in misalignment in terms of data captured. In addition, the quality of data collected from primary data sources in communities has varied in terms of accuracy and therefore reliability according to the device that was used to collect the data.
In Uganda, the overall quality of data - and therefore its utility and reliability - is affected by non-existent information; in some cases, simple things like a missing date-of-birth or missing contact information can cause bottlenecks in the wider ecosystem.
-
There is no generally recognized and implemented national framework to guide health data standardisation and interoperability in Nigeria, with different organisations in the country generating data in line with their own set of internal criteria. This results in the production of non-interoperable data and data formats with varying standards and metrics.
For comparison, the comprehensive privacy regulation General Data Protection Regulation (GDPR) in the European Union is designed to protect personal data and regulate data is processed and stored. No similar multi-national standard around data exists for Africa, however, several countries have enacted their own data laws, such as the Protection of Personal Information Act (POPIA) in South Africa. That said, the African Union (AU) has adopted the African Union Convention on Cyber Security and Personal Data Protection, but the convention has not yet been ratified by enough member states to come into effect.
There are no data transfer mechanisms between government, privately owned, or non-government dashboards, and vaccine data is transferred by submission of paper-based forms despite the associated risks.
Financial Sustainability
-
Whilst vaccination services are primarily provided at government-owned healthcare centres, which make up 67% of health facilities in Nigeria, the remaining 33% includes privately-owned healthcare centres which provide vaccination services at a cost. Data about healthcare financing, which captures the ability of individuals to pay for healthcare services, suggests that personal finances influence vaccine hesitancy by affecting ease of access and convenience, and as a result, complacency or hesitance.
-
The lack of funding and other resources to support effective and efficient health data services is a major drawback for health data management. In Nigeria, there are multiple challenges, including inadequate annual budgetary allocations at the national and state government levels often results in plans being rolled over from year to year and the reprioritization of funds at the national or state government level leads to delays or failures in procurement.
In both Uganda and Nigeria, irregularity and delays with payments, and the lack of a budget tracking system, means that there are delays and that funds often go unspent. The lack of an effective Monitoring and Evaluation system, and therefore missing impact and outcome indicators, problematises efforts to remedy challenges in this area.
-
Where national funding is limited, international and donor organisations have been active participants in funding healthcare initiatives. For example In Nigeria, UNICEF supports community polling conducted by the Nigeria Primary Health Care Development Agency (NPHCDA), the Centers for Disease Control and Prevention (CDC) funds other initiatives at the national level, and the Bill & Melinda Gates Foundation and Dangote are active contributors to funding the data bank in Kaduna State.
While this international and donor funding is critical, there is sometimes overlap. Funders and donor communities coming together to pool funding and coordinate resources can help to address this issue of replication, duplication, or fragmentation of what is being supported.
Funders and donor communities coming together to pool funding and coordinate resources can help to address this issue of replication, duplication, or fragmentation of what is being supported.
Innovation and Technology
-
The DHIS2 is a global public good transforming health information management around the world. For Uganda, a landlocked nation that depends on trade with its neighbours, the DHIS2 Tracker and DHIS2 Andriod Capture App were used for Port of Entry screening to track incoming drivers during the COVID-19 pandemic. QR-code-enabled traveller passes were issued for enhanced truck driver tracking and follow-up, supporting their safe clearance for travel across borders and at different checkpoints within the country. In Nigeria, DHIS2 has been used to improve immunization logistics by implementing a unified system for quality data collection and analysis. The unified system has improved the transmission of vaccine stock data and cold-chain equipment status, enhancing last-mile visibility and facilitating timely decision-making for better outcomes.
-
During the COVID-19 pandemic in Nigeria, the government conducted monthly telephone polling focused on gauging public perception, behaviours, and attitudes on a range of issues, including COVID-19 vaccine acceptability. A key component of these efforts was the routine collection of multiple rounds of population-level polling data, alternating monthly between national and or state levels, with sample sizes ranging from 500 to 5000. The NPHCDA have reflected that this method of data collection provided valuable insights relating to behavioural change, based on which actions and responses around vaccine provision and information were made more possible.
-
In Nigeria, social listening in the context of the COVID-19 vaccine and vaccination process involved actively monitoring relevant conversations across social networks. This involved the tracking of social media platforms using AI to search for keywords, and then analysing this data for opportunities to intervene by creating targetted responses to ongoing conversations. The intention was to improve the public’s confidence and trust in the COVID-19 vaccines and vaccination processes.There are perhaps lessons to be learnt here about how social media rumour management and interventions regarding public mood and perception are to be considered important parts of the wider response ecosystem alongside other factors like managing logistics and tracking hard data about vaccine provision.
Governance and Policies
-
In Nigeria, hyperlocal data is accessible and disseminated via both open-source and closed-source platforms, depending on who owns or wants to access that data.
In Uganda, because of a lack of clarity and policy on what can and can't be shared with whom, the tendency is to not share data and restrict access. Policies regarding access and usage, therefore, vary across the ecosystem, and gatekeeping of data among stakeholders results in duplicated efforts. Some platforms provided by international or donor organisations, such as GRID3’s dashboard which provides grid-level coordinates of health facility data, are open-source and publicly available for use.
Semi-open platforms, like eHealth Africa’s data portal, require sending requests for data before access is granted or not depending on a range of factors. And government ministries disseminate data through a closed-source system, whereby data can only be accessed by other government entities or upon request and approval from relevant authorities.
In the round, a complicated picture of data governance and access emerges which can lead to various forms of systems inefficiency. Clarity around rules and procedures for data sharing, as well as clarity on who should be involved in setting those rules and procedures, would lead to efficiencies in the data ecosystem.
-
The Nigeria Primary Health Care Development Agency (NPHCDA) is responsible for managing, leading, overseeing, and administering operations as it relates to healthcare data in Nigeria. A primary objective for the agency is to develop guidelines for State Primary Health Care Boards, which then implement health initiatives and validate data inputted by health personnel at local health facilities. Other program data are collected by routine immunisation officers, immunisation program officers, state immunisation officers, cold chain officers, and monitoring and evaluation officers at the local level. The demands of primary health care are met at the ward and local government levels by the Local Government Health Authorities (LGHAs). However, the absence of coordination and policies guiding the delineation of roles for stakeholders across the ecosystem can lead to conflict in inter-agency interventions and roles.
Clarity around rules and procedures for data sharing, as well as clarity on who should be involved in setting those rules and procedures, would lead to efficiencies in the data ecosystem.
Experiment and learn through a portfolio of interventions
Having mapped the system together with people from across the vaccine data system in Nigeria and Uganda, we were then ready to identify opportunities for interventions to strengthen the system, or learn more about what to improve.
We created a portfolio of grants in each country (read more on Nigeria here and Uganda here). This meant we were able to gather learning from a portfolio of interventions in data policy, data-for-decision-making tools and capacity development.
Read on to explore our portfolio, and see how they connect to learning needs in the system: what we and others need to better understand in order to take action.
Nigeria: Learning Areas
-
The CoLab found that whilst correct, consistent, and comparable data exists, it isn’t always effectively coordinated nor widely accessible between organisations. We learnt that there is often a lack of leadership or common policy around data sharing and a lack of structures and guidelines to enable vaccine data transfer between different types of users.
Sydani Group, with our support, developed an inclusive health data governance framework for immunisation in Nigeria, rooted at the national level in Abuja. They aimed to strengthen data governance within the National Primary Health Care Development Agency (NPHCDA) by putting shared standards in place for data quality and governance. This initiative aligned with the Nigeria Strategy for Immunization and PHC System Strengthening version 2.0 (NSIPSS 2.0) and Nigeria's digitization roadmap for immunisation data.
Sydani Group engaged a mix of government and non-government stakeholders, including FMOH NPHCDA, NBS, NDPC, NITDA, Data.Fi, AFENET, HISP, and UNICEF. Through qualitative interviews, reviews of data policy documents, and stakeholder workshops, they collaboratively crafted the data governance framework. This process gave us a clearer understanding of data governance.
-
We identified that even where hyperlocal data exists, key individuals and organisations across the vaccine data ecosystem need further skills- and knowledge-related support to better understand, utilize, and act on that data. Due to these competency gaps, data often goes underused. Relatedly, a lack of alignment and clarity around what problems need to be solved results in analysts and scientists being unsure how best to target their efforts in ways that are action-oriented and relevant to decision-makers.
Brooks Insights focused on enhancing the use of data for better immunisation programs. They used adult learning methods to help Program Managers (PMs) and Health Workers (HWs) at state and local levels understand and apply Human-Centred Design (HCD) for tailored immunisation strategies. This aligned with the NSIPSS 2.0 plan, which emphasises reaching communities with many unimmunised children. Working alongside the FCT Training Working Group, they carried out their project in three councils in Federal Capital Territory (FCT): AMAC, Bwari, and Gwagwalada, involving a variety of stakeholders including state and local PMs, HWs, and community members. Their strategy involved active participation from these stakeholders, reviewing existing materials, developing new training curriculum, and conducting hands-on HCD sessions with the community.
As a result, they identified areas where knowledge and skills were lacking, trained PMs to guide HWs in using HCD, and saw increased confidence among HWs in using the HCD approach for immunisation. The project was well-received, with strong support from the NPHCDA ACSM (Advocacy, Communication and Social Mobilization) unit and active collaboration with the FCT technical working group (TWG) in designing and overseeing the project. Importantly, the skills acquired were also applied by HWs to other health programs.
“Actually, after the training, I was able to know how to enter the community to know their problem [diagnose], after knowing their problems, give them listening ears … and to put into design and implementation. In our primary healthcare facilities, there is an improvement because of the knowledge we acquired in July in the human centred design program." - Health worker
-
In hearing from a range of data users in Nigeria, the CoLab discovered that there is a lack of standardisation in terms of how data is collected and stored. This lack of data interoperability results in the siloing and subsequent underutilisation of data. The absence of common tools, methodologies, and formats, leads to inefficiencies including resources and capacity being lost to the duplication of efforts.
We supported Sydani Group to develop a new framework for data governance, which brings different actors involved in data together and clarifies responsibilities. They reviewed current policies and global good practices to develop a framework which they had started testing with different government stakeholders. This helped us learn about opportunities to enhance quality and data protection.
Chisom Obi-Jess, Principal/CEO at Brooks Insights
Yunusa Medugu, Chief Data Processing Officer at NPHCDA
Uganda: Learning Areas
-
The CoLab found that even where immunisation data has been collected, tensions between protecting privacy and enabling access lead to limited data sharing which in turn impacts decision making and service delivery. Despite this challenge, we heard that there is an opportunity to explore policy options to strike an effective balance. We also heard that there are unexplored partnerships, including cross-sector ones, which have the potential to unlock data for decision-making.
We supported HISP to develop a consolidated vaccine data visualisation dashboard at national level. They wanted to create an interactive and customisable dashboard that unifies vaccine data from various sources into one platform, and visualises them. The dashboard is now online, after user testing, and with supporting documents on how to use it available. It hasn’t yet formally launched.
This helped us learn about fragmentation in data platforms, interoperability and ownership consideration, as well as the needs of different users.
"Accessing vaccine data from a singular platform revolutionizes the EPI landscape." Besigye Albert, UNEPI.
-
Hearing from health and vaccine data users in Uganda, the CoLab discovered that there are often quality issues with data at the first stages of the data value chain (data collection). However, we also heard that sufficient HR and training on data collection would improve data reliability. Additionally, while technologies such as Smart Paper Technology exists and have high potential, there is an opportunity to innovate further and improve how existing technologies are utilised to improve data quality.
We supported CUAMM to pilot how a combination of training, Smart Paper technology, mentorships, and facilitated conversations about quality, can improve capability. CUAMM trained district-based mentors who supported 243 frontline health workers, with content based on data quality assessments. They raised awareness about vaccine data analysis tools and helped facilities close gaps in data completeness, timeliness, and accuracy. They introduced Smart Paper Technology to 17 health facilities in Moroto. Their work revealed the unreliability of vaccine data in unsupported settings, and the health workers they supported are better at detecting and correcting data inconsistencies.
This helped us learn about the day to day challenges of making data strengthening innovations work at local level, barriers to scaling solutions, and the importance of collaboration amongst multiple stakeholders to scale interventions.
‘‘In my 22 years of working in Immunization, no one has ever mentored me on how to handle the issue of vaccine data. I have just been figuring out things by myself or asking colleagues’’ Health worker in Lokales HC II in Amudat
-
The CoLab found that there is often low motivation among healthcare workers who collect and analyse vaccine and health data. We also heard that in many cases workers do not perceive data and data collection to be as important as other parts of health service delivery. The stakeholders we spoke with suggested that there are opportunities to improve the availability of quality data if levels of motivation improved and incentives were introduced.
We supported ABBRS to develop a policy brief to highlight gaps and bottlenecks that hinder efficient data sharing among stakeholders at national, subnational and institutional levels in immunisation programming. They did a document review and key informant interviews to identify gaps and inconsistencies in policies. This helped us learn about strengths and weaknesses in the policy framework for data sharing.
“People go about data sharing in ways that end up harming the person from whom data was collected, and yet there are responsible ways of generating and sharing it.” Policy Developer.
Stakeholders’ meeting to share the accomplishments of the Vaccine Data-CoLab (VDCL), October 2023. The outcomes of this dynamic engagement will have a significant impact on implementation of the National Immunizarion strategy and shaping future decision-making processes, ultimately resulting in improved vaccine uptake.
Gathering insights and spotting opportunities for the future
Three key insights from our portfolio across Nigeria and Uganda
-
We've learned that the details truly matter when it comes to making data accessible, and more crucially, actionable.
Single-aspect interventions to enhance data collection and accessibility at the local level may not work, especially if they are focused on bringing new tools and practices to health facilities. Grasping the nuances of the 'last mile' in digitising vaccine data is paramount. We've realised that this isn't just about having computers or digital systems in place. It's about comprehensive evaluations of the basic essentials: Is there consistent electricity? How computer literate are the staff? Do they have requisite tools, like scanners? And are these tools safeguarded adequately against potential damages or misuse?
Delving into the practicalities, when CUAMM introduced Smart Paper in Karamoja to digitise vaccine records, CUAMM knew there would be additional related needs. Beyond the paper, they required additional stationary, scanners, lockable cupboards for security, and even dust sheets to protect records. We anticipated some, but not all of these costs. Because CUAMM’s model combined the introduction of tools with mentoring and had strong feedback loops during their implementation, they could spot and address other gaps preventing improvement- for example, time poor health workers were used to writing fast on forms then typing them up later, but they needed help to adapt to new habits, writing carefully on a form that will be directly scanned into digital records.
Yet, having data and the tools to collect it is just one facet of the challenge. Tech and data literacy remains low, as we learned through Brooks Insights’ training needs assessment and our simulation exercises. We found that not everyone can readily interpret data or its visual representations. Time is a luxury that overstretched staff often can't afford, making them hesitant to invest in training. But a pivot in our approach showed promise: by training local officers who, in turn, trained their peers, we tapped into a sense of empowerment. This peer-driven model not only made it more probable for local officers to allocate time for training but also fostered deeper engagement, ensuring the lessons resonated and were retained.
-
A closer look at the vaccine data system and efforts to strengthen it revealed a mosaic of local challenges that must shape national frameworks and national infrastructural and capacity issues stymieing grassroots scalability.
To begin with, consider the local health worker's perspective, often the frontline guardian of public health. In places like Karamoja, the nomadic nature of communities makes health workers question the utility of meticulous data collection. If they perceive that they and their patients benefit little from collecting certain data fields, because of their high mobility, they might omit them, leading to substantial gaps in national databases. Recognising these unique regional dynamics and health worker perspectives is thus pivotal for any national policy aiming to be effective.
Then there is the pressing issue of digital infrastructure and digital literacy. The push for digitisation strains local resources — bandwidth, server space, and maintenance costs. Also, if a rural clinic's system can't communicate with national databases due to compatibility issues, or if the digital leap feels too abrupt for local health workers, the ambition of a connected digital health system stumbles.
We also found that local digitisation may well outstrip national resources too- the government of Uganda at a national level didn’t have the server space to accommodate the data from the relatively small Smart Paper pilot. A coordinated effort between national ICT authorities and health departments, right from a project's inception, could preempt many of these challenges.
Lastly, while national data policies are crafted with the best intentions, the long policy cycle makes alignment with the realities of data governance at the local level harder. For national systems to be scaled effectively, they must resonate with local immunisation practices, infrastructural constraints, and the myriad challenges health workers face daily. For example, a national standard for clinics to offer vaccination 24/7 isn’t realistic nor efficient for health facilities.
The promise of robust public health hinges on harmonising the symphony between the local and national, the micro and macro. When building advanced digital networks, we also need to ground them in the experiences of those at the very heart of public health.
-
Data-driven decision-making is an essential tool for modern government functions. Ensuring its efficacy demands collaboration between multiple departments and stakeholders. Addressing the challenges inherent in this process can significantly streamline governance.
Governments and international partners the world over are grappling with issues of fragmentation and poor alignment, despite often having the best intentions. Siloisation happens both within government, and amongst international partners, and both face strong incentives that go against greater alignment.
The governments of Uganda and Nigeria both make significant efforts to coordinate international assistance and encourage collaboration between partners. This did help us: in Uganda, the government was able to spot the connection between the work of HISP and CUAMM to enable joining forces. However, they took on the burden of joining the dots on lessons learned from multiple internationally funded data digitisation efforts across multiple districts, and are not getting the data/learning they need in order to decide what to scale and not to scale.
Everyone is looking for solutions that work, and as humans we often are biased towards ‘innovative tools’. Those of us working in Nigeria and Uganda are no exception. With a plethora of tools and platforms emerging across different regions and departments, a consistent strategy is vital for their effective deployment at scale. While coordination is widely acknowledged in government circles, we also saw another gap: frameworks guiding the scaling of interventions. A cohesive learning framework can offer valuable insights, collating evidence from diverse interventions and guiding investments towards scalable solutions. Implementing such frameworks not only enhances scalability but also bolsters interdisciplinary collaboration.
Another challenge we saw is the fragmentation across disciplines. Domains such as "data for development", "evidence-informed policy making", and "health systems strengthening" often operate in isolation. Similarly, within the government, crucial sectors like digitisation, health data systems, and vaccine data systems can sometimes function independently. This separation can delay crucial decisions and prevent a holistic approach to challenges.
The sharing and credibility of data are paramount concerns. The parameters defining 'credible' data are often ambiguous, resulting in potential hesitancy in data dissemination. Many departments and affiliated researchers are either unsure of data-sharing protocols or lack the requisite platforms. Discrepancies in departmental data privacy norms further compound the interoperability challenges.
It's vital to understand that raw data isn't always the complete solution, especially in nuanced sectors like vaccination. Initiatives, such as those undertaken by the Behavioural Insights Team in Indonesia, highlight the differential community responses to data-driven programmes. Emphasising human-centred design and encouraging participatory decision-making can assist communities in leveraging data effectively.
Open dialogue across all government departments and with stakeholders is essential. Promoting a culture of collaboration over instruction ensures alignment in objectives and facilitates the seamless flow of information. This is especially pertinent when tackling data access issues where privacy concerns might inhibit data sharing. Collaboratively developed guidelines can ensure that data remains accessible to all relevant entities while maintaining security.
For governments aiming to harness the power of data-driven decision-making, the path is clear: collaboration.
The Vaccine Data Health Systems Map
Our key insights from the Vaccine Data CoLab led us to develop the Vaccine Data Health Systems map. This highlights the interconnections between local, national and global factors in strengthening vaccine data systems. It brings together a behavioural, political and spatial lens on WHO’s 7 technical pathways for data systems strengthening.
When we started out in Nigeria and Uganda, we framed our thinking and learning through a systems lens created by the WHO called the 7 Strategic Pathways Framework (see above.) We created the Vax Data Health Systems Map because we couldn’t ‘fit’ what we have been learning about vaccine data system strengthening onto the WHO 7 Pathways without adding another set of perspectives.
We built on the 7 pathways already identified by the WHO, which speak to the technical aspects of systems change, adding:
A behavioural lens: thinking about what influences people’s behaviour, including norms and the choices they have available to them
A political lens: thinking about how power and politics influence the system, including individual and institutional incentives
A spatial lens: how the behavioural and political lenses, alongside the ‘technical’ lens offered by the 7 pathways, play out at local, national and global levels, and how they relate to each other
These lenses helped us draw out the connections between different efforts at local and national level, and unpack why we were seeing certain patterns.
Mindset & capacity: leadership, skills incentives
Policy: awareness, enforcement, localisation
Infrastructure: power, buildings, transport
Product: tools, platforms, consumables
Partnerships: coordination, incentives
Governance: decision making power and processes
Learning: cross-local learning, learning for scale
Infrastructure: servers, ICT
Skills & mindsets: learning materials & methods
Incentives & availability: staffing levels, rewards etc.
User needs
Interoperability & scalability
Ownership & financial sustainability
Political power and decision making
Learning frameworks and mechanisms
Learning frameworks and mechanisms
Vision
Legal and policy framework
Effective decentralisation
Financing
Coordination and information sharing
Learning frameworks and mechanisms
Vision
Capacity & mindset
Legal and policy standards
Opportunities to enhance vaccine data system strengthening efforts
Our portfolio of grants illuminated key opportunities for strengthening vaccine data systems in Nigeria and Uganda. We surfaced these through the dedicated studies and literature reviews carried out by grantees, feedback from collaborators and training participants, and in in-person workshops with stakeholders across the vaccine data system, in both countries.
Here are eleven ways that we can all improve our efforts to strengthen vaccine data systems, and health data systems in general, based on our learning. These emphasise a multi-pronged and collaborative approach, which looks at all aspects of the system.
This involves anticipating and addressing the challenges of 'last mile' delivery, aligning scaling efforts with strategic approaches, fostering collaboration among data development partners, and prioritising capacity building that encompasses skills, knowledge, behavioural change, and institutional adaptability.
-
In our work, we explored how to use ‘national’ data management tools (like the App Studio, by HISP) at the local level, and a suite of interventions which would make it easier and more cost-effective for local stakeholders to collect and use data.
There are, we learned, both up front and ongoing costs associated with transitioning to more digital ways of working- not all of which can be anticipated, and not all of which will be ‘sustainable’ in the sense used by many traditional donors.
For example, consider the deployment of digital data collection tools like Smart Paper in rural healthcare facilities. While these tools offer significant advantages in terms of data accuracy and accessibility, they may also require additional resources for maintenance, training, and ensuring that they are resilient to local conditions.
Addressing these 'last mile' challenges proactively can help optimise the use of data solutions in healthcare delivery- but government and funders need to test and learn about the short and long term costs of new tools and approaches, and the best ways of covering them. In the short term, funders may need to accept a greater share of ‘unsustainable’ consumables costs, for the long term payoff of sustained capacity.
One such controversial aspect of these costs is the need for local officials to travel. When we talk about connecting national policies with local realities, some exchanges are necessary. In many cases, doing the work effectively requires officials to travel. For example, in our training of trainers and mentoring schemes, district officials travelled to different locations to carry out mentoring/training. This was a very effective model, but is resource-intensive, and funders and government can consider the best ways of allowing for this in the short and long term.
-
Scaling data solutions locally involves not only acknowledging the associated costs but also ensuring that these investments yield value for money over time. To achieve this, a strategic approach to decision making about and funding of scaling is essential, involving government, donor, and private sector investments. Within this, we see four opportunities:
Portfolio Approaches: Embracing a portfolio approach to data system strengthening allows for a more holistic view of interventions. By treating data initiatives as part of a broader portfolio, stakeholders can make informed decisions about where to allocate resources and efforts.
Learning Frameworks: Establish learning frameworks which consider ‘what we need to know, to decide whether to scale this idea’. This would require a collaborative approach to generate cross-stakeholder buy-in, clarity on what the ambitions are, and new spaces for international funders to try out being collaborative.
Shared Learning: Sharing experiences, challenges, and best practices across different health projects and programs can lead to a more comprehensive understanding of what works and what doesn't. This shared learning can inform decision-making and improve scalability.
Government, Donor, and Private Investment: Effective scaling requires a mix of funding sources, including government, donor, and private sector investments. Combining these resources strategically can ensure sustainable and impactful data solutions.
The hope is that these together enable decision-makers to determine which interventions should be scaled up and which should not, based on evidence and performance, and reduce fragmentation in international and governmental system strengthening efforts.
-
In the pursuit of strengthening healthcare systems and optimising data-driven healthcare solutions, there’s an opportunity to better integrate data system strengthening and ICT development efforts in the face of fragmentation.
Cross-department cooperation: Despite their significant coordination efforts government departments still often operate in silos. For example, different departments holding the ICT/health data system/immunisation data system mandates without incentives and spaces to cooperate might miss opportunities to align, or underestimate risks.
Cross-funder collaboration: Similarly, international funders used to working in our ‘data for development’, ‘health system strengthening’, and ‘innovation’ silos could make a concerted effort to reach out beyond disciplinary boundaries- especially as there are fewer barriers to doing so than many governments face. Future data strengthening efforts could look beyond the ‘usual’ funders and partners.
Better collaboration might improve some aspects of data system strengthening interventions:
Plan ahead for infrastructural constraints: When introducing new digital tools, we need to consider the national infrastructural constraints: anticipating challenges related to server space, internet connectivity, and data security. Involving ICT experts from the outset of healthcare data projects could ensure that infrastructure limitations are considered and mitigated
Design tools for the context: Recognizing that in countries like Nigeria and Uganda, data may be expensive, with limited bandwidth, and conditions that are dusty, humid, or prone to power cuts is important when designing and deploying new digitisation tools. We could explore developing applications and platforms that optimise for those conditions.
Legislative and policy alignment for the AI era: Acknowledge that policy and legislative environments may need to evolve to accommodate advancements in data collection and storage. For example, establishing a robust international data protection legal framework is crucial to instil trust in cloud computing solutions; governments also need to think about the implications of AI for data protection and privacy.
-
We all recognise that collaboration among development partners involved in strengthening data systems is essential to avoid duplicating efforts and streamline initiatives. In general, developing shared methods and frameworks can help in harmonising approaches and stop ‘reinventing the wheel’.
In the data development field, various partners may independently work on similar initiatives, by getting on the same page about the essential ‘curricula’ for healthcare workers that will help them collect and use data, we could improve effectiveness. By sharing methods and frameworks, partners could reduce redundancy and ensure that resources are used more effectively.
For instance, if multiple organisations are supporting healthcare workers to use Human Centre Design methods, they could collaborate to create a shared framework for the process, identifying common principles and guidelines. This approach not only saves time and resources but also leads to more consistent and effective outcomes.
However, we might need to address intellectual property (IP) and copyright issues that may hinder the sharing of methods and tools among international stakeholders- while they haven’t affected this work, we have seen IP issues stymie sharing even within Consortia of NGOs operating under a shared legal framework. Governments can play a role in advocating for greater collaboration and standardisation.
-
Most partners make significant efforts to align with national policies- from state actors, to international partners. However, these national policies are often developed over longer timeframes, are complicated and often rigid, and by necessity can’t account for the myriad of contexts they must apply to.
Local governments, data users, and health research organisations may have been consulted during the policy-making process, however it would be even more helpful to have a feedback loop that allows for a policy to adjust once it’s being implemented in reality. Flexibility allows policies to evolve and better meet the needs of different regions.
We’d like to explore what more flexible and adaptive policy making in the health space could look like. What could it look like to adapt national policies based on the experiences and lessons learned from local implementation? What does an inclusive policy cycle that involves state and local actors look like?
We also think it’s important to explore how to flip the script in national vaccination programming: instead of districts implementing national campaigns (e.g. for measles vaccination), what would it look like for local governments to define the most needed vaccination campaigns, with national support towards implementation?
-
Capacity building is a multifaceted endeavour that requires a combination of efforts to enhance skills and knowledge among healthcare workers and other stakeholders. Rather than focusing on a single prong, it's essential to address various aspects of skills and knowledge.
Interpreting Data: Healthcare workers need the skills to interpret and analyse data effectively. This involves understanding the implications of data trends and using them to inform decision-making.
Appreciating Hesitancy: We found that the people planning vaccine programmes locally don’t always try to understand and address behavioural/psychological reasons for vaccine hesitancy, so it may help to integrate that into learning materials.
Participatory Approaches: Encouraging participatory decision-making methods is crucial: healthcare workers and district officials really appreciated experiencing participatory decision making alternatives.
Digital literacy: digital and tech literacy may be poor in areas where digitisation efforts are going ahead.
We also need to consider scaling more effective learning models:
Adult Training: Traditional classroom-based training may not be sufficient for adult learners. Exploring alternative approaches to training, such as on-the-job learning and mentorship, can be more effective.
Combining Digital and Non-Digital Tools: In contexts with high attrition rates and limited digital literacy, combining digital and non-digital tools for training and capacity building can ensure that all healthcare workers, including new recruits, have consistent access to meaningful learning opportunities.
-
Addressing vaccine hesitancy is a critical aspect of healthcare delivery. While research on hesitancy is essential, it's equally important to translate this research into clear guidance and practical tools that can help community-level individuals explore the reasons behind hesitancy.
Practical Tools: Develop user-friendly tools that community-level healthcare workers and volunteers can use to engage with hesitant individuals. These tools should provide insights into the specific concerns and questions of community members.
Research Insights: Use research findings to inform the development of behavioural toolkits and communication strategies that address vaccine hesitancy effectively. These tools should be evidence-based and tailored to the local context.
-
People can’t always put their new skills and knowledge into practice, and nor is training necessarily enough for people to start using a new tool. Introducing new tools and practices often requires behaviour change and overcoming institutional obstacles.
Support behaviour change: Healthcare workers may need to adjust their behaviours and routines to incorporate new skills and tools into their practice. Understanding the factors that influence behaviour change and providing support in this regard is essential.
Institutional obstacles: Institutions and organisations may have existing procedures and protocols that resist change. Identifying and addressing these barriers is crucial to put new ways of working into practice: initiatives like ours could better consider up front what those barriers might be.
-
Local networks and trusted partners play a vital role in bridging the gap between healthcare initiatives and communities. Traditional leaders, community organisers, and non-governmental organisations can act as intermediaries to facilitate engagement and communication.
Community Health Workers: Empower community health workers and volunteers to serve as intermediaries between healthcare providers and community members. They can help build trust and facilitate communication.
Trusted Partners: Collaborate with local non-governmental organisations and community leaders who have established trust within the community. These partners can assist in outreach and awareness campaigns.
-
Efforts to share healthcare data should be accompanied by well-defined frameworks that balance data sharing and privacy concerns. We’ve recommended creating guidelines and mechanisms that promote data sharing while safeguarding individuals' privacy.
Privacy Protection: Establish clear guidelines for protecting the privacy of healthcare data. This includes data encryption, access controls, and adherence to data protection laws.
Data Sharing Frameworks: Develop data sharing frameworks that outline the conditions under which data can be shared, who can access it, and for what purposes. We’ve found that ambiguity lends itself to reduced data sharing.
-
Before adding new tools to the healthcare ecosystem, it's important to consider their added value and long-term sustainability.
Leveraging Existing Resources First: Before introducing new tools into the healthcare ecosystem, explore how to maximise the potential of existing resources
Contextual Design: Develop tools with a deep understanding of the local context. Account for factors such as infrastructure limitations, digital literacy levels, and resource constraints. Tailoring tools to specific environments enhances their usability and impact.
Plan for the Long-Term: Know who is going to eventually own the tool and take over responsibility for strategy, maintenance and user-roll out, and bring them along the journey with you. This will also help with alignment with broader healthcare objectives, which help make sure that the tools contribute meaningfully to overarching goals.
Specific Entry Points
Nigeria
Last mile capacity building. On-ground staff needs consistent training to manage evolving data tools, understand behavioural factors affecting hesitancy, and use participatory decision making approaches. These could use shared methods and frameworks. We have started to pilot approaches such as this 1-day data simulation exercises in Cross River State.
Innovations to enhance data quality. In addition to continual assessment and management of data quality, exploring next generation technology’s potential to spot and address quality issues.
Transition to digital data collection and reporting. Effectively try and scale tools which make data collection and reporting more efficient and easier, and which digitises data.
More research into hesitancy. Better understanding the reasons for hesitancy, and ensuring that local communities can act on that evidence.
Collaborative multi-stakeholder effort behind data system strengthening. Continuing to bring multiple stakeholders across silos together to imagine a better future, using shared learning frameworks.
Data Source Consolidation. There's a need to consolidate multiple data sources to streamline information gathering and decision-making.
Investment in ICT Infrastructure. Robust ICT infrastructure is required to ensure safe and efficient data collection, storage, and sharing. This will facilitate collaboration among various stakeholders.
Uganda
Last mile capacity building. On-ground staff needs consistent training to manage evolving data tools, understand behavioural factors affecting hesitancy, and use participatory decision making approaches. These could use shared methods and frameworks.
Transition to digital data collection and reporting. Effectively scale tools which make data collection and reporting more efficient and easier, and which digitises data.
More research into hesitancy. Better understanding the reasons for hesitancy, and ensuring that local communities can act on that evidence.
Collaborative multi-stakeholder effort behind data system strengthening. Continuing to bring multiple stakeholders across silos together to imagine a better future, using shared learning frameworks.
Data Sharing Policies. The recent draft by MoH about data sharing, access, and quality is a significant step. However, policies must be adapted and updated to address current challenges and needs.
Data Privacy and Security. Ensuring data privacy is essential, especially for sensitive health data. This requires up-to-date laws, regulations, and policies that consider the current needs and challenges.
Data Source Consolidation. There's a need to consolidate multiple data sources to streamline information gathering and decision-making.
Investment in ICT Infrastructure. Robust ICT infrastructure is required to ensure safe and efficient data collection, storage, and sharing. This will facilitate collaboration among various stakeholders.
What’s next?
The Vaccine Data CoLab in this form ended in October 2023. We’d like to thank the Government, Non-Government, International Organisations, our country partners Dev Afrique in Nigeria and Infectious Disease Institute (Makerere University) in Uganda and others who have contributed to our learning so far. We know that Better Futures are realised when instead of building incrementally from today, we convene a group of experts, innovators, and funders and work collaboratively. These partners are the vital “Co” in the Vaccine Data CoLab.
We are passionate about strengthening health systems and data for decision making.
We’d love to know what you think about what we’ve learnt, and if you also see opportunities to continue the collaboration and spread and scale the approach to other countries and health topic.
If you work in the vaccine data ecosystem in any capacity, whether you’re based in Nigeria or Uganda or elsewhere, please do get in touch with us: vaccinedata@makingbetterfutures.org.