Knowledge From the Afrikan World, For the World

PMC Articles

A health technology assessment of COVID-19 vaccination for Nigerian decision-makers: Identifying stakeholders and pathways to support evidence uptake

PMCID: PMC11210088

PMID: 38926716

Abstract

Background Nigeria commenced rollout of vaccination for coronavirus disease 2019 (COVID-19) in March 2021 as part of the national public health response to the pandemic. Findings from appropriately contextualized cost–effectiveness analyses (CEA) as part of a wider process involving health technology assessment (HTA) approaches have been important in informing decision-making in this area. In this paper we outline the processes that were followed to identify COVID-19 vaccine stakeholders involved in the selection, approval, funding, procurement and rollout of vaccines in Nigeria, and describe the process routes we identified to support uptake of HTA-related information for evidence-informed policy in Nigeria. Methods Our approach to engaging with policy-makers and other stakeholders as part of an HTA of COVID vaccination in Nigeria consisted of three steps, namely: (i) informal discussions with key stakeholders; (ii) stakeholder mapping, analysis and engagement; and (iii) communication and dissemination strategies for the HTA-relevant evidence produced. The analysis of the stakeholder mapping uses the power/interest grid framework. Results The informal discussion with key stakeholders generated six initial policy questions. Further discussions with policy-makers yielded three suitable policy questions for analysis: which COVID-19 vaccines should be bought; what is the optimal mode of delivery of these vaccines; and what are the cost and cost–effectiveness of vaccinating people highlighted in Nigeria’s phase 2 vaccine rollout prioritized by the government, especially the inclusion of those aged between 18 and 49 years. The stakeholder mapping exercise highlighted the range of organizations and groups within Nigeria that could use the information from this HTA to guide decision-making. These stakeholders included both public/government, private and international organizations The dissemination plan developed included disseminating the full HTA results to key stakeholders; production of policy briefs; and presentation at different national and international conferences and peer-reviewed publications. Conclusions HTA processes that involve stakeholder engagement will help ensure important policy questions are taken into account when designing any HTA including any underpinning evidence generation. Further guidance about stakeholder engagement throughout HTA is required, especially for those with low interest in vaccine procurement and use.

Full Text

The first confirmed case of coronavirus disease 2019 (COVID-19) in Nigeria was recorded on 27 February 2020 [1]. During the first wave of COVID-19, the epicentres of infection were in Lagos, Kano and Abuja [2]. A sero-surveillance study conducted in October 2020 in three Nigerian states with 8000 individuals found the prevalence of severe acute respiratory syndrome (SARS-CoV-2) antibodies was 23% in Lagos and Enugu States, 19% in Nasarawa State and 9% in Gombe State [3]. The diversity of SARS-CoV-2 strains indicates multiple introductions of the virus into Nigeria from different parts of the world and adds to evidence of community transmission in different states of Nigeria [4]. As of 7 June 2023, 266,675 confirmed COVID-19 cases and 3155 related deaths have been reported to WHO [5].
Nigeria commenced rollout of vaccination for COVID-19 in March 2021 as part of the national public health response to fight the pandemic [6]. It was expected that mass vaccination would reduce the risk of infection and transmission. At the commencement of the vaccination programme, Nigeria set the ambitious target of vaccinating 40% of the population by December 2021 and 70% by December 2022. This was to be achieved through a four-phase rollout schedule starting with health/frontline workers and strategic leaders (Table 1).
Financing the cost of COVID-19 vaccination has proven difficult in low- and lower-middle-income countries, including Nigeria. In addition, a sudden increase in government spending coupled with a steep decline in fiscal revenue has caused an economic downturn and financing imbalances in Nigeria [7, 8]. For the first 18 months of the pandemic, there was no significant discussion about the national purchase of COVID-19 vaccines; vaccines used in Nigeria were donated, with national authorities bearing the cost of logistics, training and distribution. However, the Johnson & Johnson COVID vaccine was purchased by the Government of Nigeria on the 11 August 2021 at $7.50 per dose, which is lower than its price of $10 per dose, through African Union’s African Vaccine Acquisition Trust (AVAT) [9]. Plans are in place to continue vaccinating Nigerian citizens and approval has been obtained from The World Bank Board of Directors for a $400 million credit to provide upfront financing for safe and effective COVID-19 vaccine acquisition and deployment within the country [7]; further discussions on vaccine purchases (and associated prices) will be necessary. Findings from evidence-informed health technology assessment (HTA)-type approaches will be important to inform decision-making in this area.
HTA is a framework for collating evidence on healthcare interventions as well as a decision-making framework to inform resource allocation decisions and increase the value of discretionary healthcare expenditures [10, 11]. Cost–effectiveness evidence is key in HTA. Obtaining good value for money for COVID-19 vaccine procurement matters in Nigeria; in a highly resource-constrained setting, opportunity costs are high.
To ensure that evidence generated by the HTA process is taken up into policy, there is a need to involve key stakeholders in the conceptualization of the research (including model development) and the development of recommendations, as well as in communication and policy translation. Without stakeholder engagement, lack of awareness, understanding or confidence about economic models will hinder their use and impact [12]. Studies have also shown that adequate engagement of key stakeholders in evidence generation can lead to increased research uptake [13–16]. Stakeholder involvement can build political and social legitimacy for the evidence-to-policy process, which is particularly important in Nigeria, where the use of research findings by policy-makers and communities has traditionally been limited and challenging [17]. Inadequate communication between researchers and policy-makers, and a lack of involvement of policy-makers and the wider community in shaping research activities, has been observed [18].
In this paper we outline the processes followed to identify COVID-19 vaccine stakeholders involved in the selection, approval, funding, procurement and rollout of vaccines in Nigeria, and describe the process routes identified to support uptake of HTA-related information for evidence-informed policy in Nigeria. This work was part of a larger study that sought to explore the cost–effectiveness of COVID-19 vaccination in Nigeria, taking into account context-relevant policy questions [19]. The Nigerian government had already begun its phase 1 rollout of the COVID-19 vaccine when this work began (phase 1 focused on front-line health workers, etc.); our focus was on phase 2. The economic evaluation analysis we conducted broadly confirmed the “age group prioritization strategy of the Nigerian government (which focused on a 50+ cohort during phase 2 of the rollout) and different types of delivery made little difference to the results” [19].
Our approach to engaging with policy-makers and other stakeholders as part of a cost–effectiveness analysis within the HTA process of COVID-19 vaccination in Nigeria was informed by experiences documented previously in Nigeria [16] and consisted of three steps: (i) informal discussions with key stakeholders; (ii) stakeholder mapping, analysis and engagement; and (iii) communication and dissemination strategies for the HTA-relevant evidence produced. We define a stakeholder as individuals, groups or organizations which have a direct interest in the topic under scrutiny and can potentially affect the goals or the performance of a sector, plan or policy [20].
The Federal Ministry of Health approached a team of researchers from the Health Policy Research Group of the College of Medicine, University of Nigeria Nsukka (and co-authors on this study), in December 2020, at the peak of the second wave of the COVID-19 pandemic, about the need to produce evidence on the relative cost–effectiveness of different vaccines for the control of COVID-19 in Nigeria. Several informal discussions were held with the Minister of Health’s office, the Nigerian Center for Disease Control (NCDC) and the National Primary Health Care Development Agency (NPHCDA) on the need for such evidence. These stakeholders stressed that such information could support them on investment decisions with respect to COVID-19 vaccines and the wider healthcare system as part of the ongoing response to the pandemic. This method to support engagement with stakeholders in Nigeria on evidence-informed policy-making was based on strategies (1 and 2) described and summarized in Fig. 1 [16].
Four evidence-informed policy-making strategies [21]. GRIPP, getting research into policy and practice
The identification of a broad variety of pertinent stakeholders was made possible using a snowballing strategy. Initial informants were asked to name people and organizations that had a say in the COVID-19 vaccine selection, approval, funding, procurement and distribution. This technique made it possible to identify both the individuals frequently seen as significant, and those who may have been less well-known or influential but who may nevertheless have an impact on the development and use of vaccines. The strategy used was adapted from elsewhere [21].
The study team then narrowed down the stakeholders on the basis of relevance and role. This took place over 2 weeks. To support further identification of priority stakeholders to help maximize research impact, a categorization scheme was applied on the basis of a qualitative assessment of relative power and interest in the policy area [22]. Here, power is defined as the potential capacity to influence policy decisions, and the power judgment is based on an assessment of stakeholders’ resources [23]. Interest is defined as the degree to which stakeholders are likely to be affected by policy change. The degree of interest or concern stakeholders have about a policy will influence how the stakeholder’s resources, and how much of those resources, will be used in the policy debate [23]. The stakeholders were grouped into four categories depending on their interest and power in COVID-19 vaccine selection, approval, funding, procurement and rollout in Nigeria: (i) high power and high interest; (ii) high power and low interest; (iii) low power and high interest, and (iv) low power and low interest. This provided information on their relative influence and likely interest in the findings of the COVID vaccination cost–effectiveness analysis (CEA), allowing the authors to prioritize stakeholders who would need to be kept informed. In addition, the roles of the stakeholders were identified. The mapping was supported with document reviews.
Stakeholder analysis is the systematic identification, evaluation and prioritization of individuals or organizations who can influence or have an interest in a project or program. [24]. The approach can assist with the development of an effective stakeholder communication and engagement strategy and is a fundamental element of an organization’s stakeholder management plan. We used three key steps as set out elsewhere [23] to conduct the stakeholder analysis, namely:
Input from Nigerian policy-makers (some of whom are listed in Table 2) was sought through informal emails and discussions, on the key policy questions that need to be addressed to support decision-making in Nigeria and in the Africa CDC at large, with regards to COVID-19 vaccine selection, approval, funding, procurement and rollout activities. These questions were shared with colleagues from the Health Economics Programme at the Africa CDC and the economics and modelling team from the London School of Hygiene & Tropical Medicine (LSHTM). This input was important in helping determine whether the policy questions could be addressed with the modelling tools available.
The stakeholder mapping exercise highlighted the range of organizations and groups within Nigeria that could use the information from the HTA to guide decision-making. It was anticipated that these organizations would benefit from the findings of the study, either in costing their programs or in the use of the evidence for policy and decision-making and as a tool for advocacy. The results may also offer useful data and opportunities for resource allocation, strengthening of the health system and knowledge translation, resulting in improved health outcomes and better-informed decision-making. The names, roles and responsibilities of the various stakeholders that were mapped are shown in Table 2.
Figure 2 illustrates key stakeholders involved with the decision-making of COVID-19 vaccine procurement and their interaction with one another. They either make the decisions or are impacted by the decisions made. For example, the Presidential Steering Committee (PSC) reports directly to the President of Nigeria. The PSC interacts with the FMoH, which is responsible for developing vaccine policy for the country and ensuring implementation. The FMoH interacts with all the other agencies, as shown in Fig. 2. The NPHCDA receives and coordinates the distribution of the vaccines until it gets to the end users and, with the FMoH, jointly decides on the types of vaccine to be procured and the quantity needed, as well as on issues linked to storage capacity. The NAFDAC regulates and licences potential vaccines prior to use. Figure 2 also shows other important actors, including the various technical working groups, development partners (DPs) and regional bodies.
The stakeholders in Fig. 2 were categorized using a power/interest grid framework [23], as shown in Fig. 3.
The resulting matrix (Fig. 3) identified key stakeholders who could assist with the prioritization of resources within the context of COVID-19 vaccination and as such provided a foundation for a communication and engagement plan. Thus, the stakeholders were categorized into four groups:
These three policy-relevant questions were further expanded by the researchers through an interactive and consultative process into the context-specific decision problem, as set out in Table 3. This was used as the basis for modelling the cost–effectiveness of the selected vaccines.
Table 3 shows the summary of the decision problems modelled in the study. The epidemiological model of virus transmission parametrized with Nigeria-specific data shown in Table 3 was combined with a costing model that incorporated local resource use assumptions and prices, both for vaccine delivery as well as costs associated with care and treatment of COVID-19. Scenarios of vaccination were compared with no vaccination [19].
For the purpose of this HTA, the following outline dissemination plan was developed and used to support dissemination in an ongoing process (Table 4):
Table 4 shows the dissemination of the results with the different messages that were shared with different stakeholders on the basis of the stakeholder analysis done. The stakeholders included academia, policy-makers from both the national and sub-national levels, and the Africa CDC. The results were well received by these stakeholders, and it is still ongoing.
The importance of decision-makers comprehending HTA requirements, uses and scope as well as the optimal approaches to incorporating HTA components into the design of decision-making mechanisms has been emphasized by a number of authors. [25]. HTA is not a narrow technocratic exercise, and as such it is important to also identify or develop methods of wider citizen and stakeholder involvement that could be considered best practice internationally. For example, the evidence-informed deliberative processes (EDPs) framework [26]), though not used in our research due to limited resources and time, can support the decision-making process in HTA bodies and, thus, contribute to the legitimacy of recommendations and decisions. How any EDP process is implemented as part of HTA institutionalization in a particular setting would likely be shaped by local conditions and preferences.
Although stakeholder involvement is key to improving the HTA process, the form and timing of such improvements must be adapted to local contexts and data. In Nigeria, the perceived availability and accessibility of suitable local data to support HTA varies widely but is mostly considered inadequate and limited [27]. In our study, it was necessary to ensure that the cost–effectiveness analyses were consistent with the epidemiological context and that stakeholders were involved in (at least) initial discussions around study design to enhance credibility and engagement [28]. Inadequate understanding of the links between HTA-related outputs and decision-making can lead to key stakeholder exclusion and loss of policy relevance [29]. To maximize the relevance and timeliness of COVID-19 vaccine HTAs (or indeed HTAs for any topic), it is important to first identify the stakeholders and processes for decision-making on technology procurement [30].
Given the large number of stakeholders in COVID-19 vaccine procurement and use, it was necessary to identify priority actors to guide the dissemination of research findings. As noted by some authors, this process should not be considered as a one-off activity, but rather an ongoing pattern of engagement, to allow for changes in the stakeholders’ positioning [24]. Overall, the power/interest matrix is a useful tool for stakeholder analysis since it offers a structured method for comprehending stakeholder dynamics, setting priorities for engagement initiatives, and successfully managing relationships. However, other criteria could also be considered when grouping stakeholders, such as influence, legitimacy, expertise, attitude and resources, which could provide a more nuanced understanding of stakeholders and help to inform engagement strategies that are tailored to their needs and concerns [31].
The experience set out in the present study demonstrates the importance of identifying key stakeholders early on as part of any evidence-driven activity and ensuring an ongoing degree of engagement to enhance the value of the research findings. Despite the depth of the stakeholder mapping, issues with power dynamics and representation of stakeholder interests are regularly encountered when trying to acquire a true reflection of the balance of all stakeholders. [32, 33]. Ensuring representation of stakeholder interests, power balance and transparency of decision-making are critical in COVID-19 vaccine procurement and use in Nigeria. However, it is difficult to say how this played out in our study, as there are multiple stakeholders involved, and the situation is constantly evolving. We found it easier to engage some stakeholder groups such as government workers and academia, than those in the private sector such as the CACOVID in this study. Even though efforts have been made to involve stakeholders and ensure transparency, there is always room for improvement, such as inclusive stakeholder representation, early and ongoing engagement, establishing mechanisms for two-way communication, allowing stakeholders to provide feedback and making decisions that are informed by the input of all stakeholders. Ongoing communication and engagement with stakeholders are crucial to addressing their concerns and ensuring accountability. Researchers need concrete information on potential stakeholder groups affected by COVID-19 immunization to be more inclusive. Nevertheless, structured guidance on how to maximize stakeholder involvement in HTA is needed.
Policy-makers in this study highlighted the importance of supply chain consequences, especially with cold chain, and storage when decisions are made on vaccine selection. It is well known that investing in vaccinations to prevent COVID-19 infections might have a significant return on investment. [34]. However, there could be very significant budgetary or resource consequences for the government and other payers from very rapid vaccine rollouts, especially where there are important cold chain requirements. Formal cost–effectiveness analysis combined with an assessment of budget impact features typical of the HTA process are clearly needed to support appropriate vaccine selection in a given context. Information regarding the highest price charged for each COVID vaccine choice was also requested by policy-makers.
Policy-makers were interested in knowing what vaccines, and critically at what price, should be purchased. However, in addition, policy-makers were concerned about the costs of delivery to inform the best way to deliver each of the vaccines. The aim here was to support policy-makers in deciding which method provides more cost–effective coverage, considering the high level of hesitancy observed in the uptake of the COVID vaccine [35–37].
As shown in our study, a strategy for communication and dissemination should be informed by a stakeholder mapping exercise. Arguably the more important a stakeholder is (in terms of power and interest for example), the more important it is to establish an effective means of communication with that stakeholder. Timeliness is also an important consideration, and it is a challenge for HTA systems everywhere. Findings need to be delivered quickly (especially in pandemic contexts) and in a style that is clear and useful to the target group. Consequently, policy and decision-makers must be included in the study process. Additionally, it is important to share plain-language research summaries through a variety of platforms, such as social media and in-person meetings between researchers and end users [38, 39].
Aside from the production of a detailed full HTA report on COVID-19 vaccination in Nigeria, other dissemination materials were produced and disseminated by the study team, including a policy brief [40], blogs [41, 42], conference presentations and a peer-reviewed publication [19]. These products were tailored to the needs of the various stakeholders and delivered at different time points in the study.
The results of this study could have significant implications for the development of policies and the improvement of public health in Nigeria, especially in the context of managing future pandemics. The study’s primary objective of determining the optimal COVID-19 vaccines for procurement holds significant importance for the Nigerian government. To optimize resource allocation and ensure the population receives the most effective protection against the virus, it is important to understand the most relevant vaccinations based on variables such as efficacy, safety and cost–effectiveness, given the wide range of alternatives available. This has been noted by Santoli et al., who noted that economic evaluation data for mass vaccination is crucial for decision-makers to make evidence-based, value-based decisions to ensure equitable access and reduce the global COVID-19 burden [43].
The stakeholder mapping exercise done during the study emphasizes the significance of engaging a diverse array of organizations and people in decision-making processes. Incorporating the participation of public/government, private and international organizations guarantees the inclusion of a wide range of viewpoints, hence enhancing the probability of formulating policies and plans that are pertinent, agreeable and efficient across different sectors of society. This has been echoed in a study that addresses the under-researched issue of stakeholder identification and engagement in problem-structuring interventions [44].
The devised dissemination plan for the study, encompassing the disclosure of comprehensive HTA outcomes, the creation of concise policy briefs and the delivery of presentations at both national and international conferences, is crucial for effectively converting research findings into implementable policies and practices. The study’s wide dissemination of findings can have a significant impact on decision-making processes at several levels and facilitate evidence-based policy formulation not only in Nigeria but also in other regions. The study highlights the significance of involving stakeholders in HTA procedures to guarantee the examination of pertinent policy inquiries and the production of supporting evidence. Offering advice on how to engage stakeholders effectively, especially those who are not very interested in vaccine procurement and usage, will improve the inclusivity and efficacy of future HTA programmes. This, in turn, will strengthen health systems and lead to better public health outcomes. Uzochukwu et al. (2020) noted that stakeholder participation in identifying HTA topics and conducting relevant research will enhance the use of HTA evidence produced for decision-making [27].

Sections

"[{\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR1\", \"CR2\", \"CR3\", \"CR4\", \"CR5\"], \"section\": \"Background\", \"text\": \"The first confirmed case of coronavirus disease 2019 (COVID-19) in Nigeria was recorded on 27 February 2020 [1]. During the first wave of COVID-19, the epicentres of infection were in Lagos, Kano and Abuja [2]. A sero-surveillance study conducted in October 2020 in three Nigerian states with 8000 individuals found the prevalence of severe acute respiratory syndrome (SARS-CoV-2) antibodies was 23% in Lagos and Enugu States, 19% in Nasarawa State and 9% in Gombe State [3]. The diversity of SARS-CoV-2 strains indicates multiple introductions of the virus into Nigeria from different parts of the world and adds to evidence of community transmission in different states of Nigeria [4]. As of 7 June 2023, 266,675\\u00a0confirmed COVID-19 cases and 3155 related deaths have been reported to WHO [5].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR6\", \"Tab1\"], \"section\": \"Background\", \"text\": \"Nigeria commenced rollout of vaccination for COVID-19 in March 2021 as part of the national public health response to fight the pandemic [6]. It was expected that mass vaccination would reduce the risk of infection and transmission. At the commencement of the vaccination programme, Nigeria set the ambitious target of vaccinating 40% of the population by December 2021 and 70% by December 2022. This was to be achieved through a four-phase rollout schedule starting with health/frontline workers and strategic leaders (Table\\u00a01).\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR7\", \"CR8\", \"CR9\", \"CR7\"], \"section\": \"Background\", \"text\": \"Financing the cost of COVID-19 vaccination has proven difficult in low- and lower-middle-income countries, including Nigeria. In addition, a sudden increase in government spending coupled with a steep decline in fiscal revenue has caused an economic downturn and financing imbalances in Nigeria [7, 8]. For the first 18\\u00a0months of the pandemic, there was no significant discussion about the national purchase of COVID-19 vaccines; vaccines used in Nigeria were donated, with national authorities bearing the cost of logistics, training and distribution. However, the Johnson & Johnson COVID vaccine was purchased by the Government of Nigeria on the 11 August 2021 at $7.50 per dose, which is lower than its price of $10 per dose, through African Union\\u2019s African Vaccine Acquisition Trust (AVAT) [9]. Plans are in place to continue vaccinating Nigerian citizens and approval has been obtained from The World Bank Board of Directors for a $400 million credit to provide upfront financing for safe and effective COVID-19 vaccine acquisition and deployment within the country [7]; further discussions on vaccine purchases (and associated prices) will be necessary. Findings from evidence-informed health technology assessment (HTA)-type approaches will be important to inform decision-making in this area.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR10\", \"CR11\"], \"section\": \"Background\", \"text\": \"HTA is a framework for collating evidence on healthcare interventions as well as a decision-making framework to inform resource allocation decisions and increase the value of discretionary healthcare expenditures [10, 11]. Cost\\u2013effectiveness evidence is key in HTA. Obtaining good value for money for COVID-19 vaccine procurement matters in Nigeria; in a highly resource-constrained setting, opportunity costs are high.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR12\", \"CR13\", \"CR16\", \"CR17\", \"CR18\"], \"section\": \"Background\", \"text\": \"To ensure that evidence generated by the HTA process is taken up into policy, there is a need to involve key stakeholders in the conceptualization of the research (including model development) and the development of recommendations, as well as in communication and policy translation. Without stakeholder engagement, lack of awareness, understanding or confidence about economic models will hinder their use and impact [12]. Studies have also shown that adequate engagement of key stakeholders in evidence generation can lead to increased research uptake [13\\u201316]. Stakeholder involvement can build political and social legitimacy for the evidence-to-policy process, which is particularly important in Nigeria, where the use of research findings by policy-makers and communities has traditionally been limited and challenging [17]. Inadequate communication between researchers and policy-makers, and a lack of involvement of policy-makers and the wider community in shaping research activities, has been observed [18].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR19\", \"CR19\"], \"section\": \"Background\", \"text\": \"In this paper we outline the processes followed to identify COVID-19 vaccine stakeholders involved in the selection, approval, funding, procurement and rollout of vaccines in Nigeria, and describe the process routes identified to support uptake of HTA-related information for evidence-informed policy in Nigeria. This work was part of a larger study that sought to explore the cost\\u2013effectiveness of COVID-19 vaccination in Nigeria, taking into account context-relevant policy questions [19]. The Nigerian government had already begun its phase 1 rollout of the COVID-19 vaccine when this work began (phase 1 focused on front-line health workers, etc.); our focus was on phase 2. The economic evaluation analysis we conducted broadly confirmed the \\u201cage group prioritization strategy of the Nigerian government (which focused on a 50+\\u2009cohort during phase 2 of the rollout) and different types of delivery made little difference to the results\\u201d [19].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR16\", \"CR20\"], \"section\": \"Methodology\", \"text\": \"Our approach to engaging with policy-makers and other stakeholders as part of a cost\\u2013effectiveness analysis within the HTA process of COVID-19 vaccination in Nigeria was informed by experiences documented previously in Nigeria [16] and consisted of three steps: (i) informal discussions with key stakeholders; (ii) stakeholder mapping, analysis and engagement; and (iii) communication and dissemination strategies for the HTA-relevant evidence produced. We define a stakeholder as individuals, groups or organizations which have a direct interest in the topic under scrutiny and can potentially affect the goals or the performance of a sector, plan or policy [20].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Fig1\", \"CR16\"], \"section\": \"Informal discussions with key stakeholders\", \"text\": \"The Federal Ministry of Health approached a team of researchers from the Health Policy Research Group of the College of Medicine, University of Nigeria Nsukka (and co-authors on this study), in December 2020, at the peak of the second wave of the COVID-19 pandemic, about the need to produce evidence on the relative cost\\u2013effectiveness of different vaccines for the control of COVID-19 in Nigeria. Several informal discussions were held with the Minister of Health\\u2019s office, the Nigerian Center for Disease Control (NCDC) and the National Primary Health Care Development Agency (NPHCDA) on the need for such evidence. These stakeholders stressed that such information could support them on investment decisions with respect to COVID-19 vaccines and the wider healthcare system as part of the ongoing response to the pandemic. This method to support engagement with stakeholders in Nigeria on evidence-informed policy-making was based on strategies (1 and 2) described and summarized in Fig.\\u00a01 [16].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR21\"], \"section\": \"\", \"text\": \"Four evidence-informed policy-making strategies [21]. GRIPP, getting research into policy and practice\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR21\"], \"section\": \"Stakeholder mapping\", \"text\": \"The identification of a broad variety of pertinent stakeholders was made possible using a snowballing strategy. Initial informants were asked to name people and organizations that had a say in the COVID-19 vaccine selection, approval, funding, procurement and distribution. This technique made it possible to identify both the individuals frequently seen as significant, and those who may have been less well-known or influential but who may nevertheless have an impact on the development and use of vaccines. The strategy used was adapted from elsewhere [21].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR22\", \"CR23\", \"CR23\"], \"section\": \"Stakeholder mapping\", \"text\": \"The study team then narrowed down the stakeholders on the basis of relevance and role. This took place over 2\\u00a0weeks. To support further identification of priority stakeholders to help maximize research impact, a categorization scheme was applied on the basis of a qualitative assessment of relative power and interest in the policy area [22]. Here, power is defined as the potential capacity to influence policy decisions, and the power judgment is based on an assessment of stakeholders\\u2019 resources [23]. Interest is defined as the degree to which stakeholders are likely to be affected by policy change. The degree of interest or concern stakeholders have about a policy will influence how the stakeholder\\u2019s resources, and how much of those resources, will be used in the policy debate [23]. The stakeholders were grouped into four categories depending on their interest and power in COVID-19 vaccine selection, approval, funding, procurement and rollout in Nigeria: (i) high power and high interest; (ii) high power and low interest; (iii) low power and high interest, and (iv) low power and low interest. This provided information on their relative influence and likely interest in the findings of the COVID vaccination cost\\u2013effectiveness analysis (CEA), allowing the authors to prioritize stakeholders who would need to be kept informed. In addition, the roles of the stakeholders were identified. The mapping was supported with document reviews.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR24\", \"CR23\"], \"section\": \"Stakeholder analysis\", \"text\": \"Stakeholder analysis is the systematic identification, evaluation and prioritization of individuals or organizations who can influence or have an interest in a project or program. [24]. The approach can assist with the development of an effective stakeholder communication and engagement strategy and is a fundamental element of an organization\\u2019s stakeholder management plan. We used three key steps as set out elsewhere [23] to conduct the stakeholder analysis, namely:\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab2\"], \"section\": \"Stakeholder engagement: crafting the policy questions\", \"text\": \"Input from Nigerian policy-makers (some of whom are listed in Table\\u00a02) was sought through informal emails and discussions, on the key policy questions that need to be addressed to support decision-making in Nigeria and in the Africa CDC at large, with regards to COVID-19 vaccine selection, approval, funding, procurement and rollout activities. These questions were shared with colleagues from the Health Economics Programme at the Africa CDC and the economics and modelling team from the London School of Hygiene & Tropical Medicine (LSHTM). This input was important in helping determine whether the policy questions could be addressed with the modelling tools available.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab2\"], \"section\": \"Stakeholders mapping, analysis and engagement\", \"text\": \"The stakeholder mapping exercise highlighted the range of organizations and groups within Nigeria that could use the information from the HTA to guide decision-making. It was anticipated that these organizations would benefit from the findings of the study, either in costing their programs or in the use of the evidence for policy and decision-making and as a tool for advocacy. The results may also offer useful data and opportunities for resource allocation, strengthening of the health system and knowledge translation, resulting in improved health outcomes and better-informed decision-making. The names, roles and responsibilities of the various stakeholders that were mapped are shown in Table\\u00a02.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Fig2\", \"Fig2\", \"Fig2\"], \"section\": \"Stakeholders mapping, analysis and engagement\", \"text\": \"Figure\\u00a02 illustrates key stakeholders involved with the decision-making of COVID-19 vaccine procurement and their interaction with one another. They either make the decisions or are impacted by the decisions made. For example, the Presidential Steering Committee (PSC) reports directly to the President of Nigeria. The PSC interacts with the FMoH, which is responsible for developing vaccine policy for the country and ensuring implementation. The FMoH interacts with all the other agencies, as shown in Fig.\\u00a02. The NPHCDA receives and coordinates the distribution of the vaccines until it gets to the end users and, with the FMoH, jointly decides on the types of vaccine to be procured and the quantity needed, as well as on issues linked to storage capacity. The NAFDAC regulates and licences potential vaccines prior to use. Figure\\u00a02 also shows other important actors, including the various technical working groups, development partners (DPs) and regional bodies.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Fig2\", \"CR23\", \"Fig3\"], \"section\": \"Stakeholders mapping, analysis and engagement\", \"text\": \"The stakeholders in Fig.\\u00a02 were categorized using a power/interest grid framework [23], as shown in Fig.\\u00a03.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Fig3\"], \"section\": \"Stakeholders mapping, analysis and engagement\", \"text\": \"The resulting matrix (Fig.\\u00a03) identified key stakeholders who could assist with the prioritization of resources within the context of COVID-19 vaccination and as such provided a foundation for a communication and engagement plan. Thus, the stakeholders were categorized into four groups:\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab3\"], \"section\": \"Development of research questions\", \"text\": \"These three policy-relevant questions were further expanded by the researchers through an interactive and consultative process into the context-specific decision problem, as set out in Table\\u00a03. This was used as the basis for modelling the cost\\u2013effectiveness of the selected vaccines.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab3\", \"Tab3\", \"CR19\"], \"section\": \"Development of research questions\", \"text\": \"Table 3 shows the summary of the decision problems modelled in the study. The epidemiological model of virus transmission parametrized with Nigeria-specific data shown in Table\\u00a03 was combined with a costing model that incorporated local resource use assumptions and prices, both for vaccine delivery as well as costs associated with care and treatment of COVID-19. Scenarios of vaccination were compared with no vaccination [19].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab4\"], \"section\": \"Communication and dissemination to support the uptake of the HTA evidence\", \"text\": \"For the purpose of this HTA, the following outline dissemination plan was developed and used to support dissemination in an ongoing process (Table\\u00a04):\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"Tab4\"], \"section\": \"Communication and dissemination to support the uptake of the HTA evidence\", \"text\": \"Table 4 shows the dissemination of the results with the different messages that were shared with different stakeholders on the basis of the stakeholder analysis done. The stakeholders included academia, policy-makers from both the national and sub-national levels, and the Africa CDC. The results were well received by these stakeholders, and it is still ongoing.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR25\", \"CR26\"], \"section\": \"Discussion\", \"text\": \"The importance of decision-makers comprehending HTA requirements, uses and scope as well as the optimal approaches to incorporating HTA components into the design of decision-making mechanisms has been emphasized by a number of authors. [25]. HTA is not a narrow technocratic exercise, and as such it is important to also identify or develop methods of wider citizen and stakeholder involvement that could be considered best practice internationally. For example, the evidence-informed deliberative processes (EDPs) framework [26]), though not used in our research due to limited resources and time, can support the decision-making process in HTA bodies and, thus, contribute to the legitimacy of recommendations and decisions. How any EDP process is implemented as part of HTA institutionalization in a particular setting would likely be shaped by local conditions and preferences.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR27\", \"CR28\", \"CR29\", \"CR30\"], \"section\": \"Discussion\", \"text\": \"Although stakeholder involvement is key to improving the HTA process, the form and timing of such improvements must be adapted to local contexts and data. In Nigeria, the perceived availability and accessibility of suitable local data to support HTA varies widely but is mostly considered inadequate and limited [27]. In our study, it was necessary to ensure that the cost\\u2013effectiveness analyses were consistent with the epidemiological context and that stakeholders were involved in (at least) initial discussions around study design to enhance credibility and engagement [28]. Inadequate understanding of the links between HTA-related outputs and decision-making can lead to key stakeholder exclusion and loss of policy relevance [29]. To maximize the relevance and timeliness of COVID-19 vaccine HTAs (or indeed HTAs for any topic), it is important to first identify the stakeholders and processes for decision-making on technology procurement [30].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR24\", \"CR31\"], \"section\": \"Discussion\", \"text\": \"Given the large number of stakeholders in COVID-19 vaccine procurement and use, it was necessary to identify priority actors to guide the dissemination of research findings. As noted by some authors, this process should not be considered as a one-off activity, but rather an ongoing pattern of engagement, to allow for changes in the stakeholders\\u2019 positioning [24]. Overall, the power/interest matrix is a useful tool for stakeholder analysis since it offers a structured method for comprehending stakeholder dynamics, setting priorities for engagement initiatives, and successfully managing relationships. However, other criteria could also be considered when grouping stakeholders, such as influence, legitimacy, expertise, attitude and resources, which could provide a more nuanced understanding of stakeholders and help to inform engagement strategies that are tailored to their needs and concerns [31].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR32\", \"CR33\"], \"section\": \"Discussion\", \"text\": \"The experience set out in the present study demonstrates the importance of identifying key stakeholders early on as part of any evidence-driven activity and ensuring an ongoing degree of engagement to enhance the value of the research findings. Despite the depth of the stakeholder mapping, issues with power dynamics and representation of stakeholder interests are regularly encountered when trying to acquire a true reflection of the balance of all stakeholders. [32, 33]. Ensuring representation of stakeholder interests, power balance and transparency of decision-making are critical in COVID-19 vaccine procurement and use in Nigeria. However, it is difficult to say how this played out in our study, as there are multiple stakeholders involved, and the situation is constantly evolving. We found it easier to engage some stakeholder groups such as government workers and academia, than those in the private sector such as the CACOVID in this study. Even though efforts have been made to involve stakeholders and ensure transparency, there is always room for improvement, such as inclusive stakeholder representation, early and ongoing engagement, establishing mechanisms for two-way communication, allowing stakeholders to provide feedback and making decisions that are informed by the input of all stakeholders. Ongoing communication and engagement with stakeholders are crucial to addressing their concerns and ensuring accountability. Researchers need concrete information on potential stakeholder groups affected by COVID-19 immunization to be more inclusive. Nevertheless, structured guidance on how to maximize stakeholder involvement in HTA is needed.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR34\"], \"section\": \"Discussion\", \"text\": \"Policy-makers in this study highlighted the importance of supply chain consequences, especially with cold chain, and storage when decisions are made on vaccine selection. It is well known that investing in vaccinations to prevent COVID-19 infections might have a significant return on investment. [34]. However, there could be very significant budgetary or resource consequences for the government and other payers from very rapid vaccine rollouts, especially where there are important cold chain requirements. Formal cost\\u2013effectiveness analysis combined with an assessment of budget impact features typical of the HTA process are clearly needed to support appropriate vaccine selection in a given context. Information regarding the highest price charged for each COVID vaccine choice was also requested by policy-makers.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR35\", \"CR37\"], \"section\": \"Discussion\", \"text\": \"Policy-makers were interested in knowing what vaccines, and critically at what price, should be purchased. However, in addition, policy-makers were concerned about the costs of delivery to inform the best way to deliver each of the vaccines. The aim here was to support policy-makers in deciding which method provides more cost\\u2013effective coverage, considering the high level of hesitancy observed in the uptake of the COVID vaccine [35\\u201337].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR38\", \"CR39\"], \"section\": \"Discussion\", \"text\": \"As shown in our study, a strategy for communication and dissemination should be informed by a stakeholder mapping exercise. Arguably the more important a stakeholder is (in terms of power and interest for example), the more important it is to establish an effective means of communication with that stakeholder. Timeliness is also an important consideration, and it is a challenge for HTA systems everywhere. Findings need to be delivered quickly (especially in pandemic contexts) and in a style that is clear and useful to the target group. Consequently, policy and decision-makers must be included in the study process. Additionally, it is important to share plain-language research summaries through a variety of platforms, such as social media and in-person meetings between researchers and end users [38, 39].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR40\", \"CR41\", \"CR42\", \"CR19\"], \"section\": \"Discussion\", \"text\": \"Aside from the production of a detailed full HTA report on COVID-19 vaccination in Nigeria, other dissemination materials were produced and disseminated by the study team, including a policy brief [40], blogs [41, 42], conference presentations and a peer-reviewed publication [19]. These products were tailored to the needs of the various stakeholders and delivered at different time points in the study.\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR43\"], \"section\": \"Discussion\", \"text\": \"The results of this study could have significant implications for the development of policies and the improvement of public health in Nigeria, especially in the context of managing future pandemics. The study\\u2019s primary objective of determining the optimal COVID-19 vaccines for procurement holds significant importance for the Nigerian government. To optimize resource allocation and ensure the population receives the most effective protection against the virus, it is important to understand the most relevant vaccinations based on variables such as efficacy, safety and cost\\u2013effectiveness, given the wide range of alternatives available. This has been noted by Santoli et al., who noted that economic evaluation data for mass vaccination is crucial for decision-makers to make evidence-based, value-based decisions to ensure equitable access and reduce the global COVID-19 burden [43].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR44\"], \"section\": \"Discussion\", \"text\": \"The stakeholder mapping exercise done during the study emphasizes the significance of engaging a diverse array of organizations and people in decision-making processes. Incorporating the participation of public/government, private and international organizations guarantees the inclusion of a wide range of viewpoints, hence enhancing the probability of formulating policies and plans that are pertinent, agreeable and efficient across different sectors of society. This has been echoed in a study that addresses the under-researched issue of stakeholder identification and engagement in problem-structuring interventions [44].\"}, {\"pmc\": \"PMC11210088\", \"pmid\": \"38926716\", \"reference_ids\": [\"CR27\"], \"section\": \"Discussion\", \"text\": \"The devised dissemination plan for the study, encompassing the disclosure of comprehensive HTA outcomes, the creation of concise policy briefs and the delivery of presentations at both national and international conferences, is crucial for effectively converting research findings into implementable policies and practices. The study\\u2019s wide dissemination of findings can have a significant impact on decision-making processes at several levels and facilitate evidence-based policy formulation not only in Nigeria but also in other regions. The study highlights the significance of involving stakeholders in HTA procedures to guarantee the examination of pertinent policy inquiries and the production of supporting evidence. Offering advice on how to engage stakeholders effectively, especially those who are not very interested in vaccine procurement and usage, will improve the inclusivity and efficacy of future HTA programmes. This, in turn, will strengthen health systems and lead to better public health outcomes. Uzochukwu et al. (2020) noted that stakeholder participation in identifying HTA topics and conducting relevant research will enhance the use of HTA evidence produced for decision-making [27].\"}]"

Metadata

"{\"issue-copyright-statement\": \"\\u00a9 BioMed Central Ltd., part of Springer Nature 2024\"}"