Enhancing quality assessment instruments and understanding provider behavior and perceptions of quality of care within Senegal’s Performance Based Incentives program

Photo of Mother and child at their home in Nigeria

Research Overview

Enhancing Quality of Care Assessment Instruments

Senegal’s Ministry of Health and Social Affairs (MOH), with support from USAID, launched a pilot performance based incentives (PBI) program in 2012 with a focus on maternal, newborn and child health and infectious disease.* The PBI program incentivized improvements in health quality and health outcomes in the low-resource context of Senegal’s health system, with a goal of motivating health workers and strengthening the capacity of district health teams to better meet incentivized targets. The RBF program rewarded performance of health centers and providers based on the quantity of services provided, adjusted for a quality of care rating at the service delivery level.

The MOH pushed to improve the quality of care assessment instrument (QCAI) checklist to include process and results indicators that were more within providers’ scope of control (e.g. from indicators related to infrastructure or staffing to indicators related to patient safety and equipment maintenance). The updated QCAI was comprised largely of structural indicators of quality. The MOH was interested in further updating it to include more process and outcome indicators, which are a more optimal measure of quality. In addition, the MOH hoped to develop a checklist for use in hospital settings (the existing QCAI was applied only in primary care settings – health centers and health posts).

This study aimed to enhance the primary care QCAI to capture broader measures of quality of care. In addition, the study team supported the MOH in designing a secondary care QCAI, with a focus on essential obstetric and neonatal care indicators (EONC). These modifications to the existing QCAI reflected provider and clinical perceptions of quality of care, and are being used to determine distribution of reward payments to providers and facilities. Further, this study focused on process documentation in terms of the QCAI revision and development process.

Senegal Black Box Study

RBF programs in health are driven by the assumption that healthcare service providers will respond to financial incentives on the quantity and quality of care they provide. Increases in quantities of services provided and gains in scores on quality measures show that providers respond to the incentives. Little was known, however, about how specific behaviors change to attain the quality scores, and if the RBF incentives also change perceptions of what ‘quality’ healthcare should look like. To further understand what goes on inside the ‘black box’ of provider behavior and response to incentives under RBF programs, the TRAction project conducted the Black Box Study in Senegal.

Project Location

The enhancing quality of care assessment instruments study took place in the Kaffrine district of Senegal.

The Senegal Black Box Study took place in Kaffrine, Kolda and Kaolack districts.

Research Objectives and Study Approach

Enhancing Quality of Care Assessment Instruments

The study had three primary research objectives:

To revise the existing primary care QCAI and to design the secondary care QCAI to include provider, patient, and clinical perceptions of quality.
To document best practices for implementing a QCAI in primary and secondary care facilities.
To determine the effect of a quality assessment mechanism on service provision and utilization.

KIT conducted a review of existing literature from low, middle and high income countries on processes for holding providers accountable for quality of care, in order to identify analytical frameworks to guide the checklist revision process. In October of 2015, KIT facilitated a workshop with the Programme National du Financement Basé sur Ies Résultats (PNFBR) to discuss and come to consensus on how to revise the primary QCAIs and develop a secondary QCAI. After the workshop, KIT made a field visit to Kaffrine to test the enhanced primary level instruments to gather initial information about providers' perspectives. KIT drew on PNFBR's inputs and the consensus reached during the October workshop, as well as providers' perspectives from the field test and exchange of e-mails afterwards, to develop and submit a proposal for the revised checklists (one for the health center level (primary) and another for the district hospital level (secondary)) by the end of November, 2015. In March, 2016, KIT held working sessions with the PNFBR to review and discuss proposed checklists. With feedback from PNFBR, KIT updated the checklists with final proposals.

Senegal Black Box Study

The Black Box Study sought specifically to understand the relationship between RBF implementation and healthcare service provider perceptions of and behaviors concerning quality of care. As the PNFBR began to expand the RBF program in 2016 with a view to national coverage in the coming years, this study came at a critical time in assessing the effect of this financing scheme on the quality of care.

The specific objectives of the Black Box Study were to:

Assess how providers react to the quality incentives of RBF programs.
Formulate recommendations to improve quality mechanisms in the Senegalese RBF program.
Contribute to the national and international knowledge base on how to promote improved quality within RBF programs.

The study included 28 structured interviews with health center and health post workers (19 in RBF facilities and nine in comparison facilities) and 16 focus group discussions (11 in RBF facilities and five in comparison facilities) across the two pilot RBF regions and one comparison region, Kaolack. Study facilities included a selection of health centers and health posts within a selected district. The study included two data collection methods at each selected facility: (1) structured interviews with individuals and (2) focus group discussions.

Findings

Enhancing Quality of Care Assessment Instrument

KIT developed proposed revisions to the primary-level QCAI including removing and adding indicators, standardizing methods of verification and dividing into two components: (i) an annual checklist, containing the “structure” indicators; and (ii) a quarterly checklist with the “process” and “results” indicators.
Both the quarterly primary-level QCAI and the annual primary-level QCAI were finalized.
KIT designed and finalized the secondary-level QCAI with PNFBR input. Some indicators were removed or grouped together to simplify the checklist. Weighing of different indicators was also added.
Overall, KIT made marginal changes rather than considering a full “over-haul” of the instruments.

Senegal Black Box Study

Overall, RBF respondents described multi-faceted definitions of quality and cited diverse approaches to achieving quality – including internal capacity building, external engagement, and seeking additional resources. Those exposed to RBF incentives also reported on collaboration and teamwork towards achieving goals while comparison respondents did not describe such behaviors. Across all respondent groups, lack of resources was cited as a key barrier to achieving quality, although comparison facilities seemed to be less aware of how to achieve quality than RBF respondent groups. Responses to questions about barriers to attaining better quality and supervision (across RBF and comparison groups), indicate that supervision is mostly a one-way monitoring activity at present than an interactive and engaging supportive exercise.

Overall, this study indicated that RBF has had a positive impact on provider behavior and approaches to quality of care.

Research into Action

These studies are examples of a close research collaboration with the MOH and the PBI program partners to improve program processes through implementation science. Enhancements to Senegal’s QCAI checklists will be used in the expanding RBF program, and the findings on provider behavior contributed to the evidence gap on the effect of PBI programs on provider behavior. and perceptions.