The Evaluation of Effect Gammarana Intervention to Reducing Stunting during the Covid-19 Pandemic: Protocol Evaluation of Stunting Intervention in Enrekang District

Abstract

Background

Evaluation of large-scale stunting interventions in Indonesia has never been carried out, because it found limited sensitive and specific interventions that were carried out massively at the village level. The provincial government of South Sulawesi Indonesia in 2020 has implemented a stunting intervention model called Gammarana. The purpose of this evaluation is to analyze the impact of Gammarana on changes in stunting at the project site. Location project as many as 30 villages with an estimated 60,000 units population.

Design and methods

Evaluation in this study using a retrospective method and internal and external audit to document potential, then validated after the field visit Gammarana first phase in 2020. Basic Logic Model evaluation model with 22 indicators (input, process, secondary output and primary output). Proving the effect of Gammarana on changes in stunting by comparing the phenomena in the comparison village.

Results

The comparison villages were set as equal and comparable in 13 indicators that could disturb the study conclusions. The result of the initial condition is that the conditions of the two villages of Gammarana and Villages Comparison are seen as the same in various characteristics, so that whatever the results of this evaluation study are believed to be the impact of Gammarana Project.

Conclusions

this protocol eligible to evaluation of Gammarana Project Intervention in Enrekang District, South Sulawesi Indonesia.

Introduction

Indonesia is one of the countries with the 3rd largest population in the world, after China and India. This country has a high stunting of 30.8% in 2018, very far from the ideal threshold (<20%). The Indonesian government has also established a multisectoral stunting prevention policy, starting from the central to regional levels. In 2020 the Covid-19 pandemic will add to the burden of Indonesia in overcoming stunting, because access to maternal and child health services is limited and purchasing power decreases. ¹ South Sulawesi Province, took a smart step by implementing the Gammarana project (GP). The focus of this project is the provision of micronutrients, macronutrients and nutrition education to caregivers. ² This activity was carried out by nutritionist. The total target locations were 30 villages in Enrekang district as the location for the highest stunting in 2019, namely 42%. ³

The Indonesian government, even though it already has a stunting intervention scenario, is South Sulawesi which is the most unique and the largest reaching, focused and systematic. Governments from other regions are eager to know how the efficacy of the GP in Enrekang South Sulawesi in 2020. Until now, none of the stunting interventions have been evaluated for their precision efficacy at the variable level which perfectly matches the determinants of stunting factors. ⁴

Gammarana is the only pilot of a large-scale stunting intervention project in Indonesia, which coincides with Covid-19. It is certain that this is the best opportunity to learn from a pilot project designed massively and systematically when the public health crisis peaks. ⁵ It is necessary to carry out an impact evaluation, which is appropriate, accountable, and independent of this project. ⁶ The start of this evaluation is to develop an evaluation protocol that can be observed, assessed and reviewed by public health experts around the world. This protocol proves to the world that South Sulawesi has the best track of running a generation-saving package from accelerated stunting during the Covid-19 pandemic. ⁷

The specific purpose of this protocol is to define GP evaluation criteria (input, process, outcome), in accordance with the Logic Model Theoretical Framework, ⁸ and examine the comparison of determinant factors of stunting both intervention and control villages, before the pilot project started, because this is evidence of GP's causation with changes in stunting before and after the intervention.

Significance for public health

The South Sulawesi Government of Indonesia, is in need of this protocol to evaluate The Gammarana project during the Covid-19 pandemic. It is estimated that there will be an increase in stunting. Early detection can be done, if there is an appropriate protocol and time.

Design and methods

Study design

Gammarana Evaluation (GE) is a retrospective study, registered with the Registry for International Impact Evaluation (RIDIE). Baseline and End line anthropometric data of children are recorded from the 2020 GP report document. Comparative data were taken from 30 Non GP Villages. ⁹ Anthropometric data (height, weight) for August 2020 were recorded from the Community Based Nutrition Reporting and Recording (CBNR&R) Big Data Indonesia. ¹⁰ This study was approved by the Ethics Committee of the Faculty of Public Health, Hasanuddin University, Number: 797/UN4.14.1/TP.02.02/2021.

The Evaluation Framework of GP

The evaluation framework is based on the Logic Evaluation Model modified from Sherman et al., as follows (Figures 1 and 2); ¹ total indicators: 22 established; fourteen (14) composite indicators become quality indicators with the village level unit of analysis, and 8 indicators with the individual caregiver analysis unit (qualitative sub-sample study). ¹² The qualitative study was selected in the village with the highest and lowest of reduction in stunting. This subsample study is to elaborate the implementation of the I Change Model as the reason for changes in stunting at the caregiver level. The I Change model was studied in a sub-sample of mothers who succeeded in preventing stunting in their children during the mentoring period, and also mothers who had not succeeded in preventing stunting in their children during the mentoring period.

Figure 1.

Gammarana evaluation logic model.

Figure 2.

Gammarana quality evaluation process design.

The activities evaluated and the data collection methods are named according to the nomenclature in GP as follows (Matrix 1, not shown). The location of this research are 60 villages (30 villages of intervention group and 30 villages of control group) in Enrekang District South Sulawesi Indonesia. Determination of The intervention villages group based on 20 criteria (prevalence of stunting, coverage of pregnant women with chronic energy deficiency, coverage of family planning, coverage for pregnant women, program feeding coverage, coverage of attendance at growth monitoring promotion (Posyandu), coverage of maternal visiting in health care, vitamin A, immunization, diarrhea, Iron Folic Acid for young women, proper drinking water sources, proper sanitation, health insurance, of family planning, parenting classes, early childhood education, food security and postpartum care coverage). All of the intervention villages group was similar based on 20 criteria. ¹³

Data analysis and instrument validity and reliability

Analysis of stunting changes using parametric or non-parametric statistical tests in accordance with the results of the data normality test. Data normality test with Kolmogorov-Smirnov. The distribution of the HAZ-score of height before and after compared, or the distribution of the percentages of stunting before and after. Efficacy is determined based on the value of significance. The percentage reduction in stunting is calculated from the difference in the percentage before and after. This was calculated for both village groups. In addition to the change in stunting indicator, this evaluation also measures project performance with 22 indicators (7 inputs, 6 processes and 9 outcomes) (Matrix 1, not shown). The data of I Change Model was analyzed by content analysis.

Valid and reliable instruments are based on Cronbach alfa and Test the validity of the I Change Model instrument, using the source triangulation method. ¹⁴

Expected impact of the study for public health

he hypothesis in this study is to test, if a number of underline stunting interventions are carried out with maximum conditions, the reduction in stunting will be able to reach >4% per year. The result of a systematic review by Hossain ¹⁵ is that if an intervention combines sensitive and specific interventions with a strong basis for political support, community involvement and program factors, it will be able to achieve a decline of 3% per year. The results of the systematic review of researchers for the 2015-2020 publication period specifically for the Randomized Control Trial study found a decrease in annual decline for the effective intervention group was 3.23±3.73 percentage points. If the intervention is effective, it can reach 6.58±1.99 percentage points and if it is not effective it will only reach 0.57±2.42 percentage points per year. Based on the results of the systematic review study above, in this study the researcher wants to test the hypothesis that the intervention of GP in Enrekang 2020 can reduce stunting >3%, if it is able to replicate the maximum requirements as found in the systematic review results by Hossain et al 2017. A number of requirements for the components of the intervention are micronutrient supplementation, feeding, prevention of infectious diseases, food security, promotion of growth monitoring and hygiene sanitation. If the above components are supported by political commitment, community involvement and adequate program aspects, it is estimated that stunting reduction can reach 19.38% during the 2018-2024 period and the proportion of stunting in 2024 is 11.42%.

This study really needs comparative data; therefore, in this protocol the researcher examines the similarity conditions at the beginning of the GP intervention. This condition is based on 13 indicators that have the potential to confound the conclusions of this study. Selection of comparison villages with similarity criteria in the input component (13 parameters; food security, ratio of food, poor family, food security index, permanent healthy latrine access, families have access to clean water facilities, healthcare delivery, basic immunization, exclusive breastfeeding, growth monitoring and National Health Insurance) in line with the determinant factor stunting in Indonesia. Based on the results of statistical analysis (not shown), it is known that the ratio of the poor is different in the two village groups (p=0.034). This difference is believed not to have effect on the study results because the ratio of the poor in the two groups is very small <0.06 or 6 people in every 100 households in the locus village and 5 people in 100 households in nonlocus villages. Furthermore, the distribution of interventions besides GP was also mapped in the two village groups, to ensure that each group had the same exposure factor to sensitive interventions. Based on the government documents in Enrekang District, it is known that there is only one intervention package that is directly given to chronic energy malnutrition for pregnant, outside the GP, while the other packages take the form of education except in Langda Village, which is the village of GP which innovates by providing food through the hearth of family. So, it can be explained that most of the micro and macro nutrient interventions originate from the GP Interventions package.

The implementation of infant feeding education, supplement of sprinkle (taburia), proten, moringa extract capsule, by Standard Operating Procedures (SOP) has been evaluated for 5 months. Data sources for Nutritionist Volunteer reports and investigations into program beneficiaries in a sub-sample. Evaluation in the village of GP was also carried out on supporting packages, namely Food Security, Sanitation Hygiene and Growth Monitoring. This component is a natural intervention or intervention that takes place simultaneously with GP in the role of comparison. This data is measured because it becomes a reference for the completeness of the GP package as required for at least 6 combinations of stunting interventions from the results of a systematic review.

Evaluation in non-GP villages as a comparison is a collection of data on food security, sanitation hygiene and growth monitoring promotion. The role of this variable is the control variable. Materials to ensure that this variable does not become a confounding variable in the analysis of GP quality on stunting reduction. Controlling this variable is by providing equal opportunities for both villages to receive the same package. The researcher tested the hypothesis that the concentration of locus villages was geographically similar, so that the comparison in this evaluation was a neighboring village so that the similarities in topography, geography and social conditions could be controlled in giving a real effect on GP interventions.

Conclusions

The content of Gammarana's evaluation has been made systematically and accountable, so it is important to gain important recognition from the publication of the study results.