An innovative method to evaluate community support for children: Using existing datasets

How effective is help on the doorstep? A longitudinal evaluation of community-based organisation support. 

Sherr L, Yakubovich AR, Skeen S, Cluver LD, Hensels IS, Macedo A, Tomlinson M. PLoS One. 2016 Mar 11;11(3):e0151305. doi: 10.1371/journal.pone.0151305. eCollection 2016.

Community-based responses have a lengthy history. The ravages of HIV on family functioning has included a widespread community response. Although much funding has been invested in front line community-based organisations (CBO), there was no equal investment in evaluations. This study was set up to compare children aged 9-13 years old, randomly sampled from two South African provinces, who had not received CBO support over time (YC) with a group of similarly aged children who were CBO attenders (CCC). YC baseline refusal rate was 2.5% and retention rate was 97%. CCC baseline refusal rate was 0.7% and retention rate was 86.5%. 1848 children were included—446 CBO attenders compared to 1402 9-13 year olds drawn from a random sample of high-HIV prevalence areas. Data were gathered at baseline and 12-15 months follow-up. Standardised measures recorded demographics, violence and abuse, mental health, social and educational factors. Multivariate regression analyses revealed that children attending CBOs had lower odds of experiencing weekly domestic conflict between adults in their home (OR 0.17; 95% CI 0.09, 0.32), domestic violence (OR 0.22; 95% CI 0.08, 0.62), or abuse (OR 0.11; 95% CI 0.05, 0.25) at follow-up compared to participants without CBO contact. CBO attenders had lower odds of suicidal ideation (OR 0.41; 95% CI 0.18, 0.91), fewer depressive symptoms (B = -0.40; 95% CI -0.62, -0.17), less perceived stigma (B = -0.37; 95% CI -0.57, -0.18), fewer peer problems (B = -1.08; 95% CI -1.29, -0.86) and fewer conduct problems (B = -0.77; 95% CI -0.95, -0.60) at follow-up. In addition, CBO contact was associated with more prosocial behaviours at follow-up (B = 1.40; 95% CI 1.13, 1.67). No associations were observed between CBO contact and parental praise or post-traumatic symptoms. These results suggest that CBO exposure is associated with behavioural and mental health benefits for children over time. More severe psychopathology was not affected by attendance and may need more specialised input.

Abstract [1]  Full-text [free] access  [2]

Editor’s notes: This study is novel in both its research question and its methodology. The study aims to assess whether receipt of support from community-based organisations (CBOs) impacts the mental and social well-being of children in high HIV prevalence areas. The CBOs studied include many different organisations with diverse services, giving the study the benefit of assessing the overall impact of a combination of small, motivated groups. This helps lend credibility to CBOs and to convince policymakers and funders to support small-scale CBOs.

In terms of methodology, the study utilises two longitudinal datasets from southern Africa to explore the study aims. One survey is from a study of children affected by HIV served by CBOs, while the other is from a study of children affect by HIV without CBO support. There are some limitations to using two different studies, most especially unclear comparability and, in this case, lack of control data to adjust for possible differences, for example on socio-economic status or how HIV specifically affected the child. Despite these, this paper has striking results, and is an innovative effort to improve our understanding of the impact of CBOs on children’s well-being and should spur further creativity in impact evaluation methods.

Africa [9]
South Africa [10]
  • [11]