promising quality evaluation: inception report - Unicef

Promising Quality An Independent Assessment Framework: Making of Specialised Child sure we deliver Protection Service Provision excellent services for in Zimbabwe children FINAL REPORT August, 2013

FINAL REPORT: Promising Quality Assessment August, 2013

ACKOWLEDGEMENTS As noted in the original Promising Quality Framework (2012), through this assessment, Promising Quality continues on its collaborative journey. This assessment is the culmination of a partnership between the Ministry of Labour and Social Services and UNICEF Zimbabwe Child Protection Section in which both organisations have agreed to be jointly accountable for the quality of services delivered to children within the framework of the National Action Plan for Orphans and Vulnerable Children phase II 2011-2015 (NAP II). The assessment was made possible through this partnership, and in particular through the facilitation of the Ministry of Labour and social Services, Department of Social Services. Complementary Government Ministries also responsible for the protection of children were involved, including Ministry of Health and Child Welfare and Ministry of Justice and Legal Affairs. The authors also wish to acknowledge the significant contribution of NGO partners both those funded within the scope of the Child Protection Fund (CPF) in support of NAP II and those funded by UNICEF through complementary mechanisms. They also wish to acknowledge the contribution of UNICEF Child Protection Specialist in the development of the snapshot Value for Money framework.

1


CONTENTS Acknowledgements Contents 1.0 2.0 3.0 4.0 5.0 6.0 7.0

SUMMARY INTRODUCTION CONTEXT METHODS FINDINGS RECOMMENDATIONS NGO PROGRAM REPORTS Acronyms

ANNEX I

Terms of Reference

ANNEX II

Meeting Records/ Work Plan / List of Interviewees

ANNEX III

Case Studies

ANNEX IV

Feedback Workshop Agenda

ANNEX V

Data Table

2

FINAL REPORT: Promising Quality Assessment August, 2013 1.0

SUMMARY

Are NGOs using the Promising Quality Framework?

Are NGOs meetings the targets?

Reached Children under 181

93% of CPF for NAP II funded NGO partners 60 % of all NGO partners (including 2 partners who are supported by UNICEF through complementary funding mechanisms) Indicative results for Children Reached with Services (September to July, 2013) are below)

Reached Adults2 18 and +

Quality Services Total Reach

69,118

9,391

78,509

% Total Quality3 Beneficiaries Receiving a Quality Service4 76 % 59,892

ON TRACK

SURPASSED TARGETS (based on available data)

The monitoring and evaluation of Child Protection services in Zimbabwe is experiencing a fundamental shift in how everyday accountability to children is understood and practiced. Services now routinely consider not only the reach (number of children who can access services if they need them) but also the quality of these services. Since 2011, Department of Social Services and UNICEF have engaged with Child Protection actors to standardise measurements of quality that inform practitioners, children, families and others from a child’s perspective, about how the services being provided are improving the lives of children. This sector-wide paradigm shift in program monitoring and evaluation is an ongoing process. It is partially being achieved through the introduction of the Promising Quality (PQ) Framework, embedded within a wider M&E system which includes mechanisms for an on-going dialogue for continuous improvement. Currently 93 per cent of Child Protection partners funded through the Child Protection Fund (CPF) in support of the Zimbabwe National Plan for Orphans and Vulnerable Children 2011-2015 (NAP II) and 60 per cent of all Child Protection partners, including those funded by UNICEF through complementary mechanisms are implementing the PQ Framework, with encouraging, all be they early, results. Child Protection partners’ self-assessed progress towards targets for both Reach and Quality were reviewed in the assessment. Although data collection processes are still improving, and there were revisions to indicators in the first year, overall positive trends are emerging. This assessment represents a tipping point in the sector in Zimbabwe. As the M&E quality instruments are further embedded into national frameworks, Zimbabwe is realising goals set in key best practice policy and guidance in children’s related social work as well as sector-wide program goals.5 1

Between the period September 2012 – July 2013. Where the service is provided to the adult with an anticipated positive outcome for the child e.g. legal assistance for child maintenance, Family Clubs, 3 Mean score of quality in case files and coordination + %improved PSS well-being + %NGO keeping its promises + %most critical intervention fully met as per Promising Quality Reporting. Where an NGO is not yet reporting against all indicators the MCI indictor was used. If there was no MCI indicator reported against, a measure was not included in the data. 4 Total Reach x % Quality 5 Key Policy guidance includes the Children’s Act, the Orphan Care Policy, Minimum Quality Standards for OVC Programming, National Residential Care Standards, National Child Protection and Participation Guidelines, National Psycho-social Support Guidelines for Children Living with HIV and AIDS, Protocol on the Multi-Sectoral Management of Sexual Abuse and Violence in Zimbabwe 2

3

FINAL REPORT: Promising Quality Assessment August, 2013 This assessment found that for most agencies, the PQ Framework has helped to redefine successes in Child Protection work. Most organisations and government M&E systems had been set up to measure inputs and outputs in real time (e.g. number of trainings held). This work was labeled ‘process’ by stakeholders in interviews. Outcomes were measured usually at project closure. The real time ‘check in’ each quarter that the PQ Framework provides to find out about children’s wellbeing (76 % quality in self-reported improvement in well-being), about the coordination of their case (72% quality in case files) and to find out if program participants believe the organisation has kept its promises to them (88 % agencies using PQ kept their promises), has challenged the sector to be constantly accountable to the most important person in the ‘intervention equation’ – the child (and/or family). Targets for reach were overwhelmingly surpassed – ‘trends in reach’ over the longer term should stabilise as interventions BOX 1: NGO FEEDBACK become more targeted (to assist the most vulnerable children) and programs move to scale. This approach to ‘success’ has changed the reporting narrative. It is more aligned, and indeed at the forefront of global trends in ‘downward accountability,’6 client-focused service provision, in human rights based programming, and is providing a platform for real-time, child/ beneficiary informed evidenced based planning. Information is critical in Child Protection work where children’s development windows cannot wait the 3-5 formative childhood years (the usual outcome evaluation time-frame) to find out if the service is meeting their needs or not. The introduction of the PQ Framework, and the practice orientated M&E tools, is already contributing to improvements in quality of service provision. The PQ Framework is resulting in more child friendly engagement platforms, more incentive to ‘act on’ results in real time, more flexibility in programming, improved motivation towards inter-agency and government collaborations and more client focused practice.

“The Promising Quality Framework reminds us, in our everyday work, that children have many aspects to their lives. If we don’t ask children about them, we won’t know about them and so Promising Quality gives us a fuller picture”. “We have always cared about quality… and now we have a way of demonstrating that to the children, the parents and to donors”.

“It was so good to talk you about Promising Quality reporting. At the beginning it was confusing but when we just stopped to think about it we realised it is a really rewarding part of our job - to look and see the real difference we are somehow making in children’s lives.”

The introduction of a routine measure of quality offers a more nuanced understanding of service Value for Money. Although complex to measure with nascent data sets, most programs are demonstrating value for money, by finding ways to improve quality of services, reach and the cost-efficiencies simultaneously. The quality indicators can aid the routine mapping of trends in effective and efficient program delivery and inform strategic planning that would allow programs to build to national scale. The introduction of the M&E instruments, in all instances either introduced or enhanced child participation in programming and their right to have their voices heard (UNCRC Article 12) - a key challenge in social work and a key government of Zimbabwe commitment.7 The assessment noted that some organisations find it challenging to apply the tools with some groups of children, especially child survivors of violence, abuse and exploitation; children who are under 12; and / or children who have a disability, but this recognition came with a commitment to continuous

6

See for example Save the Children Fund UK, 2010 Accountability Matters in Emergencies: Listen to children and responding to their feedback during Save the Children’s Humanitarian response on Myanmar (Burma) 7 See Every Child, January, 2012. Making social work work: improving social work for vulnerable families and children without parental care around the world. Literature review, for overview of challenges and debates; See 2010 MoLSS National Child Participation and Protection Guidelines, for core principles in participation in child protection interventions in Zimbabwe & Zimbabwe National Action Plan for Orphans and Vulnerable Children phase II 2010-2015

4

FINAL REPORT: Promising Quality Assessment August, 2013 improvement to ensure inclusion of and response to the most vulnerable children and has resulted in the introduction of innovative approaches to participation (see section 5.4 below). For the full potential of the PQ framework to be realised, on-going refinement of the M&E instruments is required and is dependent on the effective roll out of the pending Zimbabwean Case Management Framework and resources, in which the tools are embedded. Although challenged by scarcity of resources for Child Protection programming in Zimbabwe, the bolstering of the Case Management system will help to overcome some of the barriers identified by partners to achieving results across all quality indicators. In particular it will assist in ensuring that inter-agency referral processes work, that the most critical interventions remain defined, that the necessary management and support is in place through the regulation and oversight of Department of Social Services – the government agency mandated for Child Protection and as such the ‘glue’ holding together the Child Protection system. For most Child Protection practitioners, the PQ tools have rekindled excitement and a drive toward results for children; however this is tempered by some organisational level anxiety linked to reporting and accountabilities. The shift away from reporting only on reach in Child Protection is an important one. This is especially the case when considered in the context where less than excellent service quality for vulnerable children can do more harm than good. Thus confidence in the quality reporting approach should be nurtured, communicated and celebrated, so the social worker, counsellor, rehabilitation officer, nurse or community development worker can trust that the ‘rush to numbers’ at ‘all costs’ approach, will not return. This is an important reflection for donors to ensure that an equitable approach is adopted which targets difficult to reach child populations - especially child survivors of violence and abuse and children with disabilities. A focus on improvement in quality over time will also assist some partners in gaining confidence in the quality focused approach. The PQ framework has made great strides ahead in encouraging an indigenous model of social work that is user friendly, encourages linkages between services and enhances child focused service provision. The PQ Framework, within a Zimbabwean case management model, has the potential to ensure services are specific to needs, flexible and informed by children, including girls, children with disabilities and children from rural areas, who are disproportionately affected by violence and abuse in Zimbabwe, who face barriers or are conventionally discouraged in freely expressing their opinions and views. 2.0

INTRODUCTION

This report presents the findings of the “Independent Assessment of Specialised Child Protection Service Provision in Zimbabwe”.8 Fifteen Child Protection service delivery partners who have Program Cooperation Agreements (PCAs) with UNICEF and who contribute to the outcomes of the Government of Zimbabwe and UNICEF rolling work plan 2011 – 2015, were assessed during the period 17 June – 31 July 2013 in Harare. The assessment was conducted by a team consisting of one international and one Zimbabwean consultant contracted by UNICEF and support and supervision was provided by a delegated representative of Department of Social Services. The purpose of the assessment was to consider the overall quality of services and the difference these services are making in children’s lives.9 Linked to this purpose were three objectives: a) Review and provide analysis on the implementation of the Promising Quality Framework (both reach and quality of service); a) Undertake a broader evaluation of service delivery ; and b) Make recommendations for improving the implementation of the Promising Quality Framework and the broader implementation of best-practice service delivery. 8

The detailed Terms of Reference is included at Annex 1 UNICEF Terms of Reference Consultancy Reference Zim/2013:15.

9

5


The assessment involved interviews (face to face and telephone), focus groups and meetings with key stakeholders both in Harare and Bulawayo (Department of Social Services personnel, representatives of Ministry of Health and Child Welfare, Ministry of Justice and other government personnel, NGO representatives including Directors, programming and M&E personnel, UNICEF ); a desk review of DSS, UNICEF and NGO program documents, PCAs , and regular field monitoring reports; collection of case studies from M&E focal points; a participatory feedback workshop with key stakeholders and report writing. Children’s thoughts and views are routinely included as the very nature of the PQ tools requires soliciting children’s inputs. A work plan (Annex II) and list of stakeholder interviews (Annex III) are provided. The report is divided into sections as follows: 1. 2. 3. 4. 5. 6. 7. 8.

3.0

Summary Introduction Context Research methods Findings Recommendations NGO Reports Annexes

CONTEXT

3.1 MEASURING QUALITY IN ZIMBABWE During implementation of the Program of Support (PoS) to NAP I (2004 - 2010) the Government of Zimbabwe and UNICEF supported over 180 partners and sub-grantees to provide multi-sectoral supports to orphans and other vulnerable children. The final Outcome Assessment of the PoS concluded that whilst over 400,000 children received some benefit from the program, the quality of the intervention needed improvement.10 Today, within the framework of the Zimbabwe NAP for OVC, the Department of Social Services and UNICEF, together with other government agencies are currently working with fifteen NGOs to provide Child Protection services including social welfare and access to justice. Eight (8) of these partnerships are funded through the multi-donor aligned child sensitive social protection funding mechanism, the Child Protection Fund (CPF).The remaining seven (7) partnerships are funded by UNICEF using complementary donor funding as well as its own resources. These partnerships are supported within a broader national, government strategy to identify and support children and their families affected by violence, exploitation and abuse with a comprehensive package of high quality interventions11.

3.1.1 A Co-coordinated System of Child Protection Actors The strategic view adopted by the CPF Child Protection Pillar is to ensure that services are aligning in a coordinated system and to promote quality assurance standards across a network of support for children, thus programs were developed through thematic lenses with technical guidance and support. The current 15 interventions, aligned to 10 11

Jimat Development Consultants 2010, Program of Support for the NAP for OVC Outcome Assessment. NAP II; page VI

6

FINAL REPORT: Promising Quality Assessment August, 2013 thematic groups are presented in Table 1. This strategy has developed a coordinated system of protection actors and has enhanced opportunities for quality evidence based programming. Specialised interventions that are not currently part of the CPF in support of NAPII arrangement have been included in Table 1 to demonstrate strategic alignment/potential alignment within the national framework.

7


Table 1. NGOs Participating in the Assessment with Strategic Alignment to overarching themes NAP II Thematic Area

Program Summary

National Case Management System Development and Practice Specialist Service Provision for Children and Women Affected by Abuse

National Case Management System policy development and case management service delivery

Specialist Service Provision for Children to be Confident and to Keep Safe

Specialist Service Provision for Children with Disabilities

Specialist Service Provision for Children to Access Justice

Specialist Service Provision for Separated Children including Children on the Move Specialist Service Provision for Social Mobilisation

Hospital clinic based provision of medical care and support, psychosocial services and assisted referrals to survivors of sexual violence Free-call hotline and network of drop in centres for children, offering counselling, advice, information and referral. Also supports the Department of Social Services with logistical and technical support to statutory case management. Emergency cash assistance for survivors of gender-based violence (GBV) Child participation and community protection programs Supports HIV positive children, other vulnerable children and their families with knowledge, skills and confidence to keep safe, to cope with the stigma and discrimination associated with sexual violence and abuse or HIV+ status Education program about child abuse and GBV with boys scouts and girls guides Small grants to community groups and individuals supporting children with a disability Support to rehabilitation structures for children with a disability Access to justice support for children with a disability Legal assistance and case management services to children in conflict with the law. Specialised services for family reunification for children living outside the family environment

Facilitation of Family Clubs

Partner Funding Organisation World Education Inc. CPF in support of NAPII

Family Support Trust

CPF in support of NAPII

Childline


Musasa

Other

Africa Community Publishing and Development Trust Africaid

Other

Bekezela

Other

Mercy Corps

Other

J.F. Kapnek Trust

CPF in support of NAPII Other

Leonard Cheshire Zimbabwe Trust Family Impact CATCH Legal Resources Foundation Child Protection Society

Concern Worldwide


Other Other CPF in support of NAPII CPF in support of NAPII


8

FINAL REPORT: Promising Quality Assessment August, 2013 3.1.2 The Promising Quality Framework A new framework for measuring quality in service delivery ‘Promising Quality’ (PQ) moves away from process measurement to report against indicators of reach and quality. This allows service providers to determine if the children they come into contact with get what they needed, where and when they needed it. The PQ Framework supports organisations to establish a baseline indicator against which they measure progress during the lifetime of their intervention, reporting quarterly. This framework has been adopted by the Department of Social Services and endorsed by the Working Party of Officials12 through the M&E Sub-committee.

BOX 2: PROMISING QUALITY FRAMEWORK The PQ Framework is a set of standardised measures for monitoring program quality. The M&E system employs four specific instruments which combined aim to tell us more about the experience of children who come into contact with services. Quality was defined by the sector in Zimbabwe as ‘ reaching a condition of wellbeing, either an individual or for an organisation’ Are we delivering what children need , where and when they need it?

How can we check? (monitoring tools/instruments)

Are you feeling good?

Ask the Expert (wellbeing measure)

Did we help you get the other kinds of support you need? Did we keep our promises to you?

Most Critical Interventions Quality Case File check list (coordination measure)

Following a number of key evaluations of Promise Cards various programs supporting the NAP II for (accountability 13 measure) OVC , an initial Quality Evaluation was conducted in 2011. This evaluation provided an For full detail on the conceptual PQ, see pages 13 -22 Promising Quality: independent overview of the quality of service Making sure that we deliver excellent services for children: Monitoring and delivery by various implementing partners and evaluating quality in child protection. UNICEF Zimbabwe, 2012 the initial design of a ‘quality’ framework (the “PQ framework”), with accompanying standards, indicators and tools.14Children were actively involved in the design of the PQ framework, as were civil society partners to ensure relevance to everyday programs. Special efforts were made to ensure a wide range of children were consulted in the development of this framework, especially those most vulnerable including children from rural areas, children living with disabilities, children affected by HIV and child survivors of abuse and violence. Over the course of 2012, a number of NGOs developed baselines and commenced the routine monitoring of quality of service delivery with guidance from DSS and UNICEF using this framework. The PQ framework includes four specific M&E instruments that emphasis quality that enable programs to track the extent to which they can answer queries about children’s wellbeing. These instruments are: 1. Most Critical Interventions: an integrated set of clearly defined quantitative indicators of quality for each program and which support the achievement of goals and objectives. The indicator is percentage of cases where most critical interventions were fully met 2. Promise Cards: designed to help children and their caregivers understand what they can expect and as a means to collect feedback on services provided. The indicator is percentage of children reporting that NGOs kept their promises.

12

The Working Party of Officials is the Cabinet mandated multi-sector national child protection committee to provide oversight to NAP II implementation A number of key evaluations have undertaken in recent years, including an outcome and impact assessment of the Program of Support in 2010. Also in 2010 and 2011, reviews of retrospective quantitative data were undertaken by Save the Children, Family Support Trust and Childline. A comprehensive child-focused monitoring tool – My Life Now – was also developed in 2010 and used to assess the outcomes for children of Program of Support interventions. 14 See UNICEF Zimbabwe 2012, Promising Quality: Making sure that we deliver excellent series for children; monitoring and evaluating quality in child protection document. 13

9

FINAL REPORT: Promising Quality Assessment August, 2013 3. Ask the Expert: A Psychosocial Support Tool centres on the concept of checking with children (or where this is not possible their caregivers) how they are feeing and doing. It is a questionnaire covering 15 topics spanning interpersonal, intrapersonal, safety and social involvement. The indicator is percentage of children with have improved psychosocial wellbeing 4. Quality Case File Checklist is a list of twenty case file and coordination standards which children’s case documents can be reviewed and scored. The indicator is percentage of quality in case files and coordination. (see pages 15 – 19 PQ 2012 Document for full description)

This assessment considers progress against these baselines, and the lessons learned during the first 12 months of implementation. For a detailed overview of the background to the ‘promising quality’ work, the situation in Zimbabwe and the baseline for some of the NGOs participating in the review, see the report from the first evaluation, 2012.15 The PQ Framework report (2012) noted that the, ‘ strategic shift from measuring reach to measuring quality had not been well understood and that although providers had a clear and comprehensive range of indicators that are reflective of the diversity of services provided to beneficiaries, they did not included quality’. One year on, this assessment found that 20 per cent of NGOs had “limited knowledge of PQ”, meaning staff had a cursory understanding of the concept but were not implementing it. Twenty percent per cent had an “awareness of PQ”, which indicates there was an understanding of the framework and that they may have begun developing indicators or are developing baselines; 60 per cent were “implementing PQ”, meaning that the NGO was routinely reporting against at least one quality measure indicator (a case file check list, percentage of Most Critical Indicators (MCIs) met, or Promise Cards, or Wellbeing (Ask The Expert). Figure 1: PQ Framework Knowledge in Participating NGOs July, 2013 N=1516

There are varying degrees of development, efficiency and confidence amongst the 60 per cent of NGOs implementing the PQ Framework. Variations can be attributed to the existing M&E capacity of the organisation, when and how PQ was introduced and the support the NGO received from DSS and UNICEF. Although, some partners may relate the success to the type of intervention, the assessment found this was unlikely and was more related to a need for more clearly defined indicators and capacity in M&E. 15

Ibid. Limited Knowledge: Musasa, ACPDT, Bekezela. Awareness of PQ: Mercy Corps, Concern Worldwide, Family Impact. Implementing PQ: LCZT, CATCH, FST, Childline, LRF, Africaid, WEI, J.F.Kapnek Trust, CPS 16

10


Figure 2 provides an overview of the support measures, also noted below, which NGOs reported have contributed to successful implementation if the PQ Framework. These measures are also partially outlined in the M&E framework for the Child Protection Fund. 1) Participatory development of the framework 2) Training provided by UNICEF 3) Developing a simplified reporting template (SRT) with standardised + MCI indicators 4) Organisation protocols for routine data collection 5) Quarterly reporting (submission of the SRT to UNICEF) 6) Quarterly meetings 7) Quarterly joint monitoring trip (DSS, UNICEF, NGO) 8) Quarterly information sharing meeting (currently for CPF funded partners) Figure 2: PQ Framework Process/Support Measures in Participating NGOs, July 2013 N = 15

BOX 3: CHILDREN’S VIEWS In line with Child Participation best practice and in consideration of the best interests of the child, the timeframes, scope of the work, child protection sensitivities and intended users of the lessons learnt through the assessment, the consultancy team did not directly engage with children. However, this assessment reviewed the existing data collected from children through the PQ Framework to provide insight into individual case experiences. The Promising Quality Framework and the M&E tools operate within a conceptual framework with child participation at the core; program participant/beneficiary feedback is deeply embedded in all the elements of the data that were reviewed in the study. Children reported their views on the effectiveness of an NGO’s performance through the use of the specialised M&E instruments. In addition case studies were relayed from partner organisations including from boys and girls in rural areas. These case stories are presented at Annex iii, for those NGOs using PQ Framework

4.0 METHODS 4.1 RESEARCH APPROACH The assessment asked if agencies had policies and procedures in place to assess quality related to the PQ Framework, and if targets were being met. It did not consider the validity of the PQ Framework or the M&E instruments, or accuracies in the administrative protocols used to collect data. The technical approach is summarised in Table 2. The determination of quality was guided by:  The best interests of the child as the paramount consideration  Adherence to international child rights standards, norms, rules and best practice; and the degree to which the service enables children to realize their rights and address child rights violations  Adherence to national frameworks and policies, where these do not conflict with international standards, norms, rules and best practice  The degree of reported satisfaction of child clients and their families  Value for money 11


Progress towards targets was reviewed, where possible, against baselines over a 12 month implementation period (depending on the length of time PQ Framework was in use). It also explored the strategies employed to deliver quality services for children, within an agreed general determination of quality. The primary research questions were related to: 1. REACH + QUALITY: Are NGOs meetings their targets; to what extent; why or why not have targets been met? 2. M&E INSTRUMENTS: Are NGOs using the Promising Quality Framework; if so how; if not why not? 3. QUALITY: What is the overall assessment of service delivery quality? The assessment was largely qualitative, but borrows from sequential mixed methods design. Figure 3 provides an overview of this approach (post inception phase).17 Figure 3: Methodology Outline

Table 2: Summary of Technical Approach Methodology Matrix Purpose: to assess the overall quality of services provided by DSS & UNICEF partner child protection organisations and the difference these services are making in children’s lives in Zimbabwe Objectives (as per ToR) Key Research Questions Data Collection tools Sources of data Review and provide  Are NGOs meetings the targets?  Desk Review Analysis (of  Recent Evaluations analysis on the Is so why? Or If not why not? In data collected by NGOs)  NGO quarterly reports implementation of the terms of both reach and quality  Key Informant interviews Promising Quality  Are NGOs using the Promising (structured dialogue – based Framework (both reach Quality Framework? If so how, on NGO Evaluation Report and quality of service) and to what extent? If not why Template and Guiding not? Question Annex ii)  Are NGOs achieving their ‘most critical intervention (MCI)’?  Are children reporting that NGOs are keeping their promises?  Are children reporting improved well-being, prior to case closure?  Are case files of 100% quality? 17

Figure 2 is adapted from Figure 4, Page 54 Evaluation Working Papers How to Design and Manage Equity Focused Evaluations available: http://mymande.org/sites/default/files/EWP5_Equity_focused_evaluations.pdf

12

FINAL REPORT: Promising Quality Assessment August, 2013 Undertake a broader evaluation of service delivery

    

Make recommendations for improving the implementation of the PQF and the broader implementation of best-practice service delivery.



What is the overall assessment of service delivery quality? Best interest of the child? Adherence to child rights standards, norms, rules and best practice? Adherence to relevant national frameworks and policies? Value for money?

  

What strategies need to be put in place to improve outcomes and quality?



As above plus: Key Informant Interviews Feedback/ Validation Workshop (Feedback Focus Group)

  



As above



As above plus: PCAs/Budgets M&E Officer, Head of Programs + Director (outcome of discussions based on structured dialogues) Internal NGO reports (data from regional field sites, case files/ filing systems, reports from DSSO/PSSO etc) As above

The assessment involved the following activities. a) Inception: the formation of the consultancy team consisting of one international and one Zimbabwean consultant contracted by UNICEF with support and supervision provided by a delegated representative of Department of Social Services; development of a work plan and agreement on methodology; presentation to and feedback from the UNICEF internal Research and Evaluation Group (REG); finalisation of the Inception Report by UNICEF and DSS. b) Desk Review: including analysis of NGO Simplified Reporting Templates (SRT) which collate quarterly statistics from the M&E instruments; review of relevant program documents including national and regional frameworks for child protection service delivery and monitoring and evaluation in Child Protection. c)Data Collection: i) Key Informant Interviews and Focus Group Discussions (FGD) - 74 key informants (Annex III), including an FGD of 20 participants were interviewed using in-depth ‘structured dialogues’ around key research themes, with a view to explaining trends in the SRTs; Where NGOs did not have a SRT, other quarterly program reports were reviewed. Key informants were selected across the organisational hierarchy and were interviewed separately. A breakdown of respondents is provided in Table 3. Interviewees also included representatives of Ministry of Labour and Social Services (MoLSS), district and national DSS, ministries of Health and Child Welfare, Women’s Affairs Gender and Community Development, Justice and Legal Affairs and UNICEF; ii) Case Studies - The assessment capitalised on the extensive reporting from children routinely collected by NGO partners using the Ask the Expert, Promise Card feedback, Case Planning sessions and other child feedback forums to gain an understanding of children’s experiences of the framework. A relevant case study per service/NGO that expresses a child’s response to, for example, the Promise Card or ‘Ask the Expert Survey’ and what this response represents or tells us about service delivery quality was collected electronically. Fifteen case studies were provided, including stories from boys and girls in rural settings. Case studies from NGOs implementing the PQ framework are included at Annex iii. d) Data Analysis and Synthesis e) Partners Feedback Workshop: A stakeholder workshop was held on 22 July with 42 attendees from partners, government and UNICEF to present initial findings and draft recommendations and to obtain feedback through participative activities. The activities provided a platform to explore and better understand any barriers and constraints in implementing the tools as well as highlighting any critical success factors in the service design, implementation, monitoring and evaluation. See Annex IV for agenda. f) Final Analysis and Report Writing

13

FINAL REPORT: Promising Quality Assessment August, 2013 Table 3. Breakdown of Interview/ Focus Group Respondents (NGOs)18 Designation

Number19

Head of Agency M&E Focal Points / Project Coordinator Users of the M&E instrument/s (Includes DSS and NGO) Study Participants whose regular duty station is not Harare Total interviewees

14 21 49 20 in FGD 32 (43% of respondents) Includes focus group participants 74 (includes 20 focus group participants) 42 (NGO, UNICEF, DSS, MoJLA, WAGCD, MoHCW and Local Universities)

Partners Feedback Workshop Attendees

4.2 LIMITATIONS The assessment coincided with announcement of the date for the Zimbabwean National Elections and the campaign period. As a result and at the request of MoLSS, field work remained in Harare. However, views from non-Harare based practitioners were included via telephone interviews with stakeholders in Bulawayo and with Provincial and District-based staff meetings held in Harare. Provincial or non-Harare based staff interviews made up 43 per cent of the interview cohort including focus groups discussions with community case workers.20 Also, at the Ministry’s request, almost all interviews were attended by the delegated representative of DSS as an observer and technical resource aid. Although not formally part of the interview team, the presence of a senior member of DSS may have influenced discussions around some issues. However, the initial perception that the group of three (international consultant, local consultant and DSS staff) were from UNICEF, was also noted. In introductions, the group made every effort to create an environment of shared learning and independence. This effort and arrangement also contributed to a learning dialogue between researchers and practitioners.

18

Some Respondents fall into more than one group. Some respondents fall into more than one category. 20 If these focus groups are not included, the total percentage is around 15%, which still allowed for issues of rural/urban divide to be discussed. 19

14

FINAL REPORT: Promising Quality Assessment August, 2013 5. FINDINGS 5.1 PROGRESS TOWARDS GOALS Finding: Sector-wide targets are being met Table 4 provides indicative results, against ‘yard stick’ targets extrapolated from the CPF logical framework, and nonCFP partner program documents. Individual program level progress is outlined in each of the NGO reports provided in section 7. Table 4: Snapshot of all agency progress against planned results. Progress statement

CPF in support of NAP II outcome



Reach Number of children reached with a service21

   

YES!

on track

on track

on track

on track

Annual Target CPF September 2013 milestone Baseline: 0 Target: 19,250

Wellbeing Self-reported wellbeing of girls/boys22

Baseline: 0 Target: 50% 23

Coordination % of case files meeting minimum quality standards24 M&E Systems A system to measure quality in place 25

Baseline: 0 Target: 50%

Quality Overall The Child Protection services being received by child survivors are of quality

No global target set

Baseline: 0 Target: 70%

Summary of Progress at July 2013 (Approx. 10 months of year 2 implementation) 69,118 (all partners – majority of beneficiaries reached by CPF partner programmes) 60,453 (all partners) beneficiaries reached with service deemed relevant and of quality (77%) 76 % mean of improvement in selfreported wellbeing (in those programs using the Ask the Expert Tool) 72% mean of case file check list scores (in those programs using the Tool) 93% of CPF partners are implementing 60% of all UNICEF partners are using PQ. Most NGO programs demonstrate trends of improvement 53 (8) % of NGOs have Value for Money Scale that includes quality 77% delivery on Most Critical Intervention

5.1.1 Progress on Reach Targets for children reached with a service and information only were mainly reached, or surpassed, with some exceptions. At July, 2013 the 15 NGO interventions have reached approximately 78,509 beneficiaries with a service over the past year, and approximately 58,892 with a quality service (76% of total beneficiaries reached with MCI26). The real figure is likely lower, when considering data collection and some refinements needed to the M&E system, 21

CPF for NAPII Revised LogFrame, 2013: Output Indicator 2.1 CPF for NAPII Revised LogFrame, 2013: Outcome Indicator 3 23 CPF for NAP II Revised LogFrame 2013 has this indicator at June, 2013 24 CPF for NAPII Revised LogFrame, 2013: Output Indicator 2.3 25 CPF for NAPII Revised LogFrame, 2013: Based on Output Indicator 4.1 - # &% of implementing partners a) reporting against quality reporting matrix and b) on time. 26 This is approximate and involves applying a general value measure for all organisations, based on some organisations ratings. 22

15

FINAL REPORT: Promising Quality Assessment August, 2013 but still in excess of the target of 19,250. These refinements include ensuring only children are counted (some indicators are yet to link children reached through caregivers), MCIs refined (which will narrow the scope of some ‘services’ – though this process is on track with most NGOs) and finalizing interagency reporting mechanisms (for example, from sharing data across ministries). Over 450,000 children were reached with information only. This includes children who received IEC materials or participated in an awareness program, but did not receive the agencies’ MCI service. The real figure is likely higher as it does not include children reached through some community mobilization programs, such as Africa Community Publishing and Development Trust Publications, who are yet to develop indicators. Nor, does it include beneficiaries of IEC materials, such as those produced in diversion and prevention awareness raising programs through Family Impact’s support to the Bulawayo Pre-Trial Diversion Sub-Committee. Targets and data collection methods and indicators to capture these groups are still being refined. 23,000 children were reached with referral service only27. This includes children who approached an NGO but was assessed to require a different service; and was subsequently referred on. Of all reach targets, this was the most likely not to have been met at an individual agency level. NGOs were not meeting set targets for referrals only mainly due to lack of reporting (many NGOs were yet to put in place a process for recording when someone ‘enters the wrong door’). Many agencies saw an increase in their service reach. This trend can be attributed to expansion strategies, including from pilot programs, better record keeping, and in some cases, more effective program strategies. The main barriers NGOs identified were time to build the right relationships, time to understand and develop adequate reporting systems, however, it is also important to note that all NGOs had identified strategies to increase reach moving forward. 5.1.2 Progress on Quality Measure of Quality Are NGOs achieving their ‘most critical intervention (MCI)’?

Results and Reflections The mean result for NGOs meeting % of MCIs was 77 % (N= 9)

Are children reporting that NGOS are keeping their promises?

The mean result for NGOs keeping their promises to children, for those NGOs using Promise Cards. was 88% (N= 8)

The majority of NGOs felt they had selected the right MCI, that the MCI was linked to the Promise Card and could show a direct outcome for children. NGOs that had difficulty in implementing the PQ framework had initially selected MCIs that related to more ‘upstream’ interventions such as formation of family groups (that needed to be linked to benefit to the child) or development of policy; rather than related directly to an outcome for children. There was a tendency among some NGOs to respond beyond the scope of their MCI. Although this ‘mission creep’ is a symptom of the gaps on the case management system, advocacy within the sector should be pursued. MCIs were helping to define the role each agency plays in the system of support for children. This is an important shift that is in line with the NAP II, which seeks to move away from “OVC packages” and towards more tailored, individualized and specialized services for vulnerable children. Better information sharing between partners on their respective MCIs will help improve referrals.

27

This figure does not include revised referral data for WEI’s case management and practice programme – where definitions of referral and service are still being revised. If the revised figure is included (see section 7.2.10 WEI NGO Report), the total for reached with a referral service is over 40,000.

16

FINAL REPORT: Promising Quality Assessment August, 2013 All organisations appreciated the Promise Cards and found them useful as a tool to enhance accountability, communicate service provision parameters and encourage feedback. The process of developing promises enhanced organizational planning, team building and mobilized staff around common goals. There remains some work in developing feedback mechanisms that are consistent, streamlined and linked to the case file, where possible – and which consider existing feedback tools already in use in NGOs and other ministries (such as MoHCW and being developed in the DSS Case Management Guidelines). The challenge remains to have organizational platforms and sector wide forums to act on feedback received systematically. The use of Promise Cards is likely to increase significantly, NGOs reported delays in printing the cards from UNICEF, and two NGOs reported delays in negotiating the content with DSS – both have affected results and ability to implement the PQ framework according to agreed protocols. The major barrier to reaching 100% in promises identified by partners was referral pathways. “The Promise cards are the best. They remind me why I am here, and they remind the child as well – eh, this is what we can do, what do you think? And if you explain it properly, they will tell you!” “We will do everything in our control, but we can’t promise for what someone else will do. Maybe we should get more weighting on what is in our control. It’s the referral partners letting us down” NGO Respondents Are children reporting improved well-being, prior to case closure?

The mean result for children’s self-reported wellbeing, for those NGOs using Ask The Expert was 76%. (N= 9) This result was above target, and although the process to reach the result has resulted in considerable progress towards developing child participation platforms, it should be read with some caution. This caution relates both to the sample size (NGOs were limited in administering this tool), to issues arising in the tools validation and to the NGOs being unclear that this is one indicator of wellbeing used for monitoring, but should not replace other routine measures of children’s health and wellbeing also being investigated through programming. Notwithstanding some of the issues in administration, NGO practitioners demonstrated in this assessment that the Ask The Expert tool provided a platform for improved child participation, was directly informing children’s case plans and encouraged a more rights centred approach to children’s service delivery. “Now the lawyers see the children as a whole person, not just a case. This means we pull in other services to respond to all their needs”. “Ask The Expert is not appropriate for children under 12, or children with intellectual disabilities”. NGO Respondents

Are case files of 100% quality?

The mean result for case files meeting minimum standards using the case file check list for those NGOs using Promise Cards was 72%. (N= 9) This tool was welcomed by all NGOs using it. Results are affected by simple issues related to scoring (where some NGOs are marking a case file down for not including a 17

FINAL REPORT: Promising Quality Assessment August, 2013 redundant document), but on the whole, there has been improvement over time across the sector in this indicator. Care needs to be taken when embedding the tool in the case management guidelines that it harmonises with existing systems (such as MoHCW case files, or the Africaid electronic case file system), and that the case file is easy to use for all practitioners including community case workers as well as lawyers. The tool has enhanced coordination. NGOs should be congratulated in putting systems in place quickly, often with the support of volunteer M&E capacity, low cost in-house training and where they existed, merging of existing systems to be in line with government standards. Agencies using the check list in each case file, rather than as an ‘audit tool’, had a stronger sense of the minimum standards. It should be noted that trends over time are an important element (as the protocols for this tool are currently arranged) yet this is not explicably analyzed in the existing SRT.28 “It’s like we understand from this check list everything we need to do. Helps me organize the case”.

28

See page 17 of PQ Document.

18

FINAL REPORT: Promising Quality Assessment August, 2013 5.2.3 Reflections on Quality Measures An analysis of results against quality indicators over the PQ implementation period indicated that most child protection programs showed trends of improvement. Although, this finding is tentative as the data is affected by the set up period of the M&E system, piloting of tools, changes in indicators, and adaption of program designs, trends of improvement are still evident. The tools developed are relevant, and given the traction so far and these preliminary results, they are of benefit to practitioners, children and the sector. They help give a more realistic picture of service provision reach. It is important, however, that DSS together with other government agencies, partners and UNICEF remain open to other measures of quality and feedback mechanisms in line with changing priorities and best practice. The trends of improvement are likely to continue as the M&E instruments are effective social work practice tools. The tools are designed to improve coordination, wellbeing and sense of accountability/rights when engaging with clients. The impact of these tools is being reflected in program participants’ responses. This is likely to improve further as the tools become part of a coherent, national case management system. The most consistently identified barrier to reaching quality targets at the NGO level, other than adjustment to the new M&E system, was effective referrals pathways and follow up. The most consistently reported enabling factor towards reaching targets, was effective relationship building BOX 4: CHILDREN’S FEEDBACK FORA and strengthened referrals pathways and follow up. Practitioners welcomed the Ask the Expert tool as a concept but the study highlighted concerns related to its implementation. Issues included difficulty in translation from English to Shona and English to Sign Language, administration to children under 12, or children in rural areas with less exposure to feedback concepts, the length of the tool, and the need to adapt the tool for children with particular disabilities. In the translation from English to Shona, NGOs working with child survivors of sexual violence highlighted the risks for children in using the translation for ‘love’, for example, that warrants urgent and accurate review – as it presents a protection risk for children interviewees. 29 There was also some confusion in regard to following administrative protocols (guidelines for sampling developed by each NGO) when using the tool with caregivers. Figure 4 presents practitioners views on ‘ease of use’ of the three tools, demonstrating the difficulties with Ask the Expert in particular and similar issues when gathering feedback mechanisms agencies had developed for the Promise Cards.

Children’s feedback is integral to the Promising Quality Framework noted above and includes application of the Ask the Expert psycho-social tool for reporting on well-being as well as the Promise Cards tool designed to help children and their care givers understand who is helping them and what they can realistically expect. NGOs must make sure that children receive and understand the promises, and then decide when the most appropriate time to gather feedback is. This may be during the service provision process or at the time a case is closed, each NGO will need to develop a protocol for how and when to collect regular feedback from children regarding the promises they made. Additionally ad hoc feedback is collected during field monitoring visits and specific participation events are convened as required. Excerpt M&E Framework for CPF

Simple adjustments to organizations administrative protocols, and basic simplification of the tools and clarity around their use would likely see a more uniformed response. Also of note, some organizations are not yet using all the measures of quality. Promise cards for example are not all printed or finalised and the case file check list still being adapted for some interventions.

29

The translation for the word ‘Love’ from English to Shona is ‘Kuda’ – meaning ‘affection’. This raised concern among some partners using the questionnaire with child survivors of rape. Alternative and culturally appropriate translations need to replace this, to ensure a culturally safe interview for children who are particularly vulnerable, especially if the tool is administered by un-trained staff.

19

FINAL REPORT: Promising Quality Assessment August, 2013 In addition, programs should consider working against a target of 100 per cent, even where this may be difficult to achieve with current resources. This goal setting will enable organisations/program teams to continue to stretch effort towards quality and as outcomes move closer to the target, the specific challenges associated with equity (and closing the gap) will come to the fore. This will encourage a continual focus on how to reach the most vulnerable and hard-to-reach BOX 5: THE SHIFT TO REPORTING populations. OUTCOMES NOT PROCESS Figure 4: Practitioners views on how easy or difficult the tools where to use

“The ground is shifting beneath our feet. This is a whole paradigm shift.” NGO Respondent “But we do all this work, and then they just say ‘where are the results?’. It’s been hard to explain using this template, all the system building, background process work that is important before you can see a result’. NGO Respondent “We need a narrative to explain the process behind the numbers”. NGO Respondent

5.3 REDEFINING SUCCESS Finding: PQ is redefining success in child protection work While all NGOs were ultimately positive about the focus PQ places on child participation and program impact, many practitioners expressed their perception and frustration that the project process work and delivery of project outputs was no longer valued, because outputs and process could not be reflected in the new Simplified Report Template (SRT) (see Box 4). This concern was especially emphasized for those NGOs who understood their program as primarily capacity building (such as WEI, Concern and Child Protection Society), and for those programs with intervention logic that targeted caregivers, or communities in order to impact children’s lives.

“From a policy perspective, we need to understand the strategies the NGOs are using. Not just get a matrix of numbers”. Government Respondent “We can’t be expected to report like that. It is not up to us, it is up to government to deliver, and we are here to build capacity of government, so this kind of reporting doesn’t show the work we are doing.” NGO Respondent “I get it, we need to look at outcomes, but this is hard for us. We are still catching up. Do you think we should do this from year 2 or maybe year 3 instead?” NGO Respondent “But how you do something in this work is important. Not just the result at the end.” NGO Respondent

This view was not always held consistently within an organisation, and demonstrates a stage along a change journey. Success in project implementation is being defined differently in the PQ Framework. It is no longer about reporting what the organisation did but ultimately asking if that process resulted in a positive change for children. The number of training events is only important when it correlates to children feeling better, and how well they think a program is meeting their needs. The assessment team found that the dialogues during interviews for this report, helped to clarify the framework for some partners. “It’s been good talking with you about it. We have been confused to be honest. We sometimes feel that we do all this work and it’s not understood and there isn’t somewhere to explain it in the report. But I can see its part of changing our practice”. NGO Respondent 20

FINAL REPORT: Promising Quality Assessment August, 2013 In the past ‘effort’ or activities-based project reporting defined success for many project implementers; the new indicators which celebrate outcomes, do not always capture the ‘methods’ towards the ‘goal’. While there is an appreciation, especially at senior levels of the organisation that this type of outcome reporting is embedded in a trusting partnership approach, resulting in increased flexibility as well as accountability for outcomes, it has also bought an anxiety. Confidence in reporting against the PQ indicators still needs to be nurtured. This nurturing involves celebrating staff competencies that result in outcomes – such as problem solving, relationship building, rather than ‘roll out’ project management. It may also involve reassuring partners that the STR demonstrates a story of trends over time in quality and there is a need to strengthen other forums for strategic discussions around process. Practitioners within the same organisations offered a range of understanding and conflicting views on the process versus outcome reporting debate, with senior management demonstrating the greatest understanding and acceptance of the paradigm shift. These responses however are specific to a point in time in the evolution of the system and will be ameliorated as the system beds in and all programs develop indicators that link process work to be accountable at an outcome for children level. “The reporting was simplified which was good, but didn’t allow a narrative. M&E has been difficult but will be easier. We have had to bring on more resources initially, at our own expense. NGO Respondent For those NGOs that have a good dialogue with DSS and UNICEF, have a clearly defined MCI and indicators and participated fully in the training, the fear and confusion was diminished. This suggests that if the planned protocols of systematic meetings, and forums to discuss strategic issues in program delivery are followed, and once PQ is well understood by all DSS and UNICEF programming focal points, the SRT is sufficient. Whilst some NGOs suggested the re-introduction of a narrative component to report on process in the SRT this could result in organisations losing sight of the critical outcomes for children and the gains PQ has made in terms of increased participation, real-time evidence based program planning, improved program practice and the organizational focus on critical interventions for children. Rather, it is our view that fostering a supportive culture among partners, strengthening all stakeholders understanding of the PQ and seeking alterative forums to celebrate, discuss and learn from gains will be increasingly beneficial, as practitioners harmonise their reporting systems.

21

FINAL REPORT: Promising Quality Assessment August, 2013 5.4 IMPROVING CHILD PARTICIPATION

INNOVATION FOR EQUITY PARTICIPATION, INCLUSION & REALTIME EVIDENCE BASED PROGRAMMING One of the unanticipated benefits of Promising Quality has been the evolution of innovative participation platforms for the most marginalised children. These are the children who may need special help because of their experience of abuse, or because they are very young, or because they cannot read or write; some children who cannot see or hear well, or who cannot understand or use usual communication methods because they have an intellectual or physical disability. The consistent feedback on the Promise Card and Ask the Expert tools, was that as presented these were not user friendly for many children, including those with a disability. In response, many organisations together with children, caregivers and other practitioners, have begun to explore innovative and creative ways to ensure program participants, and where appropriate their caregivers, can meaningfully contribute to the measurement of service quality and are able to provide real-time feedback on their experiences. These include applying the tools with siblings as proxies for their brother or sister with a disability, using ‘Ask the Expert’ with caregivers on the basis that if the caregiver feels better the child will benefit, working with groups rather than individuals, and employing the creative arts to help children express how they are feeling through drawing or flash cards. Because of the innovation required to include the most marginalised children, including children with disabilities in participation platforms, what was originally conceived as a basic M&E framework has become a real-time tool for evidence based enhanced service delivery and child-friendly, child-centred programming. The increased focus on participation has strengthened the rights-based approaches of some agencies, and is providing a stronger foundation for evidence-based advocacy in the Justice, Health and Protection sectors at large. This influence is likely to grow, as agencies further develop their expertise and ways of working. Because of the complexity of supports required by some children (see NGO report, Leonard Cheshire Zimbabwe Support Trust) this framework is also opening up the dialogue about the relationship between interventions that support ‘development with equity’ and considerations of value for money. PQ is breaking new ground in embedding participant feedback as well as developing a culture of accountability to vulnerable children. The ‘space’ PQ has created to ‘think outside the box’ has enormous potential through the use of creative arts, technology, peer M&E models, and alternative communications to influence participation and the realisation of rights of the most marginalised children, not only within specialist services, but in the wider social welfare and access to justice sectors. “Children with severe disabilities face so many barriers to accessing justice, even discrimination in court. We are there each step of the way, helping them to understand what is going on. We can find ways to make the system work so everyone can participate. It can be harrowing process- and that shows when you ask a child ‘are you happy?’ but it’s critical we check in with the child and caregiver that things are okay and we have kept our promises”. “We adapted the tools so that we can administer them in groups. We have so much fun! We use lots of different props and we get the caregivers involved, so the children’s feedback workshops tell us how we are doing, and we change our program if need be”. ‘Before this project we didn’t routinely get feedback. We interact differently now. This has influenced our work in many, many ways. It also reminds the caregivers that their child has rights’. NGO Respondents

22


Finding: the PQ M&E instruments either introduced or enhanced child participation in programming The Ask the Expert and Promise Card tools not only provided a vehicle for meaningful engagement between beneficiaries and practitioners at the individual level, but the concept of ‘downward accountability’ contributed to more child-friendly, child-rights informed practice at the organizational level. This was evident in extensive discussions on the Ask the Expert tool, where short-falls in the tool highlighted an interest in pursuing creative, meaningful and useful measures of self-reported wellbeing.30

BOX 6: PARTICIPATION IN PRACTICE ‘The truth is we didn’t have this kind of platform before. We didn’t think about it enough. We will now’ “It reminds us, in the everyday work, that children have many aspects to their lives. If we didn’t ask children, we won’t know. It gives us a fuller picture” ‘It is good, and has provided us with additional, information on a child – that sometimes has been critical to know. It influences the case plan. But it cannot become a tick in the box. We need to keep developing it. Would be dangerous to think, this is a decent enough measurement of wellbeing on its own.’ ‘It’s a good concept, and it encourages us to think about

Organisations that had developed platforms for accountability and to whom but I worry about some other organisations – how they come up with wellbeing figure. It will still regular beneficiary feedback, benefited from a need to be verified by someone independent like your team. There is systematic measurement of quality linked to this our version of quality, and there is their version!’. NGO Respondents participation. Services that involved counselling often found it easier to introduce the tool, yet warned that this measure is not sufficient on its own to ensure that a child is safe, and well. According to protocols outlined by individual agencies, feedback mechanisms including forms linked to the promise cards were at various stages of development. Many agencies were merging or adapting existing exist interviews, or service feedback tools. Work remains to link these tools to the case file and to standardize, where relevant, within the Zimbabwean Case Management Guidelines being developed by DSS.

5.5 DEMONSTRATING VALUE FOR MONEY Finding: Although complex to measure with nascent data sets, most organisations are demonstrating value for money, by finding ways to improve quality of services, reach and the cost-efficiencies. The quality indicators can aid the routine mapping of trends in effective and efficient program delivery. It is widely recognised that demonstrating ‘Value for Money’ (VfM) in child protection service delivery is particularly difficult. Previous efforts have sought to quantify cost-per-child, however these attempts note that this alone offers an insufficient measurement of value for money because it does not take into account the intensity of services, external factors that influence service delivery costs and can erroneously encourage comparisons across services that require entirely different inputs and resources. For example, Leonard Cheshire Trust Zimbabwe reached around 74 children, currently at a cost per child of $827. This does not take into account that a majority of these children were required to attend court on multiple occasions (up to 22 times) and subsequently service delivery costs appear inflated. Similarly, the cost per child between Childline ($25) and Kapnek ($73) are not directly comparable because

30

For a full discussion on the Ask The Expert feedback see Validation Report , UNICEF and University of Zimbabwe School of Social Work, 2012

23

FINAL REPORT: Promising Quality Assessment August, 2013 the inputs required to deliver each service are not the same. It was for these reasons that the PQ Baseline Evaluation which called for a more ‘in-depth economic analysis to analyze intensity of services and other costs.31’ This assessment was tasked with exploring VfM. It noted that the majority of organisations are not explicitly looking at value for money considerations in their strategic planning, proposals to funding partners, or as part of their overall monitoring and evaluation processes. However, Government, its partners and donors are finding ways to challenge and improve cost-efficiencies and to more thoughtfully reflect on progress. This dialogue is critical to ensuring that the child protection system goes to scale as effectively and quickly as possible, and to ensure most vulnerable groups and hard to reach populations receive services that have a lasting impact. A very basic framework is presented here that can be used to stimulate more thoughtful strategic discussions about VfM as the program moves forward. This effort is prefaced however, with the following cautions:   



The framework and each of the graphs offer a ‘snapshot for discussion’ which must be read in collaboration with the corresponding narrative to put it in context; The data sets currently available are very limited – spanning between 1 and 3 years (and 3 quarters for quality measurements) – and have been collected over periods of rapidly changing program designs; Each organisation is at different stages of developing their data collection systems and so there remains some inconsistency in the ways in which data (for example, reach) are collected and defined both between organisations and by the same organisations over time; As cost per child is demonstrated as problematic for assessing the value of child protection initiatives, the emphasis should be on the ‘new’ quality element. Care should be taken to use these graphs as a planning device, among others when considering expenditure over the life of a program32.

It is against this backdrop, the framework seeks to highlight three key areas which, together, offer a basis for thoughtful ongoing analysis of program design, cost and effectiveness. If this data continues to be collected, it can offer an opportunity to identify areas which require attention and focus as programs are taken to scale within a government child protection system. In this study, each organisation has been assessed against this framework and the results are presented in a graph and can be read using the following keys in the tables below. Tables 6 & 7 Value for Money Guide. Series Reach Quality

Cost per child

Definition This refers to the number of children who are delivered a service, in line with the PQ framework for reach. This refers to an overall analysis of the PQ indicators (MCI, Accountability, Wellbeing and Coordination) This refers to the overall program budget, divided by the number of children reached with a service, in line with the PQ Framework for reach.

Source Source: SMT and narrative reports submitted to UNICEF Source: SRT, narrative reports where available, key informant interviews, NGO verifications. Source: UNICEF compiled data, PoS Outcome Assessment, PQ Baseline Report, 2012, NGOUNICEF PCAs

Trend Upward trending line

Interpretation The organisation is demonstrating improvements in this area (e.g. greater cost efficiency, greater indications of quality, increased reach) Downward trending line The organisation is demonstrating reductions in this area (e.g. lower cost efficiency, lower PQ scores or decreased reach) Straight line The organisation has not increased or decreased (reach, cost efficiency or quality scores have remained constant over time) NB: where the line is dotted, this indicates that not all information was available and an overall assessment has been made drawing on anecdotal information, key informant interviews and other sources of data. 31 32

Promising Quality Baseline Evaluation, 2012 NGO Reports Annexes. There is potential to wrongly encourage up front spending to demonstrate positive trends in this model

24


Ideally for most interventions, organisations should aim to demonstrate an upward trend for all three areas, reach, quality and cost per child, and this would indicate overall value for money (notwithstanding case by case considerations for program design). The framework, or similar method, could also potentially be adopted by the NAP and other coordination bodies as a means of tracking overall cost-efficiencies and scalability of programs – to ensure that quality is considered in the value for money equation Value for money snapshots for each organisation can be found in the individual NGO reports. Figure 5 provides sample of two program ‘snapshots’. Note that trends are not to scale.

Figure 5: Examples of VfM SnapShots (From 1. program using PQ and 2. Program still to introduce PQ)

Overall, it was found that most organisations are demonstrating value for money, by finding ways to improve quality of services, reach and the cost-efficiencies that would allow the program to go to scale more rapidly. In some cases, there are occasional downward trends which may indicate an anomaly (for example, increases in staff salaries or high upfront costs for new programs). Whilst this may not be an indication of poor value for money at this stage, it will be important to continue to track and monitor these trends to ensure that they move towards a more upward trajectory in coming years. In some cases, it is believed that the downward trajectory may be an indication that program targets were exceeded and future targets have not been adjusted to reflect the improved cost-efficiencies and highlights the importance of an explicit review of value for money considerations when designing new program implementation phases.

25


BOX 7: 100% REPORTED PQ INFLUENCES PROGRAM DESIGN

5.6 REAL-TIME EVIDENCE BASED PROGRAMMING Finding: The introduction of the PQ framework and the practice orientated M&E tools is contributing to improvements in service provision quality. The PQ framework is resulting in more child friendly engagement platforms, more incentive to ‘act on’ results in real time, more flexibility in programming, improved motivation towards inter-agency and government collaborations and more client focused practice. 100% of NGOs using the PQ framework reported that it was positively influencing program design, at either the individual case level or organizational level. This influence is likely to continue as organisations develop and improve internal reporting and planning platforms and the M&E system is regularized in system wide feedback forums. The Ask the Expert Tool, for example, gave practitioners direct feedback from the child about what should be in the child’s case plan, and was influencing the type of partnerships being formed at an organizational level for referral and skills exchange between agencies. Even for agencies with a high level of child participation, the Ask the Expert Tool and the Promise Card ‘exit interview’ provided a quality control check that captured important case information.

5.7 COMMITTMENT AND BUY-IN Finding: Commitment and all stakeholder buy-in is important to build confidence in the framework While stakeholders saw value in the framework, there was evidence that understanding of the core concepts and procedures and commitment needs strengthening, including within DSS and within the UNICEF team. There was a nervousness among some NGO partners about implementing the framework, borne of both not fully understanding the requirements and fear that the M&E requirements will change in the future. Commitment and buy-in will be critical moving forward, especially as the framework starts to see and rely on linkages between partners and DSS and between DSS and other government departments (such as relevant agencies within MoHCW and MoJLA) to reach its full potential. The upcoming mid-term review of the NAP offers an opportunity to explicitly embed this framework in a national government strategy and will contribute to allaying some of these concerns. Further, a number of agencies have invested considerable time, resources and advocacy in embedding the framework. Further work needs to be done to link the tools to standardized reporting within the case management system - this will assist in clarifying some of the modalities. For example, linking the promise cards feedback forms to a case file, clearer guidance and M&E technical support on how best to do adapt protocols and tools to particular contexts.

“It has helped us help the caregiver appreciate that a child’s needs must be met holistically. I am sure there is more follow up taken up by the family now, just because they see that we are interested in more than just the legal paperwork” “It highlights gaps in our practice from the children’s perspective”. “One child answered that they were not feeling well during the Ask the Expert, it became clear that that child’s ART’s were not the right treatment plan and were making him feel sick. We learnt this, were able to change them. This made a huge difference, potentially a lifesaving difference to that child. It is not clear that this information about the child’s wellbeing would have come out in other ways.” NGO Respondents

BOX 8: CHANGE NEEDS MAJORITY BUY-IN “All partners need to be onboard otherwise we don’t trust it. Everyone needs to understand measuring quality, we still get confused” “It’s been challenging. Constantly reminding ourselves why we are reporting qualitative but with numbers, and ahh! – we don’t think it makes sense” “UNICEF was very supportive. But we had some confusion with M&E, and even they had trouble to see how our process work could fit into the framework. We sometimes got conflicting advice…” NGO respondents 26

FINAL REPORT: Promising Quality Assessment August, 2013 5.8 STRENGTHENING ZIMBABWEAN CASE MANAGEMENT PRACTICE FRAMEWORK Finding: The full potential of the PQ tools will be realised within the framework of the National Case Management Framework To realize its full potential the PQ Framework must be coherent with the National Case Management Framework. Currently, there is a tendency for some agencies to hold onto cases (beyond the ‘MCI period’), or to pick up cases outside of organizational mandate. There is also a tendency for case follow up post referral to be ad hoc. These appear natural responses in the absence of a functional case management system however, despite the best efforts of NGOs and DSS officers; the danger is that parallel systems develop as ‘de facto case management’ roles are taken up. Simple improvements to the tools, such as measures to render a case ‘in-active’ cases or ‘cases pending referral’ – while it is in case follow up mode, will help agencies pass on cases, and minimize ‘mission creep’ – a process that could undo some of the work in clarifying organizational roles within the sector, through the use of MCIs. National guidelines will assist with this process greatly. The Case File Check List, Promise Card and Ask the Expert Tool are components of guidelines for practice and record keeping – now being used by the majority of DSS partners, to some degree. Together with the Zimbabwean specific design of referral process, locally appropriate decision making, simple guidance in case conferencing (allowing for resource constraints), the tools form the basis for a childfriendly and participatory Zimbabwean model. The typology and approach for this system should consider the resource constraints in Zimbabwe, and heed warnings of ‘dominance of western models of social work’33. The PQ tools were developed locally by practitioners and government in Zimbabwe and as they continue to evolve and improve with trial as part of the case work toolkit, are proving relevant and successful. The next step is to organize these tools within the broader framework. However, this ‘Zimbabwean’ model for measuring quality in child protection service delivery has relevance both regionally and globally and can be simply adapted to specific national contexts. Furthermore, the national case management model draws heavily on a volunteer workforce. This may offer some cost efficiencies, compared with models that use paid staff, however, additional attention will need to be given to ensuring that volunteer motivation is maintained. This model is being introduced into districts which already have cadres of volunteers working on child protection related work (for example, those working with Child Protection Committee (CPCs) and Childline). Efforts should be made to integrate these workforces to minimize role confusion, duplication of work, and the challenges that will inevitably arise if there are disparities in incentives.

33

See Walton and Abo El Nasar, 1988 in Challenges and debates in social work, Chapter 5, Making Social Work Work, Everychild, 2012

27

FINAL REPORT: Promising Quality Assessment August, 2013 6. RECOMMENDATIONS 6.1 IMMEDIATE  Continue the roll-out and strengthening of the Promising Quality Framework.  Make urgent and simple change to Ask the Expert Tool when used in Shona (see foot note 27) and in line with the Validation exercise.  Develop and disseminate a ‘who, what, where’ (3W) directory to support the referral process  Continue to work with NGOs to re/define MCIs and embed PQ tools in everyday practice. MCI indicators and targets should describe outcomes for children (not adults); where adults are the direct beneficiary and children indirect beneficiary e.g. Family Clubs, the relationship between the program intervention and the outcomes for children should be clearly articulated. A child-focused MCI should also be possible for interventions where services are delivered in partnership between government and a partner. The MCI should also be linked to measurement of reach. That is, the MCI should describe the service which is being provided to children, and which is subsequently reported ‘children reached with a service’. This means that data will more accurately reflect specialized and quality services provided to children affected or at risk, rather than, generalized support. This should enhance government’s ability in planning for vulnerable groups.  Together with partners, consider all targets of quality as 100% and focus on improvement towards the goal as one key measure of success.  Roll out of the National Case Management Framework, including the PQ framework  The National Case Management Framework should be finalized, urgently. This should include guidelines for referral and respective agencies’ roles and responsibilities as well as procedural tools and training materials.  The draft National Case Management Framework should be harmonised with the PQ Framework. The plan for release and information sharing around the guidelines should be expedited.  Information sharing should include an update on available services (to enhance referrals), should include all Child Protection agencies regardless of funding source and other stakeholders including NAC, MoHCW, MoJLPA, MoWAGCD, MoESAC etc.  Framework should explicitly look at how to integrate volunteer workforces  Improve Confidence and Buy In  Ensure staff at all levels, and in all agencies, have an understanding of the PQ Framework, both as a practice tool for individual cases; and as a real time M&E tool to inform programming.  Provide a forum for M&E focal points to meet and discuss implementation of the PQ Framework.  Link support for PQ training with roll out of Case Management Guidelines.  UNICEF/DSS to continue to foster a spirit of ‘partnership’, joint accountabilities, transparency and encouragement with all NGO partners focused on child protection interventions.  Simplified Reporting Template and Strengthen Reporting Coordination  Continue to use the SRT with the following clarifications:  Clarify reporting period and align for all partners in the template, irrespective of when a PCA starts. 28

FINAL REPORT: Promising Quality Assessment August, 2013 



  



 

Improve consolidated reporting at the UNICEF/DSS level and share quarterly with NGO partners. A ‘dashboard’ can be created to build on the PCA management spreadsheets and could track progress and trends in real time (quality, reach, Value for Money) across the sector using excel. This combined sheet should include clear definitions of key terms (service, adult, child), and have standardized reporting timeframes to allow for easy consolidation of data from individual programs. Though work has been done to align programming, reporting periods are not aligned, affecting data accuracy. It should also aim to capture trends within NGOs (that need to be better captured at the organizational level, especially for the coordination and case file quality indicator) Separate reporting on children reached from adults reached. If adults are the direct beneficiary but the intervention is intended to result in an outcome for children this should be clearly articulated e.g. legal advice to a parent on maintenance issues (please see above). Explicitly link the definition of a service to the MCI. Desegregate data for girls and boys (and adapt STR so this is easy and straight forward in targets and reporting). Provide space in the SRT for brief, analytical report, specifically against the PQ indicators. For example, where a target was not met, the partner can identify the barrier and the strategy for addressing this in the following quarter. UNICEF/DSS to provided clear guidance and support on this. Systematically strengthen and implement PQ protocols for monitoring (see section 3.1.2 and figure 2 in the report that outlines the support measures for introducing and using the PQ ). This is currently ad hoc. Enhance existing forums for strategic discussion around process, lessons learned and to celebrate the effort that goes into delivering outcome level results. DSS and UNICEF to align field monitoring with the PQ framework. Together with the SRT and quarterly partner meetings, this will provide the opportunity to validate and triangulate data and inform program strategy and implementation.

 Implementation of Partner-specific recommendations All partners, together with DSS and UNICEF to meet and agree on a management response to; and implementation strategy for addressing recommendations in the partner-specific reports. 6.2 MEDIUM TERM  NGO partners should strengthen internal procedures to include platforms for evidence based and strategic planning and review, biannually as well as during regular Team Meetings.  NGOs, DSS and UNICEF to consider review of staff competencies and promote skill sets that get results for children (based on evidence, such as relationship building, problem solving, flexible programming)  Update the Promising Quality Document with a view to having a user friendly guide and reference piece for implementers. The use of diagrams is helpful. This guide should be linked to the national case management guidelines.  Consider a user friendly version of this assessment for dissemination to beneficiaries  Consider and develop further the value for money framework to inform future program strategy and design.

29

FINAL REPORT: Promising Quality Assessment August, 2013  Consider obtaining case studies, linked to PQ implementation, biannually to expose understandings and practice strategies for discussion and service improvement.

6.3 LONG TERM  DSS through the WPO M&E sub-committee harmonsise the PQ framework across other ministries (considering client goal setting frameworks in MoHCW for example, and strengthen intergovernmental working groups understanding of the PQ framework for children, such as the Inter Ministerial Committee on Disability and Rehabilitation, and create linkages between the National Program of Action for Children and the NAP II, and Legal Aid forums - and familiarize work of social service delegates across agencies (such as Medical Social Service Officers, Paralegals and Social workers).  Explore innovative options, including creative use of technology to enhance participation of different groups of children and their caregivers, such as children with intellectual disabilities and for monitoring trends. For example explore partnerships to develop iPad apps, methods to feedback over the phone etc.  Document lessons learnt on reaching children for whom accessing services and providing feedback has been difficult with view to better understanding the barriers PQ tools can remove for vulnerable groups in gaining access to services.

30


7. NGO PROGRAM REPORTS 7.1 RATING GUIDE Assessment Criteria Overall Service Description

Are program Targets being met?

Is the Promising Quality Framework in Place

Value for Money

Rating

Explanation



QUALITY SERVICE FOR CHILDREN

A good assessment of overall quality



POTENTIALLY A QUALITY SERVICE FOR CHILDREN

On track to be a quality service (evidence of improving trends in quality)



Should adapt, adjust strategies and/or needs improvement to demonstrate overall quality

 

IMPROVE TO BE A QUALITY SERIVICE FOR CHILDREN YES ON TRACK



NEEDS IMPROVEMENT

Majority of targets are off track, unlikely to be met without changes in strategy



YES



NO

  

VfM TRENDS

Programs ‘implementing PQ’ – see section 3.2.1programs were there is routine reporting against at least one quality measure indicator ((a case file check list, percentage of Most Critical Indicators (MCIs) met, or Promise Cards, or Wellbeing (Ask The Expert). 60% of participating NGOs Programs with limited knowledge or awareness of PQ (see section 3.2.1) 40% of participating NGOs Program has trends for cost per child, quality over time, reach over time

UNCLEAR

Majority of targets have been met Majority of targets are on track and likely to be met

Program yet to establish trends for cost per child, quality over time, reach over time

7.2 INDIVIDUAL PROGRAMS [see separate files: ] 7.2.1 Africa Community Publishing and Development Trust 7.2.2. Africaid 7.2.3 Bekezela 7.2.4 CATCH 7.2.5 Childline 7.2.6 Child Protection Society 7.2.7 Family Impact 7.2.8 Family Support Trust 7.2.9 J.F. Kapnek Trust 7.2.10 World Education Inc. 7.2.11 Leonard Cheshire Zimbabwe Trust 7.2.12 Legal Resources Foundation 7.2.13 Mercy Corps 7.2.14 Musasa 7.2.15 Concern World Wide Zimbabwe

31

FINAL REPORT: Promising Quality Assessment August, 2013 ACRONYMS ACPDT Africa Community Publishing and Development Trust ART Anti Retroviral Therapy CPF Child Protection Fund CRC Convention on the Rights of the Child CPC Child Protection Committees CPS Child Protection Society CWW Concern World Wide Zimbabwe DSS Department of Social Services GoZ Government of Zimbabwe LCZT Leonard Cheshire Zimbabwe Trust LRF Legal Resources Foundation M&E Monitoring and Evaluation MoESAC Ministry of Education and Sports, Arts and Culture MoHCW Ministry of Health and Child Welfare MoLSS Ministry of Labour and Social Services MoJLA Ministry of Justice and Legal Affairs N/A Not Applicable N/R Not Reported NAP II National Action Plan for Orphans and Vulnerable Children Phase II 2011 – 2015 NGO Non-Government Organisation OVC Orphans and Vulnerable Children PCA Program Cooperation Agreement PQ Promising Quality Framework PoS Program of Support (to the National Action Plan for Orphans and Vulnerable Children Phase I 2004 – 2010) QCFCL Quality Case File Check List (Promising Quality Tool) SRT Simplified Reporting Template (Promising Quality Tool) UNICEF United Nations Children’s Fund WEI World Education Inc. WPO Working Part of Officials

32

FINAL REPORT: Promising Quality Assessment August, 2013 Annex i TERMS OF REFERENCE (INTERNATIONAL CONSULTANT) United Nations Children's Fund Zimbabwe Country Office – Vacancy Announcement If you are a committed, creative professional and are passionate about making a lasting difference for children, the world's leading children's rights organization would like to hear from you. For 60 years, UNICEF has been working on the ground in 190 countries and territories to promote children's survival, protection and development. The world's largest provider of vaccines for developing countries, UNICEF supports child health and nutrition, good water and sanitation, quality basic education for all boys and girls, and the protection of children from violence, exploitation, and AIDS.

Consultancy No. Zim/2012:15

Title: Evaluation of Specialized Child Protection Service Provision, based in Harare Duration: 37 Days Background UNICEF is currently supporting 13 Non-Governmental Organizations to provide critical child protection services. Some of these partnerships fall under the Child Protection Fund in support of the second National Action Plan for Orphans and Vulnerable Children. Other partnerships have been established to support complementary justice sector programs, including the roll out of the Legal Assistance for Children Strategy and the Protocol on the Multi-Sectoral Management of Sexual Abuse and Violence. A summary of these services are outlined below. A number of key evaluations have also been undertaken in recent years, including an outcome and impact assessment of the Program of Support in 2010. Also in 2010 and 2011, reviews of retrospective quantitative data were undertaken by Save the Children, Family Support Trust and Childline. A comprehensive child-focused monitoring tool – My Life Now – was also developed in 2010 and used to assess the Program of Support. More recently, a Quality Evaluation was conducted in 2011, during the bridging phase between the Program of Support to the NAPI; and the Child Protection Fund (in support of the NAPII). This evaluation achieved the following objectives:  An independent overview of the quality of service delivery by various UNICEF-supported NGOs  The design of a ‘quality’ framework, with accompanying standards, indicators and tools. Over the course of 2012, a number of NGOs have developed baselines and have commenced the routine monitoring of quality of service delivery. Progress against these baselines, and the lessons learned during the first 12 months of implementation now requires review.

2.

Introduction

This evaluation will provide one consolidated report which will include, at a minimum: 1. a summary of progress to date, including numbers of children reached and progress against respective partners’ Promising Quality frameworks 2. an analytical review of the quality of service delivery being offered to children through these partnerships, identify areas of good practice; and alert UNICEF and its partners to areas which require strengthening. The intended audience for this evaluation is, primarily, the Ministries of Labour and Social Services; Justice and Legal Affairs; Health and Child Welfare and Women’s Affairs, Gender and Community Development; the Judicial Services Commission, UNICEF and their program partners. It will also be likely that the results of this evaluation will be used by other partners, including other Government ministries in the design, implementation, monitoring and evaluation of future child protection programs, including specialized child protection services that will be supported through the Child Protection Fund. For the purposes of this evaluation, the determination of quality is to be guided by the following:  The best interests of the child as the paramount consideration  Adherence to international child rights standards, norms, rules and best practice; and the degree to which the service enables children to realize their rights and address child rights violations (this should make explicit reference to gender sensitivity and equity principles)

33

FINAL REPORT: Promising Quality Assessment August, 2013   

3.

Adherence to national frameworks and policies, where these do not conflict with international standards, norms, rules and best practice The degree of reported satisfaction of child clients and their families Value for money

Objective (s)

To assess the overall quality of services provided by UNICEF partner child protection organisations and the difference these services are making in children’s lives. In collaboration with Monitoring and Evaluation Officers in each service, the evaluator will: 1. Review and provide an analysis on implementation of the Promising Quality Framework (both reach and quality of service delivery) 2. Undertake a broader evaluation of service delivery quality 3. Make recommendations for improving both the implementation of the Promising Quality Framework and the broader implementation of best-practice service delivery

4.

Methodology & Expected Output

Through a combination of desk review, onsite visits and consultations with children, service providers and other key partners, the consultant will: 1. Undertake a desk review, including critical analysis of data, NGO quarterly reports, spot check and field monitoring reports; and other existing evaluations and conduct key informant interviews 2. Provide an inception report, outlining the proposed method and work plan for completing remaining tasks. This inception report will include a specific report format for each NGO that takes into account the Promising Quality framework and other areas for evaluation. 3. Undertake the evaluation through site visits, in-depth interviews and focus group discussions. Direct feedback from children will also be sought. 4. Deliver a validation workshop to all partners, prior to leaving the country and prior to the finalization of key outputs. 5. Provide a final report for discussion with other stakeholders, including Government and donors. This can be completed in or out of country.

5.

Major Tasks, Deliverables, Budget & Timeframe -

Consultant engaged and contract signing by February 4, 2013 and 20% payment upfront. Inception report finalized, including desk review (4 days) Evaluation of individual services (30 days) and 30% payment upon delivery of draft report. Final report available for dissemination (3 days) and 50% payment on delivery of final report.

All reports to be final fully edited and formatted and provided in electronic copy, in English. Final payment will be made when the documents are endorsed and considered final and ready for dissemination.

6.

Consultancy Requirements 

Advanced university degree in a related field

Knowledge  Sound technical understanding of child protection programming, including international best practice  Sound technical understanding of international child rights frameworks, particularly those relating to child protection  Knowledge of qualitative and quantitative research methods

Skills and Experience

34

FINAL REPORT: Promising Quality Assessment August, 2013       

Demonstrated ability to analyse quality of service delivery and conceptualise and articulate service delivery improvement strategies Previous relevant working experience of at least 5 years as a practitioner and in the development and implementation of monitoring and evaluation programs Experience in the implementation of child participation programs Demonstrated ability to use initiative and work without close supervision Strong written and verbal communication skills Fluency in English is essential. Fluency in Shona or Ndebele would be an added advantage. Experience in Africa, particularly Southern Africa is desirable

Other requirements: The Consultant will be required to arrange their own work premises in-country, and be responsible for their own travel and logistical arrangements. Workshops will be undertaken in English. Applicants are asked to submit a budget for travel, daily rates and daily allowance as the fee will be paid as a lump sum.

Annex ii: INCEPTION MEETINGS, WORKPLAN, NGO INTERVIEWEE LIST 1. Inception Meetings Inception Meetings Date Meeting Held 17 June Head Department of Social Services (DSS) and the DSS Head Quarters National Team and NAPII Secretariat Child Protection Specialist, UNICEF Planning and Monitoring Specialist, UNICEF Local + International Consultants 18 June Chief Social Services Officer, DSS 19 June Planning and Monitoring Specialist, UNICEF Research and Evaluation Group, UNICEF + Collaborating Centre for Operational 20 June Research and Evaluation CCORE

2. Work Plan Note these dates and times are indicative only, as revisions were made according to interviewee availability. Agency Name and Date of Interview

Location (for Interview)

21 June CATCH Interview with Director Interview with M& E Officer Interview with Staff using the PQT 1 July Update DSS, NAPII, UNICEF together with Evaluation consultants on Work plan Interviews with DSS + NAPII staff Acting NAP Secretariat Coordinator Child Protection Technical Advisor 2 July Musasa Interview with Director

9-92rd Ave, Sunridge Park, Malbereign, Harare

DSS Boardroom Compensation House, Department of Social Services (DSS) Boardroom Compensation House 64, Selous Ave, Harare 35

FINAL REPORT: Promising Quality Assessment August, 2013 Family Support Trust Interview with Director Interview with Program Manager Interview with M&E Officer Interview with Staff who use the PQT (Social Workers) 3 July Family Impact Interview with Director Interview with M&E Officer Interview with Staff using the PQT Bekezela Interview with Director Interview with M&E Officer 4 July Child Protection Fund for NAP II Partners Quarterly Workshop Childline Interview with Director Interview with National Programs Manager Interview with staff using the PQT Mercy Corps Interview with M&E Focal Person Africaid Interview with Director Interview with Program Manager Interview with Staff using the PQT 5 July Legal Resources Foundation Interview with Project Manager Interview with Staff (para-legals) using the PQT 8 July J.F. Kapnek Trust Interview with Director Interview with M&E Officer Interview with Staff using the PQT Legal Resources Foundation Interview with Director 9 July Child Protection Society Interview with Director Interview with staff using the PQT (Social workers based in Harare, Mutare, Marondera) Africa Community Publishing & Development Trust Interview with Director 10 July Leonard Cheshire Zimbabwe Trust Interview with Director Interview with staff using the PQT Interview with Programme Officer 11 July Interviews with UNICEF Child Protection Section Staff

Harare Hospital

DSS Compensation House (telephone calls to Bulawayo) DSS Compensation House (telephone calls to Bulawayo) Cork Road, Harare 31 Frank Johnson Ave, Eastlea Harare

73 Harare drive 12 Stone Ridge Ave, Avondale

3rd Floor, BlueBridge, Eastgate

38 Lawson Ave, Milton Park

16 Oxford Road, Avondale, Harare Corner H. Chitepo/Snowdon Road, Belvedere, Harare

3rd Floor Exploration House, 145 Robert Mugabe Road 188 Westwood Drive, Kambuzuma

UNICEF, Belgravia

36

FINAL REPORT: Promising Quality Assessment August, 2013 Concern World Wide Zimbabwe Interview with Director Interview with M&E focal person (M&E Coordinator) Interview with staff using the PQT World Education Inc. Interview with Director Interview with M&E Focal Point Interview staff using the PQT (Harare and field-based staff focus groups) 12 July World Education Inc. Interview with Director 15 July Interview with Chief Social Services, DSS 16 July

9 Coxwell Avenue, Harare

Childline Interview with Director Interview with National Programs Manager Interview with staff using the PQT Mercy Corps Interview with M&E Focal Person 22 July Validation Workshop (47 Attendees) Representatives from participating NGOs Representatives from Ministry of Labour and Social Services (Department of Social Services and NAP II Secretariat) Representatives from Ministry of Justice Representative from Ministry of Health and Child Welfare Representative from Ministry of Gender, Community Development and Women’s Affairs Representatives from UNICEF

31 Frank Johnson Ave, Eastlea Harare

29 Lawson Ave, Milton Park

29 Lawson Ave, Milton Park DSS Compensation House

73 Harare drive

Crown Plaza Hotel, Harare

1 August Interview with representative from the Legal Aid Directorate 4 August Interview with Deputy Director, Rehabilitation Unit, Ministry of Health and Child Welfare

Telephone interview MoHCW , Corner fourth and central Ave

3. NGO Interviewee list NGO Interviewees Position Title

Organisation

Director M&E Officer Director Programs Director M&E Manager M&E Officer Case worker

Bekezela Bekezela CATCH CATCH CATCH Family Impact Family Impact

Regular Duty Station (in brackets – coverage) Bulawayo Bulawayo (outstation) Harare Harare Harare Bulawayo Bulawayo 37

FINAL REPORT: Promising Quality Assessment August, 2013 Country Director Country Director Assistant Country Director Programs M&E Coordinator Family Group Coordinator Director Programme Officer Programs Coordinator Case Management Officer (Volunteer) Director Director Child Rights Specialist Harare Clinic Manager Para Legal Para Legal Director Social Worker Social Worker M&E Officer Counsellor/Psychologist Director Program Manager Counsellor Case Officer/Social Worker M&E Manager Director Programs Coordinator M&E Coordinator Director Social Worker Social Worker Social Worker Research Assistant Program Manager Program Coordinator Child Protection Manager Assistant Country Director Chief of Party/Country Director Staff Representative M&E Research Officer Community Case

Family Impact Concern World Wide Concern World Wide

Bulawayo Harare Harare

Concern World Wide Concern World Wide

Harare Bulawayo

Musasa Leonard Cheshire Zimbabwe Trust (LCZT) LCZT LCZT

Harare Harare

LCZT Legal Resources Foundation (LRF) LRF LRF LRF LRF Family Support Trust Family Support Trust Family Support Trust Family Support Trust Family Support Trust Africaid Africaid Africaid Africaid

Harare Harare

Aficaid J.F.Kapnek Trust J.F.Kapnek Trust J.F.Kapnek Trust Child Protection Society (CPS) CPS CPS CPS CPS CPS Mercy Corps World Education Inc.

Harare Harare Harare Harare Harare

WEI

Harare

WEI

Harare

WEI

Harare

WEI WEI

Harare Epworth

Harare Harare

Harare (Roving) Harare Harare Harare Harare Harare Harare Harare Harare Harare Harare Harare Harare

Mutare/Marondera Mutare/Marondera Mutare/Marondera Harare Harare Harare Harare

38

FINAL REPORT: Promising Quality Assessment August, 2013 workers Focus Group x 9 Community Case workers Focus Group x 11 Case Management Officer Worker (within focus group of 4) Case Management Officer (within focus group of 4) Director Program Manager Case Manager M&E Officer Director

WEI

Goromonzi

WEI/DSS

Goromonzi

WEI/DSS

Harare

Childline Childline Childline Childline Africa Community Publishing and Development Trust

Harare Harare Harare Harare Harare

Annex iii CASE STORIES Case stories were collected from all 15 NGOs. Those stories from organisations currently monitoring programs using, or beginning to monitor programs using the Promising Quality Framework (10 NGOs) are presented below under the relevant NAP II Thematic areas. These are edited manuscripts submitted from the partners in July, 2013. The stories demonstrate varying levels of comfort with the PQ framework. NAP II THEME: ACCESS TO JUSTICE CASE STORY: Legal Resources Foundation (LRF) Francis’ access to Justice (Rural) Francis (not his real name), is a 17 year old male , who was charged with raping his employer’s 9 year old daughter. Francis denied the allegations and was referred to Legal Resources Foundation by the Attorney General’s office for representation. Francis is an orphan. He was born in a family of eight and is looked after by his brother. He dropped out of school at grade 6 when his father could no longer pay the fees. He secured employment as a herd boy at a homestead. Francis’s employment contract provided that he be awarded a head of cattle after one year. When the wor year was over, his employer indicated that he did not have the promised one herd of cattle and requested Francis extend his contract by two months while he found the cattle. Francis agreed to the arrangement although he was not happy with the wage amount of $30 that was proposed by his employer. At the end of the first month, Francis' employer indicated that he did not have sufficient money to pay him his wage, and offered him a blanket and a small suitcase instead of the money owed. Francis reluctantly accepted the payment in kind. In the second month Francis’s inquiries about his promised cattle and pay were quashed. Not long after his enquiries, Francis was very surprised when he was arrested around on allegations that he had raped his employer’s daughter. He was detained in police custody, where he was assaulted by the police officers and forced into confessing that he committed the offence. He was subsequently released into his brother’s custody. He managed to get his wage after taking up the matter with the Ministry of Labour and Social Services. 39

FINAL REPORT: Promising Quality Assessment August, 2013 Francis came to the Legal Resources Foundation (LRF) office for help. In our interview with him, we went through the Promise Card and we promised him that we would represent him in court and that our services would be for free. We then went through the Ask the Expert questions to assess his psychosocial wellbeing. Our assessment indicated that he was feeling stressed as a result of the false accusations and criminal charges being laid against him. We represented him at trial with the defense that the crime was fabricated because he had tried to claim his payments. LRF highlighted the inconsistencies in the allegations made by Francis’ employer and ultimately applied for discharge at close of the state case and the application was granted resulting in the acquittal of the client. In closing the file LRF administered the Measuring Legal Assistance tool which scored 100%. Francis indicated that he was very satisfied with LRF’s services as they had provided assistance on a matter which had caused him stress for long time. LRF re-administered the Ask the Expert tool and it indicated that the client was happy – an improvement from the first measure. He is currently doing a course in carpentry in the Lowveld. LRF proceeded to close the file in conjunction with the Department of Social Services officer who was handling the matter. The main challenge LRF encountered related to the administration of the Ask the Expert Tool. As legal professionals with no psychology training or experience, LRF staff was unable to draw any meaningful conclusions from the responses that were provided by the client. LRF therefore suggests that the tool be administered by Social Workers who are experts in assessing the psychosocial wellbeing of juveniles. CASE STORY: CATCH Promises are kept as Tawanda accesses justice, a family life and education Tawanda (not his real name) is a boy aged 16. He has no brothers or sisters and both his parents died in a car accident when he was 15. When the lease on his family house expired, he was evicted because he could not afford to pay the rent. Tawanda had lost touch with his relatives. He was forced to drop out of school because he could not afford school fees and was living and begging on the streets. It was not easy to get enough each day for a meal. He started stealing as a means of survival. On one occasion he was caught shoplifting by a shop assistant who called the police. Tawanda was arrested and taken into custody at Harare Remand Prison. There he met a Legal Aid Officer (LAO) and a Case Management Officer (CMO) from CATCH on one of their weekly visits to Harare Remand. CATCH goes regularly to identify new cases of children who are in custody. Upon hearing of Tawanda’s case, CATCH officers responded within 24 hours – this is CATCH’s promise. The CMO explained to Tawanda how CATCH could provide him with legal aid and psychosocial support. Tawanda was given a Promise Card as a commitment of the services that would be offered to him. As it is one of CATCH’s Most Critical Interventions (MCIs) to remove children from custody, the CATCH officers began the process of negotiating for Tawanda’s immediate release into their custody and an application was subsequently made to the Magistrate for his release. Since Tawanda lives on the streets, the CATCH officers liaised with the Department of Social Services, to arrange a place of safety for Tawanda to stay. The Magistrate ordered that he be temporarily placed at Training Institute. CATCH was given the responsibility of ensuring that Tawanda would attend all court hearings on the stipulated dates; failure to meet these conditions would result in a warrant for arrest being issued. While Tawanda was at the Training Institute, CATCH carried out several assessments and support visits. In the first visit, the CMO made a special effort to establish rapport with him, and provide counselling. The CMO also carried out case profiling using the Case Profile Tool. The collected information informed Tawanda's case plan as well as his

40

FINAL REPORT: Promising Quality Assessment August, 2013 legal defence. The LAO provided Tawanda with legal counselling sessions which informed him about how the justice system operates as well as the implications and consequences of his actions and expectations about his case. In the second visit, the CMO administered the Ask the Expert Tool that gave information about his psychosocial wellbeing. This gave the CMO ideas about relevant and pertinent areas on which to focus his counselling and intervention plan and establish any other services needed to improve his emotional and social wellbeing. As the case proceeded to trial, CATCH made sure that Tawanda was present at every hearing and that he was provided with free legal representation. In the meantime, the CMOs worked tirelessly to trace any of his relatives, knowing that once the case was concluded in court, Tawanda could not continue living at the Training Institute. Through determination and dedication the CMO identified an uncle who was happy to hear about the son of his late brother. He agreed to look after Tawanda should the outcome of the court permit. On one of Tawanda’s support visits, in-between court hearings, the CMO took Tawanda to meet his uncle and facilitate family reunification, which was successful. The court case was concluded and Tawanda was acquitted. He was provided with post trial counselling and officially reunited with his uncle and extended family. Unfortunately the uncle was financially unable to assist Tawanda with the costs associated with him returning to school. CATCH therefore referred him to alternative education options through Junior Achievement Zimbabwe where he was provided with Life Skills and Entrepreneurship Training that equipped him to start a small-scale poultry business.

CASE STORY: Leonard Cheshire Zimbabwe Trust Accessing the Justice system, no matter what language! (Rural) Jess (not her real name) is in her early teens and lives in a rural area of Zimbabwe. Last year Jess reported that she had been abused by a neighbour. No one else saw or heard the attack happen, so there wasn’t anyone who could be a key witness in court. On learning of Jess’ case in late 2012 the Leonard Cheshire team, responded within 72 hours putting the child’s best interests first, as this is one of the Most Critical Interventions in their Access to Justice Program. The trial was conducted in a Victim Friendly Court, where there are facilities designed to be friendlier for children and survivors of sexual abuse. All those who were not from Jess’ family were asked to vacate the court room, a deliberate attempt to make it less traumatic for Jess. Since birth Jess has not been able to hear. She has never been to school. The team at Leonard Cheshire Zimbabwe Trust sourced a Disability Expert to assist Jess and her mother with sign language interpretation at all stages of the court process including at the pre-trial stages when Jess and her mother needed to communicate with the police. The court proceedings were lengthy, and burdensome – and involved a number of support workers. Jess’ original interpreter was unable to interpret questions in cross examination, due to complexities in the use of her informal sign language. Another strategy of relay sign language was tried, where a hearing impaired adult interpreted Jess’ language into formal sign language then to English for the court. The court initially accepted this interpretation method but as is often the case with informal language the interpreter was known to Jess’ family and so, the court decided to dismiss his use on grounds of bias. Yet another sign language interpreter was suggested by the court to assess Jess. This expert’s assessment explained that Jess uses GESTUNO sign language and advised that this type of communication would be most effective if Jess was taken back to where the incident occurred, a process likely to require additional support for Jess.

41

FINAL REPORT: Promising Quality Assessment August, 2013 It takes a lot of courage and strength to continually be present in court. Leonard Cheshire’s support helped Jess and her caregiver at every step with the required resources, to make sure her case was heard and her rights ultimately respected. They made sure Jess and her mother could get to and from the court each time they were required and that they had food to eat on those days and knowledge about other support services that exist. The support required for Jess to exercise her right to access the justice system extended over a long period of time before the case was concluded. A Leonard Cheshire staff member explained the organisation’s Promises (using the Promise Card) to Jess’ care giver, her mother who relayed them to Jess. Jess’ mother was also asked about Jess’s wellbeing, guided by the Ask The Expert tool which was administered twice, at different stages in Jess’ and her mother’s participation in the program. There was a notable difference despite the frustrations involved in presenting the case. All of Jess’s records were filed and the record was assessed using the PQ Case File Check List. The case is still pending and the program is yet to attend to some of the child’s needs, so the file scored 85%.

NAP II THEME: SEPARATED CHILDREN CASE STORY: Child Protection Society (CPS) Ask the Expert in Children’s institutions, children reminding us of their rights (Urban) In an institution where children are living away from their families, children gave feedback on their wellbeing when interviewed using the Ask the Expert questionnaire. Their responses related to their education, their participation in community life (being limited in a group home setting), freedom from abuse, issues related to staff training in child protection response and their right to worship and play. Importantly, this feedback from children helped to identify critical training needs for caregivers and staff, within the framework of the National Residential Child Care Minimum Standards. These trainings have been implemented at two residential child care facilities – with plans underway to scale up to all children’s homes within the province. The feedback from children highlighted day to day concerns such as bullying and the right to play and religious expression. The children’s feedback provided a vehicle for CPS to respond directly, putting in place group counselling for children, and staff monitoring systems to prevent re-occurrences. The ‘expert’ feedback also identified the risks related to children in boarding style accommodation, and the urgent need to seek appropriate family care alternatives. Adventist children indicated that they were unable to observe their Sabbath due to the delegation of chores in the home, also highlighting the ‘right to play’. This was discussed with staff and a more appropriate approach for children has since been adopted. NAP II THEME: CHILDREN AND WOMEN AFFECTED BY ABUSE CASE STORY: Family Support Trust (FST) Ask the Expert – supporting a child survivor of rape to get comprehensive support (Urban) A 1734-year-old girl named Esther (not her real name) arrived at Family Support Trust (FST) with her husband after being attacked by four men. Esther was returning from church with her husband and his parents when they were 34

Prior to the adoption of the new Constitution in 2013 which sets the age of marriage at 18, girls were allowed to marry at 16 or younger under customary law

42

FINAL REPORT: Promising Quality Assessment August, 2013 attacked by a group of men armed with machetes. Esther’s husband and parents managed to run away from the armed men but Esther was raped by the group. Upon arriving at the clinic, the staff assessed Esther’s psychosocial wellbeing using the ‘Ask the Expert ’. Esther scored 46 out of 70, which was considered low, particularly in the areas of interpersonal skills and social involvement. The team therefore placed Esther into a peer support group for additional therapy. Esther was examined by the doctor and evidence of physical and sexual abuse including genital trauma was documented. Condoms were not used during the forced intercourse and Esther was not on any birth control method. As Esther had presented at FST she was tested for HIV within 72 hours, and post-exposure prophylactic anti-retroviral medicine was immediately administered. After a week, Esther returned for a review and answered the questions in the Ask the Expert; she showed some improvement with a score of 52 out of 70. Esther told the staff she was coming to terms with the situation and that her husband was supporting her. In the peer support group meetings, Esther identified with the issues under discussion and could relate with the other group members who had experienced similar cases of such attacks. Sharing her experience enabled her to regain strength, build connections and friendships with the other girls and re-gain her interpersonal skills. Esther showed great psychosocial improvement. The case was discussed in the case conferencing sessions with the Department of Social Services and it was also noted that her husband also required support through men’s forums. He was referred to relevant men’s services’ organisation. One of the perpetrators of the attack was convicted of rape and sentenced to 7 years in prison and the other three perpetrators are still to be apprehended. CASE STORY: Childline Ask the Expert – supporting a child survivor of rape to get comprehensive support (Urba Ask the Expert – supporting a child survivor of rape to get comprehensive support (Urban) Yolanda (not her real name) is 13 and lives with her paternal grandmother and uncles. Yolanda’s, mother died and her father works in another town. Yolanda has been repeatedly sexually abused by one of her uncles. Yolanda’s situation came to the attention of her neighbour, after Yolanda told her neighbour that she had been raped for the fifth time by her 19 year old uncle. The neighbour took Yolanda to her grandmother in the hope that her grandmother would do something about the rape and protecting her grandchild. However, instead of assisting the child, the grandmother began to beat her. The concerned neighbour proceeded to report Yolanda’s case to the nearby police station, but she was referred to another police station as the alleged crime had occurred outside that station’s jurisdiction. The neighbour then went to the next police station as advised, but was informed that there was no Victim Friendly Unit Officer at that particular station. After many failed attempts to assist Yolanda, the neighbour decided to contact Childline through its freephone Helpline 116. She felt that the child was in danger and needed urgent help. Childline received the call regarding Yolanda in the morning, and a Social Worker completed a home visit in the afternoon of the same day to verify the child protection report received. Yolanda explained to the Social Worker that her uncle raped her, about her fear of her grandmother and her confidence in her neighbour. The Social Worker contacted the Department of Social Services (DSS) and the local Victim Friendly police to ensure that Yolanda was safe, and that the alleged perpetrator was arrested. Yolanda received a Promise Card as an assurance of the services that Childline and DSS could provide. Childline and DSS conducted a joint visit the following day, involving counselling sessions with Yolanda, and her family, including her uncle. The family counselling sessions revealed ongoing threats being made to Yolanda’s safety and emotional wellbeing in her home. Yolanda was also referred to Family Support Trust and a medical examination was completed that same day which confirmed that she had been sexually abused. 43

FINAL REPORT: Promising Quality Assessment August, 2013 DSS and Childline contacted a family member as a possible alternative care arrangement for her, but this was unsuccessful. Instead her grandparents were willing to protect Yolanda during her court case. Several follow up home visits were made and counselling sessions were held with the child where she was given an opportunity to complete the Ask the Expert questionnaire. The questionnaire confirmed that Yolanda was unhappy, and that the fallout from the conviction of her uncle was having a negative impact on Yolanda’s home life. Counselling was offered to console the child and not to blame herself for the uncle’s arrest. Ongoing psychosocial support is still being offered to her. It was hard for Yolanda to answer the questions on the Ask The Expert tool, she had challenges in reading and writing, and the instructions were difficult for her to follow. The Social worker found it hard to understand her responses. The Social Worker also noted that the cultural sensitivities related to traditional protocols where children must be thankful for services. This tradition may have influenced her responses on the feedback to the promise card.

44

FINAL REPORT: Promising Quality Assessment August, 2013 NAP II THEME: NATIONAL CASE MANAGEMENT PRACTICE SYSTEM DEVELOPMENT AND PRACTICE CASE STORY: World Education International Tatenda’s leg is healed: Case management supporting access to medical treatment (Rural) Tatenda (not his real name) is 10 years old and lives in Rushinga. He has been living with a septic wound for about two years. His family believed that Tatenda had been bewitched and therefore traditional healing remedies were the best way to cure his wound. Even if Tatenda and his family were interested in medical treatment, accessing that kind of help was difficult; it is expensive to travel the 100 km to Bindura hospital and to pay for the services. A lead case care worker, a community outreach worker within the case management framework identified Tatenda and alerted the Department of Social Services about his case. The DSS, through the Case Management Officer (CMO), assessed Tatenda, and realized that the he was in pain and that getting his wound treated was a medical emergency. Together with Tatenda’s mother’s participation, a care plan was designed for him. An Assisted Medical Treatment Order was given to him, which allowed him to access free treatment. Tatenda and his mother were also given bus fares to travel to and from Bindura Hospital from the Emergency Fund under the case management budget. After receiving hospital treatment for 3 weeks, Tatenda’s wound healed. It was noted that the area where the wound had been, had lost sensation. As such, Tatenda was referred to a dermatologist in Harare, 190 km from his home. The CMO escorted Tatenda to Harare for the dermatologist treatment. The wound that had made the child suffer for more two years was healed in a space of a month with sensation returned. NAP II THEME: SOCIAL MOBILISATION (Family Clubs) CASE STORY: Concern World Wide Zimbabwe Charity’s improved parenting through family clubs (Rural) Charity (not her real name) has been a resident in one of the urban suburbs in Zimbabwe for the past 20 years. She lives with two of her children and two of her grandchildren. Charity’s husband passed away, and she has struggled to support her family ever since. She was forced to take on additional work in order to feed and send everyone to school. The additional financial pressures and limited personal time meant that Charity struggled to communicate effectively with her family and felt frustrated. She beat the children when they misbehaved. Charity’s children and grandchildren were frightened. Understandably it became harder for everyone to manage; the children ceased talking to Charity about their concerns. Charity found this painful, but did not have the strength or resources to communicate with her family or community to ask for help. In February 2013, the Family Clubs project was introduced in Charity’s area. After learning about the purpose of the program, Charity was excited and became a member. She has been involved in the group discussions ever since. Some of the topics of group discussions have included, communicating with children, children’s rights and responsibilities and safe ways to discipline children that does not involve violence, such as beating. Since taking part in the Family Clubs’ program, the communication between Charity and her family has improved. She now takes the time to listen to her children and address their concerns. Charity’s children now approach her and freely discuss whatever issues they are facing. Charity has learnt to talk calmly to her children, in a friendly manner, without anger and without physical violence. The change in her behaviour has improved the environment for the whole family. The Family Clubs program has taught Charity the importance of spending time and sharing information with her children and their rights.

45

FINAL REPORT: Promising Quality Assessment August, 2013 Furthermore, Charity joined together with other Family Club members to start a savings account, which is used at the end of each month to purchase groceries for each family. This collaboration has improved the health of Charity’s family, as each member is now receiving a better allocation of food. Charity is now aware that her children’s and grandchildren’s future is brighter as they are aware of their rights, and that her children will be able to share the knowledge they have learnt from Charity to their own children. NAP II THEME: CHILDREN WITH DISABILITIES CASE STORY: J. F. Kapnek Trust Nompilo and his mother access to disability services (Rural) Nompilo (not his real name) is 4 years old and lives in the rural areas of Zvimba district with his mother. He was recently diagnosed with hearing and sight impairment conditions. Nompilo’s mother got to know about the Children’s Rehabilitation Program through an outreach team in 2011. After learning about the program, she visited the Children’s Rehabilitation Unit where Nompilo was assessed and his condition explained, along with the types of support services available to her family. These services included training for caregivers to try and make every day life easier as well as support groups where they could meet and communicate with parents and children living with similar disabilities. Since Nompilo is young, the J.F Kapnek Trust Promise Card was given to Nompilo’s mother. The rehabilitation officer carefully explained each of the promises that the Children’s Rehabilitation Unit makes to them as participants in the service. Having understood the promises, Nompilo and his mother, joined activities organized by the outreach team, including caregiver workshops on hearing and sight impairment, activities about daily living, as well as the home program for caregivers. The J.F Kapnek Trust Monitoring and Evaluation (M&E) team recently caught up with Nompilo and his mother at a beneficiary feedback session exercise. Using a semi-structured interview guide, an M&E officer asked Nompilo’s mother if she had been given a Promise Card and she confirmed she had. Nompilo’s mother then answered questions relating to the set promises; highlighting that most of the promises had been honoured except for one promise: ‘…provide referrals to places for help’. Nompilo’s case had been referred to another organization to assist in obtaining a hearing aid but, so far, the device had not been received. So, although Nompilo’s mother was happy with the services provided by J.F Kapnek Trust, this promise on the card, remained un-kept. The case was referred to the social worker to organize for a follow up case conference with the other organisations to map a way forward for Nompilo.

46

FINAL REPORT: Promising Quality Assessment August, 2013 NAP II THEME: CHILDREN TO BE CONFIDENT AND SAFE CASE STORY: AfricAID Gaining confidence and participating in case planning Anesu is 17 years old, has been living with HIV since birth and lives with her paternal aunt. Her mother died of HIV related illnesses in 2005 and her father left the country for greener pastures in 2002 and has stopped communicating with the family. Anesu sat her Ordinary Level examinations and passed only one subject. She wanted to go back to school but always faced challenges with the school fees. Her problems have left her so dejected that all her confidence has been eroded. She was introduced to the Africaid Zvandiri program in 2012 and started attending the activities offered. As a new member of the program, she was asked to fill in the Ask the Expert tool, Confidence Tool and SSQ tool to assess her mental health as well as her capacity to cope with her HIV positive status. Additionally, she was also given the Promise Cards so that she is aware of the promises that Africaid and the other service providers under the National Action Plan for OVC are making to her. Anesu had been admitted recurrently at various health institutions for HIV related illnesses. She was often noted to have been failing to adhere to her treatment. When she became a member of the program, she was recommended to be switched from the ART first line to second line. She was given support through counselling and training and has responded well to the ARVs and her CD4 count has gradually improved. On initial review at Africaid, Anesu was withdrawn and reluctant to share her experiences. She explained to the counsellor that she had no hope for her future and did not see any reason for her to live. However, six months later she began showing signs of improvement in the way she related with her peers. She then expressed interest in helping her peers and suggested that she could do so by joining the Community Adolescent Treatment Supporters (CATS). She went through the selection process and was trained. She is now ready to start working as a peer counsellor in her community. Anesu pointed out that the initial review, when the tools were administered, was the start of a new life for her. She indicated that the Ask the Expert tool made her realise that she needed to work towards building her confidence as she scored very low marks in terms of how she interacts with her friends, family and the community. When the Ask the Expert tool was administered for the second time she scored highly and she indicated that it really showed the improvement in her life. As a member of the CATS program, she was trained in basic case management processes and she indicated that this was something she would like her peers to also be part of as it had helped her so much from the time she joined the program. She indicated that the case management meetings that were held between the Africaid team, herself and her aunt were very critical as she also realised her family loved her very much as opposed to what she thought about them. During one of her visits, Anesu stated that, “I would like to thank my aunties and uncles for making me a better person and every time I look at the promise card I was given at the support group, it reminds me of how you have fulfilled all the promises you have made to me.” Anesu added saying that she now looks forward to the future and will also be working towards improving her academic qualifications. This case is an example of a child who has seen the benefits of the promising quality by the Zvandiri model through the various tools. She has attributed the short time in which she has managed to get back on her feet to the sustained case management process since she became a member of the program.

47


Annex IV: FEEDBACK WORKSHOP AGENDA

[see separate file] Annex V: DATA TABLE (see separate file)

48


Annex V: NGO FEEDBACK ON REPORTS

49