Category: Resourceful Young Children | Formalisation of human resource development | 21 December, 2014 - 22:00← BACK
The Little Elephant Training Centre for Early Education (LETCEE) is a non-profit organisation (NPO) based in Greytown, KZN. Our vision is ‘for every child to develop and grow in a caring community’ and our mission is to build ‘the confidence and capacity of adults so that they will create nurturing environments in their communities.’ This is done in two ways: by providing both Early Childhood Development (ECD) training, and family-based ECD programmes in rural communities. We offer accredited and non-accredited training courses for ECD practitioners working in both centre-based and family-based ECD.
We have secured funding to strengthen our Monitoring and Evaluation (M&E) system, and to provide evidence of the impact of our programmes through evaluations. Our intention is to grow in to a learning organisation by using evidence to reflect, make changes, improve, and celebrate success.
This learning brief describes some of the steps involved in improving our monitoring and evaluation processes. It reflects on the challenges faced as well as lessons learnt regarding implementing and strengthening our M&E techniques and procedures for our Abahambi programme.
The holistic ‘Abahambi’ model
LETCEE runs a family-based ECD model that allows for a holistic approach to child development. Women, known as Abahambi (literally translated as ‘those who walk’), from within the communities, are trained to deliver ECD sessions for children in a family setting. Children who would otherwise be unable to access ECD services are provided with the opportunity for play and stimulation. The Abahambi build relationships with the children, caregivers and family, and provide advice, assistance or referrals where needed. This learning brief focuses on the M&R system as it pertains to the Abahambi programme.
What is M&E?
Monitoring and Evaluation (M&E) is an on-going process of collecting rigorous and systematic evidence. It involves collecting data in various forms such as statistics (numbers) and narrative/documentation from interviews and observations. Data need to be purposeful and intentional and should be used for a variety of purposes such as planning, learning, effective decision-making, and for accountability to funders and reporting requirements (Booth et al. 2001).
Using a participatory M&E approach
We made the decision to take a participatory approach towards M&E. It is participatory in the sense that members at all levels of our organisation should be involved, as opposed to employing outside experts to evaluate our programmes (Booth et al. 2001). A participatory approach also allows stakeholders and beneficiaries to have a voice in the evaluation process, thus embedding the approach within the organisation and enhancing its sustainability. It is inclusive, empowering, helps develop staff skills, and increases the opportunity for our organisation to gain a clear picture of ‘what is really happening on the ground’ (IDS, 1998).
The first steps in strengthening M&E
The initial step in strengthening our M&E system was to hold workshops with LETCEE staff regarding the existing approach. We articulated the intended outcomes of the Abahambi programme and discussed current monitoring techniques and procedures. We looked at participatory M&E, its purpose and its importance. Our discussions made it clear that, at that time, information collected from the field was limited, irregular, and unsystematic. Moreover, our reporting forms were primarily qualitative and took a long time for the Abahambi to complete; and then the data were laborious to analysis. It was hard to distil clear information from these reports and thus the data were not effectively used.
The next step was the development of new tools and processes of data collection that would form the basis of an improved M&E system. We revised the forms and other data gathering tools used to monitor the Abahambi programme. Our new emphasis was on gathering appropriate information, creating a better system for information flow, and using the data to inform practise and decision-making.
Efficiently monitoring the Abahambi programme
We now use two data collection tools to efficiently gather information about our Abahambi programme: the Daily Record Form and the Monthly Report Form. These newly designed tools and how we use them are described below.
Daily Record Forms
In consultation with staff and Abahambi, regarding the type of information we needed to be gathering, we revised the Daily Record Forms. The Abahambi completes the new Daily Record Form at each home session.
The forms are designed to keep a log of the children and family visited, and define the Abahambi home visits. They offer guidance for discussion with families and caregivers, and prompt reflection on child development and Abahambi practise.
During the monthly support visits a community facilitator checks the completion of the Daily Record Forms and tracks fidelity. The community facilitator attends a home visit session with each Abahambi at least once a month. The facilitator can then provide direct, on-site support and advice regarding the Abahambi’s interaction with children, facilitation skills, communication with the family and/or caregiver, and completion of Daily Record Forms. The ECD motivator will also include this support when visiting the Abahambi.
Monthly Report Forms
We have also included a Monthly Report Form that the Abahambi submits at the end of each month, using their completed Daily Record Forms. The Abahambi can use this form to reflect on child development, their own practise as an Abahambi, and highlight cases of concern; or they can use it to track their work for the month.
Prior to the monthly meetings with the Abahambi, the community facilitator collects the Monthly Reports and scans them to see if they are complete and accurate and if the Abahambi is in need of any assistance. The reports are used as a basis for discussion and reflection during the monthly meeting.
Community facilitators return the Monthly Reports to the LETCEE office at their monthly meeting the following week. A meeting is held with the community facilitators from each of the communities where LETCEE works. This is an opportunity for cross-community sharing, problem solving, and peer learning.
The LETCEE administrator captures/inputs the quantitative data onto an excel database and files the Daily Record Forms per community. The M&E officer then summarises all the quantitative and qualitative information from the reports into a community summary report. These reports for each community are then emailed to all members of the management team.
Based on the summary reports and the concerns and issues raised, the management team, along with members of the community development team, create an action plan that includes the following key points: what is the issue; how can it be dealt with; who can deal with it, and by when. The M&E officer and community facilitator are able to provide the Abahambi with feedback about parts of the reports that were well completed, strong examples of child development or Abahambi practice, interesting, or positive stories. The aim here is for further learning, reflection and sharing, and to emphasise that the reports are useful and have a purpose.
We have faced a number of challenges in this data gathering process that involve language and cultural barriers, and diverse educational and literacy levels of the people we work with. Due to these challenges the Abahambi struggle to use the tools properly, and thus we need to regularly confirm that they understand the forms and the importance of gathering accurate information. We provide continual support and feedback where needed.
Another challenge was to work out the most appropriate and effective flow of information, and to ensure that the information is used to the full. After re-working our flow of information a number of times we realised that accountability amongst staff members requires the information be made more readily available and easily accessible to all. Consequently we now send out the summary reports as attachments on a group email to all staff members. We also decided that a dedicated M&E staff member would be needed on a permanent basis, and hired an appropriate candidate.
Suggestions for others implementing new M&E systems
We learnt that flexibility and adaptability are important aspects of an M&E system. When implementing a new M&E system accept that things will change and that things sometimes go wrong. Make changes to correct the mistakes and streamline operations. Think carefully about each step taken to ensure rigorous data is collected, and work within the priorities of your organisation. If essential data are collected and used properly, management will have the evidence to make informed decisions, and the confidence to make changes.
If rolling-out a participatory M&E approach you will need to begin by first building your staff’s capacity to participate. This takes time and is an on-going activity. M&E is a continuous process and takes time to become embedded in your organisation. Therefore, regularly review the implementation progress and constantly reflect on and refine your project expectations, data gathering tools, reporting systems, and feedback protocols.
Through this process of strengthening the Abahambi M&E system we found that we increased the number of visits to our project sites for monitoring sessions and evaluations. This helped to raise awareness of our project within the community. The increased site visits by diverse LETCEE staff provided an opportunity to gain a deeper knowledge of the community and its people, and of the services, assets and challenges within that community.
Some individual staff members have seen much personal growth from being part of the M&E process. Some began to recognise the potential and resilience of the people working and living in a deep rural community. Others have come to realise that the Abahambi work under extreme situations yet remain committed to their daily tasks. And still more have a greater understanding and appreciation of the living and working conditions in these communities. Some individuals reported a strengthening of their capacity, such as improvement in observation and communication skills, increased patience, better ability to adapt, and greater respect for people and diversity. This increased personal capacity translates into more organisational capacity.
This process made us aware of the importance of M&E and so we committed to hiring an M&E officer who can focus on strengthening the systems we’ve put in place so far. This office is dedicated to ensuring the system runs smoothly, receiving reports, analysing information, and extracting trends. Evidence collected from the new systems put in place is now used for informing management meetings, decision-making, and financial and project accountability.
In committing to strengthen our participatory M&E system we have learnt a lot from the implementation as well as from the processes of learning, observing, and reflecting. The result is a system that allows us to make informed decisions, creates richer transparency and accountability across all levels of our organisation, and provides useful, relevant information that can drive strategic planning. Through participation, negotiation, learning, and flexibility we have a stronger system that gives a variety of people within LETCEE a voice. As an organisation we can learn from these many voices.
Booth, W., Ebrahim, R. and Morin, R. (2001) Participatory Monitoring, Evaluation and Reporting. An Organisation Development Perspective for South African NGOs. Braamfontein, Pact [Produced with support from the United States Agency for International Development in South Africa 1998]
IDS (1998) ‘Participatory Monitoring & Evaluation: Learning from Change’, Policy Briefing, Issue 12, available online at: http://www.ids.ac.uk/files/dmfile/PB12.pdf (accessed 28 February 2014)
130 Voortrekker Road, Greyton
(033) 413 2736
This learning brief shows that through participation, negotiation, learning, and flexibility organisations can build a stronger monitoring and evaluation system that gives a variety of people a voice. Learn from LETCEE how to strengthen you organisation’s M&E systems.