Financial Empowerment and Monitoring and Evaluation

DNA Economics was commissioned to conduct a comprehensive review of the Monitoring, Evaluation, Accountability and Learning processes on the Sireletsa Bana Project administered by Save the Children South Africa. The project consisted of two distinct but interrelated components. The first component focused on the assessing the quality of data. The second component of this evaluation was a formative evaluation to understand the progress in delivering on the results set out in the logical framework.

DNA Economics was asked by Zenex Foundation to evaluate the implementation of the Gauteng Literacy and Numeracy (LitNum) Strategy: “2015 and beyond”. This strategy aimed to improve the curriculum management practices and behaviours within schools.

DNA Economics subcontracted to Ducharme Consulting to develop an institutional analysis, logical framework and indicators for the Performance Expenditure Review on Government Fleet Services.

This national evaluation focused on the National Qualifications Framework. Commissioned by the Department of Planning, Monitoring and Evaluation, this evaluation assessed the extent to which the National Qualifications Framework Act had been implemented as intended. Given the complex nature of the National Qualifications Framework, DNA Economics used a complexity-responsive approach to analyse the effectiveness of the framework.

Bertha Centre tasked DNA Economics and Alternative Prosperity with assessing the feasibility of an Outcomes Fund for Johnson and Johnson. As part of this project, the DNA team looks at the design and possible structure of an Outcomes Fund in the education sector. The project involved identifying the social problem that could be addressed through an Outcomes Fund, designing the structure of a fund and developing models for performance-based payments to implementing agents.

DNA Economics provided technical support to JET Education Services who were commissioned by the Quality Council for Trades and Occupations (QCTO) to develop a business case, including position paper and costing model, for the future activities of the QCTO. In its role as quality council, the QCTO takes overall responsibility for the qualification development, quality assurance and certification of occupational (workplace) training in South Africa. This business case allowed the QCTO to clearly set out and cost its future operational model as part of a request for the creation of a more secure and sustainable funding stream for the organisation that will enable it to better execute its strategic responsibilities.

This evaluation of the NECT’s District Improvement Programme (DIP) Learning Programme (LP) was commissioned by the Zenex Foundation and implemented by Social Surveys Africa and DNA Economics. The evaluation assessed the extent to which the DIP was implemented as planned and achieved its intended outcomes and impacts. The evaluation draws conclusions on the implementation of the programme and its emerging impacts. Lessons from this evaluation form the basis for recommendations on how the LP and the DIP can be improved, sustained and scaled up beyond the current interventions and provinces.

This study combined a theory-based implementation and impact evaluation. The impact assessment used a quasi-experimental design that measures the differences between the matched treatment and comparison groups. Work on the evaluation began in October 2016 and was completed in July 2019.

DNA Economics helped to design the Blue Ladder School programme, a holistic intervention to improve the quality of learning and teaching in schools. As part of this project, DNA Economics developed the theory of change for the intervention, held validation workshops, designed the monitoring and evaluation frameworks and conducted the annual programmatic reviews.

This was an economic evaluation, that used cost benefit analysis to determine whether the benefits arising from the investment in forensics services outweighs the cost. DNA Economics would cost data on the implementation of the incremental investment programme within the South African Police Services. Once this information was collected, DNA Economics identified how fiscal resources were spent and where possible monetise the incremental benefits of this investment.

This evaluation focused on the Incremental Investment in Forensic Services, under the Seven-Point Plan approved by Cabinet in 2007. The Seven-Point Plan seeks to build capacity in the Criminal Justice System (CJS) to tackle the high levels of crime in South Africa. Forensic Services was chosen as the focal point for the additional funding because it influences the effectiveness of subsequent investigations, prosecution, and judicial proceedings within the CJS.

The South African Police Services (SAPS) and Department of Planning, Monitoring and Evaluation (DPME) have commissioned an evaluation of the Incremental Investment in Forensic Services (IIFS). Through the IIFS, the SAPS received an additional R6.2 billion in funding between 2009/10 and 2014/15. This funding was made available to recruit forensic personnel, train existing staff, purchase specialised equipment, modernise systems and fund the operations of the Division: Forensic Services. This evaluation sought to understand the effectiveness of the Incremental Investment in Forensic Services in improving criminal justice outcomes.

Assisted Statistics SA in writing up a sectoral report on Goal 8 for the Millennium Development Goals Report (2010, 2013 and 2015) (South Africa). This involved reporting on South Africa’s progress towards its goals within the context of other national indicators and targets, in line with global MDG standards and principles. The 2015 MDG report also focused on the transition to the Sustainable Development Goals post-2015.

The Gauteng Government has adopted a Township Economy Revitalisation (TER) strategy to build and strengthen township economies. This strategy sets out numerous action plans across several focus areas. DNA was contracted to undertake a formative evaluation of this strategy in order to test the theory of change for the TER Strategy; validate the connections and assumptions across each level of the underlying results chain; and identify a set of indicators which can be used to monitor the performance of the strategy against this theoretical framework.

This study examined the factors that drive the workload of South African Directors General; and how they allocate their work across the various demands on their time.The study used a mixed methods approach: including an analysis of their diaries and time allocated to different meetings, interviews with key informants and political principals.

DNA Economics was tasked with reviewing the Gauteng Planning Commission’s Programme of Action and its indicators. For this project, DNA Economics reconstructed the Programme Theory for each of the ten Pillars of the GPC’s POA. By applying a indicator evaluation framework, DNA Economics was able to assess the quality and robustness of each indicator in the POA. Based on this assessment, the project team proposed changes to existing indicators.

DNA Economics was appointed to design and refine the Front Line Service Delivery Monitoring tool used by the Presidency to monitor service delivery across home affairs offices, health clinics, schools, police stations, SASSA offices, licence testing centres, magistrate’s courts and municipal offices. The project involved designing a rating tool and manual for monitoring staff in the Presidency. As part of this project, the DNA team piloted the revised tool and refined it based on the findings.

Contracted to undertake a review of the KZN Agribusiness Development Agency (ADA). The review had two broad objectives: (1) To evaluate the economy, efficiency and effectiveness of ADA and its project management processes; and (2) To reflect on the Agency’s performance and identify improvement opportunities for the organisation and its project management processes. The methodology incorporated a literature review, review of KZN ADA’s documentation, stakeholder consultations and a visit to (and review of) 6 ADA project sites.

DNA Economics was contracted to conduct a performance evaluation of specific elements of the Zimbabwe Strategic Economic Research and Analysis (SERA) Program in order to inform the future programming of USAID economic interventions in Zimbabwe and its economic policy work elsewhere.

DNA Economics was contracted by USAID South Africa to undertake a final evaluation of the USAID-funded Development Grants Programme (DGP) environment portfolio. Four projects in four different countries (South Africa, Swaziland, Lesotho and the Seychelles) were selected for evaluation. These projects focused on strengthening regional capacity for utilization of clean energy technologies and expanding key stakeholder capacities to adapt to anticipated climatic changes.

DNA Economics was commissioned to analyse the Presidential Hotline Database of about 160,000 complaints across 14 sectors. This project involved categorising complaints by sector, analysing key trends in complaints and identifying common themes. The project also did an assessment of the complaints mechanisms in government departments to understand their effectiveness. As a final output of this project, DNA Economics produced an improvement plan that outlined some systemic changes that could be introduced within the government to improve the handling of complaints.

DNA Economics was part of the WYGI-led team appointed to evaluate TradeMark Southern Africa (TMSA); a GBP100m, 5-year programme funded by the UK’s Department for International Development (DFID). Matthew Stern of DNA was the team leader on this evaluation and was responsible for coordinating all activities and a team of 8 evaluators and technical specialists.

DNA Economics was contracted to undertake an assessment of the National Liquor Act (2003), focussing on whether the policy goals outlined in the Act have been achieved since the implementation of the Act in 2003. Specifically the study focussed on the regulation of manufacturing and distribution of liquor, the production of illicit liquor and the employment of minors and the extent to which the Act has successfully reduced the socio-economic harms associated with liquor abuse. The study comprised of desktop research, stakeholder consultations and focus group research of informal liquor retailers.

DNA has been commissioned to design an impact evaluation framework for BANKSETA, and to test this framework across 5 of BANKSETA’s priority skills development programmes. This has included a review of international evaluation techniques in the training sector, the development of impact or logic chains for each of the programmes, the design and implementation of a survey of 200 learners and 50 corporate beneficiaries, and the calculation of the return on investment from these programmes.

The purpose of this assignment was to conduct a performance evaluation of the USAID-funded Southern Africa Regional Environment Program. The specific objectives included:

• To document the degree to which USAID-funded environmental project interventions through Chemonics Inc. worked, or did not work;
• To identify best practices, lessons learned, and areas of improvement; and
• To provide recommendations to inform USAID’s follow-on interventions toward improved management of the Okavango river basin, with emphasis on institutional strengthening, biodiversity conservation, water and sanitation, and climate change adaptability, and assess what will be required to achieve existing strategic goals and results.

The findings from this evaluation informed USAID’s decision to extend the projec for a further year.

DNA was contracted by the DPME to identify and plot the logical framework for four implementation programmes of the Department of Trade and Industry: Small, Micro and Medium Enterprise Development; the Cooperative Development Programme; Export Promotion; and the National Liquor Authority.

SEDA’s Technology Programme (STP) was developed to provide a form of targeted support to entrepreneurs and small enterprises.The STP programme has become one of the main avenues through which technology based small enterprises can seek and obtain support. SEDA plans to further expand on this programme in order to increase the depth and coverage of its incubation support services.
In line with the proposed expansion, DNA Economics undertook a comprehensive assessment of Seda’s incubation programme through an evaluation of the performance of six existing incubators and an international benchmarking exercise.
Through the international benchmarking the study identifies key aspects of incubator design and performance that ensure the implementation of successful incubation programmes and, in turn, providing an enabling environment for clients. Six incubators selected by SEDA were evaluated and assessed in a number of areas to determine their level of performance and determine the most suitable incubation models to achieve SEDA objectives.

DNA was contracted to undertake a comprehensive review of all evidence of the Johannesburg Development Agency’s (JDA’s) outputs, outcomes and impact from existing reports and other supporting quantitative and qualitative information; and to report on progress made against the JDA’s strategic, developmental and delivery objectives.

The National Income Dynamics Study (NIDS), the first national household panel study in South Africa, was launched in 2006. The first wave of this survey was undertaken by the Southern Africa Labour and Development Research Unit (SALDRU) at UCT, and the data was released to the public in July 2009. It was hoped that this data would serve as a resource for researchers, academics and policy makers. The main purpose of this review was to assess the performance of NIDS to date, and to recommend ways in which the study could be improved.

Participated in the formal mid-term review of the DFID-funded Commark Trust.

Assessment of the status and impact of the untying of official development assistance (ODA) across seven donor projects in South Africa and Zambia as part of a larger international review of the Paris Declaration commissioned by the OECD DAC.

An assessment of the current programme of action to inform the development of a new tool for political accountability that will allow Cabinet to monitor progress of key programmes within National Government

DNA was requested, by TIPS, to provide a quick and independent assessment of the 15 proposals received in response to the call for host institutions for the SADRN (Southern African Development Research Network) thematic working groups (TMGs). DNA developed an assessment framework, evaluated the 15 project proposals, and recommended four applications for further consideration.

A review of the regulatory environment of Non-Profit Organisations for the NPO Directorate in the National Department of Social Development. The methodology used combined evaluation and regulatory impact assessment techniques, and included primary research among NPOs. The impact of a complex, multi-jurisdictional regulatory environment was unpacked and regulatory recommendations were made.