Позитивные изменения. Том 2, №4 (2022). Positive changes. Volume 2, Issue 4 (2022)
Шрифт:
The award in the Development of Social Projects Evaluation System category was won by the Center for Advanced Management Solutions (CAMS) for the introduction of assessment models into the system of public administration and the publication of methodological materials. In 2020, CAMS and the Accounts Chamber of the Russian Federation published an expert note “Application of the Theory of Change for Strategic Audit and Strategic Planning in Russia.” [17]
17
Kim, A. I., Kopytok, V. K., Filippova, Y. A. & Tsygankov, M. V. (2020). Application of the Theory of Change to Strategic Audit and Strategic Planning in Russia. Accounts Chamber of the Russian Federation, Center for Advanced Management Solutions. Retrieved from:(accessed: 09.12.2022).
As the authors of the document note, there is a gap between the strategic goals of Russia's development, which are formulated by the President of the Russian Federation in his decrees and by the Government of the Russian Federation in its strategies, and the final results. The formal achievement of indicators does not always mean achievement of the originally planned social and economic effect. Moreover, different groups of beneficiaries often do not feel the effects of the goals set and achieved. One of the tools to bridge this gap is the theory of change, an approach in which each instrument of public policy must be focused on achieving a specific socio-economic change [18] .
18
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. New approaches to evaluating community initiatives: Concepts, methods, and contexts, 1, 65–92.
One of the authors of the expert memorandum, Maksim Tsygankov, head of the Program and Policy evaluation project area at the Center for Advanced Management Solutions, shared with the editors of the Positive Changes what important things happened in 2022 from the perspective of evaluation development in our country.
“It is too early to talk about the practice of evidence-based approaches in the public sector, despite the fact that in 2022 it was possible to finalize a number of evaluation projects, such as microdata projects of the Federal Service for Labour and Employment. Thus, detailed administrative microdata on the state of the labour market became available to researchers. Research is expected to be based on this open data, the results of which will help improve organizational and technological processes of employment for citizens and recruitment for employers [19] . In addition, I am glad to see a growing number of code and repositories around the world with the goal of evaluating and analyzing data. There are also cases from Russian companies, including MTS. MTS Big Data specialists have launched the first version of Ambrosia library for working with A/B-tests into Open Source [20] . We see that it is business that is mostly interested in the proof of practice,” said Maksim Tsygankov.
19
Accounts Chamber of the Russian Federation. (2021). A collection of best practices and initiatives in the evidence-based approach to management decision-making. According to the results of the 1st Competition of the Accounts Chamber of the Russian Federation. Moscow. Retrieved from:(accessed: 09.12.2022).
20
Bairamkulov, A. & Khakimov, A. (2022). Ambrosia is an Open Source library for working with A/B tests. Retrieved from:(accessed: 09.12.2022).
On the positive side, according to the expert, we can also add that the Accounts Chamber of the Russian Federation in 2022 mediated the evidence-based policy agenda in the public sector, continuing to hold a competition of best practices and initiatives of the evidence-based approach to management decision-making.
Also, the SOL' Social Innovation Support Center was awarded in the Development of Social Projects Evaluation System category for the introduction of social reporting standards and the theory of change in the activities of social enterprises.
The standard helps various social projects and charities to form a compelling account of their social activities and to formalize the logic of the project's transformative impact. It includes all the data required for reporting, from organizational structure to finances. The standard also provides an opportunity to show the real results of its activities to investors, government authorities and other stakeholders [21] .
“We take a full-hearted approach to developing social innovation, and we believe if there is a problem, there is a solution. The social accountability standard tool allows social innovators to be clearer and closer to the people. I want to add that I spend a lot of time trying to understand the effects of social entrepreneurs, and how to help the state notice social entrepreneurs. The social impact assessment model, which is currently being developed by the Social Projects Support Fund, is still very difficult to use to compare one player with another, but it can be used for self-diagnosis – to see your own “growth zone.” The evaluation can be done by the project itself or by an independent player,” says Alexey Maslov, executive director of the SOL center and director of the Acceleration and Educational Programs Department of the Social Projects Support Fund of the Agency for Strategic Initiatives.
21
SOL SISC. (2014). Social reporting standard. How to prepare reporting based on your performance. Retrieved from:(accessed: 09.12.2022).
There are four winners in the Development of Practices for Implementation of Evaluation of Social Projects category. The first award went to the Timchenko Foundation for developing a standard of evidence-based practices for evaluating social impact in childhood and for shaping approaches to assessing the effectiveness of community development.
In 2018, with the strategic participation of the Timchenko Foundation, the Standard for Evidence-Based Social Practices appeared in Russia [22] . It was developed by specialists from expert organizations, including Center for Evidence-Based Social Design of the Moscow State University of Psychology and Education, Association of Program and Policy Evaluators, Process Consulting company, Institute for the Study of Childhood, Family, and Education of the Russian Academy of Education.
22
Standard of evidence-based social practices in childhood. (2018). Moscow. Retrieved from:(accessed: 09.12.2022).
10 organizations from Moscow, Penza, Cherepovets, St. Petersburg, Novosibirsk, Ivanovo provided descriptions of their social practices (model, technology, methodology, service, etc.) in an evidence-based format and were professionally verified. The format of the practice description implies its brief content, information on the achievement of social results, the quality of the available evidence on the achievement of positive changes in the lives of the beneficiaries, the regulated nature, the validity and good elaboration of the practice concept.
The standard is now a document that is evolving both methodologically and practically. In particular, the provisions of the standard are taken into account by the relevant ministries when developing methodological recommendations for organizations in the field of childhood, and the academic community makes extensive use of the standard's developments.
Another area of the Timchenko Foundation's work is the development of approaches to evaluating the effectiveness of local community development. In 2022, the foundation launched the Solidarity Communities project competition, which aims to solve small area problems by local communities, organizations, and initiative groups [23] .
23
Rudina-Ladyzhets, E. (2022). Communities as a driver of territorial development. The Solidarity Communities project experience. Positive Changes. 48–59.org/10.55140/2782–5817–2022–2-S2–48–59.
Maria Morozova, General Director of the Timchenko Foundation, told the Positive Changes Journal how the foundation created, approved, and independently audited a system for monitoring and evaluating programs.
“We use the ‘theory of change,' assuming how change will occur in the short, medium and long term. As part of it, we developed our own system of indicators, which helps to see the results. Each program has a different one, but they are all governed by the same standards and principles. The monitoring and evaluation system includes all participants in the activities – the foundation, regional operators, and beneficiaries – and this helps all of us develop, make informed strategic and management decisions, and achieve significant social changes,” says Maria Morozova.
The system has a pyramid structure:
• the lower level is the direct results of the foundation's work (number of events, trained specialists, newly developed methodological materials, etc.);
• the middle is the results achieved by the supported projects (changes in the lives of the people for whom the foundation works – children saved in their families, jobs, co-financing for the development of the city, etc.);
• the third, uppermost level is the long-term social impact, the change at the societal level of the situation that underlies the program. This is usually determined through external independent research.