EVALUATION OF EFFICIENCY OF HIGHER EDUCATION INSTITUTIONS’ ACTIVITIES IN UKRAINE

Inna Vlasova Candidate of Economic Sciences (Ph.D.), Associate Professor, Institute of Higher Education of the National Academy of Educational Sciences, Ukraine e-mail: i.vlasova@ihed.org.ua, orcid.org/0000-0003-3532-3136 Olha Palamarchuk Candidate of Educational Sciences (Ph.D.), Institute of Higher Education of the National Academy of Educational Sciences, Ukraine e-mail: o.palamarchuk@ihed.org.ua, orcid.org/0000-0002-5703-5496


Introduction
The issue of evaluating the effectiveness of higher education institutions is relevant in the European Higher Education Area (EHEA), as evidenced by analytical reports of the European University Association (EUA) (Estermann T., Kupriyanova V., 2019;Estermann T., Kupriyanova V., Casey M., 2018;Pruvot EB, Estermann T., 2017) and in Ukraine (MES: Conceptual principles of reforming public funding and management of higher education institutions, Strategy for reforming the public financial management system for 2017-2020 and others).
There is an urgent need to develop target performance indicators. In Ukraine, in 2019, the Verkhovna Rada of Ukraine adopted and submitted to the President a draft Law "On Amendments to Certain Laws of Ukraine on Improving Educational Activities in the Sphere of Higher Education" (Reg. № 2299 of October 22, 2019). One of the innovations of this bill was the introduction of target performance indicators of higher education institutions, the achievement of which must be ensured by a person (candidate) as the head of a higher education institution in case of signing a contract. These targets are indicators of the effectiveness of the heads of higher education institutions. Also there are draft laws in progress: "On Amendments to the Law of Ukraine" On Higher Education "to change the management system of higher education institutions" (Reg. № 8385 of 18.05.2018) and "On Amendments to the Law of Ukraine" On Higher Education "and other laws of Ukraine on the financing of higher education" (Reg. 2170 from 24.09.2019). Under these conditions, the problem of finding adequate mechanisms for evaluating the effectiveness of universities is relevant.
The purpose of the article is to develop a system of indicators for evaluating the effectiveness of Ukrainian universities based on the analysis of EUA metrics, indicators of other organizations, leading international rankings, national metrics and indicators.

Analysis of metrics of the European University Association
The report of the European University Association (T. Estermann, V. Kupriyanova, 2019) examines the effectiveness of activities at the systemic and institutional levels, efficiency in academic issues (research, teaching and learning) and efficiency in strategic management. The report offers key recommendations messages in order to increase the efficiency of universities and the metrics of complementary components of the university autonomy.
The indicators of academic activity of universities in the report include: -Rationalization of the academic offer.
-Joint study program.
-Technology enhanced learning.
-Coordination of study portfolio.
-Coordination of student flows.
-Sharing of teaching space.
-Joint application services, student recruitment and mobility.
-Sharing or exchange of teaching staff. We fully share the authors' view that efficiency is a collective responsibility of all higher education stakeholders. Effective and efficient universities and frameworks can only be achieved through ongoing dialogue and joint action by politicians, universities and their networks.
In the report "University Autonomy in Europe III: The Scorecard" (University Autonomy in Europe III: The Scorecard, 2017) in the field of academic autonomy Scorecard focuses on the academic activities of universities: student work, program implementation, language of instruction, quality assurance and curriculum development to ensure the efficiency of universities. In order to assess the effectiveness of universities, considerable attention is paid to issues of academic autonomy, in particular: -total number of students; -mechanisms of reception; -entry and termination of graduation programs; -the ability to choose the language of instruction; -the possibility of choosing quality assurance mechanisms; -the possibility of developing educational content; This report pays more attention to ensuring academic autonomy and the implementation of its components, but these components can be taken to develop performance indicators for the universities.
The report of the European University Association (EUA), "Quality assurance: a reference system for indicators and evaluation procedures" (European University Association, 2004) reveals the content, types and features of the use of performance indicators, as well as procedures and models of quality assessment in some European countries. In the context of our study, it is valuable to consider the following indicators in the field of university governance and management. Namely (European University Association, 2004): Composition of governing bodies (availability, number and method of appointment of scientific and pedagogical representatives, other employees, students, graduates to the Board, Senate or research committees).
Mechanisms for recognizing student participation in the governing bodies of the university. Mechanisms for allocating budget resources in accordance with the strategic goals of the university development (availability of mechanisms and responsibility for the effective use of budget allocations).
Diversification of financial resources (budget allocations, tuition fees, competitive research funding, research contracts with private or public organizations, donations from individuals or organizations).
Mechanisms of institutional strategic planning (definition of strategic directions of activity, responsibility for development the strategy of activity).
Staff turnover ratio (scientific and pedagogical representatives). Mechanisms for developing interdisciplinary educational programs (number of programs with interdisciplinary components, number of scientific and pedagogical representatives participating in interdisciplinary activities, number of interdisciplinary research centers, etc.).
Institutional adaptability (number of the new educational programs, average number of hours spent on developing new educational programs).
Openness of the university to the external environment (the number of social events in which the university participates, the number of scientific and pedagogical representatives involved in such activities; the number of new jobs created as a result of research, etc.).
This report does not contain indicators of the effectiveness of financial activities of universities, but the group of indicators called "Resources for teaching and learning" contains a financial indicator -"The ratio of the operating budget per 1 student." In addition, the listed mechanisms and tools include not only managerial but also financial (diversification of financial resources) and academic components (number of the new educational programs). This demonstrates, on the one hand, an integrated approach to evaluation and, on the other hand, the difficulty of assessing the effectiveness of university governance / management and identifying specific measurable indicators.

Analysis of indicators of other organizations
A valuable document for the analysis is the Teaching Excellence Framework (Teaching Excellence Framework, 2016), which is based on six key indicators, three of which are taken from the results of the annual National Student Survey (NSS). These indicators reflect the student's vision of the quality of teaching, assessment and academic support received. The fourth indicator of teaching excellence is related to the level of student withdrawal, and the fifth and sixth -relate to the graduates, particularly the level of satisfaction with teaching and learning and the level of their employment in the future.
Quantitative indicators: -Teaching on the course.
-Employment / further study. -Highly skilled employment or further study. Qualitative indicators: -Quality of teaching.
-Student outcomes and Learning Gain. Another important document for the analysis in the context of our study is "The Higher Education Code of Governance" (UK Committee of University Chairs, 2020). This paper identifies key values and elements that form the framework for effective governance in higher education. The values / principles of effective governance are the following: integrity, sustainability, inclusivity, excellence, innovation and growth, community. Key elements of governance in higher education include: accountability, sustainability, reputation, inclusion and diversity, efficiency, engagement / involvement.

Analysis of international ratings
Let's look at the main international rankings to find out if they take into account the performance indicators of higher education institutions.
The leading international rankings "QS World University Rankings" (QS World University Rankings, 2021) and "Academic Ranking of World Universities" (Academic Ranking of World Universities) lack indicators for assessing the effectiveness of organizational and financial activities of universities.
The Times Higher Education World University Rankings 2021 contains the following indicators of the financial performance of higher education institutions, namely: Institutional income (2.25%); Industry Income (knowledge transfer) (2.5%); Research income (6%).
Of the above indicators, the largest share is "Research income" (6%), which indicates its importance. Indicators for assessing the effectiveness of organizational activities of the university are not present in this ranking.
Another international ranking, THE Impact Rankings 2020, assesses the performance of universities in the context of UN-defined sustainable development goals. According to this ranking, the comparison of universities is carried out in three broad areas: research, education and good governance in the light of the 17 UN Sustainable Development Goals. This ranking reflects the impact of universities on social and economic development. Each goal contains certain indicators in the field of university governance, financial / economic performance and social impact, in particular: by component "decent work and economic growth, SDG 8": Expenditure per employee (15.4%). By component "industry, innovation and infrastructure (SDG 9)": Research income from industry (38.4%). By component "peace, justice and strong institutions, SDG 16": University governance measures (26.6%): Elected representation on the university's governing body (3.35%); recognition of a students 'union (3.35%); policies to engage local stakeholders (3.35%); participatory bodies to engage local stakeholders (3.35%); publish university principles on organized crime, corruption and bribery (3.35%); policy supporting academic freedom (6.6%); publish university financial (3.25%).
Of the listed indicators, University governance measures have the largest share of "Policy supporting academic freedom" (6.6%), which indicates its importance.
Let's move on to the indicators of academic activity presented in international and national rankings. Thus, we see that indicators such as the ratio of teachers and students, the academic reputation of the university and the assessment of the quality of teaching and learning are the same for international rankings. Also common to The Europe Teaching Rankings (The Europe Teaching Ranking, 2019) and for the Formula is the employability of the graduates. There is no indicator of admission to the university in international and national rankings.

Analysis of national metrics and indicators
Regarding national metrics, it is advisable to consider national rankings and the formula for allocating state budget expenditures on higher education among higher education institutions.
The National Academic Rating of Higher Education Institutions of Ukraine "Top-200 Ukraine 2020" (Nezalezhne otsiniuvannia universytetiv) was developed in accordance with international standards and contains the following indicators for assessing the performance of universities: QS World University Rankings -academic activities; Scopus -research and publishing activities; Webometrics -evaluation of research achievements of universities through comparison of their Internet sites; participation in Erasmus + programs of the European Union -international activities; Google Scholar Citations -citation of scientific works of scientists of the university; UniRank -the quality of presentation and popularity of free economic resources in the Internet space on the basis of independent webometric indicators; results of All-Ukrainian student Olympiads and competitions of scientific works 2018/2019 (by the amount of points)quality of training; scholarships of the President of Ukraine and the Cabinet of Ministers of Ukraine for young scientists -educational and scientific work; the number of patents received by scientists of the university in 2019. -inventive activity; The average weighted value for the ratings of free economic zones in terms of the number of applications submitted by entrants and the average competitive score in 2019 is the attractiveness of the university for entrants.
At the same time, this rating does not contain indicators of organizational and financial / economic performance of higher education institutions.
Resolution of the Cabinet of Ministers of Ukraine "On the distribution of state budget expenditures between higher education institutions on the basis of indicators of their educational, scientific and international activities" (Pro rozpodil vydatkiv derzhavnoho biudzhetu) dated December 24, 2019 № 1146 approved the formula for distribution of state budget expenditures on higher education between institutions of higher education. This formula contains a comprehensive indicator of the activity of the i-th institution of higher education of state ownership (Ai) and is calculated by the following formula: where RKi -the estimated contingent of applicants for higher education who study on the terms of the state order; Mi -an indicator of the scale of activity; FPi -an indicator of regional support; No -an indicator of scientific activity; MBi -an indicator of international recognition; PVi is an indicator of employment of graduates.
The indicator of scientific activity of the i-th institution of higher education (Ni) is determined depending on the amount of income to the special fund based on the results of scientific and scientific-technical works on international cooperation projects, the results of scientific and scientific-technical works under economic agreements and the results of scientific services per one scientific and pedagogical worker at the main place of work (ONPi) on average for the previous three calendar years.

Conclusions
Thus, possible indicators for assessing the effectiveness of organizational activities of higher education institutions may be the following: -Research and teaching staff working in the Free Economic Zone at the main place of work -the share of the total number of research and teaching staff, %.
-Scientific and pedagogical workers who work in the Free Economic Zone at the main place of work and have a scientific degree / degree of Doctor of Arts and / or scientific (honorary for artistic Free Economic Zone) title -share of the total number of scientific and pedagogical workers, %.
-Ratio of the number of scientific and pedagogical workers with a scientific degree / degree of Doctor of Arts and / or academic title to the number of applicants for higher education, %.
-Staff turnover rate (scientific and pedagogical representatives) -the ratio of the number of dismissed scientific and pedagogical workers to the total number of employees, %.
-Communication with employers -the share of higher education applicants who have an internship at enterprises outside the Free Economic Zone, of the total number of higher education applicants, %.
In order to assess the effectiveness of financial / economic activities of higher education institutions, the following can be used: -Total revenues of free economic zones per one scientific and pedagogical worker, thousand UAH.
-Share of grants, charitable contributions in the total revenues of the Free Economic Zone, %.
-Special fund of free economic financing -share in the free economic budget, %.
-The amount of funding for free economic zones in 2020 (except for: 1) funding to provide in accordance with the legislation of social benefits and other expenditures; 2) reserve, except for the adjustment of the total amount of funding taking into account the lower and upper limits of funding compared to the previous year), thousand UAH.
The development of indicators of the effectiveness of academic activities of the university should be individual. The selected indicators should correspond to the mission of each university and at the same time include a common set of indicators to assess its effectiveness.
Based on the analysis of international and national rankings, we propose to take as a basis the indicators of the effectiveness of academic activities of the university from The Europe Teaching Ranking, which in our opinion most widely presented indicators of academic activities and expand its author's vision.
The system of indicators of efficiency of academic activity of the university are the following: -The ratio of academic staff and students; -Scientific and pedagogical staff involved in professional development activities (number of academic staff who received certificates of professional development); -Admission of students (annual ranking of universities by number of students (entrants)); -Employment of graduates (satisfaction of graduates with the acquired knowledge, satisfaction of employers with the received specialists); -Involvement of external experts in the teaching process (percentage or share of international academic staff, involvement of employers and other stakeholders in the educational process); -Mobility of academic staff and students (number of internships and exchange of international experience among academic staff and students); -Competitiveness of the university (the place in international rankings and the ability to compete with leading world-class universities); -Assessment of the reputation of the university (academic reputation is determined by the presence of a higher education institution in international rankings or top 10 in the national ranking "Top 200", the quality of educational services); -Assessment of teaching and learning (determined by the student's survey, rating of academic staff).