A comparative analysis of higher education quality in the global knowledge economy: education in engineering and economics
The study was conducted in four research areas:
1. A comparative analysis of higher education quality in the global knowledge economy: education in engineering and economics,
2. An assessment of Higher Education Learning Outcomes in Economics on the OECD AHELO project methodology basis,
3. An international comparative study of challenges and prospects of rural school development in countries with economy in transition: Russia and China,
4. A monitoring study of the quality of enrolment on the basis of data published on the websites of Russian higher education institutions (both public and private).
The study results of each area are presented in separate parts of the report.
Area 1. A comparative analysis of higher education quality in the global knowledge economy: education in engineering and economics
This work is the third part of a cross-country comparative project (the first one was implemented in 2009, and the second one in 2010), which was initiated by the Stanford University School of Education (SUSE) and aimed at the study of the quality of engineering and economic education in higher education institutions of BRIC countries.
In 2011 the project was focused on the study of the situation in economic education in Russia and a comparison of the received data with the results of the study of Russian engineering education.
The object of the research was Russian higher professional education in economics and engineering.
The aim of the research is to detect the changes in the system of higher professional education in the higher education institutions which train economists, and to make a comparison with the data received in the study of engineering education in Russia, in the context of Russia’s entry into the global knowledge economy.
The empirical basis for the study consisted of data from an expert survey among the lecturers of economic departments at classical universities and industry-specific state higher education institutions of economics and management in Moscow (27 interviews, 4 institutions), and an online poll among the last-year students of undergraduate/specialist’s programmes (87 students, 11 institutions in 6 cities – Moscow, Vladivostok, Yakutsk, Barnaul, Krasnoyarsk and Novosibirsk).
Analysis of the received data confirmed the hypothesis that economic higher education institutions are largely unprepared to train experts for the work in an innovation economy. The following phenomena were detected:
– The professional community lacks mutual understanding of goals and tasks for a modern economist’s training;
– The higher education institutions are largely unprepared to integrate teaching and research work;
– There is no cooperation with market players;
– Faculty members are poorly prepared to work in accordance with third-generation educational standards and are suspicious towards the innovations introduced by their administrations.
At the same time, unlike in engineering universities, in the Russian economic universities the factor of self-identification relative to the evaluation of the Soviet educational heritage does not play a determining role. While the university representatives mention their ‘patrimonial link’ with the Soviet economic school, they do not consider this heritage as a role model and do not see their task in fully keeping Soviet traditions. The university administrations and lecturers are more aimed at accepting and implementing innovations. In addition to that, in economic higher education institutions the methods of teaching are considered to be a more important problem than in the engineering ones. The teachers are generally looking to experiment with the formats of education in order to find those which will better comply with the requirements of the development of competencies necessary for an economist today.
Area 2. An assessment of Higher Education Learning Outcomes in Economics on the OECD AHELO project methodology basis
The research matter
The research surveys the quality of education (learning outcomes) of students who are at the end of a four-year undergraduate degree (Bachelor or Specialist) in Economics programs, offered by Russian higher education institutions (HEIs).
The research objective
The purpose of the research is to assess whether it is possible to measure what undergraduate degree students in Russia know and can do in Economics by using the international AHELO methodology and assessment instruments and to develop the assessment instrument to measure the learning outcomes in Economics and their context.
In 2011 the research was aimed at:
1. the forming of a consortium of Russian participating HEIs, the development of a network of institutional coordinators and the organization of training to learn the AHELO research methodology and instruments;
2. the development (translation, adaptation and verification) of assessment instruments;
3. the testing of assessment instruments in students focus-groups in participating HEIs;
4. analysis of the AHELO tasks on the basis of focus-groups results;
5. presentation of the research results at international and Russian conferences and expert meetings;
6. preparation of recommendations on how to implement further large scale representative research among Russian students following the AHELO methodology.
The empirical base for research
The results of 18 focus-groups in Russian and 3 focus-groups in English realized were used as an empirical base for the research implementation. Totally 204 Russian students from 19 participating HEIs took part in focus-groups.
The results of research
The main results of this stage of research are:
1. A consortium of 20 participating HEIs was created and we formed a network of institutional coordinators. Training for institutional coordinators to learn the international AHELO methodology and instruments were organized. The national research web-site () has been created to communicate with participating HEIs.
2. The international assessment instruments were translated, adapted and verified following the AHELO international guidelines. Instruments to assess the learning outcomes in Economics include 2 Constructed Response Tasks (CRT) and 50 Multiple Response Questions (MCQ). Instruments to assess the context of the quality of education include 3 questionnaires: Student Context Instrument (SCI), Institutional Context Instrument (ICI), Faculty Context Instrument (FCI).
3. Focus-groups in 19 HEIs are organized to test the assessment instruments.
4. The focus-groups results were analyzed by using the Item Response Theory (IRT) and were reported to the OECD.
5. Hypothesizes and recommendations were formulated to implement further large scale research in Russian HEIs.
6. The results of this stage of the research were presented at national and international conferences and expert meetings.
The areas of the research results application
The research results could be used by national governmental bodies and agencies responsible for national quality assurance systems in education as well as by the HEIs for institutional quality assurance purposes.
The results of this stage of the research will be used for the next large scale research among Russian students of participating HEIs.
National and international partnersThe 20 Russian HEIs that were included in the research consortium are the main national partners:
- The Russian Presidential Academy of National Economy and Public Administration
- The State University of Management
- Far Eastern Federal University
- Kazan (Volga Region) Federal University
- Moscow State University International Relations
- Lomonosov Moscow State University
- Moscow State University of Economics, Statistics and Informatics
- National Research University – Higher School of Economics
- Novosibirsk State University
- Plekhanov Russian University of Economics
- Saint Petersburg State University
- Saint Petersburg State University of Economics and Finance
- Northern (Arctic) Federal University
- Yakutsk State University
- Siberian Federal University
- Ural Federal University named after the First President of Russia B.N. Yeltsin
- Federal State-Funded Educational Institution of Higher Professional Education “Financial University under the Government of the Russian Federation”
- Southern Federal University
- Altai State University
- Novosibirsk State University of Economics and Management
The Ministry of Education and Science of the Russian Federation supported the creation of a consortium of Russian HEIs participating in the AHELO project.
The international partners are the Organization for Economic Cooperation and Development which is responsible for the overall management of the AHELO feasibility study and the international Consortium led by the Australian Council for Educational Research (ACER), contracted for the different international research work implementation, that includes:
CAE (Council for Aid to Education), USA
ETS (Educational Testing Services), USA
NIER (National Institute for Educational Policy Research), Japan
University of Florence (Italy)
CHEPS (Centre for Higher Education Policy Studies), the Netherlands
CPR (Center for Postsecondary Research), USA
Area 3. An international comparative study of challenges and prospects of rural school development in countries with economy in transition: Russia and China
The topicality of the international comparative study was largely determined by the challenges the countries in transition – Russia and China - face in the field of rural education.
The object of the study is the aspirations of secondary school senior students and their parents in the pilot regions of Russia and China. Basic elements of the conceptual scheme developed to analyse education, career and migration paths of the rural students were as follows:
1. Economic well-being of families; family’s cultural and social capital; parents’ social and occupational status;
2. Parents’ expectations regarding education, career and migration paths of their children, including realistic assessment of the family’s financial resources to secure further education for children, awareness of the situation on the labour market and the level of social optimism;
3. Students’ expectations regarding the most probable education, career and migration paths;
4. School as a set of factors that determines the students’ socialization and reflects the specifics of the existing educational standards;
5. Regional characteristics which determine employment opportunities for the school graduates in urban and rural areas and opportunities for their further professional training and development.
The aim of the study is to do a comparative research of education, career and migration opportunities for senior schoolchildren in rural areas of Russia and China, as well as of expectations of their parents, to suggest development strategies for regional education systems taking into account family’s preferences in countries in transition.
Three regions have been chosen in each country – in Russia and China – for the study, with various climates, available transport services, levels and profiles of social and economic development, demographic trends and settlement systems; in one of the areas in each of the regions a survey was conducted (in the form of interview and questionnaire survey) of the senior and graduate students in the secondary schools and of their parents.Empirical research database includes two sets of data:
– 2166 questionnaires of secondary school senior students and their parents from rural areas in the following Russian regions: Altai krai, Moscow oblast, Ulyanovsk oblast and the Sakha Republic (Yakutia).
– 1876 questionnaires of secondary school students and their parents from rural areas in the following Chinese provinces: Zhejiang, Anhui and Shaanxi.
The empirical study results show that the most expected by the students education, career and migration path includes further higher education and migration to urban areas – more than 68% of students. A less popular, about 25% of students, strategy is to have secondary vocational education. And the least attractive educational strategy, about 5 % of students, is to continue training in a trade school; virtually none of the respondents will get employed right after leaving secondary school.
Among all the factors of the empirical model – family, environment and school – the family factor has the most impact on the students’ career path, in other words, rural students’ social mobility and their aspirations is largely, by 80-90%, determined by the level of their parents’ human capital.
Based on the empirical results, the study made it possible to bring forward recommendations for development of regional educational systems with regard to the families’ preferences. Prospective development of rural education in the Russian regions is connected with the establishment and promotion of educational centers and social and cultural complexes which prove to be effective models of educational institutions providing cooperation and integration of different rural establishments and communities and reinforcing the parent family capital.
The results of the empirical research conducted in the Chinese provinces show that senior secondary education is in great demand in the rural areas in China; the survey also shows that the students are reluctant to continue their training in technical vocational schools, less than one fifth of the respondents, besides their motivation is very low. The current tendencies in rural areas make it difficult to maintain distribution of the students’ flows between senior secondary school and secondary vocational education at the ratio 1:1 that was imposed by a Central government directive. Currently, this directive looks like an artificial attempt to recover non-competitive secondary vocational education.
The students’ academic performance has a decisive influence on rural students’ choice between senior secondary school and secondary vocational training; the family’s economic well-being largely determines the choice between further employment and secondary vocational training. The study results have implications for the current Chinese government policy to provide financial support and scholarships for students studying in secondary vocational schools in rural areas. Government support would help rural families cover the expenses of their children’s vocational education; and rural students who are unwilling or unable to continue their study in the senior secondary school, on the one hand, and students from poor families, on the other hand, would be willing to have training in secondary vocational school and not go into the labour market prematurely.
The main partners in China:
– the UNESCO International Research and Training Centre for Rural Education (INRULED) and Beijing Normal University. Project leaders - Prof. Dong Qi, Vice-rector of Beijing Normal University, director of INRULED and Prof. Zhu Xiaoman, Deputy Director of INRULED.
Area 4. A monitoring study of the quality of enrolment on the basis of data published on the websites of Russian higher education institutions (both public and private).
As part of the project, Russian higher education institutions’ websites were monitored in terms of their transparency for prospective students (1st stage of the study) and the quality of enrolment was evaluated (2nd stage of the study).
Object of research:
1st stage: Higher education institutions’ website transparency monitoring – Websites of public and private higher education institutions in Russia. Materials published on free access and addressed for prospective students of first higher education (full-time) as well as master’s programmes were analyzed.
2nd stage: Evaluation of the quality of enrolment – Lists of applicants recommended for enrolment and acts on enrolment in the programmes of first higher education (full-time) published on free access at the higher education institutions’ websites. Unified State Examination (USE) grades of the students enrolled by competition in the programmes of first higher education (full-time) on budget-financed places and fee-paying basis were analyzed.
Aim of the research:
1st stage: Higher education institutions’ website transparency monitoring – To evaluate the transparency of the 2011 enrolment campaign; to carry out comparative analysis of enrolment campaigns of 2011 and 2010 as well as to compare the accessibility of public and licensed private higher education institutions for prospective students.
2nd stage: Evaluation of the quality of enrolment – To evaluate the quality of enrolment in Russian universities by means of calculation of the average and the minimal USE grades among the enrolled students. Average and minimal USE grades were calculated by higher education institutions, by areas of study and by consolidated groups of areas of study.
Results of the study:
In general, as compared to 2010, the Russian higher education institutions’ websites have become more transparent for prospective students. There is also an important trend which equalizes the differences in different universities’ websites’ transparency: in 2011 there were considerably less institutions which have not published any information at all or published very poor information.
As compared with 2010, the quality of enrolment: improved in the strong institutions and in ‘strong’ areas of study; worsened in the weak institutions; stayed at the same level or insignificantly improved in the ‘weak’ areas of study. Just like in 2010, higher average USE grades are specific for prospective students of socio-economic and medical universities. The weakest USE grades belong to the students enrolled in agricultural and teacher training institutions.
Practical results of the research and application:
Russian university websites’ transparency ranking
Quality of enrolment in Russian universities ranking
A ‘USE grade calculator’ was created and published on free access. It allows the 2012 prospective students to evaluate their chances of enrolment in the selected institution.