Goal of research: to conduct a comparative research of preparedness of BRIC schools graduates (particularly graduates who are entering engineering majors) for university, and of the quality of their education after two years of university.
Objectives for 2014:
- To develop tools for assessment of students’ knowledge in Mathematics and Physics at the beginning of their education and at the third grade;
- The approbation of the instruments in Russia and China.
Research subject: the quality of students’ training in mathematics and physics, as well as their level of critical thinking.
The expert-oriented approach was used for the test development. This approach allows creating a test with the existing items pools. Test items were provided by participating countries - Russia, China and the USA. All test items were translated to English and then to Russian and Chinese.
Empirical base of research:
A pilot study was conducted in Russia and China in October and November 2014. The study involved more than 5,000 students from 21 universities.
Results of research:
The following toolkits were developed and piloted:
- The general cognitive skills (physics and mathematics) and critical thinking tests for the 1 grade students;
- The general cognitive skills (physics and mathematics) and critical thinking tests for the 3 grade students;
- Contextual questionnaires for students, teachers and administration.
Based on the results of the pilot study, next year we plan to run the main study on the bigger sample. The baseline of the 1st year students will be compared with the results of the 3rd year students, to find their mean progress during the training. There also will be conducted the analysis of factors that are associated with the students’ results and their progress during the training. The particular emphasis will be made on the international comparison between the countries.
Research purpose :
This research direction consists of three parts. One part had rather methodological focus. We were interested in how different approaches to family socio-economic status operationalization relate to discrepancies in research results. Other two parts were devoted to different factors driving secondary school students’ educational outcomes and trajectories in Russia. In the second part we investigated factors of secondary school students’ self-efficacy in a subject. Finally, in the third part we focused on effect of the whole class and referent classmates on high school students’ performance and after-school educational trajectory choice.
Different methods were used depending on the research questions. The first part of the research was based on deep literature review. We analysed approaches to family SES operationalization. For the second part of the study we did regular OLS regressions. For the third part we did social network analysis and series of logistic regressions.
Empirical base of research:
The results of the first part of the study have shown that there is quite a wide range of SES proxies used in social science research. The most frequently used operands are parental occupation and education, indicators of family financial capital (income), cultural capital indicators, such as books in the home and other cultural possessions. Our analysis shows that question about parental occupation has the most parents-child agreement. Question about parental education, on the contrary, has low parents-child agreement and is more tended to be missed by the children. All variables need additional conversion to become appropriate for use in comparative international studies. Number of books in home is a widespread indicator of cultural capital or family educational resources. This question has quite high level of agreement and low non-response rate especially when its categories grouped in bigger ranges. Concerning practical needs of research it can be said that researchers should pay more attention to parents’ occupation rather than education when they operationalize SES.
While doing the second part of the research we found that self-efficacy in math is correlated with some out-of-school factors such as gender and family SES. Among in-school factors the most important is average class achievement in particular subject. When low performing student is in a high-performing class this decreases his or her self-esteem. The same time if teacher uses different math tasks for students with different level of abilities in math this will strengthen big-fish-in-little-pond effect for lower-performing students.
The third part of the study was devoted to peer-effect on performance and educational trajectory choice. We found that student’s achievements in algebra are more related to the whole class average performance, rather than to the referent classmates’ outcomes. However in physics student’s achievements show statistically significant correlation with both class average scores and referent classmates average scores. High performance in physics increases a chance of STEM choice, while high performance in math does not. It could be so probably because performance in math may reflect general ability rather than interest to STEM. Proportion of classmates who are going to enter STEM department in college turned out to be not significantly related to student trajectory choice. However if some referent classmates choose STEM and these classmates are central persons in class network this increases the probability for a student to go to STEM as well. These inferences demonstrate that studying peer-effect only by estimation of average performance in class can lead to biased estimates. Therefore it is crucial to include in-class network characteristics in analysis when peer-effect is in interest.
In order to form the scales, evaluate their psychometric properties, and check the cross-cultural equivalence of latent constructs, two approaches were used, Item Response Theory (IRT) and multi-group confirmatory factor analysis (CFA). The analysis was conducted using Winsteps and MPlus software. Also some statistical methods were used: correlation, ANOVA and descriptive statistics. To collect qualitative methods data for validity study the structural interview were conducted.
Empirical base of the research :
To estimate and describe teachers’ beliefs and practices the NorBA survey was conducted. Also the data from qualitative study were collected.
The validity and reliability of NorBA questionnaire were checked. Two scales emerged, measuring the level of constructivism and the level of traditionalism in teacher beliefs. Both scales are unidimensional, and all items have good psychometric characteristics and fit the model. The validity was estimated by DIF analysis, CFA, correlation with other scales and qualitative methods. All analyses showed a good quality of the instrument.
Differences between math teachers’ beliefs in different countries are statistically significant. Russian teachers are more constructivists than teachers in Latvia and Estonia. Most of teachers in Estonia compromise both approaches: their views about good teaching include construction of knowledge and accept also transmission of knowledge in combination with it. Russian teachers are strongly oriented towards systematic approach to math teaching. It supports the claim that in Russia traditions of high quality mathematics education are still strong; emphasis on rigorous proofs, logic, exact definitions and a precise use of the mathematical language is a characteristic of Russian mathematics education. Teachers’ beliefs are realized in their classroom practice.
This research direction consists of three parts:
The other study was dedicated to developing of a principal framework for word math problems that are intended for assessment of a competence to apply mathematics to real life situations.
Firstly, we conducted an experiment to understand the difficulties that Russian students meet when they solve PISA math problems. Each of them was asked to do two contextual mathematical tasks. Thinking aloud protocols (both concurrent and retrospective) were used to gather data. Protocols and video records were coded and analysed.
Empirical base of the research:
For the first part we conducted an experiment with math tasks and for that purpose 60 students from 8-10 grades were selected in one public secondary school in Moscow. And in order to estimate and describe math teaching practices we used data of TIMSS 2011 and PISA 2012 study in Russia. An ability to apply school knowledge in the non-academic context was considered as a student’s results in TIMSS and PISA.
The results of the first part suggested that partly failures in problem solving could be explained by difficulties in modeling process, that is they were specific for these particular problems. However the vast majority of mistakes reflected the poor general problem solving skills. We found that inability to visualize clearly a disposition of the key task elements has a detrimental effect on performance. Also the need to develop a solving strategy, validate their own solution made difficulties for students. We draw conclusions for teaching mathematics and assessing a competence to use mathematics to solve problems in real life.
In the results of the other study we proposed one possible theoretical model, based the key characteristics real everyday situations where math knowledge would be applicable. We have considered the following model parameters as obligatory: 1) everyday semantic; 2) novelty, untypical for a problem solver; 3) consistency between a question and a context (a situation) of a problem; 4) moderate structuring; 5) requiring some curricular math knowledge for solving. We have shown how the proposed model parameters can be applied to distinguish between different math contextual problems and develop appropriate problems.
Finally, there was conducted analysis of math teacher practices in classes with poor and excellent PISA results as it was suggested that specific teaching practices are correlated with students’ results in PISA. At the first stage, the most and the less effective schools were identified by regression analysis with PISA and TIMSS scores on participants of longitude survey. However, further statistical analysis in those different of classes hasn’t revealed the difference in math teacher practices, and therefore it was suggested to start a series of video-observation in math classes and in-depth interviews with those math teachers.