AI-SUPPORTED PEER ASSESSMENT PLATFORMS IN THE METHODOLOGICAL TRAINING OF PRE-SERVICE MATHEMATICS TEACHERS: A COMPARATIVE ANALYSIS AND THE IMPLEMENTATION OF PEERSCHOLAR

Abstract

Introduction. The modern paradigm of higher education is characterized by a transition from traditional hierarchical methods of control to participatory models, among which peer assessment occupies a key place. This mechanism involves the involvement of education seekers in assessing the learning outcomes of their colleagues, which contributes to the development of critical thinking and self-regulation skills. However, the wide implementation of peer assessment in the university environment faces a number of challenges: the subjectivity of the assigned scores, a significant cognitive load on the teacher, and the problem of the “depth deficit” of feedback. The integration of artificial intelligence technologies, in particular natural language processing (NLP) methods, opens up new opportunities for addressing these problems.

Purpose. The purpose of the study is to conduct a systematic comparative analysis of AI-supported peer assessment platforms and assess the effectiveness of PeerScholar in developing assessment competencies among future mathematics teachers in Ukrainian higher education.

Methods. To comprehensively assess the effectiveness of the PeerScholar platform, a mixed-method study design was used, combining quantitative and qualitative data collection methods. The quantitative component included a structured survey of students after the completion of the pilot using a specially developed questionnaire containing 32 questions with Likert scales (1-5 points) and 7 open-ended questions for detailed comments. The qualitative component of the study included in-depth semi-structured interviews with six students selected according to the principle of maximum variability (two students with a high level of success, two with an average and two with a lower than average level), as well as an analysis of peer assessment artifacts (samples of feedback provided by students and revisions of work after receiving comments from other students). Additionally, objective indicators of the platform were analyzed: the average completion time of each stage, the number of revisions made after receiving feedback, the degree of consistency of assessments between different reviewers (inter-rater reliability).

Statistical processing of quantitative data was carried out using the IBM SPSS Statistics 27 package and included descriptive statistics (means, standard deviations, frequency distributions), correlation analysis (Spearman's ρ correlation coefficient to identify relationships between different aspects of platform perception) and assessment of internal consistency of questionnaire blocks (Cronbach's alpha coefficient). Qualitative data from open-ended questions and interviews were processed using thematic analysis to identify recurring categories, count the frequency of mentions of key themes, and select illustrative quotes.

Results. Comparative analysis of peer assessment platforms revealed differentiated EdTech solutions from adaptive systems (ALEKS) to comprehensive reflective ecosystems (PeerScholar, RiPPLE), with the critical role of affordability for Ukrainian HEIs. The PeerScholar pilot demonstrated statistically significant positive student perceptions of the methodological (M=4.36, SD=0.61) and professional-reflective (M=4.35, SD=0.65) aspects of the platform. Strong correlations were found between the identification of alternative approaches and the value of formative feedback (r_s=0.913, p<0.01), as well as between the development of methodological and reflective skills (r_s=0.713, p<0.05). Inter-rater reliability increased from r=0.51 to r=0.71 over the semester, confirming the gradual development of assessment competence. Students showed a higher interest in constructive feedback (M=4.81) compared to receiving points (M=4.42), which indicates a request for professional communication.

Originality. Based on the results of the testing, a number of recommendations were formulated for the effective implementation of PeerScholar in the educational process. First, it is advisable to conduct an introductory training session on the principles of constructive feedback and working with the platform before the start of peer assessment, which will reduce the initial uncertainty of students and improve the quality of feedback. Second, it is necessary to set sufficient time frames for each stage (at least 60 minutes for assessing one work), taking into account the real time costs of students. Third, it is recommended to gradually increase the complexity of tasks throughout the semester to develop assessment skills from simpler to more complex methodological situations. Fourth, the teacher should conduct selective quality control of the feedback provided by students, especially in the initial stages, providing meta-comments on the quality of the reviews.

Conclusion. PeerScholar integration creates a quality environment for professional dialogue, where peer assessment stimulates the transition from routine problem solving to deep methodological reflection. The sequence is revealed: identification of alternative approaches → critical analysis → request for qualitative feedback. Time costs (M=3.40) require planning adjustments, and the technological barrier of the English-language interface reduces efficiency for some students. The study offers practical recommendations for choosing platforms for institutions with limited resources and implementing structured peer assessment in teacher training.