TRAINING STUDENTS FROM PEDAGOGICAL SPECIALTIES OF TRAKIA UNIVERSITY (STARA ZAGORA, BULGARIA) TO DEVELOP ELECTRONIC TESTS

Background: The evolution and application of digital didactic tests for the evaluation of the knowledge and skills of pupils has become a major element in the academic training of students who opted for pedagogical specialties. This reflects the strong propensity for the optimization and improvement of the efficacy of teaching. This article puts forward a scientific research as well as a case study on first-hand experience in training students from pedagogical specialties in academic diagnostics and examination, together with practical tasks geared towards the formation of professional digital competence with regard to the employment of ICT in diagnostic procedures through classical, face-to-face, and mixed-form training. Objectives: The main aim of the study is to examine particular characteristics and opportunities, and to analyze the efficiency of the training provided for the development and design of electronic tests by future teachers in the course of the scholastic training. Methods: The experiment involved qualitative and quantitative research. The following techniques were applied: observation, content analysis, and mathematical and statistical methods of data processing. The statistical methods employed were the nonparametric Mann-Whitney and Kruskal-Wallis tests. Results: The survey included 150 students from three specialties in the Faculty of Pedagogy at Trakia University of Stara Zagora. The framework and composition of the e-learning course and the details of the face-to-face follow-up training are explained briefly. Aggregated data from the learning management system’s, Moodle’s, logs is also provided alongside representative observation data, and a summary with remarks on the findings from the practical classes with the students is also presented. Conclusion: A comparative scrutiny of the learning outcomes was conducted and the principal dependencies and typical merits and demerits of the two forms of training were identified and portrayed in the article. The variation in the end results was found to be statistically significant. UDC classification: 378, DOI: https://doi.org/10.12955/pss.v1.41


Introduction
Over recent years, the profile of Bulgarian students has changed considerably. The educational setting now comprises of students from different generations, with different social experiences, different skills and knowledge, different motivations for studying, different years of receiving school education, with different textbooks, different values and interests, and a significant difference in language as a means of communication. It is, thus the goal and duty of the higher educational institutions to aid all students in the development of certain professional competencies in order to establish their capability in the labor market. (Delcheva, 2018). According to the Higher Education Act of the Republic of Bulgaria, the education process in tertiary institutions is carried out in compliance to the training documentation, which includes a number of mandatory documents, such as specialty curriculum and individual subject curricula. Every discipline consists of credits that outline the hours from lectures, practical trainings, seminary studies, independent studying, examinations and other forms of assessment that the institute requires. (Higher Education Act, 1995). The Act also specifies the different types of distance learning. (Section 11, from 2004).

Characteristics of the Digital Course in the Subject "Development (Design and Creation) of Digital Tests: Working with Hot Potatoes"
The program for electronic education of the Trakia University of Stara Zagora emanates from the learning management system, Moodle (http://moodle.org). It was created with funds from a project called, "Development of Electronic Forms for Distance Learning" (2013Learning" ( -2015. The system provides the following services: chatrooms, discussion forums, wiki pages, workshops, single file uploads (by students), message boards, etc. The study units are arranged in the following hierarchy: course, topic (module), and learning activities or exercises.
A typical learning procedure involves a trainer-initiated registration in the course, and a free access to learning activities and resources. The electronic course in the subject of "Development (design and creation) of electronic tests: Working with Hot Potatoes" was advanced as a resource for training future elementary school teachers in digital and professional technical competencies, in three bachelor's degree disciplines: Preschool and Elementary School Pedagogy, Elementary School Pedagogy with a Foreign Language, and the Pedagogy of Education in Information Technology. The course is also listed in two master's degree programs -Elementary School Teacher, and Elementary School Pedagogy with Information and Communication Technologies. With regard to general curriculum, the course is an optional facility with a duration of fifteen hours (five weeks). It aims at introducing some of the major theoretical (educational and technological) aspects of pedagogic electronic tests for assessing the knowledge of pupils. The first part of the course outlines the means of assessment and the assessment as a significant factor for control and feedback. The second part focuses on displaying the practical use of the software and the opportunities, benefits, and technicalities of working with the software. The course content is divided into three central topics, which are further organized into sub-topics, independent work and answers to control questions. The completion of the course depends on a course project, such as developing a sample electronic test. It can be opted for independently, or in combination with a course called, "Pedagogical Diagnostics: Developing didactic testsmethodology, theory and technology", of which the subject matter and primary modules were developed in compliance to the current standards for digital courses at the Trakia University. Training in this optional study is done by a single lecturer, who also authors the concept and the electronic resources. The lecturer also makes the course curriculum, and the course is scheduled as a type three mixed training, based on the classification by Jara and Mohamed (2007). During 'face-toface' training, the job of the associate professor is to ensure that the contents provided by the electronic course have been mastered by each student, and that the acquired knowledge qualifies to serve as a foundation for the next phase of the formation of functional skills in the subject.

Profile of Student Groups and Course Organization
For the full-time Preschool and Elementary School Pedagogy (PESP) specialty groups, the course is offered during the second year (fourth semester). In the last four years, the number of students in Preschool and Elementary School Pedagogy has risen significantly, from 50 to about 70-80 people in a class. This calls for a prompt optimization of the educational process, especially with regard to the usage of electronic education. With full-time students, training is conducted once a week for a total of five weeks. Students are assigned specific tasks to exercise individual use of the electronic course, which are integrated into the practice sessions in the faculty course. The following drawbacks were recorded while working with larger groups (by way of analyzing the electronic system logs, monitoring student activity during practice sessions, and from individual conversations with students):  A fairly small amount of students participate in the preliminary preparation for the course which involve the use of electronic resources. These students are the typically responsible and wellprepared type.  Feedback is received only from active participants, which does not assure the fulfillment of educational goals for all the participants from the group.  As much as 45% of the students do not prepare in advance for the practice sessions, or prepare only formally and indifferently.  Students focus their efforts mainly in the procurement of theme knowledge and activity algorithms, hardly paying any attention to the objectives and outcomes of the course or to the planning or requirement of the individual work.  During the practice sessions, the students have to work in pairs, owing to infrastructural limitations, which lead to ambiguous responsibilities and obligations.  In about 10% of the cases, a student might be absent for more than two 'face-to-face' sessions.
For the full-time Elementary School Pedagogy with a Foreign Language (ESPFL) specialty groups, the course takes place during the third year (fifth semester). The structure of education in the practice sessions is similar, but the number of students in a group is comparatively small (up to 22). Practice sessions are conducted once a week for five weeks, with a duration of three hours. The faculty course includes 'face-to-face' trainings. The following observations were made:  Up to 50% of the students do not prepare in advance for practice sessions, or prepare only formally and randomly.  They, sometimes, work in pairs during the practice sessions.  A significant proportion of the students might be absent from more than two 'face-to-face' sessions; For the groups from the Pedagogy of Education in Information Technology (PEIT) specialty, the course is offered during the second semester of the first year. The following observations were recorded:  In contrast to other groups from other specialties a smaller amount of students participate in the preliminary preparation for the course and there is no feedback.  Around 70% of the students do not prepare in advance for practice sessions.  Each student works independently on a computer in practice sessions.  Plenty of students are absent for several consecutive 'face-to-face' sessions.

Analysis of Current Results from the Training
The study sample size was estimated on the basis of an investigation on the entries of the electronic course for a three-year period (204 users). Users that had been registered but spent less than 90 minutes in the course (2 study sessions) were excluded. The final sample size comprised of 72 students from the full-time PESP specialty (48.0%), 62 students from the ESPFL specialty (41.3%) and 16 students from PEIT specialty (10.7%). The technology for procuring knowledge and developing skills, in order to use the software for electronic tests, is based on dynamic algorithms. The total study time for the formation of skills -3 lectures and 9 study hours, is enough to start, and to a certain extent, stabilize the process. The following was revealed during the observations on the ongoing work of the students:  It was noted that, in accordance with the Krathwohl taxonomy, the students displayed skills, mainly of the "imitation" and "manipulation" types (Krathwohl, Bloom, Masia, 1964). Towards the end of the course, a certain level of "combining" and "naturalization" is achieved, but not in totality and not by all the students.  The behavior that students exhibit is observed at two different states "receiving" and "reaction", respectively. There is an incomplete manifestation of "compliance", which is a mass and unprovoked abiding of the rules of conduct. The students also try to explain and justify their activities. At the time of the analysis of the operational versions of the exercises and assessment forms, created by the students during the course, it was observed that whilst working with all the five modules of 'Hot Potatoes', the generated problems had differences in terms of contents and functioning and they generally targeted different cognitive stages. Most problems tested the extent of "knowing" and "understanding", except for one problem, which measured the level of "application". At this stage, the main challenges that the students encountered were:  Inaccurate choices and inappropriate matches of targets and issues.  Errors when working with multimedia objects, and faults when working with absolute and relative addresses.  Difficulties in the creation of quality feedback forms contained within the exercises.  Trouble when combining the individual exercises into a common framework. Finally, the students took a practical exam on the development of electronic tests. They naturally succeeded in the designing and formation of the electronic tests and they generally followed the guidelines, but their most common weaknesses in the submitted course works (151) were:  Good application of individual problems, but the test was submitted in separate files instead of an archive.  Some forms of the problems were not included.  Invalid linking to multimedia objects.  Navigation faults in separate problems or in the overall structure.  Good implementation, but the test interface was not localized.  Project files were submitted instead of working exercises.
A check, for the purpose of reviewing the statistical importance of the discrepancies in the compiled training results, was carried out by means of employing a non-parametric K-Sample test of Kruskal-Wallis, and a Post Hoc test of Mann-Whitney as a non-parametric alternative to a one-factor analysis of variance, for comparison between the three categories (specialties). The prerequisites for the utilization of the statistical test were met and two hypotheses were formulated: H0: There is no difference in the distribution of the training results of the course between the groups. H1: There is a difference in the distribution of the training results. To identify the exact nature of the difference, i.e., to find out which groups are distinct in a statistically significant way, we needed to conduct a supplementary Post Hoc test by Mann-Whitney, one for each pair of independent variables from the groups (specialties). We applied Bonferroni correction (coefficient p) and determined the effect size coefficient (r). The results from the Kruskal-Wallis test presented that there was a statistically significant difference between the training results for the three groups. When the results of the students from the Elementary School Pedagogy with a Foreign Language and those from the Preschool and Elementary School Pedagogy were compared using the Mann-Whitney test, we recorded that U = 1118.5; Z = -4.896; p = 0.000; r = -0.43. According to Cohen's scale, the effect size of the training (r = -0.43) was considered typical and slightly greater than the typical (0.30 < r < 0.50). When comparing the results of the Preschool and Elementary School Pedagogy and those of the Pedagogy of Education in Information Technology, we observed that U = 378.50; Z = -2.018; p = 0.044; r = -0.22. According to Cohen's scale, the effect size of the training (r = -0.22) was lower and a little closer to the typical (0.10 < r < 0.30). When comparing the survey answers by Elementary School Pedagogy with a Foreign Language and those of the Pedagogy of Education in Information Technology, we observed that U = 160.50; Z = -4.389; p = 0.000; r = -0.50. According to Cohen's scale, the effect size (r = -0.43) was deemed typical and a bit greater than the typical (0.30 < r < 0.50). With the help of the Bonferroni correction (р = 0.0167) we established that there was a statistically significant difference between the training results of the course between the representatives of the Preschool and Elementary School Pedagogy, the Elementary School Pedagogy with a Foreign Language, and the Pedagogy of Education in Information Technology specialties. The most plausible reason for this difference is that the course is offered to the last group at a very early stage (first year) of their education, when they have only just started to obtain general pedagogical and didactic knowledge.

Conclusion
In the wake of the observations and findings from the process and effects of mixed education, we can say that the methods and techniques used in the process of education and applied by the lecturer are well chosen and competent enough. This is further reaffirmed with the help of relatively overviewed median results, as well as group results from the training. Consequently, future pedagogues develop subjective and methodical professional competencies, alongside digital competencies. As the elements, arranged and sufficiently substantiated during the exercises and course control, we can reliably declare that most students are professionally capable of innovating electronic forms of control and assessment through:  Adequately demonstrated skills for the analysis of training documentation, study programs and diverse variants of handbooks as a normative and informational resource for education.  Sufficiently developed skills for searching, selecting, and, when required, adapting to digital resources for teaching, to take tests, and for assessments. Throughout the study, we explored and revealed the development of digital competence in students. It should be noted that this process largely depends on the individual entry level and the functional digital literacy shaped in high school. Therefore, the findings also consist of statistically significant differences in the performance of students from different specialties. Most students respond satisfactorily to the tasks assigned to them. However, in a few cases we observe refusal to confront a particular issue, with this behavior being the most common in students from the Pedagogy of Education in Information Technology. Even though the final results from the training are reasonable enough, in terms of requirements and milestones, we can ascertain two routes towards future optimization:  To accelerate the shift in the motivation of students to use digital resources for training and tricky content within different subjects; further development of the complicated pragmatically oriented methodical training tasks.  To optimize the productiveness of the forms, methods and techniques for training students in optional subjects and subjects from the general methodological cycle; expand the impact of electronic education in order to improve the overall performance.