E-evaluation of professional skills: Comparative study of different technological choices by video conference: Case of the CRMEF of Taza

Introduction

The COVID-19 pandemic was a determining factor in changing the philosophy of teaching, training and learning assessment in Morocco, thus benefiting from the contribution of digital technology and while trying to overcome some constraints related to face-to-face training, a new system of online training and assessment is implemented.  

The CRMEFs like all educational institutions have adopted exclusive distance training during the confinement period, followed by a hybrid mode organized by Ministerial Note N ° 42 of the year 2020. distance saw the light of day following the decree 2.20.474 which defined the conditions, the actors, and the orientations in order to legitimize the training and the remote evaluation in Morocco.          

Faced with a multitude of choices of techniques and methods, the trainer is called before addressing his trainees , to set his educational scenario, and more precisely in the case of a course in virtual mode (online) it is essential to answer the following central questions: what is the technological choice which will guarantee more chance to evaluate the learning of the trainees in virtual mode ? And what approach (time / staff) can be adopted to respond to the characteristics of the context and the conditions of learning and evaluation of the trainees.      

In this regard, given the interest that technology represents for providing online courses and assessments in difficult contexts, we opted in this research for the comparison of four technological choices in videoconferencing used by trainers in the assessment. (Diagnostic-formative and summative `` oral '') of teachers and trainee administrators (zoom , Google-meet , Microsoft teams and Skype) and this with a fundamental objective of identifying the advantages and disadvantages of each technological tool on the one hand and to guide the trainers and trainees towards choices adapted to the real levels and needs in order to set up technical-pedagogical scenarios making it possible to guarantee an optimal quality in the training.   

Our comparative study was based on an experiment made by our sample made up of 42 trainers of TAZA Centre belonging to three different primary training cycles , qualifying and the educational administration cycle with 251 trained (63 primary ; 80 qualifying, 108 administrators), we thus administered two online questionnaires, the first for the trainers and the second for the trainees, in order to carry out a survey on the advantages, the constraints and the functional and spatiotemporal adaptations of each technological choice.   

We will try through this work to verify the satisfaction and the obstacles of this mode of evaluation on the one hand and to model the benefits of each videoconference tool in the process of evaluation and development of the skills of the trainees on the other hand go.

1. Theoretical frame

To understand and contextualize our research, we will first of all present a definition of the concept of evaluation while specifying the types and modes used in the training of trainees, Thus in a second part, we will frame the digital concept while presenting a modeling of technological choice in videoconferencing

Finally, we will define the different training methods from enriched face-to-face to absolute distance.

• Evaluation in training

Evaluation is a polysemous concept, it is reduced for most users to a simple expression of evaluation on paper format which is used to detect a mark or a classification of learning. an assessment of skills in an initial training, the concept of '' evaluation '' requires a lot of resources and investment, and covers several dimensions while respecting a set of criteria namely:  

• Formative evaluation: it is a process evaluation that is part of a continuous improvement process. It in turn has two dimensions :
• Live evaluation: that is to say, the training evaluation process is carried out immediately after the training action. The purpose of this form of evaluation is to measure the progress as well as the adjustment of the learning path (Aichi Y et al., 2020).
• Cold evaluation: this involves evaluating the training action after its completion in a few days or weeks. Its purpose is to determine the achievement of the training objectives expressed in terms of professional skills.
• Summative evaluation : in the training mode summative evaluation is characterized by the certification function, it is generally placed at the end of the training and its final objective is to validate the competence of the training.
• Self-assessment : this method of assessment remains accessible and available for those trained, especially with the use of new technologies and the flashback of assessment situations (Astalini D et al., 2019)
• Co-evaluation : the use of videoconferencing tools offers opportunities for social support and learning by assigning immediate or subsequent evaluations in a group or pair work; in short, this mode favors the evaluation of social learning.
  • Technological choice in videoconferencing

A videoconference is defined as a teleconference allowing, in addition to the transmission of knowledge and graphic documents, the transmission of moving images of distant participants.

Videoconferencing software, sometimes called videoconferencing, web-conferencing or more simply “visio” - are computer tools that make it possible to establish live connections, between two or more distant parties, via the Internet, in video mode, to simulate face to face meetings, courses or assessments. (Baneres D et al., 2016)

Videoconferencing then makes it possible to bring together in a virtual way fromes who would not normally be able to meet face-to-face with their trainers without the condition of appearing in class .

The different components of a videoconferencing system are :

• A network for data transfer. This is usually a high speed internet connection
• Video cameras or webcams that provide video input.
• Microphones: external or integrated into the user's device (smartphone, PC, Mac, conference room equipment).
• A screen, monitor, television or projector to broadcast video output.
• Headphones, in-device speakers, or external speakers for audio output
• And a software solution for encoding and decoding (codecs) and encryption.

These last points constitute the subject of our article based on the choice of software or in other words the technological choice, in this sense we advance the idea that there are several conference vision software, we tried in our training to adopt those which work for free for economic reasons in the majority of the trainees (Marjan G et al., 2013).

• Remote assessment (e-evaluation)

The issue of remote evaluation is discussed since 1850, date of the 1 st correspondence course in France by Emile Pigier. By Jean-Pierre Lehnisch . In the professional sector, remote evaluation is a system included in the training which allows training without going to the training place and without the physical presence of a trainer. In other words The evaluation of knowledge and the learning activities take place outside the direct face-to-face relationship, known as “face-to-face” between the trainer and the trainee (trainees) (Beatty B., 2010). 

Several definitions have identified this new paradigm of remote evaluation among which we distinguish:

''Distance assessment a mode of judgment involved in a distance education system that allows everyone to work independently, at their own pace, wherever they are, in particular thanks to the techniques provided through technology'' (Chantal C., 2019).

• Types of remote assessment

According to several researchers have distinguished between four types of distance training namely (Charlier B et al., 2006):

• Improved face-to-face : a hybrid form between face-to-face and online, also called work-study training.
• The enriched face-to-face training: The face-to-face training time here concentrates the heart of learning, and therefore remains online teaching practices
• The reduced face-to-face: a very open form of open online distance learning
• The non-existent face-to-face : long distance training par excellence .
  • The characteristics of a remote assessment

Distance learning is an organized, finalized training device, recognized as such by the actors, which takes into account the uniqueness of people in their individual and collective dimensions.

Learners are then called upon to perform the tasks that were recorded at home and hand them over electronically or hand to hand to their training organization (France L et al., 2021).

• The Benefits of Remote Assessment

 Among the advantages demonstrated by the authors of this evaluation model we cite:

• The learner works individually in permanent contact with the trainer .
• Organize training around groups of learners: group videoconferencing for example .
• Learners are a self-aware group: It is effective to build a cohesive team.
• Generating its own codes: The team must invent and build its methods of collaboration
• Promoting mutual aid networks: including and perhaps even above all outside the tools made available by the trainer. (peer learning, mediation, ...).

Methodology

The methodology recommended for this work points to the investigation based on hypotheses allowing a particular reasoning to be established (Peraya, D., 2012),  , so our experiment will aim to verify the impact of recourse to technological choices at a distance for this reason we have set the following steps:  

1. Description of the sample

The purpose of our article is to verify the impact of the technological choice in a distance training mode on the development of skills our sample composed of 251 trainees who are distributed as follows:

• 63 primary teacher trainees and 80 qualifying cycle trainees :

Trainee teachers are called validated 8 training modules (planning-management- evaluation-ICT-school life-education sciences) each module contains 34 hours of training. More simulation situations or presentations followed by a discussion and a collective evaluation of the animated work.   

• 108 educational administrators

The method and the evaluation criteria for this category are identical to trainee teachers with a difference in the number of modules of 10 instead of 8 modules.

• 42 trainers

The trainers concerned by our research are 42 in number providing the different modules, as well as the management of the regional center to provide each trainer with a laptop computer to ensure distance training.

In the same perspective, each trainer has set a technological choice to evaluate his trainees in remote mode.

Among several choices we have identified 4 videoconferencing tools namely : zoom, Google-meet, Microsoft teams and Skype

2. Spatiotemporal context

The special research context is located at the regional center for education and training professions in Taza, Morocco

While the temporal limits are summed up in the training year 2020-2021 which is also a separate year compared to preventive health measures

After having provided a hybrid training 50% in face-to-face mode and 50% of the non-existent face-to-face hourly volume '' of six months from January 05, 2021 to June 26, 2021, the trainers also began two modes of evaluation : one of validation face-to-face modules and remote assessments for continuous checks in training modules, to validate the oral practice of practical training and to assess the defenses of end-of-study projects at a distance

3. Research Tools

Data collection tools

To begin our research, we considered two tools for data collection and the evaluation of distance learning, namely.

• Questionnaire for trainees : technical and cognitive assessment

Although Our survey is based on the implementation of a remote assessment, the questionnaire is also distributed online for those trained to judge their motivations and their cognitive engagement in the assessment process. multiple choice, open and closed questions concerning the technological aspect as a method of evaluating the technological choice used by the trainer

• Questionnaire for trainers : technical and ergonomic assessment

This questionnaire intended for the ergonomic and technical evaluation of different technological choices mobilized in the process of online skills assessment.

The questionnaires were distributed immediately after each semester. (months of April and July 2021)

4. Results Analysis Tools

For the stage of analysis and interpretation of the results we adopted the model of Krippendorff 2003, of which a set of data collection means and methods of analysis of quantitative and qualitative results are mobilized (Sphinx, Trideux and Excel) . In order to allow us to confirm the validity of our results (Nafidi, Y et al., 2018).

Results

The results from our first questionnaire are as follows:

The use of remote assessment: by trainers

The table reveals the percentage of use of technology in the acts of training and skills assessment

Table 1. Percentage of use of distance assessment: by trainers

The results shown in Table 1 show the strong will to integrate new technologies into the training process, However this will is not presented by the same figures concerning when talking about the evaluation of skills, of which we have recorded a resistance of 47% of the trainers, against a value of 53% which uses the technological averages in their acts of evaluation.

In the same perspective, 100% of this group of trainers use technology to regulate learning, while only 33% of users use remote evaluation to certify or validate training modules.  

Technological choices preferred by our trainers 

The answers to this question show the evolution of the use of technological choices during the 2020-2021 training year, the results are illustrated in tables 2 and 3.

       Table 2. percentage of preference in Technological choices: by trainers     

Table 3. Evolution of technological choices preferred by our trainers

We report that for the other section 10% of trainers (4) preferred to evaluate and train trainees using individualized e-learning platforms (2 with Moodle and 2 with Chamilo).

Figure 1. Evolution of technological choices preferred by our trainers

Table 3 and graph 1 explain the mobile and modifiable nature of the technological choice of trainers, but the most relevant remark is the increasing percentage of Googlemeet compared to other videoconferencing tools, which will push us more towards the analysis of mobility and permeability between technological choices and the foundations of trainers to choose one or another technological tool.

Degree of permeability and the transition from one technological choice to another

To measure the degree of permeability between the technological choices, we determined the nature of the different technological choices used by the trainers at the end of each month of training and after each evaluation action.

Table 4. Degree of permeability and the transition from one technological choice to another

We notice through the reading of the results of table 4 that the immigration of one tool towards another is marked by only one direction, that of the 3 choices towards the fourth which is Googlemeet, in the same way this change is characterized by a progressivity striking from the start of training to the end . 

Still analyzing the data in Table 4, the choice of individualized platforms has only seen a slight change from time to time, but the 10% of trainers have kept the same remote evaluation strategy during all the sessions of the training.

Reasons for choosing technological software

To justify the technological choices mobilized in the digital assessment on the one hand and the causes of immigration from one tool to another during the training, we present the data illustrated in Table 5. and which represent the ergonomic part of our questionnaire:

 Table 5. basis for choosing the technological tools for videoconferencing

Table 5 clearly shows that the choice and immigration to Googlemeet is explained by several points namely : the average level in technological skills facilitated by the programming options and access to courses offered by this technological choice. Adding also the facility of the Equipment, the temporal durability and the traceability offer by this choice, without forgetting the important capacity in participants. 

For people who have chosen Zoom and who continue to do so, justify their choice by the quality of the image and sound provided by this technological tool, finally for the choice of Microsoft Teams, users value the option of organizing tasks and the planning of actions facilitated by this technological choice.               

Limits of each technological choice

The trainers who used remote evaluation during the training revealed several limits of the use of each technological choice.We present them in table 6.

Table 6. Limits of each technological choice declared by the trainers

The analysis of the results of Table 6 shows that the main limitations revealed by the trainers concern the Zoom application, the latter is qualified as a less secure choice and which is easy to scam, as well as the technical problems and malfunction that it does. present has decreased its jobs by our population, in another aspect the Microsoft teams is a choice which presents several problems of compatibility and functioning online. Finally, the Googlemeet is evaluated by our trainers as the most secure choice despite the presence of some problems related to the connection and access to the application after starting the program (Taber K., 2018).

To complete our analysis it is essential to evaluate the feedbacks of the trainees in relation to such evaluation, the results of the second questionnaire discussed the following points: 

The use of remote evaluation : by trainees  

Table 7 reveals the positions of the trainees in relation to the use of technology in the evaluation.

Table 7. Feedback from trainees on the use of remote evaluation

We detect that the majority of trainees value the digitization of evaluation, especially for future administrators who no longer find funding problems for their training, against a percentage that varies between 12 to 15 % of future teachers who still find it difficult. socioeconomic to adhere to this method of evaluation. 

Technological choices preferred by trainees

To assess the trainees' preferences in relation to the technological choices used by the trainers, we questioned them after several evaluation actions. The results of our survey are mentioned in table 8. 

Table 8. Technological choices preferred by trainees

Figure 2. Technological choices preferred by trainees

For all categories, graph 2 the preference of the evaluation provided by Google meet as a first technological choice followed by Zoom as a choice also admired by our population, whose reasons for choosing these two tools remain the same as for the trainers.

Distance assessment methods preferred by trainees

Table 9 shows the preferred modes of our student population compared to the preferred assessment mode. 

Table 9. Distance assessment methods preferred by trainees

 As shown in our table 9, evaluation by group or following collective work remains the most valued by our trainees, followed by the oral mode which gives immediate interpretation and direct questions, while the presentation of presentations is also a mode. preferred by 27% of our trained population.  

Discussion

The results unveiled in the previous part show that the issue of digitization of evaluation has experienced significant development and successful integration into regional education and training centers. evaluation gave birth to a new training strategy based on technological choices throughout the training process.

In this perspective, the use of videoconferencing tools has participated in the modification of the representations of trainers and trainees in relation to the question of digitization. In this sense, our survey unveiled several advantages of remote evaluation, we cite : immediate feedback which is an essential element in the development of skills, instantaneous processing of results which participates in the management of performance and educational roundness , the saving of means and resources unlike a face-to-face evaluation which requires the establishment of a willing and a strict protocol, and finally a wide possibility of adapting the time of the evaluation and organizing it according to a educational contract and the modeling of difficulties and problematic situations according to each group of trainees.       

Limitations of the research

As our research presented a set of advantages due to the use of technology in the evaluation process, several limitations and obstacles were reported by our study population we cite: 

• On the organizational level : 

Users reported a major difficulty related to the individualization of the evaluation process around the concept of '' differentiated pedagogy '' in order to respect the pace of the trained, especially with courses started online.

A second difficulty still in the organizational framework is related to the preparation time which remains quite important and requires more work by the trainers.

• On the human and social level

The use of remote evaluation requires a certain availability of trainers and trainees, adding to the working environment which no longer favors this process among the majority of those evaluated.

• On the material level:

Material problems are also a limiting factor in our research because the quality of the sense image and the connection are automatically linked to the nature of the Equipment used (camera - headset - computer - capacity - speed - RAM).

In the same aspect, this digitization process requires a subscription to the aforementioned technological choices in order to access all the options and avoid repetitive technical problems.

Ethically (and value)

The trainers have spotted some cases of cheating and plagiarism which directly influence the objectivity and fallibility of our e-evaluation action.

Conclusion

At the end of our article, we recall that the empirical study carried out on the experimentation of different technological choices within the regional center of the education and training professions of Taza, in an online training mode allowed us to highlight the changes in the practices of 42 trainers who used technological tools to assess the learning of their trainees.

In this sense, the survey carried out by the two questionnaires which were carried out and distributed online confirmed that the multitude of technological choices gave trainers more opportunity to vary their assessment mode and strategy , by giving trainees other avenues and opportunities to learn and accompanying them to develop their professional skills either in an individual or also collaborative way 

However, the context of the training, the nature of the knowledge, the intellectual and socioeconomic level of the trainees and the material resources remain factors limiting the wide choice of technological tools to evaluate. In other words, the technological choice mobilized in the evaluation of learning is a unique and specialized action characterizing the wishes and resistance within our center exclusively, and it cannot be more generalized for all training sectors and in all circumstances. of learning. 

Finally, confirming that the use of technological choices to ensure remote evaluation at critical moments has undoubtedly constituted a motivating factor for learning, for self-evaluation and for co-evaluation in a strategy based on a multitude of modes and methods likely to guarantee maximum socio-professional performance.

Acknowledgment 

My field research work was facilitated by the administration of the regional center for education and training trades of TAZA and I would like to express my particular gratitude to the trainers and trainees who agreed to participate in the experiments. and surveys of my research.

Conflict of interest

The authors declare no potential conflict of interest regarding the publication of this work. In addition, the ethical issues including plagiarism, informed consent, misconduct, data fabrication and, or falsification, double publication and, or submission, and redundancy have been completely witnessed by the authors.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article