Technopedagogical evaluation of a digital device for the continuing training of teachers in Morocco: Case of the e-takwine platform

Introduction

Since independence, the Moroccan education system has seen many projects that have tried to meet different social expectations such as the generalization of education or the decentralization allocated to regional academies and universities. In 2005, the Moroccan government adopted a strategy aimed at making information and communication technologies accessible in public schools.

As part of the Maroc Numérique project, the GENIE Program is at the heart of the reform of the Moroccan education system. In the strategic action plan of the Ministry of National Education and Vocational Training 2013-2016, and within the framework of the new orientations of the GENIE program 2013-2016, a new model of distance training devices is put into effect. in the form of MOOCs (Massive Open Online Course) 1.

In the same perspective, Morocco is committed to a Moroccan-American partnership to develop the infrastructure, management and skills of second cycle teachers, this project is known by Millennium Challenge Account-Morocco which has put in the spotlight. availability of Moroccan teachers and administrators 20 distance training modules in the platform, e-takwine.

While these environments are seen as a learning medium, special attention should also be paid to evaluation modalities to ensure their expected effectiveness and relevance. Faced with this situation, improving these supports becomes a major concern for those responsible for these systems.

In this context, we commit ourselves through this article to addressing the following problematic question: what evaluation method to measure the technopedagogical functioning of the tool in a distance continuing education?

In order to respond to our problem, we will deal with the following four axes through this work:

The first axis is concerned with the situation of distance continuing education in Morocco as well as the definition of two key concepts which are distance training and the distance continuing education system. For the second axis, it deals with the issue of evaluation procedures to determine the quality of a material or device in continuing education. As for the third axis, it presents the methodological framework of a survey on the evaluation of the e-takwine system for the continuing education of college and qualifying teachers. Finally, the fourth axis presents the results of practical research by answering the question of technopedagogical or ergonomic evaluation of the device subject of our research.

Theoretical frame

1. The situation of distance learning in Morocco

In such a debate on the Moroccan school, teachers have always been seen as the main actors in any reform. Between motivation and promotion, the reform provides them, according to the national education and training charter, "social and professional conditions allowing them to devote themselves fully and in dignity to what is more than a simple profession, a vocation. ".

Such involvement of teachers also requires professional qualification in relation to new pedagogical models. In this regard, the national education and training charter has set up continuing education actions and programs to help teachers work under optimal conditions and while mastering the necessary teaching methods and tools. The report of the Special Education and Training Commission (COSEF 2008) states that the Emergency Program offers continuing training sessions in its E3 P5 project . It has scheduled around 1.5 million days of continuing education for school education staff (Ben Ouahi et al., 2021)..

At the same time, the Mohamed VI Foundation for Social Works for Education-Training conducted a needs survey in 2009 in order to determine the expectations of its members with regard to online training services. As a result, almost 90% of the respondents were interested in distance training, hence the implementation of a training device of the E- le @ rning project. Despite everything, digital culture is not sufficiently anchored in the practices of teachers. In 2011, the barometer of open and distance learning (ODL) in Morocco states that 72% of the training provided is face-to-face while 18% represents blended learning .

In 2013, the Moroccan education system experienced the first online distance continuing education session for teachers. This was a MCE "Microsoft Certified Educator" certified training course that lasted two months with a face-to-face certification exam.

In 2014, and as part of the GENIE program of the Ministry of National Education and Vocational Training, a TICE and Professional Development Program (PDP TICE) was set up allowing teachers to be users of TICE while developing teaching-learning situations instrumented by ICT. This training also aims to use technological communication tools and define educational uses

Generally, in the last five years, there has been a great deal of interest in the use of distance learning. However, face-to-face training encountered several constraints: large audience, high training cost, cascade training hence the need to resort to a new system of distance training with the implementation of a new system. training in the form of MOOC: Massive Open Online Courses. In this sense, it is necessary to define two key concepts: distance training (FAD) and the distance training system. (Taber, K. S. 2018).

2. The distance training system

Usually, flexibility is a key concept in distance learning. But, before defining what a distance learning device is, it is better to come back to the concept of a device. (Montandon C.,2002) specifies that a device represents:

"Term initially derived from the technical field, is always characterized by a set: set of elements [...] implemented for a specific intervention" or "a set of means arranged in accordance with a plan"

As for Blandin, it was defined as: "a set of means, arranged, with a view to facilitating a learning process". To this definition, we can add that of (Peraya D., 2012) who speaks of “techno-semio-pragmatic device” As for the distance training device, it designates: “A set of actors (learners, tutors, training managers) and technical tools (educational resources, exchange tools, platforms) organized in space and time, according to a learning goal ” (Blandin B., 2002).

From this definition, it can be deduced that the distance learning device represents a very complex system. On the one hand, by its actors in relation to their interactions: the device is considered as a place for the construction of autonomy as well as the construction of cognitive and metacognitive strategies. On the other hand, by a complex relationship between the two notions of mediation and mediatization.

3. The e-takwin platform

Created in 2016 by the CNIPE on the occasion of the reform of recruiting contract professors (AREF senior professors from 2018), this portal was created for initial training objectives, first of all before move to a support and reinforcement platform for face-to-face training in 2017 and 2018 .

The e-takwine training platform in version 2020 is made up of two essential components

a- The part intended for secondary school teachers:

The portal presents support training in the modules:

Planning of learning. Learning Management-Management of common classes. Learning assessment. The use of ICT in teaching. And Exercises and quizzes for the assessment of prior learning (reading, grammars, multidisciplinary project, etc.)

Each CRMEF trainee has their own personal session integrated into a discussion forum with educational themes.

We point out that the trainee is not obliged either to validate one or the other module or to have a mark at the end of the training

b- Part intended for directors in training and outgoing CRMEF directors:

This part is intended for directors of schools, it presents reinforcement training in basic modules of the school's project, school coaching, inclusive education and new methods of governance and support assistance in the school. use of computer applications (MASSAR, GRESA and ESIS).

The platform provides participants with a range of courses and online resources intended to support them before, during and after training. Participants have free access to courses and resources according to their needs and according to their pace. (Bernard M., 2001). These courses are designed and integrated into the platform in such a way that trainees can consult them before even starting the training in.

4. Evaluation of the e-takwine platform

For Parmentier, to evaluate means:

"... examine the degree of fit between a set of criteria appropriate to the objective set, in order to make a decision. The evaluation is therefore part of an ongoing process of seeking improvements ".

This evaluation aims to improve the effectiveness of the training, the methods and the technologies used.

In this sense, evaluating the effectiveness of continuing training is very important. The evaluation process makes it possible to determine the quality of a material or device.

In this context, the evaluation of a training is done in two stages (Linard M., 2008):

• "On-the-spot" evaluation : that is, the training evaluation process is carried out immediately after the training action. The purpose of this form of evaluation is to measure progress as well as the adjustment of the path and the program.
• The "cold" assessment : this is to assess the training action after its completion in a few days or weeks. Its purpose is to determine the achievement of the training objectives expressed in terms of professional skills. In this form of assessment several tools are important such as: observation guide, self-assessment questionnaire and follow-up interview.

At the end of the 1950s, (Kirkpatrick D., 2006) defined a training evaluation model based on four levels of evaluation: evaluation of reactions, evaluation of learning, evaluation of transfer level and finally evaluation of results. In relation to what is already mentioned, (Pottiez J., 2018), thinks that:

"… The 'hot' assessment generally corresponds to Kirkpatrick's level 1 (and sometimes to level 2, if the learning assessment is carried out at that time), while the 'cold' assessment corresponds at Kirkpatrick     level 3”.

As for (Gerard F., 2010), he considers that there are evaluations of the effectiveness of the training. These evaluations are broken down into three axes:

The first axis concerns the evaluation of prior learning, or the evaluation of teaching effectiveness in relation to the objectives achieved.

The second axis concerns the level of transfer with regard to the learning outcomes and which are applied in the field.

The third concerns the impact of training in terms of new skills of the trainees acquired and developed in work situations.

The evaluation of distance learning must be the subject of several evaluations: evaluation of behavior, evaluation of results, technical and ergonomic evaluation of the techno-pedagogical tools used, etc (Nafidi Y., 2018).

It is in this context that our research is being launched, which will try to define the technopedagogical or ergonomic evaluation modality as one of the evaluation modalities of a distance continuing education system for teachers.

Methodology

This stage of our research consists of evaluating a remote continuing training system for e -takwine teachers accessible through the following link: http://e-takwine-tanmia.men.gov.ma,  Its' This is a continuous distance training spread over four months April –May and June 2020, intended for teachers of the two cycles (college and qualifying), we will limit ourselves during our research to those trained from the AREF of Fés –Meknes

To conduct our investigations, we will combine both qualitative and quantitative research elements. For this reason, we will adopt the mixed methodology.

Data collection tools and procedures

1. the questionnaire

Our survey is based on the implementation of an online questionnaire for teachers (trainees) who have participated in distance learning via the e-takwine platform. This questionnaire brought together multiple choice, open and closed questions concerning the technological aspect as a method of evaluating a distance continuing education system for teachers.

1. The ergonomic evaluation grid

There are two types of ergonomic assessment methodologies for web interfaces: user testing and ergonomic inspection.

During user testing, one or more users participate in free exploration to perform real tasks. During these tests, we are interested in the performance and behavior of users when interacting with the interface (MAZOUAK A et al., 2019).

For ergonomic inspection, it is a method that applies to the characteristics of interfaces performed by evaluators, whether or not they are experts in ergonomics. Among the inspection methods are:

• Assessment of compliance with a set of recommendations (guideline reviews).
• Analysis of compliance with standards (inspection standards), dimensions, heuristics.

As part of our research, we will proceed with a combination of the two methods: we will use the two users of the evaluated platform and an ergonomic inspection while referring to the ergonomic standards of (Scapin et al.,1993), of which here are these criteria and under criteria:(Table 1)

Table 1. Disciplines represented in the questionnaire

How many reviewers?

The number of people who will participate in this ergonomic inspection is not interesting since the objective of this evaluation is to detect that an existing problem is indeed encountered by at least one of the evaluators. We will proceed iteratively with a small group of 5 testers. The argument put forward by (Nielsen and Landauer,1993),   is that it is better to use 15 users by correcting the defects identified by the first 5, then repeating the process three times, than to use all 15 users up front. For this, we will administer this grid to the 5 students of the IFTIC master's degree (distance master's degree in training engineering and information and communication technology) at Mohamed Premier University (Oujda-Morocco), because they have already studied during their master's degree the ergonomic criteria of Scapin and Bastien. Among these evaluators, there are two users of the e-takwine platform who have pursued this continuing education remotely.

Results and Discussion

Sampling elements

To define the size of a sample, (Dornyei., 2007) provides some details :

- apply a rate of between 1 and 10% of the population, with a minimum of 100 participants, this is an approximate rule used by researchers;

- the sample must have a normal distribution, which implies a minimum threshold of 30 people;

- set the minimum sample threshold according to the sub-groups identified in the population;

- leave sufficient leeway to compensate for abandonment during research.

In the context of our research, the online questionnaire is the data collection instrument. But for fear of poor representativeness of respondents, the questionnaire was administered to all registered participants. There are 362 teachers from both cycles (college and secondary), belonging to the AREF Fés Meknes region.

In addition, the percentage of respondents to the questionnaires was generally representative since we retained the response of 241 teachers, for a percentage of 74.8%. These respondents are distributed according to the education cycle as follows (Table 2).

Table 2. Distribution of respondents according to the educational cycle

According to those questioned, 61% of them have already pursued distance training compared to 39%. (Table II) The latter (who have never pursued previous distance education) are distributed according to their teaching cycle as follows:

Table 3. Distribution of respondents according to the educational cycle

Results of the ergonomic evaluation

1. According to the questionnaire

Before starting the results of our ergonomic evaluation grid which was sent to five evaluators, we will have to give the results obtained from the online questionnaire with regard to the difficulties encountered at the technical level.

a- Percentage of consultation

Fig 1, clearly shows the regression in the frequency of consultation, which dropped from 96% at the start of the training to 23% around the last week of the course, which calls into question the quality of the content offered and the training methods adopted.

Fig. 1. Evolution of consultation frequency

b- Functional aspects

b-1 : Ease of navigation

 As Graph 2 shows, 55% of users showed that they rather agree that the platform is easy to navigate, clearly state the units for each quantity in an equation.

Fig. 2. the degree of ease of navigation of respondents in the platform

b-2: Account management

According to graph 3, 65% of users expressed that they tend to agree with the quality of managing and personalizing their accounts on the e-takwine platform

Fig. 3. Account management capability

b-3: Forums and exchanges (social learning)

Graph 4 shows that 83% of users value the social learning offered by the e-takwine platform

Fig. 4. Contribution of forums on exchanges

c- Technical aspects

Among the main technical problems with which the 12.5% of the trainees were confronted, we have collected, in the following table, the main part of their answers and we have classified them by type of difficulty:

Table 4. The technical difficulties encountered by respondents by type.

d- Educational aspects

d-1: Guides and orientation

In terms of guidance, 55% of users tend to agree that they are perfectly guided in terms of navigation on the platform, while only 2.5% who tend to disagree, as shown in the following graph:

Fig.5. the level of agreement of respondents with the platform guides

d-2: Updated and renovated courses

According to the responses of the respondents, 75% declared that the training content did not correspond to the changing needs of the trainees.

Fig.6. updating training content

d-3  Impact on the competence of planning - management and evaluation of learning

In general, more than 50% of the trainees rated negatively the contribution of the e-takwine platform to their planning, management or learning assessment skills.

Table 5. impact on the competence of planning - management and evaluation of learning

We note following our first survey that: despite the efforts made to create and administer the e-takwine platform, we identified several points of dissatisfaction among our research population:

- In relation to updating the words that remain frozen without taking into account the evolution of the levels and needs of the trainees.

Expressed difficulties with connections, compatibility or also technological skills

- Finally, for educational performance, only half of our sample declared the direct impact of training courses on their teaching practices.

2. Results according to the ergonomic evaluation grid

As already specified, the objective of this evaluation is to detect that an existing problem is actually encountered by at least one of the evaluators. The following table summarizes these various techno-pedagogical problems encountered by the ergonomic evaluators with the aim of treating them and subsequently improving them by the designers of the remote continuing training system:

Table 6. Use of technology in teacher assessment

Of the platform, that is to say participants in remote continuing education (college professor and secondary professor and at the same time students of the Master).

Results interpretation

According to the table, while trying to answer the research question concerning the techno-pedagogical aspect, it can be predicted that, so that the distance continuing education system implemented for teachers can respect ergonomic aspects, he must review the ergonomic criteria related to:

• Guidance as a criterion which indicates the means
• implemented to advise, guide and inform the user, but the revision of this distance training device must target the following two sub-criteria: on the one hand at the level of immediate feedback c 'that is to say the answers as immediate as possible which must be provided to the user to inform him about the accomplished action and about its result. On the other hand, at the level of the sub-criterion which is readability concerning the lexical characteristics of the presentation of information on the screen.
• The Workload as a criterion which means all the elements of the interface that play a role in the perception and memorization of information. Two sub-criteria, which participate in the workload criterion, must be revised: brevity, and information density.
• Explicit control concerns two main factors: first, the taking into account by the system of the explicit actions of the users, then the control that users have over the processing of their actions. The remote continuing education system implemented must revise the user control sub-criterion because this user control over the dialogue is a very important factor in terms of system acceptance.
• Adaptability which consists in providing the user with different procedures and commands allowing them to achieve the same goal. This device must take into consideration, at the level of recommendations, the two sub-criteria: flexibility which targets the means made available to users to personalize the interface and taking into account the user experience.
• Error management which represents the means implemented to avoid or reduce errors, and correct them when they occur. The sub-criterion that must be corrected is that of protection against errors because it is necessary each time to warn users in the face of disruptive harmful actions or commands.
• Compatibility in terms of the presentation of useful functions and objects important for carrying out the proposed tasks. This criterion also concerns the degree of similarity between the various environments or applications.

Conclusion

Our research focused on the theme of the evaluation of a distance continuing education system for teachers. The objective of this research is to address the issue of complexity related to the evaluation of the e-takwine platform. Given the specificities of this new mode of continuing education, and given the interest that evaluation represents for maintaining and improving its quality, we are committed to answering the following problematic question: which evaluation method technopedagogic of a distance continuing training device for teachers in Morocco?

We initially asked ourselves about the feedback from users on the use and the functional and structural parameters of the e-takwine platform. secondary school. The analyzes of the results that we have collected show that the frequency of use of the platform remains regressive from March to June, add that the general satisfaction rate is average for the majority of the indicators, whether for functional capacity as for design and operation.

Despite the contributions and the theoretical and practical advantages demonstrated in the process of evaluating and interpreting the results and in the responses provided to the problem, our work also presents a number of nuances and limitations.

In this sense, human factors, the level and experience of those trained, the period of evaluation and training remain the variables influencing the success or failure of the evaluation process.

To conclude, the evaluation of a distance training system is not an easy task given the consideration of several components as well as the characteristics linked to the distance training mode. In this sense, Alan Jay Perlis said: "Programmers should not be evaluated by their ingenuity or their logic, but by the completeness of their case study".

 Acknowledgment 

A special thank you to all the teachers and students who took the time to complete the questionnaire.

 Conflict of interest

The authors declare no potential conflict of interest regarding the publication of this work. In addition, the ethical issues including plagiarism, informed consent, misconduct, data fabrication and, or falsification, double publication and, or submission, and redundancy have been completely witnessed by the authors.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article