Artificial intelligence is a good servant in the evaluation of online tests. But the Lords remain men

Artificial intelligence (AI) needs mantinels. We therefore welcome the European Parliament's efforts to provide a transparent environment for the use of AI in education. Our proctoring solution uses AI to monitor the regularity of the test — but artificial intelligence is only an assistant that makes people's work easier. It is they, trained evaluators, who decide if the applicant has played by the rules.

Let's get straight to the point. Scio is taking online college exams in Exceptional position. As one of the few in Europe testing:

• we implement
• and at the same time we have created our own proctoring system in which the exam takes place — ScioLink.

Companies usually only deal with one part. Since we have both pieces in our hands, we have 100% control over the rules and we decide what role we play with artificial intelligence. And the role assigned to her is to highlight potentially troublesome moments during the exam, but never to decide whether or not such a moment was actually a violation of the rules. Artificial intelligence escapes the context of human action. It identifies a suspicious situation, but is not always able to determine exactly why the situation occurred and cannot make decisions in marginal situations. For this reason, the control is overseen by experienced human evaluators, who have the final say in the final decision on the exam.

How the proctored online exam takes place

Candidate before starting the test, legitimizes with the help of a webcam -- shows ID and then himself. The photo of the pass is checked by the assessor as in the case of the full-time exam. Thanks to this, we know that the right person is taking the exam. The candidate then uses a webcam to film the surroundings of his workplace, i.e. the table and the room in which he takes the exam. “This shows that he does not have any unauthorised devices ready anywhere and there is no friend nearby to advise him.” explains Jáchym Vintr, head of development of the ScioLink test application.

Before the test begins, ScioLink provides desktop sharing, monitor events, and external display connections, which verifies what is happening on the monitor or if an external help screen is connected next to it. And then time runs out, and the applicant at ScioLink answers the questions. “Both the webcam and microphone are running throughout the testing. At the end of the test, ScioLink stores both the answers and all video and audio recordings. And the tracking ends there.” describes Vintr.

We described online proctoring in detail in a separate article >>

What Artificial Intelligence Does at ScioLink

Assists. It helps. Practically, this means that AI marks suspicious moments from the course of the exam. For example, if a candidate moves away from a job during a test, pulls out a cell phone on which he is looking for help, or violates the rules in some other way, the AI will mark that passage of the record with a “flag”.

“We have deliberately “trained” artificial intelligence to be more sensitive. It marks the slightest hint that something suspicious has happened. Perhaps even self-apology, when the candidate says to himself what counts,” gives the example of Vintr. We have a sensitivity set this way because after artificial intelligence everything is controlled by man.

The regularity of the test is verified by approximately 100 evaluators

Let's say that even with the initial preparation, the exam lasts an hour and 500 applicants take part in it. This creates 500 hours of recording from webcam and microphone, as well as 500 hours of recording of events on the monitors of individual applicants. But, for example, the annual reality of our National Benchmarking Exams represents an even greater volume of data, and it is virtually impossible for the control to be based solely on people going through individual records from start to finish.

Artificial intelligence will mark suspicious places and only then Human evaluators are in control.. “We have a team of more than 100 people, among which the individual records are divided. They follow them mainly in accelerated mode so that they can concentrate fully on the marked suspicious locations. We train the evaluators, give them feedback, and the algorithm divides the records for review according to the key, which ensures randomness and prevents the same evaluator from reviewing the records of the same applicant.” points out Vintr.

Unlike most foreign solutions, we do not allow AI at all for the initial identification part of the exam. “Applicants often have an ID with a photo of their fifteen year old. Now they're in their 20s, and AI all too often sees the problem where it's not.” reveals Vintr. The validity of the document, the matching of the photo with the applicant, as well as the recording of the surroundings of the workplace is always checked by a person in ScioLink.

Human evaluation takes place at several levels. If the evaluator finds that the AI has used the flag incorrectly, they will remove it with one click. If the AI is right and determines that there really has been a violation of the rules, the record goes to the commissioner, who either confirms the assessor's verdict or corrects it (there is even a third instance for online National Comparative Examinations, the so-called Appeal Commissioner).

After checking all records, we send the results to the faculty, which sets the line — accepted/not accepted. And for applicants in whom we have uncovered wrongdoing against the rules, note describing where they erred. The final verdict, however, is again up to faculty representatives.

We are pleased that proctoring gets a legal framework

Over the past three years, via ScioLink, the Over 60,000 online college exams. “We made it possible for many students to take the exam from home at an important moment in their lives. They avoided complicated travel and unnecessary stress. ScioLink has also made things easier for a lot of faculties, so I am convinced that online proctoring is the right direction,” summarizes Vintr and adds: “Transparent rules will only help.”

If you are interested in how online tests are evaluated by students, we have a separate article >>

We welcome and closely monitor the current debate on the regulation of artificial intelligence in education. We agree with the opinions that:

• the tests do not need to collect quantities of data or biometric data (fingerprints, pupil scans, etc., which some other proctoring services collect),
• Moreover, the sensitive data of applicants must be carefully protected,
• tools that leave student fate free to be decided by AI need to be revamped
• Online testing must be as accessible as possible to the disabled or socially disadvantaged (either in the form of a longer time limit for the completion of the online test or by maintaining the offline version of the test).

However, we do not agree that the use of AI in education is universally perceived as harmful. If artificial intelligence does not make decisions on its own, but serves only as an assistant, it brings a lot of positives to education for students and universities.