Who can benefit from using online proctoring technology? Anyone who is looking to deliver remote assessments, whether in an educational or organizational context.
How does it work? In this article, we'll take a deep dive into the structure and operation of Constructor's Proctor solution, an AI-powered online proctoring platform, as well as the key stages for online exams.
Getting started with Proctor
The online proctoring extension works from a browser with no additional installations needed. The service is integrated into a testing system – LMS (learning management system). Proctor service is offered as a SaaS (Software as a Service), and the entire infrastructure is stored in cloud services. Students do not need to install anything on the computer. Simply open the browser, access the specified link, pass the system and identity authentication, and begin taking the exam.
Should students use a computer or a phone during a remote-proctored exam?
Based on our experience, the desktop version is the only device that provides reliable results, so we always strongly recommend it to our customers. The mobile version gives the user too many opportunities to exploit the system. However, we understand that, in some cases, there is no possibility of using a computer, for e.g., when field workers are located in a remote plant or factory.
The proctoring system requests access to the camera and microphone and verifies their functionality as well as the stability of the Internet connection, the presence of a second screen, and all attempts to simulate or imitate the screen. If the system passes all the tests, students must pass the identification process: to take and send a photo of themselves with an identity document (a driver’s license, a student ID card, or a credit card).
Afterward, the student can start the examination. At the same time, the online proctoring system acquires information from three different sources:
- Camera
- Microphone
- Desktop
After completing the exam, the online proctoring system will gather and combine information from all sources into a single combined record.
Flagging violations with Proctor
During the exam, the Proctor software observes student actions. The system is flexible, allowing the host to set the rules for each exam. For example, in some cases, students can use a calculator; in others, they may need a pen and paper.
During the examination, the system automatically detects the following types of violations:
- Changes in the direction of the gaze, i.e., when the user is looking elsewhere.
- Lack of the face in the frame, i.e., if the user is not in the field of view of the camera or the face is not fully visible.
- The presence of other figures in the frame (Note: removing any portraits from the walls is also recommended as they could be flagged automatically.
- Substitution of the student, i.e., if the person in the frame is not the same person that is intended to participate in the examination.
- Any human voice in the frame, even if it is the student's voice (e.g., the student voicing the questions out loud).
- Change the windows and tabs on the desktop, i.e., if the user decides to switch from the test window to a search engine or open a third-party application.
In the online proctoring mode, when the entire process is controlled and monitored by an individual, other types of violations can be reported:
- The use of phones, tablets, and other devices
- The use of TeamViewer, Skype, RemoteAdmin, and other desktop broadcasting software
- The use of virtual machines and thin clients to pass an examination by a third party
- The use of books, notes, and drafts
- The use of calculators (if prohibited)
- All non-verbal communication (winks, hand movements, nods, etc.)
Proctor automatically marks all potential rule violations and adds them to the report. Each flagged entry contains information about the type of violation, such as moving the eyes away from the screen or opening a new tab, and a timestamp.
Based on the results of the exam, Proctor calculates the overall probability of violations during the exam:
- the green zone means a low probability
- the yellow zone means a medium probability
- the red zone means a high probability
In other words, Proctor can immediately determine which test-taker needs to be checked first.
Based on the results of the exam, we can perform the following actions for each test-taker:
- View and listen to camera, microphone, and desktop recordings
- View all potential violations in chronological order
- Check the records of what happened at that particular moment
Types of online proctoring
Automatic proctoring
Everything is done by an AI proctor. The obvious advantage of this mode is that it reduces the overall examination costs. However, in many situations, human participation is simply irreplaceable, such as:
- In the case of high-stakes exams, the supervision and decision process cannot be left to the machine;
- Even the most accurate automation system cannot track certain non-obvious violations;
- Automation is very sensitive and cannot recognize accidental outcomes such as a mother opening and closing the door in a matter of seconds;
- There are many instances where all stages of the exam process must be controlled by a human proctor, e.g., an individual may need to use one of the predefined protocols before giving a student access to the examination;
- The operator of the proctoring system can ensure that the student follows the procedure for a particular exam. In some cases, the student must show that their palms are not full of hints and answers, while for another type of exam, they must tear the draft into small pieces.
3 human participation modes
In addition to fully automated AI proctoring, there are three modes available with human participation:
- Asynchronous mode: an operator will review the recording after the examination;
- Synchronous mode: remote online monitoring and control of the examination process by an operator of the proctoring system;
- Identification mode: the operator only confirms the identity of the student before entering the exam;
Each mode has its interface and methodology for working with operators, which we will discuss in detail in the following articles.
We hope you now understand how the online proctoring system works in general. Our next articles will explore the technologies behind the proctoring solution for online exams.