How to develop quality situational judgment test (SJT) items: a step-by-step guide
19 February, 2025
How to Develop Quality Situational Judgment Test (SJT) Items: A Step-by-Step Guide Situational Judgment Tests (SJTs) are a powerful way to assess how candidates would respond to real-world, job-related scenarios. Designed properly, SJTs can reveal the specific skills, knowledge, and competencies required for optimal job performance. Below is a clear, concise checklist for creating robust SJTs, from identifying the competencies you wish to measure to delivering your final test.
Checklist on how to develop quality SJT items
1. Define what to measure
- Identify the job role
- Conduct a job analysis with subject matter experts (SMEs) to determine critical skills and behaviors on the job
- Expected Results: List of the crucial skills, competencies, knowledge that reflect the specific demands of the job.
2. Develop realistic scenarios for Items
- Select real-world job situations to serve as a foundation for test items. Gather critical incidents from employees, managers, or archival data.
- Scenarios reflect common challenges that candidates will likely face in the role.
- Scenario descriptions clear and concise, no unnecessary complexity present.
- Expected Results: A list of scenarios with common challenges and problems for the role aligned with crucial skills, competencies, and knowledge needed to overcome those challenges.
3. Design effective response options
- A set of response options, capturing a range of effective and ineffective behaviors. Use employee interviews to establish a variety of behaviors for responses.
- There is no obviously right or wrong answers in a set of responses—responses should reflect realistic decision-making trade-offs.
- Response format (Multiple Choice, Rank-order, Rating Scale) was chosen thoughtfully and with regards to particular assessment goal and practical limitations.
- Response instructions are developed in a clear wording.
- Expected Results: A list of response options in a chosen format.
4. Ensure scoring accuracy
- Scoring key developed using rational (expert-based) or empirical (data-driven) approaches following established procedures.
- Consider hybrid scoring methods, where expert and empirical approaches are combined for reliability.
- SJT items are pilot tested with a sample group of employees to validate response effectiveness and adjust scoring as needed.
- Expected Results: A list of response options with a scoring according to the chosen item format.
5. Delivery of SJT
- Platform or an app for the SJT delivery allows to place items of the selected format on it (Multiple Choice, Rank-order, Rating Scale, video, GIF etc.).
- Item format is not distorted on the platform and allows correct scoring.
- Response instructions are clearly visible and presented in a convenient for a test-taker manner.
How Constructor Assess can help in creating SJTs
Constructor Assess is a powerful and user-friendly platform designed to streamline the development and delivery of Situational Judgment Tests. It supports multiple item types (e.g., multiple choice, rating scale) and offers intuitive tools for creating, piloting, and scoring SJT items. By leveraging Constructor Assess, you ensure your SJTs accurately measure the critical skills, behaviors, and decision-making required for any role, while providing a smooth, engaging experience for test-takers.
Join our webinar
The future of workplace assessments – best practices for talent management. Explore key characteristics of effective talent evaluations. Learn about high-quality workplace assessments, essential SJT design principles, response formats, scoring methods, and the pros and cons of SJTs.
