When an emergency occurs, time is everything – a rapid response could save lives. So, it should come as no surprise that when considering candidates for roles within the emergency services, special screening measures are used to identify the very best people for the job. People in emergency service roles such as call handlers, paramedics, and the police could be faced with a serious situation on their very first day of live action, and this is no less true when considering the situations faced by firefighters. According to the United States Fire Administration, fire kills more than 3,000 people every year, and around 100 firefighters die on duty every year.1 When there are lives on the line, it is critical that firefighters be appropriately trained to judge the situation correctly and make the right decision in terms of the necessary action.
Situational Judgment Tests
Part of the current selection protocol for frontline job posts includes situational judgment tests (SJTs), a blend of quiz and simulation where the candidates are placed in a hypothetical situation and must respond with the decision they would make if faced with this situation in reality. These tests are highly valued because of their flexibility, as well as being relevant to the matter being evaluated. For example, when selecting a firefighter, SJTs can be used to test a candidate against a concrete situation and establish how he would respond.
State-of-the-art SJTs employ multimedia resources to achieve the ultimate immersion, outperforming regular text-based SJTs in relevance and potential. So, how exactly can these exams help recruiters select the right people for the job? Ultimately, SJTs have the potential to allow candidates to respond as if they were in a real situation.
The quality of these tests relies on two dimensions: “task fidelity” and “response fidelity.” Task fidelity is the extent to which the challenges presented relate to real-life work situations. In written SJTs, participants read a description of a scenario and either write a short response or select from a choice of solutions. Text-based SJTs are lower in task fidelity than multimedia SJTs, where video and computer simulation provide depth and realism to the interaction by also including nonverbal stimuli. SJTs and, more specifically, multimedia SJTs can gauge a candidate’s innate decision making and behavioral interpretation skills, qualities necessary for a quick and accurate response from firefighters.
Response fidelity reflects the extent to which the options given to the candidate as valid answers correspond to the way individuals would react in real situations. Most SJTs adopt multiple-choice solutions, which aids the selection process when dealing with a high number of candidates. However, closed-option SJTs tend to have a low response fidelity, as they restrict the choices available and prevent applicants from providing their own solution. In the real world, firefighters are often required to select a solution from a variety of alternatives, a process that can prove difficult to simulate in just a few multiple-choice options.2
SJTs can play an important role in preparing individuals for the work ahead by giving them a taste of the reality of the job. Dr. Ali Shalfrooshan, research and development consultant at Assessment and Development Consultants Ltd., found in his research that a significant proportion of call handlers resigned from their posts after a few weeks or days. This is likely because the call handlers had a lack of mental preparation and did not understand the true nature of their role. The same scenario could be true for other emergency workers, including firefighters, indicating the need for a thorough and clear recruitment and induction regime to ensure that candidates are fully aware of what will be expected of them. SJTs can be beneficial here.
Flexibility and Relevance
According to Shalfrooshan, the strength of a well-designed SJT lies in its “flexibility and ability to be designed around a specific context/role.” These tests are relevant to many emergency service positions, and Shalfrooshan’s team has developed tests for the ambulance and police services in addition to the fire services. He has spent the past eight years devising a solid methodology for the process of designing relevant and appropriate testing for any kind of critical environment. These tests are common in the recruitment of firefighters as part of a firefighter candidate assessment. At the Los Angeles (CA) Fire Department, for example, passing this multimedia test is a minimum requirement in the recruitment process.3
Many SJTs have already been developed for recruiting in the fire service sector. Shalfrooshan says, “We have designed multiple tests for the fire service, including assessment center exercises and SJTs. The SJT we designed for the fire service were text based rather than multimedia but the principles regarding the best practice steps taken are still the same.”
Even when most of these are text based, they undergo the same intensive process involved in high-quality SJTs, which involves job shadowing and analysis. According to Shalfrooshan, “Test designers need to be truly immersed in the role for it to be realistic. Job shadowing, understanding the role, and understanding the common challenges of that tole are critical.” Test makers need to have a full understanding of the reality and challenges faced routinely by firefighters and other critical professions. As Shalfrooshan puts it, “Involvement of experts across the fire service is essential and ensures the content [of SJTs] is realistic.”
SJT development processes often include visiting training stations, job shadowing at fire stations, and interviewing a wide range of employees, among other strategies, making the best possible effort to understand the strategy of the service as a whole and identify the crucial qualities of a firefighter.
A similar development process is used in the United Kingdom, with potential firefighters being required to complete National Firefighter Ability (NFA) tests.4 Scenarios and questions were developed in workshops, with contributions from a range of fire service personnel, including trainers and human resources experts. These workshops have the same aim as the interviewing and shadowing exercises conducted by Shalfrooshan’s team, which is to ensure that the tests provide realistic information and scenarios that accurately reflect the role of a firefighter. (3)
The Future of SJTs
As the knowledge behind these tests evolves, their implementation will become more affordable and commonplace. “The only issue is the higher cost associated with such tests,” offers Shalfrooshan. “The benefits are very clear.” Indeed, SJTs offer employers a way of determining essential character traits early in the recruitment process, and in sectors where candidates will be exposed to dangerous, life-threatening situations from the very beginning, it is in everybody’s interest for the person fighting that fire to be the person best suited to the job.
References
1. United States Fire Administration, “About the U.S. Fire Administration,” 2017, available at: www.usfa.fema.gov/about/index.html [accessed MAR 2017].
2. Lievens, Filip, and De Soete, Britt, “Simulations,” The Oxford Handbook of Personnel Assessment and Selection, Neal Schmitt (ed), Oxford University Press: Oxford, 2013.
3. Los Angeles Fire Department, “Firefighter Selection Process,” 2016, available at: www.joinlafd.org/index.cfm?section=hiringprocess [accessed 22 MAR 2017].
4. Communities & Local Government, “National Firefighter Selection Process National Firefighter Ability (NFA) Tests Technical Manual,” 2009, available at: www.gov.uk/government/uploads/system/uploads/attachment_data/file/14997/1410492.pdf [accessed 24 MAR 2017].
