Denver’s Skills-based Annual Fireground Evaluation Program

In 2010, the Denver Fire Department (DFD) implemented an innovative program to test fireground capability among its ranks. The Skills-based Annual Fireground Evaluation (SAFE) was born out of the realization that our previous physical fitness testing program was not as effective as it could be. As such, we took important steps to devise a new program that would ultimately keep our firefighters fit for duty. This article will look at what makes the SAFE program unique, offer some background on the program, and explain program specifics, as well as how it has helped our department’s workforce become safer and healthier.

How It’s Different
The SAFE test is different from other programs in one key way: While similar evaluations focus on physical fitness– pushing the “go, go, go” mentality where speed is king–the DFD evaluation incorporates both physical and cognitive elements. This helps reinforce the basic tenant that firefighters should always drill on both the mental and physical aspects of the profession, together. After all, once the basic mental and physical functions become routine, where firefighters have started to develop some muscle memory, they can apply their cognitive skills to the specific problems at each incident. In short, the SAFE test reinforces safety by requiring that our people think while acting, so the two will become inseparable on the fireground.

Program Background
Denver Fire has always prided itself on being an aggressive, first-rate department. However, in recent years, we had several incidences of “mask failure” at working fires, resulting in respiratory compromise and injury. We also suffered the loss of one of our lieutenants at a fire. As such, we needed to revisit what we were teaching and how we were teaching it.
    
So in 2009, the DFD Safety and Training Division began the process of re-implementing physical fitness testing after a six-year hiatus. Our former Skills Evaluation Test (SET) had been dropped due to several issues, including its well-intended but flawed “fitness to test” criteria and a budget that did not allow for placing members unable to complete the test on light duty.

We initially modeled the resurgent program after the tried-and-true SET. We hoped that familiarity with the process would help foster acceptance and ease some of the anxiety that’s always present when a new program is introduced or, in this case, re-introduced.

The training staff began to carefully evaluate the old SET, hoping to eliminate some of the problems and perhaps add value and increase credibility. However, the issues that brought down this program had not been resolved; in fact, the prospect of some our members failing after a six-year hiatus from physical fitness testing loomed even larger than before. Plus, we had no additional funding. To be frank, in 2010, we faced one of our worst budget years, so the prospect of members failing and therefore being considered “unfit” for the fireground threatened to derail the program before it even got started.

Phase I: Fitness Evaluations
We decided to take a phased approach instead, and we started by offering Wellness-Fitness Initiative (WFI)-based fitness evaluations on a voluntary basis. We hoped that providing the fitness evaluations during the first quarter of the year would both ease members back into the expectation of a physical fitness standard and also provide a snapshot of the fitness levels on the job. Along with that, we announced our intention to conduct the SET in the early summer. We knew that our members, being a fairly competitive lot, would want to do well, and thus, hopefully, take advantage of the fitness evaluations.

We had a number of WFI-certified peer-fitness trainers on the department, so we used them for the fitness evaluations. This also served to endorse these peer-fitness trainers’ skills and their value to our membership. We hoped that members who needed improvement would feel comfortable seeking the counsel of the peer-fitness trainers to improve their fitness levels for the SET.

The first challenge we faced: conspiracy theories, complete with worries and questions such as, “That new division chief is out to get us,” “What’s in it for the administration?” “Everyone who doesn’t pass will be sent out to a subdivision,” “They are going to use your fitness information against you,” etc.

Anecdotally, we learned that most department members were going to refuse to take advantage of the fitness evaluation. Because the program was voluntary, in accordance with the WFI guidelines, we needed a reason to require all members to come out to the division, regardless of whether they were intending to go through the fitness evaluations. We seized on the mask-fit testing as the reason to require every member to attend training. Once we got them to training, we had to allay their fears and provide the necessary reassurance about the fitness testing.

Members’ three main worries: 1) privacy, 2) being laid off or sent to a subdivision and 3) getting charged for an ambulance ride. (To the third concern, in the past, several people who participated in our fitness-to-test evaluations were taken to the hospital as a precaution and were later billed.)

We addressed each of these concerns in turn. To ensure members’ privacy, we developed a randomized numbering system to de-identify any and all paperwork with members’ results. Only a universally trusted lieutenant held the key to that identity-masking system. This lieutenant is also the lead for our wellness-fitness program. Even administration-level officers were not allowed access to anything except aggregate data.

The next two issues were interrelated and caused primarily by the same well-intentioned but flawed fitness-to-test process. In the past, we had set unrealistic blood pressure benchmarks. So we worked with the wellness committee and our occupational health team to set guidelines that were realistic; we took into consideration the anxiety associated with any fitness evaluation, but still protected our members. However, even with these safeguards in place, it took an exceptional amount of diplomacy and tact to overcome the suspicion and hesitancy of a number of our members.

Fortunately, thanks to the professionalism of our team of peer fitness instructors, many members of the first round of companies took part in the fitness evaluations and received positive outcomes–and the general membership began to see the value. Just as importantly, no one suffered any adverse consequences. No one was laid off or reassigned, and no one had to pay for an ambulance ride.

The results were very positive: We identified private issues with several members and were able to refer them to their personal care provider–issues that were addressed in private, not on the fireground. Having completed the fitness evaluations, members were given a snapshot of their fitness levels and offered the opportunity to develop personal-improvement plans with the peer-fitness trainers in their districts. The evaluations gained overall acceptance and momentum. By the time we finished the evaluations for the year, a number of members who had originally opted out called to see if they could still take advantage of the opportunity.

Phase II: SAFE
Our next challenge was to implement the fireground fitness test. We started with the intent to use the old SET with some modifications.

First, we identified the things that always went wrong when we used the SET. For example, we would take a blood pressure when people came out to take the test. We all know how amped up everyone gets for a physical evaluation, and how our blood pressure drops back to normal right after the test is complete. Also, in the past, we had allowed anyone to watch as we put members through the SET. This created the feel of a “spectator sport” for bystanders. Another problem was the attitude of the instructors. They encouraged the young studs and cajoled those who struggled. They were often seen standing off to the side, joking, and to the members waiting for their turn, it appeared that they were joking about the people running through the test–not a healthy way to feel before beginning your own test. It felt like the entire system around the SET had become poisoned.

With all that in mind, we needed a completely new approach. Since we had already completed fitness evaluations, we realized that we only needed to test fireground capability. We needed a way to demonstrate members’ fireground effectiveness, so we decided to base the evaluation on the skills and abilities needed for a bread-and-butter apartment fire.

During an academy, we make our members verbalize every step of every skill as they complete it. Further, more often than not, when watching a new firefighter throw a ladder, for example, you can see them quietly verbalize the commands as they work. It happens without thinking; it has become automatic.

We in the training division constantly harp on doing a thorough size-up, evaluating where and why you throw a ladder, when you force a door, check the alarm panel before you begin operations, etc. However, when testing fireground skills, cognitive skills are too often omitted from the process. DFD Chief Jimmy Lynch had read about diminishing cognitive function when the body is under physical stress. The study had come out of the military, but we realized it was relevant for the fire service as well. It was out of these discussions about how to work in cognitive skills that the SAFE test was born.

We started with an SCBA-competency and PPE-donning station. Every police officer has to undergo a certification process with their weapon at least annually, so we felt it wasn’t much of a stretch to require our members to certify with their primary protective equipment–a process that included a series of questions about use and function.

From the PPE station, we hurry the member toward the drill tower and the prepositioned engine. The member drags a supply line off the engine, hooks it up to the hydrant, and answers a series of questions about traffic safety and the process itself. The member then approaches the drill tower carrying some forcible-entry equipment and a power saw. The member answers questions about the use and safe operation of the equipment. Then the member stages the tools and answers a series of questions about building size-up and personal safety size-up. The member then raises a prepositioned 35′ ladder and answers questions about ladder use.

This process continues through a series of nine skill stations (set up in an order similar to normal fireground operation), all requiring a cognitive response. The firefighter finishes on the top floor of the drill tower, on air, carrying a section of 2½” hose and a forcible-entry tool, and must then respond to a request to provide a conditions, actions and needs (CAN) report on the radio. The member then verbalizes our mayday procedure.
During the delivery of the SAFE, we did a little unofficial experiment. We tested a number of former training officers, all of whom had experience teaching LUNAR (Location, Unit number, Name, Assignment and Resources) reports. We asked them to give a mock LUNAR report at the start of the SAFE, and they each rattled off the report without a hitch.

Then at the end of the test, when they were at the top of the drill tower with an elevated pulse rate, we asked them to give a LUNAR again. Approximately half of them were unable to give an accurate systematic report–and this was after undergoing just the physical exertion of the SAFE; there was no heat, no smoke, no one was trapped, and yet, some of them truly struggled with the report. (As a result, we have since changed our mayday and RIT protocols and eliminated the LUNAR report in favor of the who/where/why/what report; this proved to be much easier for members to accurately relay under physical duress.) The experiment emphasized the importance of training our members to make radio reports, size-ups, maydays, etc., as almost a function of muscle memory.

By adding a question-and-answer element to the SAFE, we increased the pre-test stress level. To offset the stress of anticipation, we provide a complete packet on the SAFE that includes a layout of the entire process. We provide every question we would ask–and every answer too. The test is not a “gotcha”; we want members to succeed. We capitalized on our firefighters’ aversion to looking foolish, knowing that they would all pour over the questions and answers before coming to training. The end result: They learned (or at least memorized) the information we wanted to emphasize.

Another precaution: We isolated the drill ground during the test. Only one company was allowed on the drill ground at a time, and they were staged behind a rig so no one was on display while they went through the SAFE. Your own crew was there and had the option of cheering you on, but again, this was no “spectator sport” for bystanders. In addition, we sat down with all of our instructors and discussed how to help people be successful in the test. We need department personnel to know that we are here to do everything in our power to ensure they’re prepared to do what they do best in the safest way possible. Furthermore, we can best accomplish that by ensuring that the Training Division is the place where people feel safe to make a mistake, to learn new things and to get better.

Final Thoughts
For the Denver Fire Department, the SAFE has been a significant leap forward. All of our trainings now include the cognitive pieces. In retrospect, it seems like an obvious step, but it’s certainly one that most departments have not yet implemented. Fortunately, the DFD can finally practice the way we play.

Special Thanks
I would like to recognize the distinct contribution of the DFD’s wellness committee. Without the support of this dedicated and innovative group of people, we never would have gotten the SAFE program off the ground. The committee included representation from our union, Local 858; Kaiser Permanente, our primary healthcare provider; Lockton, the company that helped negotiate our healthcare benefits; the city’s risk manager; the Safety and Training Division; and our employee’s medical clinic. In years prior, with the cooperation of the previous administration and some $5 grocery discount cards, the committee had been able to convince most of the department members to have a “well physical.” Unfortunately, other than these well physicals, the committee members felt they were able to provide just moral support to the concept of fitness, and they were growing frustrated as a result. It was only by empowering and acting on the suggestions of this group that were we able to overcome a number of obstacles.