Image of woman evaluating candidates for a job
Skills Testing

Not Hiring the Wrong Engineer

Using the right skills tests to eliminate bad-fitting applicants as early as possible

Among the biggest mistakes that an organization can make is hiring the wrong person.

Hiring the wrong person lowers productivity, harms employee morale, and wastes all the time and money involved in training, managing, potentially firing, and then re-filling the position, re-training, etc.

An Appropriate Fit

Actually, “wrong” isn’t the best term because it implies an issue with the applicant — the applicant could be great — but they might not be a great fit.

Let me rephrase: the organization’s biggest mistake is not hiring the most appropriate person for each job role.

During my brief stint filling-in as an art director at a digital agency, I embraced the concept of appropriateness as the most crucial dimension when reviewing a design. Almost all of the designs I saw were good, but they weren’t always appropriate for our clients’ goals. Subjectivity remained, but the lens of appropriateness helped me focus on which areas needed the most improvement.

Appropriateness is also a critical consideration for effective hiring. Consider that an organization can hire a fantastic person, but their great qualities don’t guarantee that they are well suited or well aligned for the specific position.

So, how do you determine who is appropriate? The first step is ruling out those who are most clearly not a good fit.

The Problem With Resumes

The first step is evaluating resumes, while undoubtedly necessary, resumes are surprisingly not that helpful as they are a weak indicator of appropriateness. At best, resumes convey a certain level of professionalism, but they can also be deceiving since different organizations have vastly different expectations/definitions of the same job title. This fact diminishes much of the potential value of the work history section of a person’s resume. Nonetheless, a resume helps eliminate candidates that are overtly a bad-fit.

Skills Testing

Skills testing is the key feature of next two steps to narrow the applicant pool further while also reducing subjectivity.

Before I describe my approach to skills testing, it is essential to note that employers must ensure that skills tests are reliable and valid, yielding consistent results that predict success on the job. The EEOC’s Uniform Guidelines on Employee Selection Procedures detail how the EEOC will evaluate a testing method called into question; the agency offers additional guidance in its Employment Tests and Selection Procedures fact sheet.

I administer skills tests in two rounds.

Skills Test: Round One, Very Easy

The first round is an easy skills test. This test is not pass/fail, because I am looking for the candidate to not only arrive at the right answer, but I am also looking for them to get there with relative ease. I administer this test on phone-screenings, and it can take anywhere from 5 to 15 minutes. I ask candidates to write two simplistic coding functions similar to functions engineers at our organization frequently write and don’t think twice about.

I can imagine that this might seem pointless. Why administer an intentionally easy test?

I do it because many people with outstanding resumes fail at this stage. This skills test is a little awkward for the highly-skilled applicant (like asking a famous writer to recite the alphabet), but it saves both parties time, and potential embarrassment as the next skills test is more challenging.

Skills Test: Round Two, Very Typical

For practicality, the second test occurs on a subsequent day to prevent all of the scheduling and unscheduling that would result from booking all interviewers before the outcome of the first skills test.

The second skills test is to solve a problem that requires consuming multiple data sources, processing data, and making a few calculations. This test takes 25-45 minutes. This test is not overly trickly. It’s just a more in-depth problem, but it is also aligned with the type of work engineers at our organization regularly perform.

For legal compliance purposes and to make the test a useful indicator of future performance, it is crucial that the test realistically simulates normative work. There was a trend in the industry to ask unusual puzzles or test impractical knowledge, but these tests are falling somewhat out of fashion as they mostly indicate test-preparedness and don’t necessarily indicate on the job performance.

The in-depth skills test is pass/fail. Subsequent same-day interviews are canceled at the last minute if the applicant does not pass. Upon scheduling the in-person interview, I don’t indicate that there will be multiple rounds of interviews. If the applicant fails, I still try to have a pleasant chat and wrap up ahead of schedule.

Last Step: Structured Interviews

The final step of making an appropriate hire is the structured interview. This is something I plan to write about more substantially in another post. Still, generally, the structured interview should guarantee that all applicants get asked and ranked according to a uniform set of questions. The organization’s engineering progression framework provides a rubric for asking and scoring highly relevant questions.

Evaluating for Appropriate Abilities

All this testing and multiple rounds of interviews may seem like a lot of effort. As far as I can tell, no alternative yields better results.

When you think about it, the bottom line is that engineers must perform at a high-level day after day. A relevant skills test for an engineer is like watching a sprinter run — you can see right away if they have the ability or not.

Without the right skills testing, you might end up hiring an engineer who passionately follows the day-to-day news of software engineering, who studied advanced algorithms, who is extremely charismatic — but isn't very good at writing code.

About the Author

Ryan's FaceRyan Mahoney is the director of technology for the customer-facing technology department of state of Massachusetts’ public transportation agency. He has spent the past two decades leading engineering teams as a founder, director, manager, and tech lead working with brilliant engineers that make positive impacts with their work.

More Posts in Hiring