From discussions with hundreds of engineers and recruiters at companies ranging from 1 employee to over 5000, the two most common ways companies automate their screening process are: coding challenges and projects.

Coding challenges are online tests that evaluate general programming competency and algorithms knowledge. The online interface is meant to simulate a real coding environment. Several companies like Hackerrank and Codility are starting to gain traction with their code challenge solutions, revealing the many inefficiencies within the technical interviewing process.

Some companies prefer evaluating engineers using projects, having candidates submit a whole webapp, iOS app or machine learning model. The advantage of project-based evaluation is that candidates can write code in their own environment, and companies can see real software engineering work.

Both of these evaluation tools are a great improvement over resume screening because they allow companies to hire based on real skill. Here's a short analysis on the pro's and con's of each:

Challenges

Pro - Easy to grade: one big advantage of using challenges to screen engineers is that grading can be automated using test cases. This can dramatically speed up how long it takes to pass or fail an engineer. Instead of spending 5-10 minutes reading over every resume, you can instantly eliminate 70% of them since the candidate couldn't exhibit basic programming competency on a coding challenge.

Pro - Standardizes the process: sending out the same challenge problem to all candidates allows you to create better benchmarks for what a "good" prospective engineer looks like. This in turn also helps with making your hiring process more efficient.

Con - Limitations in scope: coding challenges aren't the best tool to assess certain skills. For example, OpenAI uses Hired Assessments's project-based evaluation product to assess whether a candidate can implement a model using TensorFlow. This isn't possible in a coding challenge environment that online tools like HackerRank and Codility offer.

Projects

Pro - Evaluate real software engineering tasks: coding challenges are great for determining basic programming competency and ability to write algorithms. But for certain roles, like a front-end web engineer, knowledge of algorithms might be completely pointless. A hiring manager would much rather because to see HTML and JavaScript code than a solution to a dynamic programming problem.

Pro - more realistic: allowing candidates to write code in their own environment brings your assessments closer to what an engineer would actually be doing on the job.

Con - time intensive for candidates: one difficulty with project-based evaluation is making sure that the projects are short enough that they can be completed in a reasonable amount of time. Having candidates spent days or even weeks on a project, only to get rejected, can be frustrating and tough for the candidate.

Con - difficult to manage: the process of writing a project specification, giving candidates access to a github or bitbucket repository, and emailing it to candidates individually is a hassle. Not to mention that candidates must then email the project back as a zip file to let the interviewer know that it has been completed.

If your company already evaluates engineers using projects or you've been meaning to build one into your process, try Projects with Hired Assessments. Projects centralizes the creation, invitation and evaluation of projects into a single platform. Once candidates complete the project, recruiters automatically get an email where they can view it, without any more zip files. Hired Assessments also provides the option to have the project code-reviewed by an experienced engineer.

So, which one is right for your company?

Both challenges and projects help you eliminate bias, standardize the process and make screening far more efficient. This in turns helps you cast a wider net and discover better engineers. Determining which method is right for your company depends on a number of variables, like the roles you're screening for and the volume of candidates your company sees.

Generally challenges are used by companies that are evaluating engineers on a high volume. These are ideal for screening more junior engineers or candidates that might not pass an initial resume screen but still might be technically proficient. Recruiters can also leverage coding challenges to screen engineers without even involving any of their own engineers, since grading can be done automatically.

Companies that need to evaluate certain skills outside of general programming competency and algorithms knowledge might opt to use a project-based approach. These work especially well for assessing candidates with over 3 years experience, and for general software engineering jobs, and software engineering jobs that include managerial responsibilities and that don't necessarily require knowledge of CS topics.

Here's more information on our Challenges and Projects products in case you're interested!