The mathematics of hiring failure reveals itself in what doesn't happen. A Series B company with 75 engineers screens 80 candidates per month. 20 reach final interviews. The company extends 2.5 offers monthly. It needs 5 hires to meet growth commitments.
This looks like a sourcing problem or a closing problem. It is neither. It is a measurement problem.
Twenty engineers rejected in final-round interviews were tracked for 18 months. Eighteen of the twenty demonstrated clear professional success: eight joined companies with reputations for technical rigour, five earned promotions to senior or staff engineer, three maintained significant open-source projects, two founded venture-backed companies.
The rejection reasons follow predictable patterns: "Not strong enough on algorithms." "Seemed uncertain in system design." "Communication wasn't clear in behavioural round." None correlate with observed outcomes.
The engineer who "struggled with algorithms" now works as a staff engineer at a company famous for technical standards. The one who "seemed uncertain in system design" leads a platform team that successfully executed a complex architectural migration. The percentage of rejected candidates who would have succeeded if hired appears to exceed 90% among candidates who reach final rounds.
This is not a hiring process operating near optimal performance. This is a process that systematically rejects the engineers it should hire.
The economic cost compounds across three dimensions:
First, opportunity cost: 28 unfilled positions at 12 weeks vacant produces approximately $3.5m in annual lost output.
Second, process cost: engineering teams invest 4 hours per week in interviews at $110 per hour, totalling roughly $1.7m annually.
Third and largest, false negative cost: 126 engineers rejected annually who would have succeeded (90% of 140 post-screen rejections) represent roughly $25m per year in lost contribution, calculated conservatively at $200,000 annual value per engineer.
The total cost approaches $30m annually. The company hires 30 engineers at an all-in cost of $1m each. A senior engineer's fully-loaded salary: $230,000. The company spends 4.4× the cost of employing an engineer just to hire one.
The Pattern
The standard technical interview follows a format developed at technology companies in the 2000s. Phone screen, coding round, system design, behavioural assessment, hiring manager evaluation. The process consumes 6-8 hours of candidate time and costs approximately $1,000 per candidate in interviewer time and coordination overhead.
A senior engineer at a payments company had spent seven years architecting their fraud detection system, mentoring twelve engineers and shipping features that processed $2bn in annual transaction volume. When she interviewed at a fast-growing Series B fintech, she failed the coding round.
The problem: implement a function to find the longest palindromic substring in a string. She spent 15 minutes working through edge cases—empty strings, single characters, even-length versus odd-length palindromes. She wrote working code but struggled to optimise it to linear time. The interviewer gave hints. She understood them but couldn't implement the dynamic programming solution quickly enough. Time expired. The feedback: "Struggles with algorithms. Not strong enough for senior level."
She now works at a competitor. Her manager there describes her as exceptional: "She thinks through edge cases other engineers miss. Her code is bulletproof. She's the person I want reviewing anything that touches production." The palindrome problem she failed? She has never encountered anything like it in eight years of professional work.
This pattern repeats at scale. The implicit assumptions underlying technical interviews: performance on algorithmic problems predicts performance writing production code. Whiteboard coding without IDE support assesses coding ability. System design interviews on whiteboards predict ability to design real systems. Behavioural interviews predict collaboration quality. All of these dimensions can be assessed accurately in 1-hour blocks by interviewers who have never worked with the candidate.
Research demonstrates these assumptions are false.
LeetCode performance correlates with interview success at 0.27—weak enough to explain less than 10% of variance. No research shows LeetCode performance correlates with job performance. The problems test pattern recognition in competitive programming scenarios that professional engineers rarely encounter.
Whiteboard coding assesses comfort with artificial constraints. Professional engineers never write code without syntax highlighting, autocomplete, compiler feedback, or ability to run tests. A controlled study tracked 48 developers: 34% who completed coding challenges in private failed equivalent problems under observation. The cause: anxiety, not ability. The senior engineer's failure had nothing to do with her engineering capability. It measured her comfort performing under artificial time pressure on problems divorced from actual work.
System design interviews compress weeks of research, prototyping and iteration into 45-60 minutes with no reference materials. The format rewards breadth of memorised patterns over depth of reasoning. Engineers who design excellent systems in their jobs often perform poorly because they think carefully and ask probing questions—behaviours that read as uncertainty in time-constrained interviews.
Structured behavioural interviews achieve validity coefficients of 0.51, meaning they explain 26% of variance in job performance. Unstructured interviews achieve only 0.33 validity, explaining 11%. The format assesses ability to tell compelling stories about past work, not ability to collaborate on future work.
Work sample tests achieve validity coefficients of 0.54 (85 years of hiring research confirms this). At best, interviews explain 26% of variance in job performance. The remaining 74% depends on factors interviews cannot assess: how someone responds to this codebase, this team, this manager, this domain.
When you reject someone in an interview, you have at most 26% confidence they would have been a poor performer. There is a 74% probability that other factors determine their success. This rejection represented a 74% probability that the company passed on someone who would have excelled. They will never know. She works somewhere else now.
The process therefore selects on a variable that explains one-quarter of the outcome it cares about while inadvertently selecting against qualities that matter: thoroughness reads as slowness, careful consideration reads as uncertainty, principled pushback reads as difficult personality.
The Economics
Three types of costs shape hiring decisions: false positive cost (bad hires who slip through), false negative cost (good engineers rejected) and process cost (time and money spent interviewing). The relative magnitude shifts dramatically as companies grow.
At 15 engineers, a single bad hire is catastrophic. Consider a seed-stage startup that hires its fifth engineer. This person is responsible for 20% of engineering output. If they cannot ship, the product slips by months. If they create conflict, they poison 20% of the team. The founder personally interviewed this candidate for 8 hours across four sessions. The opportunity cost seemed irrelevant compared to the risk. The process was exhaustive because false positive cost threatened company survival.
At 150 engineers, the same bad hire is a rounding error. They consume perhaps 5% of one manager's bandwidth. The team ships its roadmap with or without them. Within six months, they separate or improve. The cost is manageable: $300,000 (salary, separation, replacement). Meanwhile, the company is rejecting 70-80% of candidates to maintain this level of rigour. At 100 candidates per month, this means 70-80 rejections. If 90% of those rejected would have succeeded—as the data suggests—the company is passing on 63-72 good engineers monthly. At $200,000 annual contribution each, that is $12-14m in monthly lost output. Annual false negative cost: $150m. To avoid $3m in bad hire costs, the company spends $150m in missed opportunities. False negative cost now dominates by two orders of magnitude.
Yet most companies develop a rigorous process at 20 engineers—when the founder can no longer interview everyone—and maintain it with minor modifications from 20 to 200 engineers. Five to six rounds, consensus requirements, bar raiser approvals designed to prevent false positives. The process that was rational at 20 engineers becomes actively harmful at 100. Nobody questions it because "maintaining our bar" sounds like professionalism. The actual effect: systematic rejection of engineers who would have excelled.
The standard response to hiring velocity problems—add recruiters, expand sourcing, optimise candidate flow—addresses the wrong constraint. The company processes 80 candidates per month but hires 2.5. The bottleneck is not candidate availability. The bottleneck is a 71% rejection rate applied to a candidate pool where 90% of rejects would have succeeded.
Three approaches present themselves. First: maintain current process and accept reduced growth. This delays product delivery, pushes out the next funding round, and creates compounding constraint. Second: lower the passing thresholds while maintaining the same interview format. This increases false positives without addressing the root problem—interviews measure the wrong thing. Third: abandon prediction in favour of measurement. Rather than predicting job performance through 6 hours of interviews, measure it through 90 days of actual work.
The economic case for trial-based hiring becomes clear with some elementary calculations. Research on trial-based hiring at companies with 150-300 engineers shows that structured 90-day trials achieve 70% success rates. From 200 candidates, a trial-based process screens through two rounds with 40% rejection, filtering only obvious mismatches. 120 enter trials. With 70% passing, the result is 84 successful hires—a 42% yield per candidate.
The traditional process with 71% rejection produces 58 hires at 85% success rate: 49 successful hires—a 25% yield. Trial-based hiring produces 71% more successful hires per 100 candidates.
The cost of trial failures must be considered. Each failed trial costs approximately $50,000: three months of salary and benefits ($37,500), onboarding expenses ($5,000), and manager time ($7,500). For 200 candidates, 36 trial failures cost $1.8m. This appears expensive until compared against the cost of false negatives: 126 rejected engineers who would have succeeded represent $25m in lost annual contribution.
The trial-based approach produces 84 successful hires versus 49 under the current process—a net gain of 35 engineers. At $200,000 annual contribution each, these 35 additional engineers generate $7m in incremental value in their first year alone. The trial failure cost of $1.8m produces a 3.9:1 return in year one.
The Case Study
The VP of Engineering at a Series C company with 180 engineers faced a board mandate in March: reach 280 engineers within 12 months to hit revenue targets. Current velocity: 12 hires monthly. Simple arithmetic revealed the problem: 12 × 12 = 144 hires, leaving them 56 engineers short. The gap threatened $15m in committed roadmap delivery.
The existing process followed industry patterns. Five rounds, 7.5 hours of candidate time, consensus requirement, bar raiser approval. Time from first contact to offer: seven weeks. Post-screen rejection rate: 74%. The VP had inherited this process and initially assumed it worked well—it had been refined over three years, after all.
Then the tracking began. Over six months, the VP personally reviewed feedback for 40 engineers rejected in final rounds. The patterns were disturbing. "Not strong enough on algorithms" for a candidate with ten years of production experience. "Seemed uncertain in system design" for an engineer who had scaled a system to 50m users. "Communication wasn't clear" for someone whose technical blog had 100,000 readers.
LinkedIn research revealed the outcome. Within four months of rejection, 35 had accepted positions at comparable technology companies—some at competitors, some at companies with reputations for more rigorous hiring. At twelve months: 28 showed clear professional progression. Promotions to senior roles. Technical leadership positions. One had become a distinguished engineer. The company was systematically rejecting engineers who succeeded elsewhere.
The data went to the leadership team. "We are not hiring too slowly because we lack candidates. We are hiring too slowly because we reject 90% of people who would be good." The hypothesis: the process optimised for false positive avoidance at the expense of false negatives. The company spent tremendous effort preventing bad hires—consensus requirements, bar raisers, algorithmic rigour—while inadvertently rejecting hundreds of engineers annually who would have excelled in actual work.
The proposal: trial-based hiring for all individual contributor roles. Two-round screening to filter obvious mismatches, followed by 90-day structured trials with explicit success criteria. Not for VP or Director positions initially—those would maintain traditional evaluation—but for Software Engineer through Staff Engineer levels, which represented 85% of hiring volume.
The objections materialised immediately. The CEO: "Won't this let weak people in?" The CFO: "What's the cost if trials fail?" The Head of HR: "Are there legal risks? What if someone claims discrimination?" The engineering managers: "We don't have time to evaluate trial hires. We barely have time to interview."
The VP came prepared with a constrained proposal: three months, two volunteer teams, target of 15 trial hires. Low risk, measurable outcomes, easy to kill if it failed. The platform infrastructure team and a product engineering team agreed to participate. Other teams would continue the traditional five-round process, providing a control group for comparison.
The trial process simplified dramatically. Round 1: technical phone screen filtering for basic coding ability. Rejection rate: 28%. Round 2: hiring manager and peer session assessing whether any obvious mismatch existed—not whether the candidate was excellent, but whether clear reasons existed to believe they could not do the work. Cumulative rejection rate: 42%.
Candidates who passed entered 90-day trials. The hiring manager defined success criteria on Day 1: ship 2-3 production pull requests by Week 6, collaborate effectively, communicate clearly, complete assigned tasks independently by Week 10. Weekly one-on-ones provided continuous feedback. Week 12 concluded with formal evaluation.
Three months produced measurable results. The two pilot teams processed 85 candidates. 36 passed screening. 49 received trial offers. Trial outcomes: 34 completed successfully (69%), 10 failed (20%), 5 departed voluntarily (10%). Net successful hires: 34 in three months.
The control teams processed 120 candidates during the same period. With 74% rejection rate, 31 candidates received offers. One departed within three months. Net successful hires: 30 in three months.
Yield per 100 candidates: 40% for trial-based teams versus 25% for traditional—a 60% improvement. Time from first contact to start date: 18 days for trials versus 49 days for traditional—63% faster. Cost per successful hire: $2,400 for trials versus $4,100 for traditional.
Manager satisfaction exceeded expectations. After initial scepticism about evaluation burden, managers reported confidence in their assessments. "I know who's good after six weeks of working with them," said one manager. "In interviews, I would agonise over whether someone's hesitation meant they didn't know the answer or were just thinking carefully. Now I just watch them work. Do they ship good code? Do they ask smart questions? Do they take feedback well? It's obvious." The ambiguity of predicting performance from interviews gave way to the clarity of observing actual work.
Unexpected findings emerged. Trial failures became predictable early. Eight of ten failures showed obvious problems by Week 4—consistently missed deadlines, pull requests requiring extensive rewrites, minimal participation in team discussions, or inability to work independently. The full 90 days proved unnecessary for poor performers; the signal appeared within four to six weeks. "I usually know by Week 3," one manager admitted. "But I give them until Week 8 to be sure I'm not just seeing a rough start."
More striking: some trial passes were surprise discoveries. Four of the 34 successful hires became standout performers—top 20% in six-month performance reviews, consistently delivering high-quality work, mentoring other engineers effectively. All four would likely have failed traditional interviews.
One engineer, 52 years old, was slow in whiteboard settings but thorough in production work. His first pull request took nine days to submit—colleagues worried he was struggling. The PR had zero bugs, comprehensive tests and documentation that prevented three future incidents. "I'm not fast," he told his manager. "But I ship code that doesn't break." Six months later, he was the engineer everyone wanted reviewing their critical infrastructure changes.
Another hire lacked formal computer science vocabulary but demonstrated deep practical understanding. In her screening interview, she couldn't explain Big O notation. In production, she identified and fixed a performance bottleneck that had stumped three senior engineers. She didn't know it was an O(n²) problem; she just knew the loop was checking every item against every other item and that seemed wasteful. Traditional interviews would have rejected her for not knowing the terminology. Trials revealed she understood the concepts better than people who could recite the definitions.
Two others were careful thinkers who appeared uncertain under interview pressure but proved exceptionally reliable when given time to consider problems properly. These were exactly the engineers the traditional process filtered out—people whose strengths became visible only through real work.
Based on pilot success, leadership approved expansion to all individual contributor hiring. Months 7 through 18 constituted the measurement period—12 full months of trial-based hiring at scale.
Traditional period (six months before pilot): 720 candidates screened, 187 hired (26% conversion rate). Trial period (12 months): 1,240 candidates screened, 682 hired into trials (55% conversion rate), 478 successful permanent hires after trial outcomes.
The trial approach produced 156% more successful hires from 72% more candidates.
Of 682 trial hires, outcomes distributed as follows: 478 successful completions (70%), 135 trial failures (20%), 69 voluntary departures (10%). Average duration to failure: 6.5 weeks—confirming that poor performers became obvious early.
The critical question: did trial-based hiring produce worse engineers? Bad hire rate—defined as separations within 12 months due to performance problems—increased slightly. Traditional process: 18 bad hires of 187 total (9.6%). Trial process: 53 bad hires of 478 total (11.1%). The difference: 1.5 percentage points.
In absolute terms: traditional would generate 36 bad hires annually (18 × 2), trial generated 53. The cost: 17 additional bad hires. The benefit: 104 additional successful hires (478 versus 374 annualised traditional). The ratio: 6:1. For every bad hire the trial process admitted beyond what traditional interviews would have caught, it captured 6 good hires that traditional interviews would have rejected.
The economic analysis proved compelling. Traditional approach (annualised): process cost of $710,000, opportunity cost of $16m (70 positions continuously open), bad hire cost of $10.8m. Total: $27.5m for 374 hires. Cost per successful hire: $73,500.
Trial approach (12-month results): screening cost of $525,000, trial failure cost of $6.75m, opportunity cost of $6.8m (30 positions continuously open), bad hire cost of $15.9m, manager time of $382,000. Total: $30.4m for 478 hires. Cost per successful hire: $63,600.
The trial approach cost $2.9m more in absolute terms but produced 104 more successful hires. Output value told the complete story. Traditional approach: 374 hires × $200,000 = $74.8m annual output. Trial approach: 478 hires × $200,000 = $95.6m. Net value after costs: traditional produced $47.3m, trial produced $65.2m. The trial approach created $17.9m more value annually—a 38% improvement.
The VP presented these results to the board at the 12-month review. The CEO looked at the data for thirty seconds. "The data is clear. Why didn't we do this sooner?" The answer was careful: "Because 'lower the bar' sounds like moral failure. What we actually did was change what we measure. The bar didn't get lower. It got more accurate."
The CFO had a different question: "What about the 17 additional bad hires? That's real cost." The response: "It is. But those 17 bad hires cost us $5.1m. The 104 additional good hires generated $20.8m in value. We spent $5.1m to gain $20.8m. That's a trade I'll make every time."
The company ended the year at 298 engineers—two short of the board target but far ahead of where the traditional process would have landed them. More importantly, they had discovered something valuable: the engineers failing traditional interviews weren't worse engineers. They were engineers whose strengths didn't translate to whiteboard performance. Dozens of capable engineers who would have been rejected by the process that "maintained the bar" but thrived when evaluated through actual work.
What Companies Get Wrong
Companies speak of "maintaining the bar" as if an objective standard exists for engineering quality. There is not one. What "the bar" actually measures is performance in six hours of interviews: algorithmic ability, whiteboard communication, behavioural storytelling skill. Not ability to ship production code, collaborate over months, handle ambiguity, or grow from feedback.
The conflation creates expensive confusion. When you reject someone in an interview, you have at most 26% confidence they would have been poor performers. There is 74% probability that other factors—invisible in six hours of artificial interaction—determine their success.
The reframe changes the optimisation. "The bar" should not be "how well do they interview" but "what minimum signal is needed before we try working with them?" The company requiring exhaustive interviews before hire loses to the company requiring basic competence before trial.
Despite overwhelming economics—3.9:1 ROI, 71% more successful hires per candidate—most companies will not adopt trial-based hiring. The barriers are psychological, not economic.
The first barrier: sunk cost. The current process required years to develop. The company trained dozens of engineers to conduct interviews, built rubrics and calibration systems and established "maintaining our bar" as cultural identity. Admitting this infrastructure optimises for the wrong outcome requires admitting years of effort produced a process that systematically rejects excellent engineers.
The second barrier: visibility asymmetry. False positives are visible—poor code quality, missed deadlines, eventual separation. False negatives are invisible. When an excellent engineer is rejected in interviews, they go work somewhere else. No one at the rejecting company knows they passed on a future staff engineer. This asymmetry creates organisational bias toward "rigorous" processes that generate false negatives at scale.
The third barrier: perceived risk. "Lower the bar" sounds like moral failure. Yet the evidence shows that "the bar" measures interview performance, not job performance. The correlation between the two is weak enough (0.51 at best) that optimising for interview performance systematically excludes excellence in actual work.
The broader pattern appears across technology companies at similar growth stages. A process developed when the company had 15 engineers persists largely unchanged when the company has 100 engineers. The economics of hiring change radically with scale. At 15 engineers, the dominant risk is a bad hire destroying culture. At 150 engineers, the dominant risk is rejecting so many good engineers that the company cannot staff its roadmap.
But "lower the bar" sounds like moral failure, so companies maintain processes that cost tens of millions annually rather than admit that hiring is an economic optimisation problem that changes with scale.
The key insight: hiring is economic optimisation under changing constraints. At 10 engineers, a single bad hire threatens company survival. At 100 engineers, the same bad hire is manageable while rejecting hundreds of good engineers is catastrophic. The optimal process at 10 becomes actively harmful at 100.
The engineer who would have been the company's best hire—careful where others are careless, thorough where others are quick and principled where others are agreeable—failed the algorithm round or seemed uncertain in system design. That engineer works somewhere else now. The company that hired them probably didn't have a better interview process. They probably just moved faster, rejected fewer people, and discovered through real work what interviews fail to reveal: that the correlation between whiteboard performance and production engineering capability is weak enough that optimising for the former systematically excludes excellence in the latter.
The cost of this mistake, compounded over years and hundreds of rejected candidates, exceeds the cost of most engineering decisions companies agonise over. Yet it persists, invisible in its magnitude, because false negatives leave no trace except in the companies that benefit from them.
The only way to know if someone will succeed is to work with them. Interviews predict job performance at 0.51 correlation in structured cases, 0.33 in unstructured cases. At best, you explain 26% of variance. The remaining 74% depends on how someone responds to this codebase, this team, this manager, this domain. These factors are invisible in six hours of interviews. They become visible in 90 days of real work.