Ibrahim AbuAlhaol
Every engineer who joined the team this year finished onboarding on time, and the dashboard that counts merged pull requests showed them all on track. Most of that code was written with an AI assistant. In week six, one new hire's model for predicting which customers would leave looked almost perfect in testing. A senior engineer's review found why: one input had been computed with data from after the moment of prediction. Asked whether near-perfect was plausible, the new hire could not say.
The dashboard treated every merged pull request as proof that the engineer understood the work. It was proof that the code ran. That is not a cheating problem. It is a measurement problem, and leaders can fix it in one short conversation per new hire, with the AI asking the questions instead of answering them.
Right answers used to mean something
Any test can mislead in two ways. Someone who knows the topic can slip, and someone who does not can guess right. Every educator knows the second from multiple choice. A test works only while lucky guesses are rare.
ALEKS, the tutoring system built on Knowledge Space Theory, keeps lucky guesses rare on purpose: it avoids multiple choice and asks for answers entered much as they would be written on paper.1 The theory's founders put the stakes bluntly: in real life, "the occurrence of lucky guesses would render the result of the assessment so unreliable as to render it practically useless."2
Engineering teams run the same kind of test every day without calling it one. A merged pull request, a passing notebook and a completed onboarding checklist are each read as proof of what a new hire knows. That reading held while producing working code without the knowledge was hard. It no longer is.
AI makes every right answer suspect
Picture 20 new hires, 10 of whom understand data leakage. Ask them to spot the input that leaks the answer into a model. Without AI, nine of the 10 who understand get it right, and one of the other 10 guesses right: 10 right answers, nine of them earned. With an assistant at hand, nine of the 10 who do not understand also get it right: 18 right answers, only half of them earned.
Asking more questions does not help. Ordinary lucky guesses are random, so they average out over many questions. The assistant is one cause behind every answer, so more aided answers only make you more confident in the wrong conclusion. Only asking without the assistant removes it.
For the technical reader: the detector arithmetic
Read a right answer as an alarm that says "this hire understands." Without AI it fires for nine in 10 hires who understand and one in 10 who do not, a likelihood ratio of nine; ALEKS's own data show about seven to eight.1 With AI at hand it fires for nine in 10 of both, and the ratio falls to one: a right answer tells you nothing. Knowledge Space Theory's assessment models give every question a slip rate and a guess rate and treat each guess as independent of the next,3 which is why they grow more certain with each aided answer, of a state the hire does not hold.
The week-six model scored an area under the curve (AUC) of 0.97: it ranked leavers above stayers 97% of the time, because one feature used data from after the prediction time. Two more checking questions, with answers: a 0.4-point gain on 1,000 test examples is usually inside the noise; and a fraud model that catches 90% of fraud and flags 5% of legitimate cases, where 2% of cases are fraud, is right about only one flag in four.
The effect reaches what people learn, not only how they test. In a randomized study, 52 developers learned a new Python library with or without an AI assistant. Both groups took about the same time, but those with the assistant averaged 50% on a quiz afterwards, against 67% for those without.4 In a trial with nearly 1,000 high school students, unrestricted GPT-4 raised practice grades by 48%, then left them 17% behind peers once access was removed.5
People rarely notice from the inside. Participants who solved reasoning problems with ChatGPT scored higher but overestimated their scores, and those who knew most about AI judged themselves least accurately.6
Keep the knowledge that checks the work
The usual objection is fair: the assistant writes the data wrangling, the plots and the training loop, so why test any of it? Most of the code can go to the assistant. What cannot go is the knowledge that tells you whether the result is believable.
Call it checking knowledge, my term: what a person must know to tell whether a tool's answer is right. On an AI team that means spotting data leakage, knowing how much a score wobbles between test sets, and weighing base rates. Assessment researchers call the wider skill evaluative judgement, the capability to judge the quality of work, including a tool's.7
The stakes are real. A survey of machine-learning-based science found data leakage in 17 fields, affecting at least 294 papers, some with wildly overoptimistic conclusions.8 Researchers made those errors, and people who knew what to check found them.
Let the assistant write the code. The knowledge that checks it has to stay with the engineer. The aim is not to ban the assistant: in a study of 5,172 customer-support agents, AI help raised issues resolved per hour by 15% on average, with the largest gains for the least experienced.9
Let the AI ask the questions
The tool that hides what a new hire knows can also measure it. Give the first 15 minutes of a new hire's first design review to a short conversation with an AI assessor, run live with the reviewer present and no assistant open. That answers the first question every leader asks: how do you know it was unaided? Universities answered it long ago with the oral exam, and AI makes it cheap enough for every new hire.
The assessor asks three to five questions on the checking knowledge this project depends on, each answered in the hire's own words. Is this gain bigger than the noise? Which input could leak the answer? Of the cases this model flags, how many are real? Three to five questions cannot map everything a person knows, but they can name the one topic to teach next.
Three rules keep it fair. Try every question on your current team first, senior engineer included, and keep only those that people who know the topic pass. Show results to the new hire first, and to leaders only as topic counts. Put in writing, before the first one runs, that results never enter performance reviews. The check never decides a launch; a miss books an hour with a mentor. The questions double as practice, and practice testing earned the top rating in a major review of study techniques.10
The same logic gives you one question for any vendor of AI tutoring: show me results from a test taken after your tutor was switched off.
Grades raise the stakes
In a course, the assessor's verdict becomes a grade, and a grade is a decision about a person. Assessment researchers argue that cheating matters because it breaks validity, the claim that a result shows what it says it shows.11 The European Union's AI Act lists AI systems that evaluate learning outcomes, like those that evaluate workers' performance, as high-risk, a class that requires human oversight.12
So keep the AI assessor as ungraded practice until it shows validity with your own students. Where a check counts, tell students in advance, offer a human examiner instead, carry accommodations over, give every student the transcript and have a person review any disputed mark. I represented my department in the consortium that drafted Jordan's National AI Ethics Charter, and these are the safeguards I would ask of any graded AI check. Unaided still means a supervised room.
What we do not yet know is how well AI assessors measure, so pilot one against your own reviews before trusting it. Finding the checks a new hire cannot yet make is the questions-first move of The Missing Step, applied to a whole team.
What leaders should do
- List the checking knowledge each role must hold without help, next to the team's prerequisite map, and mark which coding can go to the assistant. The senior engineer owns the list.
- Open each new hire's first design review with a 15-minute unaided AI assessor conversation on checking knowledge, after your current team has taken it, with the no-performance-review rule in writing first.
- Track which checking-knowledge topics recur in review catches and incident reviews, by topic and never by person, and add a guide page for any topic that recurs.
- In a course, move the graded check into a short supervised oral or written task, keep AI assessor sessions as ungraded practice, and publish the appeal route in the syllabus.
