Duplicate payment
Duplicate requests and unintended side effects
- 75 min
- 4 domains
- Sample: Defense required
For anyone working with AI · Starting with tech
Graded on how you think and steer the work
Take on real-world missions with an AI agent and get a full breakdown—from how you directed the AI to what you shipped.
Practise on your own, or share your result with hiring teams.
Why the old signals say less
AI can polish it for anyone
AI handles many clearly specified tasks
It can look good either way. You can't see who decided what
So we think the output alone isn't enough anymore.
What you really want to know
Does this person know what's right,
and will they catch the AI when it's wrong?
The result
The first line states the rule in one sentence: every piece reached its required level, 7 of 7, or which piece hasn't yet and the level it needs.
A piece has levels drawn from C1 to C5; it need not have all five. Each piece has its own required level.
Time used, out of the time given.
Read the request handler and trace the retry path.
bun test requests.test.tsCaptured change available
DiffThe timeline shows recorded work; incomplete coverage is marked separately.
Sample result: the verdict, each piece's levels, and attempt details.
The first line states the rule in one sentence: every piece reached its required level, 7 of 7, or which piece hasn't yet and the level it needs.
Read the request handler and trace the retry path.
bun test requests.test.tsCaptured change available
DiffThe timeline shows recorded work; incomplete coverage is marked separately.
Sample result: the verdict, each piece's levels, and attempt details.
What we measure
A submission shows what was delivered. A captured process gives a reader something to follow. This illustration compares records, not scores based on when AI was first used.
Visible tests pass for both; the diffs look alike
The accepted work and mission version identify what was evaluated.
Messages, commands and file changes within the retained record.
A reader can follow available evidence with questions rather than inferring everything from one file.
Missing records remain gaps. They do not prove no work happened. Evaluation follows the task requirements.
Read a work-shaped brief
Sample mission topics—not the live catalog. Read the full brief and deliverables before choosing.
Duplicate requests and unintended side effects
A balance that doesn't match its records
An issue that takes multiple sources to trace
Illustrative times and piece counts only
Separate symptoms from requirements before editing
Read only the symptom
See a single error → Ask AI for an immediate fix
The requirement is still unverified
Read, then decide
Inspect terms and supplied data
Check the repair against the deliverable
There is reasoning and a check to inspect
From brief to result
Check the brief and rules, do the work in the workspace, and submit for grading.
Read the published brief and mission version. Some internal runtime details aren't shown upfront.
brief Situation and deliverables workspace Starting work files terms Time, credits and acknowledgements Each version and preset locks the timer, AI budget, allowed models, tool policy, starter files, and grader. Only the model you pick and its outputs vary.
Situation The work to handle Deliverables The required output Content language As published for the mission Check whether each piece met its required level. Doing extra on one piece won't make up for missing another.
What a result refers to
Read the version, submission and piece levels together to understand the result’s scope.
Read that version’s brief to see what this attempt must deliver.
Read the acknowledgements to understand this attempt’s rules.
Read files, direct AI and inspect changes within the mission’s rules.
Review the intended work and follow the mission guide for acceptance.
Compare each piece’s achieved level with its required level.
The pass rule
Every piece
A piece that meets its bar cannot compensate for one that does not.
C1 is the core. Deeper levels do not count until it passes.
The complete explanation and worked example are on How we measure.
Terms belong to the started attempt, not a later choice based on its outcome.
A name can span several versions. Read the version beside the result.
Every piece has its own required level.
A hold is not a fail; corrections keep their history.
Read each source for what it shows
Different sources answer different questions. This compares information, not hiring outcomes.
| Written background | Final files | Conversation | | |
|---|---|---|---|---|
| What to read | Described experience | The submitted artifact | The explanation | Each piece’s level |
| Further context | Check other sources | Compare with requirements | Read beside the work | Captured, retained evidence |
| The reviewer’s role | Assess relevance | Inspect quality | Interpret the explanation | Read terms and limitations |
A result does not guarantee employment or replace human judgment.
See which pieces reached their required level.
Pieces define what the result covers.
Open a timeline and changes within access and retention.
Unmeasured and low-scoring are different.
Corrections keep a visible history.
Read alongside other job-relevant evidence.
Put results to work
Review shared results alongside the rest of your hiring process.
In team rounds, check result coverage before reading the charts.
In classes, teachers can review shared work and leave feedback without altering the graded result.
Fits the hiring you have
Choose a position to explore an illustrative flow. Add mission evidence while your team reads its limits and makes the decision.
Job post → OperatorRank → Résumé review → Interview / live coding → Decision
Read an accessible result alongside the application, without automatic rejection.
Job post → Résumé review → OperatorRank → Interview / live coding → Decision
For an assessment step, match the brief and terms to the work.
Job post → Résumé review → Take-home → OperatorRank → Interview / live coding → Decision
Read permitted evidence before talking to understand the attempt in context.
Job post → OperatorRank → Decision
Use it as one source alongside other evidence, with people deciding.
Practice on your terms
Each result keeps a record of how you worked, so you can see what to practise next.
Compare the required and achieved level.
Review saved prompts, commands, and file edits.
Practice reaches a company only when you share it
Verify with Google, then check your balance and expiry under Credits.
Know the boundaries
A held result has no verdict to interpret.
The menu and budget belong to the preset.
Unmeasured doesn't mean a low score.
Organisations only see what your permissions and consent allow.
Missing evidence does not prove no work happened.
Check the review deadline on the result page; any correction keeps its history.
Who it's for
Read each result piece by piece. No camera proctoring.
Read the request handler and trace the retry path.
bun test requests.test.tsCaptured change available
DiffThe timeline shows recorded work; incomplete coverage is marked separately.
Practice results are not public. Our team and providers process data as the Privacy notice explains.
Questions
Can't find what you're after? Write to us at support@operatorrank.com
We're starting with software tasks; browse the catalog for live missions and their content language. The same approach fits other real work with AI: spot the issue, check the evidence, make the call, and verify what the AI did.
Use the AI within the mission's budget and rules. But you pass only when every piece reaches its level: many fixes aren't enough if a piece still misses its requirement.
The result page gives an estimate when it can. Large missions take longer to check, and queues can add time. You can close the page; you'll get a notification in the app.
No. We provide the work evidence; people on the hiring team read it and decide.
Choose work to try and read its brief before starting.