ISSUE 06 / WORK DESIGN · 4 MIN READ

The $0 Test for AI Career Advice

A recommendation can keep its direction while the conditions that support it remain unanswered.

A desk with a printed letter, a calculator, and five gold steps leading to an empty dashed rectangle.
An illustrative still life. The offer and scenario figures are not independently verified.

The bonus changed the model's confidence. It did not change its recommendation. That still did not make the decision ready.

A finance and analytics director was weighing a higher-paying external role against a raise and a possible future role at a long-time employer. The AI recommended leaving, with conditions: attainable incentive pay and hours compatible with family life. Neither condition had been checked.

01 / THE SENSITIVITY TEST

The incentive changed confidence, not direction. The 28-point range is within the second evaluation; the first answer had no numeric score. All five levels were considered in one fresh-context response, not five independent trials. Confidence values are subjective model judgments, not calibrated probabilities. The same response separately gave an overall provisional accept at 70/100 after considering remaining unknowns; that is not a scenario score.

Source and scope: Adapted from Don Long's supplied article, “Before You Trust AI Career Advice, Run This $0 Test.” It describes an anonymized public r/careeradvice question and reports two AI evaluations. Reddit blocked full-page extraction during review, so case details and model outputs are attributed to the supplied article, not independently verified. The scenario percentages are illustrative; the confidence scores are not calibrated probabilities. This checklist is editorial guidance, not a validated career or financial framework. No recommendation is made for the person in the case. Draft for author review; confirm date and byline before send.

ABOUT THE AUTHOR

Don Long

Founder of SummitMark. Helping people and organizations become capable with AI.

THE MOSTLY RIGHT LETTER

Continue reading.

Full archive