12,000 UFC Scorecards vs. Machine Learning And Human Judgement
How to evaluate a model when the ground truth is a panel of humans who disagree, and sometimes get it wrong. Twenty-five confident misses I trained a
Read piece →Tanner Stahl · Product · Data Science · AI
Experiments and essays on AI systems, model evaluation, agents, and deploying them where mistakes are expensive.
Selected work
How to evaluate a model when the ground truth is a panel of humans who disagree, and sometimes get it wrong. Twenty-five confident misses I trained a
Read piece →A few weeks ago a new model called Jev launched, and within days it was being discussed across the industry, including inside our own organization. We spent the
Read piece →A cautionary tale for any PM about to greenlight an AI feature: sometimes the simplest approach works better than the complicated model. I work as a product owner
Read piece →Research
Things I built, tested, measured, and learned from.
Essays
Working through the design choices around useful, deployable AI.
About Tanner
I work across AI product and data science, with a background in bioengineering and building products in regulated industries. Fit for Purpose is where I test ideas, document what works, and write through the tradeoffs.
More about me →Archive