# A score is only as good as its report

"96% accurate" means almost nothing on its own. It depends on what was held out, what the baseline was, and where the model is weakest.

## What a held-out report states

How well a system predicts rows whose target values it never saw. A good report records the split, the seed, the baseline, and the exact data snapshot behind the score. A common default is a stratified 80/20 split: predict the test rows, then score them against the truth.

## Metrics by task

- **Classification:** accuracy and ROC-AUC, plus per-class results, against a majority-class baseline.
- **Regression:** RMSE, NRMSE, R-squared, and interval coverage, against a mean baseline.
- **Anomaly detection:** precision, recall, and PR-AUC where labels exist; otherwise score separation, clearly marked as unsupervised.
- **Imputation:** NRMSE on known cells that were hidden on purpose.

## An example

Take a churn model scoring 96.1% on 4,480 test rows. Against a 71.0% majority baseline that is a lift of 25.1 points. The weakest slice, customers with under three months of tenure, scores 84%. Each number tells you something the headline hides.

## How to read one

- Look at the lift over the baseline, not just the headline.
- Check per-class results and the weakest slice for hidden weak spots.
- Confirm the snapshot and seed, so you know what the number applies to.
- To judge an upgrade, compare two reports and read the delta.

## What it does not tell you

A held-out score is an estimate made at creation time, tied to one snapshot. It is not live production accuracy. That only comes from labels that arrive after deployment.