Model Evaluation & Threshold Economics
LIVEsynthetic data · generator disclosedA logistic model was just trained in your browser (300 gradient-descent epochs) on a seeded synthetic fraud corpus of 1,200 transactions (148 fraudulent, base rate 12%). The generative process is in evalbench.ts — nothing here is a canned figure. The question this lab answers: where do you set the decision threshold, in dollars?
ranking quality across all thresholds
vs 12% base rate
probability quality (lower is better)
$5,436 total operating cost
ROC curve
Area 0.954 — computed from 1,201 distinct score thresholds
Precision / recall
Average precision 0.835 — the honest curve at a 12% base rate
Calibration
Reliability by decile · Brier 0.046 — do 0.8 scores mean 80%?
Threshold economics
Every flag buys a review; every miss eats a loss. The U-curve picks the operating point.
Operate the threshold
Drag the bar; the confusion matrix and the bill move together.
Visible math: score = σ(w·x + b), trained by full-batch gradient descent (L2 = 1e-3). ROC sweeps every distinct score; AUC is the trapezoid sum. Total cost(t) = flags(t) × review cost + misses(t) × fraud loss. Data is synthetic and seeded (mulberry32, seed 7) with two pure-noise features — the generative process ships in the same tested module as these formulas (@labs/engines/evalbench).