Keep selection outside the test set Training fits parameters and preprocessing. Validation chooses features, regularization, and threshold. Final test estimates performance after choices are fixed. Group users or preserve time order when that matches deployment. 2 / 6
Trace the API dictionary True: [1,0,1,0,1,0]; predicted: [1,1,0,0,1,0]. TP=2, TN=2, FP=1, FN=1; sum=6. Return {'TP':2,'TN':2,'FP':1,'FN':1}, not a nested list. 3 / 6
Name each denominator Accuracy=(TP+TN)/n. Precision=TP/(TP+FP): predicted positives. Recall=TP/(TP+FN): actual positives. F1=2TP/(2TP+FP+FN); zero-denominator policy returns zero. 4 / 6
Threshold counterexample Scores [0.9,0.8,0.7,0.4], labels [1,0,1,0]. Threshold 0.9: precision 1, recall 1/2. Threshold 0.8: precision 1/2, recall 1/2. Threshold 0.7: precision 2/3, recall 1. Lower thresholds cannot reduce recall on fixed scores; precision can rise or fall. 5 / 6
Diagnose and reconstruct All-negative predictions on 1 positive out of 100: 99% accuracy, 0 recall. Score cost as a declared rule; F1 does not encode every error cost. Check fresh models/preprocessing per fold and include remainder examples. Identical confusion counts cannot certify a leak-free split. 6 / 6