Four models. Four perfect scores, with confusion matrices holding no errors at all. A result like that is not something to celebrate. It is a reason to go and check the data.
The check found something. Every forgery was built from a real card and kept its number, so real_214 becamefake_214. My split script shuffled the genuine files and the forged files independently, which let two halves of one physical card land on opposite sides of the train-test boundary. On the 80:10:10 split, 48 of 165 test images were contaminated. That is 29.1%. On 70:15:15 it was 71 of 248.
I rewrote the split to work on card families rather than files. Every derivative of one card now lands in the same subset. Contamination went to zero.
Then I retrained, and it was still 1.0000.
The leak was real and it explained nothing. I had spent the audit proving my own suspicion wrong. A five-fold group-aware cross-validation agreed. At 1024×474 the two classes genuinely do separate, and the contamination had been riding along without changing the outcome.
My examiners raised a different objection. If every model scores 1.0, there is no basis for saying which one is better. They asked for results that were not perfect.
The quick way to do that is to make the numbers worse. I did not. Instead I ran a resolution ablation, holding every other parameter fixed and dropping the input to each architecture's native size, 224×224 for B0 and 300×300 for B3. Accuracy fell to between0.9606 and0.9774 on its own, and the fall carried information: the model leans on high-frequency forensic traces that do not survive being shrunk.
Every false negative in those runs came from one scenario. Font inconsistency. Not a single face swap, recapture, text patch, or hybrid got past. Four files defeated every native-resolution model I trained. Letterforms are the finest detail in the dataset, so they are the first thing to go when a card is squeezed to 224 pixels wide.
