§ Widget · 03 Next-token prediction on a real sequence

Real output from the model. We feed an input sequence and ask the model to predict the next token. While the most probable 6-mer may not match the target token, the marginalized base distributions often align with the target at individual positions—demonstrating the benefit of FNS for model inference.

real inference
1 · Inputs & ground truth
input sequence
→
target token
2 · Token-level prediction over all 6-mers
per-token probability cumulative mass uniform reference ground truth rank
most probable tokens
3 · Marginalized to base level (the -way distribution summed per position · click a position)
4 · All 6-mers, ranked by probability ·
most probable → least likely