§ Widget · 03 Next-token prediction on a real sequence

Real output from the model. We feed an input sequence and ask the model to predict the next token: first the raw distribution over the full six-mer vocabulary, then that same distribution marginalized down to the per-position base distributions.

real inference
1 · Inputs & ground truth
input sequence
→
target token
2 · Token-level prediction over all six-mers
per-token probability cumulative mass uniform reference ground truth rank
Most likely tokens
3 · Marginalized to base level (the -way distribution summed per position · click a position)
4 · All six-mers, ranked by probability ·
most likely → least likely