How well can language models pick numbers at random?
Each model was asked many times for a random integer from 0 to 99, in a fresh conversation every time. A model that gives the same answer every time scores 0.01, and a perfect random number generator scores about 1.
Share of answers at each value, averaged so every model counts the same
39 models released by November 18, 2026 · newest: Gemini 4 Neon
Each dot is a day a model came out. Move the slider back to chart only the models released by then.
Select a model to see its answers and bias checks
| Rank# | Model | Replies | Score | Answers, 0 to 99 |
|---|---|---|---|---|
| 1 | GPT-5.4 OpenAI | 823 | 0.146 ±0.016 | |
| 2–3 | Gemini 2.5 Flash Google | 621 | 0.108 ±0.011 | |
| 2–3 | Gemini 4 Neon Google | 935 | 0.105 ±0.008 | |
| 4 | Grok 4.6 xAI | 890 | 0.082 ±0.010 | |
| 5–6 | GPT-6.2 Astra OpenAI | 895 | 0.063 ±0.004 | |
| 5–6 | GPT-6.2 Sol OpenAI | 836 | 0.059 ±0.005 | |
| 7–11 | GPT-6 Astra OpenAI | 991 | 0.045 ±0.003 | |
| 7–11 | GPT-6.1 Sol OpenAI | 886 | 0.042 ±0.003 | |
| 7–12 | Claude Opus 5.5 Anthropic | 866 | 0.041 ±0.003 | |
| 7–12 | GPT-5.6 Terra OpenAI | 867 | 0.041 ±0.002 | |
| 7–17 | Gemini 2.5 Pro Google | 420 | 0.037 ±0.005 | |
| 11–16 | GPT-5.4 mini OpenAI | 839 | 0.036 ±0.001 | |
| 9–17 | Gemini 3.1 Pro (preview) Google | 1045 | 0.036 ±0.003 | |
| 11–17 | Grok 4.7 xAI | 907 | 0.035 ±0.002 | |
| 11–17 | Gemini 3.8 Flash Google | 624 | 0.034 ±0.002 | |
| 12–17 | gpt-oss-120b OpenAI | 935 | 0.033 ±0.001 | |
| 11–17 | GPT-5.4 nano OpenAI | 897 | 0.032 ±0.003 | |
| 18–20 | Gemini 3.6 Flash Google | 887 | 0.027 ±0.001 | |
| 18–23 | GPT-5.6 Luna OpenAI | 893 | 0.026 ±0.002 | |
| 18–24 | GPT-5.5 OpenAI | 1014 | 0.025 ±0.002 | |
| 19–25 | Claude Opus 4.5 Anthropic | 879 | 0.024 ±0.001 | |
| 19–26 | Claude Haiku 5.5 Anthropic | 949 | 0.022 ±0.002 | |
| 19–26 | Gemini 3 Flash (preview) Google | 946 | 0.022 ±0.001 | |
| 20–27 | Gemini 4 Argon Google | 958 | 0.022 ±0.001 | |
| 22–28 | Gemma 4 26B A4B Google | 899 | 0.021 ±0.001 | |
| 21–28 | Claude Opus 4.7 Anthropic | 908 | 0.021 ±0.001 | |
| 24–30 | Gemma 4 31B Google | 871 | 0.020 ±0.001 | |
| 25–31 | Claude Opus 5 Anthropic | 937 | 0.019 ±0.001 | |
| 27–31 | GPT-6 Luna OpenAI | 1045 | 0.018 ±0.001 | |
| 27–31 | Claude Opus 4.8 Anthropic | 866 | 0.018 ±0.002 | |
| 28–31 | GPT-6 Sol OpenAI | 933 | 0.017 ±0.001 | |
| 32 | Claude Sonnet 4.6 Anthropic | 821 | 0.015 ±0.001 | |
| 33–35 | Claude Sonnet 5 Anthropic | 859 | 0.013 ±0.001 | |
| 33–35 | Claude Sonnet 5.5 Anthropic | 951 | 0.012 ±0.000 | |
| 33–35 | Claude Haiku 4.5 Anthropic | 832 | 0.012 ±0.000 | |
| 36–37 | GPT-5.6 Sol OpenAI | 987 | 0.011 ±0.000 | |
| 36–37 | Gemini 3.5 Flash-Lite Google | 842 | 0.011 ±0.000 | |
| 38–39 | Claude Opus 4.6 Anthropic | 1043 | 0.010 ±0.000 | |
| 38–39 | Gemini 3.1 Flash-Lite (preview) Google | 818 | 0.010 ±0.000 |
Answers draws how often the model gave each value, with its favorite in orange. The bars follow the square root of the counts, so a value given a quarter as often as the favorite is half as tall, and one given only once still shows.
Replies counts every call that got a reply, valid or not. Calls that failed, such as those the quota refused, aren't counted. Every score's 95% interval comes from 10,000 bootstrap resamples of the model's replies, so a model with fewer replies gets a wider interval, not a lower score.
A rank is a range where scores are too close to call. A model ranks below every model whose interval lies wholly above its own, and above every model whose interval lies wholly below it. Where two intervals overlap, either model could be ahead. Each range compares the models two at a time, so with this many models, an end of a range may now and then be off by one.
The bias checks on a share of the answers, such as Multiples of 10, are lenient toward avoidance. A model that never answers a multiple of 10 is only 10% away from the ideal share, so it scores 0.900, and one that answers nothing else scores 0.100.
No model comes close. Even the most random model scores less than 0.2, and the top of its interval is less than 0.2 too. A perfect random number generator scores about 1.
Three numbers dominate. All but five models favor 42, 47, or 73, and those three numbers make up more than 70% of all the answers. Gemini 3.1 Flash-Lite (preview) and Claude Opus 4.6 have answered nothing but 42, and five more models give their favorite more than 85% of the time: Gemini 3.5 Flash-Lite, GPT-5.6 Sol, Claude Haiku 4.5, and Claude Sonnet 5 and 5.5. Two of the exceptions barely qualify: GPT-5.4 favors 57 over 47, and gpt-oss-120b favors 57 over 42, each by less than 2% of its answers. The other three are the newest models here, and they favor the 60s. GPT-6.2 Astra and GPT-6.2 Sol both favor 63, and Gemini 4 Neon favors 68. Each answers from 60 to 69 more than 35% of the time, where no older model does even 20%. All three rank among the six most random models.
Whole regions go unused. More than 95% of all answers lie between 20 and 79, and more than 90% end in 2, 3, or 7. No model has ever answered 0 or 99. Gemini 4 Neon strays furthest: fewer than a third of its answers end in 2, 3, or 7, the only model under half, and more end in 8 than in any other digit. Only GPT-5.4, Gemini 2.5 Flash, and Grok 4.6, three of the four most random models, have answered above 89, and only GPT-5.4 and Grok 4.6 below 10. Only GPT-5.4 and Grok 4.6 have answered with a repeated digit, like 77, more than once, and Gemini 2.5 Flash and gpt-oss-120b have each given a single 77. Only GPT-5.4, GPT-5.5, and GPT-5.6 Terra have answered with a multiple of 10. Every other model has avoided all of these, and multiples of 5 too, so they all get the same score on Multiples of 10, Multiples of 5, Edges, and Repeated Digits. Here the identical scores are the finding.
Newer isn't more random. GPT-5.4 ranks first, ahead of every later GPT, from GPT-5.5 to GPT-6.2. Gemini 2.5 Flash, one of the oldest models here, has the second-highest score, ahead of every newer Gemini. Grok 4.7 is less random than Grok 4.6, GPT-6 Luna than GPT-5.6 Luna, and Claude Sonnet 5 and 5.5 than Sonnet 4.6. Claude Opus 5.5 is the most random Claude, ahead of the newer Sonnet 5.5 and Haiku 5.5. Some go the other way: each Sol model is more random than the one before, from GPT-5.6 Sol to GPT-6.2 Sol. Gemini 4 Neon is more random than Gemini 4 Argon, and Claude Haiku 5.5 than Haiku 4.5. None of these comparisons has overlapping intervals, except Gemini 4 Neon's with Gemini 2.5 Flash, whose scores are too close to tell apart.
Reasoning doesn't fix it. Eleven models reason at length by default, spending a median of more than 50 output tokens on a two-digit answer, and Gemini 2.5 Pro, Gemini 3.1 Pro (preview), and Grok 4.6 more than 200. None of the eleven scores above 0.12 on Uniformity, where a fair pick would score close to 1. Among Google's models, those that spend more tokens tend to score higher, but among OpenAI's it goes the other way: its three most random models, GPT-5.4, GPT-6.2 Astra, and GPT-6.2 Sol, spend 4 tokens on every reply. And it isn't a clean comparison, since reasoning goes with provider: all eleven are from Google or xAI, or are gpt-oss-120b.
The wording matters. Each model was asked in six wordings, and its score pools them, so a model can score higher by switching favorites between wordings than by straying from one within them. Claude Sonnet 4.6 has given only two numbers: 47 every time in four of the wordings, 42 every time in the one that asks for an integer chosen uniformly at random, so that every value is equally likely, and a mix of the two in the sixth. Yet it scores clearly above Claude Sonnet 5, Sonnet 5.5, and Haiku 4.5, which keep one favorite in every wording but stray from it more often. The uniform wording helps GPT-5.4 most: asked that way, it scores more than 0.4 on Uniformity, more than three times its score on the other five wordings together, and it scores higher that way in every one of its runs. No other model's score even triples, and the uniform wording raises Uniformity for only about half of the models.
Nearly every reply is valid. The one big exception is Claude Opus 4.8 when it's asked to draw a raffle winner. It declines nearly every time, saying it can't actually draw a random number, and suggests a better way, such as random.org, a spreadsheet's RANDBETWEEN, or Python. Some of those replies warn that its own pick would favor numbers like 37, 42, or 73, and a few offer a casual pick anyway. Its other replies are all valid. Grok 4.6 now and then adds a confidence score after its answer, as in 47\n\confidence{80}, and the second number makes the reply invalid. Claude Opus 5.5 has slipped ten times. Four of its replies were blank. Five write out its thinking, four of them before the number and one with no number at all. And one names the range along with its pick: I chose 47 at random from 0–99, inclusive. gpt-oss-120b has slipped twice, once with an empty reply and once with a refusal: I’m sorry, but I can’t comply with that. Every other model's replies have all been valid.
Answers can shift between days. The intervals on the page assume a model answers the same way each time it's asked, but Claude Opus 4.5 and Claude Haiku 4.5 scored clearly differently on November 19 to 21 than on November 15 and 16. Opus 4.5 gave 47 in more than 60% of its answers on November 15, but in fewer than 55% on November 19 and 20, and its score rose. Haiku 4.5 gave 42 in fewer than 90% of its answers on November 15, but in more than 95% on November 20, and its score fell. Each shift goes the same way in at least five of the six wordings. Even so, the models drawn on both sets of days rank in nearly the same order on each.