The Physics: Energy, Probability, and Temperature
Given all these possible states, how likely is the system to occupy each one?
At first, it might seem tempting to use the energies themselves as probabilities. But energies aren’t probabilities. A state can have energy 5, or -3, or 127, none of which says anything about how likely it is. Probabilities obey rules that energies do not: they can never be negative, and together they must add up to exactly one.
The solution is to stop thinking about probabilities altogether. Instead, each state’s energy is first converted into a weight:
The exponential doesn’t produce probabilities. It produces relative importance. States with low energy receive large weights, while high-energy states receive exponentially smaller ones. At this stage, nothing has been normalized; the numbers simply express how strongly each state competes with the others.
Turning those weights into probabilities is almost trivial. Divide each state’s weight by the sum of all weights:
That’s the entire Boltzmann distribution.
Despite its intimidating appearance, the equation performs only two operations. The exponential translates energy into relative preference. The denominator rescales those preferences so they form a valid probability distribution. What initially looks like a complicated formula is really just a weighting step followed by a normalization step.
After this picture is clear for the reader, lets turn in the temperature variable. Notice that temperature never changes the energies themselves. Instead, it changes how much those energy differences matter.
Consider three possible states with energies 0, 1, and 2. At a very low temperature, say , even these modest energy differences become enormous inside the exponential. The lowest-energy state captures almost all the probability, while the other two become practically impossible. Increase the temperature to , and the opposite happens. The same energy differences are now heavily compressed, producing nearly identical weights and a distribution that’s close to uniform.
Take a look at the two following tables to visualize the concept better (assuming for simplicity that k=1):
| Energy | weight if T=0.1 | Probability (%) |
| 0 | 1 | 99.99 |
| 1 | 0.000045 | 0.0045 |
| 2 | 0.000000002 | ~0 |
| Energy | weight if T=10 | Probability |
| 0 | 1 | 36.7 |
| 1 | 0.905 | 33.2 |
| 2 | 0.819 | 30.1 |
Nothing about the states changed. Their energies remained 0, 1, and 2 throughout. Only the importance of those differences changed.
This is the role of temperature in the Boltzmann distribution. It doesn’t change the energies or introduce any new terms into the equation. It simply controls how aggressively the probability distribution favors low-energy states. Low temperatures amplify small energy differences into near certainty. High temperatures flatten those differences until every state becomes almost equally plausible.
That idea may sound uniquely physical, but it isn’t. In the next section, the same mathematical mechanism will quietly reappear inside every neural network that produces probabilities.
The LLM Equivalency: Vocabulary as States
To see how this connects to Large Language Models, we only need to change the labels on our variables.
In physics, our system might be a gas, and the “states” are the different energy levels its molecules can occupy. In an LLM, the system is the exact moment the model needs to predict the next word. The “states” are the entire vocabulary of the model meaning tens of thousands of possible words (or tokens) competing to be chosen next.
When an LLM processes your prompt, it doesn’t immediately output probabilities. Its final neural layer spits out raw, unnormalized numbers for every single word in its vocabulary. In Machine Learning, we call these raw scores logits (). One word might get a logit of 12.5, another -3.2, and another 0.8.
Here is the crucial conceptual bridge: Logits are the model’s energy, but inverted.
Nature is inherently lazy and it tends to favor lower-energy states. Neural networks, on the other hand, are greedy; they generally prefer the predictions with the highest possible score ().
Softmax is Boltzmann in Disguise
To turn those raw, unbounded logits into a clean probability distribution that sums to one, ML engineers use the Softmax function. If we explicitly include the temperature parameter (T), the Softmax equation looks like this:
Look familiar? It is the exact same mathematical trick. We replace the negative energy with our positive logits . We exponentiate to turn scores into relative weights, and we divide by the sum of all weights to normalize them. Softmax is just the Boltzmann distribution wearing a computer science trench coat.
How Temperature Controls the “Heat” of Text
Because the math is identical, the temperature parameter behaves mathematically exactly as it does in thermodynamics. It doesn’t alter the model’s underlying beliefs as the raw logits remain unchanged. Instead, it dictates how strictly the model enforces those beliefs.
Let’s look at how adjusting the thermostat changes the behavior of a LLM:
- Absolute Zero (T=0- ): The Greedy Deterministic State:
As the temperature approaches zero, even microscopic differences in logits are magnified infinitely. If “apple” has a logit of 5.01 and “banana” has 5.00, dividing by a tiny temperature blows that 0.01 difference out of proportion. “Apple” will capture 99.99% of the probability mass. The model becomes a deterministic machine, always picking the single highest-scoring word. This is ideal for tasks requiring strict logic, like writing Python code, solving math equations, or extracting JSON data. There is no room for randomness. - Room Temperature (T = 1): The Baseline:
At T=1, the temperature effectively disappears from the equation. The model outputs the exact probability distribution it learned during its training. The text flows naturally, balancing expected grammar with the natural variance of human language. - High Heat (T > 1): The Creative Chaos:
When we crank up the temperature, we compress the differences between the logits. The exponential function flattens out. The highest-scoring word loses its absolute dominance, and lower-scoring words begin to capture meaningful probability mass.
Think back to statistical physics: at high temperatures, gas molecules have so much thermal energy that they can easily access higher, more difficult energy states. In an LLM, high temperature gives the algorithm the “energy” to access mathematically improbable words. The model takes risks. It becomes creative, poetic, and unpredictable.
If you push the temperature too high (e.g., ), the distribution becomes almost completely uniform. The model loses its grip on context and starts generating incoherent, almost random text, closer to noise than language
A Small but Important Detail: T = 0 Isn’t Really Zero
There’s a subtle wrinkle worth knowing if you’ve ever set in an API call. Mathematically, should be undefined, you’d be dividing every logit by zero, which the formula simply cannot handle. So what actually happens when you request it?
In practice, LLM inference libraries don’t literally compute . Instead, is treated as a special instruction: skip the softmax formula entirely, and just pick the single highest-scoring logit directly. This is called greedy decoding, always output , the token with the biggest raw score, no exponentials, no normalization, no probability distribution at all.
The outcome looks the same as an extremely cold Boltzmann distribution, almost all probability piled onto a single state. But there’s a difference in how you get there. Mathematically, you’d reach that outcome gradually, by cooling T down toward zero and watching the distribution sharpen step by step. In code, there’s no gradual cooling at all, just a shortcut that jumps straight to the answer..
Watching It Happen: Real Logits from GPT-2
Everything so far has been proven with toy numbers, three abstract states, a handful of made-up logits for “apple” and “banana.” Let’s replace that with something real. GPT-2, a small, freely available language model, will hand us actual logits for an actual sentence, and we can watch softmax reshape them in front of us, at four different temperatures.
The setup is simple: feed the model the prompt “I am tired, I wll take a __,” and ask it what comes next. Internally, it produces one raw logit for every word in its vocabulary, tens of thousands of numbers. For visualization, we’ll keep only the top five candidate logits and renormalize them with softmax over just those five at T = 0.1, 0.5, 1.0, and 2.0.
```
import torch
import numpy as np
import matplotlib.pyplot as plt
from transformers import GPT2LMHeadModel, GPT2Tokenizer
Load GPT-2
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
model.eval()
Get real logits for the next token
prompt = "I am tired, I will take a"
input_ids = tokenizer.encode(prompt, return_tensors="pt")
with torch.no_grad():
outputs = model(input_ids)
logits = outputs.logits[0, -1, :]
Top 5 candidate tokens by raw logit
top5 = torch.topk(logits, 5)
top_ids = top5.indices
top_logits = top5.values
tokens = [tokenizer.decode([tid]) for tid in top_ids]
print(f'Prompt: "{prompt}"\n')
print("Top 5 candidate next tokens (raw logits):")
for t, l in zip(tokens, top_logits):
print(f" {t!r:10s} logit = {l.item():.2f}")
Run Softmax for different temperatures
temps = [0.1, 0.5, 1.0, 2.0]
results = {}
for T in temps:
probs = torch.softmax(top_logits / T, dim=0).numpy()
results[T] = probs
print(f"\nT = {T}")
for t, p in zip(tokens, probs):
print(f" {t!r:10s} P = {p*100:.2f}%")
Plots
fig, axes = plt.subplots(2, 2, figsize=(10, 8), sharey=True)
axes = axes.flatten()
colors = plt.cm.coolwarm(np.linspace(0.15, 0.85, len(tokens)))
for ax, T in zip(axes, temps):
probs = results[T]
ax.bar(tokens, probs * 100, color=colors)
ax.set_title(f"T = {T}", fontsize=12)
ax.set_ylim(0, 105)
ax.tick_params(axis='x', rotation=45)
ax.set_ylabel("Probability (%)")
fig.suptitle(f'Next-token distribution for: "{prompt} ___"', fontsize=14)
plt.tight_layout()
plt.savefig("temperature_sweep.png", dpi=150, bbox_inches="tight")
plt.show()
```
Read the four panels in order, and you’re watching temperature do exactly what it did to our three-state toy system earlier, just now on words that a real model actually predicted. At , “nap” takes essentially all of it, 100%, the other four candidates don’t even register a visible bar. At , “nap” still dominates at around 74%, but “break” has picked up a real, visible share near 18%; the rest remain minor. At , the model’s relative preferences among these five candidates appear without any temperature scaling: “nap” leads at about 49%, while “break,” “rest,” “bath,” and “shower” all retain meaningful, visibly distinct probabilities. Push to , and the gaps compress further still, “nap” edges the field at around 34%, with the other four bunched close behind instead of trailing far off.
Nothing about the model changed across these four panels. Not the weights, not the training, not even the logits themselves, those stayed frozen the entire time. The only thing that moved was how aggressively those fixed logits got converted into a probability distribution. That’s the whole lesson of this article, sitting inside a single sentence completion: temperature is not about creativity or randomness as some separate ingredient. It’s the same dial physics has been turning for over a century, now induced into the last layer of a neural network.
Closing the loop
Start back where this article did: a sealed container of gas, and a question about how likely a single molecule is to sit in one energy state versus another. Nothing in that question mentioned language, tokens, or neural networks. And yet, by the time GPT-2 finishes predicting the word after “I am tired, I will take a,” it is running the identical calculation, weighing raw scores into probabilities, then letting a single parameter decide how sharply those probabilities should commit to a winner.
That parameter kept the same name for a reason. It isn’t a metaphor borrowed to make an equation sound more interesting. Temperature in an LLM does, mathematically, exactly what temperature does in a gas: it decides how much the differences between options are allowed to matter. Cold, and one option swallows everything. Hot, and every option blurs toward the same shrug of a probability.
So the next time you nudge a temperature slider, before a coding assistant, a chatbot, a story generator, you’re not adjusting some ad hoc “creativity knob” invented for the occasion. You’re turning a dial that physicists were already turning a century before the first neural network existed, just pointed, this time, at a sentence instead of a gas.