noun · AI concept
Parameters
also called weights
The model's dials. Billions of tiny settings that training tuned until the model got good at its job.
The learned numeric values inside a model, whose count is the usual measure of model size and which are fixed after training.
The learned numbers, mostly weights, that every calculation inside the model multiplies and adds. Training sets them, inference only reads them, and the count (7B, 70B and so on) is the usual shorthand for model size.
Real-life example
You read that a model has 70 billion parameters and picture 70 billion facts. It is closer to 70 billion dials, each turned a tiny amount, which together make the model write like it has read a library. Open the file and you will not find a sentence you can read.
Where the analogy breaks
-
Parameters are not facts. No single number holds a fact such as "Paris is in France". Knowledge is spread across many numbers at once.
-
More parameters does not guarantee a better answer. Training data and method matter a lot, and a smaller well-trained model can beat a larger one on a given task.
-
The word has two meanings in practice. Inside a model it means learned weights. In an API it also gets used for settings you choose per request, such as temperature. Same word, different things.