Model Parameters

Model parameters are the internal numbers a machine learning model adjusts during training: mainly the weights and biases of a neural network. Together they encode everything the model has learned, and they are what gets saved when you store a trained model.

Why it matters

Parameter count is the usual shorthand for model size: "7B" means about seven billion parameters. More parameters generally give more capacity, but also more memory use, slower inference and higher serving cost. Parameters are different from hyperparameters, which you choose before training (learning rate, batch size, number of layers) and which are not learned from data.

Example

A single linear layer mapping 1,000 inputs to 500 outputs has 1,000 x 500 weights plus 500 biases:

1000 * 500 + 500 = 500,500 parameters

At 2 bytes per parameter (16-bit), that one layer takes about 1 MB. A model with 7 billion parameters at the same precision needs about 14 GB just to hold its weights, before any working memory for a request.

Common mistakes

• Treating parameter count as a direct measure of quality. Training data, architecture and tuning matter as much.
• Confusing parameters with hyperparameters or with the tokens in a prompt.
• Forgetting that memory for activations and caches is on top of the weights.

Related

• Neural network
• Quantization
• Gradient descent
• LLM

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.