Epoch

An epoch is a single full cycle through the whole training dataset. If the data is processed in mini-batches, one epoch consists of as many weight updates as there are batches, so a dataset of 10,000 samples with a batch size of 100 gives 100 updates per epoch.

Why it matters

The number of epochs controls how long the model trains. Too few and it underfits, never learning the patterns well. Too many and it may overfit, memorising the training data while validation performance gets worse. Practitioners watch training and validation loss each epoch and use early stopping to halt when validation stops improving. Very large language models are often trained for about one pass or less over huge corpora, so epochs matter there differently than in small-data work.

Example

samples = 10,000   batch_size = 100   epochs = 5
updates per epoch = 10,000 / 100 = 100
total updates     = 100 * 5      = 500

Common mistakes

• Choosing a fixed epoch count without monitoring validation loss.
• Mixing up epochs, iterations (single batch updates) and batches.
• Not shuffling data between epochs, which can bias learning toward ordering.

Related

• Batch size
• Gradient descent
• Overfitting
• Training data

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.