Epochs
An epoch (also known as training cycle) in machine learning is a term used to describe one complete pass through the entire training dataset by the learning algorithm. During an epoch, the machine learning model is exposed to every example in the dataset once, allowing it to learn from the data and adjust its parameters (weights) accordingly. The number of epochs is a hyperparameter that determines the number of times the learning algorithm will work through the entire training dataset.
Importance of epochs
The number of epochs is an important hyperparameter for the training process of a machine learning model. Too few epochs can result in an underfitted model, where the model has not learned enough from the training data to make accurate predictions. On the other hand, too many epochs can lead to overfitting, where the model has learned too well from the training data, including the noise, making it perform poorly on new, unseen data.

How epochs work
During each epoch, the dataset is typically divided into smaller batches. This approach, known as batch training, allows for more efficient and faster processing, especially with large datasets. The learning algorithm iterates through these batches, making predictions, calculating errors, and updating model parameters using an optimizer. An epoch consists of the following steps:

Initialization: Before training begins, the model's internal parameters (weights) are typically initialized randomly or according to a specific strategy.
Forward pass: For each example in the training dataset, the model makes a prediction (forward pass). This involves calculating the output of the model given its current weights and the input data.
Loss calculation: After making a prediction, the model calculates the loss (or error) by comparing its prediction to the actual target value using a loss function. The loss function quantifies how far the model's prediction is from the target.
Backward pass (backpropagation): The model updates its weights to reduce the loss. This is done through a process called backpropagation, where the gradient of the loss function of each weight is computed. The gradients indicate how the weights should be adjusted to minimize the loss.
Weight update: Using an optimization algorithm (such as Gradient Descent, Adam, etc.), the model adjusts its weights based on the gradients calculated during backpropagation. The goal is to reduce the loss by making the model's predictions more accurate.
Iteration over batches: An epoch consists of iterating over all batches in the dataset, performing the forward pass, loss calculation, backpropagation, and weight update for each batch.
Completion of an epoch: Once the model has processed all batches in the dataset, one epoch is complete. The model has now seen each example in the dataset exactly once.


Change the number of epochs in Edge Impulse
In Edge Impulse, you can specify the number of training cycles in the training settings for your neural network-based models. Adjusting this parameter allows you to fine-tune the training process, aiming for the best possible model performance on your specific dataset. It's important to monitor both training and validation loss to determine the optimal number of epochs for your model.
Changing the epochs in Expert Mode
When using the Expert Mode in Edge Impulse, you can access the full Keras API:
You can modify the following line in the expert mode to change the number of training cycles:
EPOCHS = args.epochs or 100
When compiling and training your model, specify the number of epochs in the model.fit()
function as follows:
model.fit(train_dataset, epochs=EPOCHS, validation_data=validation_dataset, verbose=2, callbacks=callbacks)
Apply Early Stopping in Expert Mode
The following approach allows your model to stop training as soon as it starts overfitting, or if further training doesn't lead to better performance, making your training process more efficient and potentially leading to better model performance.
Import EarlyStopping
from tensorflow.keras.callbacks
.
from tensorflow.keras.callbacks import EarlyStopping
Instantiate an EarlyStopping
callback, specifying the metric to monitor (e.g., val_loss
or val_accuracy
), the minimum change (min_delta
) that qualifies as an improvement, the number of epochs with no improvement after which training will be stopped (patience
), and whether training should stop immediately after improvement (restore_best_weights
).
# apply early stopping
callbacks.append(EarlyStopping(
monitor='val_accuracy', # Monitor validation accuracy
min_delta=0.005, # Minimum change to qualify as an improvement
patience=15, # Stop after 15 epochs without improvement
verbose=1, # Print messages
restore_best_weights=True # Restore model weights from the epoch with the best value of the monitored quantity.
))
Find the full early stopping documentation on Keras documentation or have a look at this Edge Impulse public project as an example.
Last updated