Increasing model performance
If your impulse is performing poorly, these could be the culprits:
- 1.There is not enough data. Neural networks need to learn patterns in data sets, and the more data the better. You can also lower the window increase (in the Create Impulse screen) to create more overlap from windows, but this does not lead to more variance in your data set. More data is thus always better.
- 2.The data does not look like other data the network has seen before. This is common when someone uses the device in a way that you didn't add to the test set. If you see this in the test set or during live classification you can push the sample to the training set by clicking
⋮, then selecting Move to training set. Make sure to update the label before training. - 3.The model has not been trained enough. Up the number of training cycles and see if performance increases. If there's no difference then you probably don't have enough data, or the data does not separate well enough.
- 4.If you have a high accuracy on your neural network, but the model performs poorly on new data, then your model might be overfitting. It has learned the features in your dataset too well. Try adding more data, or reduce the learning rate.
- 5.The neural network architecture is not a great fit for your data. Play with the number of layers and neurons and see if performance improves.
If you have much more data in one class than for other classes, say 90% of your data is labeled as "idle" and only 10% as "wave", your neural network will have trouble learning due to class imbalance. If this is the case your best bet is to increase the amount of data in the misrepresented class. However, if this is not possible you can try to rebalance your dataset during training. To do this:
- 1.On the Neural Network page click
⋮and selectSwitch to Keras (expert) mode. - 2.Below the imports (the lines starting with
from) add:
from sklearn.utils.class_weight import compute_class_weight
class_weights = dict(enumerate(compute_class_weight('balanced', np.unique(np.argmax(Y_train, axis=1)), np.argmax(Y_train, axis=1))))
- 1.At the last line (where
fit()is called), addclass_weight=classweights. E.g.:
model.fit(X_train, Y_train, batch_size=50, epochs=200, validation_data=(X_test, Y_test), class_weight=class_weights)
Quantization works by reducing the precision of the model's weights, so there will often be a bit of reduction in performance. If it performs too poorly, here are some things that can help:
- Add more data. It is especially likely that performance will be lost for classes that are in a minority in the training set. If you are working with time series (such as accelerometers), try reducing the window increase, it will generate more samples.
- Add some regularization (for example, some dropout layers). This helps force the network to be more resilient against the kind of error introduced by quantization. You may have to train a few more epochs to make up for the regularization.
- Increase the capacity of the network. Quantization error is worse for networks that are at their capacity limit, e.g. where every parameter is super important. If you add a few more neurons or layers you might find that the issue is not as bad.