Understanding Avoidable Bias!
In this article, we will discuss comparing our model accuracy with human-level performance and discuss the concepts like avoidable bias and how to tackle it!
What is avoidable bias?
The difference between human error (approximation of Bayes error) and the training error is termed avoidable bias.
The perfect level of accuracy may not be always 100% and Bayes optimal error is the very best theoretical function that can never be surpassed(best possible error).
Comparison with human-level performance
As long as our model is doing worse than humans we can use some tactics for improving our model. Although knowing about bias and variance helps and it turns out that knowing how well humans can do on a task can help us understand better how much we should try to reduce bias!
Scenario 1- You have built a cat classifier and you got error percentages for training and dev sets as 8% and 10% respectively. Plus let’s say human error is 0.5% for this dataset.
In this example:-
- Avoidable Bias is 7.5% and variance is 2%.
- The next steps should focus on reducing the avoidable bias.
How to reduce Avoidable bias?
- Train a bigger neural model.
- Train longer or with better optimization algorithms.
- Finding better neural network architecture.
Scenario 2 -You have built a cat classifier and you got error percentages for training and dev sets as 8% and 10% respectively and human error for this dataset is 7.2% (images were of poor quality s.that it is difficult for human eyes also to classify a cat in the image).
In this example:-
- Avoidable Bias is 0.8% and variance is 2%.
- The next steps should focus on reducing the variance.
How to reduce Variance?
- Get more training data.
- Regularisation (Dropout, L2 regularisation, Batch Normalisation, etc.)
- Data Augmentation
- Early stopping
- Finding better neural network architecture.
If you want to read more about these pointers mentioned above refer to my previous post.
So that’s it! You have made it to the end of the post. For more information about these concepts check out the references below!
