As of the 2020 lectures, reducing the number of parameters was considered key. Because doing so led to better training and improved general performance or generalization performance.
But today in a master class, the professor said that’s no longer the case in 2021. There’s a paper from OpenAI showing that increasing the number of parameters improves generalization performance. Also, if you have the resources to train, increasing parameters is more advantageous for both training and performance.