The explosion of the number of parameters in the large language model,
does this really matter?
Does that make a difference?
Should users care?
Well, today's wisdom says: the more parameters, the better.
But, of course, more parameters
sometimes come at a cost, because in order to train models
with a lot of parameters, you need to have a lot of data.
So, all the advances we talk about today came from OpenAI
training on the very, very large models, on the very large dataset,
GPT-3, GPT-4, GPT-3 having 75 billion parameters,
and we haven't seen such a great performance on the smaller models.
But at the same time,
what we are observing recently is that researchers are capable
of actually building and training much smaller models,
but maybe providing 80% quality of that large model,
and the models of the size several billions are actually pretty good.
There are more interesting developments happening.
There was a paper recently published
saying that all you need are textbooks, where researchers train the model
on several billion parameters, but they train it on a very, very high quality of data.
And those models are also exhibiting
very good properties comparable to large models.
So there is a big question,
the larger model you want to build, the more parameters you need to train,
the more data you need, the more energy it takes.
And so the question is, do you really need such a big model for your problem?
What impact on business decision-makers?
Should there be small or medium models?
This is a very practical question.
Businesses should look into why they really need those models.
Do you really need your model to speak to 200 languages,
or do you want your model to be a chatbot
and be able to work as a customer service representative?
Or do you want your model to help maintenance workers?
Depending on that, you might need a different type of model.
Whether you want the universal model
or the model adjusted for your business.
The model size, does this have an impact on the cost for users?
Well, using these large language models can be quite costly.
So to train them costs a lot.
Plus, if you're using the model trained
by the model provider, they will still charge you for those costs.
What you shouldn't forget
is the cost of inference or calling the model to make prediction.
Every time you want the model to make prediction,
it goes through all those billions of parameters and doing all
that multiplication, which takes a lot of energy and of course, costs you.
And so this cost aggregates very, very, very quickly.
So as a result, you need to really balance your business needs and the size
of the model to make overall to make economic sense.