Emergent Abilities of Large Language Models

LLMs Large Language Models

Emergent Abilities of Large Language Models

[ 2/May/23 ]

Giovanni Santostasi

That is a really interesting paper.

What it shows, clearly, is that the sort of “emergence” one “sees” depends on the sort of questions one asks.

If one asks sufficiently broad questions, then the emergence disappears, and one simply sees steady stochastic improvements.

If one is operating from a model based at any level in binary distinctions (eg True/False, Right/Wrong), then one will see emergence derived from those binary distinctions. If, however, all of the distinctions in use are distributed probability functions (resulting in highly dimensional context sensitive probabilistic topologies), then what one sees is steady stochastic improvement.

The entire underlying approach, that one can pre-calculate a probability function related to a “goal” is the one I find most suspect? In respect of human intelligence, it is a part of a very complex picture, certainly; but only a part. Part of human intelligence seems clearly to be actual random search, recursively applied to all levels of structures and relationships that go under the broad heading “understanding”. Some of us are more practiced in it than others, have recursively applied it to successive levels of abstraction and models approximating ourselves and the reality we find ourselves in.

And another part of being human seems clearly to be all the multiple levels of heuristics (contextually useful approximations to something) embodied in the many layers and levels of our biological and cultural being (culture in this sense including all of philosophy and science and art and mathematics/logics).

Chat GPT gave an error on my second question to it – [!
Invalid token in prompt: 50289. Minimum value is 0, maximum value is 50280 (inclusive).
There was an error generating a response]
I do not know what that error represents.