Emergent Abilities of Large Language Models

LLMs Large Language Models

Emergent Abilities of Large Language Models

[ 2/May/23 ]

Giovanni Santostasi

That is a really interesting paper.

What it shows, clearly, is that the sort of “emergence” one “sees” depends on the sort of questions one asks.

If one asks sufficiently broad questions, then the emergence disappears, and one simply sees steady stochastic improvements.

If one is operating from a model based at any level in binary distinctions (eg True/False, Right/Wrong), then one will see emergence derived from those binary distinctions. If, however, all of the distinctions in use are distributed probability functions (resulting in highly dimensional context sensitive probabilistic topologies), then what one sees is steady stochastic improvement.

The entire underlying approach, that one can pre-calculate a probability function related to a “goal” is the one I find most suspect? In respect of human intelligence, it is a part of a very complex picture, certainly; but only a part. Part of human intelligence seems clearly to be actual random search, recursively applied to all levels of structures and relationships that go under the broad heading “understanding”. Some of us are more practiced in it than others, have recursively applied it to successive levels of abstraction and models approximating ourselves and the reality we find ourselves in.

And another part of being human seems clearly to be all the multiple levels of heuristics (contextually useful approximations to something) embodied in the many layers and levels of our biological and cultural being (culture in this sense including all of philosophy and science and art and mathematics/logics).

Chat GPT gave an error on my second question to it – [!
Invalid token in prompt: 50289. Minimum value is 0, maximum value is 50280 (inclusive).
There was an error generating a response]
I do not know what that error represents.

About Ted Howard NZ

Seems like I might be a cancer survivor. Thinking about the systemic incentives within the world we find ourselves in, and how we might adjust them to provide an environment that supports everyone (no exceptions) with reasonable security, tools, resources and degrees of freedom, and reasonable examples of the natural environment; and that is going to demand responsibility from all of us - see www.tedhowardnz.com/money
This entry was posted in Technology, understanding and tagged , , , , . Bookmark the permalink.

Comment and critique welcome

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s