What makes LLMs work isn't deep neural networks or attention mechanisms or vector databases or anything like that.

What makes LLMs work is our tendency to see faces on toast.

@jasongorman No, seriously - that's exactly what it is & it's a mechanism exploited by charlatans since always: https://softwarecrisis.dev/letters/llmentalist/

@jwcph @jasongorman I’ve always felt like the way chatbots speak was a little too general and assertive for me to believe/trust them, and I’m glad to learn *why* it is like that. The psychic comparison is very effective.

@jasongorman

That and spoonfuls of confirmation bias.

I am happy that there's a good generative grammar model. That's a significant thing it may go some way to suggest that grammar can be generalised.

This is orthogonal to machine intelligence, however.

@jasongorman patterns on toast are random and we interpret it as faces. So you say that LLMs produce random text and we interpret it? That's no true at all. If that would be true, the output of LLMs would look like this:

The fan box deep is tendency let fedolin are have with no a where I been water can mouse. Big dark wall health we garage child software haben open.

@duco I'm suggesting that we project human qualities like understanding and reasoning on to the output of LLMs where there are none.

@jasongorman Fuck, Jason. That's quotable.

@GeePawHill I feel a t-shirt coming on 🙂

@jasongorman

This is why I call them Fake AI

@jasongorman if it looks like a duck and it quacks like a duck, it can't possibly be an artificial duck

@jasongorman

Pareidolia is fun and cool unless you are fooling yourself that the faces you see are real and they are talking to you.

i initially misread "charlatans" as "christians" and read the first ~45% of this article with the thought in the back of my mind how 10 years ago i had found myself doing a lot of these things to myself, knowingly, because i was that desperate to buy into a system that promised to fix what was wrong with me

RE: https://helvede.net/@jwcph/114595739680588052

@jasongorman @GeePawHill i can imagine like

first panel has the first line as a caption for a picture of this big looming shiny metal machine

second panel has the last line and the same machine, but the toast is done and has popped up

@jasongorman @GeePawHill (of course the toast has faces on them)

@jasongorman @duco on the other hand, i'm looking at those sample sentences and already almost kinda see some things that could be parsed as meaning

tell me a gorilla signed this and i'd damn well tell you she was talking about a mouse she saw in the watering can earlier and something about someone's kids doing a startup in their garage