156
Bill Gates feels Generative AI has plateaued, says GPT-5 will not be any better
(indianexpress.com)
this post was submitted on 02 Dec 2023
156 points (85.1% liked)
Technology
57418 readers
5500 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If humans are any kind of yardstick here, I’d say all this is true of us too on many levels. The brain is a shortcut engine, not a brute force computer. It’s not solving equations to help you predict where that tennis ball will bounce next. It’s making guesses based on its corpus of past experience. Good enough guesses are frankly our brains’ bread and butter and most of us get through most days on little more than this.
It’s true that we can do more. Some of us, anyway. How many people actually exercise math and logic though? Sometimes it seems like… not a lot. And how many people hallucinate fake facts? A lot.
It’s much like evaluating self-driving cars. We may be tempted to say they’re just bloody awful, but so are human drivers.
I'd say the majority of humans know what 2 + 2 is. Chat GPT doesn't. As it found the answer in some texts it will tell you 4, but all it takes is you telling it that's wrong and suddenly it's 5. So even for the most simple math problem it's extremely easy to throw the whole thing off. Which also means for any prompt you put in it can go in wildly wrong directions at times.
And this is all with good input data, there's plenty of trolls online and the data will only get worse (it already did, the original data up to 2021 was okayish, in the last year tons of crap was put out on top, some of it by Chat GPT itself. So the new model might input the crap it produced before, getting worse over time). The problem on top of that is that you don't know the sources it used. If you ask about a recent event you might receive an insane answer it picked up from a right wing conspiracy site, you simply don't know. There is no fact checking in place.
It's a stunningly good text generator, but that's all it is and it ever will be, at least until they do much more than just add more compute power to it.
Hehe. I’m imagining sitting 100 human test subjects down in a lab setting and asking them what 2+2 is, and then telling them they’re wrong when they answer 4. I don’t know how many of them would guess again but I know it’s not zero. Meanwhile, GPT can probably give a better answer to any advanced math or science query than the majority of humans.
I’m a writer and a language nerd and I watch people all the time use words incorrectly because they think they know what they mean, but they really don’t. They’re just regurgitating them in what they think is the same situation they heard them. They don’t “understand” the word and are just guessing and churning out crap.
I don’t have a dog in this race but I think it’s interesting how people judge artificial intelligence with too much credit given to what goes on with human intelligence. Most people who say it’s “just a next word predictor” read that phrase somewhere and are regurgitating it, not at all dissimilarly to what LLMs do. They use phrases like “it doesn’t actually understand” without being able to define, with any clarity or precision, and without resorting to examples, what would actually impress them as real intelligence.
Only if that answer is already out there and in the model. So pretty much a Google search away.
GPT isn't coming up with new math or science facts (at least not real ones).
It literally is a word predictor, an insanely complex one, it's the best way to describe it. If you start with layers, parameters and so on most people lose interest. But there's some really good explanations around.
Generic AI (real AI) has internal logic, can learn and improve itself and can do self motivated actions. Chat GPT can tell you exactly how to create an account and order something from Amazon, but despite being able to put that text out it will never be able to actually follow them itself.
That's not true. I wanted a vba script for Excel. I don't know vba or excel so I spent hours searching Google for help. There were explanations of functions but no working code. I tried GPT for the fun of it and it spit back working code. Code that was nowhere on the Internet.
It was able to put together functions into working code based on the definition of functions, not simply cutting and pasting what somebody else had already written.
Again, pretty similar to the vast majority of humans. How many times in your science education did you learn ann equation that you’d already figured out on your own previously?
And to be fair, GPT doesn’t have hands and the ability to conduct experiments. So we have to, in a sense, judge its success on an apples to apples basis of what it, and we, do with the corpus of written knowledge.
In contrast to humans, GPT has at least read it all ;) (I say this in jest - I know it doesn’t have access to everything, but humans are too lazy to read, for the most part, even things they have access to).
This is a really good ELI5 explanation of its limit.
That would still give it too much credit in that case. It's purely an input output system. You put text in (the prompt), you get text out (the result). If there is no input from you, there is no output. It doesn't have any intrinsic functionality that runs on its own.
Maybe a bit too much for an ELI5.
How is it a definitional limit on its intelligence that it can’t use interfaces designed for people with hands? You also cannot send an http request with your lips no matter how you try - that’s just not an interface made for you.
Bots can 100% operate websites and take online actions, conduct quality tests, write fake reviews. That doesn’t mean they are intelligent. I just can’t see how it has any bearing either way whether ChatGPT can place an Amazon order.