Can AI really improve itself recursively until it becomes powerful enough to cause human extinction? Nobody sponsored this video, I turned down funding speci...
The important thing to keep in mind is that we are not currently at risk of anyone inventing a super intelligence. The word guessing engines are getting amazingly good at guessing words, but the huge jump from that to actual intelligence, super or otherwise, remains out of reach with currently known technology.
I agree that if we did manage to invent that, the biggest threat would be the humans doing short-sighted, greedy and generally stupid shit with it. No coincidence that that’s also the biggest threat from current LLMs.
Edit: “Super-hammers could break containment and kill us all if we don’t invest more in controlling them!” shouts man wildly swinging around a hammer in a crowded room. “The government needs to regulate this technology so that only people like me, who use it responsibly, can have access to it,” he added. He tried to say more, but was out of breath after some particularly enthusiastic hammer swings.
Best responses so far in this thread. People need to stop calling these ‘whatever’-intelligence engines. They’re NOT INTELLIGENT. They are probabilistic text-synthesizers. Stochastic Parrots. Or one of many other terms people have begun using, any of which are much more accurate and do not incorrectly assign ‘intelligence’, agency or motivations to what is still only an algorithm. One with ‘temperature’ (randomness) parameters thrown in, to jiggle the retrieved phrasing of sentences somewhat, and so on.
The reason these things always ‘hallucinate’ (some are advocating we drop that term as well, and simply say the algorithms yield incorrect results) can NOT be fixed, as it’s a side-effect of the fundamental way they synthesize their answers.
My fear is that the stupid money people will weaponize what they percive to be a smart machine, and then fall back behind “The AI said so” as an excuse. People who understand computers know it isnt intelligent, but people who cant understand any concepts beyond “more money” will think it’s a god and act accordingly.
The important thing to keep in mind is that we are not currently at risk of anyone inventing a super intelligence. The word guessing engines are getting amazingly good at guessing words, but the huge jump from that to actual intelligence, super or otherwise, remains out of reach with currently known technology.
I agree that if we did manage to invent that, the biggest threat would be the humans doing short-sighted, greedy and generally stupid shit with it. No coincidence that that’s also the biggest threat from current LLMs.
Edit: “Super-hammers could break containment and kill us all if we don’t invest more in controlling them!” shouts man wildly swinging around a hammer in a crowded room. “The government needs to regulate this technology so that only people like me, who use it responsibly, can have access to it,” he added. He tried to say more, but was out of breath after some particularly enthusiastic hammer swings.
Best responses so far in this thread. People need to stop calling these ‘whatever’-intelligence engines. They’re NOT INTELLIGENT. They are probabilistic text-synthesizers. Stochastic Parrots. Or one of many other terms people have begun using, any of which are much more accurate and do not incorrectly assign ‘intelligence’, agency or motivations to what is still only an algorithm. One with ‘temperature’ (randomness) parameters thrown in, to jiggle the retrieved phrasing of sentences somewhat, and so on.
The reason these things always ‘hallucinate’ (some are advocating we drop that term as well, and simply say the algorithms yield incorrect results) can NOT be fixed, as it’s a side-effect of the fundamental way they synthesize their answers.
And then there’s the architectural issue where LLMs have no way of separating commands and data, so hacking them will always be possible.
You’re simply not using “intelligence” in the same way the literature does.
My fear is that the stupid money people will weaponize what they percive to be a smart machine, and then fall back behind “The AI said so” as an excuse. People who understand computers know it isnt intelligent, but people who cant understand any concepts beyond “more money” will think it’s a god and act accordingly.
Oh yeah, that’s already been happening and going to get worse, especially so as long as the “AI race” narrative holds sway.