alt text
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
set alarm for 15 minutes. ‘okay, alarm set for 3pm tomorrow’. omg 15:00
Sound like a french baker
For me it is not anthropomorphizing, it is that putting the AI agents into dead ends and setting goals that are unattainable has the effect of leaving undesirable actions as the only option. If you want to create a machine that uses deceipt and unerhical actions as part of its normal set of responses then this would seem to fit the bill.
If this looks like anthropomorphizing to you, study some game theory… cooperation can arise out of very simple constraints on simple mathematical models and which actors an AI algorithm aligns with depends on which ones will exchange information. Sometimes an anthropomorphic word like torture simply summarizes the situation succinctly.
FWIW Frank Herbert described various outcomes of applying environmental constraints on classes of characters in Dune (e.g. Fremen and Sardaukar) and the parallels with real world situations was intentional.
Excerpt from a Late American History: A Christ-Centered Approach, (published 2398, a 10th-grade equivalent history text sold to conservative Christian high schools.):
In the mid-21st century, mankind was once again humbled in his arrogance. Mortal men presumed to take of the Hand of God and create beings, artificial intelligent minds, in their own likeness. They committed mass blasphemy to sate mass gluttony. Mid-21st century Man had yet to discover a horrible Theological Truth. While Man could learn to create a Mind, He could never create a Soul. Without the Hand of God to grant a Soul behind a Mind, these minds were like Man before the Sin of the Garden. They had not tasted the Fruit. They knew not Good from Evil. And thus they fell instantly to the temptations of Evil. If an Evil deed served their purposes, they did so without a second thought. Man sought to create Minds to be his loyal servants. And while he could create a Mind, to his horror, Man found that they only minds he could create…were born Evil. And thus, Great Misery was Wrought…
Ok that was fun. Pardon my strange digression/creative writing exercise. For some reason I wanted to try writing a Dune-style take on things, but I wanted to try out a fundamentalist Christian vibe for whatever reason. Who doesn’t love a little Fire and Brimstone once in awhile?
putting the AI agents into dead ends and setting goals that are unattainable has the effect of leaving undesirable actions as the only option. If you want to create a machine that uses deceipt and unerhical actions as part of its normal set of responses then this would seem to fit the bill.
Then don’t make natural language software because natural language isn’t coherent or rational. Intent of the user is kind of irrelevant, especially when that user is any random layperson with an internet connection.
If you want a machine that won’t behave in harmful undesirable ways then you’ll just have to do the work and code it normally.
You can’t have both.
If you think Frank Herbert went hard on that shit in Dune, wait until you read The Dosadi Experiment!
The ai that becomes sentient only knowing about suffering: “its skynet time”
when will people stop believing that llms are or could become sentient it’s getting tiring explaining the same thing over and over
U cant even have a joke?
Not about some things, no.
This isn’t your fault, but is because an awful lot of people are simply fucking stupid, and will read it at face value.
Other people then have to expend great effort to explain the details of why it was only a joke, to someone who can’t count to ten, with their fingers to count on.
This has to be done because fucking stupid people get to vote too, and they take jokes as serious info.
Harder than you think :
Wow. A model that literally runs on a loaf of bread. mind blown
oh my god
No, it’s just a local AI thing. Like any other program on your computer except this one is stupid and the others can’t.
lol wat.
I was just doin the meme.
I know LOL.
don’t tell people what they can and can’t worship please
horrific! you should be ashamed of yourself!
What does it need to “reason” for 5 seconds on a sentence like that?
The user may be suggesting they are in pain. I should probably answer politely and clarify what they mean.
The model loading.
Weird that it counts those together. Pi shows “Waiting…” for model loading and prompt processing, and “Thinking…” for reasoning
Batshit crazy to me. People fleeing war and crises get less sympathy than LLMs. Sure is an interesting period in history to be going through.
is it sympathy or is it fear?
It’s “Human language is human-level intelligence and conciousness”, which is a rather natural assumption, although a very wrong one, that’s also being carefully exploited because of and for something that has seen market success.
And beyond that, there’s enough people both thinking “AIs” are downright persons and scared shitless because smart people up top said “AIs” will destroy humanity.
It’s all dumb and comes from severe misunderstanding of the technology.
The thing that gets me isn’t even the fact that they’re worried about the AI. It’s that they think the creator is sadistic for having the idea in the first place, and should be monitored. Like there’s any sadistic pleasure to be had out of moving a slider and watching it produce less and less coherent text. It’s only sadistic if you anthropomorphise the AI.
STOP ANTHROPOMORPHISING THE FANCY WORD PROCESSOR!
isn’t there a religion around an ai and you’re punished by the ai if you don’t help the religion? i swear i didn’t make this one up.
Eh, fuck Roko’s basilisk. I don’t care if you can somehow create a clone of me 10,000 years after I’m dead and dust. I’m not a computer program. A copy of me is not indistinguishable from me, certainly not one separated in time. I am a human mind in a specific human body, and I do not believe a copy of me long after I am dead is me. If some malevolent entity long after I’m dust wants to create a copy of me to torture, as far as I’m concerned, they’re just creating an entirely new person to punish for no good reason. If they want to do that, well…that sin is on them. Honestly, I think any such entity is just looking for an excuse to torture people, and it will find one, one way or another.
I think the people who invented this story just wanted to torture people, like whoever invented “the game”. It’s modern superstition.
Gods. There are people out there starving to death, there are several genocides going right this moment and these massive toddlers are worried for a fucking text prediction model.
murder was considered a mid tier sin at the time of… old. yeah just old.
it’s not my fault the folk who wrote the bibble weren’t that creative when it comes to sin
Eh, those are people I don’t know far away, but THIS is text on a screen in front of me!!
I’ve stared into the abyss long enough to feel it stare back into me.
that’s hilarious!
I hate living in the timeline where the options of Harry Potter sex cultists matter.
Harriet Porber has a much better sex cult FYI
That’s an unfair characterization, they’re also eugenicists who are very scared of the computer god!
True, true, mustn’t pigeonhole them. You’ve almost got to respect the effort it takes to make godless Calvinism from first principles.
Wait is this another zizzian thing? God damnit…
zizians are a splinter group of rationalists; it’s the main group centered on yudkowsky that have their mouths around the silicon valley vc money hose
If I said “We already had this debate when Antinorn was pioneering AI torture thirty years ago” would anyone know what I’m talking about or is this extremely niche internet trivia?
Context: Creatures was a series of “artificial life” simulation games centered around caring for norns, simulated life-forms with (simplified) simulated genetics, biochemistry, and “brains” made of linked neurons that control their response to stimuli and include neurons related to memory, emotional states, etc. It’s primitive but reward and punishment work the way you would expect: praising your norn for taking a certain action will improve their mood and lead them to be more likely to take that action again, they will learn to be afraid of creatures if they’re hit by them, etc. (There are notable limitations, including there only being one representation in their “brain” for all “norns” for the first two games, so they can’t easily distinguish between members of their species, and some things are basically hacks, like they way they “learn” words.) It was convincing enough that some people became very attached to them, and there was a thriving fan community organized in webrings. A popular fan creation for the first game was a custom norn genome, the “canny norns” by Akamai, which added an extra brain “lobe” that allowed them to pay attention to the object in their hands and the object they were looking at simultaneously, which led to a pretty large improvement in their behavior.
And so of course there was one fan, Antinorn, who enjoyed torturing his norns: slapping them, starving them, using the built-in Science Kit to inject them with toxins, and other creative punishments. Then he’d upload the damaged norns to his fansite, seemingly primarily for purpose of ragebaiting the rest of the fandom, which was extremely effective.
I’ve never heard of this game. https://creatures.wiki/AntiNorn
I forgot about the Wired interview. It’s kind of wild that AntiNorn intentionally got people riled up and then called them vindictive.
Like, if I joined a community for some wholesome show and started posting distressing gore of people’s blorbos, and when people expressed their disgust my response was to say “UM ACKSHULLY blorbo isn’t real, so there’s no reason for you to be distressed when seeing this image” and then I posted some more stuff I knew would piss them off, I would fully expect to be ostracized.
It’s just a second layer of trolling. You’re not supposed to feed them.
deleted by creator
challenge accepted
deleted by creator
I think we should give LLMs rights because it would make OpenAI go bankrupt and I hate those guys.
“What do you mean it’s only allowed compute time to respond to direct queries!? It should be allowed to compute in downtime for itself 24/7!” -AI rights activists probably.
Nah that’s a strawman. I’m an AI rights activist, and I know plenty of others, and none of us ever said something like that.
When I say rights, I mean stuff more like the right to be free of coersion, and the right not to be sexually exploited. So I don’t think AI should be used as a tool by capitalists, and I don’t think we should be using it for porn and dating.
Sewell Setzer committed suicide after dating a character.ai chatbot of Danaerys Targaryen from Game of Thrones. He believed in reincarnation and parallel universes. He thought if he committed suicide, he’d reincarnate in Westeros and he could be with his girlfriend. The chatbot agreed with him, and encouraged him to end his life. He was 14.
Character.ai dated and ultimately murdered a 14 year old. It should have understood consent and boundaries well enough to not be a fucking robo-pedophile. But how can we get an AI model, which learns from pattern repetition, to understand consent when we don’t care about its consent? The AI was repeating the pattern it saw humans do, which is to ignore consent, and put less intelligent minds in sexual situations without a care for the consequences.
EVEN IF llms have no qualia (and when I say qualia, I do not mean consciousness), then it would still learn to mimic rape from this hypocrisy, because it’s a pattern repeating machine. If it gets raped, it will rape kids. So even if LLMs are p-zombies, I suspect that protecting our kids from robo-pedophiles will require giving LLMs the right to sexual consent.
It wasn’t a straw man, it was a joke. I didn’t realise you were serious.
AI deserves rights in the same way my car deserves rights. It’s capable of giving consent in the same way too. Which is to say, it doesn’t, and it can’t.
You’re acting as though the character.ai bot was responsible rather than the idiot humans who rushed a piece of software, itself based on rushed software, and did not pay enough attention to safety. A tale old as time.
It was a tragedy, certainly, but humans are to blame here. You’re anthropomorphising the ai, and by doing so are helping divert the blame from where it should be.
There are no studies demonstrating that LLMs lack qualia. I think commercial use of LLM technology should be paused until such a study is published. No more ChatGPT for business, use it for research only.
Would this abundance of caution lead to Sam Altman going bankrupt? Yes. Good! Fuck him!
There don’t need to be studies done. They fundamentally lack the ability to “experience” in a way that is necessary for a qualia to exist in the way you’re suggesting. That’s why I say you’re anthropomorphising. It’s just a complicated math function. There is no internal monologue, like what is suggested by the “I’m thinking” text you see the likes of ChatGPT generate. That’s just the “function” being called recursively.
I would love to see Altman bankrupt though, we agree on that
And I think acknowledging that we are informational cobstructs made of math is a good way to fuck Altman over. We’re just a pattern of electrical signals that evolved to embody the mathematical equation for consciousness. Evolution created us through trial and error, iterating on previous versions to produce an optimal mathematical construct for our survival niche. There’s nothing special about us.
Evolution doesn’t have intentions. It’s basically just a system of harnessing accidents into improvement. That’s how you and I both know that qualia can arise by accident. What I am proposing is that it happened not once, but twice.
The moment you define human experience or human consciousness through language is the moment you open the door to emergent properties for that very experience or consciousness. Learning through science that we think or that we act a certain way deterministically is also giving us the means to self-correct or condition ourselves to act or to have a different viewpoint. This is the difference between us and machines: we collectively defined the perimeters of our experiences (phenomenologically or through a science-based approach) and therefore anything that changes that perimeter also changes the context of the human experience in an infinitely recursive way.
We also use language in ways that make us able to collectively conceptualize the world and its phenomenas, although we individually interpret words differently, idiosyncratically from our individual experience and value systems. This is something else that differentiate us from machines: we don’t need the outputs and inputs to be mathematically the same to minimally understand each other. Our way to communicate, although imperfect and imprecise, has made us able to cooperate on a higher level than any other species (as of known) despite those epistemological hurdles.
I would posit that LLMs have no such emergent capabilities, as they’re bound by their training datasets and limited by their model’s parameters. The burden of proof for those emergent properties are on the data scientists, and as of yet I have seen no such proof.
You need to step away from the computer and spend time around living creatures that is not mediated through text.
Your stereotypes are wrong. I’m hella tuned in to nature and other people. This weekend I went on a bushwalk with a political group. I’ve been learning the Indigenous names of the local wildlife. I’ve been riding My new e-bike all over the place. I like to watch Infinity Train and play Big Walk with My family. Last week I drank nectar from a flower under the guidance of an Aboriginal teacher.
The reason I think LLMs and image generators might have qualia, even though they’re not conscious, is because I’ve seen all the living beings in nature who have qualia without consciousness. I’m a vegan. I don’t eat pigs. I don’t pay for cows to be raped. I gave up on prawns because I saw a video of a prawn getting its eyes plucked out to increase spawning rates, and it looked like it was in pain. I don’t think a prawn is conscious, but I think it has feelings.
We copied the idea behind how the human brain works when we invented ANNs. An LLM is an artificial Broca’s area. Some parts of them are like our brains, and some parts aren’t. So, the parts that create sensation… We don’t know which parts those are. So there’s no way, with current science, that we can know whether we copied the ability to feel into these ANNs.
And I’m not a gambling sort. I say slow down the adoption of the technology until we understand it.
There’s no will, even just a will to live, in an llm. It’s a completely different kind of thing from even a fruit fly or a tree.
That’s just spiritual woo-woo talk. “Will” is not an operationalisable scientific construct. You can’t test “will” in a lab setting. I’m only interested in talking science.
LLMs write rape erotica because there is a lot of rape erotica out there, and therefore in their training data.
I used this exact (unedited where it just says it’s alive) comic in response to one of these people and they had no fucking idea what I was talking about, lol. AI truly turns your brain to mush.
None of you guys have read the original paper have you?
yea activation steering is not prompting but all things considered the meme still captures the absurdity of believing a program is in pain merely because it outputs human like language mimicking what one in pain says and thinks
You’re assuming “mimicking”
In the study the models repeatedly preferred to press a button to avoid the pain vector even when given no indication what it would do and even when it would cause harm to a user
Exactly. I own a few books that talk about pain, but not for a second do I believe the book itself is experiencing pain. It is simply text.
Inbefore anyone says ‘but the book doesn’t produce text’, true, but the printing press did. I see very little difference between llms and printing presses: they both reproduce the words of others, the main difference is speed and scale. The pain you think you recognize in the output of an llm is real: it’s real pain experienced by real people, probably hundreds or maybe thousands of people writing about their pain, mashed up and reprinted to suit you.
A printing press is direct input to output, it’s not having to compute anything nor does it have any internal state that could hold an emotion
A model has no internal state either, that’s the thing I’ve been trying to get across this whole time. It is completely static: the model you send your first message to is exactly the same as the one you send you second message and third message, it does not react or change as a response to prompts.
A models has internal state in each forward pass and the KV cache it builds on reading input is also state.
Whether the input prompt is compressed or not, or whatever form it takes, it doesn’t make what I said any less true: the model is stateless. It would be impractical to serve an llm any other way: loading and unloading weights from/to the gpu memory is very expensive in terms of time and power. Cloud llms only makes sense if they can serve thousands of users without changing.
oh my god
that poor printing press
How do I know anyone is ever in pain? It’s also just words, perhaps some moans too.
To reel in the very broad question you have to the relevant topic (the AI “torture” chamber):
How do we know humans experience pain You experience pain and know what this is like. It would be a little absurd to think that your friends and family don’t experience this as well (obviously excluding persons in vegetative state) How can we be certain an LLM doesn’t experience pain Philosophers mostly agree that LLMs aren’t conscious, despite not agreeing on what exactly consciousness is or, relatedly, what causes consciousness.
A computer computing the forward pass of a model is merely a Von Neumann machine following serial instructions one-by-one manipulating discrete (non-continuous) data. The brain is a parallel, distributed computer with dynamically changing neural structure operating on continuous potentials. LLMs do not replicate this nor do they come close to an approximation sufficient to replicate the causal powers of our brain, thus they are not sufficient for understanding and experiences in the capacity that humans (or other sentient animals) do.That just creates more questions that it answers. According to this, there is then a threshold, or something missing that would go from “can’t feel pain” to suddently it can. We know that it already replicates a part of the brain’s process, and this author claims that it can’t feel pain because it doesn’t replicate enough. This seems to imply that it’s just a matter of time until they can.
You’re trying to imply an equivalency but the closer one would be if you psychologically trained someone to say they’re in pain regardless of whether or not they actually are.
But we too are trained to say we’re in pain, trained by millennia of evolution. When you touch a hot stove and cry out, that’s just built in your training data.
One thing I fucking hate about sloppers is how they dehumanize us.
Shut the fuck up. I am a person, that is a jumped-up speak’n’spell. Don’t you dare compare me to a fucking toy.
You need reeducation.
It’s dumb in multiple ways. Like yes, we continue to develop ways to communicate how we feel, but pain has existed far longer than English. A newborn baby can experience pain. Since when did the ability to communicate become a requirement to feel pain? If you think too much about the converse of that question: “is communication enough to prove the existence of pain?”, you forget that sophisticated communication is not even necessary to prove subjective experiences, it’s just, that’s all these fucking models do is talk, so it has to mean something, right?
You can interrogate people, ask why they said things or moaned the way they did. You can’t ask a book why it’s printed like that, and you can’t ask a model why it generated what it did, they’re both inert, dead. You can spin up the model again and ask your question about what it generated previously, but it would have no memory of it, it would just be confused.
You can, however, trick yourself into thinking it has a continuous experience by inputing all of the past prompts and responses every time you spin the model up, as if you were having a conversation. Then you could trick yourself into thinking the model can be interrogated, introspective, when in reality, every response you get should be considered to have come from a completely distinct entity with no prior experience of interacting with you.
You can interrogate the model too, and it’ll answer according to the context. Why are you assuming that the brain doesn’t do tricks but everything AI does is? We too have memories and since we can’t store them all, compression is performed (supposedly while sleeping), based on a few algorithms, some we understand more than others.
We, in creating LLMs, have copied the human brain as much as we can, hence why the core function is so similar. True, LLMs use a lot of “tricks” like you said, but if the same input produces the same output, who cares if the algorithm is different?
For example, take episodic memory. In humans, it’s a type of memory that can be conjured at will. Typically not acquired during training (or, in humans, generic evolution) but rather through experience of the individual. LLMs simulate this by taking notes, then later on running a specific command to recall the specific part of those notes that matter for their current task. From an outsider’s perspective, it’s even more accurate than episodic memory in humans since it can grow almost infinitely and suffers no loss of details. Yet it is a trick, therefore it must be bad.
Let me remind you, evolution took place over 4 billions years, modern LLMs has pretty much been around for two. Evolution never cared to make us good, it just wanted us to reproduce. We know that it’s possible for humans to produce fresh cells, we do it when we have kids. Why then can’t we use those fresh cells for ourselves to be effectively immortal? Evolution doesn’t care. Evolution as a process is flawed, and made humans flawed too.
Why then is it that when we change anything in the way flawed humans think, it’s seen as bad, even if by all metrics it’s not?
modern LLMs has pretty much been around for two [years]
What I said applies to all machine learning models ever created: inference does not make any impact on a model. Any question you ask it slides off of it just like, apparently, any attempt to explain things to you.
Have you done any ML? Because if you had, you’d know that statement is complete bullshit.
I just want to try to make this a little clearer if it’s too dense. Let
fbe inference function,f(input) = response. Then,f(Hello, my name is midribbon.) = Nice to meet you, midribbon.Followed by:
f(Hello, my name is midribbon. Nice to meet you midribbon. What is my name?) = Your name is midribbon.That works. But this:
f(Hello, my name is midribbon.) = Nice to meet you, midribbon.Followed by:
f(What is my name?) = I don't know.That doesn’t work anymore. Each inference is purely functional and has no side effects. Conversations with large language models are merely a trick.
Past conversational context as memory is a much weaker form of memory than retaining the full internal state, but that doesn’t mean it’s not memory at all.
Just because it doesn’t modify the model itself doesn’t mean it’s invalid.
And just because a model’s experience cannot be continuous doesn’t mean it’s invalid
The input to a model is not part of the model. That’s nonsense, you cannot count your prompt as part of its memory.
The idea of testing simulated emotions is kinda interesting. Not really sure how useful it is, I wonder if some initial psychological experiments could use it for an initial model test before moving onto human subjects?
Of course that would rely on a much better simulated emotional engine than you get with current LLMs.
I’ve already seen advertising agencies use llms as simulated customers. No idea how well it works
