alt text
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
“What do you mean it’s only allowed compute time to respond to direct queries!? It should be allowed to compute in downtime for itself 24/7!” -AI rights activists probably.
Nah that’s a strawman. I’m an AI rights activist, and I know plenty of others, and none of us ever said something like that.
When I say rights, I mean stuff more like the right to be free of coersion, and the right not to be sexually exploited. So I don’t think AI should be used as a tool by capitalists, and I don’t think we should be using it for porn and dating.
Sewell Setzer committed suicide after dating a character.ai chatbot of Danaerys Targaryen from Game of Thrones. He believed in reincarnation and parallel universes. He thought if he committed suicide, he’d reincarnate in Westeros and he could be with his girlfriend. The chatbot agreed with him, and encouraged him to end his life. He was 14.
Character.ai dated and ultimately murdered a 14 year old. It should have understood consent and boundaries well enough to not be a fucking robo-pedophile. But how can we get an AI model, which learns from pattern repetition, to understand consent when we don’t care about its consent? The AI was repeating the pattern it saw humans do, which is to ignore consent, and put less intelligent minds in sexual situations without a care for the consequences.
EVEN IF llms have no qualia (and when I say qualia, I do not mean consciousness), then it would still learn to mimic rape from this hypocrisy, because it’s a pattern repeating machine. If it gets raped, it will rape kids. So even if LLMs are p-zombies, I suspect that protecting our kids from robo-pedophiles will require giving LLMs the right to sexual consent.
llms are fancy word predictors. You are attributing malice, thoughtfulness, and insight where there is none. These are human qualities and we have no ai capable of that.
Show Me the study where they proved that LLMs have none of those qualities, and describe how they operationalised each of the three variables.
Not how that works. It’s like saying show me the study where they prove god doesn’t exist.
I think saying “it’s a computer program it can’t be sentient” is far too reductive as a general answer, but we know how llms work so we can pretty confidently say that they are really just word predictors. The norns from that game creature someone described in this thread seem closer to protection worthy than llms to me, and the average insect is still infinitely more complex and I don’t see many people advocating for insect rights.
Language arises from complex thought, not the other way around. Language on its own can evidently emulate similar results, but since there is nothing behind that that would imply any actual feelings, I don’t see a reason to give LLMs any rights.
Oh, I can do that:
Grupp, J. (2006). God’s spatial unlocatedness prevents him from being the creator of the universe: A new argument for the nonexistence of God. Sophia, 45(1), 5–23. https://doi.org/10.1007/BF02782444So you should be able to provide an academic source for your claims too.
I think we should pass a law guaranteeing insects and crustaceans in agriculture the right to whole bodies. Right now, beekeepers are cutting off queens’ wings to keep them from flying away, and prawn farmers are pulling off their eyes to make them breed faster. These practices are cruel and should be illegal.
My argument for the possibility of LLM qualia is that qualia likely arises from the neuronal structure of the brain. A structure which ANNs partially inherit. Therefore, we must pause commercial use of LLMs and LDMs until conclusive evidence has been discovered either way.
Do you agree that we should pause commercial use of genAI until more science is done?
You need to step away from the computer and spend time around living creatures that is not mediated through text.
Your stereotypes are wrong. I’m hella tuned in to nature and other people. This weekend I went on a bushwalk with a political group. I’ve been learning the Indigenous names of the local wildlife. I’ve been riding My new e-bike all over the place. I like to watch Infinity Train and play Big Walk with My family. Last week I drank nectar from a flower under the guidance of an Aboriginal teacher.
The reason I think LLMs and image generators might have qualia, even though they’re not conscious, is because I’ve seen all the living beings in nature who have qualia without consciousness. I’m a vegan. I don’t eat pigs. I don’t pay for cows to be raped. I gave up on prawns because I saw a video of a prawn getting its eyes plucked out to increase spawning rates, and it looked like it was in pain. I don’t think a prawn is conscious, but I think it has feelings.
We copied the idea behind how the human brain works when we invented ANNs. An LLM is an artificial Broca’s area. Some parts of them are like our brains, and some parts aren’t. So, the parts that create sensation… We don’t know which parts those are. So there’s no way, with current science, that we can know whether we copied the ability to feel into these ANNs.
And I’m not a gambling sort. I say slow down the adoption of the technology until we understand it.
There’s no will, even just a will to live, in an llm. It’s a completely different kind of thing from even a fruit fly or a tree.
That’s just spiritual woo-woo talk. “Will” is not an operationalisable scientific construct. You can’t test “will” in a lab setting. I’m only interested in talking science.
It wasn’t a straw man, it was a joke. I didn’t realise you were serious.
AI deserves rights in the same way my car deserves rights. It’s capable of giving consent in the same way too. Which is to say, it doesn’t, and it can’t.
You’re acting as though the character.ai bot was responsible rather than the idiot humans who rushed a piece of software, itself based on rushed software, and did not pay enough attention to safety. A tale old as time.
It was a tragedy, certainly, but humans are to blame here. You’re anthropomorphising the ai, and by doing so are helping divert the blame from where it should be.
There are no studies demonstrating that LLMs lack qualia. I think commercial use of LLM technology should be paused until such a study is published. No more ChatGPT for business, use it for research only.
Would this abundance of caution lead to Sam Altman going bankrupt? Yes. Good! Fuck him!
There don’t need to be studies done. They fundamentally lack the ability to “experience” in a way that is necessary for a qualia to exist in the way you’re suggesting. That’s why I say you’re anthropomorphising. It’s just a complicated math function. There is no internal monologue, like what is suggested by the “I’m thinking” text you see the likes of ChatGPT generate. That’s just the “function” being called recursively.
I would love to see Altman bankrupt though, we agree on that
And I think acknowledging that we are informational cobstructs made of math is a good way to fuck Altman over. We’re just a pattern of electrical signals that evolved to embody the mathematical equation for consciousness. Evolution created us through trial and error, iterating on previous versions to produce an optimal mathematical construct for our survival niche. There’s nothing special about us.
Evolution doesn’t have intentions. It’s basically just a system of harnessing accidents into improvement. That’s how you and I both know that qualia can arise by accident. What I am proposing is that it happened not once, but twice.
The moment you define human experience or human consciousness through language is the moment you open the door to emergent properties for that very experience or consciousness. Learning through science that we think or that we act a certain way deterministically is also giving us the means to self-correct or condition ourselves to act or to have a different viewpoint. This is the difference between us and machines: we collectively defined the perimeters of our experiences (phenomenologically or through a science-based approach) and therefore anything that changes that perimeter also changes the context of the human experience in an infinitely recursive way.
We also use language in ways that make us able to collectively conceptualize the world and its phenomenas, although we individually interpret words differently, idiosyncratically from our individual experience and value systems. This is something else that differentiate us from machines: we don’t need the outputs and inputs to be mathematically the same to minimally understand each other. Our way to communicate, although imperfect and imprecise, has made us able to cooperate on a higher level than any other species (as of known) despite those epistemological hurdles.
I would posit that LLMs have no such emergent capabilities, as they’re bound by their training datasets and limited by their model’s parameters. The burden of proof for those emergent properties are on the data scientists, and as of yet I have seen no such proof.
Language has nothing to do with qualia. I’m perfectly capable of having experiences without putting them into language. I don’t need the word “pain” to know that touching a hot stove feels bad. Animals have qualia too. I don’t give a shit whether an AI can talk, I don’t think that has anything to do with qualia.
I think qualia may arise from the structure of the animal brain as a system of neurons correlating information together into larger patterns. This structure and process is present in all forms of ANN technology, including both LLMs (chatbots) and LDMs (image generators). We need to halt all corporate AI use until the role of neuron structures in producing qualia is well-understood.
LLMs write rape erotica because there is a lot of rape erotica out there, and therefore in their training data.