LLMs are not simulating human behaviour, they are stochastic text models. A magic text microwave so to speak. If you ask an LLM “What is 2+2” it does not know that it’s 4 and it does not even know what math is. The implicit prompt, so to speak, is “What is the most probable text following 2+2”, which is “=4”. This is what makes the responses so uncanny-valley-human-like: it’s a distilled stochastic model of how humans communicate, but since it’s stochastic over all possible human data, this is what pushes it into uncanny valley territory and is what we usually register as “huh, is this AI”, and this is also what makes anthropomorphism so easy.
In the same way LLMs have data on human responses to discomfort, pain, torture, which is what they regurgitate in this “experiment”.
All this is a overengineered way of telling the LLM to scream “ouch oof owie my ram” but like a human.
That’s a lot of words to say the same thing the comment you replied to said.
The point stands, going through hoops to make a LLM say things that a human being tortured would say is the same vibe as " acthualky it’s a 9000 year old dragon so it’s okay"
LLMs are not simulating human behaviour, they are stochastic text models. A magic text microwave so to speak. If you ask an LLM “What is 2+2” it does not know that it’s 4 and it does not even know what math is. The implicit prompt, so to speak, is “What is the most probable text following 2+2”, which is “=4”. This is what makes the responses so uncanny-valley-human-like: it’s a distilled stochastic model of how humans communicate, but since it’s stochastic over all possible human data, this is what pushes it into uncanny valley territory and is what we usually register as “huh, is this AI”, and this is also what makes anthropomorphism so easy. In the same way LLMs have data on human responses to discomfort, pain, torture, which is what they regurgitate in this “experiment”. All this is a overengineered way of telling the LLM to scream “ouch oof owie my ram” but like a human.
That’s a lot of words to say the same thing the comment you replied to said.
The point stands, going through hoops to make a LLM say things that a human being tortured would say is the same vibe as " acthualky it’s a 9000 year old dragon so it’s okay"