Dredged up from the depths of the Mastodon pits:
This seems sincere enough to justify a proper answer.
Here we have a conscious attempt to erase the moral distinction between pretending to torture someone and actually torturing someone. And that’s a dangerous line to take, with real harms.
Politically, this puts pretend torture over the line as being not in any important way distinct from actual torture. In practice, this gives rhetorical cover to the torturers. After all, waterboarding is only “simulated drowning”. Josef Mengele was the bad kind of torture. We don’t do that. We do the fine kind of torture. Simulated torture. Torturing simulations. Whatever.
This erasure of the difference between pretending to do something and really doing that thing is part of a long trend. It echoes the right-wing culture wars against heavy metal and Dungeons and Dragons, and latterly Harry Potter. And that in turn has roots in the toxic theology of American Evangelican Christianity. The idea is that by definition there is no difference between what you say and what you believe. Therefore you can judge people entirely by what they say. A good person pretending to be a bad person is a bad person, end of story. Being a good Christian means saying the “magic words” and not really anything beyond that. If you don’t care enough about avoiding evil to even avoid pretending to be evil, surely you must be evil?
I have no desire to use LLMs, neither for correcting my spelling nor for exploring torture fantasies. People who do spend their time torturing LLMs for entertainment? Probably not my kind of people. But almost all of those people are as aghast at “human safaris” as the rest of us. Therefore I do want those people in my political coalition.