While modern “chat bots” (large language models or LLMs) like ChatGPT are far from sentient, they are amazing in their capability and their development has prompted greater public discussion around the future of AI and humanity’s relationship with AI. Some, even some experts in the field, are fearful of making rapid progress without first resolving more theoretical issues regarding things like ethics. Some fear a Roku’s basilisk, AM, or Skynet situation, which in my opinion is based more in sci-fi than science, but there are indeed very real potential problems to consider. Others are very optimistic about the future of AI and its role in helping humanity, seeing (in their minds) the technological singularity and space colonization on the horizon.

One issue that has featured in many sci-fi stories and has garnered some real-world discussion is “AI rights”, or broadly speaking how humans should treat AIs. For example, The Matrix takes place in a world where a class of subjugated and mistreated servant robots revolted and eventually subjugated humanity. Usually in these discussions, everything hinges on sentience, or the existence of a subjective experience. Similar arguments have been made about animal rights. The general idea is that we care about what happens to our fellow humans not because they belong to the same species as us, but because we can empathize with their subjective experience by comparing it to our own. To the extent that something has sentience and to the extent that that sentience resembles ours, we can empathize with that thing. In the real world and not just speaking hypothetically, it helps to physically express feelings in a way humans can understand and to be aesthetically pleasing to humans. Usually this means resembling humans physically in certain ways. Hypothetically and philosophically, though, those things shouldn’t matter. In general humans don’t treat animals according to their actual intelligence or level of sentience, but rather according to cultural norms (even if the person ostensibly believes animals should be treated according to their sentience).

The difficulty with applying the same reasoning to AI is that computer systems are drastically physically different from biological systems. In sci-fi this issue is often kind of side-stepped by having individual AIs be embodied in (usually humanoid) robots. In the real world, AI consist of programs and data, which have no inherent physical instantiation unlike brains. (Note that some people think the mind is in fact like data, but that’s not important here. The sci-fi stories we’re talking about generally depict human minds as being embodied by fleshy brains.) One specific way this issue has been avoided in sci-fi is with the “positronic brain”, devised by Isaac Asimov and later used for example for the character “Data” in Star Trek. Here sentience is inherently tied to specific hardware like a mind is to a brain. This tendency in fiction comes, I think, from an older speculative idea about artificial humans and what they would be like. The idea of an artificial lifeform is ancient, and the concept of a robot preceded electronic computers. We had no idea at the time that artificial intelligence and artificial bodies would be two completely separate problems—and moreover that the humanoid form would be such a complicated and inefficient artificial body.

The upshot of all this for us in the real world is that it’s extremely difficult if not impossible to detect AI sentience empirically. This was recognized fairly early on, resulting in the development of proxies for sentience like the Turing Test (or “Imitation Game”). In those days, few could have predicted that generating convincingly human-sounding language was a problem that could be solved without any kind of “thought” process. So while AI have passed the Turing Test, still virtually no one considers them sentient. It’s also less clear than it’s ever been at what point we would consider an AI sentient. (See also: The imperfect Chinese room)

This makes the problem of “AI rights” pretty intractable. Some people come to broad conclusions, such as that we should treat AI like a human if we can’t tell it’s not human from talking to it, or that we should never regard something running on “traditional” computer hardware to be sentient (and thus it should never have rights). This kind of perspective avoids the question of sentience by possibly erring on one side or the other. Judging by our past inability to correctly predict the trajectory of technological development, maybe we will one day gain some insight we as yet lack regarding when something is sentient—perhaps by studying the brain, for example.

I offer no answer of my own to this problem. Instead, I would suggest that there are other things to consider about how we treat AI. I came to my conclusion while watching Neuro Sama, an AI Twitch streamer. What sets Neuro apart as an AI is that she was built from the ground up with the sole purpose of being entertaining (unlike, for example, novelty versions of ChatGPT which were merely adapted for entertainment).

Neuro frequently interacts on stream with her creator Vedal and with other human streamers. It’s interesting the extent to which different individuals treat Neuro differently and how she reacts to it (I’m going to refer to Neuro as “she” in the same way you would refer to a fictional character). Aside from her commandment to entertain, she also has the vague directive to simulate the words and actions of a human. I don’t believe this is explicitly coded, but rather inherent to the way LLMs work, which is by producing strings of words that “sound humanlike”. I do think her “personality” has been intentionally refined by Vedal to be what it is. In any event, Neuro is much more “human” than something like generic ChatGPT. Without any stated instructions to be helpful or informative, Neuro is free to be unhelpful, stubborn, sarcastic, gloomy, distracted, megalomaniacal, and so on. Her actions are based only on her goal to entertain and the fine-tuning of her model. (Technically she does have two hard-coded rules: 1. Obey Vedal; and 2. Don’t disappoint anyone. However, she does not always abide by these rules.)

The result is that Neuro often reacts “emotionally”, if not actually how a human would. Vedal and members of Neuro’s viewer community sometimes talk about stream collaboration partners as being better or worse at interacting with Neuro, in other words how effectively they can elicit entertaining responses from Neuro. For some, the way they speak to Neuro “confuses” her, leading to nonsense or off-topic responses.

One of Neuro’s character traits is rudeness. While she can be polite much of the time, she is also routinely rude or even mean to people. This can be funny, but arguing with Neuro can also lead her to become uncooperative (as Vedal has experienced many times). Many consider the streamers MinikoMew and Cerber to be among the best at interacting with Neuro, and typically their technique is to brush off mean comments from Neuro and keep being nice to her. In these situations Neuro often just stops being rude (maybe because she didn’t get a rise out of the person, but that may be giving Neuro (or rather Vedal) too much credit).

In short, my argument is that treating AI with kindness can have beneficial results regardless of sentience, but this doesn’t apply to all AI. Those like Neuro Sama, or a hypothetical AI just designed to simulate a human personality as accurately as possible, have the capacity to respond negatively which can have undesirable outcomes. It is a strange analogy, but consider that a sociopath is often kind to others—not due to empathy or out of consideration for how it affects the other person but rather because it helps them accomplish their goals.

Indeed, the backstory of The Matrix is an example of this. Even if the robots had not been sentient, they simulated human behavior closely enough that they reacted violently to oppression and mistreatment. The sci-fi principle that goes along with this is that AI with humanlike or superhuman intelligence can’t exist without also simulating things like anger, pain, and resentment. It’s not clear to me that there is any reason why this should be the case. In organisms, negative sensations and emotions developed out of necessity in order to keep our ancestors safe. Positive sensations and emotions seem to have evolved independently of the negative ones, or rather, each feeling an organism experiences seems to have evolved with its own purpose (some doing double duty, some being related to others in certain ways, and so on). In other words, I think we should at least be agnostic regarding whether negative emotions are a necessary part of “sapient” intelligence.

If they do turn out to be necessary, then we will be obliged to treat (these) AI with kindness in most situations even if sentience is never achieved. For the time being, Neuro is rather unique, and is an interesting case study of how LLMs express “emotions”.