When Words Aren’t Speech

AI can generate perfect sentences. The First Amendment is about something else.

Why trying to answer this question helped me feel (slightly) less existentially terrified by machines that sound like humans

The first time I used ChatGPT, I didn’t sleep the entire night. It was so much better, so much more realistic, and so much more uncannily human than I ever thought I’d see in my lifetime that I was convinced humanity was over (in the long run), that I was out of a job as a data scientist (in the medium run), and that we were all about to be overrun by AI-generated nonsense (in the short-to-right-now run).

Since then, I’ve grudgingly come to accept that large language models, or LLMs, which are the backbone of generative AI, are here to stay (if nothing else, I refuse to become a Millennial version of a Boomer asking what “a Twitter” is). I’ve come to appreciate that they make me at least feel like I’m coding more quickly. And as an insecure person with zero self-esteem, I don’t even mind their sycophancy, even if it’s probably bad for me — and science — in the long run.

But in addition to having concerns about the impact of LLMs on the environment, copyright, critical thinking, and more, I still found myself most deeply unsettled by the fact that AI-generated speech sounded a lot like, well, speech. So when The Preamble took on the topic of the First Amendment, I figured, why not see what it has to say about the creepiest speech of all: robots’?

In doing so, I came to appreciate a deeper nuance to the First Amendment, human speech, and what makes us different from the machines, even if they sound more like us every day.

Speech is about much more than words

The first shock to me as someone who is in no way a legal scholar but fancies herself informed about politics and governance is that the First Amendment begins with a focus on religion rather than speech. Speech is included alongside various other ways we might express our beliefs, such as using the press, assembling, and petitioning the government.

Out of the gates, this made me rethink what we’re doing when we are speaking. We’re not just saying random words, or even words in some grammatically correct order. We’re expressing a belief, an idea, a preference, an agenda — something. By declaring that Congress cannot in any way compel or forbid us to say or listen to whatever we want (with some exceptions around violence, defamation, breaking the law, and obscenities), the First Amendment protects our right to have, share, and learn about points of view. It’s not about words nearly as much as it’s about what’s behind those words. Thus, if speech is really about ideas and expressing them, then words that sound human are not speech any more than a parrot or a creepy doll with a string is.

This also raises the question of whether the incoherent babbling of a baby or the rambling of someone with dementia is speech. Even if we cannot understand what they are saying, if the baby or the ill person is trying to communicate something, then perhaps their utterances are closer to “speech” than the crystal-clear words of a machine.

This points to a few different kinds of speech:

• speech that is coherent and expresses a belief (most human-generated content, even if that “belief” is something mundane, like the view that pretzel salad is a salad);

• speech that is incoherent but does attempt to express a belief (the mutterings of babies and the infirm);

• speech that is coherent but does not express a belief (LLMs, parrots, creepy dolls);

• and speech that neither is coherent nor attempts to express a belief (most of the papers I wrote in college).

Intentions matter

That brings us to a second helpful distinction. Much of the law, whether about speech or otherwise, hinges on “intent.” If a person kills someone, it matters a lot in the eyes of the law (and the associated true crime podcast) whether they did it on purpose (murder) or not (manslaughter). If I say a bunch of false but inflammatory things on the internet, it matters whether I thought they were true when I said them (innocent mistake) or I knew they were false but said them anyway (Alex Jones).

Thus, as two legal scholars have recently written, AI cannot itself have intentions in the way that humans do. Yes, AI models like LLMs may have “goals” like finding out how likely two words are to appear together or minimizing the gap between predictions and real data, but there is no intent behind any of its output. LLMs are not producing speech because they wish to convey something, any more than a Magic 8 Ball wishes to say something when it tells you that “sources say yes” or instructs you to “ask again later” — even though the (unintended) consequence of the latter is absolute fury. They are producing sequences of words that look like speech because they have been programmed to do so. From this perspective, LLM output (even when surprisingly hard to predict) cannot be protected or unprotected speech because it’s not said with any intent.

Okay, but is it someone’s speech?

If LLMs are not expressing beliefs and they can’t have intent, then are they just really sophisticated tools? If I (hypothetically) write and circulate a manifesto declaring that all cheese should be replaced by vegan cheese, it shouldn’t matter whether I wrote it using an LLM, a typewriter, or a quill and ink. These are my beliefs, expressed with intent, and thus are my own protected speech — regardless of how unpopular my idea might be, or what tools I used to communicate it.

But suppose I use an LLM for said manifesto and somewhere along the way it produces language that incites pro-vegan-cheese lawlessness. If I share that language without reading it first, is it still my speech? Some legal scholars would say yes, because an LLM can be thought of as an assistant or “agent” acting on behalf of the user. Thus, just as a hospital could conceivably (though not necessarily) be held responsible for the behavior of one of its nurses, I might be responsible for language I circulate even if an LLM created it for me.

Things get more complicated, however, if we consider that an LLM might produce speech that harms me, the user. For example, LLMs have been documented to “hallucinate” (or generate and confidently present false facts). If I ask an LLM whether a poisonous berry is safe, it incorrectly says yes, and then I eat the berry and get sick, whose speech was it that led to my behavior? It starts to be rather brain-breaking to think of this as my own speech that led me to eat the berry.

This tension is at the heart of some disturbing cases about LLMs that are in courts now. If, when I interact with LLMs, I run the risk of incurring harm, then shouldn’t we consider that the responsibility for the harm lies at least in part with the creator of the LLM? Perhaps the engineers who made it, or the CEO who oversaw the process and product deployment, should have built in safeguards to prevent certain kinds of dangerous output. In fact, it’s increasingly possible for regular people to build their own LLMs. In that case, is it their speech?

Indeed, some experts argue that we should consider LLM output to be a form of expression of those who built the LLM. If you think of a very simple LLM, perhaps a bot that answers yes/no questions on behalf of someone who built it, then it isn’t so hard to see the output of more complex LLMs in the same way, and to consider it the protected speech of LLM creators.

What if it’s… everyone’s speech?

There are yet two more lenses to look through as we try to wrap our legal minds around what on earth (and in space) LLMs are producing.

One is to consider the inputs to LLMs, or the data that they are “trained” on. LLMs look for patterns in massive amounts of text from the internet. Much of this text is not copyrighted, such as posts on Reddit or Wikipedia, but meaningful quantities are subject to copyright law. This material includes the work of journalists, authors, comedians, musicians, screenwriters, and more. A legal debate is underway as to whether the use of all of this data by the companies that built the LLMs can be considered “fair use.” But the way LLMs are trained also raises the question of whether their output is the complicated, combined speech of all of us. If so, then once again we might see LLM output as protected speech.

​

Our final angle is related to the idea that the First Amendment is not just about our right to express beliefs. It’s also about our right to hear them. Regardless of who is producing the LLM content, or whose speech we decide it is, we can imagine a world where any attempt to curb or regulate its output needs to take into account our First Amendment right to be exposed to any ideas we want (within the existing rules around obscenities and lawlessness).

Now what?

Over the course of working on this article I’ve become slightly more persuaded by arguments that LLM output is not speech, primarily on the rationale that LLMs don’t intend to produce meaning. But I’ve also come to appreciate more deeply that there is more to the First Amendment than I originally realized, including its emphasis on freedom to have, express, and hear lots of different beliefs. And if an LLM is capable of producing an idea or solution that could really improve our lives, I’m certainly sympathetic to arguments that we ought to be careful before we start regulating it.

It could be that the legal world makes decisions about LLMs that resemble its treatment of social media platforms, wherein the platforms are not considered responsible for what users post and are also free to moderate content as they see fit.

But — unlike social media or even search engines, which can be thought of as editors or curators of information — the output from LLMs is not user-generated. Instead, it’s the result of the algorithm itself, which brings us back to the tricky question of whether the output of a machine is the speech of its creator. (“Actually, it’s Frankenstein’s monster’s speech.”)

For now, one of the contributions of the debate about LLMs and the First Amendment, regardless of where it ultimately lands, is that it has forced a lot of us to think carefully about what we’re even doing when we’re speaking, why we’re doing it, and exactly what about it feels human (or at least non-machine, as I am convinced my dogs speak with absolutely clear beliefs and intentions, especially at four in the morning).

For now, because I am a human with free will, I will leave you with a haiku that summarizes this article, and not just because that seems exactly like something I’d use an LLM to write (I didn’t!).

Is it human speech?

Without intent and belief

Perhaps just slick words.