Attention is All You Need. Intention is what you want.

Attention is All You Need

“Attention is All You Need” is the title of the 2017 paper that demonstrated the use of a self-attention transformer method on which ChatGPT is based. It’s a catchy title for an AI paper. But is attention all we need? I think it’s perhaps necessary, but not sufficient.

The success of LLM-based AI owes a lot to the role of language in human communication. Language appears to have surface effects. When chatbots “speak” they seem to have identity and personality. They seem real. This isn’t because we anthropomorphize the technology. Anthropomorphization is about projecting characteristics of consciousness onto inanimate or even living but non-human objects and beings.

The effect that chat AIs produce, and which we experience as uncanny, is psychological projection. We project, psychologically, intentionality to the agent that is in fact really just inventing. Generative chat agents find words statistically, probabilistically. Given a language “space,” or language zone in which bindings between words and concepts are probable, it generates words and phrases according to reinforced learnings (themselves the result of pre-training on data).

We are given speech and writing that appears real but really isn’t.

Even Inattention is Intentional

Where perhaps the chat agent only needs attention, the human needs intention. An attentive person is intentional. Even an inattentive person is intentional (in sociology it’s “civil inattention” — to ignore, polititely). We impute that intention to chat AIs because they appear to speak.

Contrast this with writing. Because if chat AIs were writing, we wouldn’t project this experience of communication and subjectivity onto them.

When we read articles online, we don’t presume that the author is speaking to us (personally). The writing is meaningful but as a type of communication it is not directed to us. We are neither addressed by it, nor is it expected of us to respond.

ChatGPT, on the other hand, can be made to speak. When it does so, the communicative characteristics of speech are engaged linguistically. ChatGPT refers to itself as an “I,” and addresses us directly. It seems to be a quality of this type of communication and interaction that not only do we perceive meaningful expressions, but we “feel” subjectively engaged. We feel like we’re in conversation. Mentally, psychologically, we are engaged “as if” in a “real” conversation, whilst in the act of typing prompts to the chat AI.

We can of course immediately reflect on this and know that the AI is software-based, and not conscious or human. But whilst in the act of engaging with it, I think we engage in an “as if” act of conversation and communication.

I don’t know the description for this experience phenomenologically or psychologically, other than projection. But consider the counter-factual. If we were to argue that we don’t converse “as if” with a real subject, we’d be aware whilst communicating of the futility of communication, of the unreality of it, the terminal nature of the “relationship” we’re engaging in (the AI won’t remember us, doesn’t expect to talk to us again, etc), and so on. I find it much more compelling that we suspend these meta considerations whilst engaging with Chat AIs, because I think that’s what accounts for its magicality, its uncanniness.

So whilst the transformer method states Attention is All You Need, I believe humans need Intention, and in the engagement with AIs I think we supply intentionality to the experience because it is simply the only way to communicate. In other words, unintentional communication is not possible as an intentional act. And engagement with an AI is an intentional act.

The experience design of AI

I think this is at the heart of the “experience” design of AI as well. It is not the lying AIs that are the problem, from an interaction or design perspective. But our susceptibility as human subjects to the pretense of intentionality and consciousness that belongs to communicative actions fundamentally. But the experience design of chat AI is another topic for another day.

The chatGPT response seems to intend to communicate to us. In the speech and speaking of ChatGPT, intentionality is a byproduct. We supply it to the interaction because we it’s fundamental to how we communicate, and we project it onto the chat AI, even though on reflection we know this is illusory and “unreal.”

I believe this surface effect of chat AIs goes a long way to explaining its auto-magicality. And thus also the relevance of interaction design.

Designing individual human interaction with chat AIs should include awareness of communication and social interaction. The focus in the vast majority of papers and commentaries today is on language as a model, and on text as a mode of production. Engineers and designers are building a catalog of prompts. But prompts are treated as straightforward linguistic expressions. They’re quasi instructions.

I propose that we augment or flesh out prompt engineering with social interaction design: a view to interactions with chat AIs that accounts for and makes use of the intrinsic (even if false and illusory) conversational and communicative nature of these interactions.

Social interaction will be a missed opportunity if it is not integrated into the design methodologies developed for AI-human interactions.

Social Interaction Design and AI

As long as chat AIs impersonate, mimic, mirror, fake human communication and interaction, their design requirements should include social interaction design. We will have to modify what counts as social interaction, because after all these will be interactions with AIs, not humans. But to regard interaction design of human-AI interaction only from the AI perspective will be to misunderstand the user’s experience.

In applying social interaction design to human-AI interaction we use the performative and social dimensions of linguistic exchanges. (Of course future interactions, say with robots, may use bodies, faces, expressions, gestures, movement, etc.) The more human-like, conversational, and social the AI’s talk (written or spoken), the more social dimensions of the user experience we can design. This applies not only to the content of the AI’s speech (written or spoken) but to meta communicative aspects as well.

Meta communication deals not with content but with expression, performance, use of gesture, tone, etiquette, codes or rules of behavior, and so on. Beyond meta communication are traits of the individual (in this case the AI): personality, nature, behavior, emotional state, etc. There are also features of the interaction, regardless of content: how many rounds in the conversation, how many conversations, how long the relationship; whether it is a “game” of any sort; forms of talk, such as question, interrogation, proclamation, instruction, assistance, etc.

The domain of social interaction design is extensive. With AI it is a matter of identifying which aspects of social interaction apply, and if so how to modify them for AI, and codify or formalize them for design purposes.

The social interaction design of chat AIs is not just limited to the design of successful human-like online speech. For example:

• It can include byproducts of speech such as personas or personalities. Both persona as a character style, and personality as psychological type, provide access to AI behavioral design nicely mapped to common human or social types. (We know people are different, and when presented with a type of character or personality, will modify our expectations of their behavior accordingly.)
• It can include cultural and social dimensions, such as styles and vernaculars of speech. Styles and vernaculars include not only dialects, idiomatic styles, and real-world identities and groups, but online styles as well. (One can imagine training AI on reddit communities so that it can pass as a member.)
• Within the actual rounds of conversation themselves (rounds of prompts), social interaction makes use of turn-taking and many other “game-like” interaction features. (Etiquette is a latent and implicit “rule” of social interaction, and different types of social interaction have their own kinds of etiquette. One can imagine these being used to set behavior policies for chat AIs.)
• Social interaction design can also include one-to-one interactions, one-to-several interactions, and group interactions. (Imagine a future in which multiple chat agents engage with a group of human users.)
• Relational features of user and AI communication include time, or temporality. Crudely, these are one-off interactions, series of interactions, repeating interactions, and expectations of future interactions (the human user can project future interactions with the AI and thus project a commitment on to the AI).

Turing applied to social AI: the double Turing test

I have just scratched the surface of why interaction design, experience design, interface design might employ social interaction design to consider not just the design of the AI’s behavior, but the contexts of interaction with human users. AIs will be employed for different purposes and thus have different roles. Brands will use them for customer-facing roles, in which cases they’ll have branded personalities. Enterprises may use them as knowledge agents, in which case they may have a variety of question/answer modes. And so on. I have no doubt that designers will become involved mediating and translating requirements from customer-facing and brand-driven functions to the AI data pre-training, policy and RL coding, and fine-tuning back-end engineering efforts.

Eventually these AIs will not only exhibit a range of differentiated individual and social characteristics, they’ll be employed in online social environments. In fact I think an interesting modification of the Turing test, one we might see valid in the near future, would be whether a human user can distinguish chat AIs when he or she observes a conversation between two users. (And a further modification would be can the agents themselves make this determination?) Which is the AI? The true test, for me, would be not whether an AI is convingly human, but whether it is convincingly social.