Big Medium

While there's tons of focus right now on how we talk directly with LLMs, one of the things that I'm most excited about is how they can talk to other systems. When AI-mediated experiences are interoperable, exciting things happen. Reliable structured data is at the root of it—It's what makes agents go and what enables radically adaptive UI. So I was unreasonably excited to see OpenAI announce the addition of new discipline to the model to help ensure the reliability of the structured data it returns. Most of us experience OpenAI’s GPT models as a chat interface, and that’s certainly the interaction of the moment. But LLMs are fluent in lots of languages—not just English or Chinese or Spanish, but JSON, SVG, Python, etc. One of their underappreciated talents is to move fluidly between different representations of ideas and concepts. Here specifically, they can translate messy English into structured JSON. What this means for product designers: Moving nimbly between structured and unstructured data is what enables LLMs to suggest and deliver interface elements, to make function requests to other systems, to act on your behalf, or to share information in new ways. The interoperable ability of Sentient Design systems to "speak machine" is the stuff that will animate the next generation of interaction design. Alas, as in all things LLM, the models sometimes drift a bit from the specific ask—the JSON they come back with isn’t always what we asked for. This latest update is a promising direction for helping us get disciplined responses when we need it—so that Sentient Design experiences can reliably communicate with other systems.

Is your new AI-powered product feature a hermit or an expat? An automaton or a shapeshifter? A “jack of no trades” or a polymath? These are a few of the nuanced “postures” that I uncovered when exploring the forms of user experience that intelligent interfaces can take. Here’s a whiteboard sketch that I did to explore those postures for the NPC (non-player character) design pattern. More on that in a sec. I’ve adopted the term “posture” to describe how a feature or interaction positions itself in relation to the user. More than just distinct functionality, each posture has its own interaction style, manner of communication, and expectations that it sets. When cooking machine intelligence into interfaces, I’ve found it fundamentally important to identify not only the high-altitude posture (chat, agent, copilot, or tool) but to drill into subtle variations by adjusting the dials of a few key attributes. I wrote a thing called “The Shape of Sentient Design” a few days ago, sharing a visual framework to explore these postures and full landscape of AI-mediated user experiences. (Sentient Design is the term that Veronika and I use for intelligent interfaces that are aware of context and intent so that they can be radically adaptive to user needs in the moment.) You can find that article here: https://lnkd.in/edRumKCm I’ve found the triangle diagram that I shared there to be a powerful thinking tool for explaining and exploring Sentient Design experiences. But! A 2D representation of a complex reality has its limitations—so when I apply one of its design patterns to my product work, I find it helpful to pluck that pattern out of the triangle framework and turn its dials for how grounded, interoperable, and radically adaptive the experience is. The sketch here shows some work I did while developing a NPC product feature. (NPC means “non-player character,” a term from the gaming world that refers to an automated character with pre-determined behaviors. In more general Sentient Design experiences, NPCs might appear as Slack bots, Figma users, or Miro sidekicks; they have a user account and some agency, but the system runs them.) The sketch shows what happens when you waltz the NPC pattern along the spectrum of those three core attributes. What’s cool is that each node really starts to describe degrees of personality, capability, and interaction. And then you can mix and match notes to get “shape-shifting local guides” or “automaton diplomats.” None of this is one-size-fits-all. New ideas emerge. It’s revealing—and super fun.

My first post on the Big Medium website! While researching for my upcoming book with Josh Clark, Sentient Design, I've noticed a recurring theme of corporate anxiety surrounding AI. This reminds me of the fall of 2022 and the release of ChatGPT. I spent that time on campus at NYU, and after an initial period of uncertainty, I saw teachers, students, and AI tools find a productive rhythm by the following spring. At Big Medium, we're bringing that same calm, forward-thinking approach to our work with clients. We even have some ideas on how to help YOUR team build a positive and productive relationship with these new sentient tools. More below! #SentientDesign #AI #UX #DigitalExperience #DesignInnovation

Forget sparkles; let’s talk triangles. This triangle diagram is a great thinking tool for exploring the UX landscape of AI features. Use it to get your bearings and organize how you approach emerging AI product experiences. Big thanks and huge props to Matt Webb for developing the original visualization. I tweaked Matt’s version to help it click into place for me—and rhyme better with the Sentient Design framework. But most of the diagram comes directly from Matt's giant brain. The triangle plots AI-mediated experiences based on how much each experience blends three characteristics: - Grounded: has the info/training needed for reliable results - Interoperable: shares data and instructions with other systems - Radically adaptive: morphs in real-time to user needs and context Dialing these attributes up and down yields a rich variety of features, but when you pull up to a higher altitude, four fundamental archetypes emerge: tools, copilots, agents, and chat. Each of these (overlapping) archetypes suggests specific functionality, but just as interesting, they also present their own *posture* to the user: - Tools are controlled by the user through prompts or controls (often through iteration that sculpts the result). - Agents flip the script and do their work with little oversight until they return with their results. - Chat presents as a peer to the user in conversations that can go in any direction. - Copilots stake out the middle of the diagram and blend attributes of all of these to provide real-time collaborative support. Dive down into more detail, and specific nodes in the diagram light up to represent AI-mediated experiences, each with its own opportunities and constraints. I’ve found the map helpful for developing ideas and concepts. I take the problem to solve and then roam the triangle to workshop how different AI postures could address it. I’ve also used it for product positioning by mapping clients’ competitors to identify market gaps or to reinforce our clients’ differentiating features. This framework is hugely useful, but it remains (like all maps) a simplified representation. It doesn’t catalog every possible AI interaction model—and that’s okay! Think of the diagram as more of a compass than a map. Rather than a complete catalog of what’s possible, it suggests directions to explore Sentient Design opportunities. Sentient Design is what Veronika and I call intelligent interfaces that feel almost self-aware in their response to user needs. We describe Sentient Design experiences as: aware, radically adaptive, collaborative, multimodal, continuous/ambient, and deferential. Those characteristics are great describing the vibe; what I love about the triangle is that it’s great at describing the *form*. The shape of Sentient Design is triangular. I wrote up lots more about how the map works (and how it doesn’t) along with how and why I adapted it from Matt’s original: https://lnkd.in/edRumKCm

Sentient Design is a thing. And it will literally be a thing when we publish Josh Clark and Veronika Kindred's book on it later this year or early in 2025... Sign up for notification here:

Registration for our free AI and Design Systems virtual event is now open! Join us on June 13th at 12PM ET as we dig into how AI can be used to supercharge many aspects of design system work. #designsystems #ai #generativeai #event #ui #design #development

Registration for our free AI and Design Systems virtual event is now open! Join us on June 13th at 12PM ET as we dig into how AI can be used to supercharge many aspects of design system work. #designsystems #ai #generativeai #event #ui #design #development

For your viewing pleasure! Here’s my talk introducing Sentient Design. If you and your design team are trying to figure out how to create actually meaningful applications of AI in your product work, I hope this helps. I’d love to hear what you think. Sentient Design is what I call the already-here future of intelligent interfaces: experiences that feel almost self-aware in their response to user needs. Sentient Design moves past static info and presentation to embrace UX as a radically adaptive story. These experiences are conceived and compiled in real time based on your intent in the moment—AI-mediated experiences that adapt to people, instead of forcing the reverse. I think it’s a fun talk! It’s clear-eyed about what AI is crummy at, along with its special strengths and curious characteristics. Turns out that what most of us THINK generative AI is good at is actually what it's worst at. There are other, better opportunities. This is so much more (and in exciting ways, so much less) than chatbots, text prompts, and systems that can write, speak, or draw. In a nutshell, Sentient Design is all about the new opportunities to add intelligence to our interfaces (sometimes just a dash, other times a heaping spoonful). Some of this is so familiar that you might consider it just plain dull (content recommendations or prediction), but there’s a lot of surprising new possibility here, too. It’s all part of our kit as designers now—and especially powerful in combination. Sentient Design describes the form of this new user experience, but it’s also a framework and a philosophy for working with machine learning and AI as design material. And! It’s ALSO a book—or will be soon. More details to come, but I’m writing the Sentient Design book with Veronika Kindred for Rosenfeld Media. But first, the talk. It’s available at the link below in video, audio, text, and slides. (And hey: big, big thanks to Friends of Figma, Miami for hosting the talk and for providing and sharing the video.) https://lnkd.in/ecgWrG5J