What We’re Reading

A great reminder from Ethan Mollick of how quickly things have changed in AI generation quality in the last 18 months. AI right now is the worst that it will ever be; only getting better from here. Good inspiration to keep cranking!

When I started this blog there were no AI chatbot assistants. Now, all indications that they are likely the fastest-adopted technology in recent history.

Plus, super cute otters.

OpenAI introduced a bit of discipline to ensure that its GPT models are precise in the data format of their responses. Specifically, the new feature makes sure that, when asked, the model responds exactly to JSON schemas provided by developers.

Generating structured data from unstructured inputs is one of the core use cases for AI in today’s applications. Developers use the OpenAI API to build powerful assistants that have the ability to fetch data and answer questions via function calling(opens in a new window), extract structured data for data entry, and build multi-step agentic workflows that allow LLMs to take actions. Developers have long been working around the limitations of LLMs in this area via open source tooling, prompting, and retrying requests repeatedly to ensure that model outputs match the formats needed to interoperate with their systems. Structured Outputs solves this problem by constraining OpenAI models to match developer-supplied schemas and by training our models to better understand complicated schemas.

Most of us experience OpenAI’s GPT models as a chat interface, and that’s certainly the interaction of the moment. But LLMs are fluent in lots of languages—not just English or Chinese or Spanish, but JSON, SVG, Python, etc. One of their underappreciated talents is to move fluidly between different representations of ideas and concepts. Here specifically, they can translate messy English into structured JSON. This is what allows these systems to be interoperable with other systems, one of the three core attributes that define the form of AI-mediated experiences, as I describe in The Shape of Sentient Design.

What this means for product designers: As I shared in my Sentient Design talk, moving nimbly between structured and unstructured data is what enables LLMs to help drive radically adaptive interfaces. (This part of the talk offers an example.) This is the stuff that will animate the next generation of interaction design.

Alas, as in all things LLM, the models sometimes drift a bit from the specific ask—the JSON they come back with isn’t always what we asked for. This latest update is a promising direction for helping us get disciplined responses when we need it—so that Sentient Design experiences can reliably communicate with other systems.

In The New Yorker, Kyle Chayka bemoans the creeping blandness that settled into his Spotify listening experience as the company leaned into algorithmic personalization and playlists.

Issues with the listening technology create issues with the music itself; bombarded by generic suggestions and repeats of recent listening, listeners are being conditioned to rely on what Spotify feeds them rather than on what they seek out for themselves. “You’re giving them everything they think they love and it’s all homogenized,” Ford said, pointing to the algorithmic playlists that reorder tracklists, automatically play on shuffle, and add in new, similar songs. Listeners become alienated from their own tastes; when you never encounter things you don’t like, it’s harder to know what you really do.

This observation that the automation of your tastes can alienate you from them feels powerful. There’s obviously a useful and meaningful role for “more like this” recommendation and prediction engines. Still, there’s a risk when we overfit those models and eliminate personal agency and/or discovery in the experience. Surely there’s an opportunity to add more texture—a push and pull between lean-back personalization and more effortful exploration.

Let’s dial up the temperature on these models, or at least some of them. Instead of always presenting “more like this” recommendations, we could benefit from “more not like this,” too.

Scratching your head about diffusion models versus frontier models versus foundation models? Don’t know a token from a transformer? Jay Peters assembled a helpful glossary of AI terms for The Verge:

To help you better understand what’s going on, we’ve put together a list of some of the most common AI terms. We’ll do our best to explain what they mean and why they’re important.

Great, accessible resource for literacy in fundamental AI lingo.

Oliver Reichenstein shares strategies for using AI to elevate your own writing instead of handing the job entirely to the robots. (This rhymes nicely with the core principle of Sentient Design: amplify judgment and agency instead of replacing it.)

Let’s turn the tables and have ChatGPT prompt us. Tell AI to ask you questions about what you’re writing. Push yourself to express in clear terms what you really want to say. Like this, for example:

I want to write [format] about [topic]. Ask me questions one at a time that force me to explain my idea.

Keep asking until your idea is clear to you.

Reichenstein is CEO and founder of iA, the maker of iA Writer. One of its features helps writers track facts and quotes from external sources. Reichenstein suggests using it to track AI-generated contributions:

What if the ChatGPT generated something useful that I want to keep? Paste it as a note Marked as AI. Use quotes, use markup, and note its origin. … iA Writer greys out text that you marked as AI so you can always discern what is yours and what isn’t.

It’s a good reminder that you can design personal workflows to use technology in ways that serve you best. What do you want AI to do for you? And as a product designer, how might you build this philosophy into your AI-mediated features?

In The New York Times, Teddy Rosenbluth reports on doctors using AI tools to automate their fight with insurance companies’ (automated) efforts to refuse or delay payment:

Doctors and their staff spend an average of 12 hours a week submitting prior-authorization requests, a process widely considered burdensome and detrimental to patient health among physicians surveyed by the American Medical Association.

With the help of ChatGPT, Dr. Tward now types in a couple of sentences, describing the purpose of the letter and the types of scientific studies he wants referenced, and a draft is produced in seconds.

Then, he can tell the chatbot to make it four times longer. “If you’re going to put all kinds of barriers up for my patients, then when I fire back, I’m going to make it very time consuming,” he said.

I admire the dash of spite in this effort! But is this an example of tech leveling the playing field, or part of an AI-weaponized red-tape arms race that no one can win?

There’s a lot of “ready, fire, aim” in the industry right now as execs feel pressure to move on AI, even though most admit they don’t have confidence in how to use it. At The Wall Street Journal, Ray A. Smith rounds up some recent surveys that capture the situation:

Rarely has such a transformative, new technology spread and evolved so quickly, even before business leaders have grasped its basics.

No wonder that in a recent survey of 2,000 C-suite executives, 61% said AI would be a “game-changer.” Yet nearly the same share said they lacked confidence in their leadership teams’ AI skills or knowledge, according to staffing company Adecco and Oxford Economics, which conducted the survey.

The upshot: Many chief executives and other senior managers are talking a visionary game about AI’s promise to their staff—while trying to learn exactly what it can do.

Smith also points to a separate spring survey of 10,000 workers and executives that cited AI as a reason 71% of CEOs and two-thirds of other senior leaders said they had impostor syndrome in their positions.

With limited confidence at the top, AI innovation is trickling up from the bottom. (This rhymes with our strong belief at Big Medium that to be expert in a thing, you have to use the thing.)

In fact, much of what business leaders are gleaning about AI’s transformative potential is coming from individual employees, who are experimenting with AI on their own much faster than businesses are building bespoke, top-down applications of the technology, executives say.

In a survey of 31,000 working adults published by Microsoft last month, 75% of knowledge workers said they had started using AI on the job, the vast majority of whom reported bringing their own AI tools to work. Only 39% of the AI users said their employers had supplied them with AI training.

Take five minutes to watch A Protopian Future, an Ignite talk by Jenny Johnston. She offers a provocation to really think about and describe the future you imagine will come of the things you/we are trying so hard to change right now.

Here, Jenny asks what the world might look like 50 years after nuclear weapons are abolished. Your thing might be something different. You’re probably PUSHING for something to be changed / added / removed in the world; but what future are you PULLING toward? What’s the good and the bad—intentional or unintentional—of the future that you’re designing today?

Protopian stories imagine better futures but not perfect futures. They embrace a kind of messy progress. The reason we’re seeing this protopian surge right now is because humanity is in a weird place. We have this tangle of existential threats in front of us that we’re having a hard time seeing past and certainly our way through…. Protopian stories are powerful tools for reorienting ourselves toward hope and possibility and not dystopian dread.

After you watch Jenny’s video, go check out the farfutures.horizon2045.org project she edited. So great.

John Gruber shares an under-reported tidbit from Apple’s many “Apple Intelligence” reveals:

The most unheralded aspect of Apple Intelligence is that the data centers Apple is building for Private Cloud Compute are not only carbon neutral, but are operating entirely on renewable energy sources. That’s extraordinary, and I believe unique in the entire industry.

LLMs are crazy-expensive across many dimensions, including environmental cost. Great to hear at least one company is tackling this head-on.

As usual, John also has lots of other insights on the announcements.

The inimitable Mandy Brown reminds us that the f*cks we have to give are a limited resource. Spend them in the right place:

Why love your work? It won’t, of course, love you back. It can’t. Work isn’t a thing that can love. It isn’t alive, it isn’t and won’t ever be living. And my answer is: don’t. Don’t give a f*ck about your work. Give all your f*cks to the living. Give a f*ck about the people you work with, and the people who receive your work—the people who use the tools and products and systems or, more often than not, are used by them. Give a f*ck about the land and the sea, all the living things that are used or used up by the work, that are abandoned or displaced by it, or—if we’re lucky, if we’re persistent and brave and willing—are cared for through the work. Give a f*ck about yourself, about your own wild and tender spirit, about your peace and especially about your art. Give every last f*ck you have to living things with beating hearts and breathing lungs and open eyes, with chloroplasts and mycelia and water-seeking roots, with wings and hands and leaves. Give like every f*ck might be your last.

Because here’s what I’ve learned: if you give your f*cks to the unliving—if you plant those f*cks in institutions or systems or platforms or, gods forbid, interest rates—you will run out of f*cks.

Illuminate is an experimental project from Google that generates accessible, podcast-style interviews from academic papers:

Illuminate is an experimental technology that uses AI to adapt content to your learning preferences. Illuminate generates audio with two AI-generated voices in conversation, discussing the key points of select papers. Illuminate is currently optimized for published computer science academic papers.

The service has a waitlist, but you can try out some generated conversations (and I recommend that you do!). The enthusiasm, intonation, and ums & ahs are convincing and feel authentic to the genre that the project mimics. (See also the PDF to Podcast project which does similar things but with flatter voice results.)

But it’s not the seeming authenticity that feels important here. Machine-generated voices—even at this level of fidelity—are nothing new. What’s more interesting is how this project demonstrates what large language models (and now large multimodal models) are truly great at: they are prodigious translators and transformers of symbols, whether those symbols are for language, visuals, or broad concepts. These models can shift those symbols nimbly among formats: from English to Chinese to structured data to speech to UI components to audio to image. These are systems that can understand a concept they are given and then work their alchemy to present that concept in a new medium or language or format.

There are exciting opportunities here for unlocking content that is trapped in unfriendly formats (where the definition of “unfriendly” might be unique to the individual). This application leans into what generative AI is good at (understanding, transforming) around tightly scoped content—and avoids what these models are uneven at: answering questions or building content from scratch. How might this kind of transformation support education efforts, particularly around accessibility and inclusivity?

Has breathless AI mania primed you for a ragey rant about overhyped technology and huckster pitchmen? Nikhil Suresh has you covered:

Most organizations cannot ship the most basic applications imaginable with any consistency, and you’re out here saying that the best way to remain competitive is to roll out experimental technology that is an order of magnitude more sophisticated than anything else your IT department runs, which you have no experience hiring for, when the organization has never used a GPU for anything other than junior engineers playing video games with their camera off during standup, and even if you do that all right there is a chance that the problem is simply unsolvable due to the characteristics of your data and business? This isn’t a recipe for disaster, it’s a cookbook for someone looking to prepare a twelve course f***ing catastrophe.

How about you remain competitive by fixing your shit?

There’s no such thing as a quick fix for a broken organization. And there’s no silver bullet for product excellence. AI is capable of amazing things, but you can’t shortcut great execution or ignore its very real downsides.

In another context, I often say, “High-performing teams have design systems, but having a design system won’t make you a high-performing team.” The same is true for AI.

There’s only one route to success: get your process and operations in order, understand the technologies you’re using, know their strengths and weaknesses, and above all: start with the right problem to solve.