âMachine learning should be used as a research tool, not as a forecasting tool. It should be used to identify new theories, and once you identify a new theory, you throw the machine away, you donât want the machine.â âMarcos López de Prado
A caveat: López de Prado is speaking specifically about machine learning for market predictions, and he notes that markets resist prediction. “Markets evolve,” he said. “You are an investor and when you extract money from the market, the market learns to prevent you from extracting profits next year.”
Still, this resonates with a philosophy that has deepened for me the more I’ve worked with AI and machine learning: machine learning is better at signals than answers.
The first generation of mainstream AI applications has over-dialed on presenting just-the-facts answers. A one-true-answer mentality has created a whole raft of problems, some of them dangerous. Here’s the thing: the machines are flaky, with narrow and literal interpretations of the world. That means they’re brittle for decision-making. Instead of replacing human judgment, AI should amplify it. Machine learning is a mediocre substitute for human judgment and individual agency, but it’s an excellent signal booster for both.
I love the way López de Prado frames it: use the machines to surface patterns, signals, and suggestions to develop a theory for actionâand let humans make the decisions from there.
Because most chatbots understand only a very limited vocabulary, using them can become a guessing game to arrive at the precise incantation to make them do your bidding. The more we talk to robots, the more we talk like robots.
Will Oremus wrote this report in October about Slack’s expansion of support for third-party plugins. Those plugins were previously limited to text-only chatbots—via either conversational UI or specific “slash commands”—but can now offer more traditional GUI elements like windows, buttons, forms, and so on.
It seems Slack’s users found the chat-only UI too challenging because of its rigid command-line syntax. Discoverability was a challenge, and users found it hard to remember the precise words to make the bots go, or even which bots were installed. “Nobody should have to be a specialist in the dozens of apps they interact with on a daily or weekly basis,” said Andy Pflaum, Slack’s head of platform, in an interview.
Bots will “continue to exist and have their role in
Slack,” Pflaum said. But the company’s research has
found that “the typical user isn’t as comfortable with
those, or forgets how to use those methods.” Testing
of more graphical interfaces has generated “so much
positive response,” he added, and should make apps
accessible to “a much broader base of users.”
Slack’s investment in feature expansion at once suggests the success of the plugins (1800 third-party apps and counting), but also the limiting nature of plain-text UI at a moment when bots still have very narrow language understanding. This will get better as natural language processing (NLP) improves and bots get more flexible in what they can understand. We’re already seeing that happen in the latest generation of NLP (see AI Dungeon for a fun example).
In the meantime: when you can take advantage of the full range of UI on a specific platform, you should—and that’s exactly what Slack is doing here. The future of interaction is increasingly multi-modal (and multi-platform for that matter). Enabling people to move nimbly among modes and platforms is as important as the ability to move among services, the very point of third-party plugins in the first place.
“Mobile normalized the idea of computing beyond the screen” is a tidbit I offered. Not only did mobile make touch mainstream as an input alternative to keyboard and mouse, it also made sensor-based computing utterly normal. The camera, the microphone, the accelerometer, GPS—all became expected, everyday surfaces for interaction in the first full decade of the smartphone. For that matter, “invisible” interactions became commonplace, too, with notification-based interfaces driven by data-backed predictive services.
At an even more fundamental level, mobile changed consumer expectations of what software should be—in quality, ease of use, and even personality. “Applications used to be gray, bland, functional affairs imposed upon us to do the mundane tasks of the day,” I suggested to Carmel. “Mobile really blew that up.”
Carmel’s central theme is not only that mobile was the key driver for digital product design in the last decade, but that this sea change also had several follow-on (and follow-on-follow-on effects). “Mobile sped up the pace of everything, design included,” she writes. A few of her call-outs:
The collection, storage, and use of personal data exploded.
Data-driven design and A/B testing became part of the standard toolkit for designers (for better and for worse).
As more companies built on top of identical infrastructure (AWS, cloud services, open source software), design became the distinguishing factor for products, not the code.
Big companies started hiring big design staffs, instead of outsourcing design work to agencies.
Educational programs, bootcamps, and self-serve courses about design have proliferated to meet the need for more designers.
Design tools exploded to meet the growing, varied, and dynamic needs of interaction design.
What comes next?
Carmel closes her essay by noting that designers are now charged with figuring out how to use their new and growing influence to focus tech on meaningful opportunities—and limit tech’s potential and demonstrated dangers. “Many people we interviewed mentioned the moral responsibilities that lie ahead,” she writes.
For me, that will be all about how we choose to feed the algorithms, present their results, and limit their risks. Carmel and I talked about this in our interview for the article, though it didn’t make the final cut. Here are a few of my comments from that conversation:
If mobile defined the last decade of digital product design, machine learning is already defining the next.
Algorithmic interfaces already drive so many of the digital products all of us use every day. For better and for worse, algorithms determine the news we see, the movies we watch, the products that are surfaced, even the way we drive home from work. For designers, the next decade is all about understanding our roles and responsibilities in using and shaping the algorithm as a design material. How do we use data and sensors and machine learning in ways that are meaningful, personal, useful—and most of all, respectful and responsible? That’s the opportunity and challenge that will be mobile’s legacy—and the work of design in the coming years.
Are you or your team wrestling with how to adopt and design for machine learning and AI? Big Medium can help—with executive sessions, workshops, or full-blown engagements for product design and development. Get in touch.
There is a lie—a simple but damaging fiction—that seeps under the doors of our yoga studios and burrows into our meditation cushions. It dresses up in cute quotes on Pinterest and has hacked its way into countless Instagram accounts. It smells weird; like someone tried to burn sage over a garbage can. But if we wrinkle up our noses and ask if anyone else smells that weird smell, all we get in response is nervous laughter. Our ears ache from the shrillness of the hollow words; we’re choking on the dust of toxic positivity; and through the haze we think we can see a place where the air is clean–but standing between us and that clearing stands a group of people wearing t-shirts with ancient holy symbols on them, which were purchased from a big box store.
The lie, Liza writes, is the insidious suggestion that you’re not good enough. If only you would hustle harder, do more yoga, keep up your Headspace streaks, and just “work on yourself” more… you could finally be the better person you were meant to be. “Anyone who wants to fix you thinks that you are broken,” Liza warns:
This idea—that we are broken and need to be fixed—is what I call The Golden Cut. It is self aggression masquerading as a journey towards self acceptance. It is a damaging fiction that has invaded the lives and hearts of so many of us, an unchecked moral imperative that has cloaked itself in the language of wellness and well-being.
Constant striving has become the planned obsolescence of the wellness world. It’s precisely how capitalism has invaded wellness.
Liza puts her finger on something that has always felt off-putting to me about aspects of mindfulness as an industry. While mindfulness (and yoga and meditation and its other many cousins) have real and evident benefits, they’re often wrapped in a superficial commercial culture that broadcasts a smug kind of judgment, superiority, and shame to those “not in the club.” For a movement that promises personal growth, that’s just gross.
Liza is explicit about separating out the practices and services of wellness from their toxic alter ego, the wellness industrial complex. It’s not wellness or wellness-related businesses that are bad, she writes; it’s the cynical and manipulative marketing that suggests you are broken and need to be “improved.” It’s healthy to seek and cultivate personal development, and of course to acknowledge our flaws, “but none of this means that we are broken; it means that we are human.”
So what’s this got to do with design?
The very purpose of interaction design is to shape behavior. It guides users through an intentional path to a desired outcome. Commercial design seeks to shape that behavior to the benefit of the company. When done well, of course, it also benefits the customer; both get what they need. All too often, though, that power dynamic becomes lopsided, and the interests of the company steamroll those of the individual.
That’s when you get abusive anti-patterns that incite false urgency or FOMO or shame in order to get that purchase. The confirmshaming anti-pattern in particular reminds me of the “you’re broken” message of the wellness industrial complex. Like this “Nope, I don’t care enough” example:
Confirmshaming is a familiar tactic that fronts as tongue-in-cheek playfulness even as it forces you to say you’re a jerk for not buying the product. (And whattya know, that one’s for a wellness service to boot.)
Every time you design to stir this kind of guilt or false urgency (“75 people are looking at this hotel room”) or interruption (popover ads and notifications) or other unease, you take advantage of the customer. The trouble, of course, is that the tactics work, at least in the short term. If you’re just measuring conversions, you’ll see those metrics rise. But at what cost? Cynical sales tactics sully the very product they aim to sell; they erode trust. How are you measuring that? And how does that result fit with the brand you’re trying to build, or the effect your business is trying to have for your customers?
Simply wild how much in life is fucked solely because someone somewhere would make less money if it were better.
In the wellness world, Liza shows us that the wellness industrial complex deepens jittery unease and dissatisfaction, instead of the calm and peace it promises on the surface. It is harder work to make something that is both profitable and respectful. But you know what? That’s the work.
Writing for Hong Kong’s South China Morning Press, Jack Lau tells the story of Couch to 5K, the running schedule I created in 1996. Get the scoop on how the thing came to be and, um, compare photos of 25-year-old Josh running in 1996 to 49-year-old Josh running now.
I created the schedule well before I became a designer, but looking back, I consider Couch to 5K (C25K) to be my first big UX project. The “brief”: onboard skeptical would-be runners to a regular, sustainable running habit. Twenty-four years later, many millions of runners have used C25K to do exactly that.
The secret behind the nine-week plan is to offer kind encouragement to go (very) slow and advance (very) gently. Many come to C25K after defeating experiences with fitness: pain, self-flagellation, boredom, or “failure.” The program turns that around by delivering attainable victories. As Jack calls out in the article, the process has deeper effects than the merely physical. I’ve received a truly remarkable number of notes from people who tell me that completing the program revealed new confidence in what they might achieve.
“If a depressed 33-year-old with bad knees who hates cardio can do it, enjoy it, and start to thrive, anyone can,” one C25K alum told Jack. And that’s pretty cool.
Fast Company’s Aaron Cohen shares the story of Dotdash, the network formerly known as About.com. Big Medium had a big role in this tale, and it may be the most successful design- and business-turnaround story we’ve ever been involved with.
Three years ago, About.com’s audience and ad revenue were plummeting, and CEO Neil Vogel told us the company was “circling the drain” and needed drastic change. We helped the company develop a new vertical strategy, carving out the content from the main network into branded premium experiences. The new network, Dotdash, relaunched its vast archive of content with a collection of great-looking, fast, and premium websites, powered by a single CMS and a themed design system. Big Medium led the design of three of those early properties—Verywell, The Balance, and The Spruce—and the network has since grown to nearly a dozen.
Maybe you’ve never even heard of Dotdash, but its service content reaches about 90 million Americans a month. … Collectively, Dotdash’s sites have increased traffic by 44% year over year in Q3 2019. Driven by advertising and e-commerce, the company’s annual revenue grew by 44% in 2018 and 34% as reported in Q3 2019 earnings.
A big part of this success boils down to some very intentional design and technology bets that we made together:
Make more money… by showing fewer ads
Create a respectful UX that celebrates content instead of desperate revenue grabs
Create a front-end architecture that is modular and nimble
Make the sites fast
It’s worth noting that all of these choices are counter to what most media companies are doing. Most are pouring on more ads, imposing design that abuses readers and content with popovers etc, slowing their sites with heavy scripts and trackers. No kidding, it was a seriously brave and non-obvious choice to reject those paths. Fast Company describes the impact of Dotdash’s industry-bucking choices:
While other independent media companies were engineering their coverage around social media, video, and trending topics, Dotdash doubled down on text-based articles about enduring topics and avoided cluttering them with ads. … Dotdash sites run fewer ads, with no pop-ups or takeovers, and because the ads are relevant to each article, they perform better. At a time when digital ad rates have continued to crater for most online publishers, Vogel says the company’s ad rates have increased nearly 20 percent each year since 2016, and 25 percent of 2019 revenue came from affiliate marketing fees (bonuses paid to the publisher after Dotdash visitors made purchases via ads on the sites.)
The sites load very quickly, and the company’s proprietary content management system is designed for efficiency: Designers and editors can choose from fast-loading templates that include images, video, and interactive applications. And there’s an emphasis on creating the kinds of detailed, informative articles that turn up in search results. At Verywell, for example, each article is updated at least once every nine months and reviewed by medical professionals.
Dotdash has not only turned itself around, it’s been expanding as other media companies have contracted, selling themselves off piece by piece. Big congrats to our friends at Dotdash: they’ve demonstrated that ad-supported websites can be presented in ways that are both respectful and (very) profitable.
It is pitch black. You are likely to be eaten by a generative text model.
AI Dungeon is a pretty astonishing example of generative fiction—in the form of old-school text adventure game: “Imagine an infinitely generated world that you could explore endlessly, continually finding entirely new content and adventures. What if you could also choose any action you can think of instead of being limited by the imagination of the developers who created the game?”
This experiment by Nick Walton is built on top of OpenAI’s recent GPT–2 model for generating text after being primed with an arbitrary input (seed text to establish context). Unlike many prior language models, this one generates entire coherent paragraphs that feel a world ahead of prior-art attempts that simply string plausible words together.
Walton loaded the model with training data from a collection of text adventures obtained from chooseyourstory.com, and then let ’er rip. “The model at times shows remarkable understanding and writing quality like in this sample where the player tries to find love on a deserted island,” Walton writes:
You head to the island and explore it. You find it completely deserted save for a single tree. You walk up to it and examine it closely. It seems to be made of wood, but its leaves seem to be green rather than normal leafy green. > find love
You take a deep breath and stare intently at the tree. Your eyes meet with the tree’s. You immediately feel a strong sense of euphoria. You smile and touch the tree. Its bark feels warm to the touch. You sniff the tree and find that its scent is very pleasant.
The unfolding adventures are reasonably coherent, too, and generally hang together as you go along. Unlike the virtual assistants many of us use today, this one actually remembers the thing you said to it moments before. Walton says every command is accompanied by the last eight commands and responses, effectively giving the game a short-term memory to maintain context. Even as you throw curve balls at the story engine, it works hard to make it fit what came before. Here’s a snippet from one of the resulting adventures:
"I am a goblin!" you say proudly. "And I'm glad to be one". "Well then, congratulations," says the orc captain. "We'll let you live". You nod and walk away from the camp. > ask if you can join the orc band
The orcs don't want to fight anymore so they're looking for someone to help them out in their new endeavors. They offer you a chance to join up with them as a drummer. It would allow you to play music while helping out around camp. > tell them we should start a musical troupe
They look at you skeptically but agree to give you a try. You tell them about your plans to create a musical troupe and how you plan on using the orcs to help you out. > teach them how to play the flute
You teach them how to play the flute and soon enough they're playing for you all the time. The orcs seem to enjoy it and they begin to think of you as their leader. Soon other orcs come to you asking to join too. > accept them into your musical groupe and train to be the best musicians
You continue to teach them and eventually they become better than you. You learn that you need to practice more and more often until you can master the art of playing the flute. Eventually you find yourself practicing every day and finding it very difficult to sleep. > take orcs to perform at a nearby festival
You take the orcs to perform at a local festival where you get to meet some of the other orcs you've trained. This is followed by another success when the orcs manage to attract the attention of the Emperor himself who grants you permission to use his personal carriage to travel around the Empire.
Why this matters
This is a toy, of course, but it’s a nifty demonstration of how the GPT model can be used on an arbitrary data set to create fairly compelling responses. This stuff remains primitive compared to human prose; AI Dungeon isn’t going to write award-winning fiction, but it does hint at ways that it could help human writers by proposing directional text. In a Hacker News thread, Walton wrote:
This doesn’t reach near the level of good human authors. There’s no long term plot or deep human themes in this. I don’t think this will ever replace quality human writing, but it may be able to augment it in cool ways. I personally would love if rather than every guard in Skyrim telling the exact same story, if each guard could have their own stories or comments generated based on things about their life. Human authors could provide high level details and let AI generators fill in the smaller details.
As with so many instances of machine learning, in other words, the best application here is not to replace human efforts but to augment them. What might be the role for this in supporting common or repetitive writing tasks? In supporting customer-support teams providing tailored responses to frequently asked questions? In giving automated agents better comprehension of the task we want them to accomplish?
There are big differences in the role of a typical product designer and an AI designer. Rather than launching a product feature that shows up in an app in an immediate and obvious way, our output is often clarity for engineers on how the technology could be applied. Because AI capabilities might take 2–3 years to develop, it’s important for designers to help developers understand the potential of different solutions and their impact on people’s lives when developing AI.
Linden details several roles that designers play in shaping AI at Facebook—not just how it’s applied and presented, but how it’s conceived and built:
Designing AI prototypes
Shaping new technology
Developing AI centered products
Collecting data for AI to learn
Designing AI developer tools
We’re in a peculiar moment when many designers have a hard time imagining a role with artificial intelligence and machine learning, because it departs in so many ways from traditional product design. Here’s the thing: design’s superpower is understanding how technology can support human goals and ambitions, how to make technology fit our lives instead of the reverse. Developers and algorithm engineers have shown us what’s possible with AI. Now it’s the designer’s role (and responsibility!) to shape how it’s conceived and presented for meaningful use. That’s why AI and machine learning matter for design teams.
The Wall Street Journal reports that companies are using UK’s omnipresent security cameras as cultural permission to bring facial-recognition tech to semi-public spaces, tracking criminal history but also ethnicity and other personal traits. “Retailers, property firms and casinos are all taking advantage of Britain’s general comfort with surveillance to deploy their own cameras paired with live facial-recognition technology,” writes Parmy Olson for the Journal ($). “Companies are also now using watch lists compiled by vendors that can help recognize flagged people who set foot on company property.” For example:
Some outlets of Budgens, a chain of independently owned
convenience stores, have been using facial-recognition
technology provided by Facewatch Ltd. for more than
a year. Facewatch charges retailers for the use of
a computer and software that can track the demographics
of people entering a store, including their ethnicity,
and screen for a watch list of suspected thieves through
any modern CCTV camera. The system works by sending
an alert to a staff member’s laptop or mobile device
after detecting a face on the watch list. Retailers
then decide how to proceed.
Why this matters
Assumptions about appropriate (or even inevitable) uses of tech become normalized quickly. As constant surveillance becomes the everyday, it’s all too easy to become resigned or indifferent as that surveillance deepens. Once the cultural foundation for a new technology sets, it’s difficult to change the associated expectations and assumptions—or see the status quo as anything other than inevitable, “just the way things work.” We see it in the decades-long expectation that online content is free and ad supported. We see it in the assumption that giving up personal data is just table stakes for using the internet. And now, with surveillance cameras—at least in the UK—we may be settling into a new expectation that simply moving through the world means that we are seen, tracked, monitored in a very granular, personal way.
The Journal suggests that the UK’s “comfort” with surveillance cameras makes it ripe for this. A 2013 survey found that Britain had the highest density of surveillance technology outside of China. Since then, the number of surveillance cameras in the UK has nearly doubled from six million to 10 million—one camera for every seven people.
This anti-theft surveillance affects more than just the guilty. Facial recognition is still pretty iffy in real-world conditions, and the false negatives these systems generate could lead to harassment for no good reason except that you walked into the store.
James Lacey, a staff member at one Budgens store in
Aylesbury, southern England, said the system can ping
his phone between one and 10 times a day. People have
been known to steal large quantities of meat from the
store’s refrigeration aisle when staff members are
in the stock room, he said. The new system has helped,
he said, though about a quarter of alerts are false.
A spokesman for Facewatch said a maximum of 15% of
alerts are false positives, based on its own analysis.
(Related: an ACLU study in 2018 found that Amazon’s facial-recognition service incorrectly matched the faces of 28 members of Congress to criminal mugshots.)
Automated identification has implications beyond crime prevention. What’s OK for these corporate systems to track in the first place? Gender? Race and ethnicity? Income? Browser history? Social relationships? Voting record? Sexual preference? The folks at Facewatch promise vaguely that tracking ethnicity “can help retailers understand their marketplace.” This smacks of a shrugging sensibility that “we can do it, so why wouldn’t we?” And that’s the worst reason to use a technology.
Regulation is evolving, but remains vague and often unenforced. Europe’s well-intentioned privacy regulation, the GDPR, puts facial and other biometric data in a special category that requires a company to have a “substantial public interest” in capturing and storing it. That’s fuzzy enough that companies can arguably allow companies to use the technology to fight crime. Tracking ethnicity to “help retailers understand their marketplace” seems like less of a slam dunk. There is also a gray area around how long businesses can hold on to such footage, or use it for other business purposes.
We should adopt a position on this stuff both culturally and civically. If we don’t, the technology will decide for us. What will your company’s position be? And how about you? What’s your stance as a practitioner designing the technology that will set the behaviors and expectations of the next generation?
Facebook sent employees home for the holidays with robot talking points—in case the family had any questions about, y’know, the company’s cynical, grasping, overreaching, damaging, and irresponsible business model and use of technology. (Bots, it seems, are the only ones left who can deliver these lines with a straight face.) The New York Times reports:
If a relative asked how Facebook handled hate speech, for example, the chatbot — which is a simple piece of software that uses artificial intelligence to carry on a conversation — would instruct the employee to answer with these points:
Facebook consults with experts on the matter.
It has hired more moderators to police its content.
It is working on A.I. to spot hate speech.
Regulation is important for addressing the issue.
It would also suggest citing statistics from a Facebook report about how the company enforces its standards.
Grooming data for the machines has a human cost. The Verge reports that startup Vainu is using prisoners in Finland to tag Finnish-language articles. The company uses Mechanical Turk to do this for other languages, but Finnish-speaking turks are hard to come by. So they get (and pay) prison inmates to do it.
There are legit concerns of exploiting prisoners for low-wage labor, but perhaps a broader concern is that this hints at a bleak future of work in the age of the algorithm. Indeed this “future” is already here for a growing segment of humans—with Mechanical-Turk-level labor turns out to be, literally, prison labor.
This type of job tends to be “rote, menial, and repetitive,”
says Sarah T. Roberts, a professor of information science
at the University of California at Los Angeles who
studies information workers. It does not require building
high level of skill, and if a university researcher
tried to partner with prison laborers in the same way,
“that would not pass an ethics review board for a study.”
While it’s good that the prisoners are being paid a
similar wage as on Mechanical Turk, Roberts points
out that wages on Mechanical Turk are extremely low
anyway. One recent research paper found that workers
made a median wage of $2 an hour.
As we design the future of technology, we also design the future of work. What might we do to improve the quality and pay of labor required to make automated systems work?
Mat Honan’s review of the Google Pixel 3 smartphone is a funny, harrowing, real-talk look at the devices that have come to govern our lives. “We are captives to our phones, they are having a deleterious effect on society, and no one is coming to help us,” he writes. “On the upside, this is a great phone.”
The Buzzfeed review is a world-weary acknowledgement of the downside of our personal technologies—its effect on our relationships, on our privacy, on our peace of mind. He does point out the new “digital wellbeing” features in Android, but offers other alternatives:
Another idea: You may instead choose to buy a device
with a lousy screen and a lousy camera and a terrible
processor. Maybe you would use this less. Or maybe
you should walk to the ocean and throw your phone in
and turn around and never look back**.
**Please do not do this. It would be very bad for the ocean.
Related recommendation for designers and product makers: check out Liza Kindred’s Mindful Technology for strategies and techniques for making products that focus attention instead of distract it.