Clippy’s Glow-Up

Recently, Niantic (of Pokémon GO fame) released a short film in which they lay out their “vision of the future where AI and AR come together to enrich and enliven the real world”. It features a writer interacting with multiple animated augmented reality “agents” to help her perform various creative tasks, monitor her physical environment, check in on her stress levels, and even match her on a date. On the surface, it’s compelling and well-executed, but to me, it’s just that–surface. It’s a superficial representation of a new app economy user experience powered by AR and underpinned by artificial intelligence. Each of the agents is likely to be equivalent to an app, with its own enablement, AI models, interfaces, security, business model, etc. In that respect, it’s not especially new–particularly in light of the deluge of generative AI and LLM-powered services and solutions that have flooded the market in 2023. What is new is that these agents are embodied and adorable. And maybe that’s not such a good thing. Do we want a more fully actualized version of Clippy?

In a follow-up blog post Niantic caveats the scenario presented as speculative fiction. Specifically, according to Niantic CEO John Hanke, their vision is “a world where humans leverage AI to be better humans, where the information we share and the help we get remains firmly in our control.” Beyond the AR UX depicted in the film as several anthropomorphized characters, the underlying assumption of their speculation is that we will arrive at a point where we have sufficiently ubiquitous AR hardware (‘normal’ looking eyeglasses in this case) and an ecosystem of ring-fenced data feeding personalized applications that can ‘talk’ to each other. Ultimately, it’s a vision for personalization that overlaps with what could be an episode of Black Mirror in a Venn diagram of cute and creepy. And it’s a personalization trope with which I am quite familiar, having explored it more than once in the past 20-plus years.

Back in 1999, I was on a team that assembled a concept pitch for a U.S. telco that included a detailed scenario in which users would receive an SMS message when in proximity to various locations and items that had information personalized for them based on stated preferences. As with Maya in the Niantic film, the users we portrayed would be going about their everyday lives, and ‘useful’ text alerts would be delivered to the little green screen on the dumb little device in their pocket. Obviously, our vision was at least 25 years premature but the overall goal hasn’t changed materially in the decades since. I know this because several years ago, I was involved with a large urban development project that lent itself to a similar vision, albeit one that could realistically be brought to fruition in the coming years.

There’s a city being built in Queensland, Australia on a vast empty space in the middle of a thriving and vibrant region. They’ve been able to build not just from the ground up, but they’ve extended their infrastructure below ground to include subsea cables and subterranean fiber optic conduits that tie into a high-speed data ecosystem and connect to fixed physical assets like light posts, waste bins, public furniture, and smart displays above ground. This development represents a unique opportunity to revisit the vision that the team and I pitched to the U.S. telco and reframe it in the context of how far connectivity, hardware, and software have advanced all these years later.

This city, with its lack of legacy infrastructure and the support of the local government and its constituents, is an outlier but it’s also a beacon for how other cities and communities will evolve over time. A personalized experience of applications that are aware of their environment and can combine individual data with public data in a user-friendly and near frictionless interface remains the holy grail. This, I think, is what Niantic is getting at. But they clearly have a user interface bias, which is pretty evident in their film.

Given the advancements in computing and where consumer hardware (Vision Pro anyone?) is heading, a user’s physical surroundings actually hardly matter provided they have a robust data connection. Whether it’s Niantic’s cute little animated AR Agents, a Her-esque hardware device in your pocket like what Humane is working on, or another design-forward physical manifestation of some description that’s likely to emerge from the recently announced collaboration between OpenAI and LoveFrom, we’re still fundamentally talking about personalization realized in some secure and tangible form factor.

In a Fast Company article entitled “AI companions are about to be absolutely everywhere,” Mark Wilson highlights this issue as well. Rather than focusing on the interface, Wilson writes about the promise of AI-powered assistive technology while not making assumptions or judgments about how we might experience or access them in the future. He refers to the current incarnation of mostly text and voice interfaces as a “mental exoskeleton” that already gives us access to a superpower without new hardware. Amazon talks about this in the context of “ambient computing”, Apple’s term is “special computing”. Meta and Google both have their own particular angles as well. I’d rather not let consumer devices drive how we utilize our own data but It’s inevitable that some hardware innovation is likely to transform how we interact with computing resources in general. And I have to admit that I’m both curious and excited about what new paradigms are be on the horizon.

We’ve come pretty far since 1999 in imagining what interfaces are likely to take hold amongst a significant proportion of the human population. But we still have some ways to go. We should absolutely keep imagining, building, and testing new gadgets and interfaces but, in the meantime, we also need to focus on the intangibles including security, privacy, and accessibility. And also humanity! The Niantic character-cum-app’s cuteness notwithstanding, we must imagine our future of interacting with whatever user interface emerges as one that prioritizes agency and eliminates emotional manipulation to ensure that we remain firmly in control of our own data as well as our own decisions.