Agents are clearly getting better and better at writing code and at knowledge work in general. We are moving up the adoption curve; the conversation has moved from "are LLMs good enough to do task X?" to "what is the right UX for managing this agent?". In other words, the agent is no longer the bottleneck - the human is.
tl;dr: if you want to use AI to get work done, then you will most likely be spending your time in a ChatGPT-style conversation interface. But when AI gets really really good, you don't really want to be spending 8 hours a day in a chat window. There are better ways to design our interfaces so that you can get more done and have more fun while doing so.
Even if agents are getting better at completing tasks and overall productivity is (hypothetically) going up, users still have the same amount of energy and time available to spend. Consider the example of LLMs in journalism: a human-in-the-loop has a fixed budget by which they can review LLM outputs (unless they relax their standards, which is not a good idea). Agentic tools therefore need to have UX optimised so that this budget is spent effectively.
We can think about designing UX so as to speed up the loop, but it's not purely a game of quantity. Consider reviewing LLM-generated software - it is tiring to read code. It's no surprise therefore that over the last week or two I've seen a range of UX which go beyond text-based interactions:
- Clawdbot reaches out via WhatsApp (on Meta Ray-Bans no less) for executive approval.
- Claude gives you a phone call when it's done; you can respond by voice (which Simon Willison was doing way back in September 2024). Sometimes you just want to leave your agent a voice note!
- Multiple examples of deliberately making the interface more game-like, either because games provably work for resource management, or to make it more fun (which is a valid rationale for improving UX. In fact, sometimes users just want to "vibe it out"; having an agent communicate in image format also creates "fun vibes").
A UX pattern which exemplifies this central idea of 'making it as easy as possible for the user to provide feedback' would be where the agent provides an interactive prototype and observes how the user 'plays' with it, deducing feedback from this interaction. Or we could jump straight to having other AI doing the playing (which begs the question of how to design prototypes that are optimal for agents to interact with) if it's appropriate for the human to move up a level into a project management role.
The argument for better UX is even more obvious when considering the shift towards multi-agent orchestration, where a 'traditional review' of multiple outputs is highly impractical (due to e.g. context switching). We're seeing multi-agent monitoring systems emerge, such as:
- Agents monitored via a kanban board (which is arguably a visual overview of project status) reach out if blocked. Another example here.
- Gas Town using the 'high-level overseer' abstraction - i.e. "The Mayor: the main agent you talk to most of the time" - which mitigates the burden on the user to "actively prompt and manage" every subagent.
- Giving agents a 3D embodiment within a virtual world.
In summary, the main features being explored by these UX patterns include:
- Integrating with more natural channels (e.g. WhatsApp)
- Calling the user's attention only when required (e.g. if blocked)
- Providing a visual overview of status (e.g. kanban)
- Allowing for implicit feedback (e.g. via interaction instead of conversational feedback)
Of course, interface UX is tightly coupled with agentic capabilities - both of which will continue to evolve rapidly. It's going to be a valuable space to watch, not just because it could enable more knowledge work to be completed in a more fun way, but because it should also drive overall quality. I'll continue to monitor the situation!