I’ve been thinking about what good AI is supposed to do in the world, and what that means for the work I spend my time on.
The technology itself is a means to an end. We do not build AI for the sake of doing things with AI. Making the world better is the reason to build it, but that can become abstract. What would better feel like in an ordinary person’s day, including someone who has no interest in AI?
Dozens of companies out of SF have launched some form of the personal assistant in an attempt to do this. They offer to manage your inbox, schedule meetings, and take care of small jobs on your behalf. A morning briefing is useful, and so is help with emails, but I don’t have that many personal tasks I need done every day.
So much of our lives involves dealing with institutions. We pay rent, get our electricity and internet from a provider, buy groceries, and order things that someone else has to deliver. Even when we’re doing something for ourselves, getting it done often depends on one of them. We can submit everything we’ve been asked for and still be waiting for a decision, a payment, or someone to get back to us. In the meantime, we have to keep track of it all alongside the rest of our lives.
A personal agent could do quite a lot to help me with those things. It could figure out which documents I need, find another flight, or keep chasing a claim long after I would have given up. But it eventually runs into the same dependency I do: the organization on the other side has to do something.
For those of us building agents, or deciding what to deploy inside an organization, there’s a lot to work on here. We can build something that answers a question, but the person asking usually needs something to happen afterward. How far could we help them get?
Genuine care at scale feels like a worthwhile ambition for that work. To make it useful, though, we need to get specific about what caring for someone would require.
Consider a student asking a university whether a document has arrived. An agent could check and give an accurate answer. That would be helpful. But suppose the student has already submitted it to another office, their financial aid is waiting on it, and enrollment closes on Friday. They could receive a perfectly accurate answer and still lose their place.
A customer journey includes everything that happens between the interactions an organization records. A call ends, but the student still has to work out whom to contact next. They explain the situation again, forward an email, and wait. Each office may have completed its part to perfection while the student remains responsible for getting the parts to connect.
That is a lot of the friction we experience in dealing with institutions.
Hospitality offers a useful counterexample. In Unreasonable Hospitality, Will Guidara writes:
“Fads fade and cycle, but the human desire to be taken care of never goes away.”
In a restaurant, the guest generally doesn’t coordinate the host, server, and kitchen. The restaurant takes responsibility for making their work come together. And if a guest needs to leave in forty minutes to catch a flight, good hospitality means adapting to that circumstance. Perhaps the server recommends something the kitchen can prepare quickly and arranges for the bill to arrive with the food. Knowing about the flight changes what everyone does.
There is practical work underneath that experience. Information has to reach the right people at the right time. They need enough discretion to respond. Someone has to notice whether the plan is working.
The challenge is providing that same attention to everyone an organization serves.
Historically, that has been expensive. An employee can only follow so many situations at once, especially when each one involves other teams and information scattered across systems. Standard processes help manage the volume. Maintaining attention, and providing genuine care when something falls outside those processes is next to impossible.
For the first time in history, much of that work is possible with AI, but organizations have to build for it.
For our student, the agent would need to establish what is holding up the application and whether the document already exists elsewhere. With the appropriate permission, it could get that document to financial aid. Then it would need to check that the office accepted it and that the application could proceed.
Genuine care means the institution takes responsibility for proactively driving the customer’s journey.
What happens if your customer stops following up? Are you still working to get them what they need? Does someone notice when things get stuck? Or does everything quietly stop until they ask again?
There may also be a growing business reason to take genuine care seriously. In his tweet about personal agents, Patrick Collison, CEO and co-founder of Stripe, describes an imbalance of knowledge: a company can spend decades learning and optimizing how to sell to us, while we have limited time to compare prices, read the terms, or work out whether something is worth buying.
Assorted things I've been mulling over regarding agents in the economy:
— Patrick Collison (@patrickc) October 9, 2026
• Companies are in some sense the original superintelligence. It's often the case that they can use their amortized cognitive surplus to advance their interests against those of certain consumers: relative…
A personal agent working in our interests could do that research for us. It could find a cheaper policy, spot an expensive add-on, or remind us to cancel a subscription we no longer use. Revenue that depends on people being busy, confused, or forgetful could become harder to keep.
Patrick’s argument is that this could reward companies for making better products. I think the experience of dealing with a company belongs in that evaluation too. A company may have to give up revenue from customer mistakes and become better at earning it through useful products and relationships people want to keep.
Personal agents could make individuals better equipped to pursue what they need. Agents inside institutions could make those institutions better able to respond to those individual needs.
Without that second development, we may just create more persistent requests for already overwhelmed employees.
For the people these institutions serve, the result may be almost unnoticeable. A document reaches the right office before a deadline. A change to a trip gets coordinated without an afternoon of phone calls. A claim moves along without its owner having to keep checking on it.
They may never know that AI was involved, or how much work they would otherwise have had to do.
They got what they needed and went on with their day. That’s the whole point.