Anticipate and Acknowledge: Intelligent Agents

Merlin, writing about Greasemonkey, says:

The implications of things like Greasemonkey and PithHelmet catching on seem far-reaching. Think about the benefits of taking web standards to the next level and making sites that can anticipate and acknowledge your visitor’s preferences from their first visit (via standard DIV names or calls to your public “preferences” file). I wouldn’t begin to know how to make this stuff, but I can definitely see myself becoming a grateful consumer.

Which sounds great – until you start to think about the possible down sides. (Admittedly, Greasemonkey probably isn’t attempting to be as tricky as “intelligent agents” but it was a nice jumping off point for this post).

What Merlin’s talking about is similar to what academics (and other people) are calling “intelligent agents”. The idea is that a piece of software called an agent learns your preferences (or gets told about them) and then it goes off and acts on your behalf. Oft-cited examples are email-filtering, searching, gathering news headlines and the like.

One example that I recall in enough detail to recount has to do with mobile phones. Suppose your mobile has an “intelligent agent” running inside it. The agent attempts to learn your calling behaviours and then, once a certain point is reached, it acts autonomously on your behalf. Say you always send calls from Bob through to voicemail. After you’ve done that enough times, the idea is that the agent will automagically route Bob to voicemail without any intervention on your behalf. Sounds great, right?

Now, imagine the down sides. Suppose that the first few times your agent notices Bob being routed to voicemail, it’s because you’re driving, in a conversation or otherwise not in a place to chat with Bob right at that moment. However, in all other circumstances, you’d be quite happy to chat to Bob. The correct rule is “divert Bob to voicemail unless I am able to chat with Bob” but the incorrect, but learned, rule is “always divert Bob to voicemail”. It’s not really that much of a problem unless Bob begins to wonder why you’re always kicking him to voicemail instead of answering his calls.

The problem with the “intelligent” agent is that I don’t believe such an agent can ever gather enough information to truly make informed, intelligent, decisions. There will always be a case where the agent makes the wrong decision and all that mitigates against the wrong decision being a disaster is the amount of power that has been delegated to the agent. People are able to make such decisions because they have access to all their relevant context, all the time. Agents live in a necessarily impoverished world. By delegating decisions to an agent who does not have access to all the relevant context, it becomes more likely that the agent will make a bad decision. Because agents aren’t really all that intelligent, they don’t even know that they don’t have all the relevant information, so they just keep on making bad decisions – like the broom and buckets in The Sorcerers Apprentice.

A lot of this comes back to Lucy Suchman’s book, Plans and Situated Actions, though that’s a bit heavyweight for this post.