bandicot

Agents name the control, System One finds it

On an Agent Mac and in the browser, an agent says what to press in its own words; Bandicot finds the control, acts, and shows what changed.

Product

An agent used to need the exact number of a button or field before it could press it. Now it can take a step instead: it says what it wants, such as Save or the search field, with text to type if there is any. When exactly one control has that name, Bandicot presses it straight away, with no model call.

When the name is not exact, System One picks from at most 12 likely controls. Bandicot acts only when System One is at least 70% sure and 15 points surer than its next guess. Otherwise it does nothing and lists the likely controls, so the agent can pick one itself.

A step never presses a control that deletes, buys, sends, posts or signs out unless the agent named that word itself. It never types into a password field, and it never touches a control that is greyed out.

After each step the agent sees what changed: the page or window it is on now and the controls that just appeared, each marked with a star. Screenshots of an Agent Mac box each listed control in colour with its number, so the picture and the list name the same control the same way.