bandicot

Checks that test your models and agents, and a passkey you pick

Checks can now ask a model or a whole agent a question and grade the answer, and passkey sign-in lets you pick a passkey from your Mac.

Engine

A list of questions in bandicot.evals.json is now a check. Each one asks either a model directly or a full agent with its tools, then grades the answer: words it must contain, words it must not, files it must leave behind and how long it may take. Run them with bandicot evals, or name the file in a check and Versions runs them before every send.

Checks can run on a model on your own Mac, so testing every flow costs nothing and nothing leaves the machine.

Signing in with a passkey no longer asks for your email first. Press Choose a passkey and your Mac shows every passkey it holds for Bandicot; pick one and you are in. Use an email instead is still there when you want it.