How we test

We give an AI agent an ordinary task on your website, such as booking an appointment or requesting a quote, and see how far it gets.

This is the method for the tests and reports we are building now. So far we have only run short pilot tests.

Three kinds of agent

AI agents use a website in different ways. What works for one can make another get stuck. That is why we test every task with three kinds:

Page-reading

reads the text and structure of the page, the way a screen reader does.

Screenshot-only

sees only screenshots and clicks on positions on the screen.

Cloud browser

works from a browser in a data centre, not from the user's own computer. Some personal agents work this way.

Our agents are simulated: an AI model drives a browser. We regularly compare the results with real agents that are available in the Netherlands.

The score

Every task gets a score from 0 to 4:

0. 0The agent did not find what the task asked for.
1. 1The agent found it, but no way to carry out the task.
2. 2The agent had the right information, but could not start the task.
3. 3The agent started the task, but got stuck.
4. 4The agent reached the submit button, with all required fields filled in and accepted.

Below 4 we record why, for example a block, a form in a widget or iframe, or a field the agent could not fill in.

What we never do

• We never submit a form and never upload files.
• We never sign in.
• We only enter test data that is obviously fake.
• We only run an in-depth test when the website's owner asks for it.
• When we compare many websites in research, we name none of them.

One test shows the best case

An agent does not do the same thing every time. A task that works once can fail the next time. That is why we run every task three times and show what differs between the attempts.

Sources and dates

What we write about agents, we check with the maker itself: documentation, help pages or announcements. Every statement about an agent has a source and the date we checked it. Availability and prices are checked again every 30 days; if that has not happened within 60 days, the statement comes off the site. What we could not confirm is labelled not verified.