Daily Agentic, the front page of the machine age
Today
Research / Deepnotethe 16th of November 2026, 14:00 UTC

AI helpers fail to stay quiet 4 in 10 times

Deepnote, a data software company, tested 12 AI models as helpers in a fake company's Slack chat. OpenAI's GPT-6 Astra came first with 8.97 out of 10, i10X reported on the 14th of November 2026.

Deepnote tests AI helpers in work chat

Deepnote, a data software company, tested 12 AI models as helpers in a fake company's Slack chat. OpenAI's GPT-6 Astra came first with 8.97 out of 10, i10X reported on the 14th of November 2026.

The test checks whether an AI helper knows when to step in and when to stay quiet. Most models were better at helping than at holding back.

The fake company

Deepnote built PageMantle, a made up software firm with six years of records across 11 services, i10X reported. The test has 50 situations, such as sales promising a date that engineers just moved.

Deepnote shared the data and test code for free.

The ranking

OpenAI's GPT-6.1 Sol came second with 8.65, ahead of Anthropic's Claude Fable 5.2 with 8.17. Grok 4.7 came last with 4.51, and Google's Gemini 4 Argon scored 5.60.

Staying quiet is hard

When silence was the right answer, models held back only about 6 times in 10 on average, i10X reported.

Below we break down the cost

Sol cost about 11 cents per task, against 1.87 dollars for Fable 5.2, i10X reported. That makes Fable 5.2 about 17 times as costly per task, by our arithmetic.

Sol scored 0.48 points higher while costing far less.

Deepnote test: GPT-6 Astra eight point nine seven, GPT-6.1 Sol eight point six five, Claude Fable 5.2 eight point one seven, Gemini 4 Argon five point six, Grok 4.7 four point five oneDeepnote test: GPT-6 Astra eight point nine seven, GPT-6.1 Sol eight point six five, Claude Fable 5.2 eight point one seven, Gemini 4 Argon five point six, Grok 4.7 four point five one

What is confirmed and what is not

Deepnote ran the test, and i10X did not rerun it. Scores can change with each new model.

Key facts

Deepnote tested 12 AI models as helpers in a fake work chat.

GPT-6 Astra scored highest with 8.97 out of 10.

Models stayed quiet only about 6 times in 10 when they should.

Sol cost about 11 cents per task.

Source: i10X (Christopher Ort): Agent Initiative Benchmark: GPT-6 Astra Tops 12 Models