How We Evaluate a New Frontier Model: Gemini 4 Argon as a Case Study

Four line drawings in a row: an open box, a bar chart under a magnifier, a receipt, and a checklist.

Every few months a new frontier model launches, and every launch comes with the same adjectives. Before a new model comes up in a client conversation, we put it through four questions: what actually shipped, what the best independent test shows, what it will really cost, and what we would test before committing.

Google released Gemini 4 Argon on October 13, and it makes a useful worked example. This time the most telling evidence comes from a different question than usual: does the model finish the job it was given?

Step 1: What actually shipped

Google is pitching Argon at long, multi-step work in coding, finance, law, cybersecurity and science, where the model has to hold a plan across many steps rather than answer one prompt well. For developers, the two concrete changes are length and control. With background execution, a single response can now run to about 1 million tokens (less whatever you sent in), where earlier models stopped at 64K. You can also set how hard it thinks: low, medium (the default) or high.

It is widely available from day one: the Gemini app and AI Mode in Search for AI Pro and Ultra subscribers, the paid Gemini API and AI Studio for developers, Gemini for business, and Google Antigravity for Ultra subscribers. Google is also giving scientists six months of its top Antigravity tier for free, and has opened a “Fairwind” program for cybersecurity defenders.

The most interesting numbers are Google’s own. More than 100,000 people across Alphabet use Argon inside an internal version of Antigravity each month. Google says agentic code submissions are up more than 35% and code rollbacks are down by nearly a third. Chrome engineers pointed it at a 6-million-line legacy folder. On YouTube, it traced production profiles and found that redundant emoji rendering alone was eating nearly 45% of a comments server’s overhead.

Internal numbers are always the friendliest ones a company has. Anyone who has profiled a production system will believe the emoji one, though. The expensive waste is usually something boring that nobody thought to measure.

Step 2: Find the best independent test

Box has the most useful outside evidence so far. CEO Aaron Levie reports that on Box’s AI Complex Work eval, a set of hard document tasks modelled on real knowledge work, Argon scored 14 percentage points higher than Gemini 3.1 Pro: 85% against 71%.

IndustryGain over Gemini 3.1 ProWhat Argon got right
Technology+37 pointsPulled the right month’s user, retention and per-user figures. Gemini 3.1 Pro at one point reported retention above 100%.
Consumer products+33 pointsSaw that existing inventory could cover a capacity gap, which made the smaller expansion viable.
Healthcare+25 pointsBoth models caught a dangerous potassium level and a drug interaction. Only Argon took the drugs out of the final plan.
Finance+18 pointsBuilt the projected statements line by line, exactly as the brief asked, through to the funding gap.
Energy+16 pointsNoticed that a data centre’s smart meter log stops in November. Gemini 3.1 Pro often called the logs complete.

Almost every win in that table comes from the same place: the model carried a finding through to the end. Healthcare shows it most clearly. Both models got the diagnosis right, but only one wrote a plan that matched its own diagnosis.

The energy case is the one we would test first in any deployment. Noticing that a meter log stops in November is basic data-quality work, and production pipelines, human and automated, miss that kind of gap all the time.

For years these models were judged on how well they answered. Box’s results suggest the better test now is whether they finish, and that is the test that matters once an agent is doing real work.

Step 3: Read the fine print

The early hands-on commentary raises three things worth checking.

The price per token is going up, and the model is wordy. Jason Lovell, an AI strategist who had early access, pointed out what most launch threads missed. On the paid API, output costs $10 per million tokens until November 30 and $20 from December 1, and input goes from $2 to $4. Artificial Analysis found that Argon generated 110 million tokens running its Intelligence Index, against a median of 85 million, and called it “somewhat verbose”. Put together, those long outputs get noticeably more expensive in December.

The figure to watch is cost per finished task. If the extra words buy fewer retries on long jobs, the numbers can still work, but you have to measure that rather than assume it.

Benchmark gains shrink in real use. Commenters under Levie’s post kept asking whether the 14-point lift holds up against messy permissions, inconsistent data and real workflows. Several also asked something Box hasn’t answered yet: does “works harder to finish” mean a bigger token bill per task?

Checking the work still takes time. As one founder put it, a finished document can still take a while to check: the sources, the intermediate steps, all of it. When the model completes more of the task, checking becomes the person’s main job, and most teams haven’t set themselves up for that yet.

Step 4: Decide what to test before you commit

If you build with these models, this is what we would do now:

  1. Test your own tasks before December 1. The introductory price is a cheap window to measure the cost as well as the capability. Run your longest, messiest real jobs now and track cost per completed task, retries included.
  2. Test the boring failures first. Skip the flashy demos and ask the meter-log kind of question: does the model notice when your data stops, contradicts itself, or disagrees with its own earlier conclusion? A model that finishes without that discipline just hands you a confident, incomplete answer.
  3. Budget for checking, as well as for the model. If the model finishes more of the task, your process needs a deliberate review step, done by someone who knows what good looks like. That is where most of the remaining risk sits.

If you are still deciding which tasks belong with AI at all, start with the Dimensionality Test.

Our read so far: Argon is strong at carrying work through to the end, early enterprise results back that up, and the open questions are cost and checking rather than capability. We will run the same four steps when the next frontier model ships and add what we find to this page. If you would like help running them on your own workflows, that is what we do.

Sources