Gemini 4 Hydrogen vs Claude Sonnet 5.5 vs GPT-6.1 Terra
13 min read

Gemini 4 Hydrogen is not ahead everywhere. It is first in 7 of the 10 rows of Google's own table, but a closer look at that table and at the independent measurements shows this:
- On Terminal-Bench 4.1, Claude Sonnet 5.5 scores 63.9% and Hydrogen 57.0%.
- On OSWorld-2.1, which measures computer use, Sonnet 5.5 scores 83.9% and Hydrogen 77.6%.
- In Artificial Analysis's own Terminal-Bench run, Hydrogen (about 54%) also trails Sonnet 5.5 (64%).
- On the Artificial Analysis index, the order depends on the thinking level: Hydrogen scores 51 at its default setting, Sonnet 5.5 scores 55 at its highest.
- I could not find an independent index row for GPT-6.1 Terra; every number for it here comes from Google's table.
For the model itself, start with What Is Gemini 4 Hydrogen?. Free use and access are in a separate post, and the rival's side is in the Claude Sonnet 5.5 post.
This post is a news roundup and compilation; I have not tested the model, and I have not run the three models side by side on the same tasks. The numbers come from Google's November 12, 2026 announcement, the Google DeepMind model page, Artificial Analysis and news sites. Sources are at the end.
Side by side: Google's own comparison
Google compares Hydrogen with Claude Sonnet 5.5 and GPT-6.1 Terra. The table in the announcement is an image; I took the values below from the text table on the Google DeepMind model page (published with the note "as of November 2026").
| Benchmark | Hydrogen | Claude Sonnet 5.5 | GPT-6.1 Terra |
|---|---|---|---|
| SWE-sweep (find and repair bugs without breaking existing code) | 13.5% | 3.1% | 5.8% |
| Terminal-Bench 4.1 (cheating-adjusted) | 57.0% | 63.9% | not given |
| Agents' Last Exam v1.1 (end-to-end professional workflows) | 71.6% | 60.4% | 69.8% |
| AutomationBench-AA | 76.1% | 71.8% | 60.2% |
| Vending-Bench 2 (bank balance) | $14,549 | $11,165 | $9,107 |
| OSWorld-2.1 (offline subset, computer use) | 77.6% | 83.9% | 74.2% |
| MKA (Multimodal Knowledge Agent) | 53.4% | 52.0% | 43.8% |
| Chartography | 66.6% | 61.6% | 63.2% |
| Terminal-Bench-Science 0.1 | 51.4% | 59.9% | 58.3% |
| HLE-Diamond | 58.8% | 53.3% | 47.3% |
Hydrogen leads 7 of the 10 rows. Sonnet 5.5 leads three: Terminal-Bench 4.1, OSWorld-2.1 and Terminal-Bench-Science. Terra is first in none; it is ahead of Hydrogen on Terminal-Bench-Science and within two points of it on Agents' Last Exam.
Four things to keep in mind when reading the table:
- The vendor chose the benchmarks. This is the table Google prepared to introduce Hydrogen. It is good that the three rows where it trails are included, but Google decided which benchmarks made the table.
- Most rival numbers are not Google's own runs. According to Google's evaluation methodology document, Hydrogen's results are single-attempt (pass@1) scores obtained through the Gemini API, at the highest thinking setting unless stated otherwise. The numbers for non-Google models are, unless stated otherwise, the vendors' self-reported values; for Sonnet 5.5 and Terra Google uses the result at the maximum thinking setting, or the best available result where that is missing.
- Row-by-row sourcing. On SWE-sweep, Terminal-Bench 4.1 and Terminal-Bench-Science, Google ran Hydrogen itself and took the rivals' numbers from the official leaderboards. AutomationBench-AA and HLE-Diamond come from the Artificial Analysis leaderboard, Vending-Bench 2 from the Andon Labs leaderboard. On Agents' Last Exam Google ran Hydrogen and Sonnet 5.5 itself and took Terra from the leaderboard. On Chartography Google ran Hydrogen and Terra itself and took Sonnet 5.5 from its system card. On MKA Hydrogen is Google's run and the others come from Artificial Analysis.
- In two rows the conditions are not equal. On Terminal-Bench 4.1 Hydrogen was run at
high(not at xhigh, its highest level), averaged over five attempts per task. On OSWorld-2.1 the best of three runs was taken for Hydrogen and Sonnet 5.5, while Terra's value comes from its system card. These are two of the rows Sonnet 5.5 leads. As you will see below, the thinking level changes the result a lot. - The Terminal-Bench version is different. This table uses Terminal-Bench 4.1 (cheating-adjusted). The table in the Carbon comparison used Terminal-Bench 4.0 (Carbon 65.2%, Argon 57.1%). Do not put numbers from the two versions in one column.
- Everyone is low on SWE-sweep. Hydrogen's 13.5% is several times its rivals' scores, but in absolute terms none of the three models solves most of the tasks.
Independent measurements: the thinking level changes everything
Artificial Analysis measures models separately at different thinking levels. The values on the Intelligence Index v5.0.1 page (checked November 14, 2026):
| Model (thinking level) | Index | Cost per index task | Time per task |
|---|---|---|---|
| Hydrogen xhigh | 54 | $3.16 | 14.1 min |
| Hydrogen high (default) | 51 | $1.87 | 7.8 min |
| Hydrogen medium | 41 | $0.80 | 3.7 min |
| Hydrogen low | 27 | $0.29 | 1.3 min |
| Claude Sonnet 5.5 max | 55 | $10.02 | 34.0 min |
| Claude Sonnet 5.5 xhigh | 50 | $4.09 | 14.6 min |
| Claude Sonnet 5.5 high | 42 | $1.64 | 6.8 min |
| GPT-6.1 Sol max | 55 | $1.27 | n/a |
What this table shows:
- At the same level, Hydrogen is ahead. At high, Hydrogen scores 51 and Sonnet 5.5 scores 42. Their costs per task are close ($1.87 and $1.64).
- At the top setting, Sonnet 5.5 is one point ahead. Sonnet 5.5 max scores 55, Hydrogen xhigh 54. But Sonnet gets there at $10.02 per task, roughly three times Hydrogen's $3.16.
- Similar score, different bill. Hydrogen high (51) and Sonnet 5.5 xhigh (50) are almost level. Hydrogen does it at less than half the cost and in about half the time.
- Lowering Hydrogen's default is costly in quality. Going from high to medium takes the score from 51 to 41, and low drops it to 27. If you lower the level to save money, you are effectively using a different model.
- There is no Terra row. I could not find a row for GPT-6.1 Terra on Artificial Analysis. The OpenAI model in this table is GPT-6.1 Sol; the one in Google's table is Terra. They are different models, so do not substitute one's number for the other's.
For terminal work, the independent data points the same way as Google's table. As DataCamp reports, in Artificial Analysis's own Terminal-Bench 4.0 run Hydrogen scores about 54% (BenchLM lists 53.5%), Claude Sonnet 5.5 64% and GPT-6.1 Sol 56%. DataCamp's conclusion is that Hydrogen is "not the best model available for terminal-heavy work."
Where does Hydrogen sit inside the Gemini 4 family?
On the same Artificial Analysis page, at the high level, Gemini 4 Carbon scores 60, Hydrogen 51 and Gemini 4 Argon 50.
| Model (high) | Index | Cost per index task | Time per task |
|---|---|---|---|
| Gemini 4 Carbon | 60 | $5.48 | 29.0 min |
| Gemini 4 Hydrogen | 51 | $1.87 | 7.8 min |
| Gemini 4 Argon | 50 | $4.97 | 30.0 min |
So Hydrogen is in the same score band as Argon, but clearly cheaper and faster per task. Carbon is nine points ahead and asks roughly three times the cost per task in return. (The per-task costs relayed by NeuralTrust in my Carbon comparison were higher; the ones here are the values on the index's current v5.0.1 page.)
Google's own tables show a similar picture, but the Hydrogen and Carbon tables are separate tables; read each one on its own. On Agents' Last Exam v1.1, Carbon scores 77.4%, Hydrogen 71.6% and Argon 69.0%. On OSWorld-2.1, Carbon 81.5%, Hydrogen 77.6%, Argon 74.7%. On Chartography, Hydrogen (66.6%) is behind both Carbon (72.5%) and Argon (71.6%). The one reverse example is SWE-sweep: Hydrogen 13.5%, Carbon 9.6%. Carbon's comparison with its own rivals is in a separate post.
Hallucination, speed and verbosity
Hallucination rate. According to the AA-Omniscience values BenchLM gives, citing Artificial Analysis, Hydrogen's hallucination rate is 8.5% and its accuracy 50.5% (DataCamp writes the rate as "8%"). This rate is how often the model gives a wrong answer instead of saying it does not know; lower is better. Next to the values in my earlier posts, Hydrogen falls between Argon (4%) and Carbon (11%). I have no value from this benchmark for Sonnet 5.5 or Terra. Google, for its part, says fabricated claims were cut by more than 5x compared with Gemini 3.8 Flash; that is Google's own measurement, and the comparison is with the older Flash model, not with the rivals.
Speed. The Artificial Analysis model page shows 182 output tokens per second at xhigh. Time to the first answer token is 42.9 seconds at xhigh and 16.3 seconds at high. The page describes the model as slower than average per task. Google's claim that it finishes tasks 2.5x faster than Gemini 3.8 Flash at the default high setting is its own measurement, and the method has not been published.
Verbosity. Hydrogen spent 304 million output tokens to complete the index at xhigh; the median is 77.3 million. Artificial Analysis calls this "very verbose." The price per token is low, but the number of tokens spent is high; that is the reason to look at the cost-per-task table.
Price
| Model | Input (1M tokens) | Output (1M tokens) | Source |
|---|---|---|---|
| Gemini 4 Hydrogen (standard) | $1.50 | $7.50 | Google pricing page |
| Gemini 4 Hydrogen (Batch / Flex) | $0.75 | $3.75 | Google pricing page |
| Claude Sonnet 5.5 | $2 | $10 | Third-party list |
| GPT-6.1 Terra | $1 | $5 | Third-party list |
| GPT-6.1 Sol | $2 | $10 | Third-party list |
| Gemini 4 Argon and Carbon | $2 ($4 from December 1) | $10 ($20 from December 1) | Google pricing page |
I checked the Hydrogen prices on Google's pricing page on November 14, 2026. Hydrogen also has a rate-limited free API tier; Argon and Carbon do not. The Claude and GPT prices come from third-party lists and I have not verified them on the vendors' own pages; check the official page before you decide.
By list price Terra is the cheapest, then Hydrogen. But Hydrogen's thinking tokens are billed at the output price, and models spend different numbers of tokens on the same job. So the list price alone is not enough; the cost per task above is a more realistic measure.
Which one, and when?
This part is my interpretation: general conclusions I draw from the numbers above.
- Starting for free, or a small budget: Hydrogen. It has a free API tier and a low cost per task at the default setting.
- Agents that work in a terminal: Sonnet 5.5. Google's table and Artificial Analysis's run say the same thing.
- Computer use (OSWorld): Sonnet 5.5.
- End-to-end workflows and automation: Hydrogen leads in Google's table (Agents' Last Exam, AutomationBench-AA), but I have not seen independent confirmation for these rows.
- Charts and multimodal questions: Hydrogen by a small margin (Chartography, MKA).
- If you need the highest score: look one class up instead of at these three. There is one point between Sonnet 5.5 max and Hydrogen xhigh; Carbon is at 60.
- GPT-6.1 Terra: the lowest list price, and close to Hydrogen on Agents' Last Exam. Because I found no independent index data, I cannot say more.
How to read benchmarks
- A vendor's table is the vendor's choice of benchmarks. Speed, verbosity and hallucination rate are not in Google's table; they are on third-party pages.
- Ask for the thinking level. "Hydrogen scored 51" only means something once you add "at high." Check whether the two models being compared are at the same level.
- The agent harness changes the result. The same model scores differently with a different set of tools. I explained this in the post on what a harness is.
- Measure your own task. Five of your own tasks teach you more than fifty of someone else's.
To see where the models one class up stand on the same kind of benchmarks, see the Claude Fable 5.2 comparison.
Frequently Asked Questions
Is Gemini 4 Hydrogen better than Claude Sonnet 5.5?
It depends on the kind of work. Hydrogen leads 7 of 10 rows in Google's table; Sonnet 5.5 leads terminal work and computer use. On the Artificial Analysis index, Hydrogen is ahead at the same thinking level, and Sonnet 5.5 is one point ahead at its highest setting.
Is Hydrogen better than GPT-6.1 Terra?
In Google's table Hydrogen leads eight of the nine rows that have a Terra value; Terra is above Hydrogen only on Terminal-Bench-Science. I could not find an independent Artificial Analysis row for Terra, so this comparison rests on a single source.
Which one is better for coding?
For agents that run commands in a terminal, the measurements point to Sonnet 5.5. On SWE-sweep, which measures repairing bugs without breaking existing code, Hydrogen leads, but the absolute values there are low. Do not decide before trying it on your own codebase.
Which one is cheaper?
By list price: Terra ($1 / $5, third-party list), then Hydrogen ($1.50 / $7.50), then Sonnet 5.5 ($2 / $10, third-party list). By Artificial Analysis's cost per task, Hydrogen high is $1.87 and Sonnet 5.5 high is $1.64; at the upper settings Sonnet is clearly more expensive.
Does Hydrogen hallucinate more?
On AA-Omniscience its hallucination rate is 8.5%: higher than Argon (4%), lower than Carbon (11%). I have no value from the same benchmark for Sonnet 5.5 or Terra.
Why does the thinking level matter so much?
Because the same model scores very differently at different levels: Hydrogen scores 27 at low, 51 at high and 54 at xhigh. Cost and time rise with the level too.
Sources
- Google DeepMind, Gemini 4 Hydrogen: Model evaluation (November 2026): how Hydrogen's and the rivals' numbers were obtained on each benchmark.
- Google, Introducing Gemini 4 Hydrogen (November 12, 2026): comparison table (image), speed and fabrication claims.
- Google DeepMind, Gemini models page: text version of the benchmark table, same-family values for Carbon and Argon.
- Google DeepMind, Gemini 4 Hydrogen model card: benchmark values.
- Google, Gemini API pricing: Hydrogen, Argon and Carbon prices, free tier (checked November 14, 2026).
- Artificial Analysis, Gemini 4 Hydrogen: speed, time to first token, output tokens used.
- Artificial Analysis, model leaderboard: index by thinking level, cost and time per task.
- DataCamp, Gemini 4 Hydrogen (November 13, 2026): Artificial Analysis's Terminal-Bench 4.0 run, hallucination rate.
- BenchLM, Gemini 4 Hydrogen (November 13, 2026): AA-Omniscience hallucination rate and accuracy, Terminal-Bench value.
Related Posts
I Tested Gemini 4 Hydrogen: 4 Thinking Levels, 1 Task
I gave Gemini 4 Hydrogen the same coding task at four thinking levels. All four solved it; time went from 14 to 198 seconds and tokens grew 14-fold.
What Is Gemini 4 Hydrogen? Specs, Price, Benchmarks
Google opened Gemini 4 Hydrogen to everyone on Nov 12, 2026: a 1M-token window, $1.50 input pricing and a free API tier. Specs and measurements in one place.
What Is Antigravity Orchestrator? Carbon Plans, Hydrogen Codes
Orchestrator in Antigravity plans with Carbon or Argon and writes the code with Hydrogen. Google reports 24% less quota and 19% faster; Ultra and Pro only.