Hmm… the situation changes quite a bit depending on whether you’re looking for model weights or an endpoint…
If what you mainly need is a replacement for the old Groq endpoint, I would actually narrow the first pass quite a lot:
openai/gpt-oss-120b — my first production migration candidate.qwen/qwen3.6-27b — the most interesting challenger, especially if your workflow depends on parallel tool calls, vision, or Qwen’s behavior on your multilingual workload.
That is not just a model-ranking choice. Groq’s own deprecation page lists exactly those two as the recommended replacements for llama-3.3-70b-versatile.
There is one distinction I think is worth making first, though:
Groq retired the Groq model ID / offering llama-3.3-70b-versatile; Meta’s Llama 3.3 70B Instruct weights did not disappear.
Groq says this August 16, 2026 shutdown applies to Free and Developer-tier usage; Enterprise customers with committed-spend contracts are not affected. So there are really three different migration problems hiding inside the question:
Need a replacement Groq endpoint?
→ start with gpt-oss-120b
→ test qwen3.6-27b where its specific advantages matter
Need to preserve Llama 3.3 behavior as much as possible?
→ keep Llama 3.3 and change provider/runtime
Actually looking for weights to self-host?
→ then VRAM, RAM, quantization, runtime and model architecture
become first-class selection criteria
For the Groq endpoint case, gpt-oss-120b currently looks like the lowest-friction default to me, but there is one compatibility check I would do very early:
Does your existing agent actually depend on parallel tool calls?
Groq currently lists parallel tool calling as supported for the old Llama 3.3 endpoint and for Qwen3.6, but not for GPT-OSS 120B. If your workflow relies on that behavior, Qwen becomes much more interesting immediately.
A rough current comparison on Groq looks like this:
| Groq endpoint |
openai/gpt-oss-120b |
qwen/qwen3.6-27b |
| Groq status |
Production |
Preview |
| Advertised speed |
~500 tok/s |
~500 tok/s |
| Context on Groq |
131,072 |
131,072 |
| Max completion on Groq |
65,536 |
16,384 |
| Paid input / 1M tokens |
$0.15 |
$0.60 |
| Paid output / 1M tokens |
$0.60 |
$3.00 |
| Current Free-plan limits |
30 RPM / 1K RPD / 8K TPM / 200K TPD |
same |
| Local/remote function calling |
Yes |
Yes |
| Parallel tool calls |
No |
Yes |
| Remote MCP |
Yes |
Yes |
| Groq built-in tools |
Yes |
No |
| JSON Object mode |
Yes |
Yes |
| Strict JSON Schema output |
Yes |
No; use JSON Object mode |
| Vision input |
No |
Yes |
| Underlying architecture |
117B MoE, ~5.1B active/token |
27B dense |
Current sources: Groq supported models, tool-use matrix, Structured Outputs, Free-plan rate limits, GPT-OSS 120B endpoint, and Qwen3.6 endpoint.
The Production vs Preview distinction is probably the biggest operational reason I would test GPT-OSS first. Groq explicitly says Production models are intended for production use and receive a clear migration path when deprecated, whereas Preview models are evaluation-oriented and can be discontinued at relatively short notice.
That does not mean GPT-OSS is permanent. Nothing in a hosted model catalog should be treated as permanent. But given that your original problem was caused by an endpoint retirement, I would give that service-status distinction real weight.
Groq has also repeatedly used gpt-oss-120b as a migration target for other retired endpoints. I would treat that as a useful stability signal, not a guarantee.
The other major advantage for your use case is structured output. Groq currently supports strict JSON Schema constrained decoding only on GPT-OSS 20B/120B. That is materially different from merely asking a model to “return JSON.”
One caveat is important: Groq currently says its Structured Outputs feature cannot be combined with streaming or tool use. So I would not design around the assumption that one call can simultaneously give you strict constrained JSON and the normal tool-calling loop. They are better treated as separate contracts in the application.
Qwen3.6 is still very worth testing, though. Its official model card describes it as a 27B dense model aimed heavily at agentic coding and real-world utility, and Groq gives it several useful capabilities that GPT-OSS does not currently have there: parallel tool calls, image input, and explicit thinking/non-thinking modes. If your actual multilingual or agent workloads strongly prefer it, that can outweigh the Preview-status disadvantage.
I would especially keep Qwen in the comparison if any of these are important:
- multiple independent tools should be called in parallel;
- the same endpoint needs image/document-image input;
- your production workload is substantially multilingual;
- your real agent traces work noticeably better with Qwen;
- a much smaller underlying weight set matters because you may later self-host.
For multilingual use I would test the actual language boundaries in your application, rather than infer a winner from benchmark branding. OpenAI describes the gpt-oss pretraining dataset as mostly English and text-only, while Groq reports strong multilingual evaluation for the hosted GPT-OSS endpoint; Qwen is also explicitly positioned as multilingual. That makes this a good candidate for a very small application-specific test rather than a benchmark-based conclusion.
I think your existing same-prompt test is already the right starting point
I would not build a large new benchmark before trying the prompts you already have.
In fact, Groq’s own migration guide recommends starting from parameter/prompt parity first, measuring, and only then adapting prompts deliberately.
I would use two passes:
Pass 1 — migration/regression check
Keep the existing production prompts and relevant sampling settings as close as practical.
This answers:
“If I replace only the model ID, what breaks?”
Pass 2 — best supported profile
For the cases that regress, then try the model’s intended settings — reasoning mode, prompt adaptation, sampling changes, etc.
This answers:
“If I actually deploy this model properly, how good can the replacement become?”
Keeping those two passes separate prevents prompt tuning from hiding migration regressions.
I would also avoid collapsing everything into one average score. For the requirements you listed, a handful of small failure-oriented cases can reveal much more.
For example:
- RAG: one normal grounded answer, one answer that is absent from the retrieved material, and one case where the retrieved document deliberately contradicts something the model is likely to “know”.
- Prompt generation: generate a prompt from 8–10 explicit constraints, then change exactly one requirement on the next turn and see whether unrelated requirements drift.
- JSON: distinguish “valid JSON”, “schema-valid JSON”, and “the values inside the JSON are actually correct”.
- Tool use: no-tool case, single-tool case, parallel independent tools, and a dependent two-step call.
- Instruction following: retain several constraints across a few turns rather than testing only one-shot compliance.
- Multilingual: non-English user input + English tool schema or RAG documents + requested-language answer.
- Agent stability: for the few cases that actually matter, repeat them a couple of times rather than assuming one successful trajectory is representative.
For tool use in particular, I find this decomposition useful:
Did it decide correctly whether a tool was needed?
↓
Did it select the right tool?
↓
Were the arguments semantically correct?
↓
Did the API/parser produce a usable structured call?
↓
Did the tool actually execute?
↓
Did the model use the returned result correctly?
↓
Did it stop when the task was complete?
A failure in any one of those layers can look like “the model is bad at agents”, but the fix can be completely different.
What exactly was shut down?
llama-3.3-70b-versatile is a Groq model ID, not the Hugging Face repository name for Meta’s weights.
Groq’s current deprecation history says:
- deprecation announced: June 17, 2026;
- shutdown: August 16, 2026;
- affected: Free and Developer-tier usage;
- recommended replacements:
openai/gpt-oss-120bqwen/qwen3.6-27b
The underlying Meta Llama 3.3 70B Instruct model still exists and can be served elsewhere or self-hosted.
That distinction matters because there are two very different ways to minimize migration risk:
same provider + different model
vs.
same model + different provider
Neither is automatically safer in every application.
Changing only the model can preserve the API/provider behavior while changing generation behavior.
Changing only the provider can preserve the weights while changing the serving implementation, tool features, context limits, decoding defaults, latency, and API details.
So I would think of the deployable thing as more than just a model name.
Why I would start with GPT-OSS 120B on Groq
The strongest argument is operational rather than a claim that GPT-OSS is universally the better model.
Groq currently classifies openai/gpt-oss-120b as a Production model.
Its current Groq contract includes:
- ~500 tok/s;
- 131,072-token context;
- 65,536 maximum completion tokens;
- local/remote tool use;
- Remote MCP;
- Groq built-in browser/code tools;
- JSON Object mode;
- strict JSON Schema mode;
- configurable reasoning;
- current Free-plan access.
Its paid token pricing is also considerably lower than the current Qwen3.6 endpoint.
Groq’s deprecation policy says Production models are intended for production use and that a clear migration path will be supplied when one is retired.
By contrast, qwen/qwen3.6-27b is currently marked Preview, and Groq explicitly says Preview models should not be treated as production-stable because they may be discontinued on shorter notice.
Since endpoint longevity is part of the problem that brought you here, that is enough for me to make GPT-OSS the default first test.
There is also a history signal: Groq has made GPT-OSS 120B the recommended migration destination for several other retired models. I would not read that as a long-term guarantee, but it does suggest it currently occupies a fairly central place in their production catalog.
Where Qwen3.6 can still be the better choice
Qwen’s disadvantage here is mostly the current Groq service status, not an obvious lack of model capability.
The underlying Qwen3.6-27B model is Apache-2.0 licensed, 27B dense, multimodal, and explicitly aimed at agentic coding / iterative workflows.
On Groq it currently has:
- ~500 tok/s;
- 131K context;
- text + image input;
- local/remote tools;
- Remote MCP;
- parallel tool calls;
- JSON Object mode;
- thinking and non-thinking modes.
The parallel-call distinction could be particularly important.
Groq’s current tool support matrix lists:
- old Llama 3.3 70B: parallel tools = yes;
- Qwen3.6-27B: parallel tools = yes;
- GPT-OSS 120B: parallel tools = no.
So if your existing Llama-based agent routinely emitted two or more independent tool requests in one turn, I would test that before doing much else. It is a concrete migration-contract difference, not a subtle benchmark issue.
Qwen also accepts image input on Groq, while GPT-OSS 120B is text-only there.
For multilingual behavior, I would let your actual languages decide. Qwen has good reasons to be in that test set, but I would not try to infer production multilingual quality from unrelated benchmark tables.
Structured JSON and tool calling are two different contracts
For the “structured/JSON output reliability” requirement, GPT-OSS currently has a useful provider-level advantage.
Groq’s Structured Outputs documentation supports strict constrained JSON Schema mode on:
openai/gpt-oss-20bopenai/gpt-oss-120b
With strict: true, Groq constrains decoding to the supported schema subset.
For other models such as Qwen3.6, Groq currently directs you to JSON Object Mode instead. That can ensure JSON syntax, but it is not the same contract as exact schema-constrained generation.
I would still score two things separately:
- structural correctness — did it conform to JSON/schema?
- semantic correctness — are the actual field values correct?
Constrained decoding can solve the first without solving the second.
There is also an implementation boundary worth noticing: Groq currently states that Structured Outputs cannot be combined with streaming or tool use.
So for an agent design I would keep these two paths conceptually separate:
tool invocation
→ tool-call API / tool schema
final application object
→ strict Structured Outputs, where appropriate
rather than assuming one mechanism guarantees both.
If you really mean model weights / self-hosting
Then I would answer the size/VRAM part very differently.
Both of the main candidates have permissive Apache-2.0 licensing:
But their architectures make the parameter labels easy to misread.
GPT-OSS 120B
OpenAI describes it as roughly:
- 117B total parameters;
- 5.1B active parameters per token;
- 128 experts, 4 active per token;
- MoE;
- 128K-class native context.
The 5.1B active figure is mainly a compute characteristic. It does not mean the model has a 5.1B-sized weight footprint.
OpenAI’s official deployment target for its MXFP4 version is a single 80GB-class GPU.
Qwen3.6-27B
Qwen is a 27B dense model.
That means much less total weight storage than GPT-OSS 120B, even though all of its dense parameters participate rather than routing through only a few experts.
As a first-order weight-only estimate:
27B × 8 bits ≈ 27 GB
27B × 4 bits ≈ 13.5 GB
but those are not complete VRAM requirements. Real deployment also needs KV/cache or model state, runtime buffers, possibly vision components, and headroom; long context can change the picture substantially.
The Qwen model card advertises 262K native context and extension beyond that, but a hosted endpoint may expose a smaller contract — Groq currently exposes 131K. That is a good example of why model capability and endpoint capability should be recorded separately.
If self-hosting, I would compare an exact deployment tuple rather than just “Qwen vs GPT-OSS”:
model revision
+ exact quantized artifact
+ runtime/version
+ chat template
+ reasoning/tool parser
+ context allocation
+ sampling
For Qwen, its official card documents Transformers, vLLM, SGLang, local apps, and quantized deployment paths.
For GPT-OSS, OpenAI likewise treats the runtime implementation as part of what needs to be validated.
And for agent/tool workloads I would explicitly smoke-test tool arguments and structured output again after choosing the quantization. A quantization that looks fine on ordinary chat is not automatically equivalent on format-sensitive behavior.
If the main requirement is local/self-hosted compactness, Qwen3.6-27B is much easier to take seriously than the “120B” name on GPT-OSS would suggest from endpoint testing. The fact that both happen to run around 500 tok/s on Groq does not transfer to local hardware.
If you were already happy with Llama 3.3 itself
There is a third route that may be lower-risk than changing models:
keep Llama 3.3 and move the serving layer.
The Hugging Face Inference Providers model catalog currently still lists meta-llama/Llama-3.3-70B-Instruct through multiple providers.
Hugging Face also supports provider="auto" routing, including provider failover when a provider is unavailable.
Conceptually that lets you separate:
model identity
from
provider identity
which can make future provider churn easier to absorb.
I would call this a lower model-change path, not a zero-change path. Different inference backends/providers can still expose different:
- context limits;
- sampling/default behavior;
- tool support;
- structured-output support;
- speed;
- latency;
- pricing;
- serialization/API behavior.
So it still deserves a small regression test.
There is also a licensing difference worth noting if “fully open/open-weight” is part of the migration goal: GPT-OSS and Qwen3.6 use Apache 2.0, whereas Llama 3.3 remains under Meta’s Llama 3.3 Community License.
And if free usage is important, HF routing is not currently equivalent to Groq’s Free plan: HF currently gives Free users $0.10/month in routed Inference Providers credits, so this route is more compelling for behavior preservation/provider abstraction than for a generous free inference allowance.
A small migration check I would use
You already mentioned testing the same prompt set across candidates, which is probably the highest-value starting point.
I would make the existing workload the benchmark rather than trying to reproduce a large public leaderboard.
Something like this would cover most of the requirements you listed without turning migration into a research project:
| Area |
One high-information case |
| RAG |
Retrieved evidence deliberately contradicts plausible model prior knowledge |
| RAG abstention |
Required answer is not present in retrieval |
| Prompt generation |
Generate from many constraints, then modify exactly one constraint |
| JSON |
Nested object with enums/IDs; score syntax, schema and values separately |
| Tool selection |
Several similar tools; only one is appropriate |
| Tool arguments |
Correct tool but non-trivial enum/date/identifier arguments |
| Parallel tools |
Two independent calls that can run simultaneously |
| Agent chain |
Tool A returns ID needed by Tool B |
| Multi-turn |
Preserve initial constraints several turns later |
| Multilingual |
Non-English query + English tools/docs + non-English final answer |
You do not necessarily need to run every case repeatedly. If the two models agree, move on. If one case differentiates them, repeat that case a few times to see whether you found a stable difference or ordinary sampling variance.
I would also keep the first comparison intentionally boring:
same production prompt
same tool definitions
same context
same or equivalent sampling budget
same expected answer/test
Then optimize only the failures.
That gives you both:
- a drop-in migration regression view, and
- a later best achievable deployment view.
Those answer different questions, and both are useful.
For public benchmark numbers, I would use them only as hints for which cases deserve testing. Qwen publishes very strong coding/agent results on its model card, and GPT-OSS has strong reasoning/tool evaluations from OpenAI/Groq, but the evaluation harnesses, contexts, reasoning budgets and scaffolds are not identical. I would not turn those separate tables into a direct leaderboard.
So, if I had to reduce all of this to a default migration path:
If you mean “I need something to replace this Groq model ID”, I would test openai/gpt-oss-120b first.
It currently has the stronger production-service story, the same listed Free-plan allowance and roughly the same advertised generation speed as Qwen3.6, much lower paid token pricing, a larger output allowance, and strict JSON Schema support.
I would test qwen/qwen3.6-27b alongside it if parallel tool calls, vision, multilingual behavior, or your actual agent workload could change the decision. Its main problem in this particular comparison is not obvious capability — it is that Groq currently labels it Preview.
If you actually mean “I want weights that I can own and self-host”, then I would not use that ranking at all. Qwen’s 27B dense footprint becomes a major advantage, and a broader weight/runtime comparison makes sense.
And if Llama 3.3 itself was already doing exactly what you wanted, keeping the model and swapping the provider is also a perfectly reasonable migration branch rather than throwing away a known-good behavioral baseline.