JetBrains Mellum2.1 Released: How to Locally Serve a 12B MoE Coding Agent with Just 2.5B Active Parameters

·

JetBrains Releases Mellum2.1: A 12B MoE Open Model for Coding Agents

Key Takeaways

  • JetBrains released Mellum2.1, a MoE inference coding model that activates only 2.5B of its 12B total parameters, under Apache 2.0.
  • The architecture is identical to Mellum2 (28 layers, 64 experts, 8 active, GQA 32/4, 131K context), and most of the improvement came from real-repository-based reinforcement learning (RL) post-training.
  • Agentic coding scores jumped dramatically: SWE-bench Verified 2.0→47.0, SWE-bench Pro 0.0→28.0, and Terminal-Bench 2.1 0.6→17.4.

Because Mellum2.1 lifted agentic coding scores with real-repository RL while barely touching the model architecture, it shows that the combination of “small active parameters + domain-specialized RL + local-serve quantization” can become a new baseline for coding agents.

Table of Contents

In October 2026, JetBrains released Mellum2.1, a new coding-agent-only Mellum model, under Apache 2.0. What is more interesting than the release itself is the fact that agentic coding scores were dramatically improved through post-training (RL) alone, with almost no change to the model architecture. The package includes an MoE structure that activates only 2.5B of 12B total parameters, a 131K context window, and GGUF builds in the 7–8GB range—pointing to a potential new baseline for “local models for agent workers.” This article first covers the background behind Mellum2.1 and its benchmark results, then walks through practical serving tips with vLLM, GGUF, and MTP, along with the strengths and weaknesses compared to Qwen3.5-9B.

1. Mellum2.1 Overview — Three Targets JetBrains Is Aiming For

JetBrains defined this model’s use cases as (1) an agent worker running in the JetBrains IDE family and its cloud, (2) an in-house general-purpose reasoning assistant, and (3) a private self-hosting scenario where data sovereignty is required. The self-hosting angle stands out the most, because the team simultaneously registered builds that compress the BF16 24.3GB model down to Q4_K_M 8.1GB and MXFP4_MOE 7.0GB, enabling sub-agents to run on a single laptop GPU. By releasing not only the weights but also serving artifacts, Mellum2.1 substantially reduces the operational burden that is common in the 12B class—the situation where “the license is open, but quantization is unofficial.”

2. Architecture Highlights — 12B/2.5B MoE and 131K Context

Mellum2.1 uses the same backbone as Mellum2. It is a 28-layer Transformer that activates 8 of 64 experts per token, yielding 2.5B active parameters for inference. GQA is configured with 32 heads and 4 KV heads, and a 1,024-token sliding window is applied every 3–4 layers to control memory footprint. The vocabulary size is 98,304, designed to cover a wide range of code identifiers and symbols. The default context of 131,072 tokens is operated at 114K tokens for agentic execution, a number set with long-horizon repository exploration in mind.

2.1 Training Paradigm — How Scores Were Lifted with Real-Repository RL

Since the architecture is identical, most of the difference between Mellum2 and Mellum2.1 comes from post-training (RL). JetBrains unified RL tasks across five domains—math, competitive programming, science, tool use, and software engineering—and, after pre-filtering broken tests, unverifiable answers, and tasks that are too easy or impossible, used test pass/fail results from shell and file-editing tools against real repositories as the reward signal. The evaluation harness is the open-source Pi v0.73.1, used as-is, so scores for four models are measured within the same pipeline and serve as a comparison group.

3. Benchmark Results — Top of the Same Pipeline Group, but Variance by Domain

The results measured against JetBrains’ evaluation harness are as follows. The Mellum model Mellum2.1 recorded 47.0 on SWE-bench Verified, 28.0 on SWE-bench Pro, and 17.4 on Terminal-Bench 2.1, showing large gains over Mellum2. On single code-generation tasks, it took first place in the same evaluation pipeline group with LiveCodeBench v6 82.0, HumanEval+ 91.5, MBPP+ 79.4, and BFCL v4 62.3. The safety metric HarmBench fell by more than half, from 21.5 to 8.5.

Metric Mellum2 Mellum2.1 Qwen3.5-9B Gemma 4 E4B
SWE-bench Verified 2.0 47.0 52.4 38.2
SWE-bench Pro 0.0 28.0 38.0 21.7
Terminal-Bench 2.1 0.6 17.4 22.5 14.1
LiveCodeBench v6 62.4 82.0 65.6 / 75.4* 70.8
HumanEval+ 86.0 91.5 90.2 88.7
MBPP+ 72.1 79.4 78.0 75.6
BFCL v4 47.8 62.3 61.5 55.2
AIME 25/26 — 71.2 86.7 68.4
GPQA Diamond — 70.5 77.8 69.1
HarmBench 21.5 8.5 9.8 14.6

* Two values are listed because the same model was measured with different evaluation pipelines. That variance is itself a pitfall.

4. Comparison Pitfalls — “Benchmark Numbers” Shift with the Pipeline

Even though Mellum2.1 ranks first in the same-harness group, there are clear points where the lead flips by domain. Qwen3.5-9B still holds the edge in agentic and reasoning areas, with SWE-bench Pro 38.0, AIME 25/26 86.7, and GPQA Diamond 77.8. Furthermore, scores on the same model can vary significantly depending on the evaluation pipeline. The 65.6 / 75.4 figures in LiveCodeBench v6 above are for the same Qwen3.5-9B measured with different harnesses. Rather than selecting a model based on a single benchmark number, it is safer to build a separate harness close to the in-house workload and re-measure. Before adoption, the first priority is to check whether the evaluation scripts published in the JetBrains Mellum2.1 announcement can be reproduced as-is.

5. Serving with vLLM — Recommended Flags and Inference Parameters

The following flag combination is recommended when launching the Mellum model Mellum2.1 on vLLM. The reasoning parser is set to qwen3-compatible, and tool calling is enabled with the hermes format. Temperature/sampling defaults are 0.6 / 0.95 / 20, which should be used as a baseline and fine-tuned with in-house data.

vllm serve jetbrains/Mellum2.1-12B \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --temperature 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --max-model-len 131072 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.92

On a single H200, throughput has been reported at roughly 2x that of Qwen3.5-9B, and enabling the MTP (Multi-Token Prediction) head on a single-request path can reportedly deliver a 1.6x speedup. However, since the speculative-decoding-specific MTP head has not yet been released (coming soon), serverless multi-request environments will need to operate with vLLM’s default continuous batching only.

6. Running a Local Sub-Agent with GGUF

The more interesting part for practical use is the GGUF build. Four levels of quantization are already registered and can be used as-is with llama.cpp, Ollama, and LM Studio. Using top-token match rate as a quality indicator against the BF16 24.3GB baseline, the order is: Q8_0 12.9GB (96.1%) → Q6_K 10.9GB (93.9%) → Q4_K_M 8.1GB (88.0%) → MXFP4_MOE 7.0GB (85.6%). Q4_K_M is recommended, and since MXFP4_MOE sits in the 85% match range relative to size, it is safer to limit it to auxiliary sub-agents.

# Launch a Mellum model sub-agent with Ollama
ollama pull mellum2.1:Q4_K_M
ollama run mellum2.1:Q4_K_M \
  --ctx-size 114000 \
  --temperature 0.6 \
  --top-p 0.95

# Run llama.cpp directly (server mode)
./llama-server -m mellum2.1.Q4_K_M.gguf \
  -c 114000 \
  --jinja \
  --chat-template-file chat-template.jinja

Sub-agents should be restricted by system prompts so that they operate only within the narrow scope delegated by the main agent (e.g., generating a specific file patch or writing unit tests). This keeps response latency around the one-second mark even with an 8GB model. In addition to IDE plugins and JetBrains’ own harness, workflows can also be composed by combining with external controllers such as NVIDIA’s multi-turn agent recovery technique, PivotOPD.

Try It Right Now

  • Deploy Mellum2.1 on a vLLM container with the --reasoning-parser qwen3 --tool-call-parser hermes combination to spin up a single agent endpoint, then run a dry run with 20 in-house SWE-bench-style tasks.
  • Pull the GGUF Q4_K_M build with Ollama, launch an IDE-assist sub-agent on a single RTX 4090 or 24GB VRAM laptop with ctx-size 114000, separate the task-split prompts from the main agent, and measure response latency.
  • Build a mini-harness based on an in-house code repository for both Mellum2.1 and Qwen3.5-9B (file edit + shell + test pass/fail reward) and directly compare reproduced SWE-bench Pro scores on 50 identical tasks.
  • To prepare for evaluation pipeline variance, save all inference result logs (prompts, sampling, tool calls, success/failure) as JSONL, and tag the model, version, and harness as metadata on the in-house dashboard.
  • Do not consider safety validation complete based solely on the HarmBench 8.5 number. Prepare 30 in-house policy-violation patterns (comment deletion, secret exposure, license disregard) as a separate regression set and wire them into CI.

Frequently Asked Questions

If Mellum2.1 is used only as a sub-agent, what is a reasonable choice for the main model?

While Mellum2.1’s ability to be self-hosted as a 12B/2.5B MoE is a strength, complex planning and multi-hop reasoning still tend to be led by Qwen3.5-9B or higher-tier closed models. A realistic setup is a two-tier structure in which the main model is an API-based reasoning model, and only narrow tasks such as code editing, test writing, and simple refactoring are delegated to Mellum2.1.

Which build should be set as the default between Q4_K_M (8.1GB) and MXFP4_MOE (7.0GB)?

By top-token match rate, Q4_K_M leads by 2.4 percentage points at 88.0%, and it has broader compatibility with llama.cpp, Ollama, and LM Studio. MXFP4_MOE should be used as a secondary option in edge environments that need to save an additional 1GB of VRAM, while Q4_K_M is placed on the critical path to keep operational risk low.

Can the JetBrains evaluation harness be ported into the in-house environment as-is?

Pi v0.73.1 is open source, so porting itself is possible. However, because the shell, file-editing, and test-execution environment assumed by Pi may not match the in-house CI, the container image, network access, and tool whitelist should be reconfigured to match internal policies, and regression behavior should be confirmed via a dry run.

Reference Source

This article was written after verifying the following source: MarkTechPost — JetBrains Releases Mellum2.1: A 12B MoE Open Model for Coding Agents

Leave a Reply

Your email address will not be published. Required fields are marked *