Category: AI & Open Source

  • Olmo-core 3 Achieves Under 5% Throughput Loss at 47B Parameters — A Leap for the Olmo Sparse Training Stack

    Olmo-core 3 Achieves Under 5% Throughput Loss at 47B Parameters — A Leap for the Olmo Sparse Training Stack

    Key Takeaways

    • Olmo-core 3 is a meaningful upgrade to the large language model development framework, the product of redesigning and open-sourcing an open-source Mixture-of-Experts (MoE) training system.
    • The system kept active parameters at roughly 3.2B per token while expanding the expert pool from 8 to 128, holding training throughput loss to under 5%.
    • In the same benchmark, total parameter capacity grew from 4.6B to 47B, and the same infrastructure was validated at scales exceeding 1 trillion parameters.

    Analysis

    Table of Contents

    The numbers Olmo-core 3 has published are not just a routine framework update. AllenAI’s Olmo-core 3 expanded the expert pool 16x, from 8 to 128 experts, while keeping active parameters at roughly 3.2B per token, and the training throughput loss stayed under 5%. In the same experiment, total parameter capacity grew more than 10x from 4.6B to 47B, and the same stack was confirmed to operate at 1-trillion-parameter configurations.

    In my view, this is the most meaningful point. The core cost of large-scale MoE training is the routing and communication overhead that arises as the number of experts grows, and Olmo-core 3 gives the impression of resolving much of that bottleneck at the design stage.

    Olmo-core 3’s 8→128 Expert Scaling

    A throughput loss of under 5% sounds almost magical when you only look at the result. But because MoE is a sparse model that activates only a subset of experts per token, the number of active parameters is directly tied to training compute. By keeping active parameters near 3.2B and growing only the total parameter count, Olmo-core 3 pursues a tradeoff that expands model capacity while holding training FLOPs nearly constant.

    Of course, costs such as in-memory residency of active and inactive experts, gradient synchronization, and router load do not disappear entirely. The fact that the sub-5% figure was sustained is itself evidence that the communication stack and scheduler improvements that distribute the secondary costs have reached a meaningful level of maturity.

    Evolution of the Olmo Sparse Training Stack

    Olmo-core 3 did not appear in isolation. AllenAI’s sparse training lineage traces back to OlmoE, an MoE architecture with 64 routing experts. While Olmo 3 branched off into a dense structure, Olmo-core 3 has taken up the sparse training stack once again.

    Category OlmoE Olmo 3 Olmo-core 3
    Architecture MoE (64 routing experts) Dense MoE (sparse training stack)
    Main Focus Expert routing validation Dense training optimization Large-scale sparse infrastructure
    Validated Scale Mid-size Mid-size dense 47B to 1-trillion-parameter class
    Release Scope Primarily models Model + some code Full training stack

    The row practitioners should pay attention to is the last one. Olmo-core 3 differs from previous generations in that it has open-sourced not just model weights but the training stack itself.

    The MoE Dilemma — Issues After Olmo-core 3

    MoE’s limitations are well known. Because only a subset of experts is activated per token, you can pack in more learnable capacity, but the entire model still has to reside in GPU memory and be updated every step. As total parameters grow, memory and communication costs scale at least linearly, often more.

    Olmo-core 3’s sub-5% figure does not mean the dilemma has disappeared. It means routing and communication costs can be distributed right up to the point where they would otherwise eat into the efficiency gains. From a practitioner’s standpoint, the notable point is that where exactly this balance point holds is the key variable for the next one to two years.

    The Open Infrastructure Olmo-core 3 Unlocks

    The release itself carries value. When a 1-trillion-parameter-class training stack becomes open source, academic researchers and smaller labs that do not operate their own large-scale compute can port the same communication and routing patterns into their own infrastructure. AllenAI notes in the original post that Olmo-core 3’s training stack focused on communication and routing optimization. (Olmo-core 3 announcement note)

    What to Try Right Now

    • Download the routing scheduler code from the Olmo-core 3 public repository and design a small-scale reproduction experiment with an 8 to 32 expert configuration.
    • Collect GPU-to-GPU all-to-all communication logs during your own MoE training and compare them against the sub-5% loss baseline.
    • Plot the router loss curves of OlmoE and Olmo-core 3 on the same token-count basis to check how increasing the number of experts affects convergence.
    • If a 1-trillion-class configuration is not feasible, prioritize validating a setup at 47B with active parameters pinned near 3.2B.
    • Create an Olmo-core 3 adoption review page on your team wiki and track in-memory residency cost and communication overhead as separate sections.

    Frequently Asked Questions

    How is Olmo-core 3 different from OlmoE?

    If OlmoE was the generation that validated the architecture with 64 routing experts, Olmo-core 3 is the result of redesigning that sparse training stack to operate at 128 experts and at scales from 47B to 1 trillion parameters. The release scope has also expanded from models to training infrastructure.

    How was the sub-5% throughput loss achieved?

    By keeping active parameters at roughly 3.2B per token and only expanding the expert pool, the growth in training FLOPs was suppressed. AllenAI explains that this was supported by scheduler improvements that distribute routing and all-to-all communication load.

    Can smaller teams put it to use?

    Reproducing 1-trillion-parameter training outright is difficult, but porting the communication and routing patterns of the released training stack into smaller configurations is enough to diagnose bottlenecks in your own MoE pipeline.

    Will it replace dense models anytime soon?

    Not in the short term. The structural limits of MoE remain, since the entire model must reside in memory, so the tradeoff between training efficiency and inference or deployment cost will continue to exist.

    Summary of the Debate

    Olmo-core 3’s demonstration of sub-5% throughput loss is the first signal that MoE’s communication and routing costs can be absorbed through design. However, whether this balance point holds beyond 1 trillion parameters has not yet been verified, and the in-memory residency cost that comes with growing total parameters remains. With the training stack now open-sourced, the next one to two years will be the period in which outside researchers directly measure how far this balance point holds across diverse hardware configurations.

    Reference

    This article was written after checking the following source: Hugging Face Blog — Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs

  • Kumo Tabular Tops 4 Benchmarks — A Clear Look at NVIDIA’s Open Tabular Foundation Model Done in a Single Forward Pass

    Kumo Tabular Tops 4 Benchmarks — A Clear Look at NVIDIA’s Open Tabular Foundation Model Done in a Single Forward Pass

    Key Summary

    • Kumo Tabular is an open tabular foundation model that takes labeled rows as context and predicts labels for new rows in a single forward pass, handling both classification and regression without any training, hyperparameter tuning, or feature engineering.
    • Based on the source material, the training data is limited to artificial data, and three model sizes are offered ranging from 28M to 215M parameters.
    • It is distributed as an open-source library under the OpenMDW-1.1 license, which permits commercial use.

    An analytical guide that organizes a newly released tabular foundation model from the perspective of a practitioner who wants to try it immediately, while also flagging how the workflow shifts compared with the existing gradient boosting flow and what remains unverified.

    Table of Contents

    Kumo Tabular takes labeled rows as context and predicts the label of a new row in a single forward pass. No training, no hyperparameter tuning, no feature engineering. Released by NVIDIA on Hugging Face, Kumo Tabular is an open tabular foundation model that handles classification and regression through the same interface.

    It is hard to deny that gradient boosting trees (GBDT) have been the standard for structured-data prediction in industry for the past 20 years. XGBoost, LightGBM, CatBoost — virtually every organization has at some point followed the formula of “tabular data = GBDT.” Data preprocessing → training → tuning → validation → deployment, a five-step formula whose stability is now in question. The author views this point as the most meaningful aspect. It is an event where the workflow changes, not just the model itself.

    What Kumo Tabular Takes In and Puts Out

    The input is a bundle of labeled rows, that is, a context table. The output is a prediction for a new row. The model connects the two without any separate training. It belongs to the in-context learning family alongside TabPFN and TabICL, and is reminiscent of how LLMs perform new tasks from prompts alone.

    Whether classification or regression, it ends with the same call. Binary classification like churn or default, continuous-value regression like revenue, demand, or price — all of it in a single forward pass. For practitioners, think of sklearn’s fit/predict API, but with the fit step removed.

    Three Kumo Tabular Variants — Size and License

    According to the source, three variants have been released, ranging from 28M to 215M parameters. The exact middle size is not specified. All variants follow the OpenMDW-1.1 license and are accessible on both GitHub and Hugging Face as part of the NVIDIA Kumo Structured model collection.

    Variant Parameters Estimated Use Case License
    Small ~28M On-device / lightweight inference OpenMDW-1.1
    Medium Mid-scale (undisclosed) General server inference OpenMDW-1.1
    Large ~215M High-quality prediction / batch OpenMDW-1.1

    The parameter count is small compared with GBDT, but the entire context must be re-fed for each inference. Latency can grow if GPU resources are insufficient.

    Open Source Ecosystem — Simultaneous Release on Hugging Face and GitHub

    Kumo Tabular was introduced on the official Hugging Face blog. The OpenMDW-1.1 license permits commercial use, so it can be embedded into internal services or products without a separate agreement. It shares the same direction as other model families NVIDIA released around the same time — NVIDIA SoL-Pi, NVIDIA PAIR. While expanding its lineup toward local inference infrastructure and agent harnesses, NVIDIA has applied the same philosophy to the structured-data domain.

    #1 on 4 Benchmarks — But the Single-Source Caveat Must Be Noted

    The source claims first place on four benchmarks: TabArena, BeyondArena, TALENT, and ScoringBench. The core point is that a model pre-trained on artificial data outperformed GBDT on real-data-based benchmarks. However, it is difficult to independently cross-verify these figures using only the materials gathered for this analysis. Organizations evaluating adoption should look beyond the “#1” label and first examine “on which data distribution the #1 was achieved.”

    The First-Contact Workflow for Practitioners

    The first task that comes to mind is one where a labeled table already exists. Churn prediction, default classification, short-term demand forecasting, price regression — any task that has been labeled at least once in the past can be fed in as context rows to receive immediate inference.

    Applying it directly to data with no labels or where column meanings change frequently is difficult. There are likely constraints on the number of context rows and columns, and because the entire context must be re-fed for each inference, the structure is not well suited to large-volume batch processing. From a practitioner’s perspective, the most reasonable choice for the first PoC step is “the smallest table you have that already has labels.”

    Differences When Compared with the GBDT Workflow

    In the XGBoost/LightGBM workflow, five steps consumed the time of a single data scientist. Kumo Tabular effectively removes the training and tuning steps from that list. In their place, data governance — which labels may enter the context, how PII is masked, and how often the context is refreshed — takes on greater weight.

    The cost structure also changes. Because every prediction requires GPU inference, the model shifts from a “one-time training cost” to an “N-times inference cost.” For low-traffic tasks, costs can actually rise.

    Practical Application Points

    • Start with a task where a labeled table already exists, feed it as context, and run a PoC that returns a single-forward-pass result.
    • Measure accuracy, latency, and cost on the same holdout as the GBDT baseline. The gap on your own data matters more than any #1 benchmark figure.
    • Check the model card for the upper limits on the number of context rows and columns, and verify in advance whether large-volume batch processing is feasible.
    • Have the legal team review the obligations of OpenMDW-1.1 once more. Attribution requirements and redistribution restrictions are the key points.
    • Redesign the budget structure — which assumed a one-time training cost — around N-times inference cost for GPU-based inference.

    What to Try Right Now

    • Open the NVIDIA Kumo Tabular model card on Hugging Face and check the parameter count for each variant along with the full license text.
    • Pick one internal churn / default / demand dataset and feed 5,000–50,000 labeled rows as context to obtain a single-forward-pass result.
    • Compile a comparison table against LightGBM with default hyperparameters on the same holdout. Use three axes: accuracy, prediction time, and GPU memory usage.
    • Ask the data governance team whether PII columns are included in the context, and if so, how the masking policy should be structured.
    • Review the full text of the OpenMDW-1.1 license together with the in-house legal team and reach a conclusion on whether it can be embedded into commercial services.

    Frequently Asked Questions

    Can Kumo Tabular fully replace XGBoost?

    It is too early to conclude based on the source alone. The claim of #1 on four benchmarks does not mean a model pre-trained on artificial data is also superior on real distributions. A direct comparison on the same holdout must come first.

    Does it support both classification and regression?

    According to the Hugging Face blog, it predicts both classification and regression in a single forward pass. There is no need to separate the model call.

    Is it available for commercial use?

    It is distributed under the OpenMDW-1.1 license, which permits commercial use. However, there may be obligations, so reviewing the full license text is essential.

    How does Kumo Tabular differ from TabPFN and TabICL?

    All three belong to the in-context learning family. While TabPFN and TabICL are part of the academic/research stream, Kumo Tabular is a model designed with commercial distribution in mind within NVIDIA’s lineup, which sets it apart in character.

    It is still too early to declare that Kumo Tabular has fundamentally shifted the paradigm of tabular ML. There are many unverified areas: the actual upper limit on the number of context rows, how well pre-training on artificial data generalizes across domains, and the break-even point at which GPU inference cost exceeds the one-time training model. Until these three are clarified, any claim of “GBDT replacement” is premature. Even so, it is a meaningful starting point as an experiment for practitioners. If you already have a labeled table in-house, it is worth running once this week.

    Source Reference

    This article was written after checking the following source: Hugging Face Blog — NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction

  • 5 Stages of On-Device AI Adoption — Building a Local Inference Workflow with NobodyWho

    5 Stages of On-Device AI Adoption — Building a Local Inference Workflow with NobodyWho

    Key Takeaways

    • NobodyWho is an open-source on-device inference engine that runs AI models directly on the user’s device instead of through cloud APIs, designed to embed chat, image understanding, and voice capabilities into apps and games.
    • Once the model is downloaded, subsequent execution requires no internet connection or API key, and user input is processed on-device rather than transmitted to external servers.
    • It integrates easily across a range of development environments including Python, enabling local AI workflows that do not depend on cloud-based LLM APIs.

    A practice-focused analysis introducing NobodyWho as a local AI integration option that lets developers reduce cloud API dependency and solve cost, privacy, and latency challenges at the same time.

    Table of Contents

    On-device AI has moved from buzzword to a genuinely usable option. As API call costs increasingly eat into revenue, developers are turning their attention to local AI inference engines that, once downloaded, keep running without an internet connection. Right in the middle of that wave sits NobodyWho.

    NobodyWho is an open-source AI engine that runs AI models directly on the user’s device. Once the model file is downloaded, no network connection or API key is required afterward. Its goal is to port capabilities like chat, image understanding, and voice straight into apps or games. It is designed for easy integration across multiple development environments including Python, and input data never leaves the device.

    The point I find most meaningful about this project is the avoidance of pricing policy risk. Cloud APIs from OpenAI and Anthropic change prices frequently and impose usage limits. There have already been more than a few cases where token costs ate into revenue as services grew. An approach like NobodyWho that runs the model locally only costs the initial download, with near-zero variable cost thereafter.

    How On-Device AI Works — Download Once, Done

    NobodyWho’s execution flow is simple. At the first run, the model file is downloaded to the device, and all subsequent inference is processed by that device’s CPU, GPU, and NPU. The text and images a user inputs are not transmitted to external servers, and responses are generated locally. The result is network latency that is effectively close to zero.

    There are specific cases where this architecture really shines. Workloads such as medical and legal tasks where data must not leave the device, and industrial tools that run in environments with unstable internet. The fact that what a user inputs is recorded nowhere is a decisive advantage in regulated environments like GDPR and HIPAA.

    Five Developer Comparison Axes

    When deciding whether to adopt on-device AI, the most frequently compared dimensions are cost, latency, and privacy. Here they are summarized in a single table.

    Category Cloud API On-Device AI (NobodyWho) Hybrid
    Upfront Cost Low Model file download Medium
    Operating Cost Per-token billing Near zero Varies by workload
    Network Latency 200ms to several seconds Effectively none Varies by case
    Data Sovereignty Transmitted to server Processed on-device Sensitive data stays local
    Offline Operation Not possible Supported Partially supported

    As the table shows, on-device AI holds the advantage across all three axes of cost, privacy, and latency. The trade-off is that it adds hardware requirements on the device and consumes storage equal to the model size. A 7B parameter model is roughly 4–5GB, and larger models take up more. Compatible models and the device list are available in the original NobodyWho article.

    Limits You Must See Before Adopting On-Device AI

    Honestly, local inference is not the answer for every workload. Model quality itself often falls short of top-tier cloud models even at the same parameter count. The gap is especially clear on tasks that demand strong reasoning or multilingual processing.

    Battery and thermals also matter. Running a 7B model continuously on a mobile device drains the battery quickly and makes the device noticeably hot. At the user experience design stage, it is wise to predefine when to route to local and when to cloud. As NVIDIA also pointed out the drift failure mode in which agents do not fully control themselves in its agent safety platform report published on September 28, 2026, on-device AI controls must go hand in hand with sandboxing and permission separation design.

    Practical Application Points

    • At the PoC stage, separate workloads where data sovereignty matters (notes, medical, legal) as candidates for local inference.
    • Pre-measure download time, storage footprint, and battery consumption per model size as a device matrix.
    • From day one, assume a hybrid that automatically routes non-sensitive tasks to the cloud and sensitive inputs to on-device AI.
    • When adding agent-style capabilities, keep sandbox and tool-call permission scopes in a separate specification.
    • Separate the model update channel from production environments and put a rollback-capable version management system in place.

    Hybrid Design Is the Realistic Answer

    There is always a gray zone between purely local and purely cloud. Engines like NobodyWho shine brightest as the tool that fills that gray zone. A user’s text input is first classified locally, and only when the classification is ambiguous is it sent to a larger model in the cloud.

    The key to this design is sensitivity-based routing. Prompts containing personal information are processed strictly locally, while general Q&A can go to the cloud. It is the most realistic pattern for capturing both cost and privacy at the same time. Local inference routing itself was also addressed in a different way in the NVIDIA PAIR launch analysis.

    What to Try Right Now

    • Check the release notes and the compatible device list in the NobodyWho GitHub repository.
    • Pick the single workload with the largest API call cost in your current service and mark it as a local-conversion candidate.
    • Tabulate the average input/output token count and response time SLO for that workload.
    • Shortlist two to three open-source models at the same parameter tier and run device-specific benchmarks.
    • Write a sandbox and permission separation guide for the internal PoC environment first, then integrate the model.

    Who Should Adopt Now, and Who Should Wait

    There are roughly two types of teams for which adopting on-device AI right now is worthwhile. Teams whose API costs are eroding revenue, or teams for whom data leaving the device is itself a business risk. On top of that, teams building products that must work in offline environments get an even higher priority.

    Conversely, services whose core is large context or complex reasoning are still better off using top-tier cloud models. The set of workloads replaceable with sub-13B on-device models is limited. Local inference delivers results when treated as a partial-replacement strategy rather than a full one.

    Frequently Asked Questions

    What models does NobodyWho support?

    It is designed to support a broad range of mainstream open-source LLM formats along with image and voice models. The fastest way to check the concrete compatibility list is in the repository’s release notes. Device requirements may differ depending on the model file format.

    Can on-device AI eliminate API costs entirely?

    Migrating every workload to local is difficult. A realistic approach is to convert gradually, starting with low-sensitivity and low-complexity tasks. Meaningful savings require re-drawing the cost structure with a hybrid setup as a premise.

    Is data really not transmitted externally?

    NobodyWho itself does not send user input outside. However, telemetry or analytics modules may be added during app integration, so it is safer to verify the code paths directly before deployment.

    Does on-device AI work on mobile too?

    It depends on model size and device specs. 1–2B parameter models run on mid-to-high-tier mobile chipsets, but 7B and above bring significant heat and battery drain. Pre-validation based on a device matrix is essential.

    The fact that open-source AI engines like NobodyWho are growing means that ‘when local, when cloud’ is no longer a question of options but a design problem. In my view, the most important thing right now is not choosing the model, but reaching a team-internal consensus on the criteria that divide on-device AI from cloud. Once that criterion is set, NobodyWho becomes the tool that translates that criterion into code.

    Source Reference

    This article was prepared after reviewing the following original: geeknews — NobodyWho – An On-Device Inference Engine for Embedding Local AI in Apps and Games

  • IP Warranty Comparison of 5 Coding Agents — How Far Does the Liability of GitHub Copilot, Kiro, Cursor, Devin, and Windsurf Extend?

    IP Warranty Comparison of 5 Coding Agents — How Far Does the Liability of GitHub Copilot, Kiro, Cursor, Devin, and Windsurf Extend?

    Key Takeaways

    • When Microsoft updated the GitHub Copilot terms on April 3, 2026, it removed the previous structure in which a single administrator setting could void the warranty. The current “Required Mitigations” page lists no additional required actions for the GitHub product family.
    • The central interpretive issue surrounding the Copilot warranty is the “unmodified” expression: any code edited even once before release could fall outside the warranty, so legal review is essential.
    • AWS Kiro inherits the AWS Generative AI Warranty (Section 50.10) and advertises unlimited coverage for copyright claims. However, the warranty collapses if the input itself is infringing or if filtering features are disabled.

    Analysis

    Table of Contents

    When an enterprise considers adopting one of 5 coding agents, the first wall a procurement lead runs into is not licensing cost but the IP warranty clause. The enterprise AI coding agent comparison report published by MarkTechPost on September 26, 2026 placed GitHub Copilot, AWS Kiro, Cursor, and Cognition’s Devin/Windsurf on the same scale. This author finds that the questions the report puts to legal reviewers are the most practical ones.

    The 4 Questions Enterprises Ask About Coding Agents

    The report highlights four core questions. When IP infringement occurs, who is liable? Where are prompts and generated code stored? Can administrators receive audit logs? What is the actual cost when contracting at the 500-seat scale? To answer these, you have to dissect the vendor’s terms clause by clause.

    The Four Axes Used in the Comparison

    The evaluation was organized around four axes: warranty structure, data residency, administrator audit logs, and 500-seat TCO. Warranty is assessed on “what is covered, under what conditions, and to what extent”; data is assessed on storage region and retention period; logs are assessed on what is visible from the admin console; and cost is assessed on the sum of published pricing and negotiated variables.

    GitHub Copilot — What the April 3, 2026 Terms Change Means

    Microsoft updated the GitHub Copilot terms on April 3, 2026. The previous structure, in which a single administrator setting could void the warranty, has been removed, and the current “Required Mitigations” page lists no additional required actions for the GitHub product family. In practice, this change is favorable to enterprises because the hidden triggers that could erode the warranty have decreased.

    However, the word “unmodified” still gets in the way. It means any code edited even once before release can fall outside the warranty. In reality, all code goes through internal review and is modified. If this clause is applied as written, the warranty could effectively be empty, making legal review essential.

    AWS Kiro — The Two Sides of the Unlimited Section 50.10 Warranty

    AWS Kiro inherits AWS Generative AI Warranty Section 50.10. Its claim of unlimited coverage for copyright claims is the strongest among the five coding agents. There is, however, an exclusion clause that causes the warranty to collapse if the input is already infringing or if filtering features are disabled. The input-infringement exclusion refers to cases where externally licensed code imported for training is used as-is. The filter-disabling case is practically significant because filters are sometimes turned off for performance reasons during the PoC stage.

    Cursor — The Collision Between a Broad MSA and Narrow Exclusions

    Cursor’s MSA is nominally the broadest. It explicitly names “Suggestions” and separates the warranty cap from the usage-fee cap. Exclusion clauses activate for claims related to modification or unapproved combination. An interesting point is the gap between the enterprise and consumer terms. The consumer ToS, conversely, places the burden of indemnification on the user toward Anysphere. Even for the same company’s coding agent, responsibility flips depending on the license type. At the adoption stage, you must first confirm which terms apply.

    Cognition (Devin · Windsurf) — The Outlier That Disclaims Warranty

    Cognition is the only outlier among the five coding agents. Its MSA explicitly excludes generated outputs from warranty scope. After acquiring Windsurf on June 2, 2026, it changed the editor name to Devin Desktop and unified the price list and terms. The warranty disclaimer means the IP risk returns entirely to the user. Given that an agentic coding agent autonomously writes code and pushes to external repositories, this clause may be triggered more frequently than with other products. What stands out from a practitioner’s standpoint is that Cognition does not hide this fact and states it explicitly.

    Comparison Table of 5 Coding Agents

    Product IP Warranty Scope Key Exclusions Notes
    GitHub Copilot Generated outputs only Applies only to unmodified code April 3, 2026 terms change removed the admin trigger
    AWS Kiro Unlimited under Section 50.10 Input infringement, filter disabled Limited to copyright claims
    Cursor (MSA) Suggestions explicitly named, cap separated Modification, unapproved combination Consumer ToS places indemnification on the user
    Devin Generated outputs disclaimed All items Unified after the June 2, 2026 Windsurf acquisition
    Windsurf → Devin Desktop Same as Devin Same as Devin Unified price list and terms

    Limits of the Data Collected

    The original report focuses on the four questions, but the collected body text was truncated in the middle. Specific figures for the 500-seat cost table and detailed audit log items were not confirmed in the body. This article clarifies that the verification was based on the vendor’s official pages as of September 26, 2026. Since terms may change later, the latest terms at the time of legal review must be re-checked. Teams evaluating coding agent adoption are advised to obtain console and terms PDFs directly during the PoC stage.

    Practical Application Points

    • Review clauses containing conditional expressions such as “unmodified” with the legal team to determine whether they conflict with the in-house standard development workflow.
    • Verify data residency not from the marketing page but from the DPA body, checking the storage region and retention period.
    • During the PoC stage, log in to the admin console directly to confirm that the required events appear in the audit logs.
    • For agentic coding agents (e.g., Cognition), run a separate simulation of the scenarios in which the warranty disclaimer clause would be triggered.
    • Add the 500-seat TCO to a sheet together with volume discounts, multi-year commitments, and training costs, on top of the published price list.

    What to Do Right Now

    • Search the vendor’s terms for three keywords: “indemnify”, “warranty”, and “limitation of liability”.
    • Flag any responsibility-shifting expressions such as “unmodified”, “as-is”, or “your responsibility”.
    • Open the DPA and extract three fields: storage region, sub-processor list, and data retention period.
    • Request a PoC of the admin console and verify that prompt lookup, generated code download, and admin override items appear in the audit logs.
    • Obtain the 500-seat quote under two scenarios (single-year / 3-year commitment) and place them side by side in a sheet.

    Frequently Asked Questions

    Which clause should be examined first when adopting a coding agent?

    The IP warranty (Indemnification) clause. When infringement occurs, who bears the legal fees and damages is decided before cost. Price negotiation comes after that.

    Can the IP warranty be voided by administrator settings alone?

    In the past, GitHub Copilot had a structure in which certain administrator settings could void the warranty, but the April 3, 2026 terms change removed that trigger. However, conditional clauses such as “unmodified” remain.

    What should I do when the consumer and enterprise terms differ?

    Even with the same vendor, B2B and B2C terms can differ. Cursor is a representative case: the enterprise MSA has the vendor providing the warranty, while the consumer ToS places indemnification on the user toward the vendor. Confirm the applicable terms at the adoption stage.

    How should the actual 500-seat cost be calculated?

    The starting point is the official price list multiplied by the number of users. Add volume discounts, multi-year commitment discounts, training/onboarding costs, and the cost of management features such as SSO and audit logs to derive the real TCO.

    Source Reference

    This article was written after reviewing the following source: MarkTechPost — AI Coding Agents for Enterprise: IP Indemnity, Data Residency and 500-Seat Cost Compared

  • NVIDIA SoL-Pi Cuts Tokens by 49% — A Precise Breakdown of the 4 Harness Mechanisms Behind This AI-Discovered Coding Agent Layer

    NVIDIA SoL-Pi Cuts Tokens by 49% — A Precise Breakdown of the 4 Harness Mechanisms Behind This AI-Discovered Coding Agent Layer

    Key Summary

    • A joint research team from NVIDIA, NTU, and MIT has released “SoL-Pi,” an efficiency layer for the open-source Pi coding agent, distributed under the MIT license on GitHub NVlabs
    • SoL-Pi consists of four harness mechanisms, discovered by a separately trained AI running an auto-research loop across 535 environments
    • In evaluation on EdgeBench’s 51 tasks, recorded token traffic vs. Pi was reduced by 44.7–49.0%, with API costs dropping by roughly 33%

    All SoL-Pi figures come from the 40 held-out tasks out of EdgeBench’s 51, making them comparable indicators with overfitting risk controlled — though because the evaluation reflects all four mechanisms applied simultaneously, the individual contribution of each mechanism requires separate analysis

    Table of Contents

    The number that jumps out first is NVIDIA SoL-Pi cutting token traffic by 44.7–49.0% across EdgeBench’s 51 tasks. The layer, released on September 21 by a joint NVIDIA, NTU, and MIT research team, does not involve retraining the model. It is the result of layering four harness mechanisms — discovered by a separately trained AI running an auto-research loop across 535 environments — on top of the Pi coding agent.

    NVIDIA SoL-Pi structure diagram of four harness mechanisms

    The key point is that it reduced “tokens per task,” not “price per token.” The author finds this the most meaningful aspect. The same model handled the same tasks while the system prompt, tool calls, and context compression were simply rewritten. Scores held at roughly 94% of Pi’s on GPT-5.6 Sol and Opus 5, while API costs fell by about 33%.

    EdgeBench 51 Tasks: How the 49% Reduction Was Proven

    Start with the evaluation structure. Of the 51 tasks, 11 are frozen candidate 1-way acceptance tasks, and 40 are held-out. During the search phase, the optimizer cannot modify the acceptance rules of the 11 frozen tasks under any circumstances. And the results of the 40 held-out tasks are not fed back into the search loop.

    In other words, the critique that “evolved harnesses can overfit to search tasks” is blocked directly by the design. Thanks to this separation, the 49% reduction figure can be accepted as a comparable indicator. The verification pattern of separating search and evaluation data aligns with the trend that has repeatedly surfaced in the AI Research Automation September Milestones.

    How NVIDIA SoL-Pi’s Four Harness Mechanisms Work

    NVIDIA SoL-Pi’s four mechanisms focus on tool calls, context, and observation handling. Official naming is not yet finalized based on public materials, but the behavior patterns can be summarized along four axes: (1) branch compression at the tool-call stage, (2) dynamic slimming of the system prompt, (3) deduplication of observation values, and (4) early termination of failure loops. Separated analysis of each mechanism’s individual contribution has not yet been published.

    Structure of the Auto-Research Loop Across 535 Environments

    The loop has three stages: observe, propose, evaluate. It measures baseline Pi’s traffic across 535 environments, proposes a variant harness, and re-measures on the same environments. This cycle is repeated through a Ralph Loop implementation. Because independent reviewers score the search-phase results and held-out results separately, the average reduction rate of the four mechanisms is aggregated identically on both the search and evaluation sides.

    Deployment Requirements and Compatibility

    NVIDIA SoL-Pi runs on an unmodified Pi release with Pi 0.85.1 and Node.js 22.19 or higher. It is distributed under the MIT license on the GitHub NVlabs repository. Since existing Pi users simply layer it on, migration cost is minimal. The repository link and release notes can be found in the MarkTechPost original article.

    Risks and Limitations

    The evaluation is limited to two models: GPT-5.6 Sol and Opus 5. It has not been validated outside the coding agent domain. Because the result reflects all four mechanisms applied simultaneously, a separate ablation is required to know “how much would be saved by stripping out tool-call compression alone.” Whether the 1-way acceptance rule on the 11 frozen tasks would also hold outside the coding domain remains unknown.

    NVIDIA SoL-Pi vs. AWS Strands Harness

    AWS Strands harness, a frequently mentioned comparison target, targets general-purpose agents and reported about 28% lower costs than baseline on ALFWorld, ContextBench, GAIA, WebShop, τ²-bench, and Terminal-Bench 2.1 using the same model. NVIDIA SoL-Pi cut costs by about 33%, limited to coding agents. According to the Strands harness original article, the evaluation set structure also differs beyond the domain difference.

    Category NVIDIA SoL-Pi AWS Strands harness Pi Baseline
    Target Domain Coding agents General-purpose agents Coding agents
    Evaluation Set EdgeBench 51 tasks 6 sets including ALFWorld and GAIA EdgeBench 51 tasks
    API Cost Reduction ~33% ~28% Baseline
    Accuracy Retention ~94% (vs. Pi) Same-model baseline comparison 100% (baseline)
    License MIT Open source (policy to be confirmed) Original Pi policy

    The difference is large: NVIDIA SoL-Pi is domain-specific, while Strands is general-purpose. The question is less about which is better and more about which depends on the agent’s primary use. The broader trend of cost efficiency for agents can also be read alongside other organizational cases in the report that agent-based development handles 70% of PRs.

    What to Try Right Now

    • Clone SoL-Pi into your existing Pi 0.85.1 environment and run 5 tasks on an unmodified Pi with the layer applied.
    • Match Node.js to 22.19 or higher and measure the token count difference in advance.
    • Record average tokens per task, API cost per task, and pass rate before and after applying NVIDIA SoL-Pi in a table.
    • If you focus on non-coding agents, separately verify the compatibility and license of AWS Strands harness.
    • Among the four mechanisms, prioritize reviewing tool-call branch compression to gauge the reduction width.

    Practical Application Points

    If you are an existing Pi user, put NVIDIA SoL-Pi at the top of your adoption list. Thanks to unmodified compatibility, migration cost is nearly zero. However, you must first determine whether the 6% accuracy loss is acceptable for your workload, and only then set the scope of application. If you primarily handle non-coding domains, evaluate candidates like Strands harness and oh-my-pi together, and put license terms and benchmark reproducibility on your checklist. Avoiding tying your evaluation models to a single vendor is also a basic risk-diversification principle.

    Frequently Asked Questions

    Did NVIDIA SoL-Pi retrain the model?

    No. It is a harness layer layered on top of the existing Pi release. A separately trained AI discovered the four mechanisms across 535 environments and reduced token usage; the model weights themselves are unchanged.

    What does the 6% accuracy loss mean?

    It means that on GPT-5.6 Sol and Opus 5, roughly 94% of Pi’s score was retained. It is based on coding task pass rates and represents a 40-task held-out average. Adoption decisions will vary depending on business tolerance thresholds.

    Can it be used for general agents that aren’t coding agents?

    Published evaluation is limited to EdgeBench coding tasks. For general-purpose agents, AWS Strands harness or other efficiency layers must be evaluated separately.

    What is the first environment variable to check when adopting NVIDIA SoL-Pi?

    Verify that Pi version is 0.85.1 or higher and Node.js is 22.19 or higher. Then pre-measure token count and pass rate on up to 5 tasks to check for regressions.

    Reference Source

    This article was written after verifying the following original source: MarkTechPost — NVIDIA Introduces SoL-Pi: Auto-Research Loops That Cut Coding Agent Token Traffic by Up to 49%

  • Three Critiques of MCP: Why Agent Tool Integration Needs a Redesign

    Three Critiques of MCP: Why Agent Tool Integration Needs a Redesign

    Key Summary

    • As MCP (Model Context Protocol) servers each expose dozens of tools and schemas, the agent’s context window is becoming bloated.
    • Critiques argue that in today’s agent environments, where code execution and direct API calls have matured, most MCP servers are no longer essential.
    • As an alternative, the proposal is to replace MCP servers with HTTP APIs and CLIs that already offer standard interfaces.

    This analysis argues that MCP, which had established itself as the de facto standard for tool calling in the AI agent ecosystem, must reassess its utility as code execution and direct API call capabilities continue to grow.

    Table of Contents

    In September 2026, StepFun’s released Step 5 Preview was recorded as orchestrating roughly 950 web fetches in a single agent action. It features a 600B-parameter sparse MoE architecture with a deep and narrow design of 92 Transformer layers. What this case implies is more than a simple performance number. It marks a leap in the agent’s ability to handle tools and APIs directly. Against this backdrop, MCP critiques of MCP (Model Context Protocol)—long hailed as the standard for agent tool integration—have resurfaced.

    First Critique of MCP: Context Bloat

    The first issue at the center of MCP critiques is context bloat. A single MCP server defines dozens of tools and their corresponding schemas, and the agent must include them in the system prompt every time. As the number of servers grows, the context window swells exponentially, and token costs and response latency rise together.

    The losses felt in practice are more direct. When a model is in a state where there are “too many tools available,” the rate at which it selects the wrong tool increases, which in turn leads to retries and cost spikes. The author considers this the most substantive problem. Under the banner of “standardization,” each MCP server has emerged with its own schema, and as a result, agent developers are spending more time on context management than on actual integration work.

    Second Pillar of MCP Critiques: Direct Calls Become Realistic

    The second critique is more fundamental. As code execution and direct API call capabilities have grown, the argument is that the MCP intermediary layer has itself become a bottleneck. Today’s agents can spin up a shell, hit an HTTP API with a single curl call, process the response with jq, and pass the result to the next step. The Step 5 Preview’s 950 web fetches show that this direct-call pattern has become viable even at large workload scales. This is why the view that MCP’s promised “standard for tool calling” was, in fact, just one more wrapper around standard-interface HTTP API and CLI calls is gaining traction.

    Alternative: Switch Tool Exposure to HTTP APIs and CLIs

    The alternative proposed by MCP critics is surprisingly plain. Expose tools as HTTP APIs and CLIs, and let the agent call them directly through code execution. Authentication is handled via OAuth/API keys, and tool definitions are only loaded into context as function signatures when needed. This approach dramatically reduces context footprint while allowing tools to evolve freely without being tied to MCP spec changes.

    Item MCP Server Direct HTTP API Call Direct CLI Call
    Context cost High (tools/schemas loaded at all times) Low (defined only when needed) Low (shell call)
    Standardization level Dependent on MCP spec Mature standards such as OpenAPI POSIX/shell conventions
    Multi-step workflow Strength Moderate (direct implementation) Moderate (direct implementation)
    Authentication Handled internally by MCP OAuth/API key Environment variables/key files
    Maintenance Requires server update propagation Independent per endpoint Independent per binary

    As the table shows, MCP’s strength lies in “exposing multiple tools as a single bundle.” However, the core of the critique is that its standardization advantage is not large enough to justify that cost.

    A Balanced View After MCP Critiques: Where MCP Still Holds Value

    The claim that MCP is useless is premature. When you need to bundle and expose a non-standardized internal system to an agent in one go, MCP’s tool bundling is still valid. However, for workloads that are dominated by common SaaS APIs, CLIs, and web calls, it pays to skip MCP altogether. Ultimately, the answer is not a single solution but a workload-specific mix.

    Practical Choices After the MCP Critiques

    As of September 2026, agent tool integration has moved past the stage of “unifying the answer with a single protocol.” Choices diverge depending on who bears the context cost of tool definitions, how much standardization is required, and who manages authentication and authorization. The significance of MCP critiques is that they have reopened this choice.

    Practical Application Points

    • For MCP servers exposing more than five tools, consider switching to direct HTTP API calls first.
    • Monitor context cost: measure the input token share taken up by tool definitions on a weekly basis.
    • If the workflow ends in just one or two simple call steps, it is better to skip MCP’s intermediary layer.
    • Adopt a hybrid strategy as the default: bundle legacy internal systems with MCP, and separate external SaaS into direct calls.

    What to Try Right Now

    • Extract tool definitions from your currently running MCP server and measure the tokens consumed in context.
    • Redefine the same functionality as an HTTP API with an OpenAPI spec, and write code that lets the agent call it directly from the shell.
    • Simplify the authentication flow to OAuth/API keys, and reduce dependency on MCP-internal authentication.
    • Verify in logs whether context cost grows linearly as the number of tools increases.
    • Bundle only legacy systems lacking internal standards with MCP, and separate the rest into direct calls.

    Frequently Asked Questions

    Should I tear out MCP right away?

    No. MCP is still efficient when you need to bundle multiple internal legacy tools at once. For new integrations, it is reasonable to start with an HTTP API/CLI foundation.

    How much does context bloat actually affect cost?

    If tool definitions are sent with every request, cases have been reported where 20–40% of input tokens are consumed by tool metadata. Depending on the model and pricing plan, the difference can amount to several million won per month.

    What are the drawbacks of direct HTTP API calls?

    The agent must handle authentication, retries, and error handling itself. As multi-step workflows grow longer, code complexity rises quickly, so this approach is most effective when tool count is small and call frequency is high.

    Why are the 950 web fetches in Step 5 Preview important?

    Because they are evidence that an agent can reliably orchestrate large-scale tool calls without human intervention. Once this capability becomes widespread, the bottleneck in tool integration will shift from “which protocol” to “which code to write.”

    Reference Source

    This article was written after reviewing the following source: geeknews — Was MCP a flawed idea from the start?

  • K2 Horizon Analysis: IFM’s Six-Model Fleet from 0.9B to 375B Released Simultaneously Under Apache 2.0

    K2 Horizon Analysis: IFM’s Six-Model Fleet from 0.9B to 375B Released Simultaneously Under Apache 2.0

    Key Summary

    • Release scale: Six models—375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B—all simultaneously distributed under Apache 2.0 on Hugging Face (the 0.9B uses a smaller vocabulary)
    • Open stack included: Beyond model weights, the release ships the pretraining corpus, intermediate checkpoints, training code, configuration files, and detailed logs—IFM describes this as “the largest fully open-source release in AI history”
    • Training data: All six models were pretrained on approximately 20 trillion tokens, of which about 17% consists of explicit reasoning traces, and roughly 10 trillion tokens are synthetic data

    An analytical article examining what it means—for technology, licensing, and commercialization—when a large language model is released as a full-stack open-source project, and what it implies in practice to run the same architecture consistently from 0.9B to 375B.

    Table of Contents

    K2 Horizon lineup

    The fact that K2 Horizon released six models ranging from 0.9B to 375B under Apache 2.0 on the same day forces us to revisit what it means for a large language model to be open-sourced as a full stack. The bundle published in September 2026 by the Foundation Model Institute (IFM) under MBZUAI is not a single model but a “fleet” that ships the pretraining corpus, intermediate checkpoints, training code, configuration files, and detailed logs together.

    In the author’s view, the core of this announcement lies not in model size or benchmark scores but in the scope of openness. The fact that even the 0.9B model’s use of a smaller vocabulary is explicitly stated means that the same training pipeline can be repeated by varying only the scale. The ability to use an identical interface across sizes in practice becomes a prerequisite for unified operations.

    K2 Horizon Lineup: From 0.9B to 375B-A23B

    The lineup is divided into six models. The 0.9B uses a reduced vocabulary, while the 3.7B, 7B, and 32B are dense. The 36B-A4B and 375B-A23B are MoE configurations. Because all models share the same tokenizer and tool-calling interface, the same code can be run by switching only the model identifier.

    Model Total Parameters Active Type
    0.9B 0.9B 0.9B Reduced-vocab dense
    3.7B 3.7B 3.7B Local-inference dense
    7B 7B 7B General-purpose dense
    32B 32B 32B High-quality generation dense
    36B-A4B 36B 4B MoVA MoE
    375B-A23B 375B 23B Flagship MoE

    Because all six K2 Horizon models pass through the same synthetic-task generator and are trained with the same reasoning-trace ratio (about 17%), data alignment is applied consistently across the entire lineup. There is a high probability that a prompt format validated on the 0.9B will not break significantly when ported to the 375B-A23B.

    The Scope of the Open Stack Brought by K2 Horizon

    IFM used the phrase “the largest fully open-source release in AI history.” The K2 Horizon distribution bundle includes the pretraining corpus, intermediate checkpoints, training code, configuration files, and detailed logs. Compared with other open-source projects that release only weights and inference code, the scope is on a different level.

    The release of intermediate checkpoints is directly tied to reproducibility. While retraining the 375B-A23B from scratch is difficult, you can continue fine-tuning from an intermediate stage or run ablations. In effect, the surface needed to experiment by varying the synthetic-data ratio or the reasoning-trace ratio is now secured.

    Training Data Design: 20 Trillion Tokens, 100M+ Tasks

    All six models were pretrained on approximately 20 trillion tokens, of which about 17% consists of explicit reasoning traces, and roughly 10 trillion tokens are synthetic data. The research team generated more than 100 million unique synthetic tasks and progressively incorporated post-training data during the middle-training stage.

    The point practitioners should pay attention to is the scale of the synthetic tasks—the most striking number in the author’s view. 100 million is far beyond what can be hand-designed by humans. The structure has shifted from defining domain tasks and feeding them into the model to letting the model itself generate tasks while humans curate them. When examining the performance gap between the 36B-A4B and the 7B, the distribution of these 100 million tasks becomes the decisive variable.

    MoVA: A Second MoE Scaling Axis Built on Attention

    Among the K2 Horizon lineup, the 36B-A4B applies MoVA (Mixture-of-Value Attention). If MoE created a scaling axis by routing experts in the feed-forward layer, MoVA integrates routing into multi-head attention itself, adding a second scaling axis along the attention dimension. It is compatible with FlashAttention, GQA, and sparse attention, so it can be used without significantly overhauling existing serving stacks.

    What makes MoVA interesting is that it can push MoE’s parameter efficiency a step further. Whether the upper K2 Horizon lineup converges entirely on MoVA, or dense and MoVA coexist, is a point to watch.

    Tool-Calling Format: Markdown’s 18.5% Token-Efficiency Edge

    IFM trained tool definitions in three formats—JSON, XML, and Markdown—and set Markdown as the default for inference. Measurement results show that Markdown uses approximately 18.5% fewer tokens than JSON. Producing the same result with fewer tokens translates directly into lower latency and cost.

    The Markdown advantage grows as tool definitions get longer. Agents that expose 30 to 50 tools simultaneously are common, and in those cases, 18.5% is not a simple optimization but a near architectural decision. When designing a tool-calling system with K2 Horizon, it is simpler to let the model read and call Markdown tool definitions directly rather than building a separate router.

    Serving Ecosystem: Day-0 Support for vLLM, SGLang, and Ollama

    On the day of release, vLLM, SGLang, and Ollama received day-0 support, with FP8 and GGUF builds also provided. The supported range extends beyond NVIDIA to include AMD and Cerebras. According to the official IFM announcement, hosting is split across Compass, Cerebras, and Nebius APIs, plus the platform.ifm.ai gateway, so the same weights can be tested immediately on various stacks. Deployment-infrastructure choice is worth evaluating from the perspective of data-movement costs by AI chip architecture.

    Day-0 serving support is a “signal” from the announcement. New models typically stabilize only after community patches, but K2 Horizon works on three inference engines at the moment of release. This shows IFM’s strategic choice to ship open-source models as a “stack.”

    Practical Application Points

    Size selection depends on call frequency and acceptable response latency. The 0.9B and 3.7B are efficient for classification, routing, and simple transformations; the 7B fits general-purpose reasoning; and the 32B is best for high-quality generation and summarization, where resource-to-output ratio matters. The 36B-A4B is well suited to agent routers or multi-tool-call workloads, leveraging its 4B active parameters per token. The 375B-A23B carries a significant operational cost, so the safer approach is to validate prompts and tool shapes on the 0.9B first, then scale up gradually.

    What to Try Right Now

    • Download the K2 Horizon 0.9B or 3.7B weights from Hugging Face and spin up a local vLLM server.
    • Write the same tool definition in both JSON and Markdown, and compare token counts and response times.
    • Pull the 36B-A4B and run at least five multi-tool-calling scenarios with MoVA routing enabled.
    • Inspect the synthetic-task distribution in the pretraining-corpus metadata, pick the cluster closest to your in-house domain tasks, and start fine-tuning.

    Frequently Asked Questions

    Why does the K2 Horizon 0.9B model use a different vocabulary?

    It is targeted at edge devices, so the vocabulary was reduced to lower memory usage. It goes through the same training pipeline but adopts a reduced vocabulary tailored to the inference environment.

    Can it be commercialized under the Apache 2.0 license?

    Because the weights, code, and data are all released under Apache 2.0, it can be used as-is in commercial services. Note, however, that the same license-notice requirement must be followed when redistributing.

    How is MoVA different from conventional MoE?

    If MoE routes experts in the feed-forward layer, MoVA integrates routing into multi-head attention itself. Its defining feature is compatibility with FlashAttention, GQA, and sparse attention.

    Why is Markdown tool calling 18.5% more efficient than JSON?

    Markdown uses fewer metacharacters such as braces, quotes, and tags, so fewer tokens are needed to express the same meaning. The advantage grows as tool definitions get longer.

    The question K2 Horizon’s release poses is not simply “Is this yet another open-source model launch?” The fact that a fleet running consistently on the same pipeline from 0.9B to 375B has been released under Apache 2.0 shows that the axis of open-source LLM competition is shifting from weights to the “stack”—including data, code, and intermediate states. What licensing standards this change produces, and how commercial vendors respond, will be the deciding factor over the coming quarters.

    Reference

    This article was written after reviewing the following source: MarkTechPost — IFM Releases K2 Horizon: Six Apache 2.0 Models From 0.9B to 375B

  • MiniCPM5-2B: OpenBMB’s 2.5B Model Hits 53.9 Average Across 34 Benchmarks, Redefining On-Device LLM Standards

    MiniCPM5-2B: OpenBMB’s 2.5B Model Hits 53.9 Average Across 34 Benchmarks, Redefining On-Device LLM Standards

    Key Summary

    • MiniCPM5-2B is a dense causal language model with 2,516,756,480 parameters (1,981,982,720 excluding embeddings), adopting 42 layers and grouped-query attention (16 query heads, 2 key/value heads).
    • The native context window is 131,072 tokens, and because the architecture is standard LlamaForCausalLM, it runs on vLLM, SGLang, Transformers, llama.cpp, Ollama, LM Studio, MLX, and FlagOS without custom kernels or model code forks.
    • Licensed under Apache 2.0, it is benchmarked against same-class peers such as LFM2.5-2.6B, Qwen3.5-2B, and Gemma-4-E2B-it, with references including Qwen3.5-4B (51.1), granite-4.2-3B (42.7), and LFM2.5-2.6B (33.2).

    Analysis

    Table of Contents

    MiniCPM5-2B debuted on September 7 with a spec sheet of 2,516,756,480 parameters and a native context window of 131,072 tokens. Its 53.9 average across 34 benchmarks is 2.8 points ahead of the best same-lineup baseline, Qwen3.5-4B at 51.1. With less than half the parameters, it posts a higher overall score. For the author, the real significance of this model lies in numerically proving that “small doesn’t mean incapable.”

    MiniCPM5-2B Architecture: Standard Llama Fork for Instant Compatibility with 8 Inference Engines

    MiniCPM5-2B is a dense causal language model with 42 layers and grouped-query attention (16 query heads, 2 key/value heads). Excluding embeddings, it has 1,981,982,720 parameters. The key point is that the architecture is standard LlamaForCausalLM. Eight inference engines—vLLM, SGLang, Transformers, llama.cpp, Ollama, LM Studio, MLX, and FlagOS—run it as-is, without custom kernels or model code forks. Combined with the Apache 2.0 license, this makes it one of the lowest-friction options for teams looking to deploy a 2–3B-class model in real services.

    What the 53.9 Average Across 34 Benchmarks Means for MiniCPM5-2B

    The single score of 53.9 alone doesn’t tell the full story. Broken down by domain, MiniCPM5-2B’s character becomes clearer. The same-class comparison set includes LFM2.5-2.6B, Qwen3.5-2B, and Gemma-4-E2B-it, while reference points include Qwen3.5-4B (51.1), granite-4.2-3B (42.7), and LFM2.5-2.6B (33.2). The overall 53.9 is the highest figure in this reference pool.

    Benchmark MiniCPM5-2B Best Baseline Gap
    34-benchmark overall average 53.9 Qwen3.5-4B 51.1 +2.8
    LiveCodeBench v6 69.1 56.4 +12.7
    SWE-bench Verified 46.4 33.6 +12.8
    τ²-Bench Telecom 97.1 — —
    BFCL v4 66.6 — —
    τ³-Bench Banking 20.8 6.8 +14.0
    NoLiMa 68.1 43.5 +24.6
    AA-LCR 59.0 61.0 −2.0
    LongBench v2 43.7 47.3 −3.6
    MMLU-Pro 70.8 78.0 −7.2
    Humanity’s Last Exam 8.9 9.9 −1.0

    Strengths: Dominating the Baseline in Code Reasoning and Tool Use

    The largest gap is in code. MiniCPM5-2B scored 69.1 on LiveCodeBench v6 and 46.4 on SWE-bench Verified, beating the baseline of 33.6 by 12.8 points. A 2.5B model hitting 46.4 on SWE-bench means it achieves nearly a 50% success rate on tasks that go beyond simple code completion to include multi-turn debugging.

    Tool use is even steeper. The τ²-Bench Telecom score of 97.1 is essentially near-perfect, and BFCL v4 at 66.6 and τ³-Bench Banking at 20.8 (baseline 6.8) also far exceed what you’d expect from a 2.5B-class model in function calling and routing. These numbers most directly demonstrate that MiniCPM5-2B is a credible candidate for real agent workloads.

    Trade-offs: Long-Context Variability by Benchmark and General-Knowledge Limits

    It doesn’t lead on every metric. Long-context results vary by benchmark. On NoLiMa (68.1 vs. baseline 43.5) it leads by 24.6 points, but on AA-LCR (59.0 vs. 61.0) and LongBench v2 (43.7 vs. 47.3) it actually trails. Even with a 131K-token window, there are areas where measured accuracy falls short of the baseline.

    General knowledge shows a similar pattern. MMLU-Pro (70.8 vs. 78.0) and Humanity’s Last Exam (8.9 vs. 9.9) are both slightly behind. It’s natural that a model with roughly half the parameters can’t beat a 4B-class model on broad factual recall, so it’s more accurate to view MiniCPM5-2B not as an “all-rounder” but as a “coding- and agent-specialized compact model.”

    Training Pipeline: UltraData → 400B SFT → JustRL II → 16-Expert Distillation

    Behind these scores lies a staged training design. Base training ran in stable and decay phases using UltraData’s hierarchical data management, followed by 400B-token deep-thinking SFT after mid-training. The RL stage covered four separate domains—math, code, agentic, and writing—with a critic-based teacher called JustRL II guiding the training. The pipeline finishes with on-policy distillation that merges 16 expert models into a single checkpoint.

    The model card separately labels rows from Artificial Analysis and internal reproductions, proactively flagging evaluation-consistency issues—a sign that reproducibility was a deliberate priority.

    What to Try Right Now

    • Download MiniCPM5-2B via Ollama or llama.cpp and measure inference latency directly on a Mac or a single RTX 3090 GPU.
    • Pick 30 coding problems from your own domain, run multi-turn debugging, and compare response quality against Qwen3.5-2B and Gemma-4-E2B-it.
    • Build 10 BFCL-style function-calling scenarios and re-evaluate tool-use accuracy against your in-house data.
    • Stuff 50K tokens of internal documents into the 131K context and check retrieval and summarization accuracy yourself, just like NoLiMa, AA-LCR, and LongBench.
    • Confirm Apache 2.0 applicability with your legal team and verify there are no licensing risks around your in-house fine-tuning data.

    Practical Application Notes

    • When evaluating 2–3B-class models, don’t rely on a single overall average—weight domain benchmarks like LiveCodeBench, SWE-bench, and BFCL into your scoring.
    • If you’re considering MiniCPM5-2B for agent workloads, validate function-calling routing separately. Don’t assume the τ²-Bench 97.1 score generalizes to your case.
    • If long context is critical, judge models on measured benchmarks like AA-LCR and LongBench rather than the 131K window size alone.
    • For on-device deployment, note that the grouped-query attention’s 2 key/value head configuration keeps attention cache memory small but can lower batch throughput, so measure per scenario.
    • Once licensing is cleared, the practical sequence is to stand up a first baseline with vLLM or SGLang, then bolster weak areas with JustRL II-style RL fine-tuning.

    Frequently Asked Questions

    Can MiniCPM5-2B be used directly in commercial projects?

    The Apache 2.0 license permits both commercial use and redistribution. However, responsibility for the model’s outputs rests with the user, so it’s safer to decide on adoption after domain-specific evaluation.

    A 2.5B model scores higher than Qwen3.5-4B—is it really usable?

    The 53.9 average across 34 benchmarks is real. But the lead is concentrated in code and tool use, while it trails by more than 7 points on general-knowledge benchmarks like MMLU-Pro. You’ll need to re-weight the evaluation by domain.

    Does MiniCPM5-2B actually run on-device?

    Yes—because it’s standard LlamaForCausalLM, it runs out of the box on llama.cpp, Ollama, LM Studio, and MLX. In practice, though, it’s more realistic to use 8K–32K slices than the full 131K context on mobile.

    Is there a reason to migrate from previous MiniCPM models?

    Context window and code/agent scores have both been lifted compared to the previous generation. For new projects, start with MiniCPM5-2B; for existing systems, compare token usage and response latency before migrating gradually.

    The original source material was verified via MarkTechPost’s MiniCPM5-2B coverage. Comparison-group information referenced the same outlet’s IFM K2 Horizon launch article.

    Reference Sources

    This article was written after checking the following original sources: MarkTechPost — OpenBMB Releases MiniCPM5-2B: A 2.52B Dense Model Averaging 53.9 Across 34 Benchmarks and Built to Run On Device

  • AI Research Automation: September Milestone — How OpenAI’s Internal Report Reveals Coding Agents Becoming Everyday Infrastructure

    AI Research Automation: September Milestone — How OpenAI’s Internal Report Reveals Coding Agents Becoming Everyday Infrastructure

    Key Summary

    • An investigation finds that coding agents have deeply penetrated the daily workflows of OpenAI researchers, running continuously throughout the day in the form of multiple concurrent sessions
    • Since the introduction of agents, both the amount of code written by researchers and the number of experiments performed have increased
    • The nature of tasks delegated to agents is observed to be shifting beyond simple assistance toward more complex research assignments

    Analysis – An in-depth review cross-verified with primary sources on how the AI R&D process itself is being accelerated by AI agents

    Table of Contents

    A report recently disclosed that AI research automation is already part of daily life inside OpenAI. The piece titled “Research acceleration: The view inside OpenAI” is a document that unpacks with primary data how OpenAI researchers use coding agents, as can be confirmed directly from the OpenAI official report. What the author found most significant was not a simple showcase of use cases, but the fact that the same piece explicitly published the automated intern and automated researcher roadmap.

    A Different Kind of Colleague Inside the Lab

    The first piece of information in this report is that OpenAI researchers keep coding agents running all day long. The pattern of opening multiple sessions simultaneously and working on other tasks while the model writes code has become routine. People often talk about it at the level of “trying out an agent,” but analysis suggests that inside OpenAI it has already settled into something closer to ‘infrastructure for delegating multiple tasks at once.’

    The results surface in two metrics. The number of experiments run increased even though the amount of code researchers wrote directly did not decrease, and the nature of tasks delegated to agents also changed. The explanation is that the work has moved beyond the level of ‘fix this one function’ to defined research assignments like ‘test this hypothesis.’ The center of gravity appears to have shifted from simple assistance to autonomous work closer to that of an assistant.

    AI Research Automation Roadmap: September, and March 2028

    Two dates are stamped in the report. One is the ‘automated research intern’ that the company aims to secure by September of this year, and the other is the ‘automated AI researcher’ that will advance deep learning and alignment research on its own under human supervision by March 2028. The former refers to a system capable of performing defined research assignments over several days under human direction, and the latter refers to a system that can steer research direction with little to no human hands-on involvement.

    What stands out from a practitioner’s perspective is that the bar for ‘automated intern’ is not ‘an AI that writes code’ but ‘an AI that receives research assignments.’ This means the unit of delegation—hypothesis definition, experimental design, and result interpretation bundled together—is already internally valid. The fact that this bar has been met suggests that the next stage of AI research automation is not simple coding assistance but the work bundle of a single researcher.

    The Cumulative Curve of AI Research Automation

    OpenAI Chief Scientist Jakub Pachocki’s “An alien mind” post takes this flow back into the past. The explanation starts from the point in mid-2023 when the RLSlow project first confirmed the scalability of training reasoning models, and then describes how reasoning models like the o-series came to sit on top of that foundation. Pachocki diagnoses that reasoning language models are spreading rapidly across the broader economy and into the cybersecurity domain.

    Reading it through, you get the sense that the September milestone did not appear out of nowhere. A feedback loop in which reasoning models write code and that code in turn trains reasoning models has been accelerating over the past two to three years, and the term ‘automated intern’ emerged at the end of that loop. The progress of AI research automation is more naturally interpreted not as a discrete event but as a point on a cumulative curve.

    Issues: Governance, Pacing, and Control

    The reason this roadmap touches governance issues rather than being a simple engineering milestone is that the more an automated researcher decides to accelerate, the more alignment research is also accelerated. Although the phrase ‘human supervision’ appears multiple times in the report, as the rate of acceleration rises, the meaning of a single unit of supervision can become lighter. Model development pacing, internal control structures, and the timing of external disclosure—these three are likely to be the key issues over the next one to two years.

    There is also a point of contact with discussions of democratic control over AGI. If an automated researcher actually starts proposing research directions, the question of who holds the authority to decide ‘why are we doing this research’ arises. Even though the flow originated inside OpenAI, if the response from outside academia and the policy community is slower than the technology, the control vacuum could lengthen. The heaviest part of this report is that AI research automation immediately translates into a speed problem of research governance.

    Summary of Issues

    • The ‘unit’ of an automated intern is a research assignment rather than code—the very definition of AI research automation is changing.
    • The March 2028 milestone reopens the question of what ‘supervision’ means, rather than the question of ‘speed.’
    • When alignment research and capability research accelerate at the same pace, the gap between external control and internal control is the core risk.

    What to Do Right Now

    • Measure and record the number of agent sessions you keep open simultaneously for a week—the difference between casual use and real use.
    • Classify your team’s delegated work into two categories, ‘simple assistance’ and ‘defined assignments,’ and look at the ratio—to gauge the next stage of AI research automation adoption.
    • Extend your alignment and safety review checklist to include ‘hypotheses proposed by agents’—to prepare for post-September scenarios.
    • Separately tag and track PRs and experiments produced by agents into a tracking pipeline—to establish a baseline for the automation ratio.
    • Share the ‘automated researcher’ scenario with your governance lead in advance and simulate one round of decision-making delay—to gauge the length of the control vacuum.

    Frequently Asked Questions

    How is an automated research intern different from a typical coding agent?

    The automated research intern defined by the OpenAI report is not at the level of ‘writing a function,’ but a unit that performs a research assignment directed by a human over several days and reports back the results. Hypothesis formulation and experimental design are delegated as one bundle.

    Why is the March 2028 milestone important?

    It is a declaration to build a system by that date that advances deep learning and alignment research on its own under human supervision. It is significant less for the speed itself than for the fact that ‘the actual weight of the word supervision’ may be shaken.

    How does the RLSlow project connect to the current flow?

    In mid-2023, the scalability of training reasoning models was first confirmed in RLSlow, and reasoning models like the o-series came to sit on top of that. The September milestone is a point on that cumulative curve, not a sudden turning point.

    If you are already using agents, what more should you do?

    Measuring usage, classifying delegated work, and establishing a baseline for the automation ratio are actions that can be taken immediately. Without this data, designing the next stage of AI research automation will open a governance vacuum first.

    Reference Originals

    This article was written after confirming the following original source: OpenAI Blog — Research acceleration: The view inside OpenAI

  • NVIDIA PAIR Launch Analysis: How This Local Inference Router Wakes Up 5 Idle GPUs at Once

    NVIDIA PAIR Launch Analysis: How This Local Inference Router Wakes Up 5 Idle GPUs at Once

    Key Summary

    • PAIR is not a new inference engine but a virtual inference router that distributes traffic across existing Ollama/LM Studio engines on a local network
    • Released as public beta v0.1.1, with signed installers available for Windows, macOS, and Linux; the full source code is published on GitHub under the Apache 2.0 license
    • PAIR does not introduce a new cluster-specific API; instead, it proxies Ollama-compatible, LM Studio-compatible, and OpenAI-compatible endpoints, minimizing changes to existing agent harnesses

    Analysis

    Table of Contents

    One RTX on your desk, a Mac mini in the study, a DGX Spark in the living room. NVIDIA PAIR ties these scattered GPUs together so you can use them as a single endpoint. The key point first: NVIDIA PAIR is not a new inference engine. It is a virtual layer that distributes traffic across existing engines like Ollama and LM Studio on a local network.

    You might first wonder, “Why build a separate router?” In my view, the answer lies in the growth of multi-agent workflows. When the number of sub-agents grows to 5-10, a single machine’s GPU quickly becomes a bottleneck, and agent harnesses have repeatedly been asked to switch endpoints. NVIDIA PAIR proxies all three endpoint types – Ollama, LM Studio, and OpenAI-compatible. This means you can distribute traffic while barely touching your existing code.

    NVIDIA PAIR Distribution – Signed Installers and Apache 2.0

    Public beta v0.1.1 has been distributed as signed installers for Windows, macOS, and Linux. The full source code is published on GitHub under Apache 2.0, putting even organizations that cannot use commercial builds directly on the evaluation table. The specifications and download links can be confirmed in the initial MarkTechPost report.

    Node Management – mDNS Discovery, 6-Digit PIN, and mTLS

    NVIDIA PAIR first detects devices on the same subnet via mDNS. If discovery fails, IP addresses can be added manually, and pairing is completed with a single 6-digit PIN. After that, traffic is encrypted with generated-certificate-based mTLS. The fact that plaintext exposure risk is reduced when running on an office LAN without a VPN is attractive to operators.

    From a practitioner’s perspective, the standout feature is setup automation. Remote engine installation and model downloads can be triggered on paired nodes. The manual work of downloading a 30GB embedding model to five machines one by one disappears. Once the operator handles the first boot on one machine, NVIDIA PAIR fills in the rest.

    NVIDIA PAIR Compatibility – The No-Harness-Change Strategy

    It is significant that NVIDIA PAIR does not bring a new cluster-specific API. If your agent harness already knows Ollama or OpenAI endpoints, you just point it at the router address and it works as-is. The same direction is read in our article on extending local agent workflows.

    Endpoint Underlying Engine Representative Use Case
    Ollama-compatible Ollama Legacy agents, custom tools
    LM Studio-compatible LM Studio Desktop GUI workflows
    OpenAI-compatible Multiple backends LangChain, LlamaIndex families

    Performance Implications – The 5-Sub-Agent Demo

    In the reported 5-sub-agent demo, a task that took an average of 18 minutes on a single RTX Spark node was reportedly reduced to around 8 minutes with multi-node distribution. The trailing comparison context for the reported figures has not been verified, but the evidence that “workflow-level time” decreased is meaningful. This means there is now room to maintain responsiveness while increasing the number of sub-agents.

    However, these figures are the result of a specific model and prompt combination. For practical adoption, it is safer to run microbenchmarks with your own workflow. The view that NVIDIA PAIR’s value lies not in buying new GPUs but in reviving machines already on your desk is realistic. From a data governance perspective, it is also meaningful in that more traffic stays local.

    Practical Application Points

    • Prioritize an adoption path that keeps existing Ollama and LM Studio instances in place and simply layers the router on top.
    • Align the flow where 6-digit PINs and mTLS certificates are auto-issued with your internal security guidelines in advance.
    • Pre-define the candidate node list for distribution based on the number of sub-agents and model sizes in your multi-agent workflow.
    • Before enabling the remote model download trigger, verify there are no conflicts with internal proxy and bandwidth policies.

    Try It Right Now

    • Download the v0.1.1 release notes from GitHub and verify the signed installer hash.
    • Launch Ollama or LM Studio on one desktop machine and confirm that requests are proxied through the NVIDIA PAIR router address.
    • Check whether the second node on your LAN is auto-discovered via mDNS, and if not, test manual IP addition as a fallback.
    • After 6-digit PIN pairing, verify via logs that mTLS certificates are properly issued on both nodes.
    • Point your frequently used agent harness at the OpenAI-compatible endpoint and measure the response round-trip.

    Frequently Asked Questions

    Is NVIDIA PAIR a new inference engine?

    No. NVIDIA PAIR itself does not run models. It is simply a router that finds existing Ollama and LM Studio engines and sends traffic to them. Therefore, the models and prompts you already use remain unchanged.

    Do I need to learn a new cluster-specific API?

    No, you don’t. Since it proxies Ollama, LM Studio, and OpenAI-compatible endpoints, your agent harness only needs a one-line change to point at the router address.

    Does it work in an office with external internet blocked?

    Yes. Since all node-to-node communication is handled by mDNS and mTLS, you can operate it in a closed environment within your LAN. However, the initial model download will need to go out to the internet at least once.

    What is the licensing burden?

    The source on GitHub is released under Apache 2.0. You can compile, modify, and redistribute it internally, with only the obligations to preserve copyright notices and document changes.