Granite 4.2 LLMs: How They're Built

IBM’s Granite team has officially unveiled Granite 4.2, a significant evolution in their open-weights language model portfolio. Moving beyond the role of simple instruction-following assistants, the 4.2 series introduces a sophisticated reasoning-first architecture. Available in 3B, 8B, and 30B parameter sizes, these models are designed to handle complex deliberation, native tool calling, and agentic workflows, all while maintaining an Apache 2.0 open-source license.

"Granite 4.2 is our first family of dense, decoder-only reasoning LLMs, released in three sizes: 3B, 8B, and 30B. Each model is pre-trained from scratch on roughly 15T tokens with a five-phase strategy that extends the context window to 512K tokens."

Architectural Foundations

The Granite 4.2 family utilizes a dense, decoder-only transformer architecture optimized for both performance and scalability. By prioritizing a clean, efficient design, the team has ensured that these models can be deployed across a variety of hardware configurations, from local workstations to massive GPU clusters.

Core Technical Specifications

The models share a unified architectural blueprint, utilizing Grouped Query Attention (GQA) with 40 attention heads and 8 KV heads to balance inference speed with memory efficiency. Key technical choices include:

• Position Embeddings: Rotary Position Embeddings (RoPE) with a base of 10,000,000.
• Activation Function: SwiGLU, providing superior non-linear mapping for the MLP layers.
• Normalization: RMSNorm (ε = 1e-5) for stable training at scale.
• Precision: Native bfloat16 support.
• Embedding Strategy: Decoupled input and output embeddings to improve representational capacity.

The 3B, 8B, and 30B variants scale in complexity, with the 30B model utilizing 64 layers and a massive 32,768 MLP hidden size, ensuring it remains competitive in high-stakes reasoning tasks.

The Five-Phase Pre-Training Strategy

The journey to Granite 4.2 began with a foundational pre-training phase spanning approximately 15 trillion tokens. The team employed a rigorous five-phase curriculum designed to transition the models from raw data ingestion to high-fidelity reasoning.

1. Foundational Training (Phases 1-2): Establishing core linguistic and world knowledge. 2. Data Annealing (Phases 3-4): Mid-training refinement using progressively higher-quality, curated datasets. 3. Long-Context Extension (Phase 5): A specialized phase dedicated to scaling the context window to a massive 512K tokens, enabling the models to process extensive documents and long-running agentic trajectories.

Supervised Fine-Tuning (SFT) and Quality Control

To transform raw base models into capable assistants, the team curated a massive SFT corpus totaling roughly 100 billion tokens. The mixture is strategically balanced between agentic (31.6%) and non-agentic (68.4%) data.

Data Composition

• Agentic Corpus (31.6%): Heavily focused on software engineering (69%), tool calling (12.1%), and terminal operations (8.0%).
• Non-Agentic Corpus (68.4%): Covers instruction following, mathematics, coding, and multilingual capabilities.

The team implemented a strict quality control pipeline. Using GPT-OSS-120B and Gemma 4 as automated judges, they filtered out hallucinated content, invalid tool interactions, and low-quality samples. Furthermore, global deduplication using SHA-256 hashes ensured that the training data remained diverse and free of redundant noise.

The Multi-Stage Reinforcement Learning Pipeline

Perhaps the most innovative aspect of Granite 4.2 is its multi-stage, multi-environment reinforcement learning (RL) pipeline. Rather than relying on a single training pass, the team utilized a "staged curriculum" that builds capabilities incrementally.

The RL Ladder

• Foundational RL: All models undergo RLVR (Reinforcement Learning with Verifiable Rewards), focusing on math, code, and logic.
• Skill Boosters: Targeted training to sharpen specific capabilities like instruction following and competitive coding.
• Agentic RL (8B and 30B only): A specialized block where models learn to operate in real-world environments, including software engineering sandboxes, live terminal shells, and web-search interfaces.
• RLHF: The final alignment phase, which optimizes for human preference and safety while applying a "reasoning-length penalty" to prevent verbosity.

The training utilizes asynchronous GRPO (Group Relative Policy Optimization). This allows the generation workers and the trainer to operate in parallel, significantly increasing throughput without sacrificing the integrity of the policy updates.

Agentic Infrastructure: NeMo-RL and NeMo-Gym

The complexity of training an agentic model requires a robust infrastructure. IBM’s solution relies on two primary open-source components:

• NeMo-RL: Manages the training loop, utilizing Megatron-Core for distributed training and vLLM for high-speed generation.
• NeMo-Gym: Acts as the interface for environments. Whether it is a software repository, a terminal, or a web browser, NeMo-Gym provides a uniform interface that allows the model to interact with the world as if it were a simple verifier.

Performance and Benchmarking

Granite 4.2 demonstrates significant gains across the board. In agentic coding, the 30B model achieves a 57.00% resolve rate on SWE-Bench Verified, while the 8B model shows remarkable efficiency for its size.

Key Performance Highlights:

• Reasoning: The 30B model achieves an 89.17% score on AIME25 and 89.17% on HMMT Feb25, showcasing top-tier mathematical reasoning.
• General Agentic Use: The models excel in tool-use benchmarks like BFCL (v4) and τ³-bench.
• Long Context: The 30B model maintains high performance on RULER benchmarks at both 64K and 128K context lengths.

Deployment and Quantization

IBM has prioritized accessibility by providing multiple quantized variants. Developers can deploy these models using:

• FP8/FP4: Optimized for high-performance inference using LLM Compressor.
• GGUF: Fully compatible with llama.cpp for local, memory-efficient deployment.

The models are served through an OpenAI-compatible endpoint, making them "drop-in" replacements for existing agentic harnesses like OpenHands, OpenCode, and Pi.

"Every model has a thinking / non-thinking switch, a low-effort thinking mode that spends a short reasoning budget on easy questions, and native tool calling."

Conclusion

Granite 4.2 represents a shift toward models that do not just predict the next token, but actively deliberate. By combining a massive, high-quality pre-training phase with a sophisticated, multi-stage RL curriculum, IBM has created a versatile toolset for developers building the next generation of autonomous agents. With its Apache 2.0 license and seamless integration with existing agentic frameworks, Granite 4.2 is poised to become a staple in the open-source AI ecosystem.