miniTE Technical Report: Stable and Efficient Reinforcement Learning with Native NVFP4

TL;DR

• Problem and diagnosis · Native NVFP4 RL can become unstable late in training. We examine quantization, activation scaling, KV caches, and execution paths, then study how their errors affect updates.
• Two approaches · TRIAGE controls residual mismatch risk through update direction. miniTE reduces numerical execution differences and optimizes low-precision computation. We evaluate them separately.
• Measured gains · On 8×B300, miniTE with BF16 backward achieves approximately 1.33–1.37× BF16 speed for complete learner updates. A separate fixed-output serving test reaches approximately 1.64× throughput.
• Open questions · Weight conversion and synchronization limit end-to-end gains. Matching operators and obtaining a small gap do not guarantee BF16-level learning; error tolerance, update rules, and final outcomes need further study.

Abstract

Native NVFP4 reinforcement learning must remain stable under quantization and execution error while delivering practical efficiency gains. We investigate late-training instability by examining quantization formats, activation scaling, KV-cache precision, and framework differences that affect policy probabilities. TRIAGE addresses residual mismatch by adjusting updates in high-risk segments according to the mismatch and update directions, improving stability and task performance. miniTE separates quantization policy, linear layers, and compute backends, providing precision configuration, alignment checks, and performance analysis. Joint work on Transformer Engine’s per-token NVFP4 path uses contiguous layouts, device-side metadata, amax (maximum absolute value) reuse, grouped GEMM, and fusion to reduce preparation and dispatch overhead. On 8×B300, both miniTE paths with BF16 backward achieve approximately 1.33–1.37× paired speedups over BF16 for complete learner updates. A separate fixed-output serving test reaches approximately 1.64× BF16 throughput. We present the failure diagnosis, TRIAGE, and miniTE implementation and experiments in that order.

1. Introduction

Reinforcement learning for large language models repeatedly performs rollouts, policy-probability recomputation, gradient updates, and weight synchronization. Long responses increase the token volume for both generation and training, making low-precision computation valuable. Blackwell supports native NVFP4 matrix multiplication, and prior pretraining work has demonstrated low-bit training with scaling, mixed precision, and suitable rounding strategies [1]. An RL loop must additionally connect the numerical implementations used by the training and inference frameworks.

Our early NVFP4 runs improved reward initially but later developed abnormal entropy, growing training–sampling log-probability gaps, and declining reward. Quantization error, checkpoint conversion, activation scaling, KV-cache precision, and operator execution on both sides affected the outcome. Replacing a single quantization operator, or confirming that both sides used NVFP4, did not resolve all of these issues.

We first describe the failures, then use controlled experiments to identify and locate their causes. This leads to two complementary approaches. TRIAGE changes the update rule to control the effect of residual mismatch on learning. miniTE acts on numerical execution, reducing avoidable differences while turning low-precision computation into practical efficiency gains.

In a large training framework, changing scale granularity or a GEMM backend can require changes to layer wrappers, grouped layouts, communication, and autograd. miniTE separates quantization policy from these shared functions, allowing developers to replace a compute path within an existing training workflow and then evaluate its numerical correctness, learning behavior, and runtime cost.

Section 2 traces native NVFP4 RL failures and their diagnosis. Section 3 presents TRIAGE and its results. Sections 4–7 describe miniTE, implementation optimizations, experiments, and usage. TRIAGE controls update risk from residual mismatch in existing stacks; miniTE reduces avoidable implementation differences and improves execution efficiency. We report algorithmic outcomes and system performance separately, with the configuration for each experiment.

2. Why Native NVFP4 RL Becomes Unstable

2.1 Failure Develops During Training

We run GRPO on DAPO mathematics tasks with Qwen3-4B and Qwen3-30B-A3B [4,5,6]. In early native NVFP4 runs, reward increased normally before training deteriorated. In the trajectories analyzed by TRIAGE, the 4B model became unstable after approximately 300 steps, while mismatch in the 30B model began growing rapidly after approximately 700 steps. Diagnosing this process requires measurements over training time rather than a single initial error value or a short loss curve.

We track reward, entropy, the training–sampling log-probability gap, and downstream task scores. For token $t$ in response $i$, define $\delta_{i,t}=\log\pi_{\mathrm l}(y_{i,t}\mid x_i,y_{i,<t})-\log\pi_{\mathrm s}(y_{i,t}\mid x_i,y_{i,<t})$. Here $x_i$ is the prompt, $y_{i,t}$ is the current token, and $y_{i,<t}$ is its generated prefix. The learner and sampler assign conditional probabilities $\pi_{\mathrm l}$ and $\pi_{\mathrm s}$; $\log$ denotes the natural logarithm. A negative gap means the learner assigns a lower probability, and a positive gap means it assigns a higher one. Mean absolute gap averages $|\delta_{i,t}|$ over the tokens being evaluated.

2.2 Quantization Error, Conversion Error, and Execution Mismatch

Quantization error compares low-precision computation with a higher-precision reference. Training–sampling mismatch compares the actual executions of the learner and sampler. Checkpoint conversion and inference-backend adaptation can introduce additional error. Before integrating RL, we therefore compare BF16 inference, online NVFP4 quantization, and inference using offline-converted NVFP4 weights.

NVFP4 represents quantized values in E2M1: E and M give the exponent and mantissa bit counts, with one additional sign bit. Decoding gives $\hat x=q\,s_b\,s_g$, where $q$ is the E2M1 value, $s_b$ is an E4M3 block scale shared by 16 elements, and $s_g$ is an FP32 outer scale. The recipe determines whether the outer scale is computed per tensor, per expert, or per row.

NVFP4 Encoding and Two-Level Scaling

Interactive

A Qwen3-0.6B NVFP4 pretraining checkpoint initially suffered severe performance degradation because its fused QKV structure was handled incorrectly. After the fix, mean Lambda accuracy was approximately 0.3043 for BF16, 0.3056 for one-dimensional quantization, and 0.3109 for two-dimensional block-scaled quantization. Conversion and structural adaptation must therefore be checked before attributing such a failure to the FP4 format.

For Qwen3-4B, online quantization reduced Lambda-base and Math23K scores by 0.0355 and 0.047 relative to BF16. Offline conversion introduced additional reductions of 0.0089 and 0.019. These comparisons separate the loss from the format and numerical path from the additional loss introduced by conversion and scale handling.

In RL, the two sides can differ substantially even when they use the same format. Early configurations produced the following mean absolute log-probability gaps:

Table 1. Early training–sampling comparisons. The dense model is Qwen3-4B; the MoE model is Qwen3-30B-A3B. These results locate error sources within this set of configurations.

Replacing offline ModelOpt conversion with online quantization still left a gap of approximately 0.1142 for the dense model. Using the same quantization/GEMM operators on both sides also left residual differences. We therefore expanded the investigation to input organization, scale computation, and operations surrounding quantization.

2.3 Activation Precision, Scaling Granularity, and the KV Cache

With NVFP4 weights fixed, increasing activation precision tests the effect of activation quantization. W and A denote weights and activations; the following numbers give their bit widths, so W4A16 uses 4-bit weights and 16-bit activations:

Table 2. Activation-precision comparisons using the corresponding unquant inference adaptation paths. This experiment and Table 1 belong to different stages of the investigation and are not a single ablation.

W4A16 brings the gap close to the BF16 reference, identifying activation quantization as an important factor in this configuration. In another fixed configuration, changing the KV cache from FP8 to BF16 reduces the gap from 0.0721 to 0.0583. Sampler-state precision also affects cross-framework agreement.

We next examine the scaling domain. Sampler decoding processes the current token, whereas the learner typically processes a full response at once. If the activation's outer scale uses tensor-wide amax, the same token can receive different scales depending on the other tokens in its batch. Per-token amax restricts the reduction domain to the current row and removes this dependency.

Let $x$ be a reference vector and $\hat x$ its decoded approximation, each with $n$ elements. Relative L2 error is $\|\hat x-x\|_2/\|x\|_2$; signal-to-noise ratio is $\mathrm{SNR}=20\log_{10}(\|x\|_2/\|\hat x-x\|_2)$, in dB; and mean absolute error is $\mathrm{MAE}=n^{-1}\sum_{j=1}^{n}|\hat x_j-x_j|$. The notation $\|\cdot\|_2$ denotes the Euclidean norm. Relative error and SNR assume a nonzero reference vector. Lower relative L2 and MAE, or higher SNR, indicate closer reconstruction.

How the Scaling Domain Introduces Context Dependence

Interactive

Scaling granularity does not determine all quantization error. We sweep exponent and mantissa widths on real QKV inputs. With E=2 and E4M3 scales fixed, increasing M from 1 to 4 raises mean SNR from 19.88 dB to 34.96 dB and reduces relative L2 from 0.10140 to 0.01807. Increasing exponent width does not always help and can interact poorly with the scale's representable range. These simulated formats identify sensitive factors; they are not additional hardware-native NVFP4 formats.

In one configuration, per-token amax changes the initial gap only slightly, from 0.05532 to 0.05501, while later runs show smoother training. This motivates examining error direction, local structure, and its evolution during training rather than relying on the initial average alone.

2.4 Mismatch Must Be Interpreted Alongside Update Direction

Let $A_i$ denote the advantage of response $i$, measuring its reward relative to other responses in the same group. The figure abbreviates it as $A$ and writes a token’s gap as $\delta$. In GRPO, a positive advantage tends to increase sampled-token probabilities; a negative advantage tends to decrease them. At synchronized weights, the gap reflects execution differences. After learner updates, it also includes parameter changes relative to the fixed sampling policy.

When the learner underestimates a token (δ < 0), a negative-advantage update lowers its probability further and can amplify the local discrepancy; a positive-advantage update tends to reduce it. The roles reverse when the learner overestimates the token (δ > 0).

Intervention therefore needs both gap magnitude and update direction. TRIAGE uses the signs of advantage and gap to identify local risk. This local analysis motivates the algorithm; actual updates also depend on cross-token coupling and the optimizer.

Update Direction Determines Local Mismatch Risk

Interactive

A > 0, δ < 0

The learner underestimates and positive advantage raises probability, locally approaching the sampler. TRIAGE repair separately acts on negative segments in positive-advantage responses.

A > 0, δ > 0

This can locally amplify mismatch. TRIAGE does not attenuate every amplifying quadrant: its gate direction condition is negative advantage and negative gap.

A < 0, δ < 0

The learner underestimates and negative advantage lowers probability further. This meets the TRIAGE gate direction condition; attenuation also depends on segment mean gap and tail risk.

A < 0, δ > 0

The learner overestimates and negative advantage lowers probability, locally approaching the sampler.

2.5 Averages Can Hide Local Risk

In one diagnostic trajectory for each model, directional changes precede the explosion of the global tail. When responses are divided into 64-token segments, a small fraction of tail-dense segments accounts for part of the severe mismatch. A response containing these segments can still have a small average gap.

Experimental results

Figure 1. Directional distributions and negative tails from TRIAGE's supplementary analysis. The upper panels show region shares weighted by the original $|A|$; the lower panels show negative-tail depth, $-q_{0.05}(\delta)$, where $q_{0.05}$ is the fifth percentile of the token gaps and $|A|$ is the absolute advantage. The 4B analysis uses direct gaps; the 30B analysis uses residual gaps after routing replay. Stage definitions and local thresholds are given in the appendix.

These findings create two needs: an algorithm to manage risky updates in existing stacks, and shared interfaces for comparing scales, precisions, and operator paths. We next introduce TRIAGE, then describe how miniTE brings alignment, experimentation, and efficient execution into a reusable framework.

Diagnosis · §2With NVFP4 weights fixed, W4A16 brings the gap close to the BF16 reference level; moving the KV cache from FP8 to BF16 reduces the gap from 0.0721 to 0.0583.

3. TRIAGE: Direction-Aware Stabilization of Native NVFP4 RL

3.1 Mechanism

TRIAGE operates on a policy objective that already includes importance corrections such as truncated importance sampling (TIS) [10]. It retains native NVFP4 forward computation and manages residual mismatch by modifying the optimization objective. The method combines segment diagnostics, a direction-selective gate, and bounded repair.

The gate uses mean gap and deep-tail token counts in 64-token segments to assess risk. In high-risk segments with negative advantage, it attenuates update weights only for negative-gap tokens; all other tokens retain their original weights. The weighted loss is then normalized within each response, redistributing update contributions.

Repair adds a signal for negative segment offsets in positive-advantage responses. It applies a one-sided pseudo-Huber penalty with bounded derivative to mean gaps in 128-token segments, increasing learner log probabilities in active segments. The gate reduces risky updates, while repair supplies a segment-level corrective signal. Parameters and normalization are specified in Appendix B.

3.2 Stability and Task Performance

TRIAGE's NVFP4 baselines already use per-token activation scaling on both sides. The experiment measures residual mismatch in that native stack. TRIAGE completes 600 training steps for 4B and 1,700 for 30B. TIS delays but does not prevent instability for 4B; for 30B, it completes training with weaker late-stage performance.

TRIAGE Training Dynamics

Interactive

Figure 2. TRIAGE training dynamics. The upper row is 30B and the lower row 4B. From left to right: entropy loss, reward, and mean absolute gap.

Table 3. Unweighted mean over AIME24, AIME25, MATH-500, AMC23, and GSM8K [11,12,13]. † marks a checkpoint saved before instability. Final results with different training durations are distinguished from equal-budget comparisons.

At the shared 4B step-300 checkpoint, TRIAGE scores 53.03%, exceeding native NVFP4 and TIS by 5.93 and 2.27 percentage points. Its benefit therefore also appears at equal training budgets. Framework throughput is evaluated in the miniTE test environment.

3.3 Two Complementary Approaches

TRIAGE and miniTE intervene at different points in the problems identified above. TRIAGE changes updates when residual mismatch remains. miniTE directly examines and improves quantization, operators, and their invocation while reducing execution cost. The following sections introduce miniTE’s approach to numerical alignment and efficient computation. We evaluate the two approaches separately; additional gains from combining them remain to be tested.

4. miniTE: Aligning Numerical Execution and Optimizing Low-Precision Compute

4.1 Controlling Numerical Differences and Execution Cost

NVFP4 experiments repeatedly require local changes: activation-scale granularity, higher precision for selected layers, backward formats, and quantization or GEMM implementations. When these choices are tightly coupled to model modules, distributed communication, and training scripts, each experiment must also revisit surrounding logic, making errors harder to locate.

miniTE addresses these issues at the execution layer, making quantization rules explicit and comparing the learner and sampler at matching numerical boundaries. It provides ordinary linear layers, combined LayerNorm–linear modules, and MoE GroupedLinear modules. A common quantization interface connects Python references and optimized backends. Megatron and the upper-level RL system still orchestrate training, while inference frameworks such as SGLang perform sampling [7,8]. miniTE focuses on integrating, comparing, and optimizing numerical policies and operators.

miniTE Framework and Compute Paths

Interactive

4.2 Layered Interfaces and Transformer Engine Integration

The module layer retains parameters, autograd, and distributed interfaces needed by the training framework. The quantization-policy layer specifies formats and scales. Compute backends implement quantization and GEMM. Developers can validate numerical behavior with a reference before integrating CUDA, CUTLASS, or TE implementations behind the same interfaces.

Table 4. miniTE responsibilities. New numerical policies are primarily implemented in the quantization and backend layers, reusing shared module logic.

TE integration is a core design requirement. miniTE follows existing Linear interfaces and checkpoint parameter names so Megatron wrappers can replace the corresponding modules while reusing the surrounding training workflow. Parameter loading, extra state, and parallel modes are validated for each integration path to balance migration cost and behavioral consistency.

The recipe registry stores names, purposes, and dependency metadata; the parameter factory converts the selection into QLinearParams. The registry and inspection entry points use only the Python standard library. Operator imports are deferred to the corresponding factory branches, so developers can inspect a configuration before preparing its CUDA, TE, and compiled dependencies. Reference and optimized paths continue to share the upper-level modules.

4.3 Precision Recipes and Selective Quantization

The W4A16, W4A8, and W4A4 experiments in Section 2 require independent configuration of weights and activations. miniTE additionally separates gradient precision and GEMM settings for forward propagation (fprop), input gradients (dgrad), and weight gradients (wgrad). A new forward quantization policy therefore need not commit to a particular backward precision.

Version 1.0.0 selects named recipes through MINITE_RECIPE or get_qlinear_params, including nvfp4_per_token and nvfp4_per_token_bf16_bwd. Explicit layer qlinear_params take priority over environment selectors. Layer-name overrides and layer-number quantization exclusions support sensitivity analysis. Development can begin with reference quantization: establish scales and encoding, choose a GEMM backend, then compare full-precision and quantized backward paths.

miniTE CUTLASS and miniTE per-token are two implementations of MoE GroupedLinear. Both use NVFP4 forward computation and compute activation outer scales per token. CUTLASS names a matrix-multiplication implementation, whereas per-token names a scaling granularity; the terms are not mutually exclusive, and the TE per-token path also calls CUTLASS kernels. The main differences are below.

Both paths were tested with MXFP8 backward and BF16 backward. Backward precision is configured separately and cannot be inferred from the path name. The reported BF16-backward performance uses the layer bypass, which saves the original BF16 operands and skips backward quantization. The dedicated nvfp4_per_token_bf16_bwd operator is a separate implementation and requires its own performance measurements.

NVFP4 Forward with Independently Configured Backward Precision

Interactive

Public selectors resolve recipes through one registry, grouped by common, reference, and experimental use. These groups describe purpose. describe reports forward/backward behavior and requested layout; doctor checks the selected runtime prerequisites. Each result records forward and backward precision, quantization coverage, and source revision.

4.4 Numerical Monitoring, Profiling, and Alignment Diagnostics

Low-precision failures may emerge late in training. miniTE therefore provides configurable numerical and performance collection. Developers can inspect activation, weight, output, and gradient minima, maxima, means, and norms by module, instance, rank, and step, and save tensors to reproduce anomalies. The profiler produces Chrome traces and kernel timing statistics, separating computation, preparation, and gaps between calls.

Comparing the learner and sampler first requires aligning their inputs. MoE dispatch can change token order on the two sides; direct elementwise subtraction would then mistake permutation differences for numerical error. We collect routing information, expert FC1/FC2 tensors, and token log probabilities, align token and expert identities, and then compare outputs, quantized data, and scales.

Diagnostics distinguish actual kernel outputs, hook boundaries, and reference replay. Replay can locate an issue when an intermediate inside a fused kernel cannot be collected directly, but its provenance remains explicit. Agreement in replay is different from agreement in actual execution. Each recipe or kernel change can then be traced through inputs and routing, quantization, GEMM, and output probabilities.

4.5 Train–Inference Alignment and Deterministic Execution (Deterministic Mode)

Low-bit training stability depends on quantization precision and on whether operators behave consistently in training and inference. Even with the same quantization policy, small differences in operator implementations, reduction order, or parallel execution can accumulate into numerical discrepancies that affect training.

We therefore implement deterministic mode in miniTransformer: layer-by-layer replacement of Megatron operators makes them use the same kernels, formulas, and reduction order as SGLang inference. Multiple checked key layers achieve bit-exact agreement.

This layerwise alignment works with the tensor and routing-identity checks in Section 4.4. In the original record, TIS is disabled for the strict configuration and enabled for non-strict references. The figure therefore retains the joint identity of execution paths and algorithmic correction; bit-exact layer agreement and the end-to-end log-probability gap are reported separately.

NVFP4 Alignment and Training Stability

Interactive

Reward, gap, entropy, and policy-gradient loss

Figure 4. Training–sampling alignment: reward, mean absolute log-probability gap, entropy, and policy-gradient loss. Thick lines are 15-step moving averages. The aligned configuration reaches a gap of approximately $10^{-8}$ over the measured interval and maintains relatively steady late-stage reward. Multiple implementation choices change together in this comparison.

In this experiment, the aligned train–inference log-probability difference falls to approximately $10^{-8}$, with steady late-stage reward and smaller perturbations. Unaligned NVFP4 shows a late reward decline accompanied by rising entropy and log-probability difference. Within this configuration, smaller train–inference discrepancies accompany steadier NVFP4 RL dynamics. This is a single-run comparison of different implementation configurations with multiple choices changing together. Alignment and throughput measurements also have separate experimental identities; performance remains based on direct measurements of the corresponding Section 6 configurations.

miniTE provides alignment capabilities together with the configuration, debugging, and extension interfaces needed to implement them. New low-precision policies can follow the same validation process from local correctness to RL training.

Alignment · §4.5Under the strictly aligned configuration, the train–inference log-probability gap stays around 10⁻⁸, with stable late-stage reward.

5. Joint NVFP4 Implementation and Optimization with Transformer Engine

5.1 Numerical and Implementation Choices for Per-Token Scaling

We conduct joint testing and implementation work with NVIDIA around TE's NVFP4 support, using the per-token recipe and its interfaces as an important integration point [9]. Scale granularity affects numerical behavior as well as amax computation, scale storage, and GEMM epilogue cost. Accuracy and speed therefore need to be evaluated together.

NVFP4 encodes data in E2M1, uses an E4M3 scale for each block of 16 elements, and adjusts the range with an outer FP32 scale [1]. Let $s_i$ be the outer scale of row $i$, and $b_{i,j}$ its length-16 block. Then

Per-tensor scaling shares one outer scale across a tensor; per-token scaling computes it per row. The latter reduces dependence on other tokens' maxima and can mitigate block-scale underflow when row magnitudes differ greatly. It still retains within-row dynamic-range and FP4 rounding error, and requires finer-grained compute support. Public systems also combine per-token NVFP4 forward computation with higher-precision backward computation [2].

We retain multiple recipes and evaluate numerical choices separately from implementation optimizations. Lower local quantization error alone does not establish more stable RL training. The cost of additional reductions is likewise evaluated over the full computation path.

D2 · 20k-Step Language-Model Loss: Per-Token versus Per-Tensor Scaling

Experimental results

Figure 5-1. Ling MoE, single GPU, seed 11, 20,000 steps. Both NVFP4 configurations use RHT, SR, and 2D weight quantization. The left panel shows absolute loss; the right subtracts BF16 loss on the same data. Relative gaps over the final 1k steps are per-tensor +1.317% and per-token +1.060%.

Absolute losses are close at the plotted scale; subtraction from BF16 reveals the smaller differences. This supplies long-run language-model loss evidence for the scaling choice. It is distinct from the RL+TIS trajectories in Section 6.2.2 and does not independently establish RL stability.

5.2 Contiguous Layouts and Grouped GEMM Invocation

Original grouped interfaces may split quantized data and scales into separate expert tensors, then reconstruct pointers, strides, and layouts for GEMM. As expert counts increase, Python/ATen slicing, list construction, temporary allocation, and host-to-device metadata transfers can offset FP4 arithmetic gains.

We organize quantization outputs into contiguous buffers and describe expert boundaries with offsets, passing them directly to dense grouped GEMM. FP4 data, block scales, and outer scales travel through the same interface, avoiding splitting after quantization and reassembly before computation. Device-side metadata construction and workspace reuse reduce allocation and synchronous copies. Zero-token experts and padding are handled within the same grouped path.

This optimization changes data organization and invocation. Dense quantization and GEMM interfaces are checked against the original list path for operands and outputs; performance is measured for the combined implementation. Revision b641179 later combines the dense path, GEMM settings, and fusion. The core repeated tests retain their original revision identity.

5.3 Computing amax Where Data Are Produced

A separate amax kernel first scans activations, after which the quantizer reads them again. We look for operations that already traverse the data along the producer-to-consumer path.

For MoE FC1, inputs pass through dispatch and permutation, allowing rowwise amax to be accumulated during token reordering. Padding processes activations and amax together, so quantization can reuse the external statistics and skip repeated reduction.

For MoE FC2, amax is better fused with activation computation. We use _weighted_swiglu_row_amax_kernel as the fused kernel, following the JIT-compiled implementation and accumulating amax while traversing columns. The statistic applies to the SwiGLU output after router weighting, exactly the tensor passed to FC2 quantization. Reusing the existing memory and computation path avoids another complete scan of the FC2 input for amax.

The FC1 transformation is:

Original: sort → pad(activation) → amax → quantization
Fused:    sort + amax → pad(activation, amax) → quantization

FC2 produces amax together with weighted activations and reuses it in subsequent quantization:

Original: SwiGLU × router weight → activation → amax → quantization
Fused:    weighted SwiGLU + row-amax → activation, amax → quantization

The benefit must cover the producer, padding, and consumer. Module-level simulations that skip amax are not always faster, so eliminating one kernel does not by itself establish a complete-flow benefit. Section 6 uses learner timings that include surrounding work.

5.4 Kernel Optimizations and Their Scope

Kernel-level optimization addresses cross-CTA amax reduction, FC1/FC2 GEMM pipelines, scale-layout transfer, and activation/amax fusion in the epilogue. The implementation details and applicable scope follow.

ACC = block-scaled GEMM(A, B)
ACC = ACC × α_a[m] (optionally × α_b[n]) + bias
C = gate | up (pre-GLU, width N; optionally written)
D = act(gate, up) × prob (SwiGLU/GeGLU, width N/2)
Reduce D amax concurrently; FP8 can generate output scale factors

5.5 How Joint Testing Informs Implementation Choices

Joint work follows reference implementation, optimized implementation, module comparison, and model testing. Comparisons at a fixed recipe validate interface and layout changes; comparisons across recipes evaluate precision choices and their costs. BF16, native TE NVFP4, our CUTLASS path, and the upstream per-token implementation serve different baseline roles.

This process integrates TE capabilities into miniTE and brings bottlenecks found in miniTE back to TE interfaces and kernels. Public per-token implementation and discussion are available in PR #3045 [9]. Revisions, patches, and test scope accompany the results; attribution and upstream merge status follow the actual records.

5.6 Exploring Single-Pass Fusion

We also investigate online amax compensation for fused SwiGLU-FFN paths that avoid materializing the full intermediate activation. FC2 consumes activation K-tiles before the full-row amax is known. The method maintains a running rowwise amax and converts earlier partial accumulations to the new scale whenever the outer scale increases, connecting FC1, SwiGLU, quantization, and FC2.

This path primarily reduces intermediate-activation memory traffic and separate kernel calls. Compensation maintains a consistent scale when accumulating tile contributions; the E4M3 block scales and E2M1 codes produced earlier still depend on the scale and rounding used at that time. We therefore retain it as a fusion research direction, evaluating compensation error, quantization differences from two-pass execution, and the complete call-chain benefit separately.

Inputs: X, W_FC1, W_FC2
Initialize gamax = 0, gs_prev = 0, O = 0
for each FC2 K-tile i:
    Complete FC1 reduction for gate/up slice → SwiGLU → P_i
    rowmax = max(|P_i|); gamax = max(gamax, rowmax)
    If gamax = 0: this row contributes zero; skip computation
    gs = gamax / 2688
    Generate E4M3 bs and E2M1 q per 16-element block
    FC2 block-scaled MMA → ΔO_i (includes block scales, excludes gs)
    O = O × (gs_prev / gs) + ΔO_i
    gs_prev = gs
Y = O × gs_prev × gs_W2; all-zero rows output zero

6. Evaluation: Numerical Correctness, Learning, and Execution Efficiency

We validate the preceding implementation choices by checking quantization, interfaces, matrix multiplication, and gradients, then evaluating long-run learning. We next measure complete learner updates, modules, serving, and online RL stages. Each experiment reports its result and how it informs implementation choices.

The main text presents key trends and results. Complete shape, gradient, repeated-run, and configuration data accompany the corresponding sections. The web reader provides these under their titles; printing collects them in Appendix C.

6.1 Setup and Measurement Scope

This section evaluates miniTE system performance; Section 3 reports TRIAGE's algorithmic results. The main performance model is Qwen3-30B-A3B on one node with 8×B300 and TP/PP/CP/EP = 1/1/1/4. Learner tests use sequence length 4096, micro-batch size 1, and GBS 128, processing 524,288 tokens per update from the same checkpoint, seed, and frozen DAPO text. Module, learner, serving, and online RL tests have distinct timing scopes.

6.2 Correctness and Learning Behavior

6.2.1 Separate Checks for Quantization, APIs, GEMM, and Gradients

We validate local numerical boundaries before full training. API and layout optimizations compare encoding, scales, and outputs at a fixed recipe. Quantized computation is compared with FP32 references after dequantization. Backward tests check direction, magnitude, and signs against FP32 gradients under straight-through estimator (STE) semantics. This distinguishes format error from error introduced by an optimization.

miniTE per-token dense quantization and grouped GEMM pass 28/28 bitwise equivalence checks. Fused layouts add no discrepancy in seven synthetic and real-activation cases. For the tested configurations, these layout changes preserve the quantization results.

FP32 epilogues also have direct numerical support. Both ordinary miniTE CUTLASS grouped GEMM and the EVT path are compared with the FP32 golden reference after dequantization:

Table 6-4. miniTE CUTLASS GEMM accuracy. EVT reduces the norm_rmse bound from approximately 0.0033 to the 4e-5 range, supporting a high-precision epilogue in this path.

Against pure BF16 forward computation, GroupedLinear has cosine similarity around 0.99095 and norm_rmse around 0.135, substantially larger than GEMM error against its own dequantized reference. The former includes information lost to FP4; the latter checks GEMM implementation. The reference determines what the accuracy metric measures.

BF16 backward dgrad/wgrad closely match the BF16 reference, with maximum relative L2 approximately 0.001663. MXFP8 backward has relative L2 approximately 0.03765 and sign agreement around 98.8%. This supports independently selecting backward precision in Section 4; Section 6.4 measures its execution cost.

6.2.2 Supplementary Long-Run Learning Evaluation

Figure 5 compares BF16+TIS, native NVFP4+TIS, and miniTE+TIS. miniTE maintains reward growth during the first 1,000 steps and exceeds native NVFP4 later in the run; BF16 achieves higher reward over this interval. The two NVFP4 configurations have similar mean log-probability gaps but different learning trajectories. Average gap alone is therefore not a substitute for task performance.

Training Curves for the Practical miniTE Path

Interactive

Figure 5. Learning dynamics of the practical miniTE path and baselines. Each configuration has one run over steps 0–999. The left panel shows mean absolute log-probability gap; the right shows reward. Thin lines are raw values, thick lines centered 21-step means. All configurations use TIS. This is a separate experiment from the strict alignment reference.

This supplementary run uses Qwen3-30B-A3B-Base. BF16, native TE NVFP4, and miniTE all use TIS, with routing replay disabled. Table 6-8 evaluates four mathematics tasks and IFEval on the respective step-999 checkpoints [11,12,13,14]. Each AIME benchmark averages six evaluations; MATH-500, GSM8K, and IFEval each average four. IFEval uses Prompt-level Strict accuracy. The BF16 checkpoint is evaluated with BF16 inference, and both NVFP4 checkpoints with NVFP4 inference.

Table 6-8. Task scores at step 999 of the supplementary run. The last row reports percentage-point differences. The five tasks receive equal weight, independent of evaluation counts. Repeated evaluation of a checkpoint does not create independent training seeds.

miniTE improves all four mathematics scores over native NVFP4. Its five-task average increases by 1.78 percentage points and remains 2.49 points below BF16. IFEval has a different ordering: miniTE is close to BF16 but below native NVFP4. These results support improvements on the measured mathematics training tasks, with downstream gains varying by task.

This supplements the framework's long-run training evaluation. Its task set, training duration, and evaluation protocol differ from TRIAGE in Section 3; each describes learning under its own configuration.

6.3 Complete Learner Updates: Do Local Changes Reduce Update Cost?

Table 6-9 summarizes the ten-run main test, comparing BF16, native TE NVFP4, and miniTE. Results for both miniTE paths also show how backward precision affects complete learner-update throughput.

Table 6-9. Complete learner throughput. Speedups are computed by pairing runs before trimming. Timing includes forward, backward, communication, the optimizer, and data loading; it excludes rollout and weight synchronization.

Complete Learner Throughput Across Repeated Runs

Interactive

Both miniTE paths with BF16 backward achieve approximately 1.33–1.37× BF16 speed, while TE NVFP4 is near BF16. Moving miniTE CUTLASS backward from MXFP8 to BF16 further improves throughput, showing that lower-bit GEMM gains can be offset by backward quantization and preparation. miniTE per-token's ratio of aggregate throughput means against TE NVFP4 is approximately 1.396×. The 1.344× miniTE per-token/TE BF16 value is also a ratio of means, distinct from the 1.334× paired estimator here.

The subsequent complete optimization chain combines the dense path, GEMM-kind, permutation fusion, and amax reuse. miniTE per-token with BF16 backward reaches 33,540 tokens/s, or 1.39× the same-round BF16 baseline and 1.47× TE NVFP4. This combined test uses a different revision and protocol:

Key result · §6.3Both miniTE paths with BF16 backward reach about 1.33–1.37× the BF16 speed on complete learner updates.

6.4 Modules: Faster Forward Does Not Necessarily Mean Faster F+B

After measuring gains in the complete learner, module tests explain their sources. Figure 6 switches between FC1/FC2 and shows forward, directly timed F+B, or both. F+B uses directly measured timings:

GroupedLinear Forward and Direct F+B Trends

Interactive

Figure 6. Qwen3-30B-A3B GroupedLinear on B300, warm static weights, five runs. Solid lines show forward, dashed lines directly measured F+B. Both miniTE paths use quantized backward in this figure. The horizontal axis is pre-dispatch token scale; this balanced workload is distinct from full-model routing.

miniTE per-token forward is approximately 1.09–1.43× faster than BF16, while its F+B in the figure is slower. BF16 backward makes miniTE per-token F+B approximately 1.2–1.3× faster than its own MXFP8-backward path. This directly motivates measuring complete operator cost before selecting backward precision.

Module Cost Across Workloads and Backward Precisions

Interactive

With the forward recipe fixed, miniTE per-token achieves a 2.46× geometric-mean expert-forward speedup over upstream TE per-token and 1.10× for direct F+B. The difference points to forward invocation and preparation as the main sources of implementation gains. Inputs are pre-permuted, FC2 activations are prepared outside the timer, and dispatch is excluded.

Profiling explains the transition from local to full-model performance: host metadata preparation, gaps between grouped calls, expert communication, and non-GEMM operations all affect update time. Contiguous layouts reduce organization overhead, while model tests include real routing and surrounding costs.

D15 · Representative Profiles: Multi-Tensor Calls and Grouped GEMM

How dispatch and routing affect downstream GEMM scheduling. In the supplementary record's first microbatch of Step 8, routing from real weights and inputs is more uniform, while mock routing from random weights and inputs is concentrated. Both use 4,096 input tokens per rank, top-k=8, and 131,072 routed tokens within the EP group. We first compare expert distributions at this common token budget:

Load metric	Real	Mock	Interpretation
Experts with tokens	126/128	24/128	Mock concentrates tokens on fewer experts
Zero-token experts	2	104	The number of participating experts differs
CV of GroupedLinear M	0.19	0.66	The recorded Real GEMM row counts are more uniform

M is the token-row count of an expert GEMM. CV is the standard deviation divided by the mean under the same statistical definition; we retain the recorded GroupedLinear M statistic instead of reconstructing it from global expert counts. More participating experts create more matrix-multiplication problems to prepare and schedule. Host API and kernel profiles help explain that cost:

Call metric	Real	Mock	Interpretation
cuBLASLt heuristic calls / cumulative time	148,000 / 6.26 s	32,000 / 1.02 s	Real has approximately 4.6× as many calls
Median time per heuristic call	2.9 µs	2.9 µs	Similar medians, different call counts
nvte_multi_tensor_gemm calls / median time	4,608 / 2.29 ms	4,590 / 0.44 ms	Similar counts; Real's median is approximately 5.2×
nvjet kernel calls	114,000	4,600	Real has approximately 25× as many calls

The first table identifies this microbatch's routing load; the second preserves call counts, cumulative times, and per-call medians from the accompanying profile. Cumulative times are not treated as this one microbatch's wall time or added directly to GPU kernel times.

The same token budget is split into more GEMM problems. For matching expert matrix shapes, the main GEMM arithmetic scales with the sum of expert token-row counts. Real and Mock have the same token total, but Real distributes it across 126 active experts and Mock across 24. Real therefore has a smaller average M per expert and more nonempty problem entries. Its lower CV indicates a more balanced distribution, while preparation and scheduling also depend on problem count and granularity. Many smaller GEMMs have less computation over which to amortize fixed preparation and launch costs.

cuBLASLt algorithm selection repeatedly runs on the host. The original profile identifies AlgoGetHeuristic as a major BF16 bottleneck. The multi-stream, multi-tensor path still performs repeated algorithm selection and prepares shapes, layouts, and call parameters for expert problems. Both heuristic medians are 2.9 µs, but Real has approximately 4.6× as many calls, indicating that repeated work drives the extra cost. Outer nvte_multi_tensor_gemm counts are similar, while the expert problems and preparation inside each call differ. Real's median call time is consequently 2.29 ms, approximately 5.2× Mock's.

More fine-grained kernels lengthen the submission chain. Real has approximately 25× as many nvjet kernels as Mock, showing different GPU call organization for the same token budget. If CPU algorithm selection, argument preparation, and launch submission become the bottleneck, additional streams may not hide the GPU's wait for its next tasks. This is the CPU bubble to examine in that path. The counts directly support increased preparation and scheduling work; the duration of GPU gaps still requires the corresponding timeline.

Dense grouped GEMM targets this preparation and scheduling chain. Section 5's contiguous buffers, device-side offsets and metadata, and reusable allocations put expert problems on one grouped scheduling path, reducing host-side slicing, list assembly, argument construction, and repeated organization. These changes affect the work surrounding GEMM and the complete call time, so the observed difference should be interpreted together with the implementation path. Actual speed ratios remain based on comparable module or learner timings, with real weights and routing used for both BF16 and low-precision evaluation.

Recorded profile

Recorded profile

Figure 7. Representative profiles illustrating invocation structure. The screenshots do not supply paired timings with matching shapes and timers, so 3.594 ms and 156µs are not divided to report a speedup. Performance comes from the module/learner measurements above.

6.5 Rollout and Online RL: System Gains and Remaining Costs

The sampler is first tested at fixed output workload. Eight engines use real DAPO prompts, generate exactly 1024 tokens with EOS ignored, and independently select concurrency 64 after a sweep for each configuration:

Table 6-17. Fixed-output serving. BF16 uses Triton MoE; NVFP4 uses FlashInfer TRT-LLM/ModelOpt FP4 and per-token scaling. This measures the sampling stack rather than isolating miniTE training operators.

Sampling Capacity at a Fixed Workload

Interactive

Complete RL also connects learner updates and weight synchronization to generation. For a sequential workflow without overlap,

When stages overlap, measured wall time is the reference. Table 6-19 reports the recorded stages for BF16, native TE NVFP4, and miniTE per-token:

Table 6-19. Mean online RL stage times. Train includes log-probability recomputation and actor training. The two-stage sum is not a full iteration. TE NVFP4 uses its native backward configuration; miniTE per-token explicitly disables backward quantization.

Compute Gains and Conversion/Synchronization Cost

Interactive

miniTE per-token Train is approximately 1.11× faster than BF16 and TE NVFP4 approximately 1.05×. Conversion/synchronization is approximately 5.56× and 5.12× slower, respectively. miniTE per-token adds 40.84 seconds across these two stages. In this run, rollout gains did not offset synchronization cost, making conversion and transfer important targets for further optimization.

This negative result is useful: faster low-precision operators do not ensure lower system integration costs. Independent serving ratios cannot replace rollout time measured over the same run window, and summing these two stages does not yield a complete RL speedup.

Key result · §6.5In the eight-engine fixed-output test, miniTE per-token serving throughput is about 1.64× BF16.

6.6 What the Experiments Establish

Together, these experiments explain miniTE's implementation choices. Operator and gradient checks support interfaces, layouts, and backward precision. Module forward and direct F+B timings locate local costs. Real-text learner tests establish whether these choices reduce complete update cost. Serving and online RL tests reveal the additional constraints of weight conversion and synchronization.

Low-bit training benefits must therefore be validated at multiple levels: numerical correctness, sustained learning, faster local computation, and manageable integration cost. We retain core measurements and complete configuration comparisons at each level, using miniTE test records for performance results.

7. Using and Extending miniTE

7.1 Integrating Existing Models through Linear Interfaces

The implementation package is miniTransformer, and the linear layer is miniTE's basic integration unit. Linear, LayerNormLinear, and GroupedLinear provide ordinary projections, combined normalization and linear computation, and MoE expert computation. Existing Megatron wrappers can connect to these modules, or users can pass parameters constructed from a named recipe directly:

import torch
from miniTransformer.module.config import get_qlinear_params
from miniTransformer.module.linear import Linear

params = get_qlinear_params("nvfp4_per_token")
layer = Linear(
    1024, 1024,
    bias=False,
    params_dtype=torch.bfloat16,
    device="cuda",
    qlinear_params=params,
)

Recipe selection determines the numerical path; module integration determines which model layers use miniTE. The MoE launcher sets both for its miniTransformer backends. GroupedLinear imposes output-dimension and grouped-layout constraints and does not support sequence parallelism. Integration should check the specific model and parallel configuration.

miniTE retains TE/Megatron checkpoint parameter naming and compatibility interfaces. The current extra-state interface does not restore all TE internal quantization state. Parameter-loading compatibility and full internal-state compatibility are validated separately.

7.2 Explicit Recipes and Precision Boundaries

Version 1.0.0 uses names describing numerical behavior. Users select a training recipe with MINITE_RECIPE or construct independent parameters and operators with get_qlinear_params:

Table 9. Selected named entries in version 1.0.0. Performance experiments retain their own source revisions, forward/backward precision, and quantization coverage; names select the current software configuration.

For an integrated training program:

MINITE_RECIPE=nvfp4_per_token python train.py

Here train.py denotes the user's training entry point. Expressing the report's miniTE per-token BF16 layer-bypass choice through the current named interface gives:

MINITE_RECIPE=nvfp4_per_token DISABLE_BACKWARD_QUANT=1 python train.py

This layer bypass and the dedicated nvfp4_per_token_bf16_bwd operator are implemented and recorded separately. Speeds of both miniTE paths in Section 6 belong to the original experimental paths. The dedicated operator requires its own performance measurements. Historical reproduction also uses the original revision's launch configuration and fixes sampler quantization, checkpoint conversion, expert-only coverage, and TE revision.

7.3 Diagnostics as Part of Extension Development

Configuration checks begin before constructing a model. Run these commands from the miniTransformer repository root:

python -m miniTransformer recipes
python -m miniTransformer describe nvfp4_per_token
python -m miniTransformer doctor --recipe nvfp4_per_token
bash develop/script/moe_training_e2e.sh \
  --backend mini_nvfp4_per_token --profile off --dry-run

recipes and describe expose names, dependencies, and backward overrides without Torch or TE. doctor inspects visible CUDA GPUs, TE quantizer arguments, selected dense/list grouped interfaces, and operator imports. --dry-run prints the recipe, relevant environment, and complete command without starting training or creating log/cache directories. Actual runs record the recipe name, forward/backward settings, layout request, and configuration source in run_meta.json.

All 21 CPU compatibility checks for this interface update passed, covering resolution, conflicting selectors, inspection without GPU dependencies, backend aliases, dry-run, and metadata. Added GPU equivalence tests cover independent configurations, forward/backward behavior on supported Dense/GroupedLinear paths, and the two BF16 backward choices. They have not been run locally in this update. Numerical and performance results in Section 6 retain their original experiment identities.

When extending a low-precision path, we check operators, then modules and cross-framework behavior, and finally complete training. Tensor hooks collect activation, weight, output, and gradient statistics. Profiling locates kernel costs, call gaps, and allocation. Dump/compare scripts examine MoE intermediates and token log probabilities. A new recipe declares its purpose and dependencies in the registry, connects its operator through the existing parameter factory, and follows these numerical and runtime checks.

The repository README covers setup, layer integration, and launch examples; the bilingual recipe guides document selection and override rules. Model conversion, sampling, and evaluation require their own pinned versions and settings. Current software interfaces and historical experiment configurations are recorded separately.

8. Discussion: From Numerical Alignment to End-to-End Learning

The experiments address low-precision RL through two approaches: TRIAGE changes updates in the presence of mismatch, while miniTE reduces execution differences and compute cost. Neither resolves every issue. The next questions are whether faster computation yields end-to-end savings, and whether closer numerical agreement produces better learning.

8.1 Weight Updates and Synchronization Remain a Bottleneck

Here, update weight means converting, transferring, and loading the trained weights into the sampler. This remains a major bottleneck in the configurations we measured. Faster learner updates and rollouts do not imply a proportional reduction in complete RL iteration time. In the run reported in Section 6.5, conversion and synchronization costs offset the gains from faster sampling.

Further optimization should consider parameter representations, quantization and packing, transfer, loading, and synchronization frequency together. Possible directions include reusing low-precision representations within the same weight version, combining conversion with transfer, and overlapping synchronization with sampling under explicit version control. Less frequent synchronization can save transfer time but increases policy staleness. These choices should be evaluated using complete iteration time and task performance under matched training budgets. Online amax and deeper operator fusion also need evaluation within this end-to-end scope.

8.2 Why Does Low Precision Remain Sensitive After Operator Alignment?

Matching operators and quantization recipes removes some execution differences. Matching their names alone, however, does not guarantee identical floating-point computation across input shapes, batches, and parallel configurations. Floating-point addition is not associative: changes in reduction order, fusion boundaries, or tiling can change the result. A fixed execution path can be deterministic. The remaining question is where perturbations arise when execution conditions change and how they affect subsequent updates.

Our alignment experiment and long-run evaluations expose two sides of this problem. The strict alignment configuration reduces gap to approximately 10⁻⁸, while miniTE’s long-run learning and checkpoint evaluations do not match BF16 in every comparison. These results use different configurations and measurement scopes, but together they show why numerical agreement and final learning quality must be evaluated separately. Why can seemingly small perturbations be more damaging in the low-precision configurations we tested?

Candidate mechanisms include persistent error bias, rounding transitions near coarse quantization boundaries, MoE routing changes, and the accumulated response of gradient clipping and optimizer state. These remain hypotheses. More informative experiments would hold the model and data fixed, control the location, direction, and correlation of perturbations, and compare subsequent updates and long-run reward in BF16 and low precision, rather than relying on a single-step average gap.

8.3 Optimize Updates and Final Learning Outcomes

Mean log-probability difference cannot be converted linearly into reward. Similar average gaps can occur on different tokens, advantage signs, and response segments, with different effects on updates. A small gap also does not guarantee useful exploration or gradient signal. TRIAGE’s directional analysis offers one starting point. Further work can directly examine how errors alter updates, testing corrections through reward and evaluation scores under matched data, update, and wall-clock budgets.

The community’s approaches are broadening too. Together AI’s Score Centering (SC) adds a correction that removes mismatch-induced drift from the policy gradient and can be combined with importance sampling [17]. The Qwen team’s TRACE uses rollout-side quantization outcomes to guide training-side FP4 rounding and compresses the guidance information [18]. SC changes the gradient estimator; TRACE coordinates quantization decisions. TRACE still targets alignment, but both works expand the available interventions beyond matching operator implementations.

For miniTE, this means supporting joint experiments on update rules, precision allocation, and sampling policy alongside controlled, comparable numerical execution. We hope the community will investigate how to preserve task-relevant learning signal, suppress bias that compounds across updates, and translate these changes into end-to-end gains. The goal is better learning within a given resource budget, not simply a smaller consistency metric.

Appendix A. Experimental Setup and Statistics

A.1 Supplementary Algorithm and Training Materials

TRIAGE's method and results follow the release manuscript and its diagnostic appendix. Checkpoints are those in Table 3, with equal weight across five mathematics benchmarks. Updated 4B means are 58.26% for BF16 and 58.49% for TRIAGE; TRIAGE scores 53.03% at the shared step-300 checkpoint.

Characterization uses separate diagnostic runs: updates 1–324 for 4B and 1–724 for 30B, with maximum completion lengths 16,384 and 20,480. The 4B analysis uses direct learner–sampler gaps; 30B uses residual gaps after routing replay. Fixed tail thresholds over valid tokens in the healthy stage are −0.24208 and −0.03949, respectively.

These diagnostic runs use DAPO-Math-17K, learner seed 1234, rollout/data seed 42, 16 prompts per iteration, 16 responses per prompt, and GBS 256. They use RHT and gradient stochastic rounding; these numerical settings differ from miniTE's practical path, where the options are disabled. Detailed windows, length sensitivity, one-step update audits, and randomization controls are in the diagnostic appendix.

TRIAGE materials are used for algorithmic outcomes. Its image was not tuned to the same level as the miniTE performance environment, so its throughput is not used as a miniTE upper bound or an algorithm-overhead baseline.

A.2 Supplementary Long-Run RL and Checkpoint Evaluation

Figure 5 and Table 6-8 use the following training configuration and evaluation protocol:

Table A1. Supplementary miniTE long-run protocol. This differs from the fixed-text learner test on one 8×B300 node. Repeated checkpoint evaluations do not increase the number of training seeds.

The three main configurations cover steps 0–999, with one trajectory each and centered 21-step means. Figure 4 retains the four-panel plot with 15-step smoothing; the accompanying paper also contains a two-panel presentation of a related alignment experiment with 21-step smoothing. Each caption retains its own convention. Strict alignment changes several execution components together and removes TIS; its timing is separate from learner performance tests.

A.3 Fixed-Text miniTE Learner Tests

Table A2. Fixed-text performance protocol. Data construction includes approximately 4.3-fold repetition. Activation recomputation, offload, and CUDA graphs are disabled. Expert-padding costs of quantized paths are included in timing.

The core protocol runs 20 updates per process, uses five warmup updates, and measures updates 6–15. Each run's throughput is 524,288 divided by median update time. Across ten runs, the maximum and minimum are removed before averaging. Speedups are first paired by run ID and then trimmed in the same way.

The optimized protocol uses one process per configuration and 17 updates. After two warmups, the maximum and minimum of the remaining 15 update times are removed. Throughput is 524,288 tokens divided by the mean of the retained times; speedups use the same-round baseline. The estimators are reported separately. Core run 3 experienced interference from an idle serving process, and affected minimum observations are removed under the predefined trimming rule. All ten records and revision differences remain in the miniTE summary.

Table 6-9 uses paired speedups from ten independent runs. The 1.334× value is the trimmed mean of per-run ratios; 1.344× is the ratio of aggregate throughput means. The later fusion experiment uses a different revision and configuration and is reported separately.

A.4 Module, Serving, and RL Stage Timing

Module tests use one GPU, 32 local experts, four pre-dispatch token sizes, and five independent process runs with warm static weights. Dispatch is excluded and FC2 activations are prepared beforehand. Module-forward gains are distinct from learner speedups. Upstream TE per-token has different full-model quantization coverage and backward precision from miniTE; its full-model comparison is not used to isolate implementation gains.

Serving uses fixed output length 1024, ignore_eos=true, context limit 8,192 tokens, static memory fraction 0.6, and disabled piecewise CUDA graphs. A single engine uses 64 prompts to sweep concurrency 1, 2, 4, 8, 16, 32, 48, 64, and 96. Both configurations select 64 before processing 512 prompts across eight engines.

Colocated RL tests use one 8×B300 node and one GPU per engine. BF16 statistics cover performance steps 2–12; miniTE covers 2–11. Learner timing includes log-probability computation and actor training. Weight updates include Megatron→SGLang synchronization and miniTE's BF16→NVFP4 conversion. The two-stage sum excludes full rollout and other orchestration work.

API and fused-layout checks compare encodings, scales, and outputs at fixed shapes and configurations. Bitwise agreement at these boundaries does not establish full-model batch invariance. Earlier swizzle failures and the production miniTE per-token checks use different shape constraints and are interpreted separately.

Appendix B. TRIAGE Method Conventions

We use response-level normalization from the release manuscript: compute a weighted token-loss mean within each response, then average across responses. With valid-token mask $m_{i,t}$ and detached gate weight $w_{i,t}$,

The fixed gate uses 64-token segments. For negative-advantage segments, the mean-tier weight is 0.1 when mean gap is below −1.5, 0.3 for [−1.5, −0.5), and 1.0 otherwise. If $k$ valid tokens in the segment satisfy $\delta<-6$, the tail weight is $1-0.9\min(k/2,1)$. The smaller of the two segment weights is applied only to negative-gap tokens. Other tokens retain weight 1.

Repair uses 128-token segments within positive-advantage responses and applies a one-sided pseudo-Huber penalty beyond the negative boundary. Masks, segment normalization, and derivatives are described in the TRIAGE method appendix and main method section.

Component ablations branch from the full-TRIAGE step-300 checkpoint and compare continued training. The complete figure is in the TRIAGE ablation. These are continuation comparisons rather than ablations trained from scratch.

The diagrams and interactive presentation draw on The 4-bitter Lesson and the LMSYS Blackwell low-precision RL article [15,16]. Constructed examples and conditional calculations are identified separately.

References

[1] NVIDIA. Pretraining Large Language Models with NVFP4. 2025; updated 2026. arXiv:2509.25149.

[2] Cursor Research. Composer 2 Technical Report. 2026. arXiv:2603.24477.

[3] Guangming Sheng et al. HybridFlow: A Flexible and Efficient RLHF Framework. EuroSys 2025. arXiv:2409.19256.

[4] Zhihong Shao et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. 2024. arXiv:2402.03300.

[5] An Yang et al. Qwen3 Technical Report. 2025. arXiv:2505.09388.

[6] Qiying Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. 2025. arXiv:2503.14476; DAPO-Math-17K dataset.

[7] Lianmin Zheng et al. SGLang: Efficient Execution of Structured Language Model Programs. NeurIPS 2024. arXiv:2312.07104.

[8] Mohammad Shoeybi et al. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. 2019. arXiv:1909.08053.

[9] NVIDIA Transformer Engine. Add NVFP4 per-token quantization recipe, PR #3045. 2026. Implementation and discussion.

[10] Zhaopeng Qiu et al. FP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learning. 2026. arXiv:2601.18150.

[11] Karl Cobbe et al. Training Verifiers to Solve Math Word Problems. 2021. arXiv:2110.14168; official GSM8K repository.

[12] Dan Hendrycks et al. Measuring Mathematical Problem Solving With the MATH Dataset. 2021. arXiv:2103.03874. For the MATH-500 split, see Hunter Lightman et al., Let's Verify Step by Step, 2023, arXiv:2305.20050, and the official split description.

[13] Mathematical Association of America. Official AMC page and AIME introduction. Results use the AIME 2024, AIME 2025, and AMC23 evaluation versions recorded in the experiments.

[14] Jeffrey Zhou et al. Instruction-Following Evaluation for Large Language Models. 2023. arXiv:2311.07911.

[15] Ziang Li et al. The 4-bitter Lesson. humans&, 2026-07-10. Article and interactive examples.

[16] Ziang Li, humans& and Miles Team. Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles. LMSYS, 2026-07-29. Article and format explanation.

[17] Martin Marek and Max Ryabinin. Score Centering Stabilizes Off-policy Reinforcement Learning. 2026. Paper.

[18] Xin Wang et al. TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models. 2026. Paper.