The theta bundle

How a pretrained weight matrix is decomposed analytically into a shared block, a per-layer scale, and a low-rank delta.

Every weightless projection replaces its stored [out_dim, in_dim] weight with three small tensors plus one shared block. Together these are the theta bundle:

Component	Shape	Scope	Role
`W0`	`[out_dim, in_dim]`	Shared per family	The common block reused across layers.
`s`	`[out_dim]`	Per layer	A per-output-row scale of `W0`.
`U`	`[out_dim, rank]`	Per layer	Left factor of the low-rank residual.
`V`	`[in_dim, rank]`	Per layer	Right factor of the low-rank residual.

The original weight is approximated as:

weight ≈ diag(s) · W0 + U · Vᵀ

Because W0 is shared across all layers in a projection family, only the small s, U, and V tensors are stored per layer. That’s where the size reduction comes from.

Projection families

Decomposition operates per family, the seven projections that appear in every decoder layer:

q_proj  k_proj  v_proj  o_proj      (attention)
gate_proj  up_proj  down_proj       (MLP)

By default all seven are made weightless. Setting tie_families="attn" restricts weightless treatment to the attention projections (q/k/v/o) and leaves the MLP projections as plain dense max.nn.Linear layers that load their weights straight from the checkpoint.

Analytic decomposition

There’s no training or fine-tuning step. Given the shared block W0 for a family, the framework decomposes each layer’s weight in closed form:

Shared block

W0 is the mean of the family’s weight matrices across all layers, or across a band (see banded sharing).

Per-row scale

s is the per-output-row least-squares fit against W0: s = (weight · W0) / (W0 · W0), computed row by row.

Low-rank residual

The residual weight − diag(s) · W0 is approximated by a truncated SVD of the requested rank, giving U and V such that U · Vᵀ ≈ residual. U absorbs the singular values so U · Vᵀ reconstructs directly.

When rank == 0, the layer is a pure scaled shared block with no low-rank delta. The SVD uses a randomized algorithm (Halko et al., 2009) for large matrices and falls back to a full np.linalg.svd for small ones, where the randomized probe offers no speed advantage.

Key layout

The decomposition emits a flat state dict that model wrappers remap to their nested projection names:

W0.{fam}                 # G=1: one shared block per family
W0.{band}.{fam}          # G>1: one per (band, family)
layers.{i}.{fam}.s       # [out_dim]
layers.{i}.{fam}.U       # [out_dim, rank]
layers.{i}.{fam}.V       # [in_dim, rank]

Non-projection weights (embeddings, norms, biases) pass through unchanged. The adapter detects a checkpoint that already uses these keys and passes it through without re-decomposing.

To check how well theta reproduces the original weight, reconstruct_weight(s, W0, U, V) rebuilds the matrix and relative_frobenius_error(original, reconstructed) reports the relative error. See core.decompose for both.