Running a model

Serve and generate with a weightless model through MAX's standard CLI and the run_max.py wrapper.

A weightless model is a MAX custom architecture, so you run it with the same max generate and max serve commands as any MAX model. The only additions are --custom-architectures weightless_fw, which tells MAX to load the package, and the run_max.py wrapper, which fixes the environment before dispatching.

Why run_max.py

On some machines the max._core C extension resolves the Mojo import path to a stale location from a previous install and writes it into the environment, overriding the pixi SDK. run_max.py imports max._core to trigger that behavior, strips the stale MODULAR_* variables so the pixi environment’s defaults take effect, adds the repository root to sys.path so import weightless_fw resolves, and then calls the MAX entrypoint.

Always invoke the CLI through it:

pixi run python run_max.py <generate|serve> ...

How the architecture is loaded

When you pass --custom-architectures weightless_fw, MAX imports the package and reads its ARCHITECTURES list. The weightless Gemma 3 registers under the name Gemma3ForCausalLM, the same name as MAX’s built-in, and a custom architecture wins over the built-in of that name. The package also clears MAX’s lazy registration entry for that name so the built-in is never materialized after the override.

The same --custom-architectures weightless_fw also registers the weightless Gemma 4 under Gemma4ForConditionalGeneration (for example --model google/gemma-4-31B-it); both architectures load from the one package.

At load time the safetensors weight adapter auto-decomposes the HuggingFace checkpoint into a theta bundle (unless it already uses theta keys), and the pipeline model builds a graph whose linear projections are WeightlessLinear layers backed by the generated_linear kernel.

Generate

pixi run python run_max.py generate \
  --model google/gemma-3-1b-it \
  --custom-architectures weightless_fw \
  --prompt "Write a haiku about compression."

Serve

pixi run python run_max.py serve \
  --model google/gemma-3-1b-it \
  --custom-architectures weightless_fw \
  --port 3100

This exposes MAX’s OpenAI-compatible endpoints (for example /v1/chat/completions) on the chosen port.

Supported encodings

The weightless Gemma 3 architecture defaults to bfloat16 and also supports float32. It runs on a single device: multi_gpu_supported is false, because tensor parallelism doesn’t yet compose with in-kernel reconstruction.