Quickstart

Install the toolchain and run your first weightless generation with Gemma 3 1B.

Go from a clean checkout to a weightless generation in a few commands. The framework targets osx-arm64 and is driven entirely through MAX’s standard CLI.

Prerequisites

• pixi for the MAX toolchain (the project ships a pixi.toml and lockfile).
• A HuggingFace token with access to google/gemma-3-1b-it, saved at ~/.cache/huggingface/token.
• An Apple-silicon Mac (Metal). CPU execution also works.

Install

The pixi.toml pins modular (MAX + Mojo) and the Python dependencies. Resolve the environment from the project root:

cd weightless_fw
pixi install

Generate

run_max.py wraps the MAX CLI and clears stale MODULAR_* environment variables before dispatching, so the pixi environment’s SDK is used. Pass --custom-architectures weightless_fw so MAX loads the weightless version of Gemma 3:

pixi run python run_max.py generate \
  --model google/gemma-3-1b-it \
  --custom-architectures weightless_fw \
  --prompt "Explain weightless inference in one sentence."

At load time the framework auto-decomposes the HuggingFace checkpoint into a theta bundle, then reconstructs each projection inside the generated_linear kernel as tokens are produced.

Serve

The same wrapper starts an OpenAI-compatible server. The project convention is port 3100:

pixi run python run_max.py serve \
  --model google/gemma-3-1b-it \
  --custom-architectures weightless_fw \
  --port 3100

Once it’s up, call it like any MAX endpoint:

curl http://localhost:3100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-3-1b-it",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Next steps