frogggias/Ripple-65B-Preview-MLX-4bit

Ripple-65B Preview, MLX 4-bit

An unofficial MLX conversion of ricursiveAI/Ripple-65B-Preview (the instruction-tuned BF16 release, revision 240d5966cbe0) for Apple Silicon. It isn't made or endorsed by Ricursive. All linear and embedding weights are MLX affine 4-bit, groups of 64; norms and the per-layer value-residual mixes stay BF16. The tokenizer and chat template are the original's, unchanged.

• Size: 37 GB (4.5 bits per weight). Use a Mac with 64 GB or more of memory.
• An 8-bit version: frogggias/Ripple-65B-Preview-MLX-8bit.

Use

mlx-lm doesn't know Ripple's architecture yet, so this repo carries its own model file, ricursive.py, an MLX port of Ricursive's modeling_ricursive.py. mlx-lm runs it only with --trust-remote-code; it's about 280 lines of plain MLX, so please read it first.

pip install "mlx-lm>=0.32.2"
mlx_lm.generate --model frogggias/Ripple-65B-Preview-MLX-4bit --trust-remote-code \
  -p "Write a Python function that checks whether a string is a palindrome." -m 2048

from mlx_lm import load, generate

model, tokenizer = load("frogggias/Ripple-65B-Preview-MLX-4bit", trust_remote_code=True)
prompt = tokenizer.apply_chat_template([{"role": "user", "content": "Hello!"}], add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024))

OpenAI-compatible server: mlx_lm.server --model frogggias/Ripple-65B-Preview-MLX-4bit --trust-remote-code. mlx-lm parses Ripple's tool calls itself (<tool_call> JSON, the json_tools parser it infers from the chat template). Ripple's reasoning arrives inside the reply's content as <think>…</think>, because its tokenizer has no single-token think markers for mlx-lm to split on. Ricursive publishes no sampling defaults.

Quality against the BF16 original

Ricursive's own PyTorch code ran the BF16 release on three prompts (a short chat question, a 2,200-token chat about code, and a raw code completion); this conversion then read the same tokens. The 4-bit model is close to BF16: top-1 agreement 0.92–1.00, KL 0.005–0.09; greedy output equal to BF16's on two prompts, and on the long one it takes a different, equally sensible path after 10 tokens.

Speed

Measured on a Mac Studio with an M5 Ultra (256 GB) and mlx-lm 0.32.2:

• Decode: 22.0–23.3 tok/s for one stream, at 1K to 32K tokens of context.
• Prefill: 535–565 tok/s, so a fresh 32K-token prompt takes about a minute.
• Served at 32K context through mlx_lm.server: 20.2 tok/s for one user, 32.6 total for two, 42.5 for three.

Ripple is a dense 65B model, so every token reads all of the weights. Only its 16 global-attention layers keep the whole context (64 KiB of KV cache per token); the other 64 layers keep a 1,024-token window.

How it was made

mlx_lm.convert -q --q-bits 4 --q-group-size 64 --dtype bfloat16 on the BF16 checkpoint, with ricursive.py as the model. The port was checked against modeling_ricursive.py on small random-weight models: float32 logits agree to about 5e-6 for a full forward pass, cached decoding, and prompts split into chunks longer and shorter than the sliding window; the quantized forward equals the reference run with the same dequantized weights to 2e-6. Ricursive's later commits (after 240d5966cbe0) change only their SGLang serving files.

License

Apache-2.0, as the original; the license is in LICENSE. Changes from the original: the weights are quantized to MLX affine 4-bit, config.json adds "model_file": "ricursive.py", and ricursive.py is an MLX port of Ricursive's modeling_ricursive.py.