Text Generation
Transformers
Safetensors
English
Korean
code
fuse_glm
custom_code
lfm2
glm
mixture-of-experts
routed-experts
coding
code-generation
agentic
bf16
top-k-routing
trust-remote-code
conversational
Instructions to use HCHs/RivetCoder-9B-A4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HCHs/RivetCoder-9B-A4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="HCHs/RivetCoder-9B-A4B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("HCHs/RivetCoder-9B-A4B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HCHs/RivetCoder-9B-A4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HCHs/RivetCoder-9B-A4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HCHs/RivetCoder-9B-A4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/HCHs/RivetCoder-9B-A4B
- SGLang
How to use HCHs/RivetCoder-9B-A4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HCHs/RivetCoder-9B-A4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HCHs/RivetCoder-9B-A4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HCHs/RivetCoder-9B-A4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HCHs/RivetCoder-9B-A4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use HCHs/RivetCoder-9B-A4B with Docker Model Runner:
docker model run hf.co/HCHs/RivetCoder-9B-A4B
Download provenance/fusion-plan.json from HCHs/RivetCoder-9B-A4B: direct link, hf CLI and curl.
- Browser
- Download file 12.1 kB
-
https://huggingface.co/HCHs/RivetCoder-9B-A4B/resolve/main/provenance/fusion-plan.json
- Command line
-
hf download hf://HCHs/RivetCoder-9B-A4B/provenance/fusion-plan.json
-
curl -L -o fusion-plan.json https://huggingface.co/HCHs/RivetCoder-9B-A4B/resolve/main/provenance/fusion-plan.json
12.1 kB
| { | |
| "architecture": { | |
| "bridge": { | |
| "constraint": "P^T P = I", | |
| "fold_down": "W_down_folded = P^T @ W_down_glm", | |
| "fold_gate_up": "W_folded = W_glm @ P", | |
| "fold_router_init": "R_lfm_init = R_glm[selected_experts] @ P", | |
| "forward": "2048->4096", | |
| "kind": "tied frozen linear projection", | |
| "reverse": "4096->2048" | |
| }, | |
| "candidate_expert_count": 480, | |
| "donor": "zai-org/GLM-5.3-Flash", | |
| "donor_hidden_size": 4096, | |
| "expert_intermediate_size": 2048, | |
| "experts_per_layer": 16, | |
| "folded_expert_parameter_count": 6039797760, | |
| "folded_parameter_count_per_expert": 12582912, | |
| "frozen": [ | |
| "LFM host", | |
| "GLM experts", | |
| "bridge" | |
| ], | |
| "host": "LiquidAI/LFM2.5-2.6B", | |
| "host_hidden_size": 2048, | |
| "maximum_active_experts_per_token": 120, | |
| "runtime_top_k": 4, | |
| "target_layer_count": 30, | |
| "trainable": [ | |
| "per-layer router", | |
| "token gate", | |
| "bounded residual scale" | |
| ] | |
| }, | |
| "canonical_mapping_projection_sha256": "f02ef1f5152697e814f2fb41fedad3f7ea6738ac29eed1587f1d9948ed7360ac", | |
| "mapping_status": "provisional", | |
| "mappings": [ | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 149, | |
| 171, | |
| 186, | |
| 98, | |
| 286, | |
| 189, | |
| 12, | |
| 10, | |
| 202, | |
| 177, | |
| 66, | |
| 24, | |
| 118, | |
| 107, | |
| 220, | |
| 47 | |
| ], | |
| "source_layer": 3, | |
| "target_layer": 0 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 241, | |
| 243, | |
| 173, | |
| 268, | |
| 39, | |
| 45, | |
| 216, | |
| 197, | |
| 205, | |
| 223, | |
| 89, | |
| 178, | |
| 248, | |
| 93, | |
| 142, | |
| 74 | |
| ], | |
| "source_layer": 4, | |
| "target_layer": 1 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 160, | |
| 276, | |
| 104, | |
| 234, | |
| 16, | |
| 83, | |
| 209, | |
| 260, | |
| 270, | |
| 282, | |
| 55, | |
| 174, | |
| 236, | |
| 256, | |
| 137, | |
| 206 | |
| ], | |
| "source_layer": 6, | |
| "target_layer": 2 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 81, | |
| 57, | |
| 131, | |
| 233, | |
| 68, | |
| 129, | |
| 94, | |
| 145, | |
| 208, | |
| 262, | |
| 184, | |
| 48, | |
| 211, | |
| 51, | |
| 44, | |
| 72 | |
| ], | |
| "source_layer": 7, | |
| "target_layer": 3 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 129, | |
| 142, | |
| 181, | |
| 82, | |
| 74, | |
| 95, | |
| 177, | |
| 88, | |
| 202, | |
| 150, | |
| 6, | |
| 239, | |
| 98, | |
| 271, | |
| 221, | |
| 219 | |
| ], | |
| "source_layer": 9, | |
| "target_layer": 4 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 96, | |
| 98, | |
| 285, | |
| 75, | |
| 81, | |
| 284, | |
| 111, | |
| 9, | |
| 177, | |
| 268, | |
| 0, | |
| 159, | |
| 70, | |
| 28, | |
| 65, | |
| 144 | |
| ], | |
| "source_layer": 10, | |
| "target_layer": 5 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 246, | |
| 103, | |
| 109, | |
| 138, | |
| 128, | |
| 69, | |
| 178, | |
| 219, | |
| 28, | |
| 159, | |
| 220, | |
| 66, | |
| 81, | |
| 91, | |
| 166, | |
| 47 | |
| ], | |
| "source_layer": 11, | |
| "target_layer": 6 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 178, | |
| 145, | |
| 13, | |
| 25, | |
| 75, | |
| 187, | |
| 70, | |
| 169, | |
| 235, | |
| 20, | |
| 142, | |
| 209, | |
| 156, | |
| 52, | |
| 99, | |
| 177 | |
| ], | |
| "source_layer": 13, | |
| "target_layer": 7 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 143, | |
| 121, | |
| 277, | |
| 228, | |
| 46, | |
| 72, | |
| 285, | |
| 243, | |
| 70, | |
| 180, | |
| 86, | |
| 23, | |
| 150, | |
| 205, | |
| 28, | |
| 275 | |
| ], | |
| "source_layer": 14, | |
| "target_layer": 8 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 229, | |
| 219, | |
| 23, | |
| 224, | |
| 64, | |
| 244, | |
| 273, | |
| 90, | |
| 66, | |
| 185, | |
| 77, | |
| 237, | |
| 96, | |
| 136, | |
| 226, | |
| 109 | |
| ], | |
| "source_layer": 16, | |
| "target_layer": 9 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 279, | |
| 172, | |
| 114, | |
| 217, | |
| 9, | |
| 66, | |
| 157, | |
| 180, | |
| 210, | |
| 102, | |
| 150, | |
| 283, | |
| 175, | |
| 89, | |
| 32, | |
| 186 | |
| ], | |
| "source_layer": 17, | |
| "target_layer": 10 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 157, | |
| 281, | |
| 111, | |
| 228, | |
| 27, | |
| 5, | |
| 23, | |
| 21, | |
| 282, | |
| 261, | |
| 92, | |
| 81, | |
| 271, | |
| 85, | |
| 80, | |
| 136 | |
| ], | |
| "source_layer": 19, | |
| "target_layer": 11 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 269, | |
| 92, | |
| 42, | |
| 15, | |
| 173, | |
| 216, | |
| 141, | |
| 282, | |
| 29, | |
| 210, | |
| 35, | |
| 110, | |
| 260, | |
| 64, | |
| 245, | |
| 247 | |
| ], | |
| "source_layer": 20, | |
| "target_layer": 12 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 123, | |
| 165, | |
| 166, | |
| 178, | |
| 5, | |
| 182, | |
| 33, | |
| 37, | |
| 110, | |
| 211, | |
| 186, | |
| 158, | |
| 63, | |
| 106, | |
| 159, | |
| 267 | |
| ], | |
| "source_layer": 21, | |
| "target_layer": 13 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 269, | |
| 234, | |
| 202, | |
| 229, | |
| 106, | |
| 90, | |
| 67, | |
| 11, | |
| 133, | |
| 152, | |
| 254, | |
| 70, | |
| 219, | |
| 13, | |
| 57, | |
| 10 | |
| ], | |
| "source_layer": 23, | |
| "target_layer": 14 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 135, | |
| 49, | |
| 193, | |
| 207, | |
| 272, | |
| 25, | |
| 45, | |
| 101, | |
| 177, | |
| 56, | |
| 29, | |
| 23, | |
| 119, | |
| 198, | |
| 35, | |
| 159 | |
| ], | |
| "source_layer": 24, | |
| "target_layer": 15 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 227, | |
| 49, | |
| 43, | |
| 214, | |
| 245, | |
| 127, | |
| 248, | |
| 150, | |
| 25, | |
| 223, | |
| 246, | |
| 203, | |
| 145, | |
| 57, | |
| 89, | |
| 208 | |
| ], | |
| "source_layer": 26, | |
| "target_layer": 16 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 134, | |
| 196, | |
| 89, | |
| 66, | |
| 106, | |
| 205, | |
| 60, | |
| 121, | |
| 209, | |
| 224, | |
| 286, | |
| 114, | |
| 26, | |
| 248, | |
| 188, | |
| 194 | |
| ], | |
| "source_layer": 27, | |
| "target_layer": 17 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 69, | |
| 214, | |
| 198, | |
| 283, | |
| 272, | |
| 56, | |
| 89, | |
| 107, | |
| 60, | |
| 50, | |
| 245, | |
| 61, | |
| 185, | |
| 67, | |
| 97, | |
| 99 | |
| ], | |
| "source_layer": 28, | |
| "target_layer": 18 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 93, | |
| 139, | |
| 36, | |
| 279, | |
| 29, | |
| 97, | |
| 111, | |
| 64, | |
| 0, | |
| 195, | |
| 242, | |
| 24, | |
| 140, | |
| 101, | |
| 8, | |
| 136 | |
| ], | |
| "source_layer": 30, | |
| "target_layer": 19 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 147, | |
| 241, | |
| 11, | |
| 251, | |
| 178, | |
| 268, | |
| 118, | |
| 154, | |
| 103, | |
| 248, | |
| 25, | |
| 157, | |
| 197, | |
| 66, | |
| 210, | |
| 98 | |
| ], | |
| "source_layer": 31, | |
| "target_layer": 20 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 206, | |
| 72, | |
| 170, | |
| 171, | |
| 237, | |
| 8, | |
| 168, | |
| 61, | |
| 172, | |
| 202, | |
| 130, | |
| 222, | |
| 256, | |
| 128, | |
| 226, | |
| 252 | |
| ], | |
| "source_layer": 33, | |
| "target_layer": 21 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 52, | |
| 258, | |
| 18, | |
| 59, | |
| 80, | |
| 172, | |
| 137, | |
| 40, | |
| 197, | |
| 89, | |
| 35, | |
| 154, | |
| 84, | |
| 19, | |
| 255, | |
| 38 | |
| ], | |
| "source_layer": 34, | |
| "target_layer": 22 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 220, | |
| 31, | |
| 280, | |
| 106, | |
| 10, | |
| 178, | |
| 234, | |
| 36, | |
| 7, | |
| 212, | |
| 187, | |
| 199, | |
| 221, | |
| 47, | |
| 186, | |
| 147 | |
| ], | |
| "source_layer": 36, | |
| "target_layer": 23 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 36, | |
| 197, | |
| 33, | |
| 93, | |
| 181, | |
| 134, | |
| 84, | |
| 244, | |
| 264, | |
| 164, | |
| 207, | |
| 202, | |
| 71, | |
| 59, | |
| 214, | |
| 284 | |
| ], | |
| "source_layer": 37, | |
| "target_layer": 24 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 54, | |
| 143, | |
| 130, | |
| 15, | |
| 14, | |
| 107, | |
| 76, | |
| 225, | |
| 8, | |
| 195, | |
| 281, | |
| 212, | |
| 161, | |
| 177, | |
| 258, | |
| 259 | |
| ], | |
| "source_layer": 38, | |
| "target_layer": 25 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 5, | |
| 187, | |
| 78, | |
| 105, | |
| 165, | |
| 9, | |
| 156, | |
| 139, | |
| 235, | |
| 121, | |
| 62, | |
| 256, | |
| 14, | |
| 201, | |
| 180, | |
| 277 | |
| ], | |
| "source_layer": 40, | |
| "target_layer": 26 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 144, | |
| 154, | |
| 125, | |
| 98, | |
| 92, | |
| 123, | |
| 196, | |
| 80, | |
| 141, | |
| 83, | |
| 249, | |
| 250, | |
| 20, | |
| 34, | |
| 134, | |
| 184 | |
| ], | |
| "source_layer": 41, | |
| "target_layer": 27 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 249, | |
| 148, | |
| 151, | |
| 221, | |
| 18, | |
| 87, | |
| 239, | |
| 105, | |
| 207, | |
| 35, | |
| 147, | |
| 20, | |
| 84, | |
| 130, | |
| 52, | |
| 17 | |
| ], | |
| "source_layer": 43, | |
| "target_layer": 28 | |
| }, | |
| { | |
| "expert_count": 16, | |
| "expert_ids": [ | |
| 44, | |
| 212, | |
| 152, | |
| 127, | |
| 244, | |
| 138, | |
| 156, | |
| 266, | |
| 24, | |
| 135, | |
| 113, | |
| 185, | |
| 147, | |
| 186, | |
| 17, | |
| 184 | |
| ], | |
| "source_layer": 44, | |
| "target_layer": 29 | |
| } | |
| ], | |
| "schema": "lfm-glm-fixed16-top4-fusion-plan", | |
| "schema_version": 1, | |
| "source": { | |
| "layer_map": "<workspace>\\manifests\\lfm_to_glm_layer_map.json", | |
| "layer_map_method": "monotonic normalized depth over GLM main sparse layers 3..44; replace with CKA/CCA mapping after paired activation collection", | |
| "layer_map_sha256": "bd7e4efc18148ff1ebd66f58adfc994b1383a4d56d166cc2b576a973b9c4f4d4", | |
| "selection_profile": "<workspace>\\artifacts\\glm-code-expert-ranking\\selection_profile_fixed16_hybrid.json", | |
| "selection_profile_sha256": "48ee7d8528836935ceffb256050c647b087c31a801a76de3d9169235c8efc58c" | |
| } | |
| } | |