Instructions to use noumenon-labs/Fleet-Semantic-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use noumenon-labs/Fleet-Semantic-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="noumenon-labs/Fleet-Semantic-4B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("noumenon-labs/Fleet-Semantic-4B", trust_remote_code=True, device_map="auto")

PEFT
How to use noumenon-labs/Fleet-Semantic-4B with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use noumenon-labs/Fleet-Semantic-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "noumenon-labs/Fleet-Semantic-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "noumenon-labs/Fleet-Semantic-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/noumenon-labs/Fleet-Semantic-4B

SGLang

How to use noumenon-labs/Fleet-Semantic-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "noumenon-labs/Fleet-Semantic-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "noumenon-labs/Fleet-Semantic-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "noumenon-labs/Fleet-Semantic-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "noumenon-labs/Fleet-Semantic-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use noumenon-labs/Fleet-Semantic-4B with Docker Model Runner:
```
docker model run hf.co/noumenon-labs/Fleet-Semantic-4B
```

Fleet Semantic 4B

Fleet is a dynamically extensible Semantic Inference Language Model (SiLM) built on Qwen/Qwen3-4B-Instruct-2507.

SiLM is a project term used by Fleet for language models that perform semantic inference over an internal specialist bank during generation, dynamically deciding whether the base model or one or more specialists should influence each computation.

Fleet is not an external prompt router.

The routing system lives inside the causal language model itself through the custom FleetForCausalLM architecture.

Architecture

                 User prompt
                     │
                     ▼
        ┌──────────────────────────┐
        │     FleetForCausalLM     │
        │                          │
        │     Qwen3-4B backbone    │
        └────────────┬─────────────┘
                     │
              hidden states
                     │
                     ▼
        ┌──────────────────────────┐
        │ Semantic Inference Router│
        │                          │
        │ NULL / expert scores     │
        └────────────┬─────────────┘
                     │
            dynamic k = 0 / 1 / 2
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
        BASE       Expert A   Expert B
          │          │          │
          └──────────┴──────────┘
                     │
                     ▼
               next token

Fleet replaces selected Qwen projections with native FleetLinear modules.

A Fleet projection contains:

FleetLinear
├── weight
├── expert_A
├── expert_B
├── expert_scaling
├── expert_ids
└── weight_prototypes

The model also contains a dynamic semantic prototype bank:

FleetRouterState
├── semantic_prototypes
├── semantic_thresholds
├── NULL/base prototype
└── prompt/decode semantic prior

Routing therefore happens inside model inference, rather than through an external classifier that chooses a different model before generation.

Installed specialists

This checkpoint currently contains six specialists:

math_reasoning
medical_reasoning
legal_ops
intent
function_calling
structured

The base Qwen network remains a first-class path.

Fleet can choose:

k = 0  → base Qwen only
k = 1  → one specialist
k = 2  → mixture of two specialists

A specialist does not need to activate for every request.

Loading Fleet

Fleet is a custom Transformers architecture, so enable remote model code.

import torch

from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM,
)

repo = "noumenon-labs/Fleet-Semantic-4B"

tokenizer = AutoTokenizer.from_pretrained(
    repo,
)

model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype=torch.float16,
    device_map="auto",
)

print(type(model).__name__)
# FleetForCausalLM

No separate Qwen download is required at inference.

The published checkpoint contains the complete backbone and Fleet parameters.

Inference

messages = [
    {
        "role": "user",
        "content": (
            "A store sells 8 pens for $12. "
            "At the same rate, how much do 14 pens cost?"
        ),
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)

device = (
    model
    .get_input_embeddings()
    .weight
    .device
)

inputs = {
    key: value.to(device)
    for key, value
    in inputs.items()
}

prompt_length = (
    inputs["input_ids"].shape[-1]
)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=160,
        do_sample=False,
    )

answer = tokenizer.decode(
    output[
        0,
        prompt_length:
    ],
    skip_special_tokens=True,
)

print(answer)

You do not manually select an expert.

There is intentionally no normal inference API like:

model.generate(..., expert="math")

Fleet performs the routing internally.

Dynamic experts

Fleet specialists are runtime-installable.

List installed experts

model.experts()

Install

Fleet adapters use the portable .fleet package format.

model.install(
    "robotics.fleet"
)

A Hub-hosted package may also be installed through Fleet's hf:// source syntax:

model.install(
    "hf://org/repository/robotics.fleet"
)

Uninstall

model.uninstall(
    "robotics"
)

Export an installed specialist

model.export_adapter(
    "structured",
    "structured.fleet",
)

The .fleet package carries the specialist's:

LoRA A/B tensors
source-module eligibility
weight-space prototypes
semantic prototypes
semantic thresholds
metadata
semantic-space compatibility fingerprint

Why dynamic expert banks?

Traditional multi-adapter systems often assume a fixed set of adapters.

Fleet instead supports:

FleetForCausalLM.install()
FleetForCausalLM.uninstall()
FleetForCausalLM.experts()

and dynamically resizes:

FleetLinear expert banks
+
semantic prototype bank

without changing the Qwen backbone.

Individual projections maintain sparse local expert banks.

An expert that only targets attention projections does not allocate empty tensors for every MLP projection.

Heterogeneous LoRA compilation

Fleet specialists do not have to begin with the same LoRA rank or target-module set.

The Fleet compiler converts heterogeneous adapters into a shared canonical ABI.

For lower/equal source rank, Fleet can preserve the update through scaling and zero-padding.

For larger source ranks, Fleet performs a structured low-rank approximation using QR decomposition followed by an SVD of the reduced core.

Conceptually:

arbitrary compatible LoRA
        │
        ▼
source rank / targets / scaling
        │
        ▼
Fleet compiler
        │
        ▼
canonical expert ABI
        │
        ▼
FleetAdapter / FleetForCausalLM

Source-module eligibility is preserved so a compiled expert cannot accidentally activate in modules it never originally targeted.

Semantic routing

Fleet's router combines two signals:

semantic expert-vs-NULL score
+
small LoRA weight-space routing signal

The semantic signal is dominant.

The NULL route represents:

No specialist is necessary for this request.

This allows generic requests to remain primarily on the original backbone.

During decoding, Fleet keeps a strong semantic prior from the original user prompt so generated text does not easily cause the model to drift into an unrelated expert.

Validation

This checkpoint has successfully passed a standalone architecture round trip:

checkpoint
    ↓
AutoModelForCausalLM.from_pretrained(...)
    ↓
FleetForCausalLM
    ↓
dynamic expert export
    ↓
uninstall
    ↓
semantic bank shrinks
    ↓
reinstall
    ↓
semantic bank restored
    ↓
generation

Validated architecture:

FleetForCausalLM

Validated dynamic expert round trip:

structured → export → uninstall → reinstall

The model also successfully loaded under device_map="auto" with Fleet tensors offloaded through Accelerate.

Current status

Fleet is a research prototype.

The current validation proves:

standalone checkpoint loading;
native FleetForCausalLM;
generation;
dynamic expert registration;
.fleet export;
expert uninstall/reinstall;
dynamic semantic-bank mutation;
Accelerate CPU offload compatibility;
model serialization.

It does not yet establish that Fleet improves every task over the Qwen backbone.

The next evaluation stage compares:

Qwen3-4B base
vs.
individual source specialist
vs.
Fleet k=0/1/2

across specialist tasks and generic NULL-route prompts.

Current specialists

The included adapters are intended for:

quantitative reasoning;
medical reasoning;
narrow legal-operations extraction;
intent classification;
function calling;
strict structured output.

Fleet's architecture is designed so additional compatible .fleet specialists can be installed later without rebuilding the backbone.

Requirements

Recommended:

pip install -U \
  torch \
  transformers \
  accelerate \
  safetensors \
  huggingface_hub

Use a recent Transformers release with Qwen3 support.

License and third-party models

Fleet contains the Qwen3-4B-Instruct-2507 backbone plus compiled specialist transformations originating from multiple third-party adapter repositories.

Licensing and exact source revisions are retained in Fleet's model configuration and expert metadata.

Because the checkpoint combines components with multiple source licenses, the Hub repository is marked license: other.

Users should review the upstream model and adapter licenses before redistribution or commercial deployment.

Research direction

Fleet explores whether independently trained specialist LoRAs can behave as a hot-swappable semantic expert system inside one causal language model while preserving:

expert identity;
source-layer eligibility;
the base model as a NULL expert;
heterogeneous adapter provenance;
post-hoc installation and removal.

The long-term goal is a model ecosystem where capabilities can be distributed as small .fleet packages rather than requiring a new multi-billion-parameter checkpoint for every capability combination.

Architecture: FleetForCausalLM
Family: Semantic Inference Language Model (SiLM)
Backbone: Qwen/Qwen3-4B-Instruct-2507
Expert format: FleetAdapter (.fleet)
Routing: dynamic semantic k = 0 / 1 / 2

Downloads last month: 65

Safetensors

Model size

4B params

Tensor type

I64

F32

F16

Model tree for noumenon-labs/Fleet-Semantic-4B

Base model

Qwen/Qwen3-4B-Instruct-2507

Adapter

(5,877)

this model