Instructions to use ubitech-edg/commandr-35b-cpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ubitech-edg/commandr-35b-cpt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ubitech-edg/commandr-35b-cpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ubitech-edg/commandr-35b-cpt")
model = AutoModelForCausalLM.from_pretrained("ubitech-edg/commandr-35b-cpt", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ubitech-edg/commandr-35b-cpt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ubitech-edg/commandr-35b-cpt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/commandr-35b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ubitech-edg/commandr-35b-cpt

SGLang

How to use ubitech-edg/commandr-35b-cpt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ubitech-edg/commandr-35b-cpt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/commandr-35b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ubitech-edg/commandr-35b-cpt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ubitech-edg/commandr-35b-cpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ubitech-edg/commandr-35b-cpt with Docker Model Runner:
```
docker model run hf.co/ubitech-edg/commandr-35b-cpt
```

commandr-35b-cpt

File size: 3,072 Bytes

---
{
  "language": ["en"],
  "license": "apache-2.0",
  "tags": [
    "text-generation",
    "causal-lm",
    "continual-pretraining",
    "lora",
    "axolotl",
    "deepspeed",
    "transformers",
    "commandr",
    "cohere",
    "eu-hpc"
  ],
  "datasets": [
    "arxiv",
    "gov",
    "news",
    "wikipedia"
  ],
  "metrics": [
    "loss"
  ],
  "library_name": "transformers",
  "framework": "pytorch",
  "base_model": "CohereLabs/c4ai-command-r-v01",
  "model_name": "commandr-35b-cpt",
  "pipeline_tag": "text-generation",
  "task_categories": ["text-generation"],
  "model_type": "AutoModelForCausalLM",
  "inference": {
    "parameters": {
      "max_new_tokens": 512,
      "temperature": 0.7,
      "top_p": 0.9
    }
  },
  "trained_on": [
    "Leonardo EuroHPC"
  ],
  "description": "Continual pretraining (CPT) of Cohere Command-R 35B using Axolotl and DeepSpeed ZeRO-1. The model was trained on scientific, governmental, news, and Wikipedia data with LoRA adapters to improve factual grounding and reasoning."
}
---

# Command-R 35B — CPT (Continual Pretraining with LoRA)

**Model type:** Causal Language Model  
**Base model:** [CohereLabs/c4ai-command-r-v01](https://huggingface.co/CohereLabs/c4ai-command-r-v01)  
**License:** Apache 2.0  
**Framework:** [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl)

---

## Overview

`commandr-35b-cpt` is a **continual-pretrained** version of Cohere's Command-R 35B model, trained with LoRA adapters for efficient enregy doman adaptation. 
The goal of CPT is to extend the model’s general reasoning, factual grounding, and domain knowledge across science, governance, and energy-domain text.

Training was performed on the **Leonardo EuroHPC** system using Axolotl with DeepSpeed ZeRO-1 optimization.

---

## Training Setup

**Objective:** Language modeling (unsupervised continual pretraining)  
**Adapter type:** LoRA  
**Precision:** bfloat16  
**Hardware:** 8 nodes × 2 × NVIDIA A100 64GB GPUs  
**Framework:** DeepSpeed ZeRO-1, Axolotl, PyTorch 2.5.1+cu121    
**Runtime:** ~24 hours  
**Checkpoints:** Saved every 1/5 of an epoch

---

## Dataset

Public energy domain text sources:

- `arxiv.jsonl` — scientific and technical papers  
- `gov.jsonl` — public governmental documents  
- `news.jsonl` — news articles  
- `wiki.jsonl` — Wikipedia text

---

## Hyperparameters

| Parameter | Value |
|------------|-------|
| Sequence length | 2048 |
| Micro batch size | 1 |
| Gradient accumulation | 4 |
| Epochs | 1 |
| Max steps | 10000 |
| Learning rate | 0.0002 |
| LR scheduler | cosine |
| Optimizer | AdamW (8-bit) |
| Warmup steps | 10 |
| Weight decay | 0.0 |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Gradient checkpointing | ✅ |
| Flash attention | ✅ |
| Auto resume | ✅ |
| Loss watchdog threshold | 5.0 |
| Loss watchdog patience | 3 |


## Tokenizer

**Tokenizer type:** `AutoTokenizer`  
**Special token:** `<|end_of_text|>` as `pad_token`