If your GPU can run inference, it is now also capable of performing fine-tuning

Research
1

I spent the last few months building a new sparse fine-tuning method for MoE models called USAF.

The goal was simple: if your GPU can run inference on an MoE model, it should also be able to fine-tune it.

On my AMD RX 6750 XT (12 GB), I can fine-tune Qwen3-30B-A3B by training sparse expert weights and the router instead of adapters.

The project is completely open source under the Apache 2.0 license. I’m not trying to build a business, sell anything, or monetize it in any way—I just wanted to share something I built that I think is genuinely interesting.

I’d love to hear your feedback, especially from people working with MoE models.

GitHub: https://github.com/tsuyu122/usaf

2

Interesting work, perhaps I can integrate this with AReno, which I am playing and developing now.

3

This is really interesting. Getting Qwen3-30B-A3B fine-tuning down to a 12GB consumer GPU is impressive.

I’m working on a related problem from the infrastructure side: making model training easier without having to manually choose GPUs, provision compute, and configure the training environment each time.

Would be interesting to see whether a method like USAF could eventually plug into that kind of automated training workflow.

4

Category: Show and Tell
Title: Celestium: Hardware‑Agnostic Local Engine with Autonomous VRAM Cleanup

I’m sharing a small diagnostic from Celestium, a lightweight local inference engine built to run on any hardware, including CPU‑only setups.

Key behaviors (essential data):

  • Hardware‑agnostic pipeline (GPU or CPU, no config changes)

  • Automatic VRAM cleanup before each inference stage

  • FP16 / FP32 adaptive precision

  • Runs SDXL, RealESRGANx2, BuffaloCore

  • No CUDA required (full CPU fallback)

  • Stable on 4GB / 6GB / 8GB GPUs

  • No fragmentation spikes during long multi‑stage inference

Notes:

  • FP16 enables SDXL on low‑VRAM cards

  • CPU mode is slower but fully functional

  • RealESRGANx2 integrated with minimal overhead

  • VRAM stability improved across extended sessions

Sharing this to compare with other hardware‑adaptive or memory‑aware local engines.
Benchmarks or fragmentation‑mitigation approaches are welcome.