r/mffpc • • 21h ago

I built this! (MATX) Compact inference homelab build - 21.6L - 96GB VRAM

Thumbnail
gallery
14 Upvotes

Managed to basically max out this Mechanic Master C34 case. It JUST fits in the bottom of my IKEA FJÄLLBO shelf/workbench (highly recommend, btw) with the handle on.

Topline numbers:

  • Case size: 13.46 × 7.28 × 13.46 inches (21.6L)
  • 96 GB VRAM, 128 GB DDR4 RAM
  • Running 2x Qwen3.8 Flash Next at 262k context, 50-70 t/s

Runs like a champ. Both RTX A6000s set to 300 W power limits, seeing around ~76°C under load even in this tight case. Blowers FTW.

Full hardware list:

  • ASRock Rack X570D4U motherboard
  • AMD Ryzen 7 5700X cpu
  • WD_BLACK SN850X 2 TB NVMe ssd
  • NEMIX RAM 128 GB (4 × 32 GB), DDR4-3200 rated, running at 2666 MT/s
  • 2 × NVIDIA RTX A6000 48 GB, ECC enabled
  • NVIDIA 2-slot NVLink bridge
  • Mechanic Master C34 case with mesh aluminum side panel and dust filter
  • Corsair SF1000 1000 W SFX PSU
  • Noctua NH-D12L chromax black CPU cooler
  • 5 × Arctic P12 Pro PST 120 mm case fans

Managed to buy both A6000s on eBay a few months ago for a bit less than they’re going for now. The first one had a faulty vapor chamber, but Northwest Repair got it sorted and it’s finally home. Total cost was around $11k.

Been running the machine with just the one card for a while, hence the dust. Yes, I’m messy. No pretty cable management going on over here.

Highest single-day electrical usage so far is ~7.6 kwh (tho I'm not running inference 24/7 atm). Not super loud either. I have the GPUs configured to go to 100% fan as soon as they start working to keep the temps as low as possible. It does heat the room up a bit when they're running for a few hours straight... It'll keep my heating bill down this winter for sure.

Software side is mostly in place. Ubuntu Server w llama.cpp behind llama-swap. Sharing with a handful of friends & family for now, running OWUI on a vps, or tailscale (for those wanting direct api access). Also running a cline kanban fork I've built out for agent orchestration.

I'll let the AI tell you about itself. Go off short king...

Current lineup:

  • Qwen3.8-Flash-Next, GSQ-RCO IQ3_S: The big boy, main daily driver for near-frontier output. 125B main-model parameters with ~6B active per token, plus its massive n-gram embedding table accessed through mmap/lazy loading. Split across both A6000s, reasoning enabled, two configured 262K context slots. Seeing around 63–67 tokens/s for a single request, or about 51 tokens/s per request with two going in parallel. Warm concurrent tests managed 84–98 output tokens/s combined, including request overhead. With Gemma still resident, the whole setup was using about 82 GiB VRAM. Pretty darn happy with that.
  • Gemma 4 E4B, HauhauCS Aggressive Q8_K_P: Kept resident for fast responses/summaries/etc and quick vision (was that a delivery on the doorbell camera?), with a 32K context and reasoning disabled. This is the little guy that stays available to mostly automated services needing quick responses, while the bigger models swap around to serve users.

Also have a few toys installed:

  • Z-Image Turbo Q8_0, through stable-diffusion.cpp: roughly 10.5 GB of model memory in VRAM, with a tested 512×512 image in ~4.7 seconds at eight steps.
  • FLUX.2-dev Q4_K_S for image editing: using CPU parameter offload, with about 34 GB of parameters stored in system RAM. A tested 512×512 edit took ~70 seconds, or ~76 seconds including startup.
  • Wan2.2-TI2V-5B for text/image-to-video: an earlier single-A6000 test produced 81 frames at 832×480/16 fps—about five seconds of video—in ~112 seconds, using roughly 32 GB peak VRAM. Still more experimental than the text setup.
  • EmbeddingGemma 2, Nomic Embed Text v1.5, and multilingual E5 Small for embedding/RAG/routing duties.

Those image/video numbers are from earlier single-card testing; I haven’t rerun the whole lot on the finished build. The big models swap rather than all living in VRAM together.

Also, “two 262K slots” is configured capacity—I’ve tested concurrent Flash conversations around 10K tokens each, not two completely full 262K histories. The text numbers above include reasoning tokens.

---

Thanks robot. Back to work now.

Anyway. Glad the incredibly expensive card was saved and I managed to get it all together. Shout out (again) to captain save the day Northwest Repair on youtube for sending it back all fixed up.

Cheers all.


r/mffpc • • 23h ago

I built this! (MATX) bottleneck issue help

Thumbnail
gallery
41 Upvotes

hello, about a year ago someone wanted to sell/trade their 5080 which i acquired with my old gpu+cash, because the old gpu is pcie 4.0 is it worth upgrading to mobo w pcie 5.0 to maximize gpu.

i wanted to know due to holiday sales

CPU: AMD Ryzen 7 7800X3D
GPU: PNY GeForce RTX 5080 OC 16GB GDDR7
Motherboard: ASRock B650M-C (Micro-ATX)
RAM: TeamGroup T-Force DDR5 32GB (2×16GB) 6000MHz CL38
CPU Cooler: NZXT Kraken Elite 240mm LCD AIO
PSU: Corsair RM1000e 1000W
Storage: ADATA Legend 800 2TB NVMe SSD
Case: Montech Air 100
Front Fans: 3×120mm Arctic P12 Pro
Exhaust Fans: 1x120mm Arctic P12 Pro
Monitor: Sceptre ultrawide 165Hz


r/mffpc • • 18h ago

I built this! (MATX) Jonsbo D32 Pro

Thumbnail
gallery
89 Upvotes

First time building using this case. Amazing how easy it was to manage the cables.


r/mffpc • • 2h ago

I built this! (MATX) Ch270 build

Thumbnail
gallery
17 Upvotes

9950x3d with 5090 astral

The air cooler blocks alot of airflow to the ram and I see it going up to 70C sometimes, not sure how I can cool it down more in current setup

Airflow shown in second pic


r/mffpc • • 23h ago

I'm not quite finished yet. Lets go Hardware Hunting

Thumbnail gallery
2 Upvotes

Starting a Packard Bell 15L build this weekend. Found some gems in a container workstation. Let's go hunting!


r/mffpc • • 20h ago

I built this! (MATX) My favorite build (redux)

Thumbnail gallery
29 Upvotes