r/ROCm • u/PainterNo8889 • 13h ago
[Help Wanted] Testing GPU detection & TFLOPS benchmark paths (NVIDIA/AMD) for my from-scratch SLM trainer
Hi guys,
I’m a 15-year-old developer building SLM-CREATOR—a from-scratch, hardware-aware trainer for small, hyperspecialized text models (capped at 100M parameters) in Python and Rust. Instead of using high-level wrappers or fine-tuning existing models, I’m hand-coding the entire pipeline (detection, sizing, BPE tokenizer, and training loop) to learn how everything works under the hood.
Project Link: github.com
Tracking Issue (#1): github.com
Where I need help:
I’m developing this on a CPU-only machine right now, so all of my GPU detection and benchmarking code has only been tested against mock CLI outputs. Since r/llmdevs folks run all sorts of local setups—multi-GPU rigs, hybrid laptop GPUs, and ROCm environments—I’d hugely appreciate it if anyone with an NVIDIA or AMD GPU could run architecture.py and see if it breaks or misidentifies anything on real hardware.
What the script is doing under the hood:
- Vendor CLI Parsing: Detects NVIDIA (
nvidia-smi), modern AMD (amd-smi), or legacy AMD (rocm-smi/rocminfo) to extract architecture, total VRAM (MiB), device index, and GPU name. - PyTorch Index Mapping: Sets
CUDA_DEVICE_ORDER=PCI_BUS_IDbefore importingtorchand matches vendor tool names againsttorch.cuda.get_device_name()so multi-GPU systems don't target the wrong device. - Matmul TFLOPS Benchmark: Runs a 12-second thermal soak (to get past laptop boost clocks), then takes the median of 10 runs of 20
4096x4096matmuls. It checkstorch.cuda.is_bf16_supported(including_emulation=False)(or compute capability >= 8.0) so it benchmarks nativebf16and falls back tofp32rather than getting fooled by slow software-emulatedbf16. - Deterministic Sizing: Pipes VRAM, training hours, TFLOPS, and token counts into a Rust binary (
sizer_core) to compute the max parameter count across memory, Chinchilla compute-time, and data limits.
How to test it (takes ~2 mins):
git clonehttps://github.com- Set up a venv and install
requirements.txt(with your GPU's PyTorch build). - Build the Rust sizer:
cd sizer_core && cargo build --release - Run
pythonarchitecture.pyfrom the root folder. - Paste your GPU model, OS, and the terminal output (or any tracebacks) in Issue #1 or in the comments here.
I'd love any feedback on CLI parsing edge cases, ROCm quirks, or how the thermal soak behaves on your card!