r/ROCm • • 13h ago

[Help Wanted] Testing GPU detection & TFLOPS benchmark paths (NVIDIA/AMD) for my from-scratch SLM trainer

0 Upvotes

Hi guys,

I’m a 15-year-old developer building SLM-CREATOR—a from-scratch, hardware-aware trainer for small, hyperspecialized text models (capped at 100M parameters) in Python and Rust. Instead of using high-level wrappers or fine-tuning existing models, I’m hand-coding the entire pipeline (detection, sizing, BPE tokenizer, and training loop) to learn how everything works under the hood.

Project Link: github.com
Tracking Issue (#1): github.com

Where I need help:
I’m developing this on a CPU-only machine right now, so all of my GPU detection and benchmarking code has only been tested against mock CLI outputs. Since r/llmdevs folks run all sorts of local setups—multi-GPU rigs, hybrid laptop GPUs, and ROCm environments—I’d hugely appreciate it if anyone with an NVIDIA or AMD GPU could run architecture.py and see if it breaks or misidentifies anything on real hardware.

What the script is doing under the hood:

  • Vendor CLI Parsing: Detects NVIDIA (nvidia-smi), modern AMD (amd-smi), or legacy AMD (rocm-smi/rocminfo) to extract architecture, total VRAM (MiB), device index, and GPU name.
  • PyTorch Index Mapping: Sets CUDA_DEVICE_ORDER=PCI_BUS_ID before importing torch and matches vendor tool names against torch.cuda.get_device_name() so multi-GPU systems don't target the wrong device.
  • Matmul TFLOPS Benchmark: Runs a 12-second thermal soak (to get past laptop boost clocks), then takes the median of 10 runs of 20 4096x4096 matmuls. It checks torch.cuda.is_bf16_supported(including_emulation=False) (or compute capability >= 8.0) so it benchmarks native bf16 and falls back to fp32 rather than getting fooled by slow software-emulated bf16.
  • Deterministic Sizing: Pipes VRAM, training hours, TFLOPS, and token counts into a Rust binary (sizer_core) to compute the max parameter count across memory, Chinchilla compute-time, and data limits.

How to test it (takes ~2 mins):

  1. git clone https://github.com
  2. Set up a venv and install requirements.txt (with your GPU's PyTorch build).
  3. Build the Rust sizer: cd sizer_core && cargo build --release
  4. Run python architecture.py from the root folder.
  5. Paste your GPU model, OS, and the terminal output (or any tracebacks) in Issue #1 or in the comments here.

I'd love any feedback on CLI parsing edge cases, ROCm quirks, or how the thermal soak behaves on your card!


r/ROCm • • 2h ago

3 x AI 9700 build - 96gb VRAM

Thumbnail gallery
2 Upvotes

r/ROCm • • 3h ago

AMD ROCm 10.2 Released With Latest Optimizations, Expanded Ryzen Laptop Support

Thumbnail
phoronix.com
17 Upvotes

r/ROCm • • 13h ago

How to Underclock or Undervolt AMD V620 32GB?

Post image
3 Upvotes