r/IntelArc • • 11h ago

Benchmark 2× Arc Pro B70 on Windows update: Qwen3.8-27B Q8_0 now 67.5 t/s over 610 real agent requests (was 51). DSpine ported to llama.cpp, draft trees, and an honest benchmark-vs-real-use comparison

Thumbnail
reddit.com
1 Upvotes

r/IntelArc • • 12h ago

News inferzero: an LLM engine written from scratch for Intel Arc, from laptop iGPUs to 4× B60. Still far from what the cards can do, looking for contributors

20 Upvotes

I’ve been building inferzero, an open-source (Apache 2.0) inference engine only for Intel GPUs: C++ on Level Zero, kernels in OpenCL C, GGUF files as they are. It started on a Core Ultra 5 125H laptop (no XMX, DP4a only) and now runs on Arc B-series cards, Alchemist, and several GPUs at once.

Qwen3.8 27B (IQ3_S), measured by users on their own hardware:

GPU4K promptGeneration
Arc Pro B70 (Linux)2,897 tok/s35.2 tok/s, 70.6 with MTP
Arc B580 12 GB (PCIe 3.0)1,877 tok/s7.1 tok/s (the model barely fits: 0.5 GB of it in RAM, with ARC_HOST_KEEP_GB=0.75)
Arc Pro B601,726 tok/s22.7 tok/s
Arc A770 16 GB1,223 tok/s12.8 tok/s
Arc B390 (Panther Lake iGPU)592 tok/s–
Arc 140T (Core Ultra 9 285H iGPU)390 tok/s6.0, 16.9 with MTP + DFlash2
4× Arc Pro B60 (one model split, older release)1,888 tok/s53 tok/s

What’s in it:

  • Qwen3.5 / 3.6 / 3.8 (dense, MoE, Flash Next with 512 experts), Gemma 4, DeepSeek-V4.1-Flash over several GPUs.
  • MoE experts streamed from the SSD when the model is bigger than the GPU memory: the 35B-A3B and Flash Next run on a 32 GB laptop.
  • Speculative decoding (MTP heads, DFlash2) that gives exactly the text of plain decoding.
  • Every model checked against an independent f32 CPU reference (KL ≤ 0.015). Same prompt, same logits, bit for bit.
  • An OpenAI / Anthropic / Ollama-compatible server with a web UI (web search, voice, transcripts), Windows and Linux, no Python needed.

Why I’m posting: it’s still far from the hardware’s limits, on every card. The 140T’s prompt GEMM runs at ~33 % of its XMX peak, while a probe shows ~85 % is reachable. The B70’s prompt GEMMs sit at ~62 % of the int8 peak, and generation at ~62 % of memory bandwidth. Alchemist and most iGPUs are barely tuned, and on Windows multi-GPU sums still go through host memory.

Two ways to help:

  1. Run it on your Arc and post the numbers. One script writes a zip for the issue. Every card and setup we haven’t measured is the most useful thing.
  2. Code: kernels (GEMM/GEMV on XMX and DP4a), multi-GPU, Linux packaging, the web UI. CONTRIBUTING.md has where the speed is missing, how to build, and the rules a change is checked against (KL, bit-identical logits, before/after numbers).

Repo: https://github.com/rizagp/inferzero · Releases: https://github.com/rizagp/inferzero/releases/latest · How to help: https://github.com/rizagp/inferzero/blob/main/CONTRIBUTING.md


r/IntelArc • • 4h ago

Discussion B580 prices are getting worse

19 Upvotes

Just wanted to peruse eBay today. My mornings usually start with a cup(turns into a pot. Deathwish Medium roast) of coffee and me trying to find tech deals. Was curious on the prices of B580s now, cause I bought my LE at $350 maybe 3 months ago. But holy SHEET these things are being sold at an avg high of $469! And that’s the asrocks, sparkles. I love my b580, it’s a tinkerers dream for a card. But $500 for a gpu that usually gets beat out by more reliable and often cheaper cards on the used market is crazy to me.


r/IntelArc • • 6h ago

Discussion Arc Cascade: llama.cpp for Intel Arc that runs MoE models bigger than your card, plus faster Q4_0 prompts

Post image
17 Upvotes

A few people asked if Strata's approach could come to llama.cpp, so it would work with the usual GGUF files and tools. That's what this is.

I run local models on a home server with an Arc Pro B70 and an Arc B580, and the mixture-of-experts models I wanted to use are bigger than either card. Arc Cascade runs them anyway.

It's llama.cpp's SYCL backend with changes for Arc cards. It's still llama.cpp, so the server, the GGUF files and the tools you already use work the same way. A small launcher reads the model file, looks at your cards and free RAM, and decides where everything goes. It reads each card's memory itself, so it should work on any Arc card, though I've only tested a B70 and a B580.

The experts that get used most stay on the card, the next ones sit in pinned RAM, and the rest are read from the model file when they're needed. The answers come out the same as they do with everything on the card.

Here's what I get on my machine (Arc Pro B70 32 GB, Arc B580 12 GB, Linux), in tokens per second while generating. These are with the experts warmed up by an earlier request. The first request is slower: 18, 13 and 18 for the first three.

Qwen3.5-35B-A3B Q4_K_M (22 GB) on the B580 alone, 12 GB RAM budget: 33, or 31.7 on a 4.5K prompt

Qwen3.5-122B-A10B IQ3_XXS (44.7 GB) on the B70 alone, 13 GB RAM budget: 27, or 25.4 on a 4.5K prompt

The same 122B on the B70 and B580 together, 13 GB RAM budget: 35, or 33.1 on a 4.5K prompt

Qwen3.8 Flash-Next Coder IQ1_M with its draft head on the B70: about 100 on code

For the 122B on the B70, stock llama.cpp with its expert offload options does about 12 to 16, and about 1 with the experts on the CPU.

This week I also added a faster path for dense Q4_0 models. Prompts go through 8-bit matrix multiplies on the card's XMX hardware, and drafted tokens get checked in one cheap pass. Qwen3.8-27B Q4_0 on the B70 now reads a 19K prompt at about 1,900 tokens per second, up from 1,350, and writes code at about 97 tokens per second with its draft head. The launcher turns this on by itself when the model fits.

For now it's an experiment. Some of my earlier Arc work has already been merged into llama.cpp and PrismML's fork, and once this has had more testing I'll put it forward for review upstream. That's part of why I'd like to hear how it runs on other people's cards.

I haven't tested it on Windows, other Arc cards, AMD or NVIDIA, models outside the Qwen family, or more than one request at a time. If you've got an Arc card and give it a go, I'd love to hear your numbers, good or bad.

Branch: https://github.com/Torchit1/llama.cpp/tree/arc-cascade

Build steps and testing notes: docs/arc-cascade-testing.md in that branch


r/IntelArc • • 5h ago

Discussion ASRock Challenger Intel Arc B570 is on sale, comes with AIO CPU cooler & games bundle

Thumbnail newegg.com
6 Upvotes

r/IntelArc • • 3h ago

Review 4 Intel ARC B580 cards for local LLM (48GB VRAM)

Thumbnail
youtube.com
12 Upvotes

r/IntelArc • • 10h ago

Question Do you think a B580 would be a good choice for my new pc?

3 Upvotes

I bought a used pc very recently and it's current specs are:

  • CPU: AMD Ryzen 7 5800X (8 Cores / 16 Threads)
  • Motherboard: ASUS ROG Strix B550-F Gaming (Wi-Fi)
  • RAM: 32GB DDR4
  • Storage: Samsung 970 EVO 1TB NVMe SSD
  • GPU: NVIDIA GeForce GTX 1050 Ti (4GB) (This is i'm replacing)
  • PSU: 650W GT EVGA (I'm 90% sure)
  • It's also got a thermaltake liquid cooler

I just bought the arc b580 from newegg for $330. My main questions are do you think it's a good price or could I have found a better budget gpu for my pc? And do you guys recommend I buy a 1080p monitor or 1440p monitor?


r/IntelArc • • 5h ago

Discussion Did anyone used b580 for portal rtx?

4 Upvotes

I downloaded portal rtx and it says that this feature works only for nvidia gpus, BUT i found xess upscaler inside the advanced settings. Is there a way to play portal rtx not using nvidia gpu. Maybe i should try optiscaler?


r/IntelArc • • 4h ago

Build / Photo Thoughts After 2 Years as an Early Adopter

6 Upvotes
Front intake fans were replaced with the Tiyou AX12 - a budget Chinese fan (cost me ~6 USD combined) that received high scores for intake in a Chinese database.

Bought the B580 in December 2024 and have been daily driving it on a 1440p monitor ever since. Have had a few issues with drivers and OneAPI here and there but none of them were particularly major, and the 192-bit bandwidth + 12GB of VRAM have allowed the card to punch above its weight in a lot of situations. Really happy with the purchase and hoping that it will hold up as a gaming/video editing/Blender card while the RTX 3060 takes care of AI workloads, especially now that GPU unlock is a thing.

Specs:

  • CPU: AMD Ryzen 7 9800X3D (Tray)
  • Motherboard: Maxsun iCraft Aiga B850
  • GPU: Maxsun Milestone Intel Arc B580 12GB
  • GPU: GALAX Nvidia GeForce RTX 3060 1-Click OC 12GB
  • RAM: Adata D300 32GB (2×16GB) DDR5-6000 CL30
  • PSU: Sama XM750 Gold 750W
  • Case: Lian Li LANCOOL 216
  • CPU Cooler: Thermalright Phantom Spirit 120 EVO
  • Intake Fans: Tiyou AX12
  • Rear Fan: be quiet! Light Wings 140mm

Notes:

  • Link to the aforementioned Chinese database. The fan changes improved both temperatures and acoustics, so the Tiyou AX12 really lived up to its score for me lol.
  • Swapping the fans on a Lian Li case might seem crazy but mine developed fan hum after a year or two and started becoming unbearable recently. Would definitely recommend prospective Lian Li case buyers to view the stock fans as a stopgap for a couple of years.
  • Did not list my drives in the specs list because I have a bunch of NVMe and SATA drives due to dualbooting Windows 11 and Linux lol.
  • Chinese peripherals are amazing nowadays with how low they are priced for the performance they offer. Other than fans, there are now a bunch of 12-15 USD CPU coolers that are capable of cooling X3D chips while staying reasonably silent, and Chinese PSUs have also improved significantly in recent years (my PSU cost 66-67 USD).
  • Saw some users talk about motherboards that can support dual GPU setups and the Maxsun iCraft Aiga B850 is probably the cheapest offering on the market for this (supports PCIe 5.0 x8 bifurcation and has a four M.2 slots). Maxsun has really upped their game since AM5 was released and I definitely recommend builders to consider their offerings.