r/IntelArcPro • • Aug 31 '26

Arc Pro B70 Let's Talk about SR-IOV and Proxmox on Intel Arc B60 and B70!

Thumbnail
youtu.be
7 Upvotes

r/IntelArcPro • • 4d ago

Arc Pro B60 Arc Pro B60 vs RTX 3090: raw llama.cpp performance

Thumbnail
gallery
11 Upvotes

 

Some tests I ran earlier this year comparing the Arc Pro B60 with the RTX 3090 under llama.cpp. Different cards in terms of power and performance, but with same vram and within a "similar" new vs used price range.

Both cards ran Qwen3.6-27B Q4_K_M llamabench through llama.cpp’s Vulkan backend on Windows using build b8994 and FA off.

TestArc Pro B60RTX 3090
Prompt processing — PP2048423.991,192.91
Generation — TG25620.0041.11
16K context — prefill / generation286.79 / 17.78— / —
48K context — prefill / generationFAIL / FAIL458.43 / 24.97

For baseline generation, the rough takeaway was half the speed at around half the power on the B60 (HWiNFO64 showed a TBP of about 170w) so the B60 offers a similar performance per watt.

Noise was another noticeable difference. The B60 was far more bearable on the desk, while this particular 3090 its a Boeing at V1.

I put the runs, hardware comparison and fan noise samples in a video on YouTube.

I kept Flash Attention off because it gives much better results on the B60 in this setup. Its 48K context depth run failed with FA off; it needed FA enabled to complete, but performance hit was huge.

A follow-up video comparing current llama.cpp build with OpenVINO Model Server (OVMS) at baseline and 48k context depth running Qwen3.8-27b on the B60 is coming soon.


r/IntelArcPro • • 5d ago

Arc Pro B60 I was leaving performance off the table. Here is a new Dockerfile for Ollama + SYCL support, now with NEO CACHE and performance improvements built in.

5 Upvotes

I fixed it up a bit!

ARG OLLAMA_VERSION=0.40.16

ARG COMPUTE_RUNTIME_VERSION=26.40.40196.8

ARG LEVEL_ZERO_VERSION=1.34.0

ARG IGC_VERSION=2.42.6

ARG IGC_BUILD=23070

ARG GMM_VERSION=22.10.2

# =============================================================================

# Stage 1: Build ggml-sycl backend from Ollama source on Ubuntu 26.04 (oneAPI)

# =============================================================================

FROM intel/oneapi-toolkit:latest AS sycl-builder

ARG OLLAMA_VERSION

# Clone Ollama at the target version

RUN git clone --depth 1 --branch v${OLLAMA_VERSION} \

https://github.com/ollama/ollama.git /ollama

WORKDIR /ollama

# Build the SYCL backend via the llama/server cmake subproject.

RUN cmake -S llama/server -B build \

-DCMAKE_BUILD_TYPE=Release \

-DCMAKE_C_COMPILER=icx \

-DCMAKE_CXX_COMPILER=icpx \

-DBUILD_SHARED_LIBS=ON \

-DGGML_BACKEND_DL=ON \

-DGGML_NATIVE=OFF \

-DGGML_OPENMP=OFF \

-DGGML_SYCL=ON \

-DGGML_SYCL_TARGET=INTEL \

-DGGML_SYCL_F16=ON \

# -DGGML_SYCL_DEVICE_ARCH=bmg_g21 \

# -DGGML_SYCL_ENABLE_OPT=1 \

-DGGML_SYCL_DNN=ON && \

# -DCMAKE_CXX_FLAGS="-O3 -march=x86-64-v3 -fno-finite-math-only" \

# -DCMAKE_C_FLAGS="-O3 -march=x86-64-v3" && \

cmake --build build --parallel $(nproc) --target ggml-sycl && \

mkdir -p /sycl-runner && \

find /ollama/build -name "libggml-sycl.so" -exec cp {} /sycl-runner/ \;

# Collect oneAPI runtime dependencies into /sycl-runner

RUN . /opt/intel/oneapi/setvars.sh --force >/dev/null 2>&1 || true && \

CMPLR_DIR="${CMPLR_ROOT:-/opt/intel/oneapi/compiler/latest}/lib" && \

DNNL_DIR="${DNNLROOT:-/opt/intel/oneapi/dnnl/latest}/lib" && \

MKL_DIR="${MKLROOT:-/opt/intel/oneapi/mkl/latest}/lib" && \

TBB_DIR="${TBBROOT:-/opt/intel/oneapi/tbb/latest}/lib" && \

cp -L ${CMPLR_DIR}/libsycl.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libur_loader.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libur_adapter_level_zero*.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libsvml.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libimf.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libintlc.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libirng.so* /sycl-runner/ && \

cp -L ${CMPLR_DIR}/libiomp5.so* /sycl-runner/ && \

cp -L ${DNNL_DIR}/libdnnl.so* /sycl-runner/ 2>/dev/null || \

find -L /opt/intel/oneapi -maxdepth 4 -name 'libdnnl.so*' -exec cp -L {} /sycl-runner/ \; && \

cp -L ${MKL_DIR}/libmkl_core.so* /sycl-runner/ && \

cp -L ${MKL_DIR}/libmkl_intel_ilp64.so* /sycl-runner/ && \

cp -L ${MKL_DIR}/libmkl_intel_lp64.so* /sycl-runner/ 2>/dev/null || true && \

cp -L ${MKL_DIR}/libmkl_sycl_blas.so* /sycl-runner/ && \

cp -L ${MKL_DIR}/libmkl_intel_thread.so* /sycl-runner/ && \

cp -L ${MKL_DIR}/libmkl_tbb_thread.so* /sycl-runner/ && \

cp -L ${TBB_DIR}/libtbb.so* /sycl-runner/ 2>/dev/null || \

find -L /opt/intel/oneapi/tbb -name 'libtbb.so*' -exec cp -L {} /sycl-runner/ \; && \

find -L /opt/intel/oneapi -maxdepth 4 -name 'libumf.so*' -exec cp -L {} /sycl-runner/ \; && \

find -L /opt/intel/oneapi -maxdepth 4 -name 'libtcm.so*' -exec cp -L {} /sycl-runner/ \; && \

cp ${CMPLR_DIR}/libsycl-fallback*.spv /sycl-runner/ && \

cp ${CMPLR_DIR}/libsycl-native*.spv /sycl-runner/ && \

strip --strip-unneeded /sycl-runner/*.so* 2>/dev/null; true

# =============================================================================

# =============================================================================

# Stage 2: Runtime image on Ubuntu 26.04

# =============================================================================

FROM ubuntu:26.04

ARG DEBIAN_FRONTEND=noninteractive

ARG OLLAMA_VERSION

ARG COMPUTE_RUNTIME_VERSION

ARG LEVEL_ZERO_VERSION

ARG IGC_VERSION

ARG IGC_BUILD

ARG GMM_VERSION

# Base packages for Ubuntu 26.04

RUN apt-get update && \

apt-get install --no-install-recommends -q -y \

ca-certificates \

wget \

zstd \

ocl-icd-libopencl1 \

libhwloc15 && \

rm -rf /var/lib/apt/lists/*

# Intel GPU runtimes: Level Zero (u26.04 fallback to u24.04), IGC, compute-runtime, GMM

RUN mkdir -p /tmp/gpu && cd /tmp/gpu && \

wget https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze1_${LEVEL_ZERO_VERSION}+u26.04_amd64.deb 2>/dev/null || \

wget https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze1_${LEVEL_ZERO_VERSION}+u24.04_amd64.deb && \

wget https://github.com/intel/intel-graphics-compiler/releases/download/v${IGC_VERSION}/intel-igc-core-2_${IGC_VERSION}+${IGC_BUILD}_amd64.deb && \

wget https://github.com/intel/intel-graphics-compiler/releases/download/v${IGC_VERSION}/intel-igc-opencl-2_${IGC_VERSION}+${IGC_BUILD}_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/intel-ocloc_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/intel-opencl-icd_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/libigdgmm12_${GMM_VERSION}_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/libze-intel-gpu1_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

dpkg -i *.deb && rm -rf /tmp/gpu

# Install official ollama binary + CPU runners

RUN wget -qO- "https://github.com/ollama/ollama/releases/download/v${OLLAMA_VERSION}/ollama-linux-amd64.tar.zst" | \

zstd -d | tar -xf - -C /usr && \

rm -rf /usr/lib/ollama/cuda_* /usr/lib/ollama/vulkan

# Install SYCL runner + oneAPI runtime libs from build stage

COPY --from=sycl-builder /sycl-runner/ /usr/lib/ollama/sycl/

# Build-time dynamic link verification

RUN export LD_LIBRARY_PATH=/usr/lib/ollama/sycl:/usr/lib/ollama:/usr/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH} && \

MISSING=0; \

echo "==> Verifying dynamic library linkages in /usr/lib/ollama/sycl/..."; \

for lib in /usr/lib/ollama/sycl/*.so*; do \

if [ -f "$lib" ] && [ ! -L "$lib" ]; then \

if ldd "$lib" 2>&1 | grep -q "not found"; then \

echo "ERROR: Unresolved dependencies in $lib:"; \

ldd "$lib" | grep "not found"; \

MISSING=1; \

fi; \

fi; \

done; \

if [ "$MISSING" -ne 0 ]; then \

echo "FATAL: One or more shared libraries have missing dependencies. Aborting build."; \

exit 1; \

fi; \

echo "==> All /usr/lib/ollama/sycl dynamic libraries successfully resolved on Ubuntu 26.04."

# Create cache directory

RUN mkdir -p /root/.cache

# Serve ollama on all interfaces & keep models loaded

ENV OLLAMA_HOST=0.0.0.0

ENV OLLAMA_KEEP_ALIVE=24h

ENV OLLAMA_SYCL=1

ENV OLLAMA_VULKAN=false

# --- Intel oneAPI / Level Zero Multi-GPU (2x Arc Pro B60) ---

ENV ONEAPI_DEVICE_SELECTOR=level_zero:0,1

ENV ZES_ENABLE_SYSMAN=1

ENV ZES_ENABLE_SYCL_LDD=1

ENV SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1

ENV UR_L0_USE_IMMEDIATE_COMMANDLISTS=1

ENV ZE_ENABLE_PCI_ID_DEVICE_ORDER=1

# --- Disable Buggy libsycl Disk Cache, Keep In-Memory Cache ---

ENV SYCL_CACHE_PERSISTENT=0 # Bypasses intel/llvm#22853 getSortedImages() SIGSEGV

ENV SYCL_CACHE_IN_MEM=1 # Caches compiled SYCL kernel bundles in RAM while runner is loaded

ENV SYCL_IN_MEM_CACHE_EVICTION_THRESHOLD=0 # Never evict compiled kernels from RAM

# --- Enable Thread-Safe Level Zero / IGC Hardware Shader Cache ---

ENV NEO_CACHE_PERSISTENT=1 # Caches IGC-compiled Xe2 ISA binaries on disk with POSIX locks

ENV NEO_CACHE_DIR=/root/.cache/

ENV NEO_CACHE_MAX_SIZE=4294967296 # 4 GB cache limit (prevents premature eviction of oneDNN kernels)

# Multi-GPU SYCL Tuning (XMX Hardware Acceleration & V2 Graph Support)

ENV GGML_SYCL_F16=1

ENV GGML_SYCL_ENABLE_GRAPH=1 # Doesn't work right now with multi-GPUs, but will some day? Keeping it on hurts nothing.

ENV GGML_SYCL_ENABLE_FUSION=0 # Slows things down.

ENV GGML_SYCL_ENABLE_HOST_PINNED_MEM=0

ENV GGML_SYCL_ENABLE_OPT=1

ENV GGML_SYCL_ENABLE_DNN=1

ENV GGML_SYCL_FA_ONEDNN=1

ENV GGML_SYCL_ENABLE_MKL_FA=1

ENV GGML_SYCL_FORCE_MMQ=0

ENV GGML_SYCL_DEBUG=0

EXPOSE 11434

ENTRYPOINT ["/usr/bin/ollama"]

CMD ["serve"]


r/IntelArcPro • • 5d ago

inferzero update: multi-GPU Arc, Linux port underway, and a call for testers and contributors

Thumbnail
reddit.com
4 Upvotes

r/IntelArcPro • • 6d ago

LazyRig Server v0.41 — Simpler OpenVINO inference on Intel GPUs

Thumbnail
gallery
10 Upvotes

 

Hi everyone! I’ve been working on LazyRig Server, a free, open-source local inference server built around OpenVINO GenAI. I’m developing and testing it on Windows with an Arc Pro B60 and thought it might be useful to others here.

The main thing I’m trying to simplify is getting everything running, including setup, model downloads and conversion:

  • On Windows, extract the release ZIP and run start-server.bat. The first launch prepares the Python environment and dependencies automatically.
  • From the browser interface, you can download models from Hugging Face and automatically convert supported models to OpenVINO.
  • Linux setup scripts and instructions are included in the same ZIP. You’ll need Python 3.12 or 3.13 and the appropriate GPU drivers installed; the setup script prepares the local environment and dependencies.

Once a model is ready, you can use local chat, vision, image generation and editing, or connect other apps through the OpenAI-compatible API subset.

The screenshots show model management, generating a cabin image through chat, and a follow-up request to change it to nighttime.

I’m also preparing v0.42 with MCP support, so you’ll be able to connect compatible external tools to the chat. I’m planning to publish it on GitHub soon.

If that sounds useful, give v0.41 a try. I’d appreciate hearing how setup goes on your hardware, especially on A-series Arc and iGPUs. Let me know if anything is confusing or needs extra manual steps. Feedback from Linux users would be welcome too!

Download v0.41

Setup instructions, documentation and source code


r/IntelArcPro • • 6d ago

Arc Pro B70 Optimizing single B70 setup for Qwen3.8-27B for hermes and coding

Thumbnail
github.com
14 Upvotes

As the Intel battlemage inference support situation seems pretty dynamic here is my optimal Qwen3.8-27b setup tested in the last two days. If you have any tips or insights please feel free to comment.

Spoiler: vLLM XPU is currently the best option


r/IntelArcPro • • 8d ago

I did it. I got my Intel Arc B60 Pro 2x duo to work nicely with a build of Ollama!

7 Upvotes

I recently purchased a second B60 Pro to resolve my lack of super high context with Qwen3.8 27B, and put them together into my Vulkan stack.

Little did I know, but Vulkan seriously gimps inter-card communication when it compares to Sycl, but the SYCL ollama builds I've seen posted on Github fail to build properly.

Here is the Dockerfile I needed in order to get it to function correctly:

ARG OLLAMA_VERSION=0.40.14

ARG COMPUTE_RUNTIME_VERSION=26.40.40196.8

ARG LEVEL_ZERO_VERSION=1.35.0

ARG IGC_VERSION=2.42.6

ARG IGC_BUILD=23070

ARG GMM_VERSION=22.10.2

# =============================================================================

# Stage 1: Build ggml-sycl backend from Ollama source on Ubuntu 26.04 (oneAPI)

# =============================================================================

FROM intel/oneapi-toolkit:latest AS sycl-builder

ARG OLLAMA_VERSION

# Clone Ollama at the target version

RUN git clone --depth 1 --branch v${OLLAMA_VERSION} \

https://github.com/ollama/ollama.git /ollama

WORKDIR /ollama

# Build the SYCL backend via the llama/server cmake subproject.

RUN cmake -S llama/server -B build \

-DCMAKE_BUILD_TYPE=Release \

-DCMAKE_C_COMPILER=icx \

-DCMAKE_CXX_COMPILER=icpx \

-DBUILD_SHARED_LIBS=ON \

-DGGML_BACKEND_DL=ON \

-DGGML_NATIVE=OFF \

-DGGML_OPENMP=OFF \

-DGGML_SYCL=ON \

-DGGML_SYCL_TARGET=INTEL \

-DGGML_SYCL_F16=ON \

-DGGML_SYCL_DNN=OFF \

-DCMAKE_CXX_FLAGS="-O3 -march=x86-64-v3 -fno-finite-math-only" \

-DCMAKE_C_FLAGS="-O3 -march=x86-64-v3" && \

cmake --build build --parallel $(nproc) --target ggml-sycl && \

mkdir -p /sycl-runner && \

find /ollama/build -name "libggml-sycl.so" -exec cp {} /sycl-runner/ \;

# Collect oneAPI runtime dependencies into /sycl-runner

RUN \

cp -P ${CMPLR_ROOT}/lib/libsycl.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libur_loader.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libur_adapter_level_zero*.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libsvml.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libimf.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libintlc.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libirng.so* /sycl-runner/ && \

cp -P ${CMPLR_ROOT}/lib/libiomp5.so* /sycl-runner/ && \

cp -P ${MKLROOT}/lib/libmkl_core.so* /sycl-runner/ && \

cp -P ${MKLROOT}/lib/libmkl_intel_ilp64.so* /sycl-runner/ && \

cp -P ${MKLROOT}/lib/libmkl_intel_lp64.so* /sycl-runner/ 2>/dev/null || true && \

cp -P ${MKLROOT}/lib/libmkl_sycl_blas.so* /sycl-runner/ && \

cp -P ${MKLROOT}/lib/libmkl_intel_thread.so* /sycl-runner/ && \

cp -P ${MKLROOT}/lib/libmkl_tbb_thread.so* /sycl-runner/ && \

cp -P ${TBBROOT}/lib/libtbb.so* /sycl-runner/ && \

cp -P ${UMF_ROOT}/lib/libumf.so* /sycl-runner/ 2>/dev/null || \

find /opt/intel/oneapi -name 'libumf.so*' -exec cp -P {} /sycl-runner/ \; && \

cp -P ${TCMROOT}/lib/libtcm.so* /sycl-runner/ 2>/dev/null || \

find /opt/intel/oneapi -name 'libtcm.so*' -exec cp -P {} /sycl-runner/ \; && \

cp ${CMPLR_ROOT}/lib/libsycl-fallback*.spv /sycl-runner/ && \

cp ${CMPLR_ROOT}/lib/libsycl-native*.spv /sycl-runner/ && \

strip --strip-unneeded /sycl-runner/*.so* 2>/dev/null; true

# =============================================================================

# Stage 2: Runtime image on Ubuntu 26.04

# =============================================================================

FROM ubuntu:26.04

ARG DEBIAN_FRONTEND=noninteractive

ARG OLLAMA_VERSION

ARG COMPUTE_RUNTIME_VERSION

ARG LEVEL_ZERO_VERSION

ARG IGC_VERSION

ARG IGC_BUILD

ARG GMM_VERSION

# Base packages for Ubuntu 26.04

RUN apt-get update && \

apt-get install --no-install-recommends -q -y \

ca-certificates \

wget \

zstd \

ocl-icd-libopencl1 \

libhwloc15 && \

rm -rf /var/lib/apt/lists/*

# Intel GPU runtimes: Level Zero (u26.04 fallback to u24.04), IGC, compute-runtime, GMM

RUN mkdir -p /tmp/gpu && cd /tmp/gpu && \

wget https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze1_${LEVEL_ZERO_VERSION}+u26.04_amd64.deb 2>/dev/null || \

wget https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze1_${LEVEL_ZERO_VERSION}+u24.04_amd64.deb && \

wget https://github.com/intel/intel-graphics-compiler/releases/download/v${IGC_VERSION}/intel-igc-core-2_${IGC_VERSION}+${IGC_BUILD}_amd64.deb && \

wget https://github.com/intel/intel-graphics-compiler/releases/download/v${IGC_VERSION}/intel-igc-opencl-2_${IGC_VERSION}+${IGC_BUILD}_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/intel-ocloc_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/intel-opencl-icd_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/libigdgmm12_${GMM_VERSION}_amd64.deb && \

wget https://github.com/intel/compute-runtime/releases/download/${COMPUTE_RUNTIME_VERSION}/libze-intel-gpu1_${COMPUTE_RUNTIME_VERSION}-0_amd64.deb && \

dpkg -i *.deb && rm -rf /tmp/gpu

# Install official ollama binary + CPU runners

RUN wget -qO- "https://github.com/ollama/ollama/releases/download/v${OLLAMA_VERSION}/ollama-linux-amd64.tar.zst" | \

zstd -d | tar -xf - -C /usr && \

rm -rf /usr/lib/ollama/cuda_* /usr/lib/ollama/vulkan

# Install SYCL runner + oneAPI runtime libs from build stage

COPY --from=sycl-builder /sycl-runner/ /usr/lib/ollama/sycl/

# Build-time dynamic link verification

RUN export LD_LIBRARY_PATH=/usr/lib/ollama/sycl:/usr/lib/ollama:/usr/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH} && \

MISSING=0; \

echo "==> Verifying dynamic library linkages in /usr/lib/ollama/sycl/..."; \

for lib in /usr/lib/ollama/sycl/*.so*; do \

if [ -f "$lib" ] && [ ! -L "$lib" ]; then \

if ldd "$lib" 2>&1 | grep -q "not found"; then \

echo "ERROR: Unresolved dependencies in $lib:"; \

ldd "$lib" | grep "not found"; \

MISSING=1; \

fi; \

fi; \

done; \

if [ "$MISSING" -ne 0 ]; then \

echo "FATAL: One or more shared libraries have missing dependencies. Aborting build."; \

exit 1; \

fi; \

echo "==> All /usr/lib/ollama/sycl dynamic libraries successfully resolved on Ubuntu 26.04."

# Create cache directory

RUN mkdir -p /root/.cache

# Serve ollama on all interfaces & keep models loaded

ENV OLLAMA_HOST=0.0.0.0

ENV OLLAMA_KEEP_ALIVE=24h

ENV OLLAMA_SYCL=1

ENV OLLAMA_VULKAN=false

# Dual-GPU Level Zero Exposure & Layer Distribution

ENV ZES_ENABLE_SYSMAN=1

ENV ZES_ENABLE_SYCL_LDD=1

ENV ONEAPI_DEVICE_SELECTOR="level_zero:0,1"

ENV ZE_ENABLE_PCI_ID_DEVICE_ORDER=1

ENV OLLAMA_SCHED_SPREAD=1

# Multi-GPU SYCL Tuning (XMX Hardware Acceleration & V2 Graph Support)

ENV GGML_SYCL_F16=1

ENV GGML_SYCL_ENABLE_GRAPH=1

ENV GGML_SYCL_ENABLE_FUSION=0

ENV GGML_SYCL_ENABLE_DNN=0

ENV GGML_SYCL_FA_ONEDNN=0

ENV GGML_SYCL_ENABLE_MKL_FA=0

ENV GGML_SYCL_FORCE_MMQ=0

ENV GGML_SYCL_ENABLE_HOST_PINNED_MEM=0

# Level Zero V2 Immediate Submission & Kernel Caching

ENV UR_L0_USE_IMMEDIATE_COMMANDLISTS=1

ENV SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1

ENV SYCL_CACHE_PERSISTENT=1

ENV SYCL_CACHE_DIR=/root/.cache

# Default 48GB VRAM & KV Cache Tuning

ENV OLLAMA_FLASH_ATTENTION=1

ENV OLLAMA_KV_CACHE_TYPE=q8_0

ENV OLLAMA_NUM_PARALLEL=1

ENV OLLAMA_MAX_LOADED_MODELS=1

ENV OLLAMA_GPU_OVERHEAD=536870912

EXPOSE 11434

ENTRYPOINT ["/usr/bin/ollama"]

CMD ["serve"]

Let me know if it functions well for you. I currently see ~80% or slightly more of the theoretical GPU VRAM bandwidth usage, and a prefill of approximately 800/sec on Qwen3.8 27B with context of 128k.


r/IntelArcPro • • 10d ago

B390 local llm

4 Upvotes

I have been trying to run local ai models for agentic coding on my laptop for weeks and Im not sure if Im delusional to think it should work or not.

Here are my specs:

Processor Intel(R) Core(TM) Ultra X7 368H (2.00 GHz)

Installed RAM 64.0 GB

Graphics card Intel(R) Arc(TM) Pro B390 GPU (128 MB)

Latest drivers installed

Here is my setup:

Herdr

Opencode

Lm studio

I tried a bunch of models and config but couldn't get satisfying results

Should I keep trying or no? I thought the 64gb shared memory would let me run any small models easily but it always feels incredibly slow or just does not work at all. I tried asking a bunch of different AIs but they all gaslit me into thinking I could run 32B models because they would fit. Is it just a config/model thing? Is there somewhere I could find known working setups for my hardware?


r/IntelArcPro • • 12d ago

Intel Arc Pro relative Performance per $USD

Post image
6 Upvotes

When compared to other common home and small lab hosted enterprise inference system add in cards, Arc Pro was shown to provide greater performance per dollar spent. Source: https://www.intel.com/content/www/us/en/content-details/922857/bar-chart-comparing-relative-performance-per-dollar-of-intel-arc-pro-series-gpus-versus-market-competitors-showing-a-significant-value-advantage-for-arc-pro.html


r/IntelArcPro • • 16d ago

Arc Pro B60 vLlm with b60

6 Upvotes

Has anyone managed to get vLlm working with b60, and what models were you using? I got llama.cpp with SYCL working but struggling with vllm. Any pointers or documentation appreciated. Thanks.


r/IntelArcPro • • 20d ago

Arc Pro B70 Airflow over the Eiffel Tower - 2.29 Billion grid cells on 2x Intel Arc Pro B70 - FluidX3D CFD

10 Upvotes

At Forum Teratec 2026 in Paris, I showed off a pair of Intel Arc Pro B70 32GB GPUs combining their power for one super large FluidX3D CFD simulation of airflow over the Eiffel Tower. Its intricate steel structure makes for the perfect showcase for super detailed CFD, at 2.29 Billion grid cells.

And if you do the math, you'll notice: That resolution is impossible to fit on 2 Arc Pro B70's with 64GB total VRAM. This was a test of a new FluidX3D prototype I'm working on, with reduced memory footprint to make CFD even cheaper and more accessible amid the memory crisis. More work to do still, but exciting times ahead, with both new software and #Intel D270 hardware on the horizon!

Intel Arc Pro B70 32GB: https://www.intel.com/content/www/us/en/products/sku/245797/intel-arc-pro-b70-graphics/specifications.html

Intel D270 480GB: https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/d-series.html

FluidX3D on GitHub: https://github.com/ProjectPhysX/FluidX3D


r/IntelArcPro • • 29d ago

Arc Pro B70 ASRock Arc Pro B70 worked for weeks, now hangs after model load and fails a VRAM test

3 Upvotes

Wondering if anyone else has run into this with their B70.

**How it started**

The card ran local LLM inference (vLLM XPU, Qwen3.8-27B, Hermes Agent) with no problems for several weeks. Around 10 days ago it crashed while Hermes Agent was in the midst of a task. Since then, no model will start. Right after the weights load, the GuC firmware crashes and the driver marks the GPU as wedged. Only a reboot clears it.

**What I've tested**

- Cold power cycles and different Linux drivers/firmware: no change

- Fresh install of Ubuntu Server 26.04 LTS with the stock xe driver and firmware: same hang

- A simple VRAM pattern test (write, read back, compare): solid 0s and 1s pass, but alternating-bit patterns come back corrupted within the first 1.5 GB (e.g. 0x55555555 reads as 0x05050505), then the GuC crashes and the card wedges

- PCIe Gen 4 x16 and Gen 5 x16: same result

- Default power limit (I had a 230W cap before and removed it)

**System specs**

- Motherboard: Supermicro X13SAE-F, BIOS Date: 06/04/2026 Ver 5.3

- CPU: Intel Core i5-12500

- GPU: ASRock Intel Arc Pro B70 Creator 32GB

- RAM: Kingston 64GB DDR5 ECC, (KSM48E40BD8KI-32HA)

- Storage: Samsung 850 EVO 250GB SSD (TrueNAS boot), 6x WD Red Plus 12TB HDD (RAIDZ2)

- PSU: Corsair RM1200x Shift, 1200W

- OS: TrueNAS SCALE 26 Beta 3; the first crash happened on TrueNAS SCALE 27 RC1. Also tested on Ubuntu Server 26.04 LTS (stock kernel)

**Questions**

- Has anyone else had a B70 run fine and then start failing like this?

- Is there anything else worth trying before it goes back? I have an RMA open with ASRock.


r/IntelArcPro • • Oct 19 '26

Arc Pro B70 Need help with my Dual B70 set up

2 Upvotes

So I'm lucky enough to have gotten a second B70 Pro, yay! However, I have this PC:

https://www.dell.com/support/manuals/en-us/dell-pro-max-fct2250-desktop/dell-pro-max-tower-fct2250-owners-manual/installing-the-system-board?guid=guid-9c53425e-673c-483c-9f8c-18f11482551f&lang=en-us

and the PCIE lanes are- not ideal. How would y'all suggest I go about this? I have 64GB ddr5-5600, lots of NVME storage, but I don't know i need to ultimately move my setup into a z890 system with proper x8/x8. up till now i was just using one B70 and the helper card below, Running a very capable assistant, but just looking for more.

I also have an RTX 2000 ADA in there that is doing some helper work.

I'm open to everyone's ideas!


r/IntelArcPro • • Oct 19 '26

Arc Power 1.2.3: Fan Speed Hibernation Fix, Display Integration & Arc Image Sharpening!

Thumbnail
reddit.com
3 Upvotes

r/IntelArcPro • • Oct 12 '26

Arc Pro B50 [SUCCESS] Intel Arc Pro B50 on an HP EliteDesk 800 G4 SFF: 16GiB ReBAR, PyTorch XPU, 35% Faster TTS — No BIOS Flash!

Thumbnail reddit.com
3 Upvotes

r/IntelArcPro • • Oct 09 '26

Intel Arc Pro B70 on PCIe 4.0 x2: Strata Decode Performance Improves by 1.5–2.9× with Two Environment Variable Overrides

9 Upvotes

I've been experimenting with Strata on my Intel Arc Pro B70 (32 GB VRAM) and found a way to significantly improve decode performance, even with the GPU limited to PCIe 4.0 x2.

For hardware placement reasons, my B70 is installed in the second PCIe slot, so it cannot operate at a wider PCIe link. This makes memory transfers between system RAM and VRAM a potential bottleneck, especially when running large MoE models whose experts cannot all fit in VRAM.

Hardware and model

  • GPU: Intel Arc Pro B70 (32 GB VRAM)
  • PCIe link: PCIe 4.0 x2
  • Inference framework: Strata
  • Model: Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS

The problem with the default configuration

With Strata's default settings, some experts are too large to fit entirely in VRAM. Consequently, some expert data must reside in system RAM and be transferred to VRAM for computation.

Under these conditions, my initial tests showed average decode speeds of only around 10–20 tokens/s. Interestingly, the CPU remained almost completely idle, suggesting that the system was not effectively utilizing CPU resources to help mitigate the memory-transfer bottleneck (Nvidia/AMD use CPU and GPU to decode) .

This was particularly disappointing given that the B70 has 32 GB of VRAM. I wanted to see whether changing Strata's execution configuration could improve performance without changing the hardware or model quantization.

The change: Two environment variable overrides

I modified the following environment variables:

"env": {
  "STRATA_VERIFY_NO_HOST": "0",
  "STRATA_VERIFY_DEVICE_PLAN": "0"
}

After applying these overrides, decode performance improved substantially.

Importantly, I did not change the GPU, PCIe configuration, or model quantization between the two benchmark configurations.

Benchmark results

  • Test A: Both environment variables explicitly set to 0.
  • Test B: Default configuration without these environment variable overrides.
Input tokensPP A (tok/s)PP B (tok/s)TTFT A / BDecode A (tok/s)Decode B (tok/s)Decode improvementDraft acceptance A / B
2021.6321.551.09 / 1.12 s62.8921.39+194%76% / 79%
2,185348.50348.226.48 / 6.50 s59.3237.58+58%66% / 69%
8,000458.86459.3117.64 / 17.65 s65.3935.09+86%81% / 70%
40,000445.98446.2589.90 / 89.89 s57.5635.82+61%69% / 71%
128,000413.01413.13310.13 / 310.11 s52.7934.34+54%67% / 70%

PP = prefill throughput; TTFT = time to first token; Decode = generation throughput; draft acceptance = accepted draft tokens as a percentage.

Key findings

1. Decode throughput improved by approximately 1.5–2.9×.

The improvement was consistent across all five input lengths. At 128K input tokens, decode throughput increased from 34.34 to 52.79 tokens/s, a 54% improvement.

For the shortest test, throughput increased from 21.39 to 62.89 tokens/s, an improvement of 194%.

2. Prefill performance remained virtually unchanged.

Prefill throughput differed by less than 0.2% between the two configurations across all five tests. TTFT was also essentially identical.

This suggests that the performance improvement primarily affects the decode execution path rather than prefill processing.

3. The improvement persists at long context lengths.

Even with 128K input tokens, Test A maintained 52.79 tokens/s compared with 34.34 tokens/s for Test B.

4. Higher draft acceptance does not explain the entire improvement.

Draft acceptance was sometimes lower with Test A, yet decode throughput was consistently higher. This suggests that the speedup cannot simply be attributed to an increase in draft acceptance.

5. No NaN outputs or crashes observed so far.

The Intel-specific notes in intel.md mention NaN outputs and crashes. With these environment variable overrides, I have not encountered those issues during my tests. This is an observation from my own testing, not a claim that the underlying issues have been universally resolved.


r/IntelArcPro • • Oct 05 '26

Qwen3.8 Flash Next GSQ-RCO IQ3_XXS running Strata on 2x Intel B70s

Thumbnail reddit.com
4 Upvotes

r/IntelArcPro • • Sep 29 '26

Mintelica: Mica-v0.1-4B on Intel Arc (B580 + Arc Pro B70)

Thumbnail gallery
3 Upvotes

r/IntelArcPro • • Sep 27 '26

Arc Pro B70 First post here: testing a 4-user local coding-agent setup on one Arc Pro B70

Thumbnail gallery
4 Upvotes

r/IntelArcPro • • Sep 22 '26

PSA for dual Intel Arc Pro B65/B70 users running vLLM: TP=2 was destroying my performance — two independent TP=1 workers are ~3x faster

9 Upvotes

I’ve been building a local AI server around 2x Intel Arc Pro B65 32GB cards, and I finally figured out why I kept feeling like these GPUs were way slower than the hardware should be capable of.

Short version:

Tensor parallelism across the two cards was absolutely killing decode performance.

When I stopped using TP=2 and instead ran one full model independently on each B65 at TP=1, aggregate decode throughput went from about 51 tok/s to 157 tok/s.
Same machine. Same GPUs. Same model. Same vLLM build.

Hardware / software
My current Daedalus machine is roughly:
Ryzen 9 9950X
ASUS ProArt X870E Creator
2x Intel Arc Pro B65 32GB
Linux bare metal
Docker
PyTorch 2.13.0+xpu
vLLM 0.29.0
custom working XPU image: daedalus/vllm-openai-xpu:0.29.0-affinity-fixed
Model: Qwen/Qwen3-30B-A3B-GPTQ-Int4
65,536-token max context
Intel xe kernel driver
GPUs isolated with ZE_AFFINITY_MASK

The Qwen checkpoint is about 15.77 GiB, so it fits comfortably on a single 32GB B65 along with a useful KV cache.

First test: TP=2 vs TP=1

I used the OpenAI-compatible /v1/completions endpoint rather than chat completions so chat-template differences couldn’t affect the result.
Same raw prompt every time, 54 prompt tokens, 512 generated tokens, temperature 0.

2x B65, TP=2:
51.19 tok/s
51.36 tok/s
50.99 tok/s
Average: 51.18 tok/s

Then I isolated one card with ZE_AFFINITY_MASK=0 and changed only:
tensor_parallel_size: 2 → 1
GPU memory utilization from 0.42 → 0.85
Same model, same 65K context, same container/image, same prompt.

B65 #1, TP=1:
77.94 tok/s
78.81 tok/s
78.65 tok/s
Average: 78.47 tok/s

Then the second physical B65:

B65 #2, TP=1:
79.68 tok/s
81.29 tok/s
81.38 tok/s
Average: 80.78 tok/s

So one B65 was roughly 53–58% faster than both B65s running TP=2.

That was the first WTF moment.

Then I ran both cards independently at the same time

Instead of one TP=2 vLLM server, I created two independent vLLM instances:
B65 A → ZE_AFFINITY_MASK=0, port 8000
B65 B → ZE_AFFINITY_MASK=1, port 8001
Both TP=1
Both running their own copy of the same Qwen model

One important wrinkle: each worker needed its own vLLM compile/cache directory.
When both instances shared the same vLLM cache, the second worker loaded the model successfully and then segfaulted while restoring compiled artifacts.
Giving worker B its own cache fixed it.
Then I fired identical 512-token generations at both endpoints simultaneously.

Dual independent worker results:

Run 1:
B65 A: 78.24 tok/s
B65 B: 77.36 tok/s
Aggregate: 154.70 tok/s

Run 2:
B65 A: 79.97 tok/s
B65 B: 79.41 tok/s
Aggregate: 158.81 tok/s

Run 3:
B65 A: 79.37 tok/s
B65 B: 79.26 tok/s
Aggregate: 158.51 tok/s

Average aggregate throughput:
~157.34 tok/s

Compare that to:
TP=2: ~51.18 tok/s

So running two independent TP=1 inference workers gave me roughly:
3.07x the aggregate decode throughput
from the exact same two GPUs.

Even better, simultaneous operation barely affected either card individually. The combined result was about 99% of the theoretical sum of their solo performance.


r/IntelArcPro • • Sep 20 '26

Arc Pro B70 Enhancing llama.cpp to get best performance

1 Upvotes

Most of the online documentation focuses on parameter tweaking and extracting the maximum performance from existing codebases. Very few people actually look at the code itself.

This is exactly the power open source gives us, and we don't make enough use of it.

I am trying to change that:
https://github.com/anantshri/llama.cpp/tree/sycl-b70-optimization-anantshri#submitted-upstream-work
4 pull requests so far, targeting different performance bottlenecks.

All the code changes have been submitted upstream for eventual merging into the main llama.cpp project (and each pull request is linked in the readme). I was too excited to hold back, so I have also created my own branch containing all the optimization work I am doing.

#IntelArcProB70 #LocalAI #PrivateAI #SelfhostedAI #ownthestack


r/IntelArcPro • • Sep 13 '26

Arc Pro B60 4x Arc Pro B60 in action - NASA X-57 in FluidX3D CFD, 1.8 Billion grid cells in 96GB VRAM

18 Upvotes

This was the 2nd demo I had showcased at SC25 in St. Louis! Forgot to upload, here we go :)

This is the NASA X-57 multi-rotor airplane simulated in FluidX3D CFD, on 4x Intel Arc Pro B60 GPUs with 24GB each, at a massive 1.8 Billion grid cells resolution. A good showcase of the mesh center-of-mass utility function for rotation of the 3-/5-bladed rotors. Although grid resolution struggles a bit without refinement (yet ;). The simulation box is split in 4 domains, each computed by one GPU, to pool 96GB VRAM together. At domain boundaries, some data is exchanged between the GPUs over PCIe - B60 have plenty interconnect bandwidth with PCIe 5.0 x8.

  • 1814x2176x454 = 1.8 Billion grid cells
  • 19488 time steps simulated, 4872 frames rendered
  • 7h38min simulation runtime = 47min (compute) + 6h51min (rendering)
  • 160km/h airspeed

FluidX3D CFD software: https://github.com/ProjectPhysX/FluidX3D

NASA X-59 model: https://science.nasa.gov/3d-resources/x-57-maxwell/

Intel Arc Pro B60 specs: https://www.intel.de/content/www/de/de/products/sku/243916/intel-arc-pro-b60-graphics/specifications.html


r/IntelArcPro • • Sep 08 '26

Run Qwen3.6-35B-A3B split across two Intel Arc B580 GPUs or larger

Thumbnail
reddit.com
2 Upvotes

r/IntelArcPro • • Sep 07 '26

Taking a look at Intels Scalar

6 Upvotes

So I have been using llama cpp with sycl for the majority of my inference and it works great from the flexibility and rapid adaptations for things like when dflash came out. However I feel like im in an awkward position atm with qwen 3.8 27B being more of my primary model and qwen 3.6 35B as a speedy inference side model. I have a vibe coded webapp i use internally that made managing llama cpp way easier to deal with and is adapted to VLLM as well (untested). But the problem feels like qwen 3.8 is just leagues ahead of the previous generation in its agentic capability. Im getting roughly 36 t/gs in agent workloads on Q6 with 192K context window and it drops to about 16 t/gs towards the back of the context and prefill at around 350 pp/s. Its quite slow and i cant see myself bothering to switch to qwen 3.6 anymore because it feels incredibly dumb when put side by side with qwen 3.8 in real world use. So I was just curious what the main techniques are and what caviots exist on the scalar VLLM nowadays. I see pre-quantized files in openvino still being made which im curious to try. Also does Dflash 2 get properly utilized? I cant seem to get it to outperform native MTP on llama cpp which was a real bummer. Would love suggestions so I can bash my head against a wall a little less in my testing :)

Edit:

Quick return after a day or two of digging. I am not sold yet on intel scalar and will likely be abandoning this unless I missed something. Seems like an awkward proposition to be using currently. Mainline VLLM is definitely without a doubt vastly better for speed then llama.cpp but this is only the case with custom compiled versions using XPU and XPU graphing mtp patching kernal patches etc. One definite trade made was gaining overall long context efficiency but your trading memory for running XPU graphs which loses some KV/context space. So from my early operational testing im going to be moving on to just forcing mainline vllm to function for the models that really matter and llama cpp for new models and weird quant/distills/fine-tunes. For instance I simply followed Freemoose's post on the intelarc subreddit and with some tweaks im getting solid prefill, 80t/s initially dropping into 40t/s on long context with 160k context. This blows my llama cpp Q6 192k ctx, 36tg/s, and 347pp/s out of the water especially since in hermes use it drops to roughly 15tg/s. There is currently multi request handling issue in the vllm implementation but not particularly a huge deal when running it in this fashion. So credit to his work but if Im understanding how the core mechanism of his guide and sergio's then it makes me really interested to see if there really is missing overhead that can be found on other models like muse glimmer so id like to explore those fields instead.

PS: forgot to mention hardware/software but as mentioned amd 5700x 48gb 3200ddr4 and 1x B70 on fedora 44 server


r/IntelArcPro • • Sep 03 '26

Arc Power 1.1.0 - Overclocking the Arc Way

Thumbnail
reddit.com
1 Upvotes