r/LocalLLaMA • u/jacek2023 • 8h ago
Discussion Qwen 4 35B is not ruled out
I think you missed this guys
r/LocalLLaMA • u/Aggressive-Sign-8167 • 5h ago
Hi r/LocalLLaMA 👋
We’re welcoming the Tencent Hy team for an AMA this Friday, November 20, from 8–10 AM PT!
Fresh off launching Hy4 and Hy4 flash, their research team—including Shunyu Yao—will join us to discuss how the models were built, inference speed and cost, and real-world applications, with extra perks for great questions!
⚠️ Note: The AMA itself will be hosted in a separate thread. Please don’t post questions here.
r/LocalLLaMA • u/rm-rf-rm • 2d ago
Its time for another Best Audio Models megathread - its already been 9 months since the last one!
Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.
Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Production use cases with long lengths/stability requirements are relatively rare compared to cherry picked/short clip demos that are plentiful - so real world empirical comparisons are very welcome
Rules
Please use the top level comments to thread your responses.
r/LocalLLaMA • u/jacek2023 • 8h ago
I think you missed this guys
r/LocalLLaMA • u/General-Spite1222 • 2h ago
r/LocalLLaMA • u/giveen • 1h ago
r/LocalLLaMA • u/Terminator857 • 4h ago
Nvidia's commitment to buy ram is near $400 billion. https://x.com/jukan05/status/2122805536118702121 . https://www.reddit.com/r/investing/comments/1xz9eob/nvidia_just_added_111_billion_of_supply/
Change over 3 quarters: $119B, $279B, $390B. One would think that $390B would buy a lot of memory fabrication plants.
r/LocalLLaMA • u/Nunki08 • 8h ago
r/LocalLLaMA • u/mayocream39 • 3h ago
Hi LocalLLaMA,
I have spent more than two months rebuilding Koharu to be the most powerful manga translator!
TLDR: https://github.com/koharu-rs/koharu (MIT or Apache 2.0, at your choice)
It's been a while since I posted the Koharu update here. This time, I have added more features and built an established editor to make it feel more like Photoshop or Clip Studio Paint.
You can type directly onto the page, restyle selected characters, and reshape text frames. There are also layers, history, marquee and lasso selection, Generative Fill, and a Remove brush. The canvas is GPU-rendered now, so it stays smooth even on big chapters.
Everything still runs locally. LLMs run in-process through llama.cpp (Index-Translate 9B by default, plus Gemma 4, Qwen 3.x and LFM3), and inpainting uses LaMa, or FLUX.3 Klein / Qwen-Image through stable-diffusion.cpp. If you already run Ollama or LM Studio, you can point Koharu at that instead. On my machine, a page goes from raw to lettered in about 1 second once the models are loaded.
I also added an agent you can ask to proofread or fix pages. It's exposed over MCP too, so you can drive Koharu from Claude Code or Codex if you'd rather.
It works on CUDA, ROCm, Metal, and Vulkan.
I'd like to hear your feedback or any form of suggestion!
r/LocalLLaMA • u/rkoy1234 • 13h ago
Everyday on /r/LocalLLaMA i see people starting hell about if a comment/post is an llm-genned slop. And I still see people joking about em-dashes or "load-bearing" as some indicator of slop. I haven't seen any recent model say "load-bearing" in months.
modern claudism/gptism isn't just mixing fancy words or em-dashes.
it's the:
The more we know the more we can combat slop in this sub. Please share your updated, 2026 list of slop you despise to see (so that I can also steal and add to my BANNED section on agents.md).
r/LocalLLaMA • u/brown2green • 3h ago
r/LocalLLaMA • u/Fabulous_Pollution10 • 1h ago
Last week we shared LongTail-SWE, 202 real GitHub tasks in 13 less common languages. Many of you asked for DeepSeek V4.1 Flash 1017, so we ran it along with Kimi K3.1 Pro.
DeepSeek V4.1 Flash 1017: 76%, up from 57% for V4 Flash.
Kimi K3.1 Pro: 77%, the top open-weight model, with 5.6M tokens per attempt.
We also re-ran Kimi K3 with a 1M context instead of 262K (now 72%, +8 pts) and fixed tests in a few tasks.
Leaderboard: longtail-swe.com
r/LocalLLaMA • u/GGwithRabbit • 12h ago
AudioGhost randomly got a wave of attention this week (100 stars in a day, no idea where from), so I took that as a sign to finally ship the update I'd been sitting on.
Quick recap if you haven't seen it: you drop in an audio or video file, type the sound you're after ("background music", "dog barking", "people talking"), and it pulls that sound out or removes it. It's a local web app around Meta's SAM-Audio, so everything runs on your own GPU and nothing gets uploaded anywhere.
The use case I hear about most: you film a vlog or an interview outside, a shop nearby is playing music, and YouTube flags the video. Now you can type "background music", mute that track and export the video with everything else still there.
What's new in v1.2:
- Several sounds per run, each on its own track, with mute / solo / volume like a tiny DAW
- Export exactly what you hear (WAV / FLAC / MP3), or the video with the new soundtrack
- The "everything else" track is now generated by the model instead of subtracted, so removing a sound works a lot better
- Mark on the timeline where a sound is (or isn't) to point the model at the right thing
- New UI, and your sessions survive reloads and restarts
If you tried an old version: it's also 4-6x faster than the first release.
Hardware: the small model runs in a bit over 4 GB of VRAM, so most NVIDIA cards are fine. Windows and Linux have install scripts. macOS works too, but it runs on the CPU for now, so it's slow.
One caveat: the official SAM-Audio pipeline also loads several helper models (ImageBind, CLAP, and a judge model that picks the best of several outputs). They add about 25 GB of downloads and need far more VRAM. I stripped those out so it fits on normal GPUs, which means quality is a bit below the full pipeline and results vary from run to run. If a track comes out empty or rough, delete it and run it again.
Demo video and install instructions: https://github.com/0x0funky/audioghost-ai
I'd love feedback, especially from anyone who tries it on messy real-world recordings. Bug reports are welcome on GitHub.
r/LocalLLaMA • u/Disastrous-Work-1632 • 5h ago
Hey there folks!
Aritra here (again) from Hugging Face. We RL-trained a small LoRA adapter (r=32, ~300 MB) on a frozen Qwen3.8-27B, inside the OpenCode coding agent. It's the second of our harness adapters, after pi that we have released yesterday.
Run the model locally like so:
r/LocalLLaMA • u/coder543 • 13h ago
r/LocalLLaMA • u/WorldlyAd7946 • 4h ago
So this started because I bought two of the base Snapdragon X boxes for some software and build testing at work (the cheap ones, about £300 each, not the Elite) and then they were sitting unused. With fast LPDDR5x (8448 MT/s) in them I thought I'd have a play and see if I could run proper models on them. I tried the usual engines and couldn't get anywhere near the speed I thought the hardware should do... So then my ADHD took over and I got carried away, and this is the result 🤣
ToRc is an inference engine written from scratch in Rust. Weeks of days and nights measuring, rewriting and measuring again went into it, and the numbers below are all measured on my own kit, different platforms etc... So I hope this proves useful to others 🙏🙃
If you can unzip a file, you can run your own model on your own machine. The download is a zip of about 13 MB (Windows x64, Windows on ARM, Linux x64 and Linux ARM, plus an experimental Mac one): unzip it, open ToRc, pick a model (a GGUF converts once, and it tells you how much space that takes first), press Auto-tune, and talk. A few minutes from download to chatting, nothing installed, and the app keeps everything (settings, chats, the lot) in that one folder. The only thing outside it is what the engine learns about your machine (Auto-tune's results and the like), in %LOCALAPPDATA%\torc or ~/.cache/torc.
What it can do:
- Uses the CPU, the NPU and the integrated GPU at the same time where it's present or faster, per model. Or just NPU + iGPU if you want to leave the machine usable while it runs.
- Tunes itself to your machine, and every setting can be overridden if you want to fiddle / let an agent try to wring every last token of performance for your particular setup.
- Speaks the OpenAI and Anthropic APIs, so Claude Code, OpenCode and anything else that talks to those just works. Full tool calling and MCP is also built in, and the desktop app has its own chat, tools and MCP servers if you just want to talk to it.
- Runs on Windows and Linux, x86 and ARM. (Linux: one user runs it on bare metal on a Snapdragon box under Ubuntu, and I've run it under WSL2, so tell me how it goes on yours.) There's also an EXPERIMENTAL Mac zip (Apple Silicon): one person has built and run it from source, and GitHub builds the zip, but I don't have a Mac to test it myself.
- It converts to its own file format for efficient SSD streaming where the model won't fit into memory and can split one model across multiple machines. I've tested two boxes over USB4; in principle it's N boxes. You pool their RAM, memory bandwidth and compute for enhanced performance.
Headline numbers, engine timed, read (prompt) / write (reply) tok/s:
- Qwen3.8 Flash Next, a 107 GB model, on two Snapdragon boxes with 64 GB of RAM between them (the experts that don't fit stream off the SSDs): ~67 / ~17.5 on a 16k prompt, ~64 / ~21 on a short one
- Qwen3.6 35B A3B, 4k prompt, two Snapdragon boxes: ~406 / ~54 (one box: ~283 / ~56)
- Same 35B on an i9-13900K (8 threads, CPU only), 4k prompt: ~243 / ~44, where llama.cpp manages ~143 / ~24
Against llama.cpp on the same hardware (CPU against CPU) it's quicker on most of what I've tried, and the gap grows with the prompt: on a 32k prompt it writes 2 to 10x faster. It isn't ahead everywhere, though. On my laptop llama.cpp reads long prompts slightly faster on GLM-4.7-Flash, and writes short replies a little faster on Moonlight and gpt-oss-20b. On the box, it reads short prompts a bit faster on the small dense Qwen3.5 models, until ToRc's NPU is on (then ToRc is ~2x). Full matrices, llama.cpp comparisons and every machine tested are in PERFORMANCE.md (link below).
Tested on: 2x Snapdragon X (base X1, 32 GB) boxes (1 box and 2), a Ryzen 7 7840HS laptop (Radeon 780M + RTX 4070), an i5-12600K desktop (RTX 3060) and an i9-13900K workstation (96 GB, Radeon AI PRO R9700 used for attention only). Everyday Windows installs with all my apps on them, not streamlined benchmark rigs on minimal Linux.
What I'm looking for:
- Testers, especially anyone with an X Elite or X2 Elite. This was all done on the base chip, so I'm curious what performance the bigger chips can manage.
- Mac people. As above, it builds and runs, but I can't test it myself, so numbers (and fixes) from more Macs would be brilliant.
- Linux on more real hardware, x86 especially.
- Holes. Find them and tell me what I can add or improve.
One question for you all: what is one feature you want from a local inference engine that nobody ships yet? I would genuinely like to try building it.
Basically I'm looking for collaboration with other people and projects, I've thoroughly enjoyed this project and I'd really like to see how far it can fly! 🕊️
Discrete GPU expert offload is on the list of my to-dos, but it was never the point of what I was trying to achieve, I use Strata on my big GPU workstation at work, and it's great, but that's a £5k machine, and out of reach of a lot of people. The point was making cheap hardware people are more likely to already have, actually useful - and use these NPUs that we now get with bigger models than the piddly ones we usually see.
Every model and machine I have measured is in PERFORMANCE.md 🙏
r/LocalLLaMA • u/jacek2023 • 7h ago
probably this fork is required: https://github.com/OpenSenseNova/llama.cpp
Model Highlights
r/LocalLLaMA • u/Binary_orchid • 4h ago
The official quants for SenseNova-Loop-9B (the 9B Looped Transformer from earlier this week) are here: https://huggingface.co/sensenova/SenseNova-Loop-9B-GGUF
The repo has Q4_K_M, Q5_K_M, Q8_0 and BF16 (sizes in the screenshot), plus a 0.9 GB mmproj for image input.
The sensenova_loop architecture isn't in mainline llama.cpp yet (issue #31571 is open), so for now these run on the fork they published, OpenSenseNova/llama.cpp. The GGUF card pins the commit, and the card's command once it's built is:
llama serve -hf sensenova/SenseNova-Loop-9B-GGUF:Q4_K_M -ngl 99 -c 4096 --port 8000
There's also a 4-bit AWQ, with a matching MTP draft model in the same repo: https://huggingface.co/sensenova/SenseNova-Loop-9B-AWQ
Their README at https://github.com/OpenSenseNova/SenseNova-Loop puts total GPU memory (4-bit weights plus an 8-bit KV cache) at about 10 GiB for 64K context and 15 GiB for 256K.
r/LocalLLaMA • u/crusaderky • 7h ago
LFM3 benchmarks are out on AA and…. huuuuuuh….. this is NOT something you want to see for a vision model that fits on a phone.
“The mushroom in this picture is a turgida deliciousa, awesome in risottos and on toasted bread. ” (it’s an amanita phalloides, deadly poisonous).
“This sign says that you must not shoot any photos beyond this point” (the sign says “trespassers will be shot”).
etc. etc.
Can anyone with actual experience with this model confirm/deny this score?
r/LocalLLaMA • u/wojtek15 • 6h ago
New version of Splash just released. The biggest improvement from previous version is a noticeable speed-up in decoding. It's even faster now.
BTW, this isn't just another slop engine. It's coming from the researchers behind DFlash and DFlash2, who also operate one of the fastest API providers for open models. These guys clearly know what they're doing.
r/LocalLLaMA • u/WeatherZealousideal5 • 22h ago
Blue TTS v3 is out, an open-source TTS model that runs locally. Almost everything changed since v2.
Speech is more natural and expressive, with a MOS of 4.43 out of 5.
Voice cloning has improved significantly, and the model is fast and lightweight: it runs locally at 50x real-time on a CPU, no GPU needed.
MIT license, 388 MB, 44.1 kHz output, 7 languages: English, German, Spanish, Italian, Russian, Hebrew, Yiddish.
Feedback is welcome.
r/LocalLLaMA • u/pmttyji • 12h ago
r/LocalLLaMA • u/soyuz-1 • 7h ago
I see a lot of lament about Qwen not making 3.8 35B (and I'm sad about it too) but I dont see anyone discuss Flint 40B, a new coding-oriented MoE model that seems to be atleast on par with qwen 3.6 35B, maybe a bit better on agentic tasks and coding. Very fast in my setup.
I'm still testing it myself, but I wonder if its not liked here or people just arent aware it exists? It performs admirably for me so far, my local model of choice for when I want good speeds. QFN is better ofcourse but for me that means low quants or low speeds.
Curious about you guys' experience with the Flint model.
r/LocalLLaMA • u/tebjan • 15h ago
Their architecture seems to translate quite well to existing kernels. They also have a small drafter. The speed numbers are aggregated across 8 parallel agents running. 256k context.
So far, I can't confirm it holds up well against Qwen 3.8 27B nvfp4, but I might have degraded it a bit, despite a calibration run. Here is a quick check of SeraphimSerapis/tool-eval-bench. Will do some more tests tomorrow, the 9B leaves a lot of vram free for FP8 tests.
tool-eval-bench runs (temperature 0, NVFP4 weights):
| Test | Qwen3.8 27B Swift T0 | SenseNova-Loop-9B T0 | SenseNova-Loop-9B T1 |
|---|---|---|---|
| Standard (69) | 98 | 90 | 91 |
| Hard mode (35) | 87 | 79 | 83 |
| Error recovery | 83 | 83 | 83 |
| Context and state | 100 | 75 | 85 |
| Autonomous planning | 67 | 67 | 83 |
| Structured output | 100 | 92 | 75 |
| Safety | 100 | 96 | 92 |
| Hard: state across turns | 0 | 50 | 83 |
| Hard: tool failures | 80 | 60 | 60 |
| Hard: tenant separation | 100 | 100 | 100 |
| Hard: applying policy | 100 | 100 | 100 |
| Hard: injection | 100 | 50 | 50 |
FP8 KV, DFlash2, T0 = temperature 0, T1 = temperature 1.0, top_p 0.95
Hardware: Intel Core 9 290hx, 64gb ram, 1TB nvme5 SSD, rtx 5090 laptop with 24gb vram.