Cannot load in LM Studio
2025-04-26 14:40:28 [DEBUG]
clip_ctx: CLIP using CUDA0 backend
2025-04-26 14:40:28 [DEBUG]
clip_model_loader: model name: Gemma-3-27B-It
clip_model_loader: description:
clip_model_loader: GGUF version: 3
clip_model_loader: alignment: 32
clip_model_loader: n_tensors: 439
clip_model_loader: n_kv: 23
2025-04-26 14:40:28 [DEBUG]
load_hparams: text_encoder: 0
load_hparams: vision_encoder: 1
load_hparams: llava_projector: 0
load_hparams: minicpmv_projector: 0
load_hparams: minicpmv_version: 2
load_hparams: glm_projector: 0
load_hparams: model size: 134197084.58 MiB
load_hparams: metadata size: 0.15 MiB
2025-04-26 14:40:31 [DEBUG]
llama.cpp abort:2743: GGML_ASSERT(ggml_can_mul_mat(a, b)) failed
I also saw someone else having the issue. Does it load in llama.cpp fine? Might be an LM Studio specific issue
I'vent tried with llama.cpp but at Lm Studio I've tried with different llama.cpp version older and newer along with CUDA12 but same result.
Ok. Tried with llama.cpp and there it works just fine. I think it might be the LM Studio.
Same error where. " Exit code: null). Please check settings and try loading the model again. "
Using last LMstudio Version, Linux Mint, Mini PC minisforum ryzen 7940HS 64 GB 56000 DDR5 CL38, last drivers and updates for everything. Also tried only with CPU, no joy. Apps open at the time of the test, Brave browser with 2 abs open, and Torrent manager Transmission. gemma-3-12b-it-qat-UD-Q4_K_XL.gguf, downloaded one hour ago.
According to @dinhngtu :It works with upstream llama.cpp b5193. (edit: still not working with LM Studio b5173)
so it seems like we have to wait for LM Studio to update
I am using the beta CUDA runtime of LM Studio b5219, and it still does not load for me. I tried both the UD IQ4_XS version and the non UD version. I get the following error message:
🥲 Failed to load the model
Error loading model.
(Exit code: 18446744072635812000). Unknown error. Try a different model and/or config.
So I tested the QAT and this one doesnt work still...
I will ask the LM Studio team what happened
I tried in Jan AI, llama.cpp and Ollama it works fine.
I have no idea what the issue is. Does anyone know?
Someone on the LMS discord said that deleting the mmproj GGUF let it load. So something is broken with vision support.
Someone on the LMS discord said that deleting the mmproj GGUF let it load. So something is broken with vision support.
But that's the vision component
After the update about half a day ago to the latest beta version of the LM Studio CUDA 12 runtime (b5283), both the UD and non UD versions of the IQ4_XS quant load properly. The non UD version seems to be working fine, but the UD version seems to be completely borked in vision. When asked to describe or transcribe an image, it either states there is no image, or hallucinates a completely different image. This behavior is exactly the same as if I did not even provide an image in the first place.