Need mmproj file
Hi, thank you for the model.
This needs to support image inputs via LMstudio user interface. For this the problem description I give you from LLM:
LM Studio's Multimodal Requirements:
LM Studio does support multimodal models (text + image), primarily those based on the LLaVA (Large Language and Vision Assistant) architecture or similar architectures compatible with llama.cpp's multimodal features.
For this to work, you typically need two files:
The main LLM GGUF file (e.g., model-q4_k_m.gguf) which has been specifically compiled with multimodal capabilities.
A separate CLIP Vision Encoder/Projector file (often named something like mmproj-<model_type>-f16.gguf or similar). This file processes the image and translates it into embeddings the LLM can understand.
LM Studio needs both files. It usually expects the mmproj file to be in the same directory as the main GGUF model file. When it detects both compatible files, it displays the little "eye" icon next to the model name, indicating vision capabilities are enabled.
Thanks! I stole bartowski's file and uploaded it here. Let me know if that works for you @notmebug
It works excellently in LM-Studio.
Thank you very much.
ollama fail to work with configuration in ollama version is 0.6.8
error message
Error: Failed to create new sequence: failed to process inputs: this model is missing data required for image input
FROM ~/Workspace/huggingface/gemma-3-27b-it-abliterated-GGUF/gemma-3-27b-it-abliterated.q8_0.gguf
ADAPTER ~/Workspace/huggingface/gemma-3-27b-it-abliterated-GGUF/mmproj-mlabonne_gemma-3-27b-it-abliterated-f16.gguf
TEMPLATE """{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if or (eq .Role "user") (eq .Role "system") }}<start_of_turn>user
{{ .Content }}<end_of_turn>
{{ if $last }}<start_of_turn>model
{{ end }}
{{- else if eq .Role "assistant" }}<start_of_turn>model
{{ .Content }}{{ if not $last }}<end_of_turn>
{{ end }}
{{- end }}
{{- end }}"""
PARAMETER stop <end_of_turn>
PARAMETER temperature 1
PARAMETER top_k 64
PARAMETER top_p 0.95
ollama does not support llama.cpp's gemma 3 vision model, because ollama has their own gguf specs for that (long story short, they have partnership directly with google so they add the support before llama.cpp have it). unfortunately nothing we can do on our side to fix that.