ysn-rfd/TinyMistral-248M-v2.5-GGUF
This model was converted to GGUF format from Locutusque/TinyMistral-248M-v2.5 using llama.cpp via the ggml.ai's all-gguf-same-where space.
Refer to the original model card for more details on the model.
✅ Quantized Models Download List
🔍 Recommended Quantizations
- ✨ General CPU Use:
Q4_K_M(Best balance of speed/quality) - 📱 ARM Devices:
Q4_0(Optimized for ARM CPUs) - 🏆 Maximum Quality:
Q8_0(Near-original quality)
📦 Full Quantization Options
| 🚀 Download | 🔢 Type | 📝 Notes |
|---|---|---|
| Download |  |
Basic quantization |
| Download |  |
Small size |
| Download |  |
Balanced quality |
| Download |  |
Better quality |
| Download |  |
Fast on ARM |
| Download |  |
Fast, recommended |
| Download |  ⭐ |
Best balance |
| Download |  |
Good quality |
| Download |  |
Balanced |
| Download |  |
High quality |
| Download |  🏆 |
Very good quality |
| Download |  ⚡ |
Fast, best quality |
| Download |  |
Maximum accuracy |
💡 Tip: Use F16 for maximum precision when quality is critical
🚀 Applications and Tools for Locally Quantized LLMs
🖥️ Desktop Applications
| Application | Description | Download Link |
|---|---|---|
| Llama.cpp | A fast and efficient inference engine for GGUF models. | GitHub Repository |
| Ollama | A streamlined solution for running LLMs locally. | Website |
| AnythingLLM | An AI-powered knowledge management tool. | GitHub Repository |
| Open WebUI | A user-friendly web interface for running local LLMs. | GitHub Repository |
| GPT4All | A user-friendly desktop application supporting various LLMs, compatible with GGUF models. | GitHub Repository |
| LM Studio | A desktop application designed to run and manage local LLMs, supporting GGUF format. | Website |
| GPT4All Chat | A chat application compatible with GGUF models for local, offline interactions. | GitHub Repository |
📱 Mobile Applications
| Application | Description | Download Link |
|---|---|---|
| ChatterUI | A simple and lightweight LLM app for mobile devices. | GitHub Repository |
| Maid | Mobile Artificial Intelligence Distribution for running AI models on mobile devices. | GitHub Repository |
| PocketPal AI | A mobile AI assistant powered by local models. | GitHub Repository |
| Layla | A flexible platform for running various AI models on mobile devices. | Website |
🎨 Image Generation Applications
| Application | Description | Download Link |
|---|---|---|
| Stable Diffusion | An open-source AI model for generating images from text. | GitHub Repository |
| Stable Diffusion WebUI | A web application providing access to Stable Diffusion models via a browser interface. | GitHub Repository |
| Local Dream | Android Stable Diffusion with Snapdragon NPU acceleration. Also supports CPU inference. | GitHub Repository |
| Stable-Diffusion-Android (SDAI) | An open-source AI art application for Android devices, enabling digital art creation. | GitHub Repository |
- Downloads last month
- 46
Hardware compatibility
Log In
to view the estimation
6-bit
8-bit
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for ysn-rfd/TinyMistral-248M-v2.5-GGUF
Base model
Locutusque/TinyMistral-248M-v2.5Datasets used to train ysn-rfd/TinyMistral-248M-v2.5-GGUF
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard24.570
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard27.490
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard23.150
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard46.720
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard47.830
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard0.000
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard13.360
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard3.180
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard0.000
- acc_norm on GPQA (0-shot)Open LLM Leaderboard0.110