how much VRAM or how good of a gpu do I need to run this?
#3
by
Trylobot
- opened
with lmstudio it run on a 7800xt 16gb but its slow (5tk/s) i guess it offload on cpu since the cpu go up to 52%
RX 7900 XTX - LLM Studio 0.3.15:
-35.04 t/s.
its ~22 gb so on 16gb card ~7 gb go into ram and it use both cpu/gpu for the calculations...
and it use only one cpu core so the tok/s go down alot 😭