☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 11 days agoGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.mlimagemessage-square58linkfedilinkarrow-up1134arrow-down114
arrow-up1120arrow-down1imageGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.ml☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 11 days agomessage-square58linkfedilink
minus-squarestuner@lemmy.worldlinkfedilinkarrow-up3·10 days agoI run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
minus-squareCameronDev@programming.devlinkfedilinkarrow-up2·10 days agoI’ll give LM studio a go, thanks.
I run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
I’ll give LM studio a go, thanks.