Quantize model

[ztachip]

The repo provides some pre-quantized model ready for download
But how do I quantize it myself. What would be the procedure?

[DanielDaniel2201]

I quantize models using their another repo, llm-awq. There is instruction on how to quantize