lunabrain-ai/dolly-v2-12b-8bit-example

This repo loads the Dolly V2 12b model (databricks/dolly-v2-12b) using the `transformers` library. The code loads it in 8-bit quantized mode.

5 best points · 0 comments · 1 Hacker News post · 26 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Example POC to run the Dolly V2 12B model in 8bit mode (5 points, 0 comments, by freeqaz, 2023-04-14)