-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Description
Proposal
Incorporate the ability to load GPT2 models (and others in future) from the GGUF format as defined here:
https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
There is a fairly approachable library written in TypeScript that could be a good basis:
https://github.com/ahoylabs/gguf.js
Benefits
- Allows loading of models from a single file.
- Easy extensibility for new model support such as Llama2 models in the future.
GeorgeS2019
Metadata
Metadata
Assignees
Labels
No labels