Support H-Net #33
Description
Goal
Support for H-Net (https://github.com/goombalab/hnet)
Design decisions
TBD
Future Extensions
- Solve batching
- (Distilled models from Token-based to Byte-based)
Benefits
- Tokenizer-free modelling
- Adds a foundation for future byte-level models
Definition of Done
- Model train on test dataset
- Generation load model from checkpoint and generate from prompt
- Example small runnable generation example
Why Merge This?
- Immediate value unlocks byte-level training and generation