OleehyO e5b8f9a2ee feat: add caching for prompt embeddings ...

- Add caching for prompt embeddings
- Store cached files using safetensors format
- Add cache directory structure under data_root/cache
- Optimize memory usage by moving tensors to CPU after caching
- Add debug logging for cache hits
- Add info logging for cache writes

The caching system helps reduce redundant computation and memory usage during training by:
1. Caching prompt embeddings based on prompt text hash
2. Caching encoded video latents based on video filename
3. Moving tensors to CPU after caching to free GPU memory

2025-01-04 06:16:31 +00:00

..

__init__.py

feat(datasets): implement video dataset modules

2025-01-01 15:10:40 +00:00

bucket_sampler.py

feat(datasets): implement video dataset modules

2025-01-01 15:10:40 +00:00

i2v_dataset.py

feat: add caching for prompt embeddings

2025-01-04 06:16:31 +00:00

t2v_dataset.py

feat: add caching for prompt embeddings

2025-01-04 06:16:31 +00:00

utils.py

Add type conversion and validation checks

2025-01-01 15:10:42 +00:00