Test User
|
2d3f2a4d02
|
feat: add production-grade batch inference pipeline
Implements a comprehensive batch video generation tool that addresses the #1
missing feature for production users: generating multiple videos from a single
batch file instead of one-at-a-time processing.
## New Files
### tools/batch_inference.py
Production-ready batch inference script with:
**Core Features:**
- JSONL input format (one job per line, streaming-friendly)
- Support for all generation types: t2v, i2v, v2v
- Progress tracking with tqdm (progress bar, ETA)
- Robust error handling (logs errors, continues batch)
- Resume capability (tracks completed jobs, skips on restart)
**Input Schema:**
- prompt (required): Text description
- output_name (required): Output filename
- image_path (optional): For i2v generation
- video_path (optional): For v2v generation
- num_frames, guidance_scale, num_inference_steps, seed, width, height (optional)
**Multi-GPU Support:**
- Job-level parallelism via --gpu_id and --num_gpus flags
- Each GPU processes a subset of jobs (round-robin distribution)
- State file prevents duplicate work across processes
**Memory Management:**
- Loads model once, generates sequentially
- CPU offloading enabled by default
- VAE slicing and tiling enabled
### resources/example_batch_*.jsonl
Example batch files demonstrating:
- example_batch_t2v.jsonl: Text-to-video prompts
- example_batch_i2v.jsonl: Image-to-video with image_path
- example_batch_v2v.jsonl: Video-to-video with video_path
## Design Decisions
1. **JSONL over JSON**: Better for large batches, streaming, and manual editing
2. **Reuse generation logic**: Mirrors cli_demo.py patterns for consistency
3. **Single model per batch**: Memory efficient, simpler implementation
4. **State persistence**: JSON state file enables reliable resume
5. **Error isolation**: One failed job doesn't stop the batch
## Usage Examples
# Basic text-to-video
python tools/batch_inference.py --batch_file prompts.jsonl --model_path THUDM/CogVideoX1.5-5B
# Multi-GPU (4 GPUs)
for i in {0..3}; do
CUDA_VISIBLE_DEVICES=$i python tools/batch_inference.py --batch_file batch.jsonl --gpu_id $i --num_gpus 4 &
done
|
2026-02-19 03:15:10 +00:00 |
|
Yuxuan Zhang
|
39c6562dc8
|
format
|
2025-03-22 15:14:06 +08:00 |
|
Yuxuan Zhang
|
aa12ed37f5
|
Merge branch 'main' into moviepy-v2
|
2025-01-20 21:46:07 +08:00 |
|
Yuxuan Zhang
|
4615479b51
|
move to tools
|
2025-01-14 11:33:02 +08:00 |
|
Erfan Asgari
|
70ca65300c
|
upgrade to moviepy v2
|
2025-01-11 00:18:24 +03:30 |
|
zR
|
17996f11f8
|
update
|
2024-11-16 10:06:22 +08:00 |
|
Yuxuan.Zhang
|
ddd3dcd7eb
|
Merge pull request #469 from THUDM/CogVideoX_dev
CogVideoX1.5-SAT
|
2024-11-08 13:50:19 +08:00 |
|
Yuxuan.Zhang
|
d1e45fbb86
|
Merge pull request #468 from THUDM/main
merge
|
2024-11-07 23:59:21 +08:00 |
|
DefTruth
|
1e6d1bbb82
|
fix parallel rand device
|
2024-11-07 02:40:13 +00:00 |
|
DefTruth
|
bb69713fbb
|
Update parallel_inference_xdit.py
|
2024-11-05 20:21:06 +08:00 |
|
zR
|
3a9af5bdd9
|
update with test code
|
2024-11-04 14:34:36 +08:00 |
|
glide-the
|
24b2053596
|
Add new command line arguments for LoRA weights and prompt
|
2024-10-12 12:28:52 +08:00 |
|
Xibo Sun
|
3304ea7275
|
fix the example scripts in parallel_inference_xdit.py
|
2024-09-26 14:12:26 +08:00 |
|
Yuxuan.Zhang
|
dcb6795b20
|
Merge pull request #334 from chenxwh/main
Add Replicate demo and API
|
2024-09-26 12:52:50 +08:00 |
|
Xibo Sun
|
acc7eac759
|
feat: add xDiT support fro parallel inference
|
2024-09-26 11:21:32 +08:00 |
|
Chenxi
|
a2359dade4
|
Rename predict_t2v.py to tools/replicate/predict_t2v.py
|
2024-09-25 12:31:11 +01:00 |
|
Chenxi
|
e027ebd69f
|
Rename predict_i2v.py to tools/replicate/predict_i2v.py
|
2024-09-25 12:30:56 +01:00 |
|
Chenxi
|
11e0cce3c6
|
Rename cog.yaml to tools/replicate/cog.yaml
|
2024-09-25 12:30:10 +01:00 |
|
huangshiyu
|
5bdc3f65f1
|
update readme for video caption
|
2024-09-19 13:48:05 +08:00 |
|
zR
|
ee9c0b38ab
|
release cogvlm-llama3-caption
|
2024-09-19 00:41:14 +08:00 |
|
zR
|
db309f3242
|
update llm_cogvideox_flux demo test
|
2024-09-17 23:15:19 +08:00 |
|
zR
|
b410841bcf
|
llm-flux-cogvideox-i2v-tools
|
2024-09-16 23:01:37 +08:00 |
|
Yuxuan.Zhang
|
36de705978
|
Merge pull request #268 from glide-the/export_sat_lora_to_hf
从SAT权重文件 导出的lora权重
|
2024-09-14 23:38:44 +08:00 |
|
zR
|
300fc75c49
|
diffusers converter update
|
2024-09-14 15:35:29 +08:00 |
|
glide-the
|
b2b772a942
|
支持了从SAT权重文件 导出的lora权重，脚本在CogVideoX仓库 tools/export_sat_lora_weight.py
加载后使用 load_cogvideox_lora.py 推理
|
2024-09-11 16:24:31 +08:00 |
|
zR
|
f4273a7391
|
readme Update
|
2024-09-10 07:15:57 +08:00 |
|
zR
|
203f56c580
|
Update convert_weight_sat2hf.py
|
2024-09-08 09:27:21 +08:00 |
|
zR
|
4578645267
|
suggestion parm
|
2024-08-20 21:19:45 +08:00 |
|
zR
|
17e6ed8685
|
upload venhancer
|
2024-08-20 15:43:49 +08:00 |
|
Ikko Eltociear Ashimine
|
69930d8463
|
docs: add Japanese README (#76)
* docs: add Japanese README
I created Japanese translated README.
* docs: update README_ja.md
* docs: update README_ja.md
* docs: update sat/README_ja.md
|
2024-08-09 19:44:57 +08:00 |
|
huangshiyu
|
a58692e3a7
|
update caption readme
|
2024-08-06 11:21:46 +08:00 |
|
zR
|
34dd61430e
|
init
|
2024-08-06 02:12:51 +08:00 |
|