AIPython★ 1,860⑂ 378 forksApache-2.0
NVIDIA-NeMo/Curator
CURATOR: LLM DATA PREPARATION
Curator is a toolkit for preprocessing and curating data, accelerating LLM training.
// KEY FEATURES
- Automatic data cleaning and normalization
- Support for multiple formats (CSV, JSON, Parquet)
- Integration with popular LLM frameworks
#data#data-curation#data-prep#data-preparation#data-processing#data-processing-pipelines#data-quality#datacuration#datarecipes#deduplication#fast-data-processing#fine-tuning
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram