data-selection

DataFlex is a data-centric training framework that enhances model performance by either selecting the most influential samples, optimizing their weights, or adjusting their mixing ratios.

data-science data model-training data-selection data-mixture data-reweighting data-model-interaction

Updated Mar 16, 2026
Python

HKUSTDial / LEAD

Star 109

[VLDB'26] Official repository for the paper "LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning".

data-selection

Updated Jun 3, 2025
Python

georgian-io / Transformers-Domain-Adaptation

Star 85

[DEPRECATED] Adapt Transformer-based language models to new text domains

natural-language-processing deprecated transfer-learning domain-adaptation data-selection huggingface-transformers huggingface-tokenizers

Updated Feb 21, 2024
Jupyter Notebook

gszfwsb / Data-Whisperer

Star 48

Code for ACL 2025 Main paper "Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning".

transformers llama mistral data-selection efficient-training llm data-pruning qwen

Updated Aug 4, 2025
Python

waltonfuture / InstructionGPT-4

Star 42

InstructionGPT-4

data-selection multimodal-deep-learning less-is-more llm llm-training

Updated Dec 29, 2023
Python

zhuang-li / SCAR

Star 39

[ACL 2025 main] SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models

data-filtering data-selection large-language-models instruction-tuning

Updated Aug 6, 2025
Python

yichengchen24 / MIG

Star 28

[ACL2025 Findings] Official code for MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space

data-selection llms instruction-tuning

Updated Aug 30, 2025
Python

yueyu1030 / Patron

Star 24

[ACL 2023] The code for our ACL'23 paper Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach

nlp natural-language-processing cold-start language-model active-learning data-selection fine-tuning few-shot-learning prompt-learning acl2023

Updated Jun 1, 2024
Python

Implementation of TSDS: Data Selection for Task-Specific Model Finetuning. An optimal-transport framework for selecting domain-specific and task-specific training data to improve LLM finetuning and instruction tuning.

nlp machine-learning research deep-learning nearest-neighbor-search transfer-learning optimal-transport domain-adaptation model-training data-selection fine-tuning domain-specific finetuning task-specific data-centric-ai large-language-models llm dataset-curation instruction-tuning

Updated Dec 25, 2024
Python

princeton-pli / STAT

Star 16

Skill-Targeted Adaptive Training

synthetic-data data-selection large-language-models supervised-finetuning

Updated Mar 12, 2026
Python

reds-lab / projektor

Star 14

This is an official repository for "Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sources" (NeurIPS 2023).

projection performance-prediction data-selection scaling-law

Updated Oct 26, 2023
Python

Nokia-Bell-Labs / data-centric-federated-learning

Star 13

Enhancing Efficiency in Multidevice Federated Learning through Data Selection

machine-learning flower wearable-devices edge-computing data-selection wearable-computing federated-learning split-learning data-centric-ai data-centric-machine-learning

Updated Apr 15, 2024
Python

ArsamAryandoust / DataSelectionMaps

Star 12

Enhanced spatio-temporal electric load forecasts with less data using active deep learning

machine-learning deep-learning remote-sensing sampling active-learning data-selection spatio-temporal-prediction load-forecasting

Updated Feb 7, 2023
Jupyter Notebook

JoyeBright / DataSelection-NMT

Star 10

Repository for the experiments in my paper accepted to the CLIN Journal: "Selecting Parallel In-domain Sentences for Neural Machine Translation Using Monolingual Texts"

machine-translation neural-machine-translation data-selection in-domain in-domain-translation