+
+
+
+
+
+
++ Documentation | + Examples | + Blog | + Research +
+ + +Cleanlab’s open-source library helps you **clean** data and **lab**els by automatically detecting issues in a ML dataset. To facilitate **machine learning with messy, real-world data**, this data-centric AI package uses your *existing* models to estimate dataset problems that can be fixed to train even *better* models. + + +
+ 
+
+ Examples of various issues in Cat/Dog dataset automatically detected by cleanlab via this code: +
+ +```python + lab = cleanlab.Datalab(data=dataset, label="column_name_for_labels") + # Fit any ML model, get its feature_embeddings & pred_probs for your data + lab.find_issues(features=feature_embeddings, pred_probs=pred_probs) + lab.report() +``` + +- Use cleanlab to automatically check every: [text](https://docs.cleanlab.ai/stable/tutorials/datalab/text.html), [audio](https://docs.cleanlab.ai/stable/tutorials/datalab/audio.html), [image](https://docs.cleanlab.ai/stable/tutorials/datalab/image.html), or [tabular](https://docs.cleanlab.ai/stable/tutorials/datalab/tabular.html) dataset. +- Use cleanlab to automatically: [detect data issues (outliers, duplicates, label errors, etc)](https://docs.cleanlab.ai/stable/tutorials/datalab/datalab_quickstart.html), [train robust models](https://docs.cleanlab.ai/stable/tutorials/indepth_overview.html), [infer consensus + annotator-quality for multi-annotator data](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html), [suggest data to (re)label next (active learning)](https://github.com/cleanlab/examples/blob/master/active_learning_multiannotator/active_learning.ipynb). + + +--- + + +## Run cleanlab open-source + +This cleanlab package runs on Python 3.10+ and supports Linux, macOS, as well as Windows. + +- Get started [here](https://docs.cleanlab.ai/)! Install via `uv`, `pip`, or `conda`. +- Developers who install the bleeding-edge from source should refer to [this master branch documentation](https://docs.cleanlab.ai/master/index.html). + +**Practicing data-centric AI can look like this:** +1. Train initial ML model on original dataset. +2. Utilize this model to diagnose data issues (via cleanlab methods) and improve the dataset. +3. Train the same model on the improved dataset. +4. Try various modeling techniques to further improve performance. + +Most folks jump from Step 1 → 4, but you may achieve big gains without *any* change to your modeling code by using cleanlab! +Continuously boost performance by iterating Steps 2 → 4 (and try to evaluate with *cleaned* data). + + + + +## Use cleanlab with any model and in most ML tasks + +All features of cleanlab work with **any dataset** and **any model**. Yes, any model: PyTorch, Tensorflow, Keras, JAX, HuggingFace, OpenAI, XGBoost, scikit-learn, etc. + +cleanlab is useful across a wide variety of Machine Learning tasks. Specific tasks this data-centric AI package offers dedicated functionality for include: +1. [Binary and multi-class classification](https://docs.cleanlab.ai/stable/tutorials/indepth_overview.html) +2. [Multi-label classification](https://docs.cleanlab.ai/stable/tutorials/multilabel_classification.html) (e.g. image/document tagging) +3. [Token classification](https://docs.cleanlab.ai/stable/tutorials/token_classification.html) (e.g. entity recognition in text) +4. [Regression](https://docs.cleanlab.ai/stable/tutorials/regression.html) (predicting numerical column in a dataset) +5. [Image segmentation](https://docs.cleanlab.ai/stable/tutorials/segmentation.html) (images with per-pixel annotations) +6. [Object detection](https://docs.cleanlab.ai/stable/tutorials/object_detection.html) (images with bounding box annotations) +7. [Classification with data labeled by multiple annotators](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html) +8. [Active learning with multiple annotators](https://github.com/cleanlab/examples/blob/master/active_learning_multiannotator/active_learning.ipynb) (suggest which data to label or re-label to improve model most) +9. [Outlier detection](https://docs.cleanlab.ai/stable/tutorials/outliers.html) (identify atypical data that appears out of distribution) + +For other ML tasks, cleanlab can still help you improve your dataset if appropriately applied. +See our [Example Notebooks](https://github.com/cleanlab/examples) and [Blog](https://cleanlab.ai/blog/learn/). + + +## So fresh, so cleanlab + +Beyond automatically catching [all sorts of issues](https://docs.cleanlab.ai/stable/cleanlab/datalab/guide/issue_type_description.html) lurking in your data, this data-centric AI package helps you deal with **noisy labels** and train more **robust ML models**. +Here's an example: + +```python + +# cleanlab works with **any classifier**. Yup, you can use PyTorch/TensorFlow/OpenAI/XGBoost/etc. +cl = cleanlab.classification.CleanLearning(sklearn.YourFavoriteClassifier()) + +# cleanlab finds data and label issues in **any dataset**... in ONE line of code! +label_issues = cl.find_label_issues(data, labels) + +# cleanlab trains a robust version of your model that works more reliably with noisy data. +cl.fit(data, labels) + +# cleanlab estimates the predictions you would have gotten if you had trained with *no* label issues. +cl.predict(test_data) + +# A universal data-centric AI tool, cleanlab quantifies class-level issues and overall data quality, for any dataset. +cleanlab.dataset.health_summary(labels, confident_joint=cl.confident_joint) +``` + +cleanlab **clean**s your data's **lab**els via state-of-the-art *confident learning* algorithms, published in this [paper](https://jair.org/index.php/jair/article/view/12125) and [blog](https://l7.curtisnorthcutt.com/confident-learning). See some of the datasets cleaned with cleanlab at [labelerrors.com](https://labelerrors.com). + +cleanlab is: + +1. **backed by theory** -- with [provable guarantees](https://arxiv.org/abs/1911.00068) of exact label noise estimation, even with imperfect models. +2. **fast** -- code is parallelized and scalable. +4. **easy to use** -- one line of code to find mislabeled data, bad annotators, outliers, or train noise-robust models. +6. **general** -- works with **[any dataset](https://labelerrors.com/)** (text, image, tabular, audio,...) + **any model** (PyTorch, OpenAI, XGBoost,...) ++Examples of incorrect given labels in various image datasets found and corrected using cleanlab. +While these examples are from image datasets, this also works for text, audio, tabular data. +
+ + +## Citation and related publications + +cleanlab is based on peer-reviewed research. Here are relevant papers to cite if you use this package: + +