hyperml/balanced_accuracy

1---2title: Accuracy3emoji: 🤗 4colorFrom: blue5colorTo: red6sdk: gradio7sdk_version: 3.19.18app_file: app.py9pinned: false10tags:11- evaluate12- metric13description: >-14  Balanced Accuracy is the average of recall obtained on each class. It can be computed with:15  Balanced Accuracy = (TPR + TNR) / N16  Where:17  TPR: True positive rate18  TNR: True negative rate19  N: Number of classes20---21 22# Metric Card for Balanced Accuracy23 24## Metric Description25 26Balanced Accuracy is the average of recall obtained on each class. It can be computed with:27Balanced Accuracy = (TPR + TNR) / N28 Where:29TPR: True positive rate30TNR: True negative rate31N: Number of classes32 33## How to Use34 35At minimum, this metric requires predictions and references as inputs.36 37```python38>>> accuracy_metric = evaluate.load("hyperml/balanced_accuracy")39>>> results = accuracy_metric.compute(references=[0, 1], predictions=[0, 1])40>>> print(results)41{'balanced_accuracy': 1.0}42```43 44### Inputs45 46**predictions** (list of int): Predicted labels.47**references** (list of int): Ground truth labels.48**sample_weight** (list of float): Sample weights Defaults to None.49**adjusted** (boolean): If set to True, adjusts the score by accounting for chance. Useful in handling imbalanced datasets. Defaults to False.50 51### Output Values52 53- **balanced_accuracy** (float): Balanced Accuracy score. Minimum possible value is 0. Maximum possible value is 1.0. A higher score means higher balanced accuracy.54 55Output Example(s):56```python57{'balanced_accuracy': 1.0}58```59 60This metric outputs a dictionary, containing the balanced accuracy score.61 62#### Values from Popular Papers63 64Balanced accuracy is often used to report performance on supervised classification tasks such as sentiment analysis or fraud detection, where there is a severe imbalance in the classes.65 66### Examples67 68Example 1-A simple example69```python70>>> balanced_accuracy_metric = evaluate.load("balanced_accuracy")71>>> results = balanced_accuracy_metric.compute(references=[0, 1, 2, 0, 1, 2], predictions=[0, 1, 1, 2, 1, 0])72>>> print(results)73{'balanced_accuracy': 0.5}74```75 76Example 2-The same as Example 1, except with `sample_weight` set.77```python78>>> balanced_accuracy_metric = evaluate.load("balanced_accuracy")79>>> results = balanced_accuracy_metric.compute(references=[0, 1, 2, 0, 1, 2], predictions=[0, 1, 1, 2, 1, 0], sample_weight=[0.5, 2, 0.7, 0.5, 9, 0.4])80>>> print(results)81{'balanced_accuracy': 0.8778625954198473} # TODO: check if this is correct82```83 84Example 3-The same as Example 1, except with `adjusted` set to `True`.85```python86>>> balanced_accuracy_metric = evaluate.load("balanced_accuracy")87>>> results = balanced_accuracy_metric.compute(references=[0, 1, 2, 0, 1, 2], predictions=[0, 1, 1, 2, 1, 0], adjusted=True)88>>> print(results)89{'balanced_accuracy': 0.8} # TODO: check if this is correct90```91 92## Limitations and Bias93 94The balanced accuracy metric has limitations when it comes to extreme cases such as perfectly balanced or highly imbalanced datasets. For example, in perfectly balanced datasets, it behaves the same as standard accuracy. However, in highly imbalanced datasets where a class has very few samples, a small change in the prediction for that class can cause a large change in the balanced accuracy score.95 96## Citation(s)97```bibtex98@article{scikit-learn,99  title={Scikit-learn: Machine Learning in {P}ython},100  author={Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V.101         and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P.102         and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and103         Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.},104  journal={Journal of Machine Learning Research},105  volume={12},106  pages={2825--2830},107  year={2011}108}109```110 111## Further References112