language classification
code-natural-language-classification-datasetSampling from codeparrot/github-code under more permissive license ['mit', 'apache-2.0', 'bsd-3-clause', 'bsd-2-clause', 'cc0-1.0'] + sampling from minipile.
It is intended to be used for training code natural language classifier.
task1577_amazon_reviews_multi_japanese_language_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task1577_amazon_reviews_multi_japanese_language_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task1577_amazon_reviews_multi_japanese_language_classification.Code-Language-Classification
Programming Language Classification Dataset
A large-scale, balanced dataset for programming language identification from source code snippets.
Overview
This dataset contains 1.664 million cleaned and labeled source code samples across 16 programming languages, specifically designed for programming language classification and identification tasks.
Unlike many code datasets that are primarily built for code generation or retrieval, this dataset was curated… See the full description on the dataset page: https://huggingface.co/datasets/kaushik-harsh-99/Code-Language-Classification.indic-voices-indian-language-classification
Indic Language Speech Classification Dataset
A curated subset of the AI4Bharat IndicVoices-R dataset created for
Indian-language speech classification experiments on consumer hardware.
The original IndicVoices-R corpus is extremely large (~750 GB). This
dataset intentionally samples a much smaller subset so that researchers
and developers can download, preprocess, and experiment with the data
locally on machines with limited storage and memory.
Languages
The… See the full description on the dataset page: https://huggingface.co/datasets/iqbaltaha3/indic-voices-indian-language-classification.LanguageClassification
LanguageClassification
An MTEB dataset
Massive Text Embedding Benchmark
A language identification dataset for 20 languages.
Task category
t2c
Domains
Reviews, Web, Non-fiction, Fiction, Government, Written
Referencehttps://huggingface.co/datasets/papluca/language-identification
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["LanguageClassification"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/LanguageClassification.Language-Agency-Classification
Language Agency Classifier Dataset
The Language Agency Classifier Dataset was created by (Wan et al., 2023) and aims to provide data for training models to
classify sentences based on the level of agency expressed in each sentence. This repository is organized by Miri Liu (github: emmatliu).
Classifying sentence agency can help expose latent gender bias, where women may
be described with more communal (community-oriented) words and men may be described with more agentic… See the full description on the dataset page: https://huggingface.co/datasets/elaine1wan/Language-Agency-Classification.