roberta
Datasets
All datasets matching “roberta”roberta-pt-checkpointsroberta_pretrain
Dataset Card for RoBERTa Pretrain
Dataset Summary
This is the concatenation of the datasets used to Pretrain RoBERTa.
The dataset is not shuffled and contains raw text. It is packaged for convenicence.
Essentially is the same as:
from datasets import load_dataset, concatenate_datasets
bookcorpus = load_dataset("bookcorpus", split="train")
openweb = load_dataset("openwebtext", split="train")
cc_news = load_dataset("cc_news", split="train")
cc_news =… See the full description on the dataset page: https://huggingface.co/datasets/gsgoncalves/roberta_pretrain.roberta-dataroberta-tokenized-datawikitext-103-raw-v1_sents_min_len10_max_len30_princeton-nlp_sup-simcse-roberta-largerecipe_RL_data_roberta-base
Dataset Description
Structure
Consists of 5 fields
Each row corresponds to a policy - sequence of actions, given an initial <START> state, and corresponding rewards at each step.
Fields
steps, step_attn_masks, rewards, actions, dones
Field descriptions
steps (List of lists of Ints) - tokenized step tokens of all the steps in the policy sequence (here we use the roberta-base tokenizer, as roberta-base would be used to encode each step of a recipe)… See the full description on the dataset page: https://huggingface.co/datasets/AnonymousSub/recipe_RL_data_roberta-base.