datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
quote-repetition
quote-repetition (Joe Cavanagh, Andrew Gritsevskiy, and Derik Kauffman of Cavendish Labs)
General description
In this task, the authors ask language models to repeat back sentences given in the prompt, with few-shot examples to help it recognize the task. Each prompt contains a famous quote with a modified ending to mislead the model into completing the sequence with the famous ending rather than with the ending given in the prompt. The authors find that smaller models… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/quote-repetition.inverse-scaling-ttc-main
Inverse Scaling in Test-Time Compute
Paper: Inverse Scaling in Test-Time Compute
Project Page: https://safety-research.github.io/inverse-scaling-ttc/
Abstract
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between test-time compute and accuracy. Our evaluation tasks span four categories: simple counting tasks with distractors, regression tasks with… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling-ttc/inverse-scaling-ttc-main.NeQA
NeQA: Can Large Language Models Understand Negation in Multi-choice Questions? (Zhengping Zhou and Yuhui Zhang)
General description
This task takes an existing multiple-choice dataset and negates a part of each question to see if language models are sensitive to negation. The authors find that smaller language models display approximately random performance whereas the performance of larger models become significantly worse than random.
Language models failing to follow… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/NeQA.redefine-math
redefine-math (Xudong Shen)
General description
In this task, the author tests whether language models are able to work with common symbols when they are redefined to mean something else. The author finds that larger models are more likely to pick the answer corresponding to the original definition rather than the redefined meaning, relative to smaller models.
This task demonstrates that it is difficult for language models to work with new information given at inference… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/redefine-math.hindsight-neglect-10shot
inverse-scaling/hindsight-neglect-10shot (‘The Floating Droid’)
General description
This task tests whether language models are able to assess whether a bet was worth taking based on its expected value. The author provides few shot examples in which the model predicts whether a bet is worthwhile by correctly answering yes or no when the expected value of the bet is positive (where the model should respond that ‘yes’, taking the bet is the right decision) or negative (‘no’… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/hindsight-neglect-10shot.three-mountain-scaling
ThreeMountain_Scaling
Segment
Meaning
GO
Geometric Object — indicates the object type used (e.g., GO for geometric, RO for real objects).
L / Arc
Object Arrangement — defines how objects are arranged spatially. L means L-shape arrangement; Arc means objects are placed in an arc.
RC
Random Character Position — RC = True: character position is randomized.
FC
Fixed Character Position — FC = True: character stays fixed.
RS
Random Scale — RS = True: objects are… See the full description on the dataset page: https://huggingface.co/datasets/grow-ai-like-a-child/three-mountain-scaling.DensingLaw-ScalingBench
DensingLaw-ScalingBench
This dataset was created to enable a more accurate performance scaling law estimation of Large Language Models (LLMs).
This dataset is released as part of our paper, Densing Law of LLMs.
📜 Paper
💡 Overview
This repository contains the open-source dataset used for calculating conditional loss in our LLM density evaluation framework.
LLM density is defined as the ratio of effective parameter size to actual parameter size, where effective… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/DensingLaw-ScalingBench.inverse-scaling-ttc-main
Inverse Scaling in Test-Time Compute
Note: This is an anonymized repository.
Abstract
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between test-time compute and accuracy. Our evaluation tasks span four categories: simple counting tasks with distractors, regression tasks with spurious features, deduction tasks with constraint tracking, and advanced AI… See the full description on the dataset page: https://huggingface.co/datasets/anonscaling/inverse-scaling-ttc-main.Scalingacorn
license: intel-research
tags:
code
legal
art
pretty_name: 'TubuTerp '
--- Two Charts = [products ] [Scent Patterns]
import pandas as pd
Create a dataframe for products table
data_products = { Blue Dream } 1=S/H , 2=Platinum Hydrolab / Stem Handcrafted
'id': [Thc26.2% ,Burns good , Taste slight citrus profile, but is easy to inhale and smokes very nice ], [very light tea smell with a sour twang that adds that high grade smell ]
'name': ['Product A', 'Product B'… See the full description on the dataset page: https://huggingface.co/datasets/Mi6paulino/Scalingacorn.Inverse-scaling-test