syntropicsignal-ai/wildchat-asking-en-gemini-embedding-001

WildChat Asking-mode (EN) — gemini-embedding-001

English-language first-turn user prompts from `allenai/WildChat-1M`, filtered to Asking-mode prompts and embedded with Google's gemini-embedding-001 at 1536 dimensions.

The text row set is identical to the `text-embedding-3-small` release; only the embedding space differs.

What's in here

Schema

Loading

from datasets import load_dataset

ds = load_dataset("syntropicsignal-ai/wildchat-asking-en-gemini-embedding-001", split="train")
print(ds[0]["text"])
print(len(ds[0]["embedding"]))  # 1536

To stack the embeddings into a numpy matrix for ANN search:

import numpy as np
matrix = np.asarray(ds["embedding"], dtype=np.float32)  # (N, 1536)

How it was built

1. 1.Source. Streamed allenai/WildChat-1M and kept first-turn user prompts where the upstream language field was English. Deduplicated by lowercased text, length-bounded (≤ 1000 characters per prompt).
2. 2.Asking-mode filter. Applied a conservative prefix-based classifier that rejects Doing-mode (content-generation) prompts — prompts whose first content word is a canonical generative imperative (write, generate, create, summarize, translate, help me write, etc.). A leading greeting is stripped first, so "Hi, write me…" is still rejected. The taxonomy follows Chatterji et al., "How People Use ChatGPT" (NBER w34255, 2025) — Asking vs. Doing vs. Expressing. The filter is conservative: false negatives (a few Doing-mode prompts slipping through) are tolerated; false positives (rejecting genuine Asking-mode prompts) are minimized.
3. 3.Embedding. Each surviving prompt embedded once with gemini-embedding-001 at output dimensionality 1536. Gemini does not normalize sub-3072 outputs, so each vector is L2-normalized after embedding.

Intended use

Retrieval, similarity search, classification, and prompt-research workloads where pre-embedded English user prompts are useful. The Asking-mode filter makes this corpus particularly suited to representing what users ask assistants — as opposed to what users ask assistants to generate.

Limitations

• —Heuristic filter. The Asking-mode classifier is a hand-built prefix list. It will admit some Doing-mode prompts whose imperative is non-canonical, and will reject a small number of Asking-mode prompts that happen to start with a generative-sounding word.
• —Single embedding model. Embeddings are tied to Google's gemini-embedding-001 at 1536 dimensions. To use a different embedding space at query time you must re-embed the prompts (the text column is preserved in full for this purpose).
• —Domain skew. WildChat-1M reflects ChatGPT usage in 2023–2024; topical distribution and stylistic conventions may differ from later user behavior.
• —No PII guarantees beyond upstream. This dataset inherits whatever PII handling the WildChat-1M release applied; review their dataset card before publishing or sharing derivatives.

License

This dataset is released under ODC-BY 1.0, matching the upstream WildChat-1M license. Attribution to AllenAI is required when using this dataset.

Citation

@inproceedings{zhao2024wildchat,
  title     = {{WildChat}: 1M ChatGPT Interaction Logs in the Wild},
  author    = {Zhao, Wenting and Ren, Xiang and Hessel, Jack and Cardie, Claire and Choi, Yejin and Deng, Yuntian},
  booktitle = {ICLR},
  year      = {2024}
}

@techreport{chatterji2025chatgpt,
  title       = {How People Use {ChatGPT}},
  author      = {Chatterji, Aaron and Cunningham, Tom and Deming, David J. and Hitzig, Zo{\"e} and Ong, Christopher and Shan, Carl Yan and Wadman, Kevin},
  number      = {w34255},
  institution = {National Bureau of Economic Research},
  year        = {2025}
}