CoolFace
Datasetpublic

aiqualitylab/ai-natural-language-tests

NL-to-Test Training Dataset Training data for fine-tuning a code model that generates Cypress and Playwright end-to-end tests from natural-language requirements. Each example is a chat pair: a user message containing a plain-English test requirement and target URL, and an assistant message containing a complete, runnable test file that follows the conventions of the AI Natural Language Tests platform. Playwright examples embed a top-level testData object with a resolveLocator… See the full description on the dataset page: https://huggingface.co/datasets/aiqualitylab/ai-natural-language-tests.

sourceHugging Faceagpl-3.0updated 2mo agoView on Hugging Face
1likes19downloads
Dataset Card

NL-to-Test Training Dataset

Training data for fine-tuning a code model that generates Cypress and Playwright end-to-end tests from natural-language requirements.

Each example is a chat pair: a user message containing a plain-English test requirement and target URL, and an assistant message containing a complete, runnable test file that follows the conventions of the AI Natural Language Tests platform. Playwright examples embed a top-level testData object with a resolveLocator helper and iterate over test_cases; Cypress examples follow the platform's fixture-driven pattern (cy.fixture('url_test_data')).

Dataset Structure

  • —Format: JSONL, one example per line, chat format (messages: [user, assistant]) compatible with trl's SFTTrainer
  • —Fields: requirement, url, messages
  • —Split: train (202 examples: 95 Playwright/TypeScript + 107 Cypress/JavaScript)
Source sitePlaywrightCypress
the-internet.herokuapp.com4149
demoqa.com4448
saucedemo.com1010

Coverage spans login/auth, forms, text inputs, dropdowns, checkboxes/radios, buttons/links, alerts/modals, hover/drag-and-drop, tables, tabs/accordions, dynamic content and waits, iframes/windows, widgets (sliders, date pickers), and multi-step e-commerce flows.

Dataset Creation

Examples were generated by running a categorized requirement bank through the ai-natural-language-tests generation pipeline (gpt-4o-mini backend) against three public demo websites, then filtered with mechanical quality gates (no invalid selectors for the target framework, no hard-coded sleeps, no fake file paths, no markdown fences, valid JSON). Tests were not execution-verified in this version; a future version will add an execution-pass gate.

Limitations

  • —Generated tests follow one platform's conventions and may not reflect general test-writing styles.
  • —Expected values (messages, texts) were produced by an LLM without running the tests, so some assertions may not match the live sites.
  • —All data comes from public demo websites; no personal, sensitive, or user-provided data is included. User inputs to the platform are never collected for training.

License

AGPL-3.0, matching the source project.