aurele1/semantic-job-search-dataset

Semantic Job Search Dataset (Synthetic)

Overview

This dataset contains 10,000 synthetic job postings designed for semantic search. Each record represents a job listing with structured attributes (e.g., field, location, job type) plus a short natural-language description.

The dataset was generated as part of a course final project and is used to support a Gradio app that performs semantic job search using Sentence-Transformer embeddings and cosine similarity.

Dataset Size

• —Rows: 10,000
• —Columns: 9

Data Schema

Each row includes the following fields:

• —title (string): Job title
• —field (string): Domain/category (e.g., data, marketing, software)
• —location (string): Work location format (e.g., On-site / Hybrid / Remote + city)
• —job_type (string): Employment type (e.g., internship, full-time, contract)
• —company_type (string): Organization type (e.g., startup, corporate, agency)
• —required_skills (string): Comma-separated list of skills
• —tags (string): Comma-separated tags/keywords
• —description (string): Short natural-language job description
• —skill_count (int): Number of skills extracted from required_skills

Part 2: EDA Summary (Key Findings)

1) Basic Statistics

• —The dataset is balanced and fully structured with consistent formatting across records.
• —A derived feature (skill_count) was added to quantify how many skills each job requires.

2) Skill Count Distribution

skill_count (number of skills per job) summary:

• —Mean: 3.79
• —Median: 4
• —Min / Max: 0 / 10
• —Std: 2.21

3) Description Length (in words)

The job descriptions are short and consistent in length:

• —Mean: 87.58
• —Std: 1.46
• —Min / Max: 80 / 93

This consistency helps benchmarking semantic search models because it reduces variation caused by extremely short/long texts.

The following plots summarize the main checks we performed

jobs by field:

job description length:

job type distribution within each field:

Part 3 (Embeddings): The dataset was embedded using multiple Hugging Face Sentence-Transformer models (MiniLM, MPNet, and BGE). Embeddings were saved as .npy files and L2-normalized to support fast cosine-similarity retrieval via a dot product. Based on a lightweight retrieval evaluation and practical considerations (embedding size and efficiency), MiniLM was selected as the final model for deployment.

Part 4 (Input → Pipeline → Output): I implemented an end-to-end semantic retrieval pipeline: a user query is converted into an embedding, cosine similarity is computed against the precomputed job embeddings, and the system returns the Top-K most similar job postings along with their key metadata.

Part 5 (Application): A Gradio app was deployed on Hugging Face Spaces to demonstrate the semantic job search system. The app loads the dataset directly from this Hugging Face Dataset repository, loads the selected embeddings file from the Space, and provides three “Quick Starter” queries for one-click demos.

Intended Use

This dataset is useful for:

• —Semantic search / retrieval experiments
• —Embedding model comparisons (quality vs. size vs. speed)
• —Clustering, similarity analysis, and lightweight NLP pipelines
• —Building demo applications (e.g., Gradio + Hugging Face Spaces)

Notes on Synthetic Data

All entries are synthetically generated using a pretrained language model and are intended for educational and benchmarking purposes (not real job postings).

Project Files (Recommended)

To fully document the project, the dataset repo should also include:

• —EDA.ipynb (Exploratory Data Analysis notebook)
• —Synthetic_Data_Generation.ipynb (Data generation notebook)