CoolFace
Datasetpublic

lukebarousse/data_jobs

šŸ§  data_jobs Dataset A dataset of real-world data analytics job postings from 2023, collected and processed by Luke Barousse. Background I've been collecting data on data job postings since 2022. I've been using a bot to scrape the data from Google, which come from a variety of sources. You can find the full dataset at my app datanerd.tech. Serpapi has kindly supported my work by providing me access to their API. Tell them I sent you and get 20% off paid plans.ā€¦ See the full description on the dataset page: https://huggingface.co/datasets/lukebarousse/data_jobs.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
96likes11kdownloads
Dataset Card

šŸ§  data_jobs Dataset

A dataset of real-world data analytics job postings from 2023, collected and processed by Luke Barousse.

Background

I've been collecting data on data job postings since 2022. I've been using a bot to scrape the data from Google, which come from a variety of sources.

You can find the full dataset at my app datanerd.tech.

Serpapi has kindly supported my work by providing me access to their API. Tell them I sent you and get 20% off paid plans.

šŸ“˜ Data Dictionary

Column NameDescriptionTypeSource
job_title_shortCleaned/standardized job title using BERT model (10-class classification)CalculatedFrom job_title
job_titleFull original job title as scrapedRawScraped
job_locationLocation string shown in job postingRawScraped
job_viaPlatform the job was posted on (e.g., LinkedIn, Jobijoba)RawScraped
job_schedule_typeType of schedule (Full-time, Part-time, Contractor, etc.)RawScraped
job_work_from_homeWhether the job is remote (true/false)BooleanParsed
search_locationLocation used by the bot to generate search queriesGeneratedBot logic
job_posted_dateDate and time when job was postedRawScraped
job_no_degree_mentionWhether the posting explicitly mentions no degree is requiredBooleanParsed
job_health_insuranceWhether the job mentions health insuranceBooleanParsed
job_countryCountry extracted from job locationCalculatedParsed
salary_rateIndicates if salary is annual or hourlyRawScraped
salary_year_avgAverage yearly salary (calculated from salary ranges when available)CalculatedDerived
salary_hour_avgAverage hourly salary (same logic as yearly)CalculatedDerived
company_nameCompany name listed in job postingRawScraped
job_skillsList of relevant skills extracted from job posting using PySparkParsed ListNLP Extracted
job_type_skillsDictionary mapping skill types (e.g., 'cloud', 'libraries') to skill setsParsed DictNLP Extracted