AgentPublic/service-public

📢 Sondage 2026 : Utilisation des datasets publiques de MediaTech

Vous utilisez ce dataset ou d’autres datasets de notre collection MediaTech ? Votre avis compte ! Aidez-nous à améliorer nos datasets publiques en répondant à ce sondage rapide (5 min) : 👉 https://grist.numerique.gouv.fr/o/albert/forms/gF4hLaq9VvUog6c5aVDuMw/11 Merci pour votre contribution ! 🙌

🇫🇷 Service-Public.fr practical sheets dataset (Administrative Procedures)

This dataset is derived from the official Service-Public.fr platform and contains practical information sheets and resources targeting both individuals (Particuliers) and entrepreneurs (Entreprendre). The purpose of these sheets is to provide information on administrative procedures relating to a number of themes. The data is publicly available on data.gouv.fr and has been processed and chunked for optimized semantic retrieval and large-scale embedding use.

The dataset provides semantic-ready, structured and chunked data of official content related to employment, labor law and administrative procedures. These chunks have been vectorized using the `BAAI/bge-m3` embedding model to enable semantic search and retrieval tasks.

Each record represents a semantically coherent text fragment (chunk) from an original sheet, enriched with metadata and a precomputed embedding vector suitable for search and retrieval applications (e.g., RAG pipelines).

🗂️ Dataset Contents

The dataset is provided in Parquet format and includes the following columns:

🛠️ Data Processing Methodology

1. 📥 Field Extraction

The following fields were extracted and/or transformed from the original XML files:

• —Basic fields: doc_id, theme, title,surtitre, introduction, url, related_questions, web_services are directly extracted from the XML files, with some processing when needed.
• —Generated fields:
• —chunk_id: is an unique generated and encoded hash for each chunk.
• —chunk_index: is the index of the chunk of a same document. Each document has an unique doc_id.
• —chunk_xxh64: is the xxh64 hash of the chunk_text value. It is useful to determine if the chunk_text value has changed from a version to another.
• —source: is always "service-public" here.
• —Textual fields:
• —context: Optional contextual hierarchy (e.g., nested sections).
• —text: Textual content of the article chunk. This is the value which corresponds to a semantically coherent fragment of textual content extracted from the XML document structure for a same sid.

Column source is a fixed variable here because this dataset was built at the same time as the Travail Emploi Dataset. Both datasets were intended to be grouped together in a single vector collection, they then have differents source values.

2. ✂️ Generation of 'chunk_text'

The value includes the title and introduction of the article, the context values of the chunk and the textual content chunk text. This strategy is designed to improve semantic search for document search use cases on administrative procedures.

The Langchain's RecursiveCharacterTextSplitter function was used to make these chunks (text value). The parameters used are :

• —chunk_size = 1024
• —chunk_overlap = 0
• —length_function = bgem3tokenizer

3. 🧠 Embeddings Generation

Each chunk_text was embedded using the **`BAAI/bge-m3`** model. The resulting embedding vector is stored in the embeddings_bge-m3 column as a string, but can easily be parsed back into a list[float] or NumPy array.

🎓 Tutorials

🔄 1. The chunking doesn't fit your use case?

If you need to reconstitute the original, un-chunked dataset, you can follow this tutorial notebook available on our GitHub repository.

⚠️ The tutorial is only relevant for datasets that were chunked without overlap.

🤖 2. How to load MediaTech's datasets from Hugging Face and use them in a RAG pipeline ?

To learn how to load MediaTech's datasets from Hugging Face and integrate them into a Retrieval-Augmented Generation (RAG) pipeline, check out our step-by-step RAG tutorial available on our GitHub repository !

📌 3. Embedding Use Notice

⚠️ The embeddings_bge-m3 column is stored as a stringified list of floats (e.g., "[-0.03062629,-0.017049594,...]"). To use it as a vector, you need to parse it into a list of floats or NumPy array.

Using the datasets library:

import pandas as pd
import json
from datasets import load_dataset
# The Pyarrow library must be installed in your Python environment for this example. By doing => pip install pyarrow

dataset = load_dataset("AgentPublic/service-public")
df = pd.DataFrame(dataset['train'])
df["embeddings_bge-m3"] = df["embeddings_bge-m3"].apply(json.loads)

Using downloaded local Parquet files:

import pandas as pd
import json
# The Pyarrow library must be installed in your Python environment for this example. By doing => pip install pyarrow

df = pd.read_parquet(path="service-public-latest/") # Assuming that all parquet files are located into this folder
df["embeddings_bge-m3"] = df["embeddings_bge-m3"].apply(json.loads)

You can then use the dataframe as you wish, such as by inserting the data from the dataframe into the vector database of your choice.

🐱 GitHub repository :

The project MediaTech is open source ! You are free to contribute or see the complete code used to build the dataset by checking the GitHub repository

📚 Source & License

🔗 Source :

• —Service-Public.fr official website
• —Data.Gouv.fr : Fiches pratiques et ressources de Service-Public.fr Particuliers
• —Data.Gouv.fr : Fiches pratiques et ressources Entreprendre - Service-Public.fr

📄 Licence :

Open License (Etalab) — This dataset is publicly available and can be reused under the conditions of the Etalab open license.