Juliabelloni/weather-forecasting

Monitoring and Forecasting Air Pollution in Utrecht using Machine Learning

This is the repository for air pollution forecasting for the Machine Learning for Industry course.

Using API's, the data has been collected from two governmental websites (RIVM and KNMI). This data was merged into one dataset to do feature engineering. Some additional features were created and the correlation between the final features was inspected, resulting in the removal of highly correlated features. Lagging features and rolling average features were added for more accurate prediction Standardization was applied to the data to ensure fixed ranges of the data.

Several separate models were developed to predict the pollution levels of NO2 and O3.

Prediction system explanation

In the loading_pipeline weather stations that measure the pollutants and the weather can be adjusted. For now, these are set to measure the data for Utrecht. The period for which the model will be trained on can be adjusted too. The train, evaluation, and test set split as default is set to 70%, 20%, 10%.

The preprocessing pipeline uses the merged dataset that was fetched in the loading_pipeline. In the preprocessing pipeline, the months that the model will be tested on are set to be December, April, July, and October. These can be adjusted when needed.

The preprocessing pipeline handles O3 and NO2 data separately, resulting in 6 separate .csv files that contain the train, validation, and test set of NO2 and O3. These files are used to train, validate and test the models.

The models can be run from the notebooks, located in the \notebooks folder. They implement functions that are defined in files in the \models folder and use the preprocessed data that are saved in .csv files in the \data\processed folder. The notebooks containing the models are separated for the target features O3 and NO2.

Instructions:

Here's how you can start using our app!

Local Setup

Both the application and the models have their own virtual environments. To use the model for prediction, simply set up a virtual environment of your choice and download the requirements from requirements.txt. For the application, fisrt change the directory to app, then create a venv for the application and install its dependencies from requirements_app.txt

Docker Setup

1. 1.Clone the repository

git clone https://github.com/Gosia-Kwap/ML4IProject.git

1. 1.Adjust the permission for the websocket In the Dockerfile, replace the hugging face id with localhost:7860 in the following line:

 ENV BOKEH_ALLOW_WS_ORIGIN=juliabelloni-weather-forecasting.hf.space

This will allow the connection to Websocket through localhost.

1. 1.Build the docker image

docker build -t pollution-monitoring-app .

1. 1.Run the docker container

docker run -p 7860:7860 pollution-monitoring-app