Juliabelloni/weather-forecasting
0
1�
2 "!g� � �Z � d dl mZ d dlZd dlZd dlZd dlZddlmZ G d� de� � Z dS )� )� timedeltaN� )�LoadABCc �N � e Zd Zd3dededededef4d�Zdedededej fd�Z d S ) �
LoadPollution�5 �6station_id�7start_time�end_time� max_retries�returnc �4 � d}|� d|� d|� d|� �}i }i }d} d}8| |k �rU t j |||�� � } | � � � t j |9� � | � � � d } n.# t t f$ r}
t
d |� d10|
� �� � �d }
~
ww xY w| S # t j j $ r t d |� d
�� � �t j j $ r t d|� d�� � �t j j $ r*}t j � d|� d11|� �� � �d }~wt j j
$ r*}t j �
d|� d12|� �� � �d }~ww xY wt j �
d|� d|� d�� � �)a 13 Retrieves measurements from a given station in the given time period.14 15 Args:16 station_id (str): The ID of the station to retrieve the measurements from.17 start_time (str): The start time of the period to retrieve the measurements from.18 end_time (str): The end time of the period to retrieve the measurements from.19 max_retries (int, optional): The maximum number of retries to perform if the request fails. Defaults to 10.20 21 Returns:22 list: The measurements retrieved from the station in the given time period.23 24 Raises:25 TimeoutError: If the request times out after max_retries attempts.26 ConnectionError: If a network problem occurs while trying to retrieve the data after max_retries attempts.27 HTTPError: If an HTTP error occurs while trying to retrieve the data after max_retries attempts.28 RequestException: If any other request exception occurs while trying to retrieve the data after max_retries attempts.29 zhttps://api.luchtmeetnet.nlz&/open_api/measurements?station_number=zE&formula=&page=&order_by=timestamp_measured&order_direction=desc&end=z&start=r � )�headers�datar z:Invalid JSON response or missing 'data' field for station z: NzRequest to station z timed out.z:Network problem while trying to retrieve data for station �.z HTTP error occurred for station z1Error occurred while requesting data for station z$Failed to retrieve data for station z after z30 attempts.)�requests�get�raise_for_status�time�sleep�json�31ValueError�KeyError�32exceptions�Timeout� TimeoutError�ConnectionError� HTTPError�RequestException)�selfr r33 r r � gateway_url�url�payloadr �retries�delay�responser �e�http_err�req_errs �N/Users/juliabelloni/Desktop/ML_industry/ML4IProject/src/data/load_pollution.py�_get_station_infoz LoadPollution._get_station_info s� � �& 4� � � s� s�J� s� s� V^� s� s� gq� s� s����������� �#�#�
H�#�<��W�7�K�K�K�� �)�)�+�+�+��34�5�!�!�!�u�#�=�=�?�?�6�2�D�D��"�H�-� u� u� u�$�%s�bl�%s�%s�pq�%s�%s�t�t�t�����u����� ���&�.�
R�
R�
R�"�#P��#P�#P�#P�Q�Q�Q��&�6�
r�
r�
r�%�&p�cm�&p�&p�&p�q�q�q��&�0�
u�
u�
u� #�-�7�7�8s�[e�8s�8s�iq�8s�8s�t�t�t������&�7�
H�
H�
H� �)�:�:� <G�oy� <G� <G� ~E� <G� <G� H� H� H�����
H���� �!�2�2� 4E�Zd� 4E� 4E�mx� 4E� 4E� 4E� F� F� FsH �?B+ � A; �:B+ �;B&� B!�!B&�&B+ �+A E1� %D0�0E1�%E,�,E1c �^ � g }t j � |� � }t j � |� � }||k r�t |t d�� � z |� � }| � ||� � � |� � � � � }d|� d|� � � � d|� � � � �}| j � |�� � |� |� � |t d�� � z }||k ��t j |� � S )a� 35 Retrieves all data from a given station for a given time period.36 37 This function does so by making multiple requests for 7 day periods, 38 as the API does not allow for larger time ranges.39 40 Parameters41 ----------42 station_id : str43 The id of the station to retrieve data from.44 start_time : str45 The start time of the data to retrieve (inclusive).46 end_time : str47 The end time of the data to retrieve (inclusive).48 49 Returns50 -------51 pd.DataFrame52 A DataFrame containing all the data from the given station for the given time period.53 � )�daysz(Data successfully retrieved for station z for period z to )�info_strr ) �dateutil�parser�parse�minr r, � isoformat�_logs� load_logs�extend�pd�json_normalize) r! r r54 r �all_data�
current_start�next_endr r0 s r+ �get_datazLoadPollution.get_dataI s< � �* �� ��-�-�j�9�9�
��?�(�(��2�2���h�&�&��=�9�!�+<�+<�+<�<�h�G�G�H��)�)�*�m�6M�6M�6O�6O�QY�Qc�Qc�Qe�Qe�f�f�D�W� �W� W�!.�!8�!8�!:�!:�W� W�@H�@R�@R�@T�@T�W� W�H� �J� � �H� � 5� 5� 5�
�O�O�D� !� !� !� $�y�a�'8�'8�'8� 8�M� �h�&�&� � ��*�*�*� N)r )55�__name__�56__module__� __qualname__�str�int�listr, r9 � DataFramer> � r? r+ r r s� � � � � � �;F� ;F�C� ;F�S� ;F�C� ;F�^a� ;F�jn� ;F� ;F� ;F� ;F�|#+�3� #+�C� #+�3� #+�2�<� #+� #+� #+� #+� #+� #+r? r )57�datetimer r1 r r �pandasr9 �load_abcr r rG r? r+ �<module>rK s� �� � � � � � � ���� ���� � � � � � � � � � � � � � �c+� c+� c+� c+� c+�G� c+� c+� c+� c+� c+r?