seai2526-uniba-TheClouds/Code-Comment-Classification-Api
1
1�
2 �*i� � �T � U d dl Z d dlmZ d dlmZ g d�Zee ed< dd�Zd� Z dS ) � N)�List)�Dataset)�java�python�pharo�LANGSc � � | � t j � dd� � } |�t }t j � | � � st d| � d�� � � ddl}n"# t $ r}t d� � |�d}~ww xY wi }|D ]}dD ]z}|� d |� d3�}t j � | |� � }t j � |� � st d |� �� � �|�4 |� � } | ||� d |� �< �{��|S ) ua Load dataset splits from CSV files under data/raw.5 6 Expects files like data/raw/java_train.csv, data/raw/java_test.csv, etc.7 Returns a dict mapping split names (e.g. "java_test") to pandas DataFrames.8 9 Raises:10 FileNotFoundError: se la directory base o un file atteso non esiste.11 ImportError: se pandas non è installato.12 N�data�rawz CSV datasets not found under z ; cannot load dataset splits.r z)pandas is required to load dataset splits)�train�test�_z.csvz Expected dataset file missing: ) �os�path�joinr �isdir�FileNotFoundError�pandas� Exception� ImportError�isfile�read_csv)13�base_dir�langs�pd�e�datasets�lang�split�fnamer �dfs14 �E/home/kali/SwEn/TheClouds/codecommentclassification/modeling/utils.py�load_dataset_splitsr# s^ � � ���7�<�<���.�.�� �}���
�7�=�=�� "� "� 15� � S�H� S� S� S�16� 17� 18�N������� � N� N� N��E�F�F�A�M�����N���� �H�� -� -��&� -� -�E��)�)�e�)�)�)�E��7�<�<��%�0�0�D��7�>�>�$�'�'�
R�'�(P�$�(P�(P�Q�Q�Q����T�"�"�B�*,�H��&�&�u�&�&� '� '�
-� �Os � A$ �$19B�.A>�>Bc �H � d� }| d � |� � | d< | S )Nc � � t | t � � rf| � � � }|� d� � r |� d� � r20|dd� }d� |� � � D � � S dd l}t | |j � � r d� | � � � D � � S n# t $ r Y nw xY wt | t t f� � r d� | D � � S t d t | � � � d21| ��� � �) N�[�]� �����c �0 � g | ]}|�t |� � ��S � ��int)�.0�toks r" �22<listcomp>z;parse_labels_column.<locals>._parse_one.<locals>.<listcomp>5 s# � �9�9�9��S�9�C��H�H�9�9�9� r c �, � g | ]}t |� � ��S r+ r, �r. �vs r" r0 z;parse_labels_column.<locals>._parse_one.<locals>.<listcomp>9 s � �3�3�3�1��A���3�3�3r1 c �, � g | ]}t |� � ��S r+ r, r3 s r" r0 z;parse_labels_column.<locals>._parse_one.<locals>.<listcomp>= s � �&�&�&�q�C��F�F�&�&�&r1 z Formato labels non gestito: z -> )�23isinstance�str�strip�24startswith�endswithr �numpy�ndarray�tolistr �list�tuple�25ValueError�type)�x�s�nps r" �26_parse_onez'parse_labels_column.<locals>._parse_one0 s$ � � �a�� � � :���� � �A��|�|�C� � �
�Q�Z�Z��_�_�
��a��d�G��9�9���� � �9�9�9� 9� � � � � ��!�R�Z�(�(�
4�3�3����27�28�3�3�3�3�
4��� � � � �D� ���� �a�$�� � '� '� '�&�&�A�&�&�&� &��J��Q���J�J�Q�J�J�K�K�Ks �=6B5 �529C�C�labels)�apply)r! rE s r" �parse_labels_columnrH / s6 � �L� L� L� �h�<�%�%�j�1�1�B�x�L�
�Ir1 )NN)30r �typingr r r r r7 �__annotations__r# rH r+ r1 r" �<module>rK s~ �� � � � � � � � � � � � � � � � � � /�.�.��t�C�y� .� .� .�$� $� $� $�N� � � � r1