CoolFace
Apppublic

Tonic/l-android-control

sourceHugging Faceupdated 1y agoView on Hugging Face
1likes
imports.cpython-311.pyc139 linesDownload Raw Back to __pycache__
12�#�h�(�
��ddlZddlmZddlZddlmZmZddlm	Z				dde3ezde4de5dzde6dzd	e7dzd8dfd�Z			dde9ezde10de11dzde12dzd	e13dzd14dfd
�ZdS)�N)�Path)�deploy�utils)�
SQLiteStorage�csv_path�project�name�space_id�15dataset_id�returnc��tj|��rtd|�d����t|��}|���std|�����t
j|��}|jrtd���tj16|j�����}|�
|���}d}|jD]}|���dkr|}n�|�td���|�|j}g}	g}17g}g}|jD]Y}
|
|kr�	|
d	kr�	t
j||
d18���|�|
���C#tt$f$rY�VwxYw|���D]�\}}i}|D]=}
||
}t)t
j|����rt-|��||
<�>|r�|	�|��|19�t/||����d	|jvrPt)t
j|d	����r)|�t1|d	������|�d����|	rtj|||	|20|�
��t5dt7|	���d|�d|�d|�d�	��t5dd�|	d���������tj||��\}}|�!|t>j d<t5d|����|�tj!|��dStEj#||��tEj$|��tEj%||��t5dtDj&�'|�������dS)a121    Imports a CSV file into a Trackio project. The CSV file must contain a "step" column, may optionally22    contain a "timestamp" column, and any other columns will be treated as metrics. It should also include23    a header row with the column names.24 25    TODO: call init() and return a Run object so that the user can continue to log metrics to it.26 27    Args:28        csv_path: The str or Path to the CSV file to import.29        project: The name of the project to import the CSV file into. Must not be an existing project.30        name: The name of the Run to import the CSV file into. If not provided, a default name will be generated.31        name: The name of the run (if not provided, a default name will be generated).32        space_id: If provided, the project will be logged to a Hugging Face Space instead of a local directory. Should be a complete Space name like "username/reponame" or "orgname/reponame", or just "reponame" in which case the Space will be created in the currently-logged-in Hugging Face user's namespace. If the Space does not exist, it will be created. If the Space already exists, the project will be logged to it.33        dataset_id: If provided, a persistent Hugging Face Dataset will be created and the metrics will be synced to it every 5 minutes. Should be a complete Dataset name like "username/datasetname" or "orgname/datasetname", or just "datasetname" in which case the Dataset will be created in the currently-logged-in Hugging Face user's namespace. If the Dataset does not exist, it will be created. If the Dataset already exists, the project will be appended to it. If not provided, the metrics will be logged to a local SQLite database, unless a `space_id` is provided, in which case a Dataset will be automatically created with the same name as the Space but with the "_dataset" suffix.34    �	Project 'z:' already exists. Cannot import CSV into existing project.zCSV file not found: zCSV file is empty)�columnsN�stepz/CSV file must contain a 'step' or 'Step' column�	timestamp�raise)�errors��r�run�metrics_list�steps�35timestamps�* Imported z rows from z into project '�36' as run '�'z* Metrics found: �, r�TRACKIO_DATASET_ID�:* Trackio metrics will be synced to Hugging Face Dataset: �* View dashboard by going to: �r37)(r�get_runs�38ValueErrorr�exists�FileNotFoundError�pd�read_csv�emptyr�simplify_column_namesr�tolist�rename�lower�stem�39to_numeric�append�	TypeError�iterrows�bool�notna�float�int�str�bulk_log�print�len�join�keys� preprocess_space_and_dataset_ids�os�environ�print_dashboard_instructionsr�create_space_if_not_exists�wait_until_space_exists�upload_db_to_space�	SPACE_URL�format)rrr	r40r�df�column_mapping�step_column�colrrr�numeric_columns�column�_�row�metrics�values                  �J/opt/conda/envs/codecanvas/lib/python3.11/site-packages/trackio/imports.py�41import_csvrP42s5��,��g�&�&�43��[��[�[�[�44�45�	46��H�~�~�H��?�?���C�� A�x� A� A�B�B�B�	��X�	�	�B�	�x�.��,�-�-�-��0���1B�1B�1D�1D�E�E�N�	���>��	*�	*�B��K��z�����9�9�;�;�&� � ��K��E�!����J�K�K�K��|��}���L��E��J��O��*�47�48���[� � ���[� � ��	��M�"�V�*�W�5�5�5�5��"�"�6�*�*�*�*���I�&�	�	�	��H�	�����+�+�-�-�&�&���3���%�	/�	/�F���K�E��B�H�U�O�O�$�$�
/�"'��,�,������	&�����(�(�(��L�L��S��-�.�.�/�/�/��b�j�(�(�T�"�(�3�{�;K�2L�2L�-M�-M�(��!�!�#�c�+�&6�"7�"7�8�8�8�8��!�!�"�%�%�%���49�����%��!�	50�	51�	52�	53�54�g�c�,�'�'�g�g�H�g�g�W�g�g�`d�g�g�g����5556A�d�i�i��Q��(<�(<�(>�(>�?�?�57A�58A�B�B�B� �A�(�J�W�W��H�j���+5��59�'�(�
�W�:�W�W�X�X�X���
�*�7�3�3�3�3�3��)�(�J�?�?�?��&�x�0�0�0��!�'�8�4�4�4�
�Y�V�-=�-D�-D�h�-D�-W�-W�Y�Y�	60�	61�	62�	63�	64s�*1E�E0�/E0�log_dirc65���	ddlm}n#t$rtd���wxYwtj|��rtd|�d����t
|��}|���std|�����|t|��dh���}|j66}|jrtd	|�����d}	g}67|�d��D�]\}}	|d68krd}
n|}
|r|�d|
��}
|jrtd
|�d����6g}g}g}|���D]�\}}t|d��}t|d��}t!|d��}||i}|�|��|�|��d|jvrPt't)j|d����s)|�t|d������|�d69����|r�tj|t|
��|||���|	t/|��z
}	|70�|
��tdt/|���d|�d|
�d���tdd�t3|d�����������#t4$r }td|�d|����Yd}~��d}~wwxYw|71std���td|	����tdd�|72������t7j||��\}}|�!|t:jd <td!|����|�t7j|��dStAj!||��tAj"|��tAj#||��td"t@j$�%|�#������dS)$aG73    Imports TensorFlow Events files from a directory into a Trackio project.74    Each subdirectory in the log directory will be imported as a separate run.75 76    Args:77        log_dir: The str or Path to the directory containing TensorFlow Events files.78        project: The name of the project to import the TensorFlow Events files into. Must not be an existing project.79        name: The name prefix for runs (if not provided, will use directory names). Each subdirectory will create a separate run.80        space_id: If provided, the project will be logged to a Hugging Face Space instead of a local directory. Should be a complete Space name like username/reponame" ororgname/reponame", or just "reponame" in which case the Space will be created in the currently-logged-in Hugging Face user's namespace. If the Space does not exist, it will be created. If the Space already exists, the project will be logged to it.81        dataset_id: If provided, a persistent Hugging Face Dataset will be created and the metrics will be synced to it every 5 minutes. Should be a complete Dataset name likeusername/datasetname" or "orgname/datasetname", or just "datasetname" in which case the Dataset will be created in the currently-logged-in Hugging Face user's namespace. If the Dataset does not exist, it will be created. If the Dataset already exists, the project will be appended to it. If not provided, the metrics will be logged to a local SQLite database, unless a `space_id` is provided, in which case a Dataset will be automatically created with the same name as the Space but with the_dataset suffix.82    r)�
SummaryReaderz�The `tbparse` package is not installed but is required for `import_tf_events`. Please install trackio with the `tensorboard` extra: `pip install trackio[tensorboard]`.rz@' already exists. Cannot import TF events into existing project.zTF events directory not found: �dir_name)�
extra_columnsz#No TensorFlow events data found in r�mainrKz* Skipping directory z: no scalar data found�tagrNr�	wall_timerrz scalar events from directory 'rrz* Metrics in this run: rz* Error processing directory z: Nz1No valid TensorFlow events data could be importedz* Total imported events: z* Created runs: rrr r!)&�tbparserS�ImportErrorrr"r#rr$r%r6�scalarsr(�groupbyr8r1r4r5r/rr2r&�isnar7r9r:�set�	Exceptionrr<r=r>r?rr@rArBrCrD)rQrr	r83rrS�path�readerrE�total_imported�
imported_runsrT�group_df�run_namerrrrKrLrWrNrrM�es                        rO�import_tf_eventsrg{s���$84�)�)�)�)�)�)�)���85�86�87��
v�88�89�	90�91����92��g�&�&�93��a��a�a�a�94�95�	96���=�=�D��;�;�=�=�J�� H�$� H� H�I�I�I��]�3�t�9�9�Z�L�
A�
A�
A�F�	��B�	�x�G��E�t�E�E�F�F�F��N��M�!�j�j��4�4�8�8���(�7	��2�~�~�!���#���
0�"�/�/�X�/�/���~�
��N�h�N�N�N�O�O�O���L��E��J�"�+�+�-�-�
*�
*���3��#�e�*�o�o���c�'�l�+�+���3�v�;�'�'����,���#�#�G�,�,�,����T�"�"�"��(�"2�2�2�4��G�C��,�-�-�<�<�2��%�%�c�#�k�*:�&;�&;�<�<�<�<��%�%�b�)�)�)�)��
S��&�#��H�
�
�!-��)������#�l�"3�"3�3���$�$�X�.�.�.��s�#�l�"3�"3�s�s�T\�s�s�hp�s�s�s�����Q��	�	�#�h�u�o�:N�:N�0O�0O�Q�Q�R�R�R����	�	�	��A�(�A�A�a�A�A�B�B�B��H�H�H�H�����	�����N��L�M�M�M�	�976�n�986�996�7�7�7�	�1007�T�Y�Y�}�5�5�1017�1027�8�8�8� �A�(�J�W�W��H�j���+5��103�'�(�
�W�:�W�W�X�X�X���
�*�7�3�3�3�3�3��)�(�J�?�?�?��&�x�0�0�0��!�'�8�4�4�4�
�Y�V�-=�-D�-D�h�-D�-W�-W�Y�Y�	104�	105�	106�	107�	108s'�	�#�.J6�F(J6�6109K �K�K )NNN)
r=�pathlibr�pandasr&�trackiorr�trackio.sqlite_storagerr6rPrg��rO�<module>rnsQ��	�	�	�	�����������!�!�!�!�!�!�!�!�0�0�0�0�0�0���!�n110�n111��D�j�n112�
�n113���*�n114��D�j�	n115116�d�117�n118�119�
n120�n121�n122�n123�h��!�z124�z125�
�4�Z�z126�
�z127���*�z128��D�j�	z129130�d�131�z132�133�
z134�z135�z136�z137�z138�z139rm