CoolFace
Apppublic

seai2526-uniba-TheClouds/Code-Comment-Classification-Api

sourceHugging Facemitupdated 9mo agoView on Hugging Face
1likes
sync_models.cpython-311.pyc83 linesDownload Raw Back to __pycache__
1�

2ub5iY����dZddlZddlZddlZddlmZddlZddlmZej	e3��ZdZdefd�Z
ded	efd4�Z		dd
eezdeddfd�Ze5dkr"ejej���e��dSdS)zTSynchronise champion MLflow models from the remote registry to the local filesystem.�N)�Path)�MlflowClient)�python�java�pharo�returnc�r�tjd��}|rtj|��t	��S)aVReturn an MLflow client configured from environment variables.6 7    If ``MLFLOW_TRACKING_URI`` is defined, it is passed to8    :func:`mlflow.set_tracking_uri`. Authentication (for example on DagsHub)9    is handled by MLflow itself via the standard environment variables10    ``MLFLOW_TRACKING_USERNAME`` and ``MLFLOW_TRACKING_PASSWORD``.11    �MLFLOW_TRACKING_URI)�os�getenv�mlflow�set_tracking_urir)�tracking_uris �,/home/kali/SwEn/TheClouds/api/sync_models.py�_get_mlflow_clientrs7���9�2�3�3�L��.����-�-�-��>�>���client�langc��|�d�}|�d�}|���D]�}|j}|�|��s�	|�||���}t�d|||j��|cS#t$rt�d|��Y��wxYwt�d|��dS)a�Return the champion model version for the given language, if any.12 13    The function searches all registered models and looks for models whose name14    starts with ``"<lang>-"`` (for example ``"python-transformer"``). For each15    matching model it tries to resolve the alias ``"<lang>-champion"`` using16    :meth:`MlflowClient.get_model_version_by_alias`.17 18    Args:19        client: Initialised MLflow client.20        lang: Language identifier, such as ``"python"``, ``"java"`` or21            ``"pharo"``.22 23    Returns:24        The matching :class:`mlflow.entities.model_registry.ModelVersion` if a25        champion is found, otherwise ``None``.26    z	-champion�-)�name�aliasz,Found champion model for %s: %s (version %s)z0Alias not defined for model %s, trying next one.zNo champion model found for %s.N)	�search_registered_modelsr�27startswith�get_model_version_by_alias�logger�info�version�	Exception�warning)rr�28alias_name�prefix�rm�29model_name�mvs       r�#_find_champion_version_for_languager&s��(�#�#�#�J��Z�Z�Z�F��-�-�/�/�����W�30��$�$�V�,�,�	��	��2�2�� �3���B�
�K�K�>����31�	
�
�
��I�I�I���	�	�	��K�K�J�J�W�W�W��H�	�����N�N�4�d�;�;�;��4s�:A<�<%B$�#B$�models�api�models_root�32api_subdirc	���t��}t|��}|r||z}|�dd���t�d|�����tD�]�}t||��}|��|j}	|�	dd��\}}n+#t$rt�d|��Y�_wxYw||krt�d|||��||z|z}	|	�
��rtj|	��|	�dd���t�d	||j|	�����	tt"j�|jt+|	���33����}34|35dz}|���r[|���D],}tjt+|��|	|jz���-|36|	krtj|37�����#t2$r-}
t�d||j|
��Yd}
~
���d}
~
wwxYwdS)
aDownload champion models from MLflow and write them to disk.38 39    For each language in :data:`LANGUAGES`, this function looks up the model40    version with alias ``"<lang>-champion"`` and downloads its artifacts. After41    download, the directory structure is normalised so that the final layout is:42 43    .. code-block:: text44 45        models/46          <api_subdir>/47            python/48              <model_type>/49                ...50            java/51              <model_type>/52                ...53            pharo/54              <model_type>/55                ...56 57    For transformer models logged via ``mlflow.transformers``, the inner58    ``model/`` directory is flattened so that the Hugging Face files59    (``config.json``, ``model.safetensors``, ``tokenizer.json``, and so on)60    live directly under ``<model_type>/``.61 62    Args:63        models_root: Base directory under which models are written. Can be a64            string or :class:`pathlib.Path`. Defaults to ``"models"``.65        api_subdir: Optional subdirectory appended under ``models_root`` (for66            example ``"api"``). If empty, models are stored directly under67            ``models_root``.68 69    Raises:70        OSError: If creating directories, moving files, or removing directories71            fails at the OS level.72    T)�parents�exist_okzSyncing best models to: %sNr�z Unexpected model name format: %sz3Language mismatch for model %s: expected %s, got %sz*Downloading model '%s' version %s to %s...)�artifact_uri�dst_path�modelz4Failed to download/reshape model '%s' version %s: %s)rr�mkdirrr�resolve�	LANGUAGESr&r�split�73ValueError�errorr �exists�shutil�rmtreerr
�	artifacts�download_artifacts�source�str�is_dir�iterdir�mover)r)r*r�rootrr%r$�lang_from_name�74model_type�dest_dir�downloaded_path�model_subdir�item�es              r�sync_best_models_to_diskrJNs���P �
!�
!�F�����D��!��j� ���J�J�t�d�J�+�+�+�75�K�K�,�d�l�l�n�n�=�=�=��;�;��
0���
>�
>��
�:���W�76�	�)3�)9�)9�#�q�)A�)A�&�N�J�J���	�	�	��L�L�;�Z�H�H�H��H�	�����T�!�!��N�N�E����	
�
�
��$�;��+���?�?���	$��M�(�#�#�#����t�d��3�3�3����8���J������		77�	78�	79�	�"�� �3�3�!#�� ��]�]�4�����O�+�W�4�L��"�"�$�$�
3�(�0�0�2�2�A�A�D��K��D�	�	�8�d�i�+?�@�@�@�@�#�h�.�.��M�/�2�2�2����	�	�	��L�L�F���80��	
�
�
�
�
�
�
�
�����	����k;�;s+�B(�(%C�C�2B4H(�(81I�2"I�I�__main__)�level)r'r()�__doc__�loggingrr9�pathlibrr
�mlflow.trackingr�	getLogger�__name__rr4rr>r&rJ�basicConfig�INFO�rr�<module>rVs4��Z�Z�����	�	�	�	�
�
�
�
�������
�
�
�
�(�(�(�(�(�(�	��	�8�	$�	$��'�	��L�����.��.�82
�.�.�.�.�d'��k�k��t��k��k�83�k�k�k�k�\�z����G��g�l�+�+�+�+���������r