declare-lab/tango2

1a

2��'d�K�@s�
dZd
dl
Z
d
dlZd
dlZd
dlZd
dlZd
dlZd
dlZd
dlm	Z	
d
dl3mZ
d
dlm
Z
mZmZ
d
dlmZ
d
dlmZmZmZmZ
dd	lmZ
d4dlmZmZmZ
dZe�e�
Zd
d�Z dd�Z!ee"ej#fd�
dd�Z$dd�Z%dd�Z&dd�Z'dd�Z(dd�Z)d%ee"ej#fe"eee"ej#fe*e*ee
e"e"feee*e"fee"e*d�	d d!�Z+d&ee"ej#fe"ee"eee"ej#fe*e*ee
e"e"feee*e"fee"e*d"�5d#d$�Z,dS)'z3Utilities to dynamically load objects from the Hub.�N)
�
StrictVersion)
�Path)�Dict�Optional�Union)
�request)�HfFolder�cached_download�hf_hub_download�6model_info�)
�__version__�
)�DIFFUSERS_DYNAMIC_MODULE_NAME�HF_MODULES_CACHE�loggingzchttps://raw.githubusercontent.com/huggingface/diffusers/{revision}/examples/community/{pipeline}.pycCs,d
}t�
t�|�
���
d��}
t|
td�S)Nz$https://pypi.org/pypi/diffusers/json�releases)
�key)�json�loadsr�urlopen�read�keys�sortedr)�urlr�r�Z/home/deep/Projects/audio_diffusion/diffusers/src/diffusers/utils/dynamic_modules_utils.py�get_diffusers_versions+s


rc
CsHtt
jvrd
St
j�t�

tjtdd�
tt�
d}|��sD|��
d
S)z_7    Creates the cache directory for modules with an init, and adds it to the Python path.8    NT�
�exist_ok�__init__.py)	r�sys�path�append�os�makedirsr�exists�touch)
�	init_pathrrr�init_hf_modules1s9




r))
�namec
CsPt�
t
t�
|}
|
j��s&t|
j�

tj|
d
d�
|
d}|��sL|��
dS)zF10    Creates a dynamic module in the cache directory for modules.11    Trr N)	r)rr�parentr&�create_dynamic_moduler$r%r')r*Zdynamic_module_pathr(rrrr,@s
12
13



r,c
Csjt|d
dd��}
|
�
�}Wd�
n1s,0


Y
tjd|tjd�}|tjd|tjd�7}tt|�
�
S)z�14    Get the list of modules that are relatively imported in a module file.15 16    Args:17        module_file (`str` or `os.PathLike`): The module file to inspect.18    �
r�utf-8�
�encodingNz^\s*import\s+\.(\S+)\s*$�
�flagsz^\s*from\s+\.(\S+)\s+import)�openr�re�findall�	MULTILINE�list�set)�module_file�
f�contentZrelative_importsrrr�get_relative_importsOs19
&r<c
s�d
}
|g
}g�|
s�g}|D]}|�t
|�
�

qt|�
j�
�
f
dd�|D�
}�f
dd�|D�
}dd�|D�
}t|�
dk}
��|�

q�S)a)
20    Get the list of all files that are needed for a given module. Note that this function recurses through the relative21    imports (if a imports b and b imports c, it will return module files for b and c).22 23    Args:24        module_file (`str` or `os.PathLike`): The module file to inspect.25    Fc
sg|]}
t�|
�
�qSr)
�str)�.0�
m)
�module_pathrr�26<listcomp>t�z-get_relative_import_files.<locals>.<listcomp>c
sg|]}
|
�v
r|
�qSrr�r>r:)
�all_relative_importsrrrAurBc
Ssg|]}
|
�d��qS)
�.pyrrCrrrrAvrBr
)�extendr<rr+�len)r9Z	no_changeZfiles_to_checkZnew_importsr:Znew_import_filesr)rDr@r�get_relative_import_filesas




27



rHc
	Cs�t|d
dd��}
|
�
�}Wd�
n1s,0


Y
tjd|tjd�}|tjd|tjd�7}dd	�|D�
}tt|�
�
}g}|D]0}zt�|�

Wq�t	y�


|�28|�

Yq�0q�t|�
d29kr�t	dd�|�
�d
d�|�
�d��
�
t
|�
S)zi30    Check if the current Python environment contains all the libraries that are imported in a file.31    r-r.r/Nz^\s*import\s+(\S+)\s*$r1z^\s*from\s+(\S+)\s+importc
Ss$g|]}
|
�d�
s|
�
d�
d
�qS)�
.r
)�32startswith�split)r>�imprrrrA�rBz!check_imports.<locals>.<listcomp>r
z\This modeling file requires the following packages that were not found in your environment: �, z. Run `pip install �
 �
`)r3rr4r5r6r7r8�	importlib�
import_module�ImportErrorr#rG�joinr<)�filenamer:r;�importsZmissing_packagesrLrrr�
check_imports~s*
&








�
��rVcCs4|
�t
jjd
�}
t�|
�
}|dur*t|�
St||�S)zY33    Import a module on the cache directory for modules and extract a class from it.34    rIN)�replacer$r"�seprPrQ�find_pipeline_class�getattr)�35class_namer@�modulerrr�get_class_in_module�s36
37

r]c
Cs�d
dlm
}

tt�|tj��
}d}|��D]b\}}||
jkr*t||
�r*|j	�38d�
ddkr*|du
r�td|
j�d|j�d	|�d39|�d�	�
�
|}q*|S)z�40    Retrieve pipeline class that inherits from `DiffusionPipeline`. Note that there has to be exactly one class41    inheriting from `DiffusionPipeline`.42    r)
�DiffusionPipelineNrIr
�	diffusersz#Multiple classes that inherit from z have been found: z, and z). Please make sure to define only one in )�	pipelinesr^�dict�inspect�43getmembers�isclass�items�__name__�44issubclass�45__module__rK�46ValueError)Z
loaded_moduler^Zcls_membersZpipeline_class�cls_name�clsrrrrY�s,
���

47
�
���rYF)	�pretrained_model_name_or_pathr9�	cache_dir�force_download�resume_download�proxies�use_auth_token�revision�local_files_onlyc	Cs�t|�
}t
j�||
�}	t
j�|	�
r.|	}48d
}�
nn|�d�
dk�
r6t�}dd�t�d�
dd��
}
|dur�|
|vrt|
nd}t	�49d	|�d��

nD||vr�d|��}n0|dkr�|}n"td50|�dd�|dg
�
�d��
�
tj
||d
�}z&t||||||dd�}51d}|d}
Wn.t�
y2


t	�d|
�d|�d��

�Yn0nfz6t||
||||||d�}52t
j�d
d�|�d�
�
�}Wn.t�
y�


t	�d|
�d|�d��

�Yn0t|53�
}tt
jj|}t|�

tt�
|}|d
k�
s�|dk�rt�|54||
�
|D]*}|�d�}t�t
j�||�||�
�
q�n�t|t��r0|}n|du�rDt��}nd}t|||d�j}||}|t
jj|}t|�

||
���s�t�|55||
�
|D]4}||���s�t ||�d�|||||||d�	
�q�t
j�||
�S)a�5657    Prepares Downloads a module from a local folder or a distant repo and returns its path inside the cached58    Transformers module.59 60    Args:61        pretrained_model_name_or_path (`str` or `os.PathLike`):62            This can be either:63 64            - a string, the *model id* of a pretrained model configuration hosted inside a model repo on65              huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced66              under a user or organization name, like `dbmdz/bert-base-german-cased`.67            - a path to a *directory* containing a configuration file saved using the68              [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`.69 70        module_file (`str`):71            The name of the module file containing the class to look for.72        cache_dir (`str` or `os.PathLike`, *optional*):73            Path to a directory in which a downloaded pretrained model configuration should be cached if the standard74            cache should not be used.75        force_download (`bool`, *optional*, defaults to `False`):76            Whether or not to force to (re-)download the configuration files and override the cached versions if they77            exist.78        resume_download (`bool`, *optional*, defaults to `False`):79            Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists.80        proxies (`Dict[str, str]`, *optional*):81            A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',82            'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.83        use_auth_token (`str` or *bool*, *optional*):84            The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated85            when running `transformers-cli login` (stored in `~/.huggingface`).86        revision (`str`, *optional*, defaults to `"main"`):87            The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a88            git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any89            identifier allowed by git.90        local_files_only (`bool`, *optional*, defaults to `False`):91            If `True`, will only try to load the tokenizer configuration from local files.92 93    <Tip>94 95    You may pass a token in `use_auth_token` if you are not logged in (`huggingface-cli long`) and want to use private96    or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models).97 98    </Tip>99 100    Returns:101        `str`: The path to the module inside the cache.102    �local�
/r
�
vrIN��mainzDefaulting to latest_version: z`custom_revision`: z3 does not exist. Please make sure to choose one of rM)rr�pipelineF)rmrnrprorsrq�gitrEzCould not locate the z inside z--T)rr�token�rmrnrorprqrrrs)!r=r$r"rS�isfile�countrr
rK�logger�infori�COMMUNITY_PIPELINES_URL�formatr	�EnvironmentError�errorr103rVrrXr,rr�shutil�copy�104isinstancer�	get_tokenr�shar&�get_cached_module_file)rlr9rmrnrorprqrrrsZmodule_file_or_urlZresolved_module_file�	submoduleZavailable_versions�latest_versionZ105github_urlZmodules_neededZfull_submoduleZsubmodule_pathZ
module_neededr{�commit_hashrrrr��s�;











��








�	



106







�107







108
 

109
110













�111r�)112rlr9r[rmrnrorprqrrrsc113Ks,t||
|||||||	d
�	}t
||�dd��S)a�114    Extracts a class from a module file, present in the local folder or repository of a model.115 116    <Tip warning={true}>117 118    Calling this function will execute the code in the module file found locally or downloaded from the Hub. It should119    therefore only be called on trusted repos.120 121    </Tip>122 123    Args:124        pretrained_model_name_or_path (`str` or `os.PathLike`):125            This can be either:126 127            - a string, the *model id* of a pretrained model configuration hosted inside a model repo on128              huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced129              under a user or organization name, like `dbmdz/bert-base-german-cased`.130            - a path to a *directory* containing a configuration file saved using the131              [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`.132 133        module_file (`str`):134            The name of the module file containing the class to look for.135        class_name (`str`):136            The name of the class to import in the module.137        cache_dir (`str` or `os.PathLike`, *optional*):138            Path to a directory in which a downloaded pretrained model configuration should be cached if the standard139            cache should not be used.140        force_download (`bool`, *optional*, defaults to `False`):141            Whether or not to force to (re-)download the configuration files and override the cached versions if they142            exist.143        resume_download (`bool`, *optional*, defaults to `False`):144            Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists.145        proxies (`Dict[str, str]`, *optional*):146            A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',147            'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.148        use_auth_token (`str` or `bool`, *optional*):149            The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated150            when running `transformers-cli login` (stored in `~/.huggingface`).151        revision (`str`, *optional*, defaults to `"main"`):152            The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a153            git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any154            identifier allowed by git.155        local_files_only (`bool`, *optional*, defaults to `False`):156            If `True`, will only try to load the tokenizer configuration from local files.157 158    <Tip>159 160    You may pass a token in `use_auth_token` if you are not logged in (`huggingface-cli long`) and want to use private161    or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models).162 163    </Tip>164 165    Returns:166        `type`: The class, dynamically imported from the module.167 168    Examples:169 170    ```python171    # Download module `modeling.py` from huggingface.co and cache then extract the class `MyBertModel` from this172    # module.173    cls = get_class_from_dynamic_module("sgugger/my-bert-model", "modeling.py", "MyBertModel")174    ```r|rE�)r�r]rW)rlr9r[rmrnrorprqrrrs�kwargsZfinal_modulerrr�get_class_from_dynamic_moduleq
sL








�r�)NFFNNNF)NNFFNNNF)-�__doc__rPrbrr$r4r�r!�distutils.versionr�pathlibr�typingrrr�urllibr�huggingface_hubrr	r175rr�r
rrrr��176get_loggerrfrrr)r=�PathLiker,r<rHrVr]rY�boolr�r�rrrr�<module>s�










�177 





�








�0






�









�