CoolFace
Modelpublic

palsetuf/BacPT-large

sourceHugging Facemitupdated 10d agoView on Hugging Face
0likes21downloads
inference.cpython-311.pyc70 linesDownload Raw Back to __pycache__
1�

2��jV����dZddlZddlZddlmZddlZddlZddlm	cm3Zddlm
Z
ddlmZmZddlmZmZdZdZd	Zd4�Zd�Zd�Zd
�Zd�Zdd�Zd�Zedkre��dSdS)zHRun a local BacPT Hugging Face release on an ordered protein FASTA file.�N)�Path)�	load_file)�
RobertaConfig�RoFormerConfig)�5BacPTLarge�6BacPTSmalli�i��c�H������g�g�d�g�t��������fd�}t|�����5}|D]�}|�d��r@|��|dd����d��std���g��W|���r8��td�����|�������	ddd��n#1swxYwY|���std�����fS)	zCRead a protein FASTA and omit sequences longer than 2,250 residues.Nc�b����dSd����}|std��������vrtd�����������t|��tkr���|f��dS���t|��f��dS)N�zEmpty sequence for zDuplicate protein ID: )�join�7ValueError�add�len�MAX_PROTEIN_LENGTH�append)�sequence�chunks�omitted�8protein_id�retained�seens ������$huggingface/BacPT-large/inference.py�
finish_recordz!read_fasta.<locals>.finish_records�������F��7�7�6�?�?���	A��?�:�?�?�@�@�@������B�j�B�B�C�C�C���������x�=�=�.�.�.��O�O�Z��2�3�3�3�3�3��N�N�J��H�
�
�6�7�7�7�7�7��>�rzFASTA header has no protein IDz)FASTA sequence appeared before its headerz)No proteins remain after length filtering)�setr�open�9startswith�splitr�stripr)	�pathr�stream�linerrrrrs	    @@@@@r�10read_fastar&s���������H��G��J�
�F��5�5�D�8�8�8�8�8�8�8�8�8�11�d�����	�	�,�f��12	,�13	,�D����s�#�#�	
,��
����!�!�"�"�X�^�^�-�-�a�0�14�!�G�$�%E�F�F�F���������
,��%�$�%P�Q�Q�Q��
�
�d�j�j�l�l�+�+�+��15	,�,�,�,�,�,�,�,�,�,�,�,����,�,�,�,��M�O�O�O��F��D�E�E�E��W��s�B'C:�:C>�C>c��tj�dd��\}}|�|�����|fS)z8Load the original FAIR ESM2 35M model through Torch Hub.zfacebookresearch/esm:v2.0.0�esm2_t12_35M_UR50D)�torch�hub�load�to�eval)�device�model�alphabets   r�	load_esm2r1=sF���i�n�n�%�';���O�E�8��8�8�F��� � �"�"�H�,�,rc���t|��\}}|���}g}tj��5|D]�\}}|�dd��}|||fg��\}}}	||	�|��tgd���dt}16|�|17ddd�f�d���	���18������	d	d	d	��n#1swxYwY~|jd19krtj�
��tj|���tjd���S)zDMean-pool ESM2 layer 12 over amino-acid positions for every protein.�Jz<mask>F)�repr_layers�return_contacts�representationsrr�����N�cuda)�copy)r1�get_batch_converterr)�inference_mode�replacer,�20ESM2_LAYERr�mean�cpu�numpy�typer8�empty_cache�np�stack�astype�float32)�proteinsr.r/r0�	converter�vectorsrr�_�tokens�representations           r�embed_proteinsrMEs�����'�'�O�E�8��,�,�.�.�I��G�	�	�	�	�J�J�$,�	J�	J� �J���'�'��X�6�6�H�$�9�z�8�&<�%=�>�>�L�A�q�&�"�U��	�	�&�!�!�21�|�U�����!�!+�-�N�
�N�N�>�!�Q�r�T�'�2�7�7��:�:�>�>�@�@�F�F�H�H�I�I�I�I�
	J�J�J�J�J�J�J�J�J�J�J�J����J�J�J�J�	�
�{�f���
�22��� � � �
�8�G���#�#�B�J�U�#�;�;�;s�B?D�D�Dc�j�|dz���5}tj|��}ddd��n#1swxYwY|d}|dkr$tj|��}t|��}n<|dkr$t
j|��}t|��}ntd|�����t|dzd�	��}d23|vr+d24|�25��vr|�d26��|�|d���|�
|�����|fS)
zFLoad the variant, configuration, and safetensors weights in model_dir.zconfig.jsonN�
bacpt_variant�small�largezUnsupported BacPT variant: zmodel.safetensorsr?)r.zembeddings.position_idsT)�strict)r�jsonr+r�from_pretrainedrrrrr�27state_dict�pop�load_state_dictr,r-)�	model_dirr.r$�28raw_config�variant�configr/�states        r�29load_bacptr]Xsx��30�m�31#�	)�	)�	+�	+�'�v��Y�v�&�&�32�'�'�'�'�'�'�'�'�'�'�'����'�'�'�'���)�G��'����.�y�9�9���6�"�"���	�G�	�	��/�	�:�:���6�"�"����@�w�@�@�A�A�A��i�"5�5�e�D�D�D�E� �E�)�)�.G�u�O_�O_�Oa�Oa�.a�.a�
�	�	�+�,�,�,�	���%���-�-�-��8�8�F��� � �"�"�G�+�+s
�9�=�=c�|�tj|d���5}|d}|d}ddd��n#1swxYwY|jdks#|jdkstj|dk��rt	d���|�tj��|z33|z�tj��S)	z@Apply the fitted BacPT StandardScaler parameters without pickle.F)�allow_pickler>�scaleN)i�rzInvalid BacPT scaler)rCr+�shape�anyrrE�float64rF)�34embeddings�scaler_path�scalerr>r`s     r�scale_embeddingsrgls���	���5�	1�	1�	1� �V��f�~���w��� � � � � � � � � � � ���� � � � ��z�V���u�{�f�4�4���u��z�8J�8J�4��/�0�0�0�
�
�
�r�z�
*�
*�T�
1�U�:�B�B�2�:�N�N�Ns�4�8�8Fc���	�tt|��t���	tj|d�	����d���|��}tj|dddt�	z35f��}tj	dtdftj36|���}d|dd�d�	�dd�f<tj��5tj�
|jtj|jdk���5|||���\}}ddd��n#1swxYwYddd��n#1swxYwY|dd�	�f���������|ddd�	�f���������d	�}|r#t'j�	fd37�|D����|d<�	|fS)z@Apply historical padding, attention masking, and CUDA precision.Nrr)�dtyper.r8)�device_typeri�enabled)�
inputs_embeds�attention_maskr7)�reconstruction�last_hidden_statec���g|]D}|dd��f�����������ES)rN)�floatr?r@)�.0�layer�counts  �r�38<listcomp>zrun_bacpt.<locals>.<listcomp>�sJ���O�O�O��U�1�f�u�f�9�
�
#�
#�
%�
%�
)�
)�
+�
+�
1�
1�
3�
3�O�O�Or�
hidden_states)�minr�MAX_PROTEINSr)�39from_numpy�	unsqueezer,�F�pad�zeros�intr;�amp�autocastrA�float16rqr?r@rCrD)40r/rdr.�41all_layers�inputs�maskrnrv�resultrts42         @r�	run_bacptr�vs������J����.�.�E�
�
�j��%��0�
1�
1�
;�
;�A�
>�
>�
A�
A�&�
I�
I�F�
�U�6�A�q�!�\�E�%9�:�
;�
;�F��;��<��+�5�9�V�L�L�L�D��D����F�U�F�A�A�A���	�	�	�	�Y�Y���!3�!3��K��m���v�%�"4�"�"�Y�Y�43).��F�SW�(X�(X�(X�%��
�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y����Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y�Y����Y�Y�Y�Y�)��F�U�F��3�9�9�;�;�?�?�A�A�G�G�I�I�*�2�.�q�&�5�&�y�9�?�?�A�A�E�E�G�G�M�M�O�O���F��44�"$�(�O�O�O�O��O�O�O�#45�#46�����&�=�s6�;E�D)�E�)D-	�-E�0D-	�1E�E�Ec��tjt���}|�dtdd���|�dtdd���|�dtt	t47�����j�	��|�d48dd�
��|�dd���|���}|j	dkr5tj	tj���rdnd��}ntj	|j	��}|j
dkr3tj���s|�d��t|j��\}}t#||��}t%||jdz��}t)|j|��\}}t+||||j��\}	}49d�|D��}|50�t1j|d|	�t4���t1jd�|D��t4���t1j||	d�t4������t1j|jfi|51��t;d|�d|	�dt=|���dt>�dt=|��|	z52�dt@�d�
��dS) N)�descriptionz--fastaTz1protein FASTA already ordered by genomic position)rA�required�helpz--outputz53output NPZz--model-dir)rA�defaultz--device)�autor?r8r�)�choicesr�z--all-layers�54store_true)�actionr8r?z%CUDA was requested but is unavailablez55scaler.npzc��g|]\}}|��S�r��rrrrJs   rruzmain.<locals>.<listcomp>�s��<�<�<�-�*�a�:�<�<�<r)ric��g|]\}}|��Sr�r�r�s   rruzmain.<locals>.<listcomp>�s��5�5�5�M�J��Z�5�5�5r)�protein_ids�omitted_protein_ids�truncated_protein_idszSaved BacPT-z representations for z proteins; omitted z
 longer than z residues and truncated z beyond position �.)!�argparse�ArgumentParser�__doc__�add_argumentr�__file__�resolve�parent�56parse_argsr.r)r8�is_availablerA�errorr&�fastarMrgrXr]r�r��updaterC�asarray�str�savez_compressed�output�printrrrx)�parser�argsr.rGr�raw_embeddings�scaled_embeddingsr/rZrtr�r�s            r�mainr��s���
�
$��
9�
9�
9�F�57���	��t�P��R�R�R�58���59���<��P�P�P�60���
�D�$�x�.�.�:P�:P�:R�:R�:Y��Z�Z�Z�61���62�,C�V��T�T�T�63����|��<�<�<������D��{�f������64�(?�(?�(A�(A�L�f�f�u�M�M�����d�k�*�*��
�{�f���U�Z�%<�%<�%>�%>�����<�=�=�=�"�4�:�.�.��H�g�#�H�f�5�5�N�(����,�9V�W�W������7�7�N�E�7��e�%6����P�P�M�E�6�<�<�8�<�<�<�K�65�M�M��J�{�6�E�6�2�#�>�>�>��J�5�5�W�5�5�5�S�66�67�68�!�j��U�V�V�)<�C�H�H�H���������.�.�v�.�.�.�	�	M�w�	M�	M�U�	M�	M��w�<�<�	M�	M�.@�	M�	M���]�]�U�*�	M�	M�=I�	M�	M�	M�����r�__main__)F)r�r�rS�pathlibrr@rCr)�torch.nn.functional�nn�69functionalr{�safetensors.torchr�transformersrr�modeling_bacptrrrrxr=r&r1rMr]rgr�r��__name__r�rr�<module>r�sJ��N�N��������������������������������'�'�'�'�'�'�6�6�6�6�6�6�6�6�1�1�1�1�1�1�1�1�����
�70�%�%�%�P-�-�-�<�<�<�&,�,�,�(O�O�O�����0$�$�$�N�z����D�F�F�F�F�F��r