Aluode/PerceptionLabPortable
0
1o
2 ��Yi�- � @ sP d dl Z d dlZd dlZddlmZ ddlmZ ddlmZ G dd� de�Z dS ) � N� )�AbstractFileSystem��infer_storage_options� )�3MemoryFilec s� e Zd ZdZdZdZdZdZ d&� fdd� Ze d d4� �Z5e d'd d
��Z e dd� �Z
e dd� �Ze dd� �Zd(dd�Zd)dd�Ze � fdd��Zedd � �Z d*d d �Zd(d d!�Zd)d"d#�Zd$d%� Z� ZS )+�GithubFileSystema� Interface to files in github6 7 An instance of this class provides the files residing within a remote github8 repository. You may specify a point in the repos history, by SHA, branch9 or tag (default is current master).10 11 For files less than 1 MB in size, file content is returned directly in a12 MemoryFile. For larger files, or for files tracked by git-lfs, file content13 is returned as an HTTPFile wrapping the ``download_url`` provided by the14 GitHub API.15 16 When using fsspec.open, allows URIs of the form:17 18 - "github://path/file", in which case you must specify org, repo and19 may specify sha in the extra args20 - 'github://org:repo@/precip/catalog.yml', where the org and repo are21 part of the URI22 - 'github://org:repo@sha/precip/catalog.yml', where the sha is also included23 24 ``sha`` can be the full or abbreviated hex of the commit you want to fetch25 from, or a branch or tag name (so long as it doesn't contain special characters26 like "/", "?", which would have to be HTTP-encoded).27 28 For authorised access, you must provide username and token, which can be made29 at https://github.com/settings/tokens30 z9https://api.github.com/repos/{org}/{repo}/git/trees/{sha}zChttps://api.github.com/repos/{org}/{repo}/contents/{path}?ref={sha}Zgithub)�< r Nc s� t � jd i |�� || _|| _|d u |d u A rtd��|| _|| _|d ur(|| _|d u rJd}tj |j31||d�fd| ji| j ��} | � � | �
� d }|| _| �d� zddlm}32 |33d i |��| _W d S typ d | _Y d S w )34Nz%Auth required both username and tokenz)https://api.github.com/repos/{org}/{repo}��org�repo�timeoutZdefault_branch� r )�HTTPFileSystem� )�super�__init__r r �35ValueError�username�tokenr
�requests�get�format�kw�raise_for_status�json�root�ls�httpr �http_fs� ImportError) �selfr r �shar r r
�kwargs�u�rr �� __class__r ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\fsspec/implementations/github.pyr , s8 ��� 36 �zGithubFileSystem.__init__c C s | j r d| j | jfiS i S )N�auth)r r �r! r r r( r J s zGithubFileSystem.kwTc C s@ t jdddg| � d|� d�| jd�}|�� dd� |�� D �S ) av List repo names for given org or user37 38 This may become the top level of the FS39 40 Parameters41 ----------42 org_or_user: str43 Name of the github org or user to query44 is_org: bool (default True)45 Whether the name is an organisation (True) or user (False)46 47 Returns48 -------49 List of string50 zhttps://api.github.com/ZusersZorgs�/z/repos)r
c S � g | ]}|d �qS ��namer )�.0r r r r( �51<listcomp>f � z*GithubFileSystem.repos.<locals>.<listcomp>)r r r
r r )�clsZ org_or_userZis_orgr% r r r( �reposP s �zGithubFileSystem.reposc C �F t jd| j� d| j� d�fd| ji| j��}|�� dd� |�� D �S )zNames of tags in the repo� https://api.github.com/repos/r+ z/tagsr
c S r, r- r �r/ �tr r r( r0 q r1 z)GithubFileSystem.tags.<locals>.<listcomp>�r r r r r
r r r �r! r% r r r( �tagsh � ���zGithubFileSystem.tagsc C r4 )z Names of branches in the repor5 r+ z /branchesr
c S r, r- r r6 r r r( r0 | r1 z-GithubFileSystem.branches.<locals>.<listcomp>r8 r9 r r r( �branchess r; zGithubFileSystem.branchesc C s | j | jd�S )z#Named references, tags and branches�r: r< r= r* r r r( �refs~ s zGithubFileSystem.refsFc 52 s� | � � �� � dkr|p
| j}|du ra� �d��d�}d�|p | j}|D ]=}| j�d||d�}��r5d| n|7 ��fdd�|D �}|sGt� ��|d }|d d53kr\|rX|g S � S |d }q#� | jvsm|| jdfvr�tj| j j54| j | j |d �fd
| j
i| j��} | jdkr�t� ��| �� d55dd��� �fdd�| �� d D �}|| jdfv r�|| j� < n| j� }|r�|S tdd� |D ��S )a List files at given path56 57 Parameters58 ----------59 path: str60 Location to list, relative to repo root61 detail: bool62 If True, returns list of dicts, one per file; if False, returns63 list of full filenames only64 sha: str (optional)65 List at the given point in the repo history, branch or tag name or commit66 SHA67 _sha: str (optional)68 List this specific tree object (used internally to descend into trees)69 r Nr+ T)r" �_shac s g | ]70}|d � kr|�qS r- r )r/ �o)�so_farr r( r0 � s z'GithubFileSystem.ls.<locals>.<listcomp>r �type�filer" )r r r" r
� � directory)Zblob�treec sX g | ](}|d �v r� r� d |d n|d |d �|d |� dd�|d d��qS )rB r+ �path�mode�sizer r" )r. rH rB rI r" )r �r/ �f)rG �typesr r( r0 � s �7172�rF c S r, r- r rJ r r r( r0 � r1 )�_strip_protocolr �rstrip�splitr �FileNotFoundError�dircacher r �urlr r r r
r � status_coder r �sorted)73r! rG �detailr" r? r# �parts�part�outr% r )rG rA rL r( r � sR 747576 7778���7980 81� 82�83zGithubFileSystem.lsc C s | j �� d S �N)rQ �clear)r! rG r r r( �invalidate_cache� s z!GithubFileSystem.invalidate_cachec s* t |�}d|vrt� �|�S |d �d�S )Nr rG r+ )r r rM �lstrip)r2 rG �optsr& r r( rM � s z GithubFileSystem._strip_protocolc C s> t | �}d|vr84i S |d |d d�}|d r |d |d< |S )Nr �passwordr85 �hostr" r )rG r] rX r r r( �_get_kwargs_from_urls� s z&GithubFileSystem._get_kwargs_from_urls�rbc K s� |dkrt �| jj| j| j||p| jd�}tj|fd| ji| j ��}|j86dkr+t |��|� � |�
� } | d rIt�| d �}87|88�d�sItd d |89�S | jd u rRtd��| jj| d f|||d �|��S )90Nra �r r rG r" r
rD �contents# version https://git-lfs.github.com/zRPlease install fsspec[http] to access github files >1 MB or git-lfs tracked files.Z download_url)rH �91block_size�
cache_options)�NotImplementedError� content_urlr r r r r r r
r rS rP r r �base64� b64decode�92startswithr r r �open) r! rG rH rd re r" r# rR r% Z content_jsonrc r r r( �_open� s6 �9394 95����zGithubFileSystem._openc C s0 | j |||d�}t|�D ] }| j||d� q d S )N)� recursive�maxdepth)�message)Z expand_path�reversed�rm_file)r! rG rm rn ro �pr r r( �rm s �zGithubFileSystem.rmc K s$ | j std��| �|�}| �|�}|sC| jj| j| j|�d�| j d�}t96j |fd| j i| j
��}|jdkr9t|��|�� |�� d }| jj| j| j|| j d�}| j }|pYd|� �|d�|rbd |ini �} t97j|f| | j d98�| j
��}|�� � d d �}99t�d
|100�r�d} t| ��|�� | �|� dS )aW 101 Remove a file from a specified branch using a given commit message.102 103 Since Github DELETE operation requires a branch name, and we can't reliably104 determine whether the provided SHA refers to a branch, tag, or commit, we105 assume it's a branch. If it's not, the user will encounter an error when106 attempting to retrieve the file SHA or delete the file.107 108 Parameters109 ----------110 path: str111 The file's location relative to the repository root.112 message: str, optional113 The commit message for the deletion.114 zAuthentication requiredr+ rb r
rD r" zDelete )ro r" �branch)r r
ro r zBranch .+ not foundzTRemove only works when the filesystem is initialised from a branch or default (None)N)r r rM �_get_sha_from_cacherg r r r r\ r r r r
r rS rP r r �delete�re�searchr[ ) r! rG ro r# r" rR r% Z115delete_urlrt �data�
error_message�errorr r r( rq s: 116117�118 � �� zGithubFileSystem.rm_filec C sL | j �� D ] }|D ]}|�d�}|r"||kr"d|v r"|d S q qd S )Nr. r" )rQ �valuesr )r! rG �entries�entry�119entry_pathr r r( ru G s 120��z$GithubFileSystem._get_sha_from_cache)NNNN)T)FNNrY )ra NNN)�__name__�121__module__� __qualname__�__doc__rR rg �protocolr
r �propertyr � classmethodr3 r: r< r> r r[ rM � staticmethodr` rl rs rq ru �
__classcell__r r r& r( r s@ � 122 123124 125126 127128129B130 131�13231336r )134rh rw r �specr �utilsr Zmemoryr r r r r r( �<module> s