Aluode/PerceptionLabPortable
0
1o
2 ��Yi@� � @ sh d dl mZ d dlZd dlZd dlZd dlZd dlZd dlZd dlm Z d dl3m Z d dl m
Z
mZmZ d dlmZ d dlmZ d dlmZ d d lmZmZ d d4lmZ d d lm Z d d l m Z d d
l m Z d dl!m"Z" d dl#m$Z$ d dl%m&Z& d dl'm(Z( e
r�d dlm)Z) e�*d�Z+G dd� de&�Z,G dd� de �Z-G dd� de-�Z.G dd� de.�Z/G d d � d �Z0dS ) � )� annotationsN)�Callable)�rmtree)�
TYPE_CHECKING�Any�ClassVar)�5filesystem)�DEFAULT_CALLBACK)�compr)� BaseCache� MMapCache)�BlocksizeMismatchError)�create_cache_mapper)�
CacheMetadata)�ChainedFileSystem)�LocalFileSystem)�AbstractBufferedFile)� Transaction)�infer_compression)�AbstractCacheMapperz
fsspec.cachedc @ s e Zd Zddd�ZdS )�WriteCachedTransactionTc C sV dd� | j D �}dd� | j D �}|r| j�||� | j �� d| j_d | j_d | _d S )Nc S � g | ]}|j �qS � ��path��.0�fr r ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\fsspec/implementations/cached.py�6<listcomp>" � z3WriteCachedTransaction.complete.<locals>.<listcomp>c S r r ��fnr r r r r # r F)�files�fs�put�clear�_intransZ _transaction)�self�commit�rpaths�lpathsr r r �complete! s 78z WriteCachedTransaction.completeN�T)�__name__�9__module__� __qualname__r, r r r r r s r c s� e Zd ZU dZdZded< dZ 10 d<d=� fdd�
Zedd� �Z dd� Z11dd� Z dd� Z dd� Z
dd � Zd d � Zd d � Zd>d!d"�Zd#d$� Z % & d?d'd(�Zd)d*� Zd@d0d1�Zd2d3� ZdAd4d5�Z� fd6d7�Zd8d9� Zd:d;� Z� ZS )B�CachingFileSystemaS Locally caching filesystem, layer over any other FS12 13 This class implements chunk-wise local storage of remote files, for quick14 access after the initial download. The files are stored in a given15 directory with hashes of URLs for the filenames. If no directory is given,16 a temporary one is used, which should be cleaned up by the OS after the17 process ends. The files themselves are sparse (as implemented in18 :class:`~fsspec.caching.MMapCache`), so only the data which is accessed19 takes up space.20 21 Restrictions:22 23 - the block-size must be the same for each access of a given file, unless24 all blocks of the file have already been read25 - caching can only be applied to file-systems which produce files26 derived from fsspec.spec.AbstractBufferedFile ; LocalFileSystem is also27 allowed, for testing28 )Z29blockcache�cachedz ClassVar[str | tuple[str, ...]]�protocol)�foN�TMP�30 F�: �31same_names� bool | None� cache_mapper�AbstractCacheMapper | Nonec st t � jd
i | �� |du r|du rtd��|du |du A s!td��|dkr5t�� } | g}
t�� � j| � n t|t �r>|g}
n|}
t32j |
d dd� |
� _ |pOi � _
|� _|� _|� _| � _d� _|durl|33durltd��|34durt|35� _n t|dur{|nd �� _t|t �r�|n
t|jt �r�|jn|jd36 � _t� j �� _� �� |dur�|nt|fi � j
��� _� fd d �}|� _ dS )a� 37 38 Parameters39 ----------40 target_protocol: str (optional)41 Target filesystem protocol. Provide either this or ``fs``.42 cache_storage: str or list(str)43 Location to store files. If "TMP", this is a temporary directory,44 and will be cleaned up by the OS when this process ends (or later).45 If a list, each location will be tried in the order given, but46 only the last will be considered writable.47 cache_check: int48 Number of seconds between reload of cache metadata49 check_files: bool50 Whether to explicitly see if the UID of the remote file matches51 the stored one before using. Warning: some file systems such as52 HTTP cannot reliably give a unique hash of the contents of some53 path, so be sure to set this option to False.54 expiry_time: int55 The time in seconds after which a local copy is considered useless.56 Set to falsy to prevent expiry. The default is equivalent to one57 week.58 target_options: dict or None59 Passed to the instantiation of the FS, if fs is None.60 fs: filesystem instance61 The target filesystem to run against. Provide this or ``protocol``.62 same_names: bool (optional)63 By default, target URLs are hashed using a ``HashCacheMapper`` so64 that files from different backends with the same basename do not65 conflict. If this argument is ``true``, a ``BasenameCacheMapper``66 is used instead. Other cache mapper options are available by using67 the ``cache_mapper`` keyword argument. Only one of this and68 ``cache_mapper`` should be specified.69 compression: str (optional)70 To decompress on download. Can be 'infer' (guess from the URL name),71 one of the entries in ``fsspec.compression.compr``, or None for no72 decompression.73 cache_mapper: AbstractCacheMapper (optional)74 The object use to map from original filenames to cached filenames.75 Only one of this and ``same_names`` should be specified.76 Nz9Please provide filesystem instance(fs) or target_protocolz@Both filesystems (fs) and target_protocol may not be both given.r5 �����T��exist_okzMCannot specify both same_names and cache_mapper in CachingFileSystem.__init__Fr c s � j �t� ��| ��S �N)r$ �_strip_protocol�typer �r( r r r@ � � z3CachingFileSystem.__init__.<locals>._strip_protocolr ) �super�__init__�77ValueError�tempfile�mkdtemp�weakref�finalize�_remove_tempdir�78isinstance�str�os�makedirs�storage�kwargs� cache_check� check_files�expiry� compression� _cache_size�_mapperr r3 �target_protocolr � _metadata�79load_cacher r$ r@ )r( rX Z
cache_storagerR rS � expiry_timeZtarget_optionsr$ r8 rU r: rQ �tempdirrP r@ �� __class__rB r rE C sT 7��8081���� 82zCachingFileSystem.__init__c C s$ zt | � W d S ty Y d S w r? )r � Exception)r\ r r r rK � s83 �z!CachingFileSystem._remove_tempdirc C s t j| jd dd� d S )Nr< Tr= )rN rO rP rB r r r �_mkcache� s zCachingFileSystem._mkcachec C s. | j du r| jd }td�j|dd�| _ | j S )z�Return size of cache in bytes.84 85 If more than one cache directory is in use, only the size of the last86 one (the writable cache directory) is returned.87 Nr< �fileT)Zwithdirs)rV rP r Zdu)r( � cache_dirr r r �88cache_size� s 8990z CachingFileSystem.cache_sizec C s | j �� | �� t�� | _dS )z#Read set of stored blocks from fileN)rY �loadr` �time�91last_cacherB r r r rZ � � 92z CachingFileSystem.load_cachec C s& | � � | j�� t�� | _d| _dS )z#Save set of stored blocks from fileN)r` rY �savere rf rV rB r r r �93save_cache� s 949596z CachingFileSystem.save_cachec C sR | � � | js dS t�� | j | jk}tdd� | jD ��}|s!|s'| �� dS dS )z0Reload caches if time elapsed or any disappearedNc s s � | ] }t j�|�V qd S r? )rN r �exists)r rP r r r � <genexpr>� s � z1CachingFileSystem._check_cache.<locals>.<genexpr>)r` rR re rf �allrP rZ )r( ZtimecondZ existcondr r r � _check_cache� s �z CachingFileSystem._check_cachec C s | � |�}| �� | j�|| �S )z Is path in cache and still valid)r@ rm rY Z97check_file�r( r r r r � _check_file� rg z CachingFileSystem._check_filec C s t | jd � | �� d| _dS )z�Remove all files and metadata from the cache98 99 In the case of multiple cache locations, this clears only the last one,100 which is assumed to be the read/write one.101 r< N)r rP rZ rV rB r r r � clear_cache� s 102z CachingFileSystem.clear_cachec C sf |s| j }| �� | j�|�\}}|D ]
}tj�|�r t�|� q|r.t| j d � | �103� d| _ dS )a� Remove all expired files and metadata from the cache104 105 In the case of multiple cache locations, this clears only the last one,106 which is assumed to be the read/write one.107 108 Parameters109 ----------110 expiry_time: int111 The time in seconds after which a local copy is considered useless.112 If not defined the default is equivalent to the attribute from the113 file caching instantiation.114 r< N) rT rm rY Z
clear_expiredrN r rj �remover rP rZ rV )r( r[ Z
expired_filesZwritable_cache_emptyr" r r r �clear_expired_cache� s 115�116z%CachingFileSystem.clear_expired_cachec C s2 | � |�}| j�|�}|durt�|� d| _dS )z�Remove cached version of given file117 118 Deletes local copy of the given (remote) path. If it is found in a cache119 location which is not the last, it is assumed to be read-only, and120 raises PermissionError121 N)r@ rY Zpop_filerN rq rV )r( r r" r r r �pop_from_cache s122 123 124125z CachingFileSystem.pop_from_cache�rbTc s �� ����j� ���d|vr �jj�f||||d����S ����}|rH|\}}|d |d } }126|127du rAt�d�� t||�S t�d�� n-����} tj �128�j d | �}t � }129�| |130t
�
� �j���d �}�j��|� t�d131�� |�d �} ��� �jj�f||||d | d
�����| du r��j|d < �j��|� �jr��jdkr�t��n�j} t| �dd��d|v r�|d �jkr�td|d � d�j� d���n�j|d< ���fdd�}
�jr�dn|
}t�j�j�j||132|d��_�j � � ��fdd��_ �� � �S )a! Wrap the target _open133 134 If the whole file exists in the cache, just open it locally and135 return that.136 137 Otherwise, open the file on the target FS, and make it have a mmap138 cache pointing to the location which we determine, in our cache.139 The ``blocks`` instance is shared, so as the mmap cache instance140 updates, so does the entry in our ``cached_files`` attribute.141 We monkey-patch this file, so that when it closes, we call142 ``close_and_update`` to save the state of the blocks.143 �r)�mode�144block_size�145autocommit�
cache_optionsr" �blocksT�Opening local copy of %sz#Opening partially cached copy of %sr< ��originalr" rz re �uidz!Creating local sparse file for %s�size�none)rv rw rx ry Z146cache_typer N�inferrt �rv � blocksizezDCached file must be reopened with same block size as original (old: z, new �)c s6 �j j�gt| � dd� | D �dd� | D �fi � ��S )Nc S � g | ]}|d �qS )r r �r ru r r r r � � zBCachingFileSystem._open.<locals>._fetch_ranges.<locals>.<listcomp>c S r� )� r r� r r r r � r� )r$ �147cat_ranges�len)�ranges)rQ r r( r r �
_fetch_ranges~ s ��z.CachingFileSystem._open.<locals>._fetch_ranges)�
multi_fetcherc s �� �� �S r? )�close_and_updater )�closer r( r r �<lambda>� s z)CachingFileSystem._open.<locals>.<lambda>) r@ r$ �_openro �logger�debug�openrW rN r �joinrP �setre �ukeyrY � update_file�getr` r rU r r148 r� r
r Z _fetch_range�cacher� ri )r( r rv rw rx ry rQ �detailr" �hashrz r �compr� r� r )r� r rQ r r( r r� s� 149 ���150 151152153� 154��� 155156������157�zCachingFileSystem._openc C s | j �|�S r? )r$ �_parentrn r r r r� � � zCachingFileSystem._parentr rM �argsr �returnc G s158 | � |�S r? )rW )r( r r� r r r � hash_name� s 159zCachingFileSystem.hash_namec C s� |j rdS | �|j�}| j�||� zt�d� | �� t�d� W n ty0 t�d� Y n t y= t�d� Y nw |� d|_ dS )z9Called when a file is closing, so store the set of blocksNz
going to save�savedz&Cache saving failed while closing filez-Cache save failed due to interpreter shutdownT)160�closedr@ r rY Zon_close_cached_filer� r� ri �OSError� NameError)r( r r� r r r r r� � s 161 �162z"CachingFileSystem.close_and_updatec C s | j �||�S r? )r$ �ls)r( r r� r r r r� � s zCachingFileSystem.lsc s� � dv r � �fdd�S � dv rt �� dv r t��j���S � dv r(tt��� �S � dkr0t��S t��d�}|�d d �}� |v rD|� S |d urw� |jv rR|j� S t|�}t|� �}t �163|�set � |�rut |d164�ro|j
d u ru|�||�S |S t� �� �S ) N>, r� r� �isfilerj �to_dict� pipe_filers ro �_make_local_detailsr� r� � commit_manyrc �cat�to_jsonrm �tailrp rE Zend_transaction�165__reduce__Z_paths_from_pathZcat_file� _cat_fileZ166read_blockri �167local_file� open_manyr` rr �piper� Z168get_mapper�headr� �__eq__�__hash__Zstart_transactionrZ �__getattribute__r� �isdir� _cat_ranges�infoc s t t��� ����| i |��S r? )�getattrrA �__get__)r� �kw��itemr( r r r� � s169 ��z4CachingFileSystem.__getattribute__.<locals>.<lambda>)�
__reduce_ex__)� transaction> �transaction_typer3 �_cacher^ �__dict__r$ �__self__)�AttributeErrorrA r� r� r� �objectr� r� r� �inspect�170isfunction�isdatadescriptor�hasattrr� rD )r( r� �dr$ �cls�mr] r� r r� � s4 0 171172173�174 z"CachingFileSystem.__getattribute__c C s~ | |u rdS t |t| ��sdS | j|jko>| j|jko>| j|jko>| j|jko>| j|jko>| j|jko>| j|jko>| j |j kS )zTest for equality.TF)175rL rA rP rQ rR rS rT rU rW rX )r( �otherr r r r� � s& 176�177�178�179�180�181�182�zCachingFileSystem.__eq__c C sX t t| j��t t| j��A t | j�A t | j�A t | j�A t | j�A t | j �A t | j183�A S )zCalculate hash.) r� �tuplerP rM rQ rR rS rT rU rW rX rB r r r r� s ��������zCachingFileSystem.__hash__)184Nr5 r6 Fr7 NNNNN)r8 r9 r: r; r? )rt NTN)r rM r� r r� rM r- ) r. r/ r0 �__doc__r3 �__annotations__Z_strip_tokenize_optionsrE � staticmethodrK r` rc rZ ri rm ro rp rr rs r� r� r� r� r� r� r� r� �
__classcell__r r r] r r1 , sL 185 �q186 187 188 189�s190191 Sr1 c @ sJ e Zd ZdZdZdZdd� Zdd� Zdd � Zd192d e fd d
�Z193ddd�Z dS )�WholeFileCacheFileSystema
Caches whole remote files on first access194 195 This class is intended as a layer over any other file system, and196 will make a local copy of each file accessed, so that all subsequent197 reads are local. This is similar to ``CachingFileSystem``, but without198 the block-wise functionality and so can work even when sparse files199 are not allowed. See its docstring for definition of the init200 arguments.201 202 The class still needs access to the remote store for listing files,203 and may refresh cached files.204 � filecacheTc s dd� �D �}d�j v r��� n ���fdd�|D �S �jr!t��fdd�|D �}dd� t||�D �}�fdd�t||�D �}dd� t||�D �}|rq�j�||� �fd d�|D �}t||�D ] \} }205�j�| |206� qa�� � d207d � � � �fd d�t||�D �S )
Nc S r r r )r Zofr r r r / r z6WholeFileCacheFileSystem.open_many.<locals>.<listcomp>ru c s> g | ]}t �j|f�jtj��jd ��|��d�� ���qS )r< �rv r" )�
LocalTempFiler$ rv rN r r� rP rW �r r )rQ �208open_filesr( r r r 3 s �����c � g | ]}� � |��qS r �ro )r �sprB r r r @ � c S � g | ]\}}|s|�qS r r �r �pr� r r r r A � c s* g | ]\}}t j�� jd � �|���qS )r< )rN r r� rP rW r� rB r r r B � ��c S r� r r )r r" r� r r r r F r� c s. g | ]}|� � |�d t�� � j�|�d��qS )Tr| )rW re r$ r� r� rB r r r L s �209��c S s t | t�r | d S | S )Nr� )rL r� r! r r r � firstpartZ rC z5WholeFileCacheFileSystem.open_many.<locals>.firstpartc s* g | ]\}}t |r
� |�n|�jd ��qS )r� )r� rv )r Zfn0Zfn1)r� r� r r r ^ r� )210rv r` rU �NotImplementedError�zipr$ r� rY r� ri ) r( r� rQ �paths�detailsZdownpathZdownfn0ZdownfnZ newdetailr r� r )r� rQ r� r( r r� . s4 211212� 213�214�215 �z"WholeFileCacheFileSystem.open_manyc C sf | j �dd� |D �dd� |D �� dd� |D � |D ]}zt�|j� W q ty- Y qw d | _d S )Nc S r r r! r r r r r d r z8WholeFileCacheFileSystem.commit_many.<locals>.<listcomp>c S r r r r r r r r d r c S s g | ]}|� � �qS r �r� r r r r r e r� )r$ r% rN rq �name�FileNotFoundErrorrV )r( r� r r r r r� c s " �216z$WholeFileCacheFileSystem.commit_manyc C sX | � |�}tj�| jd |�}||dt�� | j�|�d�}| j� ||� t217� d|� |S )Nr< Tr| �Copying %s to local cache) rW rN r r� rP re r$ r� rY r� r� r� )r( r r� r" r� r r r r� n s 218219� z,WholeFileCacheFileSystem._make_local_detailsF�raisec K s| | j |||�d�d�}g }g }g } i }220|�� D ]Q} z+| �| �} | s1| �| �}
|�| � |�|
� n
t| t�r8| nd | f\} }
| �|
� W q tyh } z|dkrQ� |dkrY||221| < |� | � W Y d }~qd }~ww |rv| j222�||� | � � |� t
|�� t|| �D ]$\} }
t|
d��}|�� |223| < W d � n1 s�w Y |�d� q�t|t�r�t
|�dkr�|du r�|224|d }225|226S ) N�maxdepth)� recursiver� r� r� rt r� Fr )Z expand_pathr� �copyro r� �appendrL r� r_ rq r$ ri Zset_sizer� r� r� �readZrelative_updaterM )r( r r� �on_error�callbackrQ r� ZgetpathsZ227storepathsZfns�outr� r� r" �er r r r r� | sH � 228229230 �� � z WholeFileCacheFileSystem.catrt c K s� | � |�}d|vr-| �|�}tj�| jd |�}dd� |�� D �}t| |f||d�|��S | �|�}|r`|\}}|d |d }} | du rXt �231d |� t ||�}232|� d233�|234_
|235S td |� d ���| �|�}||d
< | �� | jr�| jj|fi |���V}236t |d��A} t|237t�r�td|238jj|239j�|240_| jdkr�t|�n| j} t| |241dd�}242d}
|
r�t |243dd�}|244� |�}
| � |
� |
s�W d � n1 s�w Y W d � n1 s�w Y n| j� ||� | � � | �||�S )Nru r< c S � i | ] \}}|d vr||�qS �)rx rw ry r �r �k�vr r r �245<dictcomp>� s246 �z2WholeFileCacheFileSystem._open.<locals>.<dictcomp>r� r" rz Tr{ r} z&Attempt to open partially cached file z as a wholly cached filerv �wbr r� rt r� r� � P )!r@ rW rN r r� rP �itemsr� ro r� r� r� r� r} rF r� r` rU r$ r� rL r r r� �fetcherr r r247 r� r� �write�get_fileri )r( r rv rQ r� r" �user_specified_kwargsr� �_rz r �f2r� �data�blockr r r r� � s\ 248249�250 251 252�253"254255�� 256257�� �� � z WholeFileCacheFileSystem._openN�rt ) r. r/ r0 r� r3 r� r� r� r� r r� r� r r r r r� s
5 258�-r� c s� e Zd ZdZdZdZeZ� fdd�Zdd� Z dd � Z259d260d � Z d � fd
d� Z d dd�Z
dd� Zd dd�Zd!dd�Z d"dd�Z d"dd�Zd#d d �Z� ZS )$�SimpleCacheFileSystema� Caches whole remote files on first access261 262 This class is intended as a layer over any other file system, and263 will make a local copy of each file accessed, so that all subsequent264 reads are local. This implementation only copies whole files, and265 does not keep any metadata about the download time or file details.266 It is therefore safer to use in multi-threaded/concurrent situations.267 268 This is the only of the caching filesystems that supports write: you will269 be given a real local open file, and upon close and commit, it will be270 uploaded to the target filesystem; the writability or the target URL is271 not checked until that time.272 273 Z simplecacheTc sV |� � }dD ]}d||< qt� jdi |�� | jD ]}tj�|�s(tj|dd� qd S )N)rR r[ rS FTr= r )r� rD rE rP rN r rj rO )r( rQ r� �keyrP r] r r rE � s 274275 ��z SimpleCacheFileSystem.__init__c C sD | � � | �|�}| jD ]}tj�||�}tj�|�r | S q d S r? )rm rW rP rN r r� rj )r( r �sharP r" r r r ro s 276277 ��z!SimpleCacheFileSystem._check_filec C � d S r? r rB r r r ri 278 � z SimpleCacheFileSystem.save_cachec C r
r? r rB r r r rZ
r z SimpleCacheFileSystem.load_cacheNc sT | j r!| �|d��}|�|� W d � d S 1 sw Y d S t� �||� d S )Nr� )r' r� r rD r� )r( r �valuerQ r r] r r r� s279 "�z SimpleCacheFileSystem.pipe_filec 280 281 K sH | � |�}g }z| jj|fddi|���� }W n ty+ } z|}W Y d }~nd }~ww d }| jr�|�d�d }| jjD ]R}|j |krR|�282||j pL|� � dd�� q<|j �
|�r�|j �d�|�d�krs|�283|j |j pm|� � dd�� q<d�|j �d�d |�d�d � �} |�284| ddd�� q<|d ur�|s�|�|r�|S td d285� |D ��S ) Nr� T�/ra �r� r rA r� r � directoryc s s � | ]}|d V qdS )r� Nr �r r r r r rk 5 s � z+SimpleCacheFileSystem.ls.<locals>.<genexpr>)r@ r$ r� r� r� r' �rstripr� r# r r� r �tell�286startswith�countr� �split�sorted)287r( r r� rQ r� r� �exZpath1r Zdnamer r r r� sJ 288������ 289� �$� zSimpleCacheFileSystem.lsc s� | � � �� | jrD� fdd�| jjD �}|r/|d jr#tj�|d j�n|d � � }� |dd�S t290� fdd�| jjD ��}|rD� ddd�S | j j � fi |��S ) Nc s g | ] }|j � kr|�qS r r r r r r r : s z.SimpleCacheFileSystem.info.<locals>.<listcomp>r ra r c 3 s � | ] }|j �� d �V qdS )r N)r r r r r r rk > s � z-SimpleCacheFileSystem.info.<locals>.<genexpr>r )
r@ r' r� r# r� rN r �getsizer" r �anyr$ r� )r( r rQ r r r r r r� 7 s 291( zSimpleCacheFileSystem.infoc K sj t |t�r| j| �|�|fi |�� d S t |t�r1|�� D ]\}}| j| �|�|fi |�� q d S td��)Nzpath must be str or dict)rL rM r� r@ �dictr rF )r( r r rQ r� r� r r r r� C s 292 293 �zSimpleCacheFileSystem.pipec � s� �t �d|� | �|�}| �|�}| �|�}|s0tj�| jd |�}| j j294||fi |��I d H t |d�� }|r=|� |� |d u rCdn||�
� }|�|�W d � S 1 sXw Y d S )Nzasync cat_file %sr< rt )r� r� r@ rW ro rN r r� rP r$ Z _get_filer� �seekr r� ) r( r �start�endrQ r r" r r r r r r� L s � 295296297 298$�z SimpleCacheFileSystem._cat_filer� c � s� �t �d|� g }t� }g } g }299|D ]2} | �| �} | d u r?| |vr?| �| �}
tj�| jd |
�} | � | � |�300| � |301� | � |� | � q| rS| j j |302| |d�I d H t
� j|||f||d�|��S )Nzasync cat ranges %sr< )r� ��max_gapr� )r� r� r� ro rW rN r r� rP r� �addr$ �_getr r� )r( r� �starts�endsr" r� rQ r+ Zrset�downloadr* r� r" r r r r r� \ s0 � 303304305306307 ���z!SimpleCacheFileSystem._cat_rangesc s� t �d|� � fdd�|D �}dd� t||�D �}dd� t||�D �}� j�||� � fdd�|D �}t� j|||f||d�|��S )Nz
cat ranges %sc r� r r� �r r� rB r r r x r� z4SimpleCacheFileSystem.cat_ranges.<locals>.<listcomp>c S s g | ]308\}}|d u r|�qS �Fr �r �lr� r r r r y � c S s g | ]309\}}|d u r|�qS r) r r* r r r r z r, c r� r r� r( rB r r r | r� r! )r� r� r� r$ r� r r� ) r( r� r% r& r" r� rQ r+ r* r rB r r� t s ���z SimpleCacheFileSystem.cat_rangesrt c K s� | � |�}| �|�}d|vr0tj�| jd |�}dd� |�� D �}t| |f|| j |d�|��S | � |�}|r<t310||�S tj�| jd |�}t � d|� ||d< | �
� d | _| jr�| jj|fi |���V}t311|d��A}t|t�rztd |jj|j�|_| jd312kr�t|�n| j} t| |d d �}d
}313|314r�t|dd�} |�| �}315|� |316� |317s�W d � n1 s�w Y W d � n1 s�w Y n| j� ||� | �||�S )Nru r< c S r� r� r r� r r r r� � s318 �z/SimpleCacheFileSystem._open.<locals>.<dictcomp>)rv rx r" r� rv r� r r� rt r� Tr� r� ) r@ rW rN r r� rP r r� r' ro r� r� r� r` rV rU r$ r� rL r r r� r r r r319 r� r� r r ) r( r rv rQ r r" r r r r� r r r r r r� � s^ 320321����322323 "324325�� 326327�� �� � zSimpleCacheFileSystem._openr? r- )NN)Nr� r )r. r/ r0 r� r3 r� r r� rE ro ri rZ r� r� r� r� r� r� r� r� r� r r r] r r328 � s& 329 330 331 332�333�
r334 c @ sh e Zd ZdZd dd�Zdd� Zd d335� Zd d � Zd
d� Zdd� Z dd� Z336e dd� �Z d dd�Z
dd� ZdS ) r� z8A temporary local file, which will be uploaded on commitr� Tr c K sP || _ t||�| _|| _|r| j�|� || _d | _|| _d| _|| _ || _337d S )NF) r" r� �fhrv r r r r$ r� rx rQ )r( r$ r r" rv rx r rQ r r r rE � s 338zLocalTempFile.__init__c C s t | j| j| jd| j| �� ffS )Nzr+b)r� r$ r r" rx r rB r r r r� � s �zLocalTempFile.__reduce__c C � | j S r? )r- rB r r r � __enter__� s zLocalTempFile.__enter__c C s | � � d S r? r� )r( �exc_type�exc_val�exc_tbr r r �__exit__� r� zLocalTempFile.__exit__c C s0 | j rd S | j�� d| _ | jr| �� d S d S )NT)r� r- r� rx r) rB r r r r� � s 339 �zLocalTempFile.closec C s | j �� t�| j� d S r? )r- r� rN rq r" rB r r r �discard� s 340zLocalTempFile.discardc C s$ | j j| jg| jgfi | j�� d S r? )r$ r% r" r rQ rB r r r r) � s $zLocalTempFile.commitc C r. r? r! rB r r r r� � s zLocalTempFile.namer� rM c C s d| j � �S )NzLocalTempFile: r rB r r r �__repr__� r� zLocalTempFile.__repr__c C s t | j|�S r? )r� r- )r( r� r r r � __getattr__� r� zLocalTempFile.__getattr__N)r� Tr )r� rM )r. r/ r0 r� rE r� r/ r3 r� r4 r) �propertyr� r5 r6 r r r r r� � s 341
342343 r� )1�344__future__r r� �loggingrN rG re rI �collections.abcr �shutilr �typingr r r Zfsspecr Zfsspec.callbacksr Zfsspec.compressionr345 Z fsspec.corer r Zfsspec.exceptionsr
Z#fsspec.implementations.cache_mapperr Z%fsspec.implementations.cache_metadatar Z fsspec.implementations.chainedr Z fsspec.implementations.localr Z fsspec.specr Zfsspec.transactionr Z fsspec.utilsr r � getLoggerr� r r1 r� r346 r� r r r r �<module> sF 347 t I O