Aluode/PerceptionLabPortable
0
1o
2 ��Yir � @ s� d dl Z d dlmZmZ d dlmZ d dlmZ d dlm Z m3Z4 d dl m Z dd� Z
d d5� ZG d d � d e6�ZG d
d� de �ZdS )� N)�Client�_get_global_client)�Worker)�7filesystem)�AbstractBufferedFile�AbstractFileSystem��infer_storage_optionsc C s$ | d u rt � S t| t�r| S t| �S �N)r �8isinstancer )�client� r
��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\fsspec/implementations/dask.py� _get_client9 s10 11r c C s12 t tj�S r13 )�boolr Z14_instancesr
r
r
r �15_in_worker s 16r c sz e Zd ZdZ d� fdd� Zedd� �Zdd� Zd d17� Zd d � Z d
d� Z18dd� Z dd� Z ddd�Z
dd� Z� ZS )�DaskWorkerFileSystema) View files accessible to a worker as any other remote file-system19 20 When instances are run on the worker, uses the real filesystem. When21 run on the client, they call the worker to provide information or data.22 23 **Warning** this implementation is experimental, and read-only for now.24 Nc sT t � jdi |�� |d u |d u A std��|| _|| _d | _|| _|| _| �� d S )NzKPlease provide one of filesystem instance (fs) or target_protocol, not bothr
) �super�__init__�25ValueError�target_protocol�target_options�workerr �fs�_determine_worker)�selfr r r r �kwargs�� __class__r
r r ! s � z DaskWorkerFileSystem.__init__c C s6 t | �}d|v rd|v rd|d � d|d � �iS i S )N�host�portr �:r )�path�sor
r
r �_get_kwargs_from_urls1 s z*DaskWorkerFileSystem._get_kwargs_from_urlsc C sZ t � r d| _| jd u rt| jfi | jpi ��| _d S d S d| _t| j�| _t� | �| _26d S )NTF) r r r r r r r r �daskZdelayed�rfs�r r
r
r r 9 s 27� �� z&DaskWorkerFileSystem._determine_workerc O �6 | j r| jj|i |�� d S | jj|i |���� d S r28 )r r �mkdirr& �compute�r �argsr r
r
r r) E � zDaskWorkerFileSystem.mkdirc O r( r29 )r r �rmr& r* r+ r
r
r r. K r- zDaskWorkerFileSystem.rmc O r( r30 )r r �copyr& r* r+ r
r
r r/ Q r- zDaskWorkerFileSystem.copyc O r( r31 )r r �mvr& r* r+ r
r
r r0 W r- zDaskWorkerFileSystem.mvc O s. | j r | jj|i |��S | jj|i |���� S r32 )r r �lsr& r* r+ r
r
r r1 ] s zDaskWorkerFileSystem.ls�rbTc K s@ | j r| jj|f||||d�|��S td| |||||d�|��S )N)�mode�33block_size�34autocommit�
cache_options)r r" r3 r4 r5 r6 r
)r r �_open�DaskFile)r r" r3 r4 r5 r6 r r
r
r r7 c s* ��� ��zDaskWorkerFileSystem._openc C sd | j r'| �||��}|�|� |�|| �W d � S 1 s w Y d S | j�||||��� S r35 )r r7 �seek�readr& � fetch_ranger* )r r" r3 �start�end�fr
r
r r; � s 36 $�z DaskWorkerFileSystem.fetch_range)NNNN)r2 NTN)�__name__�37__module__� __qualname__�__doc__r � staticmethodr$ r r) r. r/ r0 r1 r7 r; �
__classcell__r
r
r r r s$ �38 39� r c s8 e Zd Zd � fdd� Zd dd�Zdd� Zd d40� Z� ZS )
r8 r2 c s& |dkrt d��t� jdi |�� d S )Nr2 z1Remote dask files can only be opened in "rb" moder
)r r r )r r3 r r r
r r � s zDaskFile.__init__Fc C s d S r41 r
)r �finalr
r
r �
_upload_chunk� s zDaskFile._upload_chunkc C s dS )zCreate remote file/uploadNr
r' r
r
r �_initiate_upload� s zDaskFile._initiate_uploadc C s | j �| j| j||�S )z*Get the specified set of bytes from remote)r r; r" r3 )r r< r= r
r
r � _fetch_range� s zDaskFile._fetch_range)r2 )F)r? r@ rA r rF rG rH rD r
r
r r r8 � s42 43r8 )r% Zdistributed.clientr r Zdistributed.workerr Zfsspecr Z fsspec.specr r Z fsspec.utilsr r r r r8 r
r
r
r �<module> s 44q