File: //lib/python3/dist-packages/awscli/customizations/s3/__pycache__/s3handler.cpython-310.pyc
o
.&�a[ � @ s: d dl Z d dlZd dlmZ d dlmZmZmZmZm Z m
Z
d dl m Z d dl
mZ d dl
mZ d dl
mZ d dl
mZ d d l
mZ d d
l
mZ d d l
mZ d d l
mZ d d
l
mZ d dl
mZ d dl
mZ d dl
mZ d dl
mZ d dl
mZ d dl
m Z d dl
m Z d dlm Z d dlm Z d dlm Z d dlm!Z! d dlm"Z" d dlm#Z# d dlm$Z$ d d lm%Z% d d lm&Z& d d lm'Z' d d l(m)Z) e �*e+�Z,G d d!� d!e-�Z.G d"d#� d#e-�Z/G d$d%� d%e-�Z0G d&d'� d'e0�Z1G d(d)� d)e0�Z2G d*d+� d+e0�Z3G d,d-� d-e1�Z4G d.d/� d/e2�Z5G d0d1� d1e0�Z6G d2d3� d3e0�Z7dS )4� N)�TransferManager)�human_readable_size�MAX_UPLOAD_SIZE�find_bucket_key�
relative_path�create_warning�NonSeekableStream)�*create_transfer_config_from_runtime_config)�UploadResultSubscriber)�DownloadResultSubscriber)�CopyResultSubscriber)� UploadStreamResultSubscriber)� DownloadStreamResultSubscriber)�DeleteResultSubscriber)� QueuedResult)�
SuccessResult)�
FailureResult)� DryRunResult)�ResultRecorder)�
ResultPrinter)�OnlyShowErrorsResultPrinter)�NoProgressResultPrinter)�ResultProcessor)�CommandResultRecorder)�RequestParamsMapper��StdoutBytesWriter)�ProvideSizeSubscriber)�"ProvideUploadContentTypeSubscriber)� ProvideCopyContentTypeSubscriber)�!ProvideLastModifiedTimeSubscriber)�DirectoryCreatorSubscriber)�DeleteSourceFileSubscriber)� DeleteSourceObjectSubscriber)� DeleteCopySourceObjectSubscriber)�get_binary_stdinc @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) �S3TransferHandlerFactory� c C s || _ || _dS )a Factory for S3TransferHandlers
:type cli_params: dict
:param cli_params: The parameters provide to the CLI command
:type runtime_config: RuntimeConfig
:param runtime_config: The runtime config for the CLI command
being run
N)� _cli_params�_runtime_config)�self�
cli_params�runtime_config� r- �D/usr/lib/python3/dist-packages/awscli/customizations/s3/s3handler.py�__init__: s
z!S3TransferHandlerFactory.__init__c C sr t | j�}| j|_| j|_t||�}t�d|j|j � t
� }|g}| � ||� t ||�}t
|||�}t|| j|�S )ag Creates a S3TransferHandler instance
:type client: botocore.client.Client
:param client: The client to power the S3TransferHandler
:type result_queue: queue.Queue
:param result_queue: The result queue to be used to process results
for the S3TransferHandler
:returns: A S3TransferHandler instance
z7Using a multipart threshold of %s and a part size of %s)r r) �MAX_IN_MEMORY_CHUNKS�max_in_memory_upload_chunks� max_in_memory_download_chunksr �LOGGER�debug�multipart_threshold�multipart_chunksizer �_add_result_printerr r �S3TransferHandlerr( ) r* �client� result_queue�transfer_config�transfer_manager�result_recorder�result_processor_handlers�result_processor�command_result_recorderr- r- r. �__call__G s0 ��
� ���z!S3TransferHandlerFactory.__call__c C sh | j �d�rd S | j �d�rt|�}n| j �d�r t|�}n| j �d�s)t|�}nt|�}|�|� d S )N�quiet�only_show_errors� is_stream�progress)r( �getr r r �append)r* r= r> �result_printerr- r- r. r7 k s
z,S3TransferHandlerFactory._add_result_printerN)�__name__�
__module__� __qualname__r0 r/ rA r7 r- r- r- r. r& 7 s
$r& c @ s e Zd Zdd� Zdd� ZdS )r8 c C sP || _ || _| j | jj|f}t|� t|� t|� t|� t|� t|� t |� g| _
dS )a Backend for performing S3 transfers
:type transfer_manager: s3transfer.manager.TransferManager
:param transfer_manager: Transfer manager to use for transfers
:type cli_params: dict
:param cli_params: The parameters passed to the CLI command in the
form of a dictionary
:type result_command_recorder: ResultCommandRecorder
:param result_command_recorder: The result command recorder to be
used to get the final result of the transfer
N) �_transfer_manager�_result_command_recorderr: � UploadStreamRequestSubmitter� DownloadStreamRequestSubmitter�UploadRequestSubmitter�DownloadRequestSubmitter�CopyRequestSubmitter�DeleteRequestSubmitter�LocalDeleteRequestSubmitter� _submitters)r* r< r+ �result_command_recorder�submitter_argsr- r- r. r/ z s
�
�zS3TransferHandler.__init__c C s� | j �> | j�+ d}|D ]}| jD ]}|�|�r#|�|�r!|d7 } nqq | j �|� W d � n1 s5w Y W d � n1 sDw Y | j �� S )a� Process iterable of FileInfos for transfer
:type fileinfos: iterable of FileInfos
param fileinfos: Set of FileInfos to submit to underlying transfer
request submitters to make transfer API calls to S3
:rtype: CommandResult
:returns: The result of the command that specifies the number of
failures and warnings encountered.
r � N)rM rL rU �
can_submit�submit�notify_total_submissions�get_command_result)r* � fileinfos�total_submissions�fileinfo� submitterr- r- r. �call� s$
��� �� �
zS3TransferHandler.callN)rI rJ rK r/ ra r- r- r- r. r8 y s %r8 c @ s� e Zd ZdZdZdd� Zdd� Zdd� Zdd � Zd
d � Z d d
� Z
dd� Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd� Zd d � Zd d � ZdS ) � BaseTransferRequestSubmitterNc C s || _ || _|| _dS )a\ Submits transfer requests to the TransferManager
Given a FileInfo object and provided CLI parameters, it will add the
necessary extra arguments and subscribers in making a call to the
TransferManager.
:type transfer_manager: s3transfer.manager.TransferManager
:param transfer_manager: The underlying transfer manager
:type result_queue: queue.Queue
:param result_queue: The result queue to use
:type cli_params: dict
:param cli_params: The associated CLI parameters passed in to the
command as a dictionary.
N)rL �
_result_queuer( )r* r< r: r+ r- r- r. r/ � s
z%BaseTransferRequestSubmitter.__init__c C s | � |�}|s | �|�S dS )a� Submits a transfer request based on the FileInfo provided
There is no guarantee that the transfer request will be made on
behalf of the fileinfo as a fileinfo may be skipped based on
circumstances in which the transfer is not possible.
:type fileinfo: awscli.customizations.s3.fileinfo.FileInfo
:param fileinfo: The FileInfo to be used to submit a transfer
request to the underlying transfer manager.
:rtype: s3transfer.futures.TransferFuture
:returns: A TransferFuture representing the transfer if it the
transfer was submitted. If it was not submitted nothing
is returned.
N)�_warn_and_signal_if_skip�
_do_submit)r* r_ � should_skipr- r- r. rZ � s
�z#BaseTransferRequestSubmitter.submitc C � t d��)a� Checks whether it can submit a particular FileInfo
:type fileinfo: awscli.customizations.s3.fileinfo.FileInfo
:param fileinfo: The FileInfo to check if the transfer request
submitter can handle.
:returns: True if it can use the provided FileInfo to make a transfer
request to the underlying transfer manager. False, otherwise.
z can_submit()��NotImplementedError�r* r_ r- r- r. rY � s
z'BaseTransferRequestSubmitter.can_submitc C s� i }| j r | � || j� g }| �||� | jr2d| ji}| j�dd�r'd|d< |�| jdi |��� | j�d�s?| �|||�S | �|� d S )Nr: �is_moveF�move�
transfer_type�dryrunr- ) �REQUEST_MAPPER_METHODr( �_add_additional_subscribers�RESULT_SUBSCRIBER_CLASSrc rF rG �_submit_transfer_request�_submit_dryrun)r* r_ �
extra_args� subscribers�
result_kwargsr- r- r. re � s
�z'BaseTransferRequestSubmitter._do_submitc C s@ |j }| j�dd�r d}| �|�\}}| j�t|||d�� d S )Nrk Frl �rm �src�dest)�operation_namer( rF �_format_src_destrc �putr )r* r_ rm rx ry r- r- r. rs s �z+BaseTransferRequestSubmitter._submit_dryrunc C � d S �Nr- �r* ru r_ r- r- r. rp � z8BaseTransferRequestSubmitter._add_additional_subscribersc C rg )Nz_submit_transfer_request()rh )r* r_ rt ru r- r- r. rr � z5BaseTransferRequestSubmitter._submit_transfer_requestc C s | � � D ] }||�r
dS qd S )NT)�_get_warning_handlers)r* r_ �warning_handlerr- r- r. rd s
��z5BaseTransferRequestSubmitter._warn_and_signal_if_skipc C s g S r~ r- �r* r- r- r. r� s z2BaseTransferRequestSubmitter._get_warning_handlersc C s | j �d�o | j �d� S )N�guess_mime_type� content_type)r( rF r� r- r- r. �_should_inject_content_type$ s �z8BaseTransferRequestSubmitter._should_inject_content_typec C sf | j �d�s1|�� s1t�d|j|jf � | j �d�s/td|j d|j|jf �}| j� |� dS dS )N�force_glacier_transferz@Encountered glacier object s3://%s. Not performing %s on object.�ignore_glacier_warnings�s3://z�Object is of storage class GLACIER. Unable to perform %s operations on GLACIER objects. You must restore the object to be able to perform the operation. See aws s3 %s help for additional parameter options to ignore or force these transfers.TF)
r( rF �is_glacier_compatibler3 r4 rx rz r rc r| )r* r_ �warningr- r- r. �
_warn_glacier* s"
��
��
z*BaseTransferRequestSubmitter._warn_glacierc C sD dt jj }t j�|j��|�}|r t|jd�}| j�|� dS dS )Nz..z#File references a parent directory.TF) �os�path�sep�normpath� compare_key�
startswithr rc r| )r* r_ �
parent_prefix� escapes_cwdr� r- r- r. �_warn_parent_reference? s �� z3BaseTransferRequestSubmitter._warn_parent_referencec C rg )zAReturns formatted versions of a fileinfos source and destination.r{ rh rj r- r- r. r{ L s z-BaseTransferRequestSubmitter._format_src_destc C s t |�S r~ )r �r* r� r- r- r. �_format_local_pathP r� z/BaseTransferRequestSubmitter._format_local_pathc C s |� d�r|S d| S )Nr� )r� r� r- r- r. �_format_s3_pathS s
z,BaseTransferRequestSubmitter._format_s3_path)rI rJ rK ro rq r/ rZ rY re rs rp rr rd r� r� r� r� r{ r� r� r- r- r- r. rb � s$
rb c @ sN e Zd ZejZeZdd� Zdd� Z dd� Z
dd� Z d d
� Z d d � Z
d
d� ZdS )rP c C �
|j dkS )N�upload�rz rj r- r- r. rY ] �
z!UploadRequestSubmitter.can_submitc C sF |� t|j�� | �� r|� t� � | j�dd�r!|� t� � d S d S �Nrk F)rG r �sizer� r r( rF r" r r- r- r. rp ` s �z2UploadRequestSubmitter._add_additional_subscribersc C �. t |j�\}}| �|�}| jj|||||d�S �N)�fileobj�bucket�keyrt ru )r ry � _get_fileinrL r� )r* r_ rt ru r� r� �fileinr- r- r. rr g �
�z/UploadRequestSubmitter._submit_transfer_requestc C � |j S r~ )rx rj r- r- r. r� o � z"UploadRequestSubmitter._get_fileinc C � | j gS r~ )�_warn_if_too_larger� r- r- r. r� r r� z,UploadRequestSubmitter._get_warning_handlersc C sT t |d�r&|jtkr(t|j�}d|tt�f }t||dd�}| j�|� d S d S d S )Nr� z&File %s exceeds s3 upload limit of %s.F)� skip_file) �getattrr� r r rx r r rc r| )r* r_ � file_path�warning_messager� r- r- r. r� u s
����z)UploadRequestSubmitter._warn_if_too_largec C � | � |j�}| �|j�}||fS r~ )r� rx r� ry �r* r_ rx ry r- r- r. r{ � z'UploadRequestSubmitter._format_src_destN)rI rJ rK r �map_put_object_paramsro r
rq rY rp rr r� r� r� r{ r- r- r- r. rP Y s
rP c @ sF e Zd ZejZeZdd� Zdd� Z dd� Z
dd� Z d d
� Z d d � Z
d
S )rQ c C r� )N�downloadr� rj r- r- r. rY � r� z#DownloadRequestSubmitter.can_submitc C sV |� t|j�� |� t� � |� t|j| j�� | j�dd�r)|� t |j
�� d S d S r� ) rG r r� r! r � last_updaterc r( rF r# �
source_clientr r- r- r. rp � s �
��z4DownloadRequestSubmitter._add_additional_subscribersc C r� r� )r rx � _get_fileoutrL r� )r* r_ rt ru r� r� �fileoutr- r- r. rr � r� z1DownloadRequestSubmitter._submit_transfer_requestc C r� r~ )ry rj r- r- r. r� � r� z%DownloadRequestSubmitter._get_fileoutc C s | j | jgS r~ )r� r� r� r- r- r. r� � s z.DownloadRequestSubmitter._get_warning_handlersc C r� r~ )r� rx r� ry r� r- r- r. r{ � r� z)DownloadRequestSubmitter._format_src_destN)rI rJ rK r �map_get_object_paramsro r rq rY rp rr r� r� r{ r- r- r- r. rQ � s rQ c @ s> e Zd ZejZeZdd� Zdd� Z dd� Z
dd� Z d d
� Z d S ) rR c C r� )N�copyr� rj r- r- r. rY � r� z CopyRequestSubmitter.can_submitc C sJ |� t|j�� | �� r|� t� � | j�dd�r#|� t|j�� d S d S r� ) rG r r� r� r r( rF r$ r� r r- r- r. rp � s
��z0CopyRequestSubmitter._add_additional_subscribersc C s@ t |j�\}}t |j�\}}||d�}| jj||||||jd�S )N)�Bucket�Key)r� r� � copy_sourcert ru r� )r ry rx rL r� r� ) r* r_ rt ru r� r� �
source_bucket�
source_keyr� r- r- r. rr � s
�z-CopyRequestSubmitter._submit_transfer_requestc C r� r~ )r� r� r- r- r. r� � r� z*CopyRequestSubmitter._get_warning_handlersc C s | � |j�}| � |j�}||fS r~ )r� rx ry r� r- r- r. r{ � r� z%CopyRequestSubmitter._format_src_destN)
rI rJ rK r �map_copy_object_paramsro r rq rY rp rr r� r{ r- r- r- r. rR � s
rR c @ �0 e Zd ZeZdd� Zdd� Zdd� Zdd� Zd S )
rN c C � |j dko
| j�d�S )Nr� rD �rz r( rF rj r- r- r. rY � �
�z'UploadStreamRequestSubmitter.can_submitc C s0 | j �dd �}|d ur|�tt|��� d S d S )N�
expected_size)r( rF rG r �int)r* ru r_ r� r- r- r. rp � s �z8UploadStreamRequestSubmitter._add_additional_subscribersc C s t � }t|�S r~ )r% r )r* r_ � binary_stdinr- r- r. r� � s z(UploadStreamRequestSubmitter._get_fileinc C � dS �N�-r- r� r- r- r. r� � r� z/UploadStreamRequestSubmitter._format_local_pathN) rI rJ rK r
rq rY rp r� r� r- r- r- r. rN � s rN c @ r� )
rO c C r� )Nr� rD r� rj r- r- r. rY � r� z)DownloadStreamRequestSubmitter.can_submitc C r} r~ r- r r- r- r. rp � r� z:DownloadStreamRequestSubmitter._add_additional_subscribersc C s t � S r~ r rj r- r- r. r� � r� z+DownloadStreamRequestSubmitter._get_fileoutc C r� r� r- r� r- r- r. r� � r� z1DownloadStreamRequestSubmitter._format_local_pathN) rI rJ rK r rq rY rp r� r� r- r- r- r. rO � s rO c @ s. e Zd ZejZeZdd� Zdd� Z dd� Z
dS )rS c C � |j dko |jdkS )N�delete�s3�rz �src_typerj r- r- r. rY � �
�z!DeleteRequestSubmitter.can_submitc C s" t |j�\}}| jj||||d�S )N)r� r� rt ru )r rx rL r� )r* r_ rt ru r� r� r- r- r. rr � s
�z/DeleteRequestSubmitter._submit_transfer_requestc C � | � |j�d fS r~ )r� rx rj r- r- r. r{ � z'DeleteRequestSubmitter._format_src_destN) rI rJ rK r �map_delete_object_paramsro r rq rY rr r{ r- r- r- r. rS � s rS c @ s, e Zd ZdZdZdd� Zdd� Zdd� ZdS )rT Nc C r� )Nr� �localr� rj r- r- r. rY
r� z&LocalDeleteRequestSubmitter.can_submitc
C s� | � |�\}}d||d�}zDz"| j�tdddi|��� t�|j� | j�tdi |��� W W dS tyQ } z| j�t dd|i|��� W Y d }~W dS d }~ww Y dS )Nr� rw �total_transfer_sizer � exceptionTr- )
r{ rc r| r r� �removerx r � Exceptionr )r* r_ rt ru rx ry rv �er- r- r. rr s,
� �� ���� z4LocalDeleteRequestSubmitter._submit_transfer_requestc C r� r~ )r� rx rj r- r- r. r{ - r� z,LocalDeleteRequestSubmitter._format_src_dest)rI rJ rK ro rq rY rr r{ r- r- r- r. rT s rT )8�loggingr� �s3transfer.managerr � awscli.customizations.s3.utilsr r r r r r �'awscli.customizations.s3.transferconfigr � awscli.customizations.s3.resultsr
r r r
r r r r r r r r r r r r r r r r r r r! r" r# r$ �
awscli.compatr% � getLoggerrI r3 �objectr& r8 rb rP rQ rR rN rO rS rT r- r- r- r. �<module> sX
B? ",$"