File: //lib/python3/dist-packages/awscli/customizations/s3/__pycache__/filegenerator.cpython-310.pyc
o
.&�a�> � @ s� d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZ d dl m
Z
m Z d dl m Z m
Z
mZmZ d dlmZ d dlmZ eZd d
� Zd d � ZG d
d� de�ZG dd� de�ZG dd� de�ZdS )� N)�parse)�tzlocal)� ClientError)�find_bucket_key�
get_file_stat)� BucketLister�create_warning�find_dest_path_comp_key�
EPOCH_TIME)�six)�queuec C sH t �| �j}t�|�r
dS t�|�rdS t�|�rdS t�|�r"dS dS )z�
This function checks to see if a special file. It checks if the
file is a character special device, block special device, FIFO, or
socket.
TF)�os�stat�st_mode�S_ISCHR�S_ISBLK�S_ISFIFO�S_ISSOCK)�path�mode� r �H/usr/lib/python3/dist-packages/awscli/customizations/s3/filegenerator.py�is_special_file s
r c C s� t j�| �rzt �| � W dS ttfy Y dS w zt| d��
}W d � W dS 1 s.w Y W dS ttfyA Y dS w )z�
This function checks to see if a file or a directory can be read.
This is tested by performing an operation that requires read access
on the file or the directory.
F�rNT)r
r �isdir�listdir�OSError�IOError�_open)r �fdr r r � is_readable4 s" �� ����r c s, e Zd ZdZde�� Z� fdd�Z� ZS )�FileDecodingErrorz1Raised when there was an issue decoding the file.z�Please check your locale settings. The filename was decoded as: %s
On posix platforms, check the LC_CTYPE environment variable.c s< || _ || _dt| j�| j | jf | _tt| ��| j� d S )NzJThere was an error trying to decode the the file %s in directory "%s".
%s)� directory� file_name�repr�ADVICE�
error_message�superr! �__init__)�selfr" �filename�� __class__r r r( S s ���zFileDecodingError.__init__) �__name__�
__module__� __qualname__�__doc__�sys�getfilesystemencodingr% r( �
__classcell__r r r+ r r! J s ��r! c @ s e Zd Z ddd�ZdS )�FileStatNc
C s: || _ || _|| _|| _|| _|| _|| _|| _| | _d S �N) �src�dest� compare_key�size� last_update�src_type� dest_type�operation_name�
response_data)
r) r6 r7 r8 r9 r: r; r< r= r> r r r r( ` s
zFileStat.__init__)NNNNNNNN)r- r. r/ r( r r r r r4 _ s
�r4 c @ sv e Zd ZdZ d dd�Zdd� Zdd � Zd
d � Zd d
� Zdd� Z dd� Z
dd� Z dd� Z dd� Z
dd� Zdd� ZdS ) �
FileGeneratora}
This is a class the creates a generator to yield files based on information
returned from the ``FileFormat`` class. It is universal in the sense that
it will handle s3 files, local files, local directories, and s3 objects
under the same common prefix. The generator yields corresponding
``FileInfo`` objects to send to a ``Comparator`` or ``S3Handler``.
TNc C sH || _ || _|| _|| _|| _|st�� | _i | _|d ur"|| _d S d S r5 )�_clientr= �follow_symlinks� page_size� result_queuer �Queue�request_parameters)r) �clientr= rA rB rC rE r r r r( v s
�zFileGenerator.__init__c c s� � | j | jd�}|d d }|d d }|d d }|| ||d �}|D ]#\}}t||�\} }
|| |
||| jd�} | �| |� td i | ��V q%dS )
z�
This is the generalized function to yield the ``FileInfo`` objects.
``dir_op`` and ``use_src_name`` flags affect which files are used and
ensure the proper destination paths and compare keys are formed.
)�s3�localr6 r �typer7 �dir_op)r6 r7 r8 r; r< r= Nr )� list_objects�
list_filesr r= �_inject_extra_informationr4 ) r) �files�function_table�sourcer; r< �
file_iterator�src_path�extra_information� dest_pathr8 �file_stat_kwargsr r r �call� s � � �zFileGenerator.callc C s8 |d }|d |d< |d |d< |dkr||d< d S d S )Nr; �Sizer9 � LastModifiedr: rG r> r )r) rU rS r; r r r rM � s �z'FileGenerator._inject_extra_informationc c s � t jjt jjt jj}}}t jt j}}| �|�s}|s+| �|�}|r)|V dS dS ||�} g }
| D ] } | � || �sO||| �} || �rJ| t jj
} |
� | � q3| � |
t j
d� |
D ]$} ||| �} || �rr| �
| |�D ]}
|
V qkqZ| �| �}|r||V qZdS dS )a{
This function yields the appropriate local file or local files
under a directory depending on if the operation is on a directory.
For directories a depth first search is implemented in order to
follow the same sorted pattern as a s3 list objects operation
outputs. It yields the file's source path, size, and last
update
�/N)r
r �joinr �isfile�errorr �should_ignore_file�_safely_get_file_stats�)should_ignore_file_with_decoding_warnings�sep�append�normalize_sortrL )r) r rJ rZ r r[ r\ r �stats�
listdir_names�names�name� file_path�xr r r rL � sB �
� �
�
�
��zFileGenerator.list_filesc C sN zt |�\}}W n ttfy | �|� Y d S w | �||�}|||d�fS )N�rW rX )r r �
ValueError�triggers_warning�_validate_update_time)r) rg r9 r: r r r r^ � s � z$FileGenerator._safely_get_file_statsc C s* |d u rt |ddd�}| j�|� tS |S )Nz?File has an invalid timestamp. Passing epoch time as timestamp.F)r r& � skip_file)r rC �putr
)r) � update_timer �warningr r r rl � s � z#FileGenerator._validate_update_timec s |j � �fdd�d� dS )a
The purpose of this function is to ensure that the same path seperator
is used when sorting. In windows, the path operator is a backslash as
opposed to a forward slash which can lead to differences in sorting
between s3 and a windows machine.
c s | � �� �S r5 )�replace)�item�� character�os_sepr r �<lambda>� s z.FileGenerator.normalize_sort.<locals>.<lambda>)�keyN)�sort)r) re ru rt r rs r rb � s z FileGenerator.normalize_sortc C sN t |tj�st||�}tt|�|j�}| j�|� dS t j
� ||�}| � |�S )a�
We can get a UnicodeDecodeError if we try to listdir(<unicode>) and
can't decode the contents with sys.getfilesystemencoding(). In this
case listdir() returns the bytestring, which means that
join(<unicode>, <str>) could raise a UnicodeDecodeError. When this
happens we warn using a FileDecodingError that provides more
information into what's going on.
T)
�
isinstancer � text_typer! r r$ r& rC rn r
r rZ r] )r) �dirnamer* �decoding_errorrp r r r r r_ � s
�
z7FileGenerator.should_ignore_file_with_decoding_warningsc C sP | j s tj�|�r|�tj�r|dd� }tj�|�r dS | �|�}|r&dS dS )z�
This function checks whether a file should be ignored in the
file generation process. This includes symlinks that are not to be
followed and files that generate warnings.
N���TF)rA r
r r �endswithr` �islinkrk )r) r �warning_triggeredr r r r] s
z FileGenerator.should_ignore_filec C sn t j�|�st|d�}| j�|� dS t|�r$t|d�}| j�|� dS t|�s5t|d�}| j�|� dS dS )a�
This function checks the specific types and properties of a file.
If the file would cause trouble, the function adds a
warning to the result queue to be printed out and returns a boolean
value notify whether the file caused a warning to be generated.
Files that generate warnings are skipped. Currently, this function
checks for files that do not exist and files that the user does
not have read access.
zFile does not exist.TzHFile is character special device, block special device, FIFO, or socket.z File/Directory is not readable.F)r
r �existsr rC rn r r )r) r rp r r r rk s
�
z FileGenerator.triggers_warningc
c s� � t |�\}}|s|r| �|�V dS t| j�}| j�di �}|j||| j|d�D ](}|\}} | d dkrE|�d�rE| j dkrD|| fV q)|sL||krLq)|| fV q)dS )z�
This function yields the appropriate object or objects under a
common prefix depending if the operation is on objects under a
common prefix. It yields the file's source path, size, and last
update.
�
ListObjectsV2)�bucket�prefixrB �
extra_argsrW r rY �deleteN)
r �_list_single_objectr r@ rE �getrK rB r~ r= )
r) �s3_pathrJ r� r� �listerr� rw � source_pathr> r r r rK 2 s& �
�
� �zFileGenerator.list_objectsc
C s� | j dkr |d d d�fS t|�\}}z||d�}|�| j�di �� | jjd
i |��}W n( tyT } z |jd d dks>� |j� � }d| |d d < t|d��d }~ww t
|� d
��|d < t |d �}|�
t� �|d < ||fS )Nr� ri )�Bucket�Key�
HeadObject�Error�Code�404zKey "%s" does not exist�Message�
ContentLengthrW rX r )r= r �updaterE r� r@ � head_objectr �response�copy�int�popr �
astimezoner )r) r� r� rw �paramsr� �er: r r r r� T s&
�� z!FileGenerator._list_single_object)TNNN)r- r. r/ r0 r( rV rM rL r^ rl rb r_ r] rk rK r� r r r r r? n s
�
4 "r? ) r
r1 r �dateutil.parserr � dateutil.tzr �botocore.exceptionsr � awscli.customizations.s3.utilsr r r r r r
�
awscli.compatr r �openr r r � Exceptionr! �objectr4 r? r r r r �<module> s