HEX
Server: Apache
System: Linux pdx1-shared-a1-38 6.6.104-grsec-jammy+ #3 SMP Tue Sep 16 00:28:11 UTC 2025 x86_64
User: mmickelson (3396398)
PHP: 8.1.31
Disabled: NONE
$ pwd: /lib/python3/dist-packages/boto/gs/__pycache__/
Upload Files
File: //lib/python3/dist-packages/boto/gs/__pycache__/resumable_upload_handler.cpython-310.pyc
o

ckF[�z�@s�ddlZddlZddlZddlZddlZddlZddlZddlZddlm	Z	ddl
mZmZddl
mZddlmZddlmZddlmZddlmZ	Gd	d
�d
e�ZdS)�N)�md5)�config�	UserAgent)�AWSAuthConnection)�InvalidUriError)�ResumableTransferDisposition)�ResumableUploadException)�KeyFilec@s�eZdZdZejeejej	fZ
dZd*dd�Zdd�Z
dd	�Zd
d�Zdd
�Zdd�Zdd�Zd+dd�Zdd�Zdd�Zd,dd�Zdd�Zdd�Zdd �Zd!d"�Z	$d-d%d&�Zd.d(d)�ZdS)/�ResumableUploadHandleri )r���NcCs.||_||_d|_d|_|r|��d|_dS)a

        Constructor. Instantiate once for each uploaded file.

        :type tracker_file_name: string
        :param tracker_file_name: optional file name to save tracker URI.
            If supplied and the current process fails the upload, it can be
            retried in a new process. If called with an existing file containing
            a valid tracker URI, we'll resume the upload from this URI; else
            we'll start a new resumable upload (and write the URI to this
            tracker file).

        :type num_retries: int
        :param num_retries: the number of times we'll re-try a resumable upload
            making no progress. (Count resets every time we get progress, so
            upload can span many more than this number of retries.)
        rN)�tracker_file_name�num_retries�server_has_bytes�tracker_uri�_load_tracker_uri_from_file�upload_start_point)�selfrr
�r�B/usr/lib/python3/dist-packages/boto/gs/resumable_upload_handler.py�__init__@s
zResumableUploadHandler.__init__c
Cs�d}z[zt|jd�}|����}|�|�Wn<ty8}z|jtjkr.td|j|j	f�WYd}~nd}~wt
yR}ztd||jf�WYd}~nd}~wwW|r\|��dSdS|re|��ww)N�rzHCouldn't read URI tracker file (%s): %s. Restarting upload from scratch.zXInvalid tracker URI (%s) found in URI tracker file (%s). Restarting upload from scratch.)�openr�readline�strip�_set_tracker_uri�IOError�errno�ENOENT�print�strerrorr�close)r�f�uri�errrr[s.
�������
�z2ResumableUploadHandler._load_tracker_uri_from_filec
Cs�|jsdSd}z+t�t�|jtjtjBd�d��}|�|j�Wd�WdS1s+wYWdStyJ}zt	d|j|j
ftj��d}~ww)zM
        Saves URI to tracker file if one was passed to constructor.
        Ni��wz�Couldn't write URI tracker file (%s): %s.
This can happenif you're using an incorrectly configured upload tool
(e.g., gsutil configured to save tracker files to an unwritable directory))
r�os�fdopenr�O_WRONLY�O_CREAT�writerrrrr�ABORT)rr!r#rrr�_save_tracker_uri_to_filess*��&�
����z0ResumableUploadHandler._save_tracker_uri_to_filecCsTt�|�}|j��dvs|jstd|��||_|j|_d|j|jf|_	d|_
dS)z�
        Called when we start a new resumable upload or get a new tracker
        URI for the upload. Saves URI and resets upload state.

        Raises InvalidUriError if URI is syntactically invalid.
        )�http�httpszInvalid tracker URI (%s)z%s?%srN)�urlparse�scheme�lower�netlocrr�tracker_uri_host�path�query�tracker_uri_pathr)rr"�parse_resultrrrr�s
��
z'ResumableUploadHandler._set_tracker_uricCs|jS)zX
        Returns upload tracker URI, or None if the upload has not yet started.
        )r�rrrr�get_tracker_uri�sz&ResumableUploadHandler.get_tracker_uricCs6d}|jr||jvr|j|j�|�t|�d�SdS)zt
        Returns the upload ID for the resumable upload, or None if the upload
        has not yet started.
        z?upload_id=N)r�index�len)r�delimrrr�
get_upload_id�s
z$ResumableUploadHandler.get_upload_idcCs,|jrtj�|j�rt�|j�dSdSdS�N)rr%r3�exists�unlinkr7rrr�_remove_tracker_file�s
��z+ResumableUploadHandler._remove_tracker_file�*cCsd||fS)Nzbytes %s/%sr)r�
range_spec�length_specrrr�_build_content_range_header�sz2ResumableUploadHandler._build_content_range_headercCs8i}|�d|�|d<d|d<tj|d|j|j||jd�S)a~
        Queries server to find out state of given upload.

        Note that this method really just makes special case use of the
        fact that the upload server always returns the current start/end
        state whenever a PUT doesn't complete.

        Returns HTTP response from sending request.

        Raises ResumableUploadException if problem querying server.
        rA�
Content-Range�0�Content-Length�PUT�r3�	auth_path�headers�host)rDr�make_requestr5r2)r�conn�file_length�put_headersrrr�_query_server_state�s
��z*ResumableUploadHandler._query_server_statec	Cs�|�||�}|jdkrd|dfS|jdkrtd|jtj��d}|�d�}|rAt�d|�}|r@t|�	d��}t|�	d	��}d
}n|j
S|sRtdt|���tj��|j
dkr_td||f�||fS)
a+
        Queries server to find out what bytes it currently has.

        Returns (server_start, server_end), where the values are inclusive.
        For example, (0, 2) would mean that the server has bytes 0, 1, *and* 2.

        Raises ResumableUploadException if problem querying server.
        ��r�i4z1Got non-308 response (%s) from server state queryF�rangezbytes=(\d+)-(\d+)�Tz6Couldn't parse upload server state query response (%s)zServer has: Range: %d - %d.)rQ�statusrr�
START_OVER�	getheader�re�search�long�group�SERVER_HAS_NOTHING�str�
getheaders�debugr)	rrNrO�resp�got_valid_responserB�m�server_start�
server_endrrr�_query_server_pos�s<	

��
�
��
z(ResumableUploadHandler._query_server_posc	Cs�|jj}|jdkr
td�d|_i}|D]}|��dkr"tdtj��||||<qd||j	j
<|�d|jj|j|�}|�
�}|jdvrLtd	|jtj��|jd
kr_|jdkr_td|jtj��|�d
�}|sntd|tj��|�|�|��dS)zn
        Starts a new resumable upload.

        Raises ResumableUploadException if any errors occur.
        rSzStarting new resumable upload.rzcontent-lengthz5Attempt to specify Content-Length header (disallowed)�start�POST)���zEGot status %d from attempt to start resumable upload. Will wait/retryrR��z>Got status %d from attempt to start resumable upload. Aborting�LocationzINo resumable tracker URI found in resumable initiation POST response (%s)N)�bucket�
connectionr`rrr0rrr*�provider�resumable_upload_headerrM�name�readrV�WAIT_BEFORE_RETRYrXrr+)	r�keyrKrN�post_headers�kra�bodyrrrr�_start_new_resumable_uploadsR
��
����
��
z2ResumableUploadHandler._start_new_resumable_uploadc	Cs
|�|j�}	|r&|dkr||j|d}
n	|dkrd}
nd}
d}|||�|s+i}n|��}|rL||kr<|�d|�}
n|�d||df|�}
|
|d<t||�|d<tj|d	|jd
||jd�}|�	d	|j
�|D]
}|�|||�qj|��|�
d�|	r�|�|	�|jD]
}|j|�|	�q�|t|	�7}|r�|d7}||
ks�|
dkr�|||�d}|�|j�}	|	s�|�
|j�|r�|||�||kr�td||ftj��|��}|�
|j�|jd
kr�|�d�|�d�|�d�fS|jdvr�tj}ntj}td|j|jf|��)z�
        Makes one attempt to upload file bytes, using an existing resumable
        upload connection.

        Returns (etag, generation, metageneration) from server upon success.

        Raises ResumableUploadException if any problems occur.
        rUrrrAz%d-%drSrErGrHNrIz<File changed during upload: EOF at %d bytes of %d byte file.rR�etagzx-goog-generationzx-goog-metageneration)i�rirjz1Got response code %d while attempting upload (%s))rr�BUFFER_SIZE�copyrDr^r�build_base_http_requestr5r2�
putrequestr3�	putheader�
endheaders�set_debuglevel�send�	digesters�updater:r`rrr*�getresponserVrXrs�reason)rrN�	http_conn�fprO�total_bytes_uploaded�cb�num_cbrK�buf�cb_count�irP�range_header�http_requestrv�algra�dispositionrrr�_upload_file_bytes<s�

��
�



�
��
�

��z)ResumableUploadHandler._upload_file_bytesc
Cs�|j\}}|jj}	|jrzN|�|	|�\}}||_|rPtd�|�d�|d}
|
rP|�t	|j
|
��}|s:tdtj
��|jD]
}|j|�|�q=|
t|�8}
|
s)|	jdkrYtd�Wn*ty~}
z|	jdkrntd|
j�|�||�WYd}
~
nd}
~
ww|�||�|jdur�||_|d}||kr�|�|�|jj}	|	�|j|	j|	j�}|�|	j�z/z|�|	|||||||�WW|��Sttjfy�|�|	|�}|jdkr�td	tj ���w|��w)
z�
        Attempts a resumable upload.

        Returns (etag, generation, metageneration) from server upon success.

        Raises ResumableUploadException if any problems occur.
        z-Catching up hash digest(s) for resumed uploadrrSz�Hit end of file during resumable upload hash catchup. This should not happen under
normal circumstances, as it indicates the server has more bytes of this transfer
than the current file size. Restarting upload.zResuming transfer.zUnable to resume transfer (%s).Ni�z�Got 400 response from server state query after failed resumable upload attempt. This can happen for various reasons, including specifying an invalid request (e.g., an invalid canned ACL) or if the file size changed between upload attempts)!r]rmrnrrfrr�seekrr�min�
BufferSizerrrWr�r�r:r`�messagerxr�new_http_connectionr2�port�	is_securer�r�r �socket�errorrQrVr*)rrtr�rOrKr�r�rdrerN�bytes_to_go�chunkr�r#r�r�rarrr�_attempt_resumable_upload�sr
	
�
�
�
�
��

��
�
��
z0ResumableUploadHandler._attempt_resumable_uploadcCsN|jjjdkrtd�|j|�d�kr%|��|��|��t	dt
j��dS)a;
        Checks that etag from server agrees with md5 computed before upload.
        This is important, since the upload could have spanned a number of
        hours and multiple processes (e.g., gsutil runs), and the user could
        change some of the file and not realize they have inconsistent data.
        rSzChecking md5 against etag.z"'z`File changed during upload: md5 signature doesn't match etag (incorrect uploaded object deleted)N)rmrnr`rrr�	open_readr �deleterrr*)rrtryrrr�_check_final_md5�s��z'ResumableUploadHandler._check_final_md5cCsn|jtjkr|dkrtd|j��|jtjkr(|dkr#td|j�|���|dkr5td|j�dSdS)NrSzWCaught non-retryable ResumableUploadException (%s); aborting but retaining tracker filezVCaught non-retryable ResumableUploadException (%s); aborting and removing tracker filez1Caught ResumableUploadException (%s) - will retry)r�r�ABORT_CUR_PROCESSrr�r*r@)rr#r`rrr�!handle_resumable_upload_exceptions$��
��z8ResumableUploadHandler.handle_resumable_upload_exceptionTrcCs~|j|kr	d|_n
|jd7_|r|j|_|j|jkr"tdtj��t��d|j}|dkr8t	d|j|f�t
�|�dS)NrrSzaToo many resumable upload attempts failed without progress. You might try this upload again laterrUzZGot retryable failure (%d progress-less in a row).
Sleeping %3.1f seconds before re-trying)r�progress_less_iterations�digesters_before_attemptr�r
rrr��randomr�time�sleep)r�server_had_bytes_before_attempt�
roll_back_md5r`�sleep_time_secsrrr�track_progress_less_iterations s 
��z5ResumableUploadHandler.track_progress_less_iterations�
c
s�|si}d}||vr||dur||=t|d<t|t�r"|��j}n|�dtj�|��}|�d�|j	j
j}	�dur?dti�t
�fdd��pHiD���_�jdurZt�dd	d
��_d�_	�j}
t
�fdd��jD���_z8��||||||�\}�_�_�jD]}�j|��|j|<q������||��j|_|	d
kr�td�WdS�jy�}
z$|	d
kr�td|
���t|
t�r�|
j t j!kr�|j	j
j
�"�WYd}
~
nd}
~
wt#y�}
z��$|
|	�WYd}
~
nd}
~
ww��%|
d|	�q^)a0
        Upload a file to a key into a bucket on GS, using GS resumable upload
        protocol.

        :type key: :class:`boto.s3.key.Key` or subclass
        :param key: The Key object to which data is to be uploaded

        :type fp: file-like object
        :param fp: The file pointer to upload

        :type headers: dict
        :param headers: The headers to pass along with the PUT request

        :type cb: function
        :param cb: a callback function that will be called to report progress on
            the upload.  The callback should accept two integer parameters, the
            first representing the number of bytes that have been successfully
            transmitted to GS, and the second representing the total number of
            bytes that need to be transmitted.

        :type num_cb: int
        :param num_cb: (optional) If a callback is specified with the cb
            parameter, this parameter determines the granularity of the callback
            by defining the maximum number of times the callback will be called
            during the file transfer. Providing a negative integer will cause
            your callback to be called with each buffer read.

        :type hash_algs: dictionary
        :param hash_algs: (optional) Dictionary mapping hash algorithm
            descriptions to corresponding state-ful hashing objects that
            implement update(), digest(), and copy() (e.g. hashlib.md5()).
            Defaults to {'md5': md5()}.

        Raises ResumableUploadException if a problem occurs during the transfer.
        zContent-TypeNz
User-Agentrrc3s�|]
}|�|�fVqdSr=r��.0r�)�	hash_algsrr�	<genexpr>xs�
�z3ResumableUploadHandler.send_file.<locals>.<genexpr>�Botor
�Tc3s"�|]}|�j|��fVqdSr=)r�r{r�r7rrr��s
��
�rSzResumable upload complete.zCaught exception (%s))&r�
isinstancer	�getkey�sizer�r%�SEEK_END�tellrmrnr`r�dictr�r
r�getintr�rr�r��
generation�metageneration�digest�local_hashesr@r�r�RETRYABLE_EXCEPTIONS�__repr__rr�EPIPEr rr�r�)rrtr�rKr�r�r��CTrOr`r�ryr�r#r)r�rr�	send_file;sh%



�

�
��
�����z ResumableUploadHandler.send_file)NN)rArAr=)Tr)Nr�N)�__name__�
__module__�__qualname__rz�httplib�
HTTPExceptionrr�r��gaierrorr�r]rrr+rr8r<r@rDrQrfrxr�r�r�r�r�r�rrrrr
6s0
�


4:^^
�r
)rr�r%r�rYr�r�r.�hashlibr�botorr�boto.connectionr�boto.exceptionrrr�boto.s3.keyfiler	�objectr
rrrr�<module>s"
@ Telegram