HEX

File: //usr/lib/python3/dist-packages/duplicity/__pycache__/diffdir.cpython-310.pyc
o

�~�_j�@s�dZddlmZddlmZe��ddlmZddlmZddlm	Z	ddlm
Z
ddlmZdd	lZdd	l
Z
dd
lmZddlmZddlmZdd
lTdd
lTddlmZd	ad	ZGdd�de�Zdd�Zdd�Zdd�Zdd�Zd=dd�Zd=dd�Zd>dd�Zd=dd �Z d!d"�Z!d#d$�Z"d%d&�Z#d'd(�Z$d)d*�Z%Gd+d,�d,e�Z&Gd-d.�d.e�Z'Gd/d0�d0e�Z(Gd1d2�d2e�Z)Gd3d4�d4e)�Z*Gd5d6�d6e)�Z+Gd7d8�d8e)�Z,d9d:�Z-d;d<�Z.d	S)?a
Functions for producing signatures and deltas of directories

Note that the main processes of this module have two parts.  In the
first, the signature or delta is constructed of a ROPath iterator.  In
the second, the ROPath iterator is put into tar block form.
�)�division)�standard_library)�map)�next)�str)�range)�objectN)�
statistics)�util)�config)�*)�progressc@seZdZdS)�DiffDirExceptionN)�__name__�
__module__�__qualname__�rr�3/usr/lib/python3/dist-packages/duplicity/diffdir.pyr6srcCst|�S)z)
    Alias for SigTarBlockIter below
    )�SigTarBlockIter��	path_iterrrr�DirSig:srcCst|t�d��S)a
    Return a tarblock full backup of items in path_iter

    A full backup is just a diff starting from nothing (it may be less
    elegant than using a standard tar file, but we can be sure that it
    will be easy to split up the tar and make the volumes the same
    sizes).
    �)�DirDelta�io�StringIOrrrr�DirFullAs	rcCst|t�d�|�S)zN
    Return full backup like above, but also write signature to sig_outfp
    r)�DirDelta_WriteSigrr)r�	sig_outfprrr�DirFull_WriteSigM�rcCs^t��at|t�rtdd�|D��}nt|�}t||�}tj	s'tj
r+t
j��s+t
|�St|�S)z�
    Produce tarblock diff given dirsig_fileobj_list and pathiter

    dirsig_fileobj_list should either be a tar fileobj or a list of
    those, sorted so the most recent is last.
    cS�g|]}t|��qSr��sigtar2path_iter��.0�xrrr�
<listcomp>^�zDirDelta.<locals>.<listcomp>)r	�StatsDeltaProcess�stats�
isinstance�list�combine_path_itersr#�get_delta_iterr�dry_runr
�tracker�has_collected_evidence�DummyBlockIter�DeltaTarBlockIter)r�dirsig_fileobj_list�sig_iter�
delta_iterrrrrTs

�
rcCsJ|r|��}n|r|��}nJd��t�td�t�|�t�|�f�dS)zA
    Called by get_delta_iter, report error in getting delta
    rz)Both new and sig are None for some reason�Error %s getting delta for %sN)�get_relative_path�log�Warn�_r
�uexc�fsdecode)�exc�new_path�sig_path�sig_tar�index_stringrrr�delta_iter_error_handleris


�rCcs^|sJ��r
|���|j�|��}t�td�||f����fdd�}|��rV|rV|��rV|jdkrVd|_|�d�}t	|�d��}�rLt
|||���}|�t
�||��nLd|_�rutjjdkrmd	t�d
�����_ndd
����_|��s��r�����tr�tj|��7_nt	|�d��}�r�t
|||���}|�|�|�|�|jj|j_|S)zj
    Return new delta_path which, when read, writes sig to sig_fileobj,
    if sigTarFile is not None
    zGetting delta of %s and %scsTt|��_tjjdkrdt�d�����_ndd����_��	�t
�|��dS)zG
        Callback activated when FileWithSignature read to end
        ��
signature/�/�
signature/N)�len�size�sys�version_info�majorr
r=�join�name�addfiler�BytesIO)�
sig_string��index�
sigTarFile�tirr�callback�s

z get_delta_path.<locals>.callback�	signature�diff�rb�snapshotrD�	snapshot/rF�	snapshot/)�get_tarinforS�
get_ropathr9�Debugr;�isreg�difftype�open�FileWithReadCounter�FileWithSignature�getsize�
setfileobj�librsync�	DeltaFilerJrKrLr
r=rMrNrOr*�SourceFileSize�copy_attribs�stat�st_size)r?r@rT�
delta_pathrV�	old_sigfp�newfprrRr�get_delta_pathxsH
�
��

rpcCs�|jdkr'|r|r|�|�t�td�t�|���tjj	t�
|����dS|r0|r0|�|�t�td�t�|���tjjt�
|����dS)zI
    Look at delta path and log delta.  Add stats if new_path is set
    rZzA %szM %sN)
ra�add_new_filer9�Infor;r
r=r8�InfoCode�
diff_file_new�escape�add_changed_file�diff_file_changed)rmr?r*rrr�log_delta_path�s"


��

��rxc		cs��t||�}|rt�d|�}nd}|D]�\}}t�td�|o#t�|j�|o*t�|j�f�|r3|js�|r�|�	�r�|jdkr�t�
td�t�|���tj
jt�|����|r|t|j���}tjjdkrndt�|j�|_n	dd	�|j�|_|�|�t�|�t|j�Vq|r�||kr�t�tt|||f�}|r�t||t�|Vqtjd
7_qt� |�qt�!�|r�|�!�dSdS)a1
    Generate delta iter from new Path iter and sig Path iter.

    For each delta path of regular file type, path.difftype with be
    set to "snapshot", "diff".  sig_iter will probably iterate ROPaths
    instead of Paths.

    If sig_fileobj is not None, will also write signatures to sig_fileobj.
    �wNzComparing %s and %srzD %srD�deleted/sdeleted/rF�)"�
collate2itersr
�make_tarfiler9r_r;�uindexrS�type�existsrrr=r8rs�diff_file_deletedru�ROPathr]rJrKrLrNrMrOr*�add_deleted_file�robust�check_common_errorrCrprx�Errors�add_unchanged_file�close)	�new_iterr5�sig_fileobj�collatedrTr?r@rUrmrrrr.�sP�

�

��

���r.c	cs�t�d|�}d|_|D]n}t�|�}dD]}|�|�r-|t|�d�|dd�}}nqtd|f��tjj	dkrFt
t�|��d��}nt
|�d��}|dsW|dd�}t
|�}||_|d	ksf|d
krw|�|�|��rw|�|�|��|Vq|��dS)zK
    Convert signature tar file object open for reading into path iter
    �rr{)rEr[rzN���zBad tarinfo name %srDrFrWrZ)r
r}�debug�get_tarinfo_name�
startswithrHrrJrKrL�tuple�fsencode�splitr�ra�init_from_tarinfor`rf�extractfiler�)	�	sigtarobj�tf�tarinfo�tiname�prefixrNrarS�ropathrrrr#�s0�

�
r#ccs��d\}}	|s-zt|�}Wnty)|rd|fV|D]}d|fVqYdSw|j}|sTzt|�}WntyP|rC|dfV|D]}|dfVqEYdSw|j}||kr`|dfVd}n||krn||fVd\}}nd|fVd}q)a4
    Collate two iterators.

    The elements yielded by each iterator must be have an index
    variable, and this function returns pairs (elem1, elem2), (elem1,
    None), or (None, elem2) two elements in a pair will have the same
    index, and earlier indicies are yielded later than later indicies.
    �NNr{N�r�
StopIterationrS)�riter1�riter2�relem1�relem2�index1�index2rrrr|sD�	
�
�



�r|c#s|��dd������fdd���fdd�}dd�t�ttt�����D�}|r<|��|dd	V||�|s)dSdS)
a�
    Produce new iterator by combining the iterators in path_iter_list

    This new iter will iterate every path that is in path_iter_list in
    order of increasing index.  If multiple iterators in
    path_iter_list yield paths with the same index, combine_path_iters
    will discard all paths but the one yielded by the last path_iter.

    This is used to combine signature iters, as the output will be a
    full up-to-date signature iter.
    Ncs2zt�|�}Wn
tyYdSw|j||fS)zI
        Represent the next element as a triple, to help sorting
        Nr�)�
iter_index�path)�path_iter_listrr�
get_tripleVs�z&combine_path_iters.<locals>.get_triplecsr|dd}d}|t|�kr7||}|d|kr-�|d�}|r)|||<|d7}n||=ndS|t|�ksdSdS)zK
        Update all elements with path_index same as first element
        rr{N)rH)�triple_list�
path_indexr��
old_triple�
new_triple)r�rr�refresh_triple_list`s
�z/combine_path_iters.<locals>.refresh_triple_listcSsg|]}|r|�qSrrr$rrrr'rr(z&combine_path_iters.<locals>.<listcomp>r�)�reverserr,rrH�sort)r�r�r�r)r�r�rr-Gs�
 �r-cCsVt��at|t�rt|�}nt|�}t|||�}tj	s#tj
r't
j��s't
|�St|�S)a
    Like DirDelta but also write signature into sig_fileobj

    Like DirDelta, sig_infp_list can be a tar fileobj or a sorted list
    of those.  A signature will only be written to newsig_outfp if it
    is different from (the combined) sig_infp_list.
    )r	r)r*r+r,�get_combined_path_iterr#r.rr/r
r0r1r2r3)r�
sig_infp_list�newsig_outfp�
sig_path_iterr6rrrrys	

rcCstdd�|D��S)zI
    Return path iter combining signatures in list of open sig files
    cSr!rr"r$rrrr'�r(z*get_combined_path_iter.<locals>.<listcomp>)r-)r�rrrr��r r�c@s*eZdZdZdd�Zd
dd�Zdd�Zd	S)rczH
    File-like object which also computes amount read as it is read
    cC�
||_dS)zFileWithReadCounter initializerN)�infile)�selfr�rrr�__init__�s
zFileWithReadCounter.__init__r�c
Cszz|j�|�}Wn'ty/}zd}t�td�t�|�t�|jj	�f�WYd}~nd}~wwt
r;t
jt|�7_|S)N�r7)
r��read�IOErrorr9r:r;r
r<r=rNr*rirH)r��length�buf�exrrrr��s
���zFileWithReadCounter.readcCs
|j��S�N)r�r��r�rrrr��s
zFileWithReadCounter.closeN�r�)rrr�__doc__r�r�r�rrrrrc�s

rcc@s.eZdZdZdZdd�Zddd�Zdd	�Zd
S)rdzF
    File-like object which also computes signature as it is read
    i�cGs.|||_|_t�t|��|_d|_||_dS)au
        FileTee initializer

        The object will act like infile, but whenever it is read it
        add infile's data to a SigGenerator object.  When the file has
        been read to the end the callback will be called with the
        calculated signature, and any extra_args if given.

        filelen is used to calculate the block size of the signature.
        N)r�rVrg�SigGenerator�get_block_size�sig_gen�activated_callback�
extra_args)r�r�rV�filelenr�rrrr��s
zFileWithSignature.__init__r�cCs|j�|�}|j�|�|Sr�)r�r�r��update)r�r�r�rrrr��szFileWithSignature.readcCsJ|js |�|j�r	|�|j�s	d|_|j|j��g|j�R�|j��S)Nr{)	r�r��	blocksizerVr��getsigr�r�r�r�rrrr��s�
zFileWithSignature.closeNr�)rrrr�r�r�r�r�rrrrrd�s
rdc@�eZdZdZdd�ZdS)�TarBlockz5
    Contain information to add next file to tar
    cCs||_||_dS)z8
        TarBlock initializer - just store data
        N)rS�data)r�rSr�rrrr��s
zTarBlock.__init__N)rrrr�r�rrrrr���r�c@sreZdZdZdd�Zddd�Zdd�Zd	d
�Zdd�Zd
d�Z	dd�Z
dd�Zdd�Zdd�Z
dd�Zdd�ZdS)�TarBlockIterz�
    A bit like an iterator, yield tar blocks given input iterator

    Unlike an iterator, however, control over the maximum size of a
    tarblock is available by passing an argument to next().  Also the
    get_footer() is available.
    cCs@||_d|_d|_d|_d|_d|_d|_d|_d|_d|_	dS)z*
        TarBlockIter initializer
        rFN)
�
input_iter�offset�process_waiting�process_next_vol_number�previous_index�previous_block�
remember_next�remember_value�remember_block�queued_data)r�r�rrrr��s
zTarBlockIter.__init__r�cCs\t|�|_|jdtjd�}t|jtj�\}}|dkr"dtj|}nd}t|d|||f�S)z<
        Make tarblock out of tarinfo and file data
        �replace)�errors�encodingr�r�s%s%s%s)	rHrI�tobufr�
fsencoding�divmod�tarfile�	BLOCKSIZEr�)r�rSr��	file_data�headers�blocks�	remainder�filler_datarrr�tarinfo2tarblock�s
zTarBlockIter.tarinfo2tarblockcCs|jrJ�tdS)z?
        Turn next value of input_iter into a TarBlock
        N�r��XXX)r��valrrr�processs
zTarBlockIter.processcCs|jsJ�tdS)z�
        Get more tarblocks

        If processing val above would produce more than one TarBlock,
        get the rest of them by calling process_continue.
        Nr�r�rrr�process_continueds
zTarBlockIter.process_continuedcCs�|jdur
|j}d|_|S|jr|��}n|�t|j��}|j}|jt|j	�7_|j
|_||_|j
r>|j
|_||_d|_
|S)z5
        Return next block and update offset
        NF)r�r�r�r�rr�r�r�rHr�rSr�r�r�r�r�)r��result�block_numberrrr�__next__s 

zTarBlockIter.__next__cCsdS)Nirr�rrr�
get_read_size+szTarBlockIter.get_read_sizecC�|j|jfS)zM
        Return index of last tarblock, or None if no previous index
        )r�r�r�rrr�get_previous_index2�zTarBlockIter.get_previous_indexcCr�)zW
        Next time next() is called, we will return data instead of processing
        N)r�)r�r�rrr�queue_index_data8s
zTarBlockIter.queue_index_datacCsd|_d|_d|_dS)zL
        When called, remember the index of the next block iterated
        TN)r�r�r�r�rrr�remember_next_index>s
z TarBlockIter.remember_next_indexcCr�)zD
        Retrieve index remembered with remember_next_index
        )r�r�r�rrr�recall_indexFr�zTarBlockIter.recall_indexcCs&t|jtj�\}}d|_dtj|S)zA
        Return closing string for tarfile, reset offset
        rr�)r�r�r��
RECORDSIZE)r�r�r�rrr�
get_footerLszTarBlockIter.get_footercCs|Sr�rr�rrr�__iter__TszTarBlockIter.__iter__N)r�)rrrr�r�r�r�r�r�r�r�r�r�r�r�r�rrrrr��s


r�c@r�)r2z0
    TarBlockIter that does no file reading
    cCsd|��}|j}|jr
|js|�||�Str,tjd7_tj|��7_t	�
dtj�|�||�S)z7
        Get a fake tarblock from delta_ropath
        r{N)r]rSr�fileobjr�r*�SourceFilesrirer9�Progress)r��delta_ropathrUrSrrrr�\szDummyBlockIter.processN�rrrr�r�rrrrr2Xr�r2c@r�)rzK
    TarBlockIter that yields blocks of a signature tar from path_iter
    cCs�|��}|��r;t�|�d�t|����}|��}|��dd�	|j
�|_tj
jdkr3t�|j�|_|�|j
||�Sdd�	|j
�|_tj
jdkrQt�|j�|_|�|j
|�S)z@
        Return associated signature TarBlock from path
        rYrGrFrDr\)r]r`rg�SigFilerbr�rer�r�rMrSrNrJrKrLr
r=r�)r�r�rU�sfp�sigbufrrrr�ts
�zSigTarBlockIter.processNrrrrrrpr�rc@s(eZdZdZdd�Zdd�Zdd�ZdS)	r3z�
    TarBlockIter that yields parts of a deltatar file

    Unlike SigTarBlockIter, the argument to __init__ is a
    delta_path_iter, so the delta information has already been
    calculated.
    c	Csdd�}|��}|j}|jr|js,|js||d�n|jdks!J�||d�|�||�S|�d�}|�|�\}}trCtj	t
|�7_	|rf|jdkrP||d�n|jdkr[||d�nJd��|�|||�Sd	|j|jf}|d
|_||_||_
||_d|_d|_|�|||�S)
z2
        Get a tarblock from delta_ropath
        cSs,|jdkr|d|_dSd||jf|_dS)z(Add prefix to the name of a tarinfo file�.�/z%s/%sN)rN)r�r�rrr�
add_prefix�s
z-DeltaTarBlockIter.process.<locals>.add_prefix�deletedrZrYrXrzUnknown difftypezmultivol_%s/%sz/1r{r�)r]rSrr�rar�rb�get_data_blockr*�RawDeltaSizerHrN�process_prefix�
process_fp�process_ropathr�r�)	r�r�rrUrS�fpr��
last_block�	full_namerrrr��s8




zDeltaTarBlockIter.processcCs>|��}|�|�}t|�|kr|��rtd��|dfS|dfS)zH
        Return pair (next data block, boolean last data block)
        zError closing fileTF)r�r�rHr�r)r�r
�	read_sizer�rrrr�s
z DeltaTarBlockIter.get_data_blockcCs�|jsJ�|j}|��|j}}d|j|jf|_|�|j�\}}t	r,t	j
t|�7_
|r>d|_d|_d|_d|_d|_n|jd7_|�|||�S)zA
        Return next volume in multivol diff or snapshot
        z%s/%dNr{)
r�rr]rSr
r�rNrrr*r	rHr�)r�r�rUrSr�rrrrr��s
z#DeltaTarBlockIter.process_continuedN)rrrr�r�rr�rrrrr3�s
/
r3cCs�t|t�rt|jd�}nt|tdjf�rt|d�}n|}|D]}|�|j�q|�|���|�	�r4J�t|t�r?|�
�dSdS)z<
    Write block_iter to filename, path, or file object
    �wbrN)r+�PathrbrNr�	__class__�writer�r�r��setdata)�
block_iter�out_objr
�blockrrr�write_block_iter�s

�rcCs(|dkrdSt|d�d}t|tj�S)z�
    Return a reasonable block size to use on files of length file_len

    If the block size is too big, deltas will be bigger than is
    necessary.  If the block size is too small, making deltas and
    patching can take a really long time.
    i�i)�int�minr�
max_blocksize)�file_len�file_blocksizerrrr��sr�r�r�)/r��
__future__r�futurer�install_aliases�builtinsrrrrrrrJ�	duplicityr	r
r�duplicity.path�duplicity.lazyr
r*r0�	ExceptionrrrrrrCrprxr.r#r|r-rr�rcrdr�r�r2rr3rr�rrrr�<module>sT


6
8!,2%{Z