File: //usr/lib/python3/dist-packages/duplicity/__pycache__/backend.cpython-310.pyc
o
��La<j � @ s� d Z ddlmZ e�� ddlmZ ddlmZ ddlmZ ddlZddl Z ddl
Z
ddl Z ddl Z ddl
Z
ddl Z ddlZddlZddlZddlmZ ddlmZ dd lmZ dd
lmZ dd lmZ dd lmZ dd
lmZ ddlm Z ddlm Z ddlm Z ddlm Z ddlm Z ddlm!Z! ddl"Zi a#i a$da%g Z&dd� Z'dd� Z(dd� Z)dd� Z*d d � Z+d d � Z,d d!� Z-G d"d#� d#e�Z.d$d%� Z/d&d'� Z0d/d)d*�Z1G d+d,� d,e�Z2G d-d.� d.e�Z3dS )0zq
Provides a common interface to all backends and certain sevices
intended to be used by the backends themselves.
� )�standard_library)�str)�range)�objectN)�dup_temp)� file_naming)�config)�log)�path)�util)�exception_traceback)�BackendException)�FatalBackendException)�TemporaryLoadException)�ConflictingScheme)�InvalidBackendURL)�UnsupportedBackendSchemec C s� t jjd } | �d�sJ t jj��t�| �}|�� |D ]<}|�d�rX|dd� }d|f }zt|� d}W n tyI dt t
� � d � }Y nw t �
td
�||f t j� q q dS ) z�
Import files in the duplicity/backends directory where
the filename ends in 'backend.py' and ignore the rest.
@rtype: void
@return: void
r zduplicity/backendsz
backend.pyN���zduplicity.backends.%s� SucceededzFailed: � zImport of %s %s)� duplicity�backends�__path__�endswith�os�listdir�sort�
__import__� Exceptionr �sys�exc_infor �Log�_�INFO)r
�files�fn�imp�res� r( �3/usr/lib/python3/dist-packages/duplicity/backend.py�import_backendsP s"
� �r* c C �2 t |�sJ d��| tv rtd| f ��|t| < dS )a�
Register a given backend factory responsible for URL:s with the
given scheme.
The backend must be a callable which, when called with a URL as
the single parameter, returns an object implementing the backend
protocol (i.e., a subclass of Backend).
Typically the callable will be the Backend subclass itself.
This function is not thread-safe and is intended to be called
during module importation or start-up.
� backend factory must be callablez6the scheme %s already has a backend associated with itN)�callable� _backendsr ��scheme�backend_factoryr( r( r) �register_backendk � � r2 c C r+ )a�
Register a given backend factory responsible for URL:s with the
given scheme prefix.
The backend must be a callable which, when called with a URL as
the single parameter, returns an object implementing the backend
protocol (i.e., a subclass of Backend).
Typically the callable will be the Backend subclass itself.
This function is not thread-safe and is intended to be called
during module importation or start-up.
r, z6the prefix %s already has a backend associated with itN)r- �_backend_prefixesr r/ r( r( r) �register_backend_prefix� r3 r5 c C s t �dt �|� d d| �S )zI
strip the prefix from a string e.g. par2+ftp://... -> ftp://...
z(?i)^z\+� )�re�sub�escape)�
url_string�
prefix_schemer( r( r) � strip_prefix� s r< c C s t | �}|jr dS dS )zD
@return Whether the given string looks like a backend URL.
TF)� ParsedUrlr0 )r: �pur( r( r) �is_backend_url� s r? c C s� t | �sdS t| �}|jsJ d��d}tD ]}| �|d �r+t| }tt| |��} nq|du r>|jtvr9t| ��t|j }z||�W S tyX t t
d�t t �
� d � ��w )z�
Find the right backend class instance for the given URL, or return None
if the given string looks like a local path rather than a URL.
Raise InvalidBackendURL if the URL is not a valid URL.
Nz3should be a backend url according to is_backend_url�+z Could not initialize backend: %sr )r? r= r0 r4 �
startswithr< r. r � ImportErrorr
r" r r r )r: r> �factory�prefixr( r( r) �get_backend_object� s( �
�rE c C s& t jrd| } t| �}|rt|�}|S )z�
Instantiate a backend suitable for the given URL, or return None
if the given string looks like a local path rather than a URL.
Raise InvalidBackendURL if the URL is not a valid URL.
zgio+)r �use_giorE �BackendWrapper)r: �objr( r( r) � get_backend� s rI c @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) r= aw
Parse the given URL as a duplicity backend URL.
Returns the data of a parsed URL with the same names as that of
the standard urlparse.urlparse() except that all values have been
resolved rather than deferred. There are no get_* members. This
makes sure that the URL parsing errors are detected early.
Raise InvalidBackendURL on invalid URL's
c C s� || _ ttj_ztj�|�}W n
ty td| ��w z|j| _W n
ty0 td| ��w z|j| _W n
tyD td| ��w z|j | _ | j rUtj�
| j �| _ W n
tyc td| ��w z|j | _ W n
tyw td| ��w | j r�tj�
|j �| _ nd | _ z|j | _ W n
ty� td| ��w | j r�tj�
| j �| _ nd | _ z|j
| _
W n
ty� td| ��w z|j| _W n
ty� td| ��w | jr�tj�| j�| _nd | _i | _d | _z|j| _W n1 t�y | jd v r�n!d
| jv �r t�d | j ��s td |d
| jv t�d
| j�| jf ��Y nw | jtv�rM| j�r9d| j | j | _ d| _d | _
n| j �d��sM| j �d��rMd| j | _ | j�sSd S | jtv �rc| j
�sctd| ��| jtv�ry| j �d��s{td| j|f ��d S d S )NzSyntax error in: %sz Syntax error (scheme) in: %sz Syntax error (netloc) in: %szSyntax error (path) in: %sz Syntax error (username) in: %sz Syntax error (password) in: %sz Syntax error (hostname) in: %szSyntax error (query) in: %s)�rclone�rsyncz::[^:]*$z&Syntax error (port) in: %s A%s B%s C%sz::[^:]+$z//r6 �/zIMissing hostname in a backend URL which requires an explicit hostname: %sz;missing // - relative paths not supported for scheme %s: %s)r: � uses_netloc�urllib�parse�urlparser r r0 �netlocr
�unquote�username�password�hostname�query�parse_qs�
query_args�portr7 �searchrA )�selfr: r> r( r( r) �__init__� s� � � �� � � � � �
����
���zParsedUrl.__init__c C s | j S �N)r: �r[ r( r( r) �geturl\ s zParsedUrl.geturlc C s t j�| �S r] )r �backend�strip_auth_from_urlr^ r( r( r) �
strip_auth_ s zParsedUrl.strip_authN)�__name__�
__module__� __qualname__�__doc__r\ r_ rb r( r( r( r) r= � s
l r= c C s t �dd| �� �}|S )zCReturn a URL from a urlparse object without a username or password.z^([^:/]+://)(.*@)?(.*)z\1\3)r7 r8 r_ )�
parsed_url� clean_urlr( r( r) ra c s ra c C s� t |t�r|jtjjkr|jS t| d�r | �||�p tjjS t|d�rA|jtj kr-tjj
S |jtj kr7tjj S |jtj
krAtjjS tjjS )N� _error_code�errno)�
isinstancer
�coder � ErrorCode�
backend_error�hasattrri rj �EACCES�backend_permission_denied�ENOENT�backend_not_found�ENOSPC�backend_no_space)r` � operation�er( r( r) �_get_code_from_exceptionj s
rx Tc s � �fdd�}|S )Nc s �� �fdd�}|S )Nc
s� t j��d�\}}tdt jd �D ]�}z �| g|�R � W S ty: } z|a|s4|W Y d }~ S |�d }~w ty� } z�|a|sO|W Y d }~ S t� t
d�t � � |t jk}t | j
�|�}|tjjkrmd}|r��r�dd� � d��g� fdd �|D � �}tjt
d
�||jjt�|�f ||d � nt�t
d ��j||jjt�|�f � |s�t|t�r�t�d
t j � nt�t j� t | j
d�r�| j
� � W Y d }~qd }~ww d S )N)TNr z Backtrace of previous error: %sTc S s" t | tj�r t�| j�S t�| �S r] )rk r
�ROPathr r9 �uc_name)�fr( r( r) �
make_filename� s
zFretry.<locals>.outer_retry.<locals>.inner_retry.<locals>.make_filename� c s" g | ]
}|rt |t�r� |��qS r( )rk r ��.0�x�r| r( r) �
<listcomp>� s
��zCretry.<locals>.outer_retry.<locals>.inner_retry.<locals>.<listcomp>z#Giving up after %s attempts. %s: %s)rl �extraz#Attempt of %s Nr. %s failed. %s: %s� �_retry_cleanup) r �are_errors_fatal�getr � num_retriesr �_last_exceptionr r �Debugr" r rx r` rm rs �join�
FatalError� __class__rc r �uexc�Warnrk r �time�sleep�backend_retry_delayro r� ) r[ �args� errors_fatal�errors_default�nrw �at_endrl r� )�fatalr% rv r� r) � inner_retry~ sV �
�
���
�
���z/retry.<locals>.outer_retry.<locals>.inner_retryr( )r% r� �r� rv )r% r) � outer_retry} s 3zretry.<locals>.outer_retryr( )rv r� r� r( r� r) �retryz s 5r� c @ sD e Zd ZdZdd� Z dZdd� Zdd� Zd d
� Z i Z d d � Z
d
S )�BackendzU
See README in backends directory for information on how to write a backend.
c C �
|| _ d S r] )rg )r[ rg r( r( r) r\ � �
zBackend.__init__Tc C sj | j jr| j jS ztjd }W |S ty4 | jr/t�d| j j| j jf �}|tjd< Y |S d}Y |S w )a
Return a password for authentication purposes. The password
will be obtained from the backend URL, the environment, by
asking the user, or by some other method. When applicable, the
result will be cached for future invocations.
� FTP_PASSWORDzPassword for '%s@%s': N) rg rT r �environ�KeyError� use_getpass�getpassrS rU )r[ rT r( r( r) � get_password� s �� ��zBackend.get_passwordc C s | j jr t�dd|�S |S )aD
Remove password from commandline by substituting the password
found in the URL, if any, with a generic place-holder.
This is intended for display purposes only, and it is not
guaranteed that the results are correct (i.e., more than just
the ':password@' may be substituted.
z(:([^\s:/@]+)@([^\s@]+))z :*****@\3)rg rT r7 r8 )r[ � commandliner( r( r) �munge_password� s zBackend.munge_passwordc C sJ ddl m}m} t�|d �|d< ||||dd�}|�� \}}|j||fS )z�
For internal use.
Execute the given command line, interpreted as a shell command.
Returns int Exitcode, string StdOut, string StdErr
r )�Popen�PIPET)�stdout�stderr�universal_newlines)�
subprocessr� r� r �which� communicate�
returncode)r[ r� r� r� �pr� r� r( r( r) �__subprocess_popen� s
zBackend.__subprocess_popenc C s� ddl }t|ttf�rd�|�}|}n|}|�|�}| �|�}t�t d�| � | �
|�\}}}|dkr]z| j |d }|� |� W dS t
tfy\ td|||d | d f ��w |||fS )z�
Execute the given command line with error check.
Returns int Exitcode, string StdOut, string StdErr
Raise a BackendException on failure.
r Nr} zReading results of '%s')r r6 r6 z0Error running '%s': returned %d, with output:
%s�
)�shlexrk �list�tupler� �splitr� r �Infor" �_Backend__subprocess_popen� popen_breaks�indexr� �
ValueErrorr
) r[ r� r� �logstrr� �resultr� r� �ignoresr( r( r) �subprocess_popen� s*
��
zBackend.subprocess_popenN) rc rd re rf r\ r� r� r� r� r� r� r( r( r( r) r� � s r� c @ s e Zd ZdZdd� Zdd� Zeddd�d.d
d ��Zed dd�d.d
d��Zeddd�dd� �Z eddd�dd� �Z
dd� Z dd� Z dd� Z
edd d�d d � �Zedd d�d d � �Zd!d"� Zed#d d�d$d%� �Zed#d d�d&d'� �Zd(d)� Zd.d*d+�Zd.d,d-�Zd S )/rG z^
Represents a generic duplicity backend, capable of storing and
retrieving files.
c C r� r] )r` )r[ r` r( r( r) r\ r� zBackendWrapper.__init__c C s< t | jd�rt�td�t�|� � | j�||� d S t� �)N�_putz
Writing %s) ro r` r r� r" r �fsdecoder� �NotImplementedError�r[ � source_path�remote_filenamer( r( r) �__do_put s zBackendWrapper.__do_put�putT)r� Nc C s |s|� � }| �||� dS )z�
Transfer source_path (Path object) to remote_filename (string)
If remote_filename is None, get the filename from the last
path component of pathname.
N)� get_filename�_BackendWrapper__do_putr� r( r( r) r� $ s zBackendWrapper.put�movec C sN |s|� � }t| jd�r| j�||�dur|�� dS | �||� |�� dS )z�
Move source_path (Path object) to remote_filename (string)
Same as put(), but unlinks source_path in the process. This allows the
local backend to do this more efficiently using rename.
�_moveFN)r� ro r` r� �setdatar� �deleter� r( r( r) r� 0 s zBackendWrapper.mover� c C sF t | jd�r | j�||� |�� |�� s ttd�|j ��dS t� �)z0Retrieve remote_filename and place in local_path�_getz0File %s not found locally after get from backendN) ro r` r� r� �existsr
r" rz r� )r[ r� �
local_pathr( r( r) r� A s ��zBackendWrapper.getr� c s2 dd� � t | jd�r� fdd�| j�� D �S t� �)zL
Return list of filenames (byte strings) present in backend
c S s t | t�r
t�| �S | S )z+Convert a (maybe unicode) filename to bytes)rk r r �fsencode)�filenamer( r( r) �tobytesR s
z$BackendWrapper.list.<locals>.tobytes�_listc s g | ]}� |��qS r( r( r~ �r� r( r) r� _ s z'BackendWrapper.list.<locals>.<listcomp>)ro r` r� r� r^ r( r� r) r� M s
zBackendWrapper.listc C � t | jd�r | j�|�S dS )�s
Manages remote access before downloading files
(unseal data in cold storage for instance)
�pre_process_downloadN)ro r` r� )r[ r� r( r( r) r� c � �z#BackendWrapper.pre_process_downloadc C r� )r� �pre_process_download_batchN)ro r` r� )r[ �remote_filenamesr( r( r) r� k r� z)BackendWrapper.pre_process_download_batchc C sR t |t�rJ �t| jd�r| �|� dS t| jd�r&|D ]}| �|� q dS t� �)zN
Delete each filename in filename_list, in order if possible.
� _delete_list�_deleteN)rk �bytesro r` �_do_delete_list�
_do_deleter� )r[ �
filename_listr� r( r( r) r� s s �zBackendWrapper.deleter� Fc C s4 |r|d d� }| j �|� |dd � }|sd S d S )N�d )r` r� )r[ r� �sublistr( r( r) r� � s
�z BackendWrapper._do_delete_listc C s | j �|� d S r] )r` r� )r[ r� r( r( r) r� � s zBackendWrapper._do_deletec C s� i }t | jd�r| �|�}|du ri }nt | jd�r&|D ] }| �|�||< q |D ] }||vs4|| du r8i ||< dD ]
}|| �|d� q:q(|S )zF
Return metadata about each filename in filename_list
� _query_listN�_query)�size)ro r` �_do_query_list� _do_query�
setdefault)r[ r� �infor� �metadatar( r( r) �
query_info� s
� �zBackendWrapper.query_inforV c C s | j �|�}|d u r i }|S r] )r` r� )r[ r� r� r( r( r) r� � s z BackendWrapper._do_query_listc
C sZ z| j �|�W S ty, } zt| j d|�}|tjjkr&ddiW Y d }~S |�d }~ww )NrV r� ���)r` r� r rx r rm rs )r[ r� rw rl r( r( r) r� � s ��zBackendWrapper._do_queryc C s t | jd�r
| j�� dS dS )z�
Close the backend, releasing any resources held and
invalidating any file objects obtained from the backend.
�_closeN)ro r` r� r^ r( r( r) �close� s �zBackendWrapper.closec C sB |s
t �|�}|s
J d��t�|�}| �||� |�� |�d�S )z�
Return fileobject opened for reading of filename on backend
The file will be downloaded first into a temp file. When the
returned fileobj is closed, the temp file will be deleted.
z Filename not correctly parsed�rb)r rO r �new_tempduppathr� r� �filtered_open_with_delete)r[ r� � parseresults�tdpr( r( r) �get_fileobj_read� s
z BackendWrapper.get_fileobj_readc C s$ | � ||�}|�� }|�� rJ �|S )zL
Retrieve a file from backend, process it, return contents.
)r� �readr� )r[ r� r� �fin�bufr( r( r) �get_data� s zBackendWrapper.get_datar] )rc rd re rf r\ r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r( r( r( r) rG s6
rG )T)4rf �futurer �install_aliases�builtinsr r r rj r r r� r7 r� �urllib.requestrN � urllib.parse� urllib.errorr r r r r r
r �duplicity.utilr �duplicity.errorsr
r r r r r �duplicity.backendsr. r4 r� rM r* r2 r5 r< r? rE rI r= ra rx r� r� rG r( r( r( r) �<module> s^
#~
;_