File: //usr/lib/python3/dist-packages/sphinx/search/__pycache__/__init__.cpython-310.pyc
o
�$�a�D � @ s� U d Z ddlZddlZddlZddlmZ ddlmZ ddlm Z m
Z
m Z m Z m
Z
mZmZmZmZ ddlmZ ddlmZ ddlmZmZ dd lmZ dd
lm Z dd l m Z G d d
� d
�Z ddl m!Z! de"dee" fdd�Z#i dd�dd�de!�dd�dd�d d �d d �d d!�d"d#�d$d%�d&d'�d(d)�d*d+�d,d-�d.d/�d0d1�d2d3�Z$e e"e
f e%d4< G d5d6� d6�Z&e&� Z'G d7d8� d8ej(�Z)G d9d:� d:�Z*dS );z�
sphinx.search
~~~~~~~~~~~~~
Create a full-text search index for offline search.
:copyright: Copyright 2007-2021 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
� N)�
import_module)�path) �IO�Any�Dict�Iterable�List�Optional�Set�Tuple�Type)�nodes)�Node)�addnodes� package_dir)�BuildEnvironment)�
splitter_code)�jsdumpc @ s� e Zd ZU dZdZeed< dZeed< e� Z e
e ed< dZ eed< dZ eed< dZ
e�d �Zd
ed dfd d
�Zd
ed dfdd�Zded ee fdd�Zded efdd�Zded efdd�ZdS )�SearchLanguagea<
This class is the base class for search natural language preprocessors. If
you want to add support for a new language, you should override the methods
of this class.
You should override `lang` class property too (e.g. 'en', 'fr' and so on).
.. attribute:: stopwords
This is a set of stop words of the target language. Default `stopwords`
is empty. This word is used for building index and embedded in JS.
.. attribute:: js_splitter_code
Return splitter function of JavaScript version. The function should be
named as ``splitQuery``. And it should take a string and return list of
strings.
.. versionadded:: 3.0
.. attribute:: js_stemmer_code
Return stemmer class of JavaScript version. This class' name should be
``Stemmer`` and this class must have ``stemWord`` method. This string is
embedded as-is in searchtools.js.
This class is used to preprocess search word which Sphinx HTML readers
type, before searching index. Default implementation does nothing.
N�lang�
language_name� stopwords�js_splitter_code�js_stemmer_rawcodez�
/**
* Dummy stemmer for languages without stemming rules.
*/
var Stemmer = function() {
this.stemWord = function(w) {
return w;
}
}
z(?u)\w+�options�returnc C s || _ | �|� d S �N)r �init��selfr � r �8/usr/lib/python3/dist-packages/sphinx/search/__init__.py�__init__J s zSearchLanguage.__init__c C s dS )zK
Initialize the class with the options the user has given.
Nr r r r r! r N s zSearchLanguage.init�inputc C s | j �|�S )z�
This method splits a sentence into words. Default splitter splits input
at white spaces, which should be enough for most languages except CJK
languages.
)�_word_re�findall)r r# r r r! �splitS s zSearchLanguage.split�wordc C s |S )a�
This method implements stemming algorithm of the Python version.
Default implementation does nothing. You should implement this if the
language has any stemming rules.
This class is used to preprocess search words before registering them in
the search index. The stemming of the Python version and the JS version
(given in the js_stemmer_code attribute) must be compatible.
r �r r' r r r! �stem[ s zSearchLanguage.stemc C sT t |�dkp)t |�dk r dt|d � k odk n p(t|d �dk o(|| jv S )z�
Return true if the target word should be registered in the search index.
This method is called after stemming.
r � iA0 i�0 � )�len�ordr r( r r r! � word_filterh s ,��zSearchLanguage.word_filter)�__name__�
__module__� __qualname__�__doc__r �str�__annotations__r �setr r
r r �js_stemmer_code�re�compiler$ r r" r r r& r) �boolr. r r r r! r s
r )�
SearchEnglish�sourcer c C s4 t � }| �� D ]}|�d�d }|�|�� � q|S )zs
Parse snowball style word list like this:
* http://snowball.tartarus.org/algorithms/finnish/stop.txt
�|r )r5 �
splitlinesr& �update)r; �result�liner r r! �parse_stop_wordy s
rA �daz sphinx.search.da.SearchDanish�dez sphinx.search.de.SearchGerman�en�esz sphinx.search.es.SearchSpanish�fiz sphinx.search.fi.SearchFinnish�frz sphinx.search.fr.SearchFrench�huz sphinx.search.hu.SearchHungarian�itz sphinx.search.it.SearchItalian�jaz sphinx.search.ja.SearchJapanese�nlz sphinx.search.nl.SearchDutch�noz sphinx.search.no.SearchNorwegian�ptz!sphinx.search.pt.SearchPortuguese�roz sphinx.search.ro.SearchRomanian�ruz sphinx.search.ru.SearchRussian�svz sphinx.search.sv.SearchSwedish�trz sphinx.search.tr.SearchTurkish�zhz sphinx.search.zh.SearchChinese� languagesc @ sd e Zd ZdZdZdZdedefdd�Zdedefd d
�Z ded e
dd fd
d�Z d e
defdd�Z d S )�_JavaScriptIndexz�
The search index as JavaScript file that calls a function
on the documentation search object to register the index.
zSearch.setIndex(�)�datar c C s | j t�|� | j S r )�PREFIXr �dumps�SUFFIX)r rV r r r! rX � s z_JavaScriptIndex.dumps�sc C sH |t | j�t | j� � }|r|�| j�r|�| j�s td��t�|�S )Nz invalid data)r, rW rY �
startswith�endswith�
ValueErrorr �loads)r rZ rV r r r! r^ � s
�
z_JavaScriptIndex.loads�fNc C s |� | �|�� d S r )�writerX )r rV r_ r r r! �dump� � z_JavaScriptIndex.dumpc C s | � |�� �S r )r^ �read)r r_ r r r! �load� s z_JavaScriptIndex.load)
r/ r0 r1 r2 rW rY r r3 rX r^ r ra rd r r r r! rT � s rT c sV e Zd ZdZdejdeddf� fdd� Zdej de
fd d
�Z de ddfd d �Z
� ZS )
�
WordCollectorzG
A special visitor that collects words for the `IndexBuilder`.
�documentr r Nc s" t � �|� g | _g | _|| _d S r )�superr" � found_words�found_title_wordsr )r rf r �� __class__r r! r" � s
zWordCollector.__init__�nodec C sD t |tj�r |�d�dkr |�d�}|d u rdS || jjkr dS dS )N�name�keywordsr TF)�
isinstancer �meta�getr )r rl � meta_langr r r! �is_meta_keywords� s
z WordCollector.is_meta_keywordsc C s t |tj�r tj�t |tj�r=d|�dd��� v r:t�dd|� � �}t�dd|�}t�dd|�}| j
� | j �|�� tj�t |tj
�rQ| j
� | j �|� � �� d S t |tj�re| j� | j �|� � �� d S t |tj�r�| �|�r�|d }dd � |�d
�D �}| j
� |� d S d S d S ) N�html�format� z(?is)<style.*?</style>z(?is)<script.*?</script>z<[^<]+?>�contentc S s g | ]}|� � �qS r )�strip)�.0�keywordr r r! �
<listcomp>� � z0WordCollector.dispatch_visit.<locals>.<listcomp>�,)ro r
�comment�SkipNode�rawrq r& r7 �sub�astextrh �extendr �Text�titleri r rp rs )r rl �nodetextrn r r r! �dispatch_visit� s$ �z WordCollector.dispatch_visit)r/ r0 r1 r2 r
rf r r" r rp r9 rs r r� �
__classcell__r r rj r! re � s
re c @ sl e Zd ZdZeed�Zdedede deddf
d d
�Z
d e d e ddfd
d�Z
d e d e ddfdd�Zde eef de eeeeeeeef f fdd�Zde dee eee f e eee f f fdd�Zde ee f fdd�Zdefdd�Zdee ddfdd �Zd ed ed ed ejddf
d!d"�Zde ee f fd#d$�Zdee fd%d&�Z de e fd'd(�Z defd)d*�Z dS )+� IndexBuilderzi
Helper class that creates a search index based on the doctrees
passed to the `feed` method.
)r �pickle�envr r �scoringr Nc C s� || _ i | _i | _i | _i | _i | _i | _i | _t� |�}|d u r/d|v r/t� |�
d�d �}|d u r9t |�| _ n t
|t�rS|�dd�\}}tt|�|�}||�| _ n||�| _ |rwt|d��}|�� �� | _W d � n1 sqw Y nd| _t| _d S )N�_r �.� �rbrv )r� �_titles�
_filenames�_mapping�_title_mapping� _stem_cache� _objtypes� _objnamesrS rq r& r: r ro r3 �rsplit�getattrr �openrc �decode�js_scorer_coder r ) r r� r r r� �
lang_class�module� classname�fpr r r! r" � s2
��
zIndexBuilder.__init__�streamru c s� t |t�r
| j| }|�|�}t |t�r |�d�| jjkr!td��|d � tt � |d ��| _
tt � |d ��| _ dt tt
f dt ttt f f� fdd � }||d
�| _||d �| _d S )
z Reconstruct from frozen data.�
envversionz
old format�docnames� filenames�titles�mappingr c sJ i }| � � D ] \}}t|t�r� | h||< q� fdd�|D �||< q|S )Nc s h | ]}� | �qS r r )ry �i��index2fnr r! � <setcomp>% r| z8IndexBuilder.load.<locals>.load_terms.<locals>.<setcomp>)�itemsro �int)r� �rv�k�vr� r r! �
load_terms s
z%IndexBuilder.load.<locals>.load_terms�terms�
titletermsN)ro r3 �formatsrd �dictrq r� �versionr] �zipr� r� r r r
r� r� )r r� ru �frozenr� r r� r! rd s
* zIndexBuilder.loadc C s( t |t�r
| j| }|�| �� |� dS )z"Dump the frozen index to a stream.N)ro r3 r� ra �freeze)r r� ru r r r! ra , s
zIndexBuilder.dump�fn2indexc C s@ i }| j }| j}t| jj�� �D ]�\}}t|�� �D ]�\}}} }
} } |
|vr'q| dk r,qt�|�}t�|�}|� d�\}
}}|�
|
g �}z||| f }W n/ t y{ t |�}|||| f< |j
�| �}|rr|| t|�|��f||< n|| | f||< Y nw | |kr�d}n
| | d | kr�d}n| }|�||
|| ||f� qq|S )Nr r� rv �-)r� r� �sortedr� �domainsr� � get_objectsrt �escape�
rpartition�
setdefault�KeyErrorr, � object_typesrq r3 �
get_type_name�append)r r� r� �otypes�onames�
domainname�domain�fullname�dispname�type�docname�anchor�prio�prefixr� rm �plist� typeindex�otype� shortanchorr r r! r� 2 sF
�
���
� zIndexBuilder.get_objectsc s| i i f}t || j| jf�D ].\}}|�� D ]%\}}t|�dkr-|\}|� v r,� | ||< qt� fdd�|D ��||< qq
|S )Nr� c s g | ]
}|� v r� | �qS r r )ry �fn�r� r r! r{ ` s z*IndexBuilder.get_terms.<locals>.<listcomp>)r� r� r� r� r, r� )r r� �rvsr� r� r� r� r� r r� r! � get_termsW s � �zIndexBuilder.get_termsc
s� t t� j�� �� \}}� fdd�|D �}dd� t|�D �}� �|�\}}� �|�}dd� � j�� D �}� j} t ||||||| |� j
j d� S )z/Create a usable data structure for serializing.c s g | ]}� j �|��qS r )r� rq )ry r� �r r r! r{ f s z'IndexBuilder.freeze.<locals>.<listcomp>c S s i | ]\}}||�qS r r )ry r� r_ r r r! �
<dictcomp>g s z'IndexBuilder.freeze.<locals>.<dictcomp>c S s& i | ]\}}||d d |d �qS )r �:r� r )ry r� r� r r r! r� k s & ) r� r� r� r� �objects�objtypes�objnamesr� r� ) r� r� r� r� � enumerater� r� r� r� r� r� r� )
r r� r� r� r� r� � title_termsr� r� r� r r� r! r� c s
�zIndexBuilder.freezec C s d| j j| j j f S )Nz
%s (code: %s))r r r� r r r! �labelq rb zIndexBuilder.labelr� c C s| i }i }|D ]}|| j v r| j | ||< | j| ||< q|| _ || _| j�� D ]}|�|� q'| j�� D ]}|�|� q4dS )z-Remove data for all docnames not in the list.N)r� r� r� �values�intersection_updater� )r r� �
new_titles�
new_filenamesr� � wordnamesr r r! �prunet s
� �zIndexBuilder.pruner� �filenamer� �doctreec s� |� j |< |� j|< t|� j�}|�|� dtdtf� fdd� }� jj}|jD ]%}||�} || �r=� j� | t
� �� |� q'||�rL� j� |t
� �� |� q'|j D ]+}||�} || �s`||�r`|} |� j�
| t
� �v }
|| �r{|
s{� j� | t
� �� |� qPdS )z Feed a doctree to the index.r' r c s@ z� j | W S ty � j�| ��� � j | < � j | Y S w r )r� r� r r) �lower)r' r� r r! r) � s �z IndexBuilder.feed.<locals>.stemN)r� r� re r �walkr3 r. ri r� r� r5 �addrh rq r� ) r r� r� r� r� �visitorr) �_filterr' � stemmed_word�already_indexedr r� r! �feed� s,
�
��zIndexBuilder.feedc C s: | j jr | j j}n| j}| �� t�t| j j��| j|d�S )N)� search_language_stemming_code�search_language_stop_words�search_scorer_tool�search_word_splitter_code)r r �get_js_stemmer_coder rX r� r r� )r r r r r! �context_for_searchtool� s
�z#IndexBuilder.context_for_searchtoolc C s" | j jrdd� d| j jfD �S g S )z8Returns a list of non-minified stemmer JS files to copy.c S s g | ]
}t �td d|��qS )�searchznon-minified-js)r �joinr )ry �fnamer r r! r{ � s ��z8IndexBuilder.get_js_stemmer_rawcodes.<locals>.<listcomp>�base-stemmer.js)r r r� r r r! �get_js_stemmer_rawcodes� s
�z$IndexBuilder.get_js_stemmer_rawcodesc C s d S r r r� r r r! �get_js_stemmer_rawcode� s z#IndexBuilder.get_js_stemmer_rawcodec C s� | j jrNt�tdd�}tt�|d��� }|�� }W d � n1 s"w Y tt�|| j j��� }|�� }W d � n1 s@w Y d||| j jf S | j jS )z<Returns JS code that will be inserted into language_data.js.r� z minified-jsr� Nz%s
%s
Stemmer = %sStemmer;) r r r r� r r� rc r r6 )r �js_dir�js_file�base_js� language_jsr r r! r� � s
�
� �z IndexBuilder.get_js_stemmer_code) r/ r0 r1 r2 r r� r� r r3 r r" r r rd ra r� r r r� r� r� r� r r� r
rf r� r� r� r r� r� r r r r! r� � s( � #
�2% !
r� )+r2 rt r� r7 � importlibr �osr �typingr r r r r r r
r r �docutilsr
�docutils.nodesr �sphinxr r �sphinx.environmentr �sphinx.search.jssplitterr � sphinx.utilr r �sphinx.search.enr: r3 rA rS r4 rT �js_index� NodeVisitorre r� r r r r! �<module> sn , \�������� �
� � �
�����,