o ChtJ@s@UddlmZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZmZddlmZmZdd lmZmZdd lmZmZmZmZdd lmZdd lm Z dd l!m"Z"dZ#dZ$dZ%dZ&ia'e e(e"fe)d<de e(e"ffddZ*de(de"ddfddZ+de(ddfddZ,eddd dCd e-d!e(de-fd"d#Z.de/fd$d%Z0  dDd e-d&e/d'eede e(e"ffd(d)Z1Gd*d+d+eZ2Gd,d-d-e2Z3Gd.d/d/e2Z4dddde3fd0ee(efd e-d1e-d2e-d3e-d'eedeee e(e/efeddefffd4d5Z5     dEd0ee(efd e-d1e-d2e-d3e-d'eedee(fd6d7Z6      dFd0ee(efd e-d8e-d1e-d2e-d3e-d'eedeee(e ffd9d:Z7     dEd0ee(efd e-d1e-d2e-d3e-d'eedeed;ee(e(e(fffdee(efd2e-d3e-d'eede-f d?d@Z9dHde(ddfdAdBZ:dS)I)unicode_literalsN)open) lru_cache)isabs)DictListOptionalTupleTypeUnion) SplitResulturlsplit)BaseTLDSourceParserRegistry) TldBadUrlTldDomainNotFoundTldImproperlyConfigured TldIOError) project_dir)Result)TriezArtur Barseghyanz2013-2025 Artur Barseghyanz,MPL-1.1 OR GPL-2.0-only OR LGPL-2.1-or-later)BaseMozillaTLDSourceParserget_fldget_tld get_tld_namesget_tld_names_containeris_tldMozillaTLDSourceParser MozillaPublicOnlyTLDSourceParser parse_tldpop_tld_names_container process_urlreset_tld_namesr tld_namesupdate_tld_namesupdate_tld_names_cliupdate_tld_names_containerr$returncCstS)zCGet container of all tld names. :return: :rtype dict: r$r*r*J/home/air/segue/gemini/back/venv/lib/python3.10/site-packages/tld/utils.pyr3srtld_names_local_pathtrie_objcCs |t|<dS)ziUpdate TLD Names container item. :param tld_names_local_path: :param trie_obj: :return: Nr))r,r-r*r*r+r'=s r'cCst|ddS)zTRemove TLD names container item. :param tld_names_local_path: :return: N)r$popr,r*r*r+r!Ksr!T)maxsizetypedF fail_silently parser_uidcCspg}|j}|rt|d}|r|jr||j|dt|StD]\}}|r3|jr3||j|dq"t|S)zUUpdate TLD names. :param fail_silently: :param parser_uid: :return: Nr3)appendrget source_urlr%itemsall)r3r4resultsresults_append parser_clsr*r*r+r%Us        r%cCsdtjdd}|jddddd|jdd d d d d |tjdd}|j}|j}tt ||d S)zCLI wrapper for update_tld_names. Since update_tld_names returns True on success, we need to negate the result to match CLI semantics. zUpdate TLD names) descriptionr4?Nz*UID of the parser to update TLD names for.)nargsdefaulthelpz--fail-silentlyr3F store_truez Fail silently)destrAactionrBr)r4r3) argparseArgumentParser add_argument parse_argssysargvr4r3intr%)parserargsr4r3r*r*r+r&qs(  r& retry_count parser_classcCs|st}|j||dS)aBuild the ``tlds`` list if empty. Recursive. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param retry_count: If greater than 1, we raise an exception in order to avoid infinite loops. :param parser_class: :type fail_silently: bool :type retry_count: int :type parser_class: BaseTLDSourceParser :return: List of TLD names :rtype: obj:`tld.utils.Trie` r3rO)rr)r3rOrPr*r*r+rs rc @s6eZdZe d dededeeee ffddZ dS) rFrr3rOr(c Cs|dkr |rdStt}|j|vr||jdur|SzzWt|jr%|j}nt|j}t|ddd}t}|j}d}|j} |D](} d| vrFd}|rL| sLnd | vrV| d} | d d vr]q>|| |d q>t |j|| WnQt y|j|d |d7}|j||dYWz| WStyYSwty} z|rWYd} ~ Wz| WdStyYdSw| d} ~ wwWz| W|StyY|Swz| WwtyYww)z[Parse. :param fail_silently: :param retry_count: :return: rNrutf8)encodingFz===BEGIN PRIVATE DOMAINS===Tz// xn--r)/ )privater5rQ)rr$ local_pathrr codecs_openraddinclude_privatesplitstripr'closeIOErrorr%r Exception) clsr3rO _tld_namesrX local_filetrietrie_addprivate_sectionr[lineerrr*r*r+rs                   z(BaseMozillaTLDSourceParser.get_tld_namesN)Fr) __name__ __module__ __qualname__ classmethodboolrLrrstrrrr*r*r*r+rsrc@s6eZdZUdZdZeed<dZeed<dZeed<dS) rMozilla TLD source.mozillauidz4https://publicsuffix.org/list/public_suffix_list.datr8zres/effective_tld_names.dat.txtrXN) rirjrk__doc__rqrn__annotations__r8rXr*r*r*r+rs   rc@sBeZdZUdZdZeed<dZeed<dZeed<dZ e ed <d S) rromozilla_public_onlyrqz?https://publicsuffix.org/list/public_suffix_list.dat?publiconlyr8z+res/effective_tld_names_public_only.dat.txtrXFr[N) rirjrkrrrqrnrsr8rXr[rmr*r*r*r+rs   rurl fix_protocol search_publicsearch_privatec Cs|s|stdt||d}t|tsA|r|dsd|}zt|}Wnty@}z|r:dd|fWYd}~S|d}~ww|}|j} | sT|rOdd|fSt|d| } | drb| d} | d} |j } || j} d} d}d}t| }t|d d d D]7}| |}| jdurn+|| jkrn$| j|}|dur| jd }|durn| d 7} |} | jr| }| }q|dus|jr|s|jr|s|jr|rdd|fSt| d ||krd }ntd ||}| ||fS) zProcess URL. :param parser_class: :param url: :param fail_silently: :param fix_protocol: :param search_public: :param search_private: :return: zJEither `search_public` or `search_private` (or both) shall be set to True.)r3rP)z//zhttp://https://ryN)ru.rr*) domain_name)rr isinstancer startswithr ValueErrorhostnamerlowerendswithrstripr\rXrootlenrangechildren exceptionr7leafrWrmax)rur3rvrwrxrPrb parsed_urler} domain_partsr,nodecurrent_length tld_lengthmatchlen_domain_partsipartchild non_zero_ir*r*r+r"s                r"c Ksvd|vrtd|s|rtnt}t||||||d\}}} |dur#dS|dus)J|dkr0| jSd||ddS)a?Extract the first level domain. Extract the top level domain based on the mozilla's effective TLD names dat file. Returns a string. May throw ``TldBadUrl`` or ``TldDomainNotFound`` exceptions if there's bad URL provided or no TLD match found respectively. :param url: URL to get top level domain from. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param fix_protocol: If set to True, missing or wrong protocol is ignored (https is appended instead). :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type url: str | SplitResult :type fail_silently: bool :type fix_protocol: bool :type search_public: bool :type search_private: bool :return: String with top level domain (if ``as_object`` argument is set to False) or a ``tld.utils.Result`` object (if ``as_object`` argument is set to True); returns None on failure. :rtype: str as_objectzH`as_object` argument is deprecated for `get_fld`. Use `get_tld` instead.rur3rvrwrxrPNrrzr)rrrr"rjoin) rur3rvrwrxrPkwargsrrrr*r*r+rs."   rrc Cs|s|rtnt}t||||||d\}}} |durdS|dus!J|s3|dkr*| jSd||dS|dkrHd} d} | jdusDJd| j} nd|d|d} d||d|} d||d} t| | | | dS) aExtract the top level domain. Extract the top level domain based on the mozilla's effective TLD names dat file. Returns a string. May throw ``TldBadUrl`` or ``TldDomainNotFound`` exceptions if there's bad URL provided or no TLD match found respectively. :param url: URL to get top level domain from. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param as_object: If set to True, ``tld.utils.Result`` object is returned, ``domain``, ``suffix`` and ``tld`` properties. :param fix_protocol: If set to True, missing or wrong protocol is ignored (https is appended instead). :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type url: str | SplitResult :type fail_silently: bool :type as_object: bool :type fix_protocol: bool :type search_public: bool :type search_private: bool :return: String with top level domain (if ``as_object`` argument is set to False) or a ``tld.utils.Result`` object (if ``as_object`` argument is set to True); returns None on failure. :rtype: str rNrrzzNo hostname in URLr) subdomaindomaintldr)rrr"rrr) rur3rrvrwrxrPrrrrr_tldr*r*r+rs>%   rNNNc Csd|s|rtnt}zt||d||||d}|durWdS|j|j|jfWStttt fy1YdSw)zParse TLD into parts. :param url: :param fail_silently: :param fix_protocol: :param search_public: :param search_private: :param parser_class: :return: Tuple (tld, domain, subdomain) :rtype: tuple T)r3rrvrwrxrPNr) rrrrrrrrrr)rur3rvrwrxrPobjr*r*r+r &s, r valuecCs,|s|rtnt}t|dd|||d}||kS)agCheck if given URL is tld. :param value: URL to get top level domain from. :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type value: str :type search_public: bool :type search_private: bool :return: :rtype: bool Tr)rrr)rrwrxrPrr*r*r+rUsrcCs|rt|dSiadS)zReset the ``tld_names`` to empty value. If ``tld_names_local_path`` is given, removes specified entry from ``tld_names`` instead. :param tld_names_local_path: :type tld_names_local_path: str :return: N)r!r$r/r*r*r+r#ys r#)FN)FrN)FFTTN)FFFTTN)TTN)N); __future__rrFrJcodecsrrY functoolsros.pathrtypingrrrr r r urllib.parser r baserr exceptionsrrrrhelpersrresultrrdr __author__ __copyright__ __license____all__r$rnrsrr'r!rmr%rLr&rrrrr"rrr rr#r*r*r*r+s,             S " x  G  V  1  $