i*\ddlZGddZGddZGddZGdd Zy) Nc$eZdZdZddZdZdZy) RegexBuilderaBuilds regex using arguments passed into a pattern template. Builds a regex object for which the pattern is made from an argument passed into a template. If more than one argument is passed (iterable), each pattern is joined by "|" (regex alternation 'or') to create a single pattern. Args: pattern_args (iterable): String element(s) to be each passed to ``pattern_func`` to create a regex pattern. Each element is ``re.escape``'d before being passed. pattern_func (callable): A 'template' function that should take a string and return a string. It should take an element of ``pattern_args`` and return a valid regex pattern group string. flags: ``re`` flag(s) to compile with the regex. Example: To create a simple regex that matches on the characters "a", "b", or "c", followed by a period:: >>> rb = RegexBuilder('abc', lambda x: "{}\.".format(x)) Looking at ``rb.regex`` we get the following compiled regex:: >>> print(rb.regex) 'a\.|b\.|c\.' The above is fairly simple, but this class can help in writing more complex repetitive regex, making them more readable and easier to create by using existing data structures. Example: To match the character following the words "lorem", "ipsum", "meili" or "koda":: >>> words = ['lorem', 'ipsum', 'meili', 'koda'] >>> rb = RegexBuilder(words, lambda x: "(?<={}).".format(x)) Looking at ``rb.regex`` we get the following compiled regex:: >>> print(rb.regex) '(?<=lorem).|(?<=ipsum).|(?<=meili).|(?<=koda).' cX||_||_||_|j|_yN) pattern_args pattern_funcflags_compileregex)selfrrr s Q/var/www/auto_recruiter/arenv/lib/python3.12/site-packages/gtts/tokenizer/core.py__init__zRegexBuilder.__init__3s((( ]]_ cg}|jD]9}tj|}|j|}|j |;dj |}tj ||jS)N|)rreescaperappendjoincompiler )r altsargaltpatterns r r zRegexBuilder._compile;si$$ C))C.C##C(C KK   ((4.zz'4::..rc,t|jSr)strr r s r __repr__zRegexBuilder.__repr__Es4::rNr)__name__ __module__ __qualname____doc__rr rrr rrs+Z%/rrc$eZdZdZddZdZdZy)PreProcessorRegexaRegex-based substitution text pre-processor. Runs a series of regex substitutions (``re.sub``) from each ``regex`` of a :class:`gtts.tokenizer.core.RegexBuilder` with an extra ``repl`` replacement parameter. Args: search_args (iterable): String element(s) to be each passed to ``search_func`` to create a regex pattern. Each element is ``re.escape``'d before being passed. search_func (callable): A 'template' function that should take a string and return a string. It should take an element of ``search_args`` and return a valid regex search pattern string. repl (string): The common replacement passed to the ``sub`` method for each ``regex``. Can be a raw string (the case of a regex backreference, for example) flags: ``re`` flag(s) to compile with each `regex`. Example: Add "!" after the words "lorem" or "ipsum", while ignoring case:: >>> import re >>> words = ['lorem', 'ipsum'] >>> pp = PreProcessorRegex(words, ... lambda x: "({})".format(x), r'\\1!', ... re.IGNORECASE) In this case, the regex is a group and the replacement uses its backreference ``\\1`` (as a raw string). Looking at ``pp`` we get the following list of search/replacement pairs:: >>> print(pp) (re.compile('(lorem)', re.IGNORECASE), repl='\1!'), (re.compile('(ipsum)', re.IGNORECASE), repl='\1!') It can then be run on any string of text:: >>> pp.run("LOREM ipSuM") "LOREM! ipSuM!" See :mod:`gtts.tokenizer.pre_processors` for more examples. c||_g|_|D]5}t|g||}|jj|j7yr)replregexesrrr )r search_args search_funcr(r rrbs r rzPreProcessorRegex.__init__vsG   *Cse[%8B LL   ) *rc`|jD]}|j|j|} |S)zRun each regex substitution on ``text``. Args: text (string): the input text. Returns: string: text after all substitutions have been sequentially applied. )r)subr()r textr s r runzPreProcessorRegex.runs0\\ .E99TYY-D . rcg}|jD]-}|jdj||j/dj |S)Nz({}, repl='{}'), )r)rformatr(r)r subs_strsrs r rzPreProcessorRegex.__repr__sK  EA   .55aC D Eyy##rNrr r!r"r#rr0rr$rr r&r&Is*X* $rr&c$eZdZdZddZdZdZy)PreProcessorSuba@Simple substitution text preprocessor. Performs string-for-string substitution from list a find/replace pairs. It abstracts :class:`gtts.tokenizer.core.PreProcessorRegex` with a default simple substitution regex. Args: sub_pairs (list): A list of tuples of the style ``(, )`` ignore_case (bool): Ignore case during search. Defaults to ``True``. Example: Replace all occurrences of "Mac" to "PC" and "Firefox" to "Chrome":: >>> sub_pairs = [('Mac', 'PC'), ('Firefox', 'Chrome')] >>> pp = PreProcessorSub(sub_pairs) Looking at the ``pp``, we get the following list of search (regex)/replacement pairs:: >>> print(pp) (re.compile('Mac', re.IGNORECASE), repl='PC'), (re.compile('Firefox', re.IGNORECASE), repl='Chrome') It can then be run on any string of text:: >>> pp.run("I use firefox on my mac") "I use Chrome on my PC" See :mod:`gtts.tokenizer.pre_processors` for more examples. cd}|rtjnd}g|_|D]1}|\}}t|g|||}|jj |3y)Nc$dj|S)Nz{})r3)xs r r+z-PreProcessorSub.__init__..search_funcs<<? "rr)rIpre_processorsr&r) r sub_pairs ignore_caser+r sub_pairrr(pps r rzPreProcessorSub.__init__s^ #$!! +H$MGT"G9k4GB    & &r * +rcJ|jD]}|j|}|S)zRun each substitution on ``text``. Args: text (string): the input text. Returns: string: text after all substitutions have been sequentially applied. )r=r0)r r/rAs r r0zPreProcessorSub.runs+%% B66$>> import re, RegexBuilder >>> >>> def case1(): ... return re.compile("\,") >>> >>> def case2(): ... return RegexBuilder('abc', lambda x: "{}\.".format(x)).regex >>> >>> t = Tokenizer([case1, case2]) Looking at ``case1().pattern``, we get:: >>> print(case1().pattern) '\\,' Looking at ``case2().pattern``, we get:: >>> print(case2().pattern) 'a\\.|b\\.|c\\.' Finally, looking at ``t``, we get them combined:: >>> print(t) 're.compile('\\,|a\\.|b\\.|c\\.', re.IGNORECASE) from: [, ]' It can then be run on any string of text:: >>> t.run("Hello, my name is Linda a. Call me Lin, b. I'm your friend") ['Hello', ' my name is Linda ', ' Call me Lin', ' ', " I'm your friend"] c||_||_ |j|_y#tt f$r}t dt |zd}~wwxYw)Nz`Tokenizer() expects a list of functions returning regular expression objects (i.e. re.compile). ) regex_funcsr _combine_regex total_regex TypeErrorAttributeErrorr)r rGr es r rzTokenizer.__init__"s\&  #224D >* ACFq6J  s&AA  Acg}|jD]}|j|djd|D}tj||j S)Nrc34K|]}|jywr)r).0rs r z+Tokenizer._combine_regex..4s733;;7s)rGrrrrr )r rfuncrs r rHzTokenizer._combine_regex/sU$$ D KK  ((7$77zz'4::..rc8|jj|S)zTokenize `text`. Args: text (string): the input text to tokenize. Returns: list: A list of strings (token) split according to the tokenizer cases. )rIsplit)r r/s r r0z Tokenizer.run7s%%d++rc^t|jdzt|jzS)Nz from: )rrIrGrs r rzTokenizer.__repr__Cs(4##$y03t7G7G3HHHrN) r r!r"r#r IGNORECASErrHr0rr$rr rErEs'HT+--- / ,IrrE)rrr&r8rEr$rr rVs= AAHI$I$X?B?BDmImIr