ݘhP ddlZddlZddlZddlZddlZddlmZmZmZddl m Z ddl m Z ddl m Z ddlmZddlmZddlmZmZmZmZmZmZmZddlZed eZed eZd Zd ed eegeffdZ ddej!ej!dddeeeeefdeededegefdee"d ej#f dZ$GddeZ%dS)N)ArgumentDefaultsHelpFormatterArgumentParserArgumentTypeError)Iterable)copy)Enum)isclassPath)AnyCallableLiteralNewTypeOptionalUnionget_type_hints DataClass DataClassTypect|tr|S|dvrdS|dvrdStd|d)N)yestruety1T)nofalsefn0FzTruthy value expected: got zJ but expected one of yes/no, true/false, t/f, y/n, 1/0 (case insensitive).) isinstanceboollowerr)vs a/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/transformers/hf_argparser.pystring_to_boolr%$sk!Twwyy222t 4 4 4u w! w w w   choicesreturnc&d|DfdS)aN Creates a mapping function from each choices string representation to the actual value. Used to support multiple value types for a single argument. Args: choices (list): List of choices. Returns: Callable[[str], Any]: Mapping function from string representation to actual value for each choice. c.i|]}t||S)str).0choices r$ z-make_choice_type_function..<s ???VS[[&???r&c0||SN)get)arg str_to_choices r$z+make_choice_type_function..=s}((c22r&r+)r'r4s @r$make_choice_type_functionr61s(@?w???M 2 2 2 22r&)aliaseshelpdefaultdefault_factorymetadatar7r8r9r:r;c P|i}|||d<|||d<tjd|||d|S)aArgument helper enabling a concise syntax to create dataclass fields for parsing with `HfArgumentParser`. Example comparing the use of `HfArg` and `dataclasses.field`: ``` @dataclass class Args: regular_arg: str = dataclasses.field(default="Huggingface", metadata={"aliases": ["--example", "-e"], "help": "This syntax could be better!"}) hf_arg: str = HfArg(default="Huggingface", aliases=["--example", "-e"], help="What a nice syntax!") ``` Args: aliases (Union[str, list[str]], optional): Single string or list of strings of aliases to pass on to argparse, e.g. `aliases=["--example", "-e"]`. Defaults to None. help (str, optional): Help string to pass on to argparse that can be displayed with --help. Defaults to None. default (Any, optional): Default value for the argument. If not default or default_factory is specified, the argument is required. Defaults to dataclasses.MISSING. default_factory (Callable[[], Any], optional): The default_factory is a 0-argument function called to initialize a field's value. It is useful to provide default values for mutable types, e.g. lists: `default_factory=list`. Mutually exclusive with `default=`. Defaults to dataclasses.MISSING. metadata (dict, optional): Further metadata to pass on to `dataclasses.field`. Defaults to None. Returns: Field: A `dataclasses.Field` with the desired properties. Nr7r8)r;r9r:r+) dataclassesfield)r7r8r9r:r;kwargss r$HfArgr@@sRH%    khQ` k kdj k kkr&c eZdZUdZeeed<ddeeeeefffd Z e de de j fdZdefd Z dd eed ffdZddeeefded eed ffdZ ddeeejfded eed ffdZ ddeeejfded eed ffdZxZS)HfArgumentParsera This subclass of `argparse.ArgumentParser` uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed) arguments to the parser after initialization and you'll get the output back after parsing as an additional namespace. Optional: To create sub argument groups use the `_argument_group_name` attribute in the dataclass. Args: dataclass_types (`DataClassType` or `Iterable[DataClassType]`, *optional*): Dataclass type, or list of dataclass types for which we will "fill" instances with the parsed args. kwargs (`dict[str, Any]`, *optional*): Passed to `argparse.ArgumentParser()` in the regular way. dataclass_typesNc $|g}nt|ts|g}d|vr t|d<tjdi|t j|r|g}t||_|jD]}| |dS)Nformatter_classr+) r rrsuper__init__r= is_dataclasslistrC_add_dataclass_arguments)selfrCr?dtype __class__s r$rGzHfArgumentParser.__init__s  " OOOX66 0./O F * *(EF$ %""6"""  #O 4 4 0./O#O44) 1 1E  ) )% 0 0 0 0 1 1r&parserr>c d|jg}d|jvr1|d|jdd|j}t |jtrtd| dg}t |tr|g}t|jd|j}|tus1ttdrnt |tjrSt|jjvrPt!|jjdkst d|jjvrt#d |jd t d|jjvr^|jjd tur|jjd n|jjd |_t|jd|j}nwt$|jjvrdt d|jjd r|jjd n|jjd |_t|jd|j}i}|t&us4t |jt rt)|jt*rk|t&ur|jj|d <nd|jD|d <t-|d |d<|jt0jur |j|d<nwd|d<np|jt$us|jt4t$krmt |}t6|d<|jt$us|j>|jt0jur+|jt0jurdn|j}||d<d|d<d|d<nt9|rst)|t:r^|jjd |d<d|d<|jt0jur||d<nq|jt0jurd|d<nX|j|d<|jt0jur |j|d<n0|jt0jur||d<nd|d<|jg||Ri||jdurk|jt$us|jt4t$krDd|d<|jd|jd|jddfd|jd|dSdSdS)Nz--_-zpUnresolved type detected, which should have been done with the help of `typing.get_type_hints` method by defaultr7 __origin__ UnionTypezOnly `Union[X, NoneType]` (i.e., `Optional[X]`) is allowed for `Union` because the argument parser only supports one type per argument. Problem encountered in field 'z'.rr'cg|] }|j Sr+)value)r-xs r$ z;HfArgumentParser._parse_dataclass_field..s$A$A$AQW$A$A$Ar&typer9TrequiredF?nargsconst+z--no_z--no- store_false)actiondest) nameappendreplacer;rr rZr, RuntimeErrorpopgetattrrhasattrtypesrS__args__len ValueErrorr!r issubclassrr6r9r=MISSINGrr%r rIr: add_argument)rNr> long_optionsr?r7 origin_type bool_kwargsr9s r$_parse_dataclass_fieldz'HfArgumentParser._parse_dataclass_fields1 *UZ))* %*      CUZ%7%7S%A%A C C D D D$$&& ej# & & <  **Y++ gs # # iGej, CC %  GE;$?$? J{\a\kDlDl %*---EJ'((A--d5:CV1V1V E6;jEEE Dzz!4447\>G>>>v>>> =D ejD&8&8EJ(SW.s r$rJz)HfArgumentParser._add_dataclass_argumentss 50 1 1 ,,U-GHHFFF *8*?*?JJ   6e666     #g--2UY\]_Y`Y`2`2`!$#c33CBQB3G*H*H!I!I""%""N"""  !'.. 7 7E: #EJ/EJ  ' ' 6 6 6 6  7 7s?&C%%A;C  C%FTr(.c4|s|s|rttjrg}|r#|t |n`|r^ttjrE|t tjdd|rt }||td| |\}}t| | dd} | r| d| Dg} |D]?} | r)| | z } @|| |zn| tjd dz}| |\} } g}|jD]}d t%j|Dfd t| D}D]}t+| ||d i|}||t| jdkr|| |rg|| RS| rt/d | g|RS)a% Parse command-line args into instances of the specified dataclass types. This relies on argparse's `ArgumentParser.parse_known_args`. See the doc at: docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args Args: args: List of strings to parse. The default is taken from sys.argv. (same as argparse.ArgumentParser) return_remaining_strings: If true, also return a list of remaining argument strings. look_for_args_file: If true, will look for a ".args" file with the same base name as the entry point script for this process, and will append its potential content to the command line args. args_filename: If not None, will uses this file instead of the ".args" file specified in the previous argument. args_file_flag: If not None, will look for a file in the command-line args specified with this flag. The flag can be specified multiple times and precedence is determined by the order (last one wins). Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer.abspath - if applicable, an additional namespace for more (non-dataclass backed) arguments added to the parser after initialization. - The potential list of remaining argument strings. (same as argparse.ArgumentParser.parse_known_args) rz.argsrd)rZra)argsrQNc,g|]}t|Sr+r )r-ps r$rYz@HfArgumentParser.parse_args_into_dataclasses..Us&L&L&L1tAww&L&L&Lr&rUc*h|]}|j |jSr+rrcr-rs r$ z?HfArgumentParser.parse_args_into_dataclasses..b!HHHqHAFHHHr&c$i|] \}}|v || Sr+r+r-kr#keyss r$r/z@HfArgumentParser.parse_args_into_dataclasses..cs$LLLtq!!t))a)))r&z?Some specified arguments are not used by the HfArgumentParser: r+)rlr}argvrdr with_suffixrrpr,parse_known_argsvarsr2lstripextendexists read_textsplitrCr=ritemsdelattr__dict__rm)rKrreturn_remaining_stringslook_for_args_file args_filenameargs_file_flag args_filesargs_file_parsercfgcmd_args_file_paths file_args args_file namespaceremaining_argsoutputsrLinputsrobjrs @r$parse_args_into_dataclassesz,HfArgumentParser.parse_args_into_dataclassessJ  V] V/A Vc#(mm VJ J!!$}"5"56666# JCH  J!!$sx{"3"3"?"?"H"HIII N#1#3#3  --n3x-XXX-==4=HH T&*3iimmN4I4I#4N4NPT&U&U#&N%%&L&L8K&L&L&LMMMI' ? ? ##%%?!4!4!6!6!!>>I(,'79t##YRSRTRT=UD$($9$9t$9$D$D! >)  EHHK$6u$=$=HHHDLLLLtI'<'<'>'>LLLF & & 1%%%%%//&//C NN3     y! " "Q & & NN9 % % % # -W-n-- - u !scq!s!stttW;; r&rallow_extra_keysct|}g}|jD]}dtj|Dfd|D}|||di|}|||s!|rtdt|t|S)a< Alternative helper method that does not use `argparse` at all, instead uses a dict and populating the dataclass types. Args: args (`dict`): dict containing config values allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the dict contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. c*h|]}|j |jSr+rrs r$rz.HfArgumentParser.parse_dict..rr&c$i|] \}}|v || Sr+r+rs r$r/z/HfArgumentParser.parse_dict..s$AAAtq!qDyyayyyr&z0Some keys are not used by the HfArgumentParser: r+) setrrCr=rrdifference_updaterdrmsortedtuple) rKrr unused_keysrrLrrrs @r$ parse_dictzHfArgumentParser.parse_dictss $))++&& )  EHHK$6u$=$=HHHDAAAAtzz||AAAF  ) )&++-- 8 8 8%//&//C NN3     gK gePVWbPcPceeff fW~~r& json_filectt|d5}tj|}dddn #1swxYwY|||}t |S)at Alternative helper method that does not use `argparse` at all, instead loading a json file and populating the dataclass types. Args: json_file (`str` or `os.PathLike`): File name of the json file to parse allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the json file contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. zutf-8)encodingNr)openr jsonloadsreadrr)rKrropen_json_filedatars r$parse_json_filez HfArgumentParser.parse_json_files&$y//G 4 4 4 5:n113344D 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5//$9I/JJW~~s'AAA yaml_filec|tjt||}t |S)at Alternative helper method that does not use `argparse` at all, instead loading a yaml file and populating the dataclass types. Args: yaml_file (`str` or `os.PathLike`): File name of the yaml file to parse allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the json file contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. r)ryaml safe_loadr rr)rKrrrs r$parse_yaml_filez HfArgumentParser.parse_yaml_filesB&//$.i1J1J1L1L"M"M`p/qqW~~r&r1)NFTNN)F)__name__ __module__ __qualname____doc__rr__annotations__rrrG staticmethodrr=FieldrtrJrrrdictr,r r!rosPathLikerr __classcell__)rMs@r$rBrBos  m,,,,11}h}F]7]1^(_111111"g~gk>Oggg\gR 7m 7 7 7 7H!& TT y#~ TTTTltCH~RWXacfXfRg:LQsBK/0DH y#~ 2LQsBK/0DH y#~ r&rB)&r=rrr}rjargparserrrcollections.abcrrenumrinspectr pathlibr typingr r rrrrrrrrr%rIr,r6rorrr@rBr+r&r$rs. UUUUUUUUUU$$$$$$SSSSSSSSSSSSSSSSSS  GK % % --      3t 3#0D 3 3 3 3"04&)4)<# ,l,l,l eCcN+ ,,l 3-,l ,l b#g& ,l tn ,l,l,l,l,l^LLLLL~LLLLLr&