haL"ddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZddlmZmZddlmZmZmZmZmZdd lmZdd lmZdd lm Z m!Z!m"Z"dd l#m$Z$e!j%e&Z'dZ(er ddl)Z(n#e*$rddl+Z,e,j(Z(YnwxYwd e$de$fdZ-d0dZ.dZ/dZ0 d1de de1dee2fdZ3e- d2dee4e fdeee4e fde1de1deee5e4ff dZ6d3d Z7e"e-dd!ddddddddddddd"d#e4dee2d$e4d%ee1d&ee4d'ee4d(ee4d)ee1d*eeee4e4fd+eeee4e4fd,eeee4e4fd-ee4de1deee5e4fde1fd.Z8Gd/deZ9dS)4Nwraps)Path)copytree)AnyDictListOptionalUnion) ModelHubMixinsnapshot_download)get_tf_versionis_graphviz_availableis_pydot_availableis_tf_available yaml_dump) constants)HfApi)SoftTemporaryDirectoryloggingvalidate_hf_hub_args) CallableTfnreturnc<tfd}|S)Nclt|dstdjd|g|Ri|S)Nhistoryz Cannot use 'zp': Keras 3.x is not supported. Please save models manually and upload them using `upload_folder` or `hf upload`.)hasattrNotImplementedError__name__)modelargskwargsrs c/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/huggingface_hub/keras_mixin.py_innerz'_requires_keras_2_model.._inner+seui(( %er{eee r%)$)))&)))r)rr&s` r%_requires_keras_2_modelr()s3 2YY****Y* Mr'c<g}|D]w\}}|r|d|n|}t|tjr6|t ||`|||fxt|S)aFlatten a nested dictionary. Reference: https://stackoverflow.com/a/6027615/10319735 Args: dictionary (`dict`): The nested dictionary to be flattened. parent_key (`str`): The parent key to be prefixed to the children keys. Necessary for recursing over the nested dictionary. Returns: The flattened dictionary. .)items isinstance collectionsMutableMappingextend _flatten_dictappenddict) dictionary parent_keyr,keyvaluenew_keys r%r1r17s E &&(( + + U+5>Z''#'''3 e[7 8 8 + LL%''      LL'5) * * * * ;;r'cd}|jv|j}t|}tjj|d<d}|D]\}}|d|d|dz }|S)z6Parse hyperparameter dictionary into a markdown table.Ntraining_precisionz*| Hyperparameters | Value | | :-- | :-- | z| z | z | ) optimizer get_configr1kerasmixed_precision global_policynamer,)r"tableoptimizer_paramsr6r7s r%_create_hyperparameter_tablerCTs E " ?5577()9::161F1T1T1V1V1[-.>*0022 . .JC -#--%--- -EE Lr'c \tj||dddddddd dS)N /model.pngFTTB`)to_file show_shapes show_dtypeshow_layer_namesrankdir expand_nesteddpi layer_range)r=utils plot_model)r"save_directorys r% _plot_networkrSbsO K !---      r'Trepo_dirrQmetadatac|dz }|rdSt|}|r,trtrt |||i}d|d<d}|t |dz }|dz }|dz }|d z }|d z }||d z }|d z }|d z }||z }|dz }|rAt j|dr|dz }|dz }|dz }d}|d|dz }|dz }||dS)zd Creates a model card for the repository. Do not overwrite an existing README.md file. z README.mdNr= library_namez--- F)default_flow_stylez/ ## Model description More information needed z9 ## Intended uses & limitations More information needed z: ## Training and evaluation data More information needed z ## Training procedure z ### Training hyperparameters z; The following hyperparameters were used during training:  rEz ## Model Plot z
z$ View Model Plot z ./model.pngz ![Model Image](z) z
) existsrCrrrSrospath write_text)r"rTrQrU readme_pathhyperparameters model_card path_to_plots r%_create_model_cardrbps[(K2599O'+--'2D2F2F'eX&&&&H^J)H????J'JGGJQQJRRJ"11 88 VV o% d %bgnn%<%<%<==%** m# >> $ ;,;;;; n$ :&&&&&r'FrRconfiginclude_optimizertagsc dttd|jstdt |}|dd|rt |ts tdt|d|tj z d5}tj||dddn #1swxYwYi}t |tr||d <nt |t r|g|d <|d d} | @t%jd t(d |vr|d | n| g|d <|j|jjikr|d z } | rt%jd t0| dd5}tj|jj|dddddn #1swxYwYt3||||tjj||fd|i|dS)aL Saves a Keras model to save_directory in SavedModel format. Use this if you're using the Functional or Sequential APIs. Args: model (`Keras.Model`): The [Keras model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) you'd like to save. The model must be compiled and built. save_directory (`str` or `Path`): Specify directory in which you want to save the Keras model. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. include_optimizer(`bool`, *optional*, defaults to `False`): Whether or not to include optimizer in serialization. plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. tags (Union[`str`,`list`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Nz>Called a Tensorflow-specific function but could not import it.z+Model should be built before trying to saveT)parentsexist_okzAProvided config to save_pretrained_keras should be a dict. Got: ''wre task_namez>`task_name` input argument is deprecated. Pass `tags` instead.z history.jsonzZ`history.json` file already exists, it will be overwritten by the history of this version.zutf-8)encoding)indent sort_keysrd)r= ImportErrorbuilt ValueErrorrmkdirr-r3 RuntimeErrortyper CONFIG_NAMEopenjsondumpliststrpopwarningswarn FutureWarningr2rrZ UserWarningrbmodels save_model) r"rRrcrdrQremodel_save_kwargsfrUrkr\s r%save_pretrained_kerasrsF }Z[[[ ;HFGGG.))N555!&$'' trcghncocorrrss sy44 : :3 ? ? !1 Ifa  ! ! ! ! ! ! ! ! ! ! ! ! ! ! !H$" D#  " 6!%%k488I L     X   V  # #I . . . . ){HV  } = B & &!N2D{{}}  p311 NQ %-/1MMMM N N N N N N N N N N N N N N Nunj(CCC LE>llEVlZkllllls$,CCC #G99G=G=KerasModelHubMixinc$tj|i|S)a Instantiate a pretrained Keras model from a pre-trained model from the Hub. The model is expected to be in `SavedModel` format. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): Can be either: - A string, the `model id` of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - You can add `revision` by appending `@` at the end of model_id simply like this: `dbmdz/bert-base-german-cased@main` Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. - A path to a `directory` containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `./my_model_directory/`. - `None` if you are both providing the configuration and state dictionary (resp. with keyword arguments `config` and `state_dict`). force_download (`bool`, *optional*, defaults to `False`): Whether to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only look at local files (i.e., do not try to download the model). model_kwargs (`Dict`, *optional*): model_kwargs will be passed to the model during initialization Passing `token=True` is required when you want to use a private model. )rfrom_pretrained)r#r$s r%from_pretrained_kerasrsj  -t >v > >>r'z'Push Keras model using huggingface_hub.)rccommit_messageprivate api_endpointtokenbranch create_prallow_patternsignore_patternsdelete_patternslog_dirrdrerQrepo_idrrrrrrrrrrc t|}||||dj}t5}t ||z }t ||f|| ||d|| F| gnt | tr| gn| } | dt| |dz | d||||||| | | cdddS#1swxYwYdS) a Upload model checkpoint to the Hub. Use `allow_patterns` and `ignore_patterns` to precisely filter which files should be pushed to the hub. Use `delete_patterns` to delete existing remote files in the same commit. See [`upload_folder`] reference for more details. Args: model (`Keras.Model`): The [Keras model](`https://www.tensorflow.org/api_docs/python/tf/keras/Model`) you'd like to push to the Hub. The model must be compiled and built. repo_id (`str`): ID of the repository to push to (example: `"username/my-model"`). commit_message (`str`, *optional*, defaults to "Add Keras model"): Message to commit while pushing. private (`bool`, *optional*): Whether the repository created should be private. If `None` (default), the repo will be public unless the organization's default is private. api_endpoint (`str`, *optional*): The API endpoint to use when pushing the model to the hub. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If not set, will use the token set when logging in with `hf auth login` (stored in `~/.huggingface`). branch (`str`, *optional*): The git branch on which to push the model. This defaults to the default branch as specified in your repository, which defaults to `"main"`. create_pr (`boolean`, *optional*): Whether or not to create a Pull Request from `branch` with that commit. Defaults to `False`. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. allow_patterns (`List[str]` or `str`, *optional*): If provided, only files matching at least one pattern are pushed. ignore_patterns (`List[str]` or `str`, *optional*): If provided, files matching any of the patterns are not pushed. delete_patterns (`List[str]` or `str`, *optional*): If provided, remote files matching any of the patterns will be deleted from the repo. log_dir (`str`, *optional*): TensorBoard logging directory to be pushed. The Hub automatically hosts and displays a TensorBoard instance if log files are included in the repository. include_optimizer (`bool`, *optional*, defaults to `False`): Whether or not to include optimizer during serialization. tags (Union[`list`, `str`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Returns: The url of the commit of your model in the given repository. )endpointT)rrrrh)rcrdrerQNzlogs/*logsr") repo_typer folder_pathrrrevisionrrrr) r create_reporrrrr-r{r2r upload_folder)r"rrcrrrrrrrrrrrdrerQrapitmp saved_paths r%push_to_hub_kerasr(s`  & & &CoogUGVZo[[cG ! !% S#YY(    /!         #*"/377)_%%(   " "8 , , , Wj61 2 2 2  "))++!   5% % % % % % % % % % % % % % % % % % sB CCCcTeZdZdZdZe ddeeee ffdZ dS)raA Implementation of [`ModelHubMixin`] to provide model Hub upload/download capabilities to Keras models. ```python >>> import tensorflow as tf >>> from huggingface_hub import KerasModelHubMixin >>> class MyModel(tf.keras.Model, KerasModelHubMixin): ... def __init__(self, **kwargs): ... super().__init__() ... self.config = kwargs.pop("config", None) ... self.dummy_inputs = ... ... self.layer = ... ... def call(self, *args): ... return ... >>> # Initialize and compile the model as you normally would >>> model = MyModel() >>> model.compile(...) >>> # Build the graph by training it or passing dummy inputs >>> _ = model(model.dummy_inputs) >>> # Save model weights to local directory >>> model.save_pretrained("my-awesome-model") >>> # Push model weights to the Hub >>> model.push_to_hub("my-awesome-model") >>> # Download and initialize weights from the Hub >>> model = MyModel.from_pretrained("username/super-cool-model") ``` c&t||dSN)r)selfrRs r%_save_pretrainedz#KerasModelHubMixin._save_pretrainedsdN33333r'Nrcc  ttdtj|s!t |||dt } n|} tj| } | | _ | S)aHere we just call [`from_pretrained_keras`] function so both the mixin and functional APIs stay in sync. TODO - Some args above aren't used since we are calling snapshot_download instead of hf_hub_download. Nz>Called a TensorFlow-specific function but could not import it.r=)rr cache_dirrWlibrary_version) r=rpr[r\isdirr rr load_modelrc) clsmodel_idrrforce_downloadproxiesresume_downloadlocal_files_onlyrrc model_kwargsstorage_folderr"s r%_from_pretrainedz#KerasModelHubMixin._from_pretraineds( =^__ _w}}X&& &. !#$ . 0 0 NN&N ''77  r'r) r! __module__ __qualname____doc__r classmethodr rr{rrr'r%rrso!!F444,0((c3h(((([(((r')r))TN)NFTN)rr):collections.abcabcr.rxr[r} functoolsrpathlibrshutilrtypingrrr r r huggingface_hubr r huggingface_hub.utilsrrrrrr)rhf_apirrPrrr utils._typingr get_loggerr!loggerr=tf_kerasrp tensorflowtfr(r1rCrSboolr3rbr{rzrrrrrr'r%rs%%%%%% 33333333333333<<<<<<<<HHHHHHHHHH$$$$$$  H % % ?         i    :      "# )')')')'tn )')')')'X(,#'+ PmPm#t)$Pm T#s(^ $Pm Pm  Pm 5s# $ PmPmPmPmf5?5?5?5?p "C""& $6:7;7;!#'+#w w w w  TN w  w d^ w 3-w  C=w  SMw ~w U49c>23w eDIsN34w eDIsN34w c]w w 5s# $!w "#w w w w tPPPPPPPPPPsBBB