hdZddlZddlZddlZddlZddlmZddlmZm Z m Z ddl m Z ddl mZddlmZmZmZdd lmZejeZGd d e ZGd d ZdefdZde efdZGddZdS)a Implementation of a custom transfer agent for the transfer type "multipart" for git-lfs. Inspired by: github.com/cbartz/git-lfs-swift-transfer-agent/blob/master/git_lfs_swift_transfer.py Spec is: github.com/git-lfs/git-lfs/blob/master/docs/custom-transfers.md To launch debugger while developing: ``` [lfs "customtransfer.multipart"] path = /path/to/huggingface_hub/.env/bin/python args = -m debugpy --listen 5678 --wait-for-client /path/to/huggingface_hub/src/huggingface_hub/commands/huggingface_cli.py lfs-multipart-upload ```N)_SubParsersAction)DictListOptional)BaseHuggingfaceCLICommand)LFS_MULTIPART_UPLOAD_COMMAND) get_sessionhf_raise_for_statuslogging) SliceFileObjc.eZdZdZedefdZdS) LfsCommandsu Implementation of a custom transfer agent for the transfer type "multipart" for git-lfs. This lets users upload large files >5GB 🔥. Spec for LFS custom transfer agent is: https://github.com/git-lfs/git-lfs/blob/master/docs/custom-transfers.md This introduces two commands to the CLI: 1. $ huggingface-cli lfs-enable-largefiles This should be executed once for each model repo that contains a model file >5GB. It's documented in the error message you get if you just try to git push a 5GB file without having enabled it before. 2. $ huggingface-cli lfs-multipart-upload This command is called by lfs directly and is not meant to be called by the user. parserc|dd}|dtd|d|td }|d dS) Nzlfs-enable-largefilesz:Configure your repository to enable upload of files > 5GB.)helppathz/Local path to repository you want to configure.)typerc t|SN)LfsEnableCommandargss d/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/huggingface_hub/commands/lfs.pyz1LfsCommands.register_subcommand..?5Ed5K5K)funcF)add_helpc t|Sr)LfsUploadCommandrs rrz1LfsCommands.register_subcommand..Crr) add_parser add_argumentstr set_defaultsr)r enable_parser upload_parsers rregister_subcommandzLfsCommands.register_subcommand9s)) #*f*   ""6:k"lll""(K(K"LLL))*FQV)WW ""(K(K"LLLLLrN)__name__ __module__ __qualname____doc__ staticmethodrr(rrrr$sM( M$5 M M M\ M M MrrceZdZdZdZdS)rc||_dSrrselfrs r__init__zLfsEnableCommand.__init__G  rctj|jj}tj|st dt dtjd d|tjdt d|t ddS)Nz)This does not look like a valid git repo.z > > GGG J P P R R     ZrHargparsertypingrrrhuggingface_hub.commandsrhuggingface_hub.lfsrutilsr r r utils._lfsr get_loggerr)rYrrrMr[r!r.rrrs$  &&&&&&''''''''''>>>>>><<<<<<==========%%%%%%  H % %MMMMM+MMMD22222222,4 (4.    V9V9V9V9V9V9V9V9V9V9r