ݘhddlZddlZddlZddlmZddlmZddlmZm Z m Z m Z ddl Z ddlmZddlmZmZddlmZdd lmZmZmZdd lmZmZdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,dd l-m.Z.ddl/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8e&rddl9Z9e(re)rddl:m;Z<nddl=m;Z<e*j>e?Z@dZAe!deAe.dGddeZBe"eBjCeB_CeBjCjD.eBjCjDEdddeBjC_DdSdS)N)deepcopy)partial)AnyCallableOptionalUnion)custom_object_save) BatchFeature get_size_dict)BaseImageProcessorFast)ChannelDimensionSizeDictvalidate_kwargs)Unpack VideosKwargs)IMAGE_PROCESSOR_NAMEPROCESSOR_NAMEVIDEO_PROCESSOR_NAME TensorTypeadd_start_docstrings copy_func download_urlis_offline_mode is_remote_urlis_torch_availableis_torchcodec_availableis_torchvision_availableis_torchvision_v2_availablelogging) cached_files)requires) VideoInput VideoMetadatagroup_videos_by_shapeis_valid_video load_videomake_batched_metadatamake_batched_videosreorder_videosto_channel_dimension_format) functionalaQ Args: do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the video's (height, width) dimensions to the specified `size`. Can be overridden by the `do_resize` parameter in the `preprocess` method. size (`dict`, *optional*, defaults to `self.size`): Size of the output video after resizing. Can be overridden by the `size` parameter in the `preprocess` method. size_divisor (`int`, *optional*, defaults to `self.size_divisor`): The size by which to make sure both the height and width can be divided. default_to_square (`bool`, *optional*, defaults to `self.default_to_square`): Whether to default to a square video when resizing, if size is an int. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the video. Only has an effect if `do_resize` is set to `True`. Can be overridden by the `resample` parameter in the `preprocess` method. do_center_crop (`bool`, *optional*, defaults to `self.do_center_crop`): Whether to center crop the video to the specified `crop_size`. Can be overridden by `do_center_crop` in the `preprocess` method. do_pad (`bool`, *optional*): Whether to pad the video to the `(max_height, max_width)` of the videos in the batch. crop_size (`dict[str, int]` *optional*, defaults to `self.crop_size`): Size of the output video after applying `center_crop`. Can be overridden by `crop_size` in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the video by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `self.rescale_factor`): Scale factor to use if rescaling the video. Only has an effect if `do_rescale` is set to `True`. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the video. Can be overridden by the `do_normalize` parameter in the `preprocess` method. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`float` or `list[float]`, *optional*, defaults to `self.image_mean`): Mean to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_mean` parameter in the `preprocess` method. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `list[float]`, *optional*, defaults to `self.image_std`): Standard deviation to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_std` parameter in the `preprocess` method. Can be overridden by the `image_std` parameter in the `preprocess` method. do_convert_rgb (`bool`, *optional*, defaults to `self.image_std`): Whether to convert the video to RGB. video_metadata (`VideoMetadata`, *optional*): Metadata of the video containing information about total duration, fps and total number of frames. do_sample_frames (`int`, *optional*, defaults to `self.do_sample_frames`): Whether to sample frames from the video before processing or to process the whole video. num_frames (`int`, *optional*, defaults to `self.num_frames`): Maximum number of frames to sample when `do_sample_frames=True`. fps (`int` or `float`, *optional*, defaults to `self.fps`): Target frames to sample per second when `do_sample_frames=True`. return_tensors (`str` or `TensorType`, *optional*): Returns stacked tensors if set to `pt, otherwise returns a list of tensors. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input video. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input video. If unset, the channel dimension format is inferred from the input video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: video in (height, width) format. device (`torch.device`, *optional*): The device to process the videos on. If unset, the device is inferred from the input videos. return_metadata (`bool`, *optional*): Whether to return video metadata or not. z!Constructs a base VideoProcessor.)vision torchvision)backendsc"eZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZdZdZdZdZdZdZdZeZdgZdeeddffd Zdefd Zd d defd Z dCd e de!e"de!e#e"e$ffdZ% dCdede#e e&fde!e'de!e(de)d f dZ* dCdede!e#e+e,fde!e+de)d fdZ-e.e/dedeedefdZ0 dDde)d de'de'de1de!e"de!dd e'd!e1d"e'd#e'd$e$d%e'd&e!e#e$e)e$fd'e!e#e$e)e$fd(e!e#e+e2fdef d)Z3e4 dEd+e#e+e5j6fd,e!e#e+e5j6fd-e'd.e'd/e!e#e+e'fd0e+f d1Z7dFd2e#e+e5j6fd3e'fd4Z8e4d+e#e+e5j6fde9e&e+e:fe&e+e:fffd5Z;e4d6e&e+e:ffd7Zd:e#e+e5j6ffd;Z?d<Z@e4d=e#e+e5j6ffd>ZAe4dGd@ZBdDdAe#e+e)e+e)e)e+ffdBZCxZDS)HBaseVideoProcessorNTgp?Fpixel_values_videoskwargsreturnc rt|dd|_|D]N\}} t |||#t $r*}td|d|d||d}~wwxYw|d|j }|*t||d|j nd|_ |d|j }|t|d nd|_ t|jj|_|jD]\}||t ||||/t ||t't)||d]dS) Nprocessor_classz Can't set z with value z for sizedefault_to_square)r7r8 crop_size) param_name)super__init__pop_processor_classitemssetattrAttributeErrorloggererrorr7r r8r9list valid_kwargs__annotations__keysmodel_valid_processing_keysgetrgetattr)selfr3keyvalueerrr7r9 __class__s k/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/transformers/video_processing_utils.pyr<zBaseVideoProcessor.__init__s  & +zKBaseVideoProcessor._decode_and_sample_videos...@s" L L LE!7!7 L L LrVrdim)ristack)ruimagess rPrwz@BaseVideoProcessor._decode_and_sample_videos..?sGK L LV L L LRSTTTrVzUSampling frames from a list of images is not supported! Set `do_sample_frames=False`.ro) r)r(r&zipframes_indicesappend isinstancerD fetch_imagesrf fetch_videos) rKrTrmrnrosampled_videossampled_metadatarWrbrks rP_decode_and_sample_videosz,BaseVideoProcessor._decode_and_sample_videos$sh%V,,.vnUUU &) $ $ h)9 hN! #&v~#>#> 2 2x++X>>>*1'%%eGn555 ''1111#F-NNq ** h&)T** h"&"3"3F";";$$o *.):):6Uf):)g)g&~%%rVinput_data_formatdevicec"g}|D]}t|tjrAt|tj|}t j|}|| |}| ||S)z: Prepare the input videos for processing. ) rnpndarrayr+rFIRSTri from_numpy contiguoustor)rKrTrrprocessed_videosrWs rP_prepare_input_videosz(BaseVideoProcessor._prepare_input_videosLs + +E%,, =3E;K;QSdee(//::<<!((  # #E * * * *rVc t|t|jjdgz|jjD]'}||t ||d(|d}|d}|d}|d}|rt|j fi|nd}| ||||\}}| |||}|j d i|}|j d i||d |d } |jd d |i|} | r|| d<| S) Nreturn_tensors)captured_kwargsvalid_processor_keysrrnrrm)rmrnro)rTrr data_formatreturn_metadatarTrs)rrGrDrErF setdefaultrJr=rrlrr_further_process_kwargs_validate_preprocess_kwargs _preprocess) rKrTr3 kwarg_namerrnrrmrorpreprocessed_videoss rPrSzBaseVideoProcessor.preprocesscs "KKMM!%d&7&G&L&L&N&N!O!OScRd!d    +; K KJ   j'$ D*I*I J J J J"JJ':;;!::&899H%%$455EU_GD$6AA&AAA[_!%!?!? )-/ "@" "  ++6M^gm+nn--7777((226222  =!!! **%677.d.GGfGGG  C4B  0 1""rVdo_convert_rgb do_resizer7 size_divisor interpolationzF.InterpolationModedo_center_cropr9 do_rescaledo_padrescale_factor do_normalize image_mean image_stdrc 0t|\}}i}|D]<\}}|r||}|r|||||}|||<=t ||}t|\}}i}|D]<\}}|r|||}||| | | | |}|||<=t ||}|rtj|dn|}td|i|S)N)r7rrrrxr2)data tensor_type) r%r?raresizer* center_croprescale_and_normalizerirzr )rKrTrrr7rrrr9rrrrrrrr3grouped_videosgrouped_videos_indexresized_videos_groupedr^stacked_videosresized_videosprocessed_videos_groupedrs rPrzBaseVideoProcessor._preprocesss(0EV/L/L,,!#%3%9%9%;%; ; ; !E> E!%!4!4^!D!D !%"LXe"-""-; "5 ) )'(>@TUU0E^/T/T,,#% %3%9%9%;%; = = !E> M!%!1!1.)!L!L!77 NL*V_N/= $U + +)*BDXYYCQg5;'7Q????Wg"79I!JXfggggrVmainpretrained_model_name_or_path cache_dirforce_downloadlocal_files_onlytokenrevisionc ||d<||d<||d<||d<|dd}|-tjdt|t d|}|||d <|j|fi|\} }|j| fi|S) a Instantiate a type of [`~video_processing_utils.VideoProcessorBase`] from an video processor. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained video hosted inside a model repo on huggingface.co. - a path to a *directory* containing a video processor file saved using the [`~video_processing_utils.VideoProcessorBase.save_pretrained`] method, e.g., `./my_model_directory/`. - a path or url to a saved video processor JSON *file*, e.g., `./my_model_directory/video_preprocessor_config.json`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model video processor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the video processor files and override the cached versions if they exist. resume_download: Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v5 of Transformers. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. return_unused_kwargs (`bool`, *optional*, defaults to `False`): If `False`, then this function returns just the final video processor object. If `True`, then this functions returns a `Tuple(video_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not video processor attributes: i.e., the part of `kwargs` which has not been used to update `video_processor` and is otherwise ignored. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. kwargs (`dict[str, Any]`, *optional*): The values in kwargs of any keys which are video processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* video processor attributes is controlled by the `return_unused_kwargs` keyword parameter. Returns: A video processor of type [`~video_processing_utils.ImagVideoProcessorBase`]. Examples: ```python # We can't instantiate directly the base class *VideoProcessorBase* so let's show the examples on a # derived class: *LlavaOnevisionVideoProcessor* video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf" ) # Download video_processing_config from huggingface.co and cache. video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "./test/saved_model/" ) # E.g. video processor (or model) was saved using *save_pretrained('./test/saved_model/')* video_processor = LlavaOnevisionVideoProcessor.from_pretrained("./test/saved_model/video_preprocessor_config.json") video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False ) assert video_processor.do_normalize is False video_processor, unused_kwargs = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False, return_unused_kwargs=True ) assert video_processor.do_normalize is False assert unused_kwargs == {"foo": False} ```rrrruse_auth_tokenNrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.V`token` and `use_auth_token` are both specified. Please set only the argument `token`.r)r=warningswarn FutureWarningrfget_video_processor_dict from_dict) clsrrrrrrr3rvideo_processor_dicts rPfrom_pretrainedz"BaseVideoProcessor.from_pretrainedst({#1 %5!"%z$4d;;  % ME     l#E  #F7O'Cs'CDa'l'lek'l'l$fs}1<>. ) ) h !f>!f!f!fgg g NT2222  J#ZZ(8$??NjjN,@,@,M,Mb,QRRG'd'::6::G#99.II    ' t^D A A A A')gll>CW&X&X# 5666 M0KMMNNN    ' ' -jj)) (   ,,,rVc *|dd}|dd}|dd}|dd}|dd}|dd}|d d} |d d} |d d } |d d} |dd} |-tjdt|t d|}d| d}| | |d<t r| st dd} t|}tj |}tj |r|}d}nt|r|}t|}ntt} t!|tt"t$g||||| ||| | d }|d}n6#t&$rt($rt+d|d|dtdwxYw t-|dd5}|}dddn #1swxYwYt1j|}|d|}n&#t0j$rt+d |d!wxYw|rt d"|n t d"|d#|||fS)$a From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a video processor of type [`~video_processing_utils.VideoProcessorBase`] using `from_dict`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. Returns: `tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the video processor object. rNrFresume_downloadproxiesrrrr subfolder_from_pipeline _from_autorrvideo processor) file_typefrom_auto_classusing_pipelinez+Offline mode: forcing local_files_only=TrueT) filenamesrrrrrr user_agentrr%_raise_exceptions_for_missing_entriesrz Can't load video processor for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z filerutf-8encodingvideo_processorz"It looks like the config file at 'z' is not a valid JSON file.zloading configuration file z from cache at )r=rrrrfrrBrstrrrisdirrrrrr!rrEnvironmentError ExceptionOSErroropenreadjsonloadsrIJSONDecodeError)rrr3rrrrrrrrr from_pipelinerris_localresolved_video_processor_filevideo_processor_fileresolved_video_processor_filesreadertextrs rPrz+BaseVideoProcessor.get_video_processor_dictos$JJ{D11 $4e<< **%6==**Y-- 7D))$4d;;!::&8%@@::j$//JJ{B//  #3T::  **\599  % ME     l#E#4YY  $+8J' (    $%5 $ KKE F F F# (+,I(J(J%7==!>?? 7>>7 8 8% ,I )HH 8 9 9" #@ ,89V,W,W ) )#7  2>135I>Z'#1#$3%5)%':? 2 2 2.1Oq0Q--#      K7TKK9VKK0DKKK  3S7KKK %v{{}} % % % % % % % % % % % % % % %#':d#3#3 #7#;#;J+I5) J+5I99J+<I9=-J++#Krc |}|dd}d|vrd|vr|d|d<d|vrd|vr|d|d<|di|}g}|D];\}}t||r&t |||||<|D]}||dt d||r||fS|S)a Instantiates a type of [`~video_processing_utils.VideoProcessorBase`] from a Python dictionary of parameters. Args: video_processor_dict (`dict[str, Any]`): Dictionary that will be used to instantiate the video processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [`~video_processing_utils.VideoProcessorBase.to_dict`] method. kwargs (`dict[str, Any]`): Additional parameters from which to initialize the video processor object. Returns: [`~video_processing_utils.VideoProcessorBase`]: The video processor object instantiated from those parameters. return_unused_kwargsFr7r9NzVideo processor rs)copyr=r?hasattrr@rrBr)rrr3rr to_removerLrMs rPrzBaseVideoProcessor.from_dictsR" 488::%zz*@%HH V  *> > >+1::f+=+=  ( & [4H%H%H06 ;0G0G  -#55 455  ,,.. & &JC,, &e444  %%% " "C JJsD ! ! ! ! 888999  #"F* *" "rVct|j}|dd|dd|jj|d<|S)z Serializes this instance to a Python dictionary. Returns: `dict[str, Any]`: Dictionary of all the attributes that make up this video processor instance. rHN_valid_kwargs_namesvideo_processor_type)r__dict__r=rO__name__)rKoutputs rPto_dictzBaseVideoProcessor.to_dict sR$-(( 0$777 ($///)-)@%& rVc.|}|D]6\}}t|tjr|||<7|dd}|||d<tj|dddzS)z Serializes this instance to a JSON string. Returns: `str`: String containing all the attributes that make up this feature_extractor instance in JSON format. r>Nr6T)indent sort_keys ) rr?rrrtolistr=rdumps)rK dictionaryrLrMr>s rPto_json_stringz!BaseVideoProcessor.to_json_strings\\^^ $**,, 1 1JC%,, 1"',,.. 3&>>* This API is experimental and may have some slight breaking changes in the next releases. Args: auto_class (`str` or `type`, *optional*, defaults to `"AutoVideoProcessor "`): The auto class to register this new video processor with. rNz is not a valid auto class.) rrr transformers.models.automodelsautorrfr)r auto_class auto_modules rPregister_for_auto_classz*BaseVideoProcessor.register_for_auto_classOsn *c** -#,J666666666{J// I GGGHH H$rVvideo_url_or_urlscd}tstjdd}t|tr#t t fd|DSt ||S)z Convert a single or a list of urls into the corresponding `np.array` objects. If a single url is passed, the return value will be a single object. If a list is passed a list of objects is returned. torchcodecz`torchcodec` is not installed and cannot be used to decode the video by default. Falling back to `torchvision`. Note that `torchvision` decoding is deprecated and will be removed in future versions. r.c>g|]}|S)r|)r)ruxrorKs rPrwz3BaseVideoProcessor.fetch_videos..ys.sss\]d//EV/WWsssrV)backendro)rrrrrDr}r')rKr,ror1s` ` rPrzBaseVideoProcessor.fetch_videosis&(( $ MI   $G ' . . gsssssarssstuu u/Tefff frV)NNrR)NFFNr)F)r$)Er  __module__ __qualname__rresamplerrr7rr8r9rrrrrrrrnrdrcrmrrrEmodel_input_namesrr<r rUr#rar$rrhrfloatrldictboolrrDrrrrrBASE_VIDEO_PROCESSOR_DOCSTRINGrSrrr classmethodrPathLikerrtuplerrrrrrr r#r+r __classcell__)rOs@rPr1r1s KHJI DLIIN FJNLN CJNOL./G !5G$GGGGGG>1L1111 8%)+/ 333SM3eCJ' ( 3333r,004 &&&&&&mT12&&#4. && $H- && n  &&&&&&&&VEI $    $E#/?*?$@A   n      .&&#&#&&#  &#&#&#&#p<@!0h0h^$0h0h 0h  0h sm 0h 560h0h0h0h0h0h0hU5$u+#5670hE%e"4560h !sJ!78!0h$ %0h0h0h0hd8<$!&,0o=o=',S"+-='>o=E#r{"234o= o=  o= c4i() o=o=o=o=[o=b;-;-eC4D.E;-TX;-;-;-;-zl,,1#r{2B,Cl, tCH~tCH~- .l,l,l,[l,\*#T#s(^*#*#*#[*#X c3h    GGGGG* 05bk1A+B 0 0 0 0DDD+uS"+-='>+++[+$%%%[%2ggeCcDcO4S.TggggggggrVr1rr$zvideo processor file)object object_class object_files)Frrrrr functoolsrtypingrrrrnumpyrdynamic_module_utilsr image_processing_utilsr r image_processing_utils_fastr image_utilsrrrprocessing_utilsrrutilsrrrrrrrrrrrrrr utils.hubr!utils.import_utilsr" video_utilsr#r$r%r&r'r(r)r*r+ritorchvision.transforms.v2r,r\torchvision.transforms get_loggerr rBr9r1r__doc__formatrsrVrPrRs  111111111111444444@????? 32222222 $#####((((((                      LLL;""$$;=======::::::  H % %C" L'" ,---` g` g` g` g` g/` g` g.-  ` gF"++=+I!J!J!)5-?-K-S-Z-Z /CRh.[.."***65rV