昺h!ddlmZmZmZmZmZmZmZddlm Z ddl m Z m Z ddl mZddlmZddlmZGddeZd S) )Any AsyncIteratorDictIteratorListOptionalUnion)root_validator)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)enforce_stop_tokens)GenerationChunkc 0eZdZUdZeed<eed< dZee eefed< dZ e dee fed< dZ e ed< ed e eeffd Zed efd Zed e d e fd Z ddedeeedeeded ef dZ ddedeeedeeded ef dZ ddedeeedeeded eef dZ ddedeeedeeded eef dZdS) DeepSparseaHNeural Magic DeepSparse LLM interface. To use, you should have the ``deepsparse`` or ``deepsparse-nightly`` python package installed. See https://github.com/neuralmagic/deepsparse This interface let's you deploy optimized LLMs straight from the [SparseZoo](https://sparsezoo.neuralmagic.com/?useCase=text_generation) Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") pipelinemodelN model_configgeneration_configF streamingreturnc8|j|j|j|jdS)zGet the identifying parameters.rrrrrselfs k/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain_community/llms/deepsparse.py_identifying_paramszDeepSparse._identifying_params+s)Z -!%!7    cdS)zReturn type of llm. deepsparsers r _llm_typezDeepSparse._llm_type5s |rvaluesc ddlm}n#t$rtdwxYw|dpi}|jd d|dd||d<|S) z2Validate that ``deepsparse`` package is installed.r)Pipelinez[Could not import `deepsparse` package. Please install it with `pip install deepsparse[llm]`rtext_generationr)task model_pathrr!)r r% ImportErrorcreate)clsr#r%rs rvalidate_environmentzDeepSparse.validate_environment:s  + + + + + + +   G   n-3 ,X_ "g    z  s #promptstop run_managerkwargsc |jr#d}|jd|||d|D] }||jz } |}n$|jdd|i|jjdj}|t ||}|S)Generate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") llm("Tell me a joke.") r-r.r/ sequencesrNr!)r_streamtextrr generationsrrr-r.r/r0combined_outputchunkr7s r_callzDeepSparse._callNs& >  O%DkEK . . 5:-"DD III$2HIIQ    &tT22D rcK|jr)d}|jd|||d|23d{V}||jz }6|}n$|jdd|i|jjdj}|t ||}|S)r2r3r4Nr5rr!)r_astreamr7rrr8rr9s r_acallzDeepSparse._acallts& >  O,t} Dk  EK   . . . . . . .e 5:- #DD III$2HIIQ    &tT22D s/c+K|jd|dd|j}|D]C}t|jdj}|V|r||jDdSaYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like object containing a string token. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse( model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none", streaming=True ) for chunk in llm.stream("Tell me a joke", stop=["'"," "]): print(chunk, end='', flush=True) T)r5rr)r7)tokenNr!rrrr8r7on_llm_new_tokenrr-r.r/r0 inferencerBr;s rr6zDeepSparse._streams8"DM   040F   ? ?E#):1)=)BCCCEKKK ?,,5:,>>>  ? ?rc K|jd|dd|j}|D]J}t|jdj}|WV|r!||jd{VKdSrArCrEs rr>zDeepSparse._astreams8"DM   040F   E EE#):1)=)BCCCEKKKK E!222DDDDDDDDD  E Er)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrr rboolpropertyrr"r r,rr r<r r?rrr6rr>r!rrrr s  MMM JJJV-1L(4S>*111E15uT3_-444) It8  T#s(^   X 3X^$4,%):> $$$tCy!$67 $  $ $$$$R%)?C $$$tCy!$;< $  $ $$$$R%):> $?$?$?tCy!$?67 $?  $? / " $?$?$?$?R%)?C $E$E$EtCy!$E;< $E  $E  ' $E$E$E$E$E$ErrN)typingrrrrrrr langchain_core.pydantic_v1r langchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_community.llms.utilsrlangchain_core.outputsrrr!rrrVsLLLLLLLLLLLLLLLLLL555555433333>>>>>>222222WEWEWEWEWEWEWEWEWEWEr