昺h<8ddlmZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z ddlmZddlmZddlmZddlmZmZmZmZdd lmZdd lmZdd lmZmZerdd l m!Z!d Z"dZ#dZ$ddZ%ddZ&GddZ'GddeeZ(Gddee(Z)dS)) annotationsN)ABC) TYPE_CHECKINGAnyDictIteratorListMappingOptional)CallbackManagerForLLMRun)LLM)GenerationChunk) BaseModelExtraFieldroot_validator)get_from_dict_or_env)enforce_stop_tokens)get_num_tokens_anthropicget_token_ids_anthropic)Configz Human:z Assistant:zCError: Prompt must alternate between ' Human:' and ' Assistant:'. input_textstrreturnc|}dD]I}||d|z}tdD]}|d|zd|z}J|S)N)Human: Assistant: z )replacerange)rnew_textwordis h/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain_community/llms/bedrock.py_add_newlines_before_har&srH(HH##D&4-88q H HA''4$GGHH H Oc|ddks0|d|dkrd|vr tdz|z}|ddkr |tz}|dt ddkrd|z}t |}d}t t |D]}|||t tztkr.|dzdkr|dz }ntjtd|z|||t tztkr.|dzdkr|dz }tjtd|z|dzdkr |tz}|S) Nrrr rrz Received ) countfind HUMAN_PROMPTASSISTANT_PROMPTlenr&r!warningswarnALTERNATION_ERROR)rr+r$s r%_human_assistant_formatr3's!!Q&&!!JOOL$A$AAA J & &!C'*4  %%**"22 /CMM/"h..j( (44J E 3z?? # # M M a!c,//// 0L @ @qyA~~  /2Kz2K2KKLLL a!c"23333 48H H HqyA~~  /2Kz2K2KKLLL qyA~~"22 r'cleZdZdZdddddZedd ZeddZe dddZdS)LLMInputOutputAdapterzAdapter class to prepare the inputs from Langchain to a format that LLM model expects. It also provides helper function to extract the generated text from the model response. completion outputTexttext generation) anthropicamazoncoheremetaproviderrprompt model_kwargsDict[str, Any]rci|}|dkrt||d<n0|dvr||d<n&|dkrt}||d<i||d<n||d<|dkr d|vrd|d<|S) Nr:r?)ai21r<r=r; inputTexttextGenerationConfigmax_tokens_to_sample)r3dict)clsr>r?r@ input_bodys r% prepare_inputz#LLMInputOutputAdapter.prepare_inputTs& % { " "#:6#B#BJx 3 3 3#)Jx  ! !J&,J{ #1AL1AJ- . .&,J{ # { " "'=Z'O'O14J- .r'responserc|dkr`tj|d}|dStj|d}|dkrA|ddddS|d kr.|d ddS|d kr|d S|d ddS)Nr:bodyr6rC completionsrdatar8r< generationsr=r9resultsr7)jsonloadsgetreaddecode)rIr>rL response_bodys r%prepare_outputz$LLMInputOutputAdapter.prepare_outputis3 { " " Jx||F';';'@'@'B'B'I'I'K'KLLM $$\22 2 Jx||F';';'@'@'B'BCCM v   $$]33A6::6BBFFvNN N  ! ! $$]33A6::6BB B    $$\22 2 $$Y//266|DD Dr'NstopOptional[List[str]]Iterator[GenerationChunk]c#K|d}|sdS||jvrtd||D]}|d}|rtj|d}|dkr"|ds||j|dkrdSt ||j|VdS) NrNz4Unknown streaming response output key for provider: chunkbytesr< is_finishedz )r8)rUprovider_to_output_key_map ValueErrorrSrTrWr)rIr>rLrZstreameventr^ chunk_objs r%prepare_output_streamz+LLMInputOutputAdapter.prepare_output_streamzsf%%  F 39 9 9QxQQ   EIIg&&E  Juyy'9'9'@'@'B'BCC x''m,( !?!IJ$%%FF&"3#A(#KL  r')r>rr?rr@rArrA)r>rrLrrrN)r>rrLrrZr[rr\) __name__ __module__ __qualname____doc__ra classmethodrKrYrfr'r%r5r5Fs33" ""[(EEE[E GK[r'r5cLeZdZUdZedZded<dZded< edd Zded < dZ d ed < d ed< dZ ded< dZ ded< dZ ded< dddddZ ded<ed+dZed,dZd-d Zed.d!Z d/d0d(Z d/d1d*ZdS)2 BedrockBasezBase class for Bedrock models.T)excluderclientNz Optional[str] region_name)defaultrpcredentials_profile_namezOptional[Config]configrmodel_idzOptional[Dict]r@ endpoint_urlFbool streamingstop_sequences stopSequences)r:r;rCr<zMapping[str, str]#provider_stop_sequence_key_name_mapvaluesrrc|d|S ddl}|d||d}n|}t|dd|j|d<i}|dr |d|d<|d r |d |d <|d r |d |d <|jdi||d<n:#t $rt d t$r}td |d}~wwxYw|S)zJValidate that AWS credentials to and python package exists in environment.rqNrrt) profile_namerrAWS_DEFAULT_REGION)rsrwrubedrock-runtimezRCould not import boto3 python package. Please install it with `pip install boto3`.zCould not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid.)r) boto3Sessionrrrrq ImportErrorModuleNotFoundError Exceptionrb)rIr}rsession client_paramses r%validate_environmentz BedrockBase.validate_environmentsi (  'M$  LLL01=--V    *    sB#B00"C'C""C'Mapping[str, Any]c |jpi}id|iS)zGet the identifying parameters.r@)r@)self _model_kwargss r%_identifying_paramszBedrockBase._identifying_paramss()/R  }-  r'cB|jddS)N.r)rvsplitrs r% _get_providerzBedrockBase._get_providers}""3''**r'c2|dkS)Nr:)rrs r%_model_is_anthropiczBedrockBase._model_is_anthropics!!##{22r'r?rZr[ run_manager"Optional[CallbackManagerForLLMRun]kwargsc |jpi}|}i||}t|||}t j|} d} d} |j| |j| | } t || } n<#t$r/}td| |j d}~wwxYw|t| |} | S)Napplication/jsonrNmodelIdaccept contentType!Error raised by bedrock service: )r@rr5rKrSdumpsrq invoke_modelrvrYrrbwith_traceback __traceback__r)rr?rZrrrr>paramsrJrNrrrLr8rs r%_prepare_input_and_invokez%BedrockBase._prepare_input_and_invokes)/R %%'',M,V,*8866RR z*%%#(  {//4=[0H)77(KKDD   DDDEETT    &tT22D s>B C"*C  Cr\c+PK|jpi}|}|r9||jvrtd|d|||j|<|dkrd|d<i||}t |||}tj|} |j | |j dd} n$#t$r} td| d} ~ wwxYwt || |D]$} | V||| j| %dS) NzStop sequence key name for z is not supported.r<Trcrrr)r^)r@rr|rbrUr5rKrSrrq!invoke_model_with_response_streamrvrrfon_llm_new_tokenr8) rr?rZrrrr>rrJrNrLrr^s r% _prepare_input_and_invoke_streamz,BedrockBase._prepare_input_and_invoke_stream!s)/R %%''  YtGGG N(NNN UYM$BFFxPP Q x  &*M( #,M,V,*8866RR z*%% F{DD ). EHH  F F FDDDEE E F+@@ h   F FEKKK&,,UZu,EEE  F Fs#C C" CC")r}rrr)rrrrrrxNN r?rrZr[rrrrrr r?rrZr[rrrrrr\)rhrirjrkrrq__annotations__rrrtrur@rwryr|rrpropertyrrrrrrmr'r%roros((%%%%F%%%%!%K%%%%/4eD$.O.O.OOOOO $F####LMMML$(L''''1"&L&&&&CI(&! " >>'^----^   X ++++333X3 %):> F%):> *F*F*F*F*F*F*Fr'roceZdZdZ eddZeddZeddZed d Z Gd d Z d!d"dZ d!d#dZ d$fd Z d%fd ZxZS)&BedrockaBedrock models. To authenticate, the AWS client uses the following methods to automatically load credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html If a specific credential profile should be used, you must pass the name of the profile from the ~/.aws/credentials file that is to be used. Make sure the credentials / roles used have the required policies to access the Bedrock service. rrcdS)zReturn type of llm.amazon_bedrockrmrs r% _llm_typezBedrock._llm_typejs  r'rxcdS)z9Return whether this model can be serialized by Langchain.TrmrIs r%is_lc_serializablezBedrock.is_lc_serializableos tr' List[str]c gdS)z*Get the namespace of the langchain object.) langchainllmsbedrockrmrs r%get_lc_namespacezBedrock.get_lc_namespacets0///r'rAc,i}|jr |j|d<|S)Nrr)rr)r attributess r% lc_attributeszBedrock.lc_attributesys%%'   9(,(8J} %r'c eZdZdZejZdS)Bedrock.Configz'Configuration for this pydantic object.N)rhrirjrkrforbidextrarmr'r%rrs55 r'rNr?rZr[rrrrr\c $|jd|||d|S)aCall out to Bedrock service with streaming. Args: prompt (str): The prompt to pass into the model stop (Optional[List[str]], optional): Stop sequences. These will override any stop sequences in the `model_kwargs` attribute. Defaults to None. run_manager (Optional[CallbackManagerForLLMRun], optional): Callback run managers used to process the output. Defaults to None. Returns: Iterator[GenerationChunk]: Generator that yields the streamed responses. Yields: Iterator[GenerationChunk]: Responses from the model. r?rZrrm)r)rr?rZrrs r%_streamzBedrock._streams4.5t4 +  AG   r'c t|jr"d}|jd|||d|D] }||jz } |S|jd||d|S)a`Call out to Bedrock service model. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Example: .. code-block:: python response = llm("Tell me a joke.") r)r?rZrm)ryrr8r)rr?rZrrr6r^s r%_callz Bedrock._calls, > J%DkEK ) )ej(  -t-QV$QQ&QQQr'r8intcr|jrt|St|Srg)rrsuperget_num_tokensrr8 __class__s r%rzBedrock.get_num_tokenss4  # 0+D11 177))$// /r' List[int]cr|jrt|St|Srg)rrr get_token_idsrs r%rzBedrock.get_token_idss4  # /*400 077((.. .r'rr)rr)rrArrr)r8rrr)r8rrr)rhrirjrkrrrlrrrrrrrr __classcell__)rs@r%rrNsZ      X [000[0X%):>      <%):> RRRRR@000000 //////////r'r)rrrr)* __future__rrSr0abcrtypingrrrrr r r langchain_core.callbacksr #langchain_core.language_models.llmsr langchain_core.outputsrlangchain_core.pydantic_v1rrrrlangchain_core.utilsrlangchain_community.llms.utilsr'langchain_community.utilities.anthropicrrbotocore.configrr-r.r2r&r3r5rorrmr'r%rs"""""" NNNNNNNNNNNNNNNNNN======333333222222NNNNNNNNNNNN555555>>>>>> '&&&&&& #M >PPPPPPPPfrFrFrFrFrF)SrFrFrFj~/~/~/~/~/c;~/~/~/~/~/r'