昺hDddlZddlZddlmZmZddlmZmZmZm Z m Z m Z ddl Z ddl mZddlmZddlmZmZmZmZmZmZdgZGdd eeZd eeefd efd Zd eeefd efd Zd eeefd efdZGddeZGddeZd efdZ d efdZ!d efdZ"GddeZ#dS)N)ABCabstractmethod)AnyCallableDictListMappingOptional)CallbackManagerForLLMRun)LLM) BaseModelExtraField PrivateAttrroot_validator validator Databricksc eZdZUdZeed<eed<dedededefdZdedefd Zdededefd Z e dded e e d efdefdZ edefdZd S)_DatabricksClientBasez0A base JSON API client that talks to Databricks.api_url api_tokenmethodurlrequestreturncdd|ji}tj||||}|jst d|jd|j|S)N AuthorizationzBearer )rrheadersjsonzHTTP z error: )rrequestsrok ValueError status_codetextr)selfrrrrresponses k/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain_community/llms/databricks.pyrz_DatabricksClientBase.requestsv"$>dn$>$>?#sG'   { TRX%9RR8=RRSS S}}c0|d|dS)NGETr)r%rs r'_getz_DatabricksClientBase._get%s||E3---r(c0|d||S)NPOSTr+)r%rrs r'_postz_DatabricksClientBase._post(s||FC111r(Ntransform_output_fn.cdSN)r%rr0s r'postz_DatabricksClientBase.post+s  r(cdS)NFr3r%s r'llmz_DatabricksClientBase.llm1sur(r2)__name__ __module__ __qualname____doc__str__annotations__rrr,r/rr rr4propertyboolr7r3r(r'rrs:: LLLNNNccc......22s2s2222PT   19(38:L1M    ^ TXr(rr&rc*|dddS)Nchoicesrr$r3r&s r'_transform_completionsrC6s I q !& ))r(c*|dddS)N candidatesrr$r3rBs r'_transform_llama2_chatrF:s L !! $V ,,r(c6|ddddS)NrArmessagecontentr3rBs r'_transform_chatrJ>s I q !) ,Y 77r(c"eZdZUdZeed<eed<eed<dZeed<dZe ed<dZ e eed <d effd Z e d e fd Zeddeeefd eeeffdZ ddede edefd efdZxZS) _DatabricksServingEndpointClientz:An API client that talks to a Databricks serving endpoint.host endpoint_namedatabricks_uriNclientFexternal_or_foundationtaskdatac tjdi| ddlm}||j|_n"#t $r}t d|d}~wwxYw|j|j}| dd dv|_ |j | d|_ dSdS) Nr)get_deploy_clientzMFailed to create the client. Please install mlflow with `pip install mlflow`. endpoint_type)external_modelfoundation_model_apirRr3) super__init__mlflow.deploymentsrUrOrP ImportError get_endpointrNgetlowerrQrR)r%rSrUeendpoint __class__s r'r[z)_DatabricksServingEndpointClient.__init__Ls  4     < < < < < <++D,?@@DKK   C   ;++D,>??&.ll?B&G&G&M&M&O&OT ' # 9  V,,DIII  s8 AAArc|jdvS)N) llm/v1/chatllm/v1/completions llama2/chat)rRr6s r'r7z$_DatabricksServingEndpointClient.llmasyPPPr(TprevaluescJd|vr|d}|d}d|d|d}||d<|S)NrrMrNhttps://z/serving-endpoints/z /invocationsr3)clsrjrMrNrs r' set_api_urlz,_DatabricksServingEndpointClient.set_api_urlesG F " "&>D"?3MUUU-UUUG 'F9  r(rr0.c|jrd|j|j|}|r ||S|jdkrt |S|jdkrt |S|Sd|gi}|j|j|}|d}t|tr|dn|}|jdkrt|S|r ||n|S)N)rbinputsrerfdataframe_records predictionsrrg) rQrPpredictrNrRrJrC isinstancelistrF)r%rr0respwrapped_requestr&predspreds r'r4z%_DatabricksServingEndpointClient.postns  & N;&&0B7&SSD" 1**4000yM))&t,,,222-d333K 3WI>O{**+O+H]+E)%66A588EDyM))-d3330CM&&t,,, Mr(r2)r8r9r:r;r<r=rPrrQr?rRr r[r>r7rrrnrr4 __classcell__rcs@r'rLrLBslDD IIIFC#(D(((D(3--s------*QTQQQXQ^c3hDcNQUNNN19(38:L1MN NNNNNNNNr(rLceZdZUdZeed<eed<eed<eddeeefdeeeffd Z dd ed e e d efdefdZ d S)#_DatabricksClusterDriverProxyClientzBAn API client that talks to a Databricks cluster driver proxy app.rM cluster_idcluster_driver_portTrhrjrc^d|vr(|d}|d}|d}d|d|d|}||d<|S)NrrMr~rrlz/driver-proxy-api/o/0//r3)rmrjrMr~portrs r'rnz/_DatabricksClusterDriverProxyClient.set_api_urlsW F " "&>D -J/0DPPPZPP$PPG 'F9  r(Nrr0.cV||j|}|r ||n|Sr2)r/r)r%rr0rvs r'r4z(_DatabricksClusterDriverProxyClient.posts5zz$,00,?I""4(((TIr(r2) r8r9r:r;r<r=rrrrnr rr4r3r(r'r}r}sLL IIIOOO^c3hDcNQUJJJ19(38:L1MJ JJJJJJr(r}c^ ddlm}|S#t$rtdwxYw)zhGets the notebook REPL context if running inside a Databricks notebook. Returns None otherwise. r get_contextzBCannot access dbruntime, not running inside a Databricks notebook.)!dbruntime.databricks_repl_contextrr]rs r'get_repl_contextrsU AAAAAA{}}     P    s,c<tjd}|sJ tj}|st dn$#t $r}t d|d}~wwxYw|ddd}|S)z|Gets the default Databricks workspace hostname. Raises an error if the hostname cannot be automatically determined. DATABRICKS_HOSTz(context doesn't contain browserHostName.zshost was not set and cannot be automatically inferred. Set environment variable 'DATABRICKS_HOST'. Received error: Nrlzhttp://r)osgetenvrbrowserHostNamer" Exceptionlstriprstrip)rMras r'get_default_hostrs 9& ' 'D   #%%5D M !KLLL M   OKLOO   ;;z " " ) )) 4 4 ; ;C @ @D Ks$= AAActjdx}r|S tj}|st dn$#t $r}t d|d}~wwxYw|S)z|Gets the default Databricks personal access token. Raises an error if the token cannot be automatically determined. DATABRICKS_TOKENz!context doesn't contain apiToken.zyapi_token was not set and cannot be automatically inferred. Set environment variable 'DATABRICKS_TOKEN'. Received error: N)rrrapiTokenr"r)rras r'get_default_api_tokenrsI0111y $&&/  B@AA A B     LHI L L    s$? A  AA c 6eZdZUdZeeZeed< ee Z eed< dZ e eed< dZ e eed< dZe eed< dZe eeefed < dZe eed < dZe ed efed < d Zeed< dZeed< dZeed< dZe eeed< dZe eed< eeZeeefed< dZe eed< e Z!e"ed<GddZ#e$deeeffdZ%e&dddedeeefde efd Z'e&dddedeeefde efd!Z(e&d dde eeefde eeeffd"Z)d#effd$ Z*e$deeeffd%Z+e$de,eeffd&Z-e$defd'Z. d,d(ede eed)e e/d*edef d+Z0xZ1S)-ra Databricks serving endpoint or a cluster driver proxy app for LLM. It supports two endpoint types: * **Serving endpoint** (recommended for both production and development). We assume that an LLM was deployed to a serving endpoint. To wrap it as an LLM you must have "Can Query" permission to the endpoint. Set ``endpoint_name`` accordingly and do not set ``cluster_id`` and ``cluster_driver_port``. If the underlying model is a model registered by MLflow, the expected model signature is: * inputs:: [{"name": "prompt", "type": "string"}, {"name": "stop", "type": "list[string]"}] * outputs: ``[{"type": "string"}]`` If the underlying model is an external or foundation model, the response from the endpoint is automatically transformed to the expected format unless ``transform_output_fn`` is provided. * **Cluster driver proxy app** (recommended for interactive development). One can load an LLM on a Databricks interactive cluster and start a local HTTP server on the driver node to serve the model at ``/`` using HTTP POST method with JSON input/output. Please use a port number between ``[3000, 8000]`` and let the server listen to the driver IP address or simply ``0.0.0.0`` instead of localhost only. To wrap it as an LLM you must have "Can Attach To" permission to the cluster. Set ``cluster_id`` and ``cluster_driver_port`` and do not set ``endpoint_name``. The expected server schema (using JSON schema) is: * inputs:: {"type": "object", "properties": { "prompt": {"type": "string"}, "stop": {"type": "array", "items": {"type": "string"}}}, "required": ["prompt"]}` * outputs: ``{"type": "string"}`` If the endpoint model signature is different or you want to set extra params, you can use `transform_input_fn` and `transform_output_fn` to apply necessary transformations before and after the query. )default_factoryrMrNrNr~r model_kwargstransform_input_fn.r0 databricksrOg temperaturenstop max_tokens extra_paramsrR_clientc eZdZejZdZdS)Databricks.ConfigTN)r8r9r:rforbidextraunderscore_attrs_are_privater3r(r'ConfigrVs '+$$$r(rrch|j|jd}|jr |j|d<|j |j|d<|S)N)rrrr)rrrr)r%paramss r' _llm_paramszDatabricks._llm_paramsZsK +" "  9 '!YF6N ? &#'?F<  r(T)alwaysvrjc|r|drtd|drdS|r|S tjx}r|Std#t$r}td|d}~wwxYw)NrNz-Cannot set both endpoint_name and cluster_id.z"Context doesn't contain clusterId.zuNeither endpoint_name nor cluster_id was set. And the cluster_id cannot be automatically determined. Received error: )r"r clusterIdr)rmrrjras r'set_cluster_idzDatabricks.set_cluster_idfs  ( LMM M O $ 4  H (**441H !EFFF    # ## sAA A0A++A0c|r|drtd|drdS|tdt|dkrtd||S)NrNz6Cannot set both endpoint_name and cluster_driver_port.z  O1$$$&R$$$???$N???r(rSc tjdi||j|jt d|jt jdt|jr4t|j |j |j|j |j |_dS|jr5|jr.t#|j |j |j|j|_dSt d)Nz.Cannot set both extra_params and extra_params.z-JMNN N   * MN"      ;Y."0#2Y DLLL_ !9 >Y.?$($< DLLLV r(c |j|j|j|j|j|j|j|j|j|j |j |j d S)zReturn default params. rMrNr~rrOrrrrrrrRrr6s r'_default_paramszDatabricks._default_paramssUI!//#'#;"1 -+I/ -I   r(c|jSr2)rr6s r'_identifying_paramszDatabricks._identifying_paramss ##r(cdS)zReturn type of llm.rr3r6s r' _llm_typezDatabricks._llm_types |r(r run_managerkwargsc :d|i}|jjr||j||jp|j|||r||d<|jr |jdi|}|j||jS)zAQueries the LLM endpoint with the given prompt and stop sequence.rr)r0r3) rr7updaterrrrr4r0)r%rrrrrs r'_callzDatabricks._calls$,V"4 <  - NN4+ , , ,t(=D,=>>>v  #"GFO  " 9-d-8888G|  d>V WWWr()NN)2r8r9r:r;rrrMr<r=rrrNr r~rrrrrrr0rOrfloatrrrrrdictrrRrrrrr>rrrrrr[rr rrr rrzr{s@r'rrsa//b&6777D#777U+@AAAIsAAA$(M8C=''' !%J $$$*.#--- .2L(4S>*111 .2*111 9=(38"45<<<'NC&&&FK# AsJJJ7 $D(49 $$$ $J $$$3#(5#>#>#>L$sCx.>>>7D(3-&1[]]G "222,,,,,,,, T#s(^   X Y|D)))sDcNx}*)&Y$T222  T#s(^ QT   32 Y~d+++$sCx.!9htCQTH~>V,+ s: c3h   X *$WS#X%6$$$X$3X%):> XXXtCy!X67 X  X XXXXXXXXr()$rrabcrrtypingrrrrr r r langchain_core.callbacksr langchain_core.language_modelsr langchain_core.pydantic_v1r rrrrr__all__rr<rCrFrJrLr}rrrrr3r(r'rs ########????????????????======...... .IsB*T#s(^*****-T#s(^-----8d38n88888ENENENENEN'<ENENENPJJJJJ*?JJJ0  #     #(s&KXKXKXKXKXKXKXKXKXKXr(