昺hyP : d dl Z d dlZd dlmZ d dlmZ d dlmZmZm Z m
Z
m Z m Z d dl
mZmZ d dlmZ d dlmZ d dlmZmZmZmZmZmZ d efd
Zd e e ef d e fd Z dde de de dede
e e ef d e f dZ de de d efdZ! G d dee Z"dS ) N)deepcopy)Path)AnyDictListOptionalSequenceUnion) AgentAction AgentFinish)BaseCallbackHandler) LLMResult)BaseMetadataCallbackHandler flatten_dict hash_string
import_pandas import_spacyimport_textstatreturnc L ddl } n # t $ r t d w xY w| S )zJImport the wandb python package and raise an error if it is not installed.r NzTo use the wandb callback manager you need to have the `wandb` python package installed. Please install it with `pip install wandb`)wandb ImportError)r s t/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain_community/callbacks/wandb_callback.py import_wandbr sL
L
Ls ! json_pathc ~ t | d 5 }t j | }ddd n # 1 swxY w Y |S )zLoad json file to a dictionary.
Parameters:
json_path (str): The path to the json file.
Returns:
(dict): The dictionary representation of the json file.
rN)openjsonload)r fdatas r load_json_to_dictr# ! sx
i y|| Ks 266Ttextcomplexity_metrics visualizenlp
output_dirc i }t }t }t }|rwi d| | d| | d| | d| | d| | d| | d| | d|
| d | | d
| | d |
| d | | d
| | d| | d| | d| | } | | |rA|r>|; || }
|j |
ddd } t- |t/ d| dz } | dd | |j |
ddd }
t- |t/ d | dz }| dd |
| t7 | | t7 | d }| | |S ) a Analyze text using textstat and spacy.
Parameters:
text (str): The text to analyze.
complexity_metrics (bool): Whether to compute complexity metrics.
visualize (bool): Whether to visualize the text.
nlp (spacy.lang): The spacy language model to use for visualization.
output_dir (str): The directory to save the visualization files to.
Returns:
(dict): A dictionary containing the complexity metrics and visualization
files serialized in a wandb.Html element.
flesch_reading_easeflesch_kincaid_grade
smog_indexcoleman_liau_indexautomated_readability_index dale_chall_readability_scoredifficult_wordslinsear_write_formula gunning_fog
text_standardfernandez_huertaszigriszt_pazosgutierrez_polinicrawfordgulpease_indexosmanNdepFT)stylejupyterpagezdep-z.htmlwzutf-8)encodingentzent-)dependency_treeentities) r r r r* r+ r, r- r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 updatedisplacyrenderr r r writeHtmlstr)r$ r% r& r' r( resptextstatr spacytext_complexity_metricsdocdep_outdep_output_pathent_outent_output_pathtext_visualizationss r analyze_textrS / s? ( D H NNE NNE -#
!8#?#?#E#E#
"H$A$A$$G$G#
(--d33#
!("="=d"C"C #
*8+O+OPT+U+U #
+H,Q,QRV,W,W
#
x 7 7 = =#
$X%C%CD%I%I#
8//55#
X 3 3D 9 9#
9 9$ ? ?#
x 7 7 = =#
9 9$ ? ?#
))$// #
h 5 5d ; ; #
X^^D))!#
$
+,,, )S )Z 3c$ii.'' ue$ (
z;}d}}+E+E+OPP S7 3399'BBB.'' ue$ (
z;}d}}+E+E+OPP S7 3399'BBB %zz#o*>*> ? ?
3#7#788
'((( K prompt
generationc t }| dd }| dd }| d| d| dd S )zConstruct an html element from a prompt and a generation.
Parameters:
prompt (str): The prompt.
generation (str): The generation.
Returns:
(wandb.Html): The html element.
z
z
z>:
z"
F)inject)r replacerG )rU rV r formatted_promptformatted_generations r )construct_html_from_prompt_and_generationr] t s}
NNE ~~dF33%--dF;; :: -
rT c e Zd ZdZ d1dee dee dee dee d ee d
ee d ee d ed
ededdf fd
Zde fdZ
de ee f de e de ddfdZ
dede ddfdZdede ddfdZdede ddfdZde ee f d e ee f de ddfd Zd e ee f de ddfd Zdede ddfd Zde ee f d!ede ddfd"Zd#ede ddfd$Zdede ddfd%Zd&ede ddfd'Zd(ede ddfd)Zd*e de de fd+Z de fd,Z d2d.e d/ed(edee dee dee dee d ee d
ee d ee d ee d
ee ddfd0Z xZ S )3WandbCallbackHandlera Callback Handler that logs to Weights and Biases.
Parameters:
job_type (str): The type of job.
project (str): The project to log to.
entity (str): The entity to log to.
tags (list): The tags to log.
group (str): The group to log to.
name (str): The name of the run.
notes (str): The notes to log.
visualize (bool): Whether to visualize the run.
complexity_metrics (bool): Whether to log complexity metrics.
stream_logs (bool): Whether to stream callback actions to W&B
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response using the run.log() method to Weights and Biases.
Nlangchain_callback_demoFjob_typeprojectentitytagsgroupnamenotesr& r% stream_logsr c t } t t t } t || _ || _ || _ || _ || _
|| _ || _ || _
| | _ |
| _ t! j | _ | | j | j | j | j | j
| j | j | _ d}
| |
d g | _ g | _ | | _ || _
| d | _ dS )z Initialize callback handler.)ra rb rc rd re rf rg zDEPRECATION: The `WandbCallbackHandler` will soon be deprecated in favor of the `WandbTracer`. Please update your code to use the `WandbTracer` instead.F)repeaten_core_web_smN)r r r r super__init__ra rb rc rd re rf rg r& r% rh tempfileTemporaryDirectorytemp_dirinitruntermwarncallback_columnsaction_recordsr r' )selfra rb rc rd re rf rg r& r% rh r rK warning __class__s r rm z WandbCallbackHandler.__init__ s=
""4 & 355
,1JJ]L;** -7 -
-
') $&"4 ":: .//rT c $ d | j D S )Nc i | ]}|d S )N ).0ks r
z3WandbCallbackHandler._init_resp.. s 777A4777rT )rt )rv s r
_init_respz WandbCallbackHandler._init_resp s 77!67777rT
serializedpromptskwargsc * | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t | | | |D ]k}t | }||d< | j | | j
| | j r| j
| ldS )zRun when LLM starts. action on_llm_startr N)step
llm_startsstartsr rC r get_custom_callback_metar on_llm_start_recordsappendru rh rr log)rv r r r rI rU prompt_resps r r z!WandbCallbackHandler.on_llm_start s
Q 1 q X~./// L,,--- D1133444 * *F"4..K%+K " % , ,[ 9 9 9 & &{ 3 3 3
* [)))
* *rT tokenc | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j r | j
| dS dS )z#Run when LLM generates a new token.r on_llm_new_token)r r N) r llm_streamsr rC r on_llm_token_recordsr ru rh rr r )rv r r rI s r r z%WandbCallbackHandler.on_llm_new_token s Q A 1EBBCCC D1133444 !((... ""4((( HLL rT responsec 4 | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t |j pi | | |j D ]}|D ]}t | }| t |
| t |j | j
| j | j | j j | j | | j | | j r| j | dS )zRun when LLM ends running.r r
on_llm_end)r% r& r' r( N)r llm_endsendsr rC r
llm_outputr generationsr dictrS r$ r% r& r' rp rf on_llm_end_recordsr ru rh rr r )rv r r rI r rV generation_resps r r z WandbCallbackHandler.on_llm_end s Q
Q X|,--- L!4!:;;<<< D1133444#/ 2 2K )
2
2
"*4.. &&|JOO4E4E'F'FGGG && "+/+B"&. H#'=#5 '..???#**?;;;# 2HLL 111
2 2 2rT errorc F | xj dz
c_ | xj dz
c_ dS )zRun when LLM errors.r Nr errorsrv r r s r on_llm_errorz!WandbCallbackHandler.on_llm_error % Q q rT inputsc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t | | | |d }t |t rmt | }||d< | j
| | j | | j
r | j | dS dS t |t r|D ]{}t | }| | | j
| | j | | j
r| j | |dS t# d )z Run when chain starts running.r r on_chain_startinputz Unexpected data format provided!N)r chain_startsr r rC r r
isinstancerH r on_chain_start_recordsr ru rh rr r list
ValueError)rv r r r rI chain_input
input_respinps r r z#WandbCallbackHandler.on_chain_start$ s
Q Q q X /0111 L,,--- D1133444 Wo k3 ' ' A!$ J"-Jw ' . .z : : : & &z 2 2 2
) Z(((((
)
)
T
*
* A"
-
- %d^^
!!#&&&+22:>>>#**:666# -HLL,,,
-
- ?@@ @rT outputsc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d d | | | j | | j | | j r | j
| dS dS )z Run when chain ends running.r on_chain_endoutput)r r N) r
chain_endsr r rC r on_chain_end_recordsr ru rh rr r )rv r r rI s r r z!WandbCallbackHandler.on_chain_endE s Q 1 Q ~'(:KLLMMM D1133444 !((... ""4((( HLL rT c F | xj dz
c_ | xj dz
c_ dS )zRun when chain errors.r Nr r s r on_chain_errorz#WandbCallbackHandler.on_chain_errorT r rT input_strc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d | t | | | | j | | j | | j
r | j | dS dS )z Run when tool starts running.r
on_tool_start)r r N)
r tool_startsr r rC r r on_tool_start_recordsr ru rh rr r )rv r r r rI s r r z"WandbCallbackHandler.on_tool_startY s
Q A q YGGHHH L,,--- D1133444 "))$/// ""4((( HLL rT r c | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j r | j
| dS dS )zRun when tool ends running.r on_tool_end)r r N) r tool_endsr r rC r on_tool_end_recordsr ru rh rr r )rv r r rI s r r z WandbCallbackHandler.on_tool_endk s Q ! Q }??@@@ D1133444 ''--- ""4((( HLL rT c F | xj dz
c_ | xj dz
c_ dS )zRun when tool errors.r Nr r s r
on_tool_errorz"WandbCallbackHandler.on_tool_errorz r rT r$ c | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j r | j
| dS dS )z+
Run when agent is ending.
r on_text)r r$ N) r text_ctrr rC r on_text_recordsr ru rh rr r )rv r$ r rI s r r z WandbCallbackHandler.on_text s
Q
y$77888 D1133444 ##D))) ""4((( HLL rT finishc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|j d |j d | | | j | | j
| | j r | j | dS dS )z Run when agent ends running.r on_agent_finishr )r r r N)
r
agent_endsr r rC
return_valuesr r on_agent_finish_recordsr ru rh rr )rv r r rI s r r z$WandbCallbackHandler.on_agent_finish s Q 1 Q + .x8 z
D1133444 $++D111 ""4((( HLL rT r c | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|j |j |j d | | | j
| | j
| | j r | j
| dS dS )zRun on agent action.r on_agent_action)r tool
tool_inputr N)r r r r rC r r r r on_agent_action_recordsr ru rh rr )rv r r rI s r r z$WandbCallbackHandler.on_agent_action s Q A q + $ / z
D1133444 $++D111 ""4((( HLL rT c t }| | j }| | j }|g d d ddid }g }g }| j rg d}| j rddg}|g d |z |z d d
d d d }| ||gd }|d
d g d d |d< |S )z=Create a dataframe with all the information from the session.)r r rf r )axisr prompt_step)r* r+ r, r- r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 rA rB )r r$ token_usage_total_tokenstoken_usage_prompt_tokens token_usage_completion_tokens output_stepr )r r$ r c : t | d | d S )Nr r )r] )rows r zBWandbCallbackHandler._create_session_analysis_df.. s AIH
rT chat_html)
r DataFramer r dropnarenamer% r& concatapply) rv pdon_llm_start_records_dfon_llm_end_records_dfllm_input_prompts_dfcomplexity_metrics_columnsvisualizations_columnsllm_outputs_dfsession_analysis_dfs r _create_session_analysis_dfz0WandbCallbackHandler._create_session_analysis_df so
__"$,,t/H"I"I " T-D E E
$$?$?$? @
VV^^
VV]+!V
4
4
&("!# " * * * && > E&7%D "
" -- ))
VV^^
V]H==AV
F
F !ii)=~(NUVi W W+>
!,
%
K( #"rT Tlangchain_assetresetc
` t }
t }| |
| j }| | }| j ||d |r^t | j j
d }| dd }| |d | |d |
| | t | t! | |_ n# t$ $ rN | | | t | t! | |_ Y nAt( $ r5}t+ d t+ t- | Y d
}~nd
}~ww xY w| j | |s|rF| j | j | |rq| |r|n| j |r|n| j |r|n| j |r|n| j |r|n| j | r| n| j
|
r|
n| j! | r| n| j" | r| n| j# d
S d
S ) aS Flush the tracker and reset the session.
Args:
langchain_asset: The langchain asset to save.
reset: Whether to reset the session.
finish: Whether to finish the run.
job_type: The job type.
project: The project.
entity: The entity.
tags: The tags.
group: The group.
name: The name.
notes: The notes.
visualize: Whether to visualize.
complexity_metrics: Whether to compute complexity metrics.
Returns:
None
) dataframe)ru session_analysisz
model.jsonmodel)rf typeru )rf r zCould not save model.N) ra rb rc rd re rf rg r& r% )$r r Tabler ru r rr r r rp rf Artifactaddsaveadd_filerH r# metadatar
save_agentNotImplementedErrorprintrepr log_artifactr cleanupreset_callback_metarm ra rb rc rd re rg r& r% )rv r r r ra rb rc rd re rf rg r& r% r r action_records_tablesession_analysis_tablelangchain_asset_pathmodel_artifactes r
flush_trackerz"WandbCallbackHandler.flush_tracker s D __ ${{R\\$BU5V5V{ W W !&6688 "- "
"
"6$:
2#'
(:L#I#I "^^w^ G GN 3:J K K K 5S %TNS
S S S S S S S SrT r_ )TTNN)#r rn copyr pathlibr typingr r r r r r
langchain_core.agentsr r langchain_core.callbacksr
langchain_core.outputsr #langchain_community.callbacks.utilsr r r r r r r rH r r# r
rS r] r_ r{ rT r r s! = = = = = = = = = = = = = = = = : : : : : : : : 8 8 8 8 8 8 , , , , , , c sDy!1 d $-1 B B
B B B
B
sDy)* B
B B B BJc s s 4} } } } }68K } } } } }rT