蘺h<ddlZddlmZmZddlmZddlmZmZm Z m Z m Z ddl m Z ddlmZddlmZmZddlmZdd lmZmZdd lmZdd lmZdd lmZmZdd lm Z ej!e"Z#GddeeZ$Gdde$Z%Gdde$Z&Gdde$Z'Gdde$Z(GddeZ)dS)N)ABCabstractmethod)islice)AnyDictIterableListOptional) get_client)BaseLanguageModel) BaseMessageget_buffer_string)BasePromptTemplate) BaseModelField)LLMChain)BaseChatMemory)ENTITY_EXTRACTION_PROMPTENTITY_SUMMARIZATION_PROMPT)get_prompt_input_keyc eZdZdZed dedeedeefdZededeeddfdZededdfd Z edede fd Z ed d Z dS)BaseEntityStorez%Abstract base class for Entity store.NkeydefaultreturncdS)zGet entity value from store.Nselfrrs _/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain/memory/entity.pygetzBaseEntityStore.get  valuecdS)zSet entity value in store.Nrrrr$s r setzBaseEntityStore.setr"r#cdS)zDelete entity value from store.Nrrrs r deletezBaseEntityStore.delete$r"r#cdS)z Check if entity exists in store.Nrr)s r existszBaseEntityStore.exists)r"r#cdS)zDelete all entities from store.Nrrs r clearzBaseEntityStore.clear.r"r#NrN) __name__ __module__ __qualname____doc__rstrr r!r'r*boolr,r/rr#r rrs//  s Xc] hsm   ^  s 8C= T   ^  # $   ^  # $   ^    ^   r#rceZdZUdZiZeeeefed<d dedeedeefdZ dedeeddfd Z deddfd Z dede fd Z dd ZdS)InMemoryEntityStorezIn-memory Entity store.storeNrrrc8|j||Sr0)r:r!rs r r!zInMemoryEntityStore.get9sz~~c7+++r#r$c||j|<dSr0r:r&s r r'zInMemoryEntityStore.set<s 3r#c|j|=dSr0r=r)s r r*zInMemoryEntityStore.delete?s JsOOOr#c||jvSr0r=r)s r r,zInMemoryEntityStore.existsBsdj  r#c4|jSr0)r:r/r.s r r/zInMemoryEntityStore.clearEsz!!!r#r0r1)r2r3r4r5r:rr6r __annotations__r!r'r*r7r,r/rr#r r9r94s!!&(E4Xc]" #(((,,s,Xc],hsm,,,, s 8C= T    #$!#!$!!!!""""""r#r9ceZdZdZ ddeded ed ed eed eed edeffd Ze defdZ ddedeedeefdZ dedeeddfdZ deddfdZ dedefdZddZxZS)UpstashRedisEntityStorezUpstash Redis backed Entity store. Entities get a TTL of 1 day by default, and that TTL is extended by 3 days every time the entity is read back. r memory_storeQ session_idurltoken key_prefixttl recall_ttlargskwargsc< ddlm} n#t$rtdwxYwtj|i| | |||_n*#t $rtdYnwxYw||_ ||_ ||_ |p||_ dS)Nr)RediszbCould not import upstash_redis python package. Please install it with `pip install upstash_redis`.)rIrJz.Upstash Redis instance could not be initiated.) upstash_redisrQ ImportErrorsuper__init__ redis_client ExceptionloggererrorrHrKrLrM) rrHrIrJrKrLrMrNrOrQ __class__s r rUz UpstashRedisEntityStore.__init__Ps  + + + + + + +   F   $)&))) K %#U ; ; ;D   K K K LLI J J J J J K%$$+s $A$A;:A;rc$|jd|jSN:rKrHr.s r full_key_prefixz'UpstashRedisEntityStore.full_key_prefixo/55DO555r#Nrc |j|jd||jp|pd}td|jd|d|d|S)Nr]exrDzUpstash Redis MEM get '': ''rVgetexr_rMrXdebugrrrress r r!zUpstashRedisEntityStore.getss{   # #t';$C$Cc$C$C # X X    Ut/CUUcUUsUUUVVV r#r$c |s||S|j|jd|||jt d|jd|d|d|jdS)Nr]rbzRedis MEM set 'rd' EX r*rVr'r_rLrXrhr&s r r'zUpstashRedisEntityStore.set| $;;s## # !5====uRRR Td2 T TS T Te T T$( T T     r#cN|j|jd|dSr\rVr*r_r)s r r*zUpstashRedisEntityStore.delete0   D$8!@!@3!@!@AAAAAr#cR|j|jd|dkSNr]rVr,r_r)s r r,zUpstashRedisEntityStore.exists/ ''4+?(G(G#(G(GHHAMMr#cxdtdtffd }|d}|dkr|||dkdSdS)Ncursorrcrj|jd\}}jj||S)N:*)rVscanr_r*)rxkeys_to_deleters r scan_and_deletez6UpstashRedisEntityStore.clear..scan_and_deletesK%)%6%;%;4/333&& "FN %D  $n 5 5Mr#r)int)rr}rxs` r r/zUpstashRedisEntityStore.clearsl C C      !##kk OF # # #kkkkkkr#)rrDrDrErFrGr0r1)r2r3r4r5r6r r~rrUpropertyr_r!r'r*r7r,r/ __classcell__rZs@r rCrCIs$()$4,,,, ,  , c] ,SM,,,,,,,,>6666X6sXc]hsm s 8C= T    B#B$BBBBN#N$NNNN $ $ $ $ $ $ $ $r#rCcReZdZUdZeed<dZeed<dZeed<dZ e e ed<d Z e e ed < dded edede e d e e d edeffd Z edefdZddede ede efdZdede eddfdZdeddfdZdedefdZddZxZS)RedisEntityStorezRedis-backed Entity store. Entities get a TTL of 1 day by default, and that TTL is extended by 3 days every time the entity is read back. rVrrHrErKrFrLrGrMredis://localhost:6379/0rIrNrOcX ddl}n#t$rtdwxYwtj|i| t |d|_n6#|jj$r$} t | Yd} ~ nd} ~ wwxYw||_ ||_ ||_ |p||_ dS)NrzRCould not import redis python package. Please install it with `pip install redis`.T) redis_urldecode_responses)redisrSrTrUr rV exceptionsConnectionErrorrXrYrHrKrLrM) rrHrIrKrLrMrNrOrrYrZs r rUzRedisEntityStore.__init__s  LLLL   >   $)&))) *S4 P P PD  /   LL         %$$+s"AB %BB rc$|jd|jSr\r^r.s r r_z RedisEntityStore.full_key_prefixr`r#Nrc |j|jd||jp|pd}td|jd|d|d|S)Nr]rbrDzREDIS MEM get 'rdrerfris r r!zRedisEntityStore.gets{   # #t';$C$Cc$C$C # X X    Mt';MMcMMsMMMNNN r#r$c |s||S|j|jd|||jt d|jd|d|d|jdS)Nr]rbzREDIS MEM set 'rdrlrmr&s r r'zRedisEntityStore.setrnr#cN|j|jd|dSr\rpr)s r r*zRedisEntityStore.deleterqr#cR|j|jd|dkSrsrur)s r r,zRedisEntityStore.existsrvr#cdttdtdttfd}||j|jddD]}|jj|dS)Niterable batch_sizerc3Kt|}tt||x}r%|Vtt||x}#dSdSr0)iterlistr)rriteratorbatchs r batchedz'RedisEntityStore.clear..batchedsnH~~Hx < <===%   x < <===%     r#rzi)rrr~rV scan_iterr_r*)rrkeybatchs r r/zRedisEntityStore.clears hsm  #         ' '4+?(C(C(C D Dc   0 0H %D  $h / / / 0 0r#)rrrErFrGr0r1)r2r3r4r5rrArHr6rKrLr r~rMrUrr_r!r'r*r7r,r/rrs@r rrs J$J$$$%C#%%% 0J 000$-()$4 ,,,, , c] , SM ,,,,,,,,<6666X6sXc]hsm s 8C= T    B#B$BBBBN#N$NNNN 0 0 0 0 0 0 0 0r#rc eZdZUdZdZeed<dZeed< ddedededed ef fd Z e d efd Z ddZ ddede ed e efdZdede ed d fdZded d fdZded efdZddZxZS)SQLiteEntityStorezSQLite-backed Entity storerrHrE table_name entities.dbdb_filerNrOc ddl}n#t$rtdwxYwtj|i||||_||_||_|dS)NrzVCould not import sqlite3 python package. Please install it with `pip install sqlite3`.) sqlite3rSrTrUconnectconnrHr_create_table_if_not_exists)rrHrrrNrOrrZs r rUzSQLiteEntityStore.__init__s  NNNN   @   $)&)))OOG,, $$ ((*****s"rc$|jd|jS)N_)rrHr.s r full_table_namez!SQLiteEntityStore.full_table_namer`r#Ncd|jd}|j5|j|ddddS#1swxYwYdS)Nz( CREATE TABLE IF NOT EXISTS zZ ( key TEXT PRIMARY KEY, value TEXT ) rrexecute)rcreate_table_querys r rz-SQLiteEntityStore._create_table_if_not_exists s (,(<    Y 2 2 I  0 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ;??rcd|jd}|j||f}|}| |d}|S|S)Nz+ SELECT value FROM # WHERE key = ? rrrrfetchone)rrrqueryrxresultr$s r r!zSQLiteEntityStore.getsa &    ""53&11""  1IELr#r$c|s||Sd|jd}|j5|j|||fddddS#1swxYwYdS)Nz$ INSERT OR REPLACE INTO z0 (key, value) VALUES (?, ?) )r*rrr)rrr$rs r r'zSQLiteEntityStore.set!s $;;s## # $($8   Y 3 3 I  ec5\ 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3sAAAcd|jd}|j5|j||fddddS#1swxYwYdS)N DELETE FROM rr)rrrs r r*zSQLiteEntityStore.delete+s -   Y - - I  ecV , , , - - - - - - - - - - - - - - - - - -s=AAcd|jd}|j||f}|}|duS)Nz' SELECT 1 FROM z7 WHERE key = ? LIMIT 1 r)rrrrxrs r r,zSQLiteEntityStore.exists3sS &    ""53&11""T!!r#cd|jd}|j5|j|ddddS#1swxYwYdS)Nrz r)rrs r r/zSQLiteEntityStore.clear>s -   Y % % I  e $ $ $ % % % % % % % % % % % % % % % % % %r)rrrEr1r0)r2r3r4r5rHr6rArrrUrrrr r!r'r*r7r,r/rrs@r rrs$$J$J$$$$$( ++++ +  +  ++++++,6666X62222  s Xc] hsm    3s38C=3T3333-#-$---- "# "$ " " " "%%%%%%%%r#rcteZdZUdZdZeed<dZeed<eed<e Z e ed<e Z e ed<gZeeed <d Zeed <d Zeed <eeZeed<edeefdZedeefdZdeeefdeeeffdZdeeefdeeefddffd ZddZxZ S)ConversationEntityMemorya<Entity extractor & summarizer memory. Extracts named entities from the recent chat history and generates summaries. With a swappable entity store, persisting entities across conversations. Defaults to an in-memory entity store, and can be swapped out for a Redis, SQLite, or other entity store. Human human_prefixAI ai_prefixllmentity_extraction_promptentity_summarization_prompt entity_cachekhistorychat_history_key)default_factory entity_storerc|jjS)zAccess chat memory messages.) chat_memorymessagesr.s r bufferzConversationEntityMemory.bufferas((r#cd|jgS)zMWill always return list of memory variables. :meta private: entities)rr.s r memory_variablesz)ConversationEntityMemory.memory_variablesfs D122r#inputscDt|j|j}|jt ||j}n|j}t |j|j dzd|j |j }| |||}| dkrg}nd| dD}i}|D] }|j|d ||<!||_|jr|j|j dzd} n|} |j| d |iS) ae Returns chat history and all generated entities with summaries if available, and updates or clears the recent entity cache. New entity name can be found when calling this method, before the entity summaries are generated, so the entity cache values may be empty if no entity descriptions are generated yet. rpromptNrr)rinputNONEc6g|]}|Sr)strip).0ws r zBConversationEntityMemory.load_memory_variables..s ===a ===r#,rDr)rrr input_keyrrrrrrrpredictrsplitrr!rreturn_messagesr) rrchainprompt_input_key buffer_stringoutputrentity_summariesentityrs r load_memory_variablesz.ConversationEntityMemory.load_memory_variablesnsgTXd.KLLL > !3FD>V # #HH>=6<<+<+<===H I IF'+'8'<' !3FD(:DcN(:t(:(:(:(:(:(:T""""""""r#r)*loggingabcrr itertoolsrtypingrrrr r #langchain_community.utilities.redisr langchain_core.language_modelsr langchain_core.messagesr rlangchain_core.promptsrlangchain_core.pydantic_v1rrlangchain.chains.llmrlangchain.memory.chat_memoryrlangchain.memory.promptrrlangchain.memory.utilsr getLoggerr2rXrr9rCrrrrr#r rsg########66666666666666::::::<<<<<<BBBBBBBB55555577777777))))))777777877777  8 $ $     i   :"""""/"""*K$K$K$K$K$oK$K$K$\P0P0P0P0P0P0P0P0fY%Y%Y%Y%Y%Y%Y%Y%xW"W"W"W"W"~W"W"W"W"W"r#