ݘhYddlmZmZddlmZddlmZmZddlZ ddl Z ddl m Z ddl mZmZd Zd ZGd d eZGd deZGddeZGddZdS))ABCabstractmethod)UserDict)OptionalUnionN)add_start_docstrings) ConstraintConstraintListStatea Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) next_scores (`torch.FloatTensor` of shape `(batch_size, 2 * num_beams)`): Current scores of the top `2 * num_beams` non-finished beam hypotheses. next_tokens (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): `input_ids` of the tokens corresponding to the top `2 * num_beams` non-finished beam hypotheses. next_indices (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): Beam indices indicating to which beam hypothesis the `next_tokens` correspond. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, list[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. beam_indices (`torch.LongTensor`, *optional*): Beam indices indicating to which beam hypothesis each token correspond. group_index (`int`, *optional*): The index of the group of beams. Used with [`~PreTrainedModel.group_beam_search`]. Return: `UserDict`: A dictionary composed of the fields as defined above: - **next_beam_scores** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Updated scores of all non-finished beams. - **next_beam_tokens** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Next tokens to be added to the non-finished beam_hypotheses. - **next_beam_indices** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Beam indices indicating to which beam the next tokens shall be added. a Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) final_beam_scores (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The final scores of all non-finished beams. final_beam_tokens (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The last tokens to be added to the non-finished beam_hypotheses. final_beam_indices (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The beam indices indicating to which beam the `final_beam_tokens` shall be added. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, list[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. Return: `torch.LongTensor` of shape `(batch_size * num_return_sequences, sequence_length)`: The generated sequences. The second dimension (sequence_length) is either equal to `max_length` or shorter if all batches finished early due to the `eos_token_id`. c"eZdZdZeeedejdej dejdejde ej f dZ eee dejdej dejdejdedejf d Zd S) BeamScorerz Abstract base class for all beam scorers that are used for [`~PreTrainedModel.beam_search`] and [`~PreTrainedModel.beam_sample`]. input_ids next_scores next_tokens next_indicesreturnc tdNzThis is an abstract method.NotImplementedError)selfrrrrkwargss k/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/transformers/generation/beam_search.pyprocesszBeamScorer.processas""?@@@ max_lengthc tdrr)rrrrrrrs rfinalizezBeamScorer.finalizems""?@@@rN)__name__ __module__ __qualname____doc__rr PROCESS_INPUTS_DOCSTRINGtorch LongTensor FloatTensortupleTensorrFINALIZE_INPUTS_DOCSTRINGintrrrrr[s- 233A#A&A% A & A u| AAA43^A344 A# A& A% A & A  A   A A A54^ A A ArrceZdZdZ d dededejd eed ee e e fd eed eed eefdZ e de fdZ d!dejdejdejdejdee eejfdee eeeejfdeejdeedeedee ejffdZ d"dejdejdejdejd edee eejfdee eeeejfdeejdeedeejfdZdS)#BeamSearchScorera [`BeamScorer`] implementing standard beam search decoding. Adapted in part from [Facebook's XLM beam search code](https://github.com/facebookresearch/XLM/blob/9e6f6814d17be4fe5b15f2e6c43eb2b2d76daeb4/src/model/transformer.py#L529). Reference for the diverse beam search algorithm and implementation [Ashwin Kalyan's DBS implementation](https://github.com/ashwinkalyan/dbs/blob/master/dbs/beam_utils.lua) Args: batch_size (`int`): Batch Size of `input_ids` for which standard beam search decoding is run in parallel. num_beams (`int`): Number of beams for beam search. device (`torch.device`): Defines the device type (*e.g.*, `"cpu"` or `"cuda"`) on which this instance of `BeamSearchScorer` will be allocated. length_penalty (`float`, *optional*, defaults to 1.0): Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences, while `length_penalty` < 0.0 encourages shorter sequences. do_early_stopping (`bool` or `str`, *optional*, defaults to `False`): Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values: `True`, where the generation stops as soon as there are `num_beams` complete candidates; `False`, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates; `"never"`, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm). num_beam_hyps_to_keep (`int`, *optional*, defaults to 1): The number of beam hypotheses that shall be returned upon calling [`~transformers.BeamSearchScorer.finalize`]. num_beam_groups (`int`, *optional*, defaults to 1): Number of groups to divide `num_beams` into in order to ensure diversity among different groups of beams. See [this paper](https://huggingface.co/papers/1610.02424) for more details. max_length (`int`, *optional*): The maximum length of the sequence to be generated. ?Fr N batch_size num_beamsdevicelength_penaltydo_early_stoppingnum_beam_hyps_to_keepnum_beam_groupsrc J|_|_|_|_|_|_jjz_d_fdt|jzD_ tj dt|jzDtj j_ t|tr|dkrt!d|dt|tr||ks ||zdkrt!d |d |d dS) NFcTg|]$}tjjj%S)r1r3early_stoppingr)BeamHypotheses group_sizer3r4.0_rrs r z-BeamSearchScorer.__init__..sN    /#2#5%       rcg|]}dSFr,r>r?s rr@z-BeamSearchScorer.__init__..s E E EqU E E Erdtyper2r A`num_beams` has to be an integer strictly greater than 1, but is G. For `num_beams` == 1, one should make use of `greedy_search` instead.r`num_beam_groups` has to be an integer smaller or equal than `num_beams` and `num_beams` has to be divisible by `num_beam_groups`, but is  with `num_beams` being .)r1r2r3r4r5r6r<_is_initrange _beam_hypsr%tensorbool_done isinstancer+ ValueError) rr0r1r2r3r4r5r6rs ` `r__init__zBeamSearchScorer.__init__s# ,!2%:"..D,@@      :(<<==   \ E EE*t/C"CDD E E EUZ`d`k    )S)) Y!^^CT]CCC  /3// Oi4O4OU^apUptuUuUuq;Jqqdmqqq VvUurrc4|jSNrPallrs ris_donezBeamSearchScorer.is_donez~~rrrrrr pad_token_id eos_token_id beam_indices group_indexdecoder_prompt_lenc |jddz} t|j|jz} | |jd|jzkrW|jdkr&t d|jdd|jdt d|jdd|jd|j} tj| |jf|j | } tj| |jf|j | }tj| |jf|j | }|Ft|tj s,t|tr|g}tj |}t| D]}||jz|z}|j|rj|jt|j|krt d |jd ||t d d| |ddf<|||ddf<d||ddf<d}t#t%||||||D]\}\}}}||jz|z}|||vrs||jk}|r<|||}||fz}nd}|j|||||| | z n|| ||f<||||f<||||f<|dz }||jkrn||jkr-t d|jd||d|d||d |j|pK|j|||| | |j|<t1| d|d|ddS)Nr rA group beam size of 0 is used as the input, but a group beam size of is expected by the beam scorer.A beam size of * is used as the input, but a beam size of rD#Batch can only be done if at least  beams have been generatedMGenerated beams >= num_beams -> eos_token_id and pad_token have to be definedr] generated_lenAt most tokens in can be equal to `eos_token_id: `. Make sure  are corrected.next_beam_scoresnext_beam_tokensnext_beam_indices)shapelenrMr6r<rRr2r%zerosrErQr)r+rNrLrPr1 enumeratezipitemaddclonerYmaxrview)rrrrrr[r\r]r^r_cur_lenr0r2rrrsrt batch_idxbatch_group_idxbeam_idxbeam_token_rank next_token next_score next_indexbatch_beam_idx&is_beam_token_worse_than_top_num_beams beam_indexs rrzBeamSearchScorer.processs/"%)))T-AA )/!,? @ @#a'' QIOA,>QQ#QQQ !Iioa&8IIIII ! ; DO'DKL]flmmm ; DO'DKL]flmmm!KT_(E\M_hnooo  #J|U\,R,R #,,, . ,~  < 55Lz**9 9 I'$*>>LOz/* >C(H$I$III$%u4>%u%u%uvvv'<+?$%tuuu12 AAA.1= AAA.23!)QQQ,/HIRK *K ,BLQZD[\\JJ  E!E*j*"+T_!2C%C %) OO488!.17799"))%/&-0B&B 9=G$Y%89y3I[[[ +/*_*E+YhIiIqIqI&**,,1133W>PJJDJ ' '$4$9$9"$=$=$4$9$9"$=$=%6%;%;B%?%?     rfinal_beam_scoresfinal_beam_tokensfinal_beam_indicesc t|j|jz} |Ft|tjs,t|t r|g}t j|}t|jD]\} } |j | rt|j D]f} | |j z| z}|| }||}|||nd}|j d| z }| ||||g|| |jz}g}g}t j| |jz|jtj}t| D]}|j||jz|dz|jz}d|D}t)|d}t|jD]}|}|d}|d}|d }t|||j|z|z<||||||||jz|z<| dz} |t1| |n| }!|| |jz|!}"t|dkr'|d|| |jz|!}#nd}#| | kr&|t3d |"||#|#dtt7||D]`\}\}$}%|$|"|d||f<|#(t j|%|#|dt|%f<|||!kr|d|"|||f<at9|"||#d S) Nrarjr2rEr c&g|]}|jD]}|Sr,)beams)r>beam_hypbeams rr@z-BeamSearchScorer.finalize..ks)bbbS[Sabb4tbbbbrc|dSNrr,xs rz+BeamSearchScorer.finalize..ls !rkeyrr `pad_token_id` has to be defined sequencessequence_scoresr])rvrMr6rQr%r)r+rNrxrPrLr<rzrur{newr5rwr2float32sortedpopappendr}minrRfill_ryr)&rrrrrrr[r\r]r_r0rrindex_per_groupr final_score final_tokensrrk sent_lengthsbest best_indices best_scoresibeam_hyps_in_batchcandidate_beams sorted_hypsjbest_hyp_tuple best_scorebest_hyp best_indexsent_lengths_max sent_max_lendecodedindiceshypobest_idxs& rrzBeamSearchScorer.finalize@s`))T-AA  #J|U\,R,R #,,, . ,~  < 55L*34?)C)C n n %OXz/* $)#9#9 n n!04?!B_!T/?DDFF (8 =I=U\.99[_ , 22 69K K  \;Z_l mmmm  n!}}Z$2L%LMM  k*t/I"IRVR]ejersss z"" M MA!%T5I1IQQRUVZVjLj1j!k bb4FbbbO nnEEEK4566 M M!,!2!2+A. )!,+A. CFx== T7!;a?@ H%%%##J///BL A ::Q>?? M (++--2244q8 > Ah,0GA(a(( )".3l8.D.D?S]]?*+A--.:1o<?*+$#. '     rr/Fr r N)NNNrrNNNr)r r!r"r#r+r%r2rfloatrrOstrrSpropertyrYr&r'r)listdictrr(rr,rrr.r.{s$$V+.8=/0)*$(//// / ! / $E$)$45 / (}/"#/SM////b    X <@FJ37%&,-g g #g &g % g & g uS%,%678 g uS$s)U\%ABCg u/0g c]g %SMg  c5< g g g g `<@FJ37,-` ` #` !,` !+ ` ", `  ` uS%,%678` uS$s)U\%ABC` u/0` %SM`  u ` ` ` ` ` ` rr.c eZdZdZ d*dededeed ejd e e d e e e e fd e ed e ede efdZede fdZdZdZ d+dejdejdejdejdejde e eejfde e eeeejfde ejde edeejfdZ d,dedejd ejd!ejd"ejd#ejd$e fd%Z d+dejd&ejd'ejd(ejdede e eejfde e eeeejfde ejde edeejfd)ZdS)-ConstrainedBeamSearchScorera$ [`BeamScorer`] implementing constrained beam search decoding. Args: batch_size (`int`): Batch Size of `input_ids` for which standard beam search decoding is run in parallel. num_beams (`int`): Number of beams for beam search. constraints (`list[Constraint]`): A list of positive constraints represented as `Constraint` objects that must be fulfilled in the generation output. For more information, the documentation of [`Constraint`] should be read. device (`torch.device`): Defines the device type (*e.g.*, `"cpu"` or `"cuda"`) on which this instance of `BeamSearchScorer` will be allocated. length_penalty (`float`, *optional*, defaults to 1.0): Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences, while `length_penalty` < 0.0 encourages shorter sequences. do_early_stopping (`bool` or `str`, *optional*, defaults to `False`): Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values: `True`, where the generation stops as soon as there are `num_beams` complete candidates; `False`, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates; `"never"`, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm). num_beam_hyps_to_keep (`int`, *optional*, defaults to 1): The number of beam hypotheses that shall be returned upon calling [`~transformers.BeamSearchScorer.finalize`]. num_beam_groups (`int`, *optional*, defaults to 1): Number of groups to divide `num_beams` into in order to ensure diversity among different groups of beams. See [this paper](https://huggingface.co/papers/1610.02424) for more details. max_length (`int`, *optional*): The maximum length of the sequence to be generated. r/Fr Nr0r1 constraintsr2r3r4r5r6rc 8 |_|_|_|_|_|_jjz_|_d_ fdt|D_ tj dt|Dtj j_t|t r|dkrt#d|dt|t r||ks ||zdkrt#d |d |d dS) NFcTg|]$}tjjj%Sr9)r;r1r3r4r=s rr@z8ConstrainedBeamSearchScorer.__init__..sN    .#2#5%       rcg|]}dSrBr,rCs rr@z8ConstrainedBeamSearchScorer.__init__..s"D"D"DQ5"D"D"DrrDr rFrGrrHrIrJ)r1r2r3r4r5r6r<rrKrLrMr%rNrOrPrQr+rR) rr0r1rr2r3r4r5r6rs ` `rrSz$ConstrainedBeamSearchScorer.__init__s# ,!2%:"..D,@@&      :&&   \"D"D% 2C2C"D"D"DEJ_c_jkkk )S)) Y!^^CT]CCC  /3// Oi4O4OU^apUptuUuUuq;Jqqdmqqq VvUurrc4|jSrUrVrXs rrYz#ConstrainedBeamSearchScorer.is_donerZrc:fdt|DS)NcLg|] }tdjD!S)c6g|]}|Sr,)copy)r> constraints rr@zQConstrainedBeamSearchScorer.make_constraint_states...s"$Z$Z$Z:Z__%6%6$Z$Z$Zr)r r)r>r?rs rr@zFConstrainedBeamSearchScorer.make_constraint_states..s5nnn`a#$Z$ZIY$Z$Z$Z[[nnnr)rL)rns` rmake_constraint_statesz2ConstrainedBeamSearchScorer.make_constraint_statess'nnnnejklememnnnnrcp|dd}|||jS)Nr r)rreset completed)rsequence new_states rcheck_completes_constraintsz7ConstrainedBeamSearchScorer.check_completes_constraintss5//2215 !!!""rrrrrrscores_for_all_vocabr[r\r]r_c |jddz} t|j} | |jd|jzkrW|jdkr&t d|jdd|jdt d|jdd|jd|j} tj| |jf|j | } tj| |jf|j | }tj| |jf|j | }|Ft|tj s,t|tr|g}tj |}t|jD]^\}}|j|r_|jt|krt d |jd ||t d d| |d d f<|||d d f<d||d d f<rd}tt#||||||D]\}\}}}||jz|z}|||vr||jk}|r<|||}|rY|||}||fz}nd }|||||| | z n|| ||f<||||f<||||f<|dz }||jkrn||||| |||||\}}}|| |<|||<|||<||jkr-t d|jd||d|d||d |j|p@|||| | |j|<`t5| d|d|ddS)al Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) next_scores (`torch.FloatTensor` of shape `(batch_size, 2 * num_beams)`): Current scores of the top `2 * num_beams` non-finished beam hypotheses. next_tokens (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): `input_ids` of the tokens corresponding to the top `2 * num_beams` non-finished beam hypotheses. next_indices (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): Beam indices indicating to which beam hypothesis the `next_tokens` correspond. scores_for_all_vocab (`torch.FloatTensor` of shape `(batch_size * num_beams, sequence_length)`): The scores of all tokens in the vocabulary for each of the beam hypotheses. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, list[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. beam_indices (`torch.LongTensor`, *optional*): Beam indices indicating to which beam hypothesis each token correspond. decoder_prompt_len (`int`, *optional*): The length of prompt that is included in the input to decoder. Return: `UserDict`: A dictionary composed of the fields as defined above: - **next_beam_scores** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Updated scores of all non-finished beams. - **next_beam_tokens** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Next tokens to be added to the non-finished beam_hypotheses. - **next_beam_indices** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Beam indices indicating to which beam the next tokens shall be added. rar rrbrcrdrerfrDNrgrhrirjrlrmrnrorprq)rurvrMr<r6rRr2r%rwrErQr)r+rNrxrPr1ryrzrtolistr{r|step_sentence_constraintrYr}rr~)rrrrrrr[r\r]r_rr0r2rrrsrtrrrrrrrrrcompletes_constraintr new_scores new_tokens new_indicess rrz#ConstrainedBeamSearchScorer.processs5h/"%))) )/!,? @ @#a'' QIOA,>QQ#QQQ !Iioa&8IIIII ! ; DO'DKL]flmmm ; DO'DKL]flmmm!KT_(E\M_hnooo  #J|U\,R,R #,,, . ,~  < 55L#,T_#=#=H H  Ixz)$ >CMM11$%u4>%u%u%uvvv'<+?$%tuuu12 AAA.1= AAA.23!)QQQ,/HIRK *K ,BLQZD[\\JJ" " E!E*j*"+T_!y3I[[[ %)Jy$9%X=M=MI&**,,1133W>P>>DJy ! !$4$9$9"$=$=$4$9$9"$=$=%6%;%;B%?%?     rr vocab_scoressent_beam_scoressent_beam_tokenssent_beam_indices push_progressc |d}|j} ||} ||} ||z|dz|z} } || | }|| | }tj|||dfd}|ggggd}t|D]\}}| |}|||| |}|||j sbtj | tj | }|D])}| d}||tj||dfd}||d vr|d ||d | |z|d ||d ||||d |+|r tj||d\}}tj||dfd}| |}|}||||d vr|d ||d ||d ||d ||d |t%|d dkrtj |d | }tj|d | }tj|d | }| |d z}tj||fd} tj||fd}!tj d|D| }"|"dz|!z}#|#dj}$|"|$}%d}&|%d}'g}(|%D]'})|)|'kr|&dz }&nd}&|)}'|(|&(tj t/j|(d}*|$|*d|}$|!|$}| |$}tj||f|$}|||fS)Nrr ra)dim)new_seqs new_statesrrrrDT)statefulrrrrr)r2c6g|]}|Sr,)get_bank)r>ones rr@zHConstrainedBeamSearchScorer.step_sentence_constraint..s %K%K%Kcllnn%K%K%Krd) descending mergesort)kind)sizer2rr%cat unsqueezerrxrrrNadvancelongrr{rtaker}rvstacktosortrnpargsort)+rrrrrrrrorig_lenr2topk_contraint_statesadvance_constraint_statessidxeidxthis_batch_input_idsthis_batch_token_scoresfull_hypotheses track_newseq_idxpre_seq topk_state advance_stateadvance_tokens advance_tokenr advance_seq new_score new_tokenrrr all_states all_tokens all_scores all_bankszippedr sorted_bankscountercur_bank incrementsbank rearrangerss+ rrz4ConstrainedBeamSearchScorer.step_sentence_constraints(%))!,,")!% ; ;H E E$($?$?$I$I!)IMX+Ed(d3".tDy"9)Y/@%ACSC]C]^`CaCa$bhjkkk(..00   !**> ? ?? B? B GW/w7J   _W5<<>> ? ? ?5g>M    0 0 1 1 1 *1 B!&m.C.C.E.EUZ`f!g!g!g%3 B BM - 2 2D 2 A AIMM-"6"6"8"8999"')Wm6M6Ma6P6P,QSU"V"V"]"]"_"_K")J*???!*-44[AAA!-077wGGG!,/66}EEE!,/667Nw7W7\7\]j7k7klll!,/66yAAA B" B((-y1H1QST'U'U$ 9#i)2E2Ea2H2H(I2NN 9' B )0022 ##K000i &;;;j)00===m,33G<<<l+229===l+229===l+22=AAA y' ( (1 , ,,y'?OOOKY|%<==@@HHJY|%<==@@HHJ.<1HHJ$4j#A2FFJ$4j#A2FFJ %K%K %K%K%KTZ[[[I_z1FkkTk22:G$W-LG#AHJ$ + +8##qLGGG#H!!'****,rz*;'O'O'OPPKk*9H95G)'2 )'2  % +.js qtrrrr rrr)rvrMrQr%r)r+rNrxrPrLr1rzrrrur{rr5rrwr2rrrrr}rrRrryr)&rrrrrrr[r\r]r_r0rr ids_collectbeam_idrrrrrrkrrrrrrrrrrrrrrrrrs& rrz$ConstrainedBeamSearchScorer.finalize-s))  #J|U\,R,R #,,, . ,~  < 55L$-T_#=#=   Ixz)$  K 00 0 0!*T^!;g!E/?DDFF (8 '+'G'G H[H[H]H]'^'^$'0AMAYn!=!=_cJ$0$6r$:=O$OMLL{cpLqqq&&w///;$"<<<$T^44Gk11)2T^)Cg)M&7&G&L&L&N&N '0'@ (4(:2(>AS(S  \;m \\\;''4+EEEF!}}Z$2L%LMM  k*t/I"IRVR]ejersss %T_55 M MKAx ^^DDDK4566 M M!,!2!2+A. )!,+A. CFx== T7!;a?@ H%%%##J///BL A ::Q>?? M (++--2244q8 > Ah,0GA(a(( )".3l8.D.D?S]]?*+A--.:1o<?*+$#. '     rrrrB)r r!r"r#r+rr r%r2rrrrOrrSrrYrrr&r'r)r(rrrr,rrrrs`""T+.8=/0)*$(++++*% +  + ! +$E$)$45+ (}+"#+SM++++Z    X ooo###<@FJ37,-] ] #] &] % ] & ] $/ ] uS%,%678] uS$s)U\%ABC] u/0] %SM]  u| ] ] ] ] N$KEKEKE#KE' KE  + KE  * KE!+KEKEKEKEKEh<@FJ37,-r r #r !,r !+ r ", r  r uS%,%678r uS$s)U\%ABCr u/0r %SMr  u r r r r r r rrc eZdZddedededeefdZdZ dde j d ed ee j d eefd Z ddededeedefdZ dS)r;Nr1r3r:rc||_||_||_||_g|_d|_t |jts|jtddSdS)z7 Initialize n-best list of hypotheses. geANzWhen `do_early_stopping` is set to a string, `max_length` must be defined. Ensure it is passed to the BeamScorer class instance at initialization time.) r3r:rr1r worst_scorerQrOrR)rr1r3r:rs rrSzBeamHypotheses.__init__su-,$" $-t44 9PE   9P9Prc*t|jS)z3 Number of hypotheses in the list. )rvrrXs r__len__zBeamHypotheses.__len__s4:rhyp sum_logprobsr]rkc||||jzz }n||jd|jzz }t||jks ||jkr|j|||ft||jkrTtdt|jD}|j|dd=|dd|_dSt||j|_dSdS)z3 Add a new hypothesis to the list. Nrac$g|] \}\}}}||fSr,r,)r>idxsr?s rr@z&BeamHypotheses.add..s&,b,b,b.#y1aaX,b,b,brrr ) r3rurvr1rrrrrxr)rr r!r]rkscoresorted_next_scoress rr{zBeamHypotheses.adds  $ M43F$FGEE!CIbMT5H$HIE t99t~ % %1A)A)A J  uc<8 9 9 94yy4>))%+,b,bIVZV`LaLa,b,b,b%c%c"J1!4Q78#5a#8#;   #&ud.>#?#?   *B)Arrbest_sum_logprobsrr_rcHt||jkrdS|jdurdS|jdur|||z |jzz }|j|k}|S|jdkr0|j|krt d||j|z |jzz }n|||z |jzz }|j|k}|S)z If there are enough hypotheses and that none of the hypotheses being generated can become better than the worst one in the heap, then we are done with this sentence. FTgz3max_length is not larger than decoder prompt length)rvr1r:r3rrrR)rr(rr_highest_attainable_scorerets rrYzBeamHypotheses.is_dones t99t~ % %5  $ & &4 E ) )'8GFX>CJ "S((?&888$%Z[[[%;M)MRVRe(ee)( ,=J\@\aeat?t+t("&>>CJrrU)NN)r) r r!r"r+rrOrrSrr%r&r{rYr,rrr;r;s#ud`hil`m"48'+ @@  @@u/0 @  } @@@@2""""RZ[^R_"hl""""""rr;)abcrr collectionsrtypingrrnumpyrr%utilsr beam_constraintsr r r$r*rr.rr;r,rrr2sn $####### """""""" ((((((========"H8AAAAAAAA@e e e e e ze e e P | | | | | *| | | ~SSSSSSSSSSr