hS\>ddlmZ ddlZddlZddlZddlZddlZddlmZm Z ddl m Z m Z m Z mZddlmZmZmZmZddlZddlmZddlmZerddlZdd lmZdd lmZdd lmZddZeje Z!ej"#e dZ$Gdde%Z&Gdde&Z'Gdde&Z(Gdde(Z)Gdde(Z*Gdde(Z+Gdde(Z,Gdd e&Z-Gd!d"e&Z.Gd#d$e&Z/Gd%d&e&Z0Gd'd(e&Z1Gd)d*e1Z2Gd+d,e&Z3Gd-d.e3Z4Gd/d0e&Z5Gd1d2e3Z6Gd3d4e3Z7Gd5d6e3Z8Gd7d8e3Z9Gd9d:e6Z:Gd;de3Z<Gd?d@e&Z=GdAdBe3Z>GdCdDe&Z?GdEdFe&Z@GdGdHeAZBGdIdJeAZCGdKdLe&ZDGdMdNe&ZEGdOdPeEZFGdQdReEZGGdSdTeGZHGdUdVeGZIGdWdXeEZJGdYdZeEZKGd[d\eEZLGd]d^eEZMGd_d`eEZNeOeFePeGeQeHeReJeSeIeTeKe%eLeUeMeVeNi ZWdddZXdddeddmZYddpZZej[j\j]ej[j\j^ej[j\j_ej[j\j`ej[j\jafZbddvZcedZeeeddwddzZfddZgddZheeddddZiddZjGddZkdddZlddZmddZn dddZoddZpddZqddZr dddZs dddZtdS)) annotationsN)autoEnum) extract_stack format_exc format_list StackSummary)AnyNoReturnOptional TYPE_CHECKING)config)counters) CompileId)InstructionTranslatorBase)DynamoFrameType case_namestrreturnc4d|ddzS)NzhFor more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#_-)replacers Y/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/torch/_dynamo/exc.pyexportdb_error_messager5s% I   C % % & graph_breaksceZdZdS)TorchDynamoExceptionN__name__ __module__ __qualname__rrr!r!ADrr!ceZdZdS)InternalTorchDynamoErrorNr"r&rrr)r)Er'rr)c2eZdZUded<ddd fd ZxZS) RestartAnalysis Optional[str]restart_reasonN)r-argsr rNonecB||_tj|dSN)r-super__init__)selfr-r. __class__s rr3zRestartAnalysis.__init__Ls%,$r)r.r r-r,rr/r#r$r%__annotations__r3 __classcell__r5s@rr+r+IsX!!!!CG            rr+ceZdZdS)SpeculationRestartAnalysisNr"r&rrr;r;Qr'rr;ceZdZdS)UnspecializeRestartAnalysisNr"r&rrr=r=Ur'rr=ceZdZdS) CompileCollectiveRestartAnalysisNr"r&rrr?r?Yr'rr?ceZdZdS)TensorifyScalarRestartAnalysisNr"r&rrrArA]r'rrAceZdZdS) SkipFrameNr"r&rrrCrCar'rrCceZdZdS)TorchRuntimeErrorNr"r&rrrErEer'rrEc eZdZdfd ZxZS)InvalidBackendnamerrr/cRtd|ddS)NzInvalid backend: z=, see `torch._dynamo.list_backends()` for available backends.r2r3)r4rHr5s rr3zInvalidBackend.__init__js7  e e e e     r)rHrrr/r#r$r%r3r8r9s@rrGrGis=          rrGc eZdZdfd ZxZS) ResetRequiredrr/cnttjddS)Nz Must call `torch._dynamo.reset()` before changing backends. Detected two calls to `torch.compile()` with a different backend compiler arguments. )r2r3textwrapdedent)r4r5s rr3zResetRequired.__init__qs=  O       rrr/rKr9s@rrMrMps=          rrMc(eZdZd fd Zd d ZxZS) ShortenTracebackr.r first_useful_frameOptional[types.FrameType]kwargsrr/cHtj|i|||_dSr1)r2r3rT)r4rTr.rVr5s rr3zShortenTraceback.__init__}s/ $)&)))"4r typing.Selfc|j}|j| tjr|S|j|jur!|j}| Jd|j|ju!||S)Nz#internal error, please report a bug) __traceback__rTrverbosetb_frametb_nextwith_traceback)r4tbs rremove_dynamo_framesz%ShortenTraceback.remove_dynamo_framessq    " *bjFNjKk!888B>>#H>>>k!888""2&&&r)r.r rTrUrVr rr/)rrX)r#r$r%r3r`r8r9s@rrSrS|sQ555555 ''''''''rrSc eZdZd fd ZxZS) BackendCompilerFailed backend_fnr inner_exception ExceptionrTrUrr/ct|dd|_||_d|jdt|jd|}t ||dS)Nr#?zbackend=z raised: z: )rT)getattr backend_namerdtyper#r2r3)r4rcrdrTmsgr5s rr3zBackendCompilerFailed.__init__sq $J C@@.k*kk_8M8M8VkkZikk 1CDDDDDr)rcr rdrerTrUrr/rKr9s@rrbrbsG E E E E E E E E E Errbc8eZdZdddfd Zdd Zddd ZxZS) UnsupportedNrrkrrr,rr/ct|tjj|_||_d|_| ||_ dSr1) r2r3torch_guardsTracingContextr real_stackrkcategory add_to_statsr)r4rkrr5s rr3zUnsupported.__init__s^ -6DDFF'+  (1rc|jJt|j|jxxdzcc<t|j|jdkrt|j|j=dSdS)Nrrrsrrkr4s rremove_from_statszUnsupported.remove_from_statssm}((()))Q.))) DM "48 , 1 1'111 2 1r unimplementedrscT||_t||jxxdz cc<dS)Nrrv)r4rss rrtzUnsupported.add_to_statss2  48$$$)$$$$$r)rkrrr,rr/rQ)ry)rsrrr/)r#r$r%r3rxrtr8r9s@rrmrmsw?C222222222222 *********rrmceZdZdS)$UnknownPropertiesDuringBackwardTraceNr"r&rrr|r|r'rr|ceZdZdS)RecompileErrorNr"r&rrr~r~r'rr~c eZdZdfd ZxZS)ArgsMismatchErrorrkrrr/cJt|dSr1rJr4rkr5s rr3zArgsMismatchError.__init__! rrkrrr/rKr9s@rrr=rrc eZdZdfd ZxZS)AttributeMutationErrorrkrrr/cJt|dSr1rJrs rr3zAttributeMutationError.__init__rrrrKr9s@rrrrrrc eZdZdfd ZxZS)InfiniteGeneratorErrorrkrrr/cJt|dSr1rJrs rr3zInfiniteGeneratorError.__init__rrrrKr9s@rrrs=rrc eZdZdfd ZxZS)SideEffectsErrorrkrrr/cJt|dSr1rJrs rr3zSideEffectsError.__init__rrrrKr9s@rrrrrrc$eZdZdZdfd ZxZS)CondOpArgsMismatchErrorz? Internal error from cond() due to arguments mismatch. rkrrr/cJt|dSr1rJrs rr3z CondOpArgsMismatchError.__init__rrr)r#r$r%__doc__r3r8r9s@rrrsGrrceZdZeZeZeZeZeZeZ eZ eZ dS) UserErrorTypeN) r#r$r%rDYNAMIC_CONTROL_FLOW ANTI_PATTERNSTANDARD_LIBRARYCONSTRAINT_VIOLATION DYNAMIC_DIM INVALID_INPUTINVALID_OUTPUT*UNSUPPORTED_ALIASED_MUTATED_DYNAMIC_INPUTSr&rrrrsl466466Ltvv466$&&KDFFMTVVN15...rrc$eZdZ d d fd ZxZS) UserErrorN error_typerrkrrr,rr/c|It|tsJ|dr|dz }n|dz }|t|z }t |||_||_dS)aG Type of errors that would be valid in Eager, but not supported in TorchDynamo. The error message should tell user about next actions. error_type: Type of user error msg: Actionable error message case_name: (Optional) Unique name (snake case) for the usage example in exportdb. N.  ) isinstancerendswithrr2r3rmessage)r4rrkrr5s rr3zUserError.__init__s  i-- - - -||C   s t  ))44 4C $ rr1)rrrkrrr,rr/rKr9s@rrrsENRrrceZdZdS)SkipCodeRecursiveExceptionNr"r&rrrrr'rrceZdZdS)RecompileLimitExceededNr"r&rrrrr'rrceZdZdS)UnsafeScriptObjectErrorNr"r&rrrrr'rrceZdZdS)UncapturedHigherOrderOpErrorNr"r&rrrrr'rrceZdZdS)IncorrectUsageNr"r&rrrrr'rrceZdZdS)FailOnRecompileLimitHitNr"r&rrrrr'rrceZdZdS) PackageErrorNr"r&rrrrr'rrceZdZdS)ObservedExceptionNr"r&rrrrDrrc,eZdZUded<d fd ZxZS) ObservedUserStopIterationz Optional[Any]valuer.r rVrr/ctdt|dkr|d|_dSd|_dS)Nzunhandled `raise StopIteration`r)r2r3lenr)r4r.rVr5s rr3z"ObservedUserStopIteration.__init__!sE :;;; t99q==aDJJJDJJJr)r.r rVr rr/r6r9s@rrrsKrrceZdZdS)ObservedLookupErrorNr"r&rrrr)rrrceZdZdS)ObservedIndexErrorNr"r&rrrr.rrrceZdZdS)ObservedKeyErrorNr"r&rrrr3rrrceZdZdS)ObservedGeneratorExitNr"r&rrrr8r'rrceZdZdS)ObservedAttributeErrorNr"r&rrrr<rrrceZdZdS)ObservedRuntimeErrorNr"r&rrrrArrrceZdZdS)ObservedNotImplementedErrorNr"r&rrrrFr'rrceZdZdS)ObservedTypeErrorNr"r&rrrrJrrrexc_typetype[Exception]type[ObservedException]c|tvrAt|dt|}td|dtfit|<t|S)Nr#ObservedError)observed_exception_maprhrrjr)rrHs rget_dynamo_observed_exceptionr\s]---xS]];;+/ "t " " "%6$8", , x( "( ++r)r.rVtxrr.Optional[list[Any]]rVOptional[dict[str, Any]]r cddlm}||||pg|pi}|j|t |)Nr)BuiltinVariable) variablesr call_function exn_vt_stackset_current_exceptionr)rrr.rVr exception_vts rraise_observed_exceptionresc+*****#?8,,::2tzr6z,unimplemented_with_warning../   rcSr1r&graph_break_msgsrrz,unimplemented_with_warning..?r metadata_fn payload_fn%s)from_exc) format_error_msg_verbosero_loggingtrace_structuredgraph_breaks_logdebuglogwarningry)rrrkrs @runimplemented_with_warningrs/q$77O N##  +*** $4111KK#""""""r)rrrr,c|tjddksJ|turt |||t ||)NBREAKFr)osenvironget_NOTHINGrm)rkrrs rryrysX "*..%00 0 0 0 0x#333A cY / / //rgb_typecontext explanationhints list[str]ct||tjddfdtdt |||||ddS)Nrc dddSrr&r&rrrz/unimplemented_v2_with_warning..rrcSr1r&rsrrz/unimplemented_v2_with_warning..rrrrTr log_warning)rrorrrrunimplemented_v2)rrrrrr rs @runimplemented_v2_with_warningrs/q$77O N##  +*** $4111Wg{EASWXXXXXXrctj|d}dd|D}tj|d}|d|d|d|d}|S)N rc3jK|].}dtj|dzV/dS)z Hint: rN)rOindentlstrip).0hints r z-format_graph_break_message..sP@D X_T62299;;;rz Explanation: z Developer debug context: )rOrrjoin)rrrr  hints_strrks rformat_graph_break_messagers /+v66==??K HMIogv..5577G         $    C JrFrrboolct||||}|rt||turt ||t |)a Called within dynamo to cause a graph break. Args: gb_type: Context-free graph break type. It should be a short string without any information specific to the tracing context (i.e. no dynamically-generated strings) context: Developer context for the graph break. It can contain tracing context/dynamic strings. explanation: User-facing context-dependent explanation for the graph break. Can be dynamic. hints: List of user-facing hints for the graph break. )rrrrrm)rrrr rrrks rrrs\& %Wg{E J JC Cx#H, c  rctd|xxdz cc<|tjddksJdS)NwarningsrrF)rrrr)rks rrrsJ Z" "*..%00 0 0 0 0 0 0rc&eZdZd dZd dZd dZd S) KeyErrorMsgrr rr/c||_dSr1)r)r4rs rr3zKeyErrorMsg.__init__s  rrc*t|jSr1)rrrws r__str__zKeyErrorMsg.__str__s4:rc*|Sr1)r%rws r__repr__zKeyErrorMsg.__repr__!s||~~rN)rr rr/)rr)r#r$r%r3r%r'r&rrr"r"sPrr"rexcexportc|ddl}d|_t|}|Nt|dkr;|d|_|dd||z }t jr&t|dr|d|j d|j dz }t j st|d r|d z }t|d r^t|j d rIt|j d r!|d|j j d|j j dz }n|d|j j dz }t|jdkrdnt|jd}t!|t"r*t%||zf|jddz|_dS||z}|f|jddz|_dS)Nrz from user code: record_filenamez! Last frame execution written to zD. To run only this frame while debugging, run torch._dynamo.replay('z'). rrz Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo" rd minifier_path buck_commandz Minifier script written to zW. Run this buck command to find the smallest traced graph which reproduces this error: rzQ. Run this script to find the smallest traced graph which reproduces this error. r) tracebackinnermost_user_frame_summaryget_real_stackrrrrreplay_record_enabledhasattrr-r[rdr.r/r.rrKeyErrorr")r(rkr)r0rrold_msgnew_msgs raugment_exc_messager8%s,'+C$$$J#j//A"5"5+5b>( Rbggi.C.CJ.O.O&P&PRRR # 5F(G(G   31D 3 3+ 3 3 3  > gc<88   J  s%&& 7 _,,  3& 7 7  U0C0QUU030C0PUUU CC _0C0Q___ C MMQ&&bbC ,<,Ts?H F%11:/6AL V rr c8ttSr1) filter_stackrr&rrget_stack_above_dynamorA`s   ( ((rframeOptional[DynamoFrameType]Optional[StackSummary]ct|dd}|dS|t}nt}tj||zS)Nrr)rhrAr from_list)r(rBrrstack_above_dynamos rr2r2dsZlD11Jt  455)^^  !"4z"A B BBrstackct}|D]D}|j d|jvrn0d|jvs|jr d|jvr/||E|S)N convert_frame eval_frameztorch._dynamo.optimize()r r<lineappend)rH user_stackrBs rr@r@s{J ! ! > !  en , , E 5> ) ) J *4 BB %    rrHcpddlm}tj|d|}|r|dSdS)Nr)TORCH_DYNAMO_RESUME_IN_PREFIXz _(\w+)_at_\d+)resume_executionrPrematchgroup)rHrPrSs rremove_resume_prefixrUsL?????? H5FFF M ME {{1~~ 4rct}|D]U}|j t|j}|r |r|dj|kr ||d<||_@||V|S)a1 When we graph break, we create a resume function and make a regular Python call to it, which gets intercepted by Dynamo. This behavior is normally shown in the traceback, which can be confusing to a user. So we can filter out resume frames for better traceback clarity. Example: File "..." line 3, in f File "..." line 5, in torch_dynamo_resume_in_f_at_80 File "..." line 10, in torch_dynamo_resume_in_f_at_120 becomes File "..." line 10, in f Nr+)r r<rUrHrM)rH new_stackrBrHs rcollapse_resume_framesrXs(I$$ > ! #EJ//  $ $)B-"4"<"<!IbMEJJ   U # # # # rr-cd|jd|jd|jd}|dz }|tz }t ||}|4|dz }|dt |z }|dz }|dz }|S) NWON'T CONVERT r line rz.========== TorchDynamo Stack Trace ========== zY ========== The above exception occurred while processing the following code ========== r,z ==========)co_name co_filenameco_firstlinenorr2rr)r(rr-rBrkrrs rrrs XWW(8WW@SWWW CCC:<<CU++J    rww{:../// t  x Jrc tjrt||||Sd|jd|jd|jdt S)NrZrr[z due to: )rr[rr\r]r^r)r(rr-rBs rformat_error_msgr`sk ~K'T?EJJJ 8DL 8 84+; 8 8  8 8)3 8 88r)rrrr)rrrr) rrrrr.rrVrrr )rr rr/)rrerrrkrrr )rkrrr rr,rr )rrerrrrrrrrr r rr ) rrrrrrr r rr)rrrrrrr r rr rrrr r)rF)r(rerkrr)rrr/)rrer9rrr:)rr r1)r(rerBrCrrD)rHr rr )rHrrr,)NN) r(rerrr-r,rBrCrr)u __future__rloggingrrRrOtypingenumrrr0rrrr r r r r torch._guardsror,rutilsrtypesrsymbolic_convertrrr getLoggerr#rrgetArtifactLoggerr RuntimeErrorr!r)r+r;r=r?rArCrErGrMrSrbrmr|r~rrrrrrrrrrrrerrrrrrrrrrrrr StopIteration LookupError IndexError GeneratorExitr5AttributeErrorNotImplementedError TypeErrorrrrr _subclasses fake_tensorDataDependentOutputExceptionDynamicOutputShapeExceptionUnsupportedOperatorExceptionUnsupportedFakeTensorException$UnsupportedMutationAliasingException!exceptions_allowed_to_be_fallbackrobjectrryrrrrr"r8r>rAr2r@rUrXrr`r&rrr|s """"""2  JJJJJJJJJJJJ999999999999'LLL'''''';;;;;;&&&&&&g!!>33HnMM     <        3        *                /                _        $        ,        )         (    '''''+'''" E E E E E, E E E*****&***(     ;        )     [ [ { /88888D888 0     !5        [        2        #7        Y        i        '        ,        1        +        ,        *        -        .        ,        "3        )   ,$"( *&4   ,,,,!%'+ + + + + + + ....< !> != !> !@ !F %!####0 688"*d000000YYYY8:B1111,-,-,-,-,-^    )))) 8<CCCCCB     J&*'+ <&*'+ 8 8 8 8 8 8 8r