yj&dZddlmZddlZddlZddlmZmZmZm Z ej e Z dZ ddZddZd dZddd!dZddd"dZgdZdS)#uStream diagnostics — per-attempt counters, exception chains, retry logging. When a streaming chat-completions request dies mid-response, we want to know why: which Cloudflare edge served the request, which OpenRouter downstream provider answered, how many bytes/chunks we got before the drop, the HTTP status, the underlying httpx error class. These helpers collect that info and emit it both to ``agent.log`` (full detail) and to the user-facing status line (compact). All helpers are extracted from :class:`AIAgent` for cleanliness. ``run_agent`` keeps thin forwarder methods so existing call sites and tests that patch ``run_agent.`` keep working. ) annotationsN)AnyDictListOptional) zcf-rayzcf-cache-statuszx-openrouter-providerzx-openrouter-modelzx-openrouter-idz x-request-idz x-vercel-idviaserverzx-forwarded-forreturnDict[str, Any]c6tjdddiddS)zReturn a fresh per-attempt diagnostic dict. Mutated in-place by the streaming functions and read from the retry block when a stream dies. Lives on ``request_client_holder`` so it survives across the closure boundary. Nr) started_atfirst_chunk_atchunksbytesheaders http_status)time7/home/ragecks/.hermes/hermes-agent/agent/stream_diag.pystream_diag_initr)s+ikk   ragentrdiag http_responseNonec|t|tsdS t|dd|d<n#t$rYnwxYw t|ddpi}i}t|dt}|D]D} ||}|rt |dd||<5#t$rYAwxYw||d<dS#t$rYdSwxYw)uSnapshot interesting headers + HTTP status from the live stream. Called once at stream open (before iterating chunks) so the metadata survives even if the stream dies before any chunk arrives. Failures are swallowed — diag is best-effort. N status_coderr_STREAM_DIAG_HEADERSx) isinstancedictgetattr ExceptionSTREAM_DIAG_HEADERSgetstr)rrrrcapturedtarget_headersnamevals rstream_diag_capture_responser+:s#JtT$:$: %m]DII]       -D99?R#% (>@STT"  D kk$''4%(XXdsd^HTN    "Y      sB0 ==/B:11B#"B:# B0-B:/B00B:: CCerror BaseExceptionr&c|g}|}|mt|dkrZ||vrnU||t|ddpt|dd}|||urn|}|t|dkZg}|D]}t|dd}t|dkr |dddz}||rt |jd |d nt |j|rd |nt |jS) uReturn a compact ``Outer(msg) <- Inner(msg) <- ...`` rendering. OpenAI SDK wraps httpx errors as ``APIConnectionError`` / ``APIError`` and only the wrapper's class is visible at the catch site — but the underlying ``RemoteProtocolError`` / ``ConnectError`` / ``ReadError`` is what tells us WHY the stream died. Walks ``__cause__`` then ``__context__`` (deduped, max 4 deep) to surface the chain in one line. N __cause__ __context__  …()z <- ) lenappendr"r&stripreplacetype__name__join)r,seenlinknxtpartsemsgs rflatten_exception_chainrEYsY!#D$)D  s4yy1}} 4<<  DdK.. ' -3 3  ;#++   s4yy1}}E PP!ffllnn$$T3// s88c>>dsd)e#C SNQ(1131111d1gg>NOOOO!& @6;;u   DKK,@@r)rkindattemptint max_attempts mid_tool_callboolOptional[Dict[str, Any]]c ||}n#t$rt|}YnwxYw|r t|dkr |dddz} t |}n$#t$rt |j}YnwxYwtj} d} d} d} d} d}d}t|tr^ t| dpd} t| dpd} t| d p| }td| |z } | d }| tdt||z } | d pi}t|tr3|r1d d |D}| d"t| d}n#t$rYnwxYwt d|||t%|ddpdt%|dd|jpd|jpdt |j|||| | | | | ddnd|d|idS#t$r t ddYdSwxYw)uRecord a transient stream-drop and retry to ``agent.log``. Always logs a structured WARNING so users have a breadcrumb regardless of UI verbosity. Subagents in particular benefit because their retries no longer spam the parent's terminal — but the file log keeps full detail (provider, error class, attempt, base_url, subagent_id). When *diag* is provided (the per-attempt stream-diagnostic dict from :func:`stream_diag_init`), the WARNING also captures upstream headers (cf-ray, x-openrouter-provider, x-openrouter-id), HTTP status, bytes streamed before the drop, and elapsed time on the dying attempt. These are the breadcrumbs needed to answer "is one CF edge / one downstream provider responsible, or is it random across runs?" Nr5r-rrr rrr3c3*K|]\}}|d|VdS)=Nr).0kvs r z#log_stream_retry..sA--'+q!1 q ------rruStream %s on attempt %s/%s — retrying. subagent_id=%s depth=%s provider=%s base_url=%s error_type=%s error=%s chain=%s http_status=%s bytes=%d chunks=%d elapsed=%.2fs ttfb=%s upstream=[%s] _subagent_id_delegate_depthz.2fsrJ)extrazstream-retry log emit failedT)exc_info)_summarize_api_errorr#r&r8rEr<r=rr r!rHr%floatmaxr>itemsloggerwarningr"providerbase_urldebug)rrFr,rGrIrJr_summary_chain_now_bytes_chunks_elapsed_ttfb _headers_repr _http_status_started_firstrs rlog_stream_retryrpxs:0CD "11%88HH " " "5zzHHH "  .H ++~-H *,U33FF * * *%[[)FFF *y{{  dD ! !  TXXg..3!44dhhx005A66 ,!7!7!?4@@sD8O44"233%U6]]X%=>>E((9--3gt,,$'HH--/6}}---%%M88M**6#&txx '>'>#?#?L          E>4 0 0 7C E,a 0 0 N !c N !c KK      $0uOOOOOc "M2/      2 DDD 3d CCCCCCDsqJ5J5%JA+*J+B  J B  8JE HJ HJHA9J&KKc2|rdnd}t||||||||jpd}d}t|tr_ |d} | 6dt d t jt| z d d }n#t$rYnwxYw | d |d |dt|j d|d|d| | d|d|dt|j dS#t$rYdSwxYw)uEmit a single user-visible line for a stream drop+retry. Both top-level agents and subagents announce drops in the UI — the parent prefixes subagent lines with ``[subagent-N]`` via ``log_prefix`` so they're easy to attribute. All cases also write a structured WARNING to ``agent.log`` via :func:`log_stream_retry` with the full diagnostic detail (subagent_id, provider, base_url, error_type, cf-ray, x-openrouter-provider, bytes/chunks, elapsed) for post-hoc analysis. The user-visible status line is intentionally compact: provider, error class, attempt N/M, plus ``after Xs`` when the stream dropped mid-flight. Full diagnostic detail goes to ``agent.log`` only — ``hermes logs --level WARNING | grep "Stream drop"`` to inspect. zdrop mid tool-calldrop)rFr,rGrIrJrrbr Nz after rOz.1frYu⚠️ z stream z (r7u — reconnecting, retry /z stream retry ) rprbr r!r%r^rr]r#_buffer_statusr<r=_touch_activity) rr,rGrIrJrrFrb_suffixstarteds remit_stream_droprys0$1 <  fD !# ~+HG$ hh|,,G"QCTY[[5>>-I$J$JQQQQ    D     @h @ @ @ @U 0D @ @w @ @'. @ @1= @ @     ,G , ,l , ,%[[) , ,           s%A B BBA)D DD)r$rr+rErpry)r r )rrrr rrr r)r,r-r r&)rrrFr&r,r-rGrHrIrHrJrKrrLr r)rrr,r-rGrHrIrHrJrKrrLr r)__doc__ __future__rloggingrtypingrrrr getLoggerr=r`r$rr+rErpry__all__rrrrs  #""""" ,,,,,,,,,,,,  8 $ $ "    >AAAAN&*[D[D[D[D[D[DJ&*8 8 8 8 8 8 v   r