yj1dZddlmZddlZddlZddlZddlZddlmZmZddl m Z ddl m Z m Z mZejeZdZdZd Zd Zd*d Zd+d,dZd-dZd.dZd/dZd0dZd1dZd+d2d Zd+d2d!Z d3d4d'Zd5d)Z dS)6uGateway lifecycle ledger — durable termination-reason evidence (NS-608). The gateway already has *graceful* shutdown forensics (:mod:`gateway.shutdown_forensics` — who sent the SIGTERM) and an exit-path diagnostic log (``gateway-exit-diag.log`` — every way ``asyncio.run`` can return). What it does NOT have is any record of an **unclean death**: a SIGKILL, a kernel OOM kill, or the whole VM dying takes the process out before any handler runs, so the next boot has no idea the previous life ended violently — support tickets like NS-608 then require manually cross-correlating four log files and two external APIs to answer "what killed the gateway?". This module closes that gap with a tiny state machine persisted to ``/state/gateway.lifecycle.json``: * On startup, :func:`record_startup` reads the sentinel left by the previous life. ``phase == "running"`` means that life never reached any exit path → it died uncleanly. The finding — including the last heartbeat's memory sample, which is the closest thing to a pre-death telemetry snapshot — is appended to ``gateway-exit-diag.log`` as a ``gateway.previous_unclean_exit`` record and logged at WARNING. The sentinel is then rewritten as ``phase=running`` for the new life. * On every clean exit path, :func:`mark_exited` rewrites the sentinel as ``phase=exited`` with the exit code and a reason string. Wired into ``_exit_after_graceful_shutdown`` (the single funnel for all graceful exits, #53107) and the two watchdog ``os._exit`` sites in :mod:`gateway.shutdown_watchdog`. :func:`sample_memory` provides the cheap (<1ms, pure /proc reads) memory snapshot that :func:`gateway.shutdown_watchdog.write_loop_heartbeat` embeds in the 30s heartbeat — giving every unclean-death report a "memory available N seconds before death" data point so OOM crash cycles are classifiable from the volume alone (no Prometheus retention races). Everything here is best-effort: a forensics failure must never affect the gateway lifecycle it is observing. ) annotationsN)datetimetimezone)Path)AnyDictOptional)statezgateway.lifecycle.json)logszgateway-exit-diag.logig?returnrctjdd}|rt |Sddlm}|S)zEHERMES_HOME for process-level identity files (ignore task overrides). HERMES_HOMEr)get_hermes_home)osenvirongetstriprhermes_constantsr)valrs >/home/ragecks/.hermes/hermes-agent/gateway/lifecycle_ledger.py_process_hermes_homer=sW *.. + + 1 1 3 3C Cyy000000 ?  homeOptional[Path]cD||n t}|jtS)z6Return ``/state/gateway.lifecycle.json``.)rjoinpath_LIFECYCLE_RELATIVE)rbases rget_lifecycle_sentinel_pathr Gs'#44)=)?)?D 4=- ..rDict[str, Any]ci} tdd5}|D]C}|dr,t|d|d<nDdddn #1swxYwYn#tt t f$rYnwxYw i}hd}td d5}|D]n}|d dd }||vrLt|d||<t|t|krnodddn #1swxYwYd |vr |d |d <d|vr |d|d<d|vrd|vr|d|dz |d<n#tt t f$rYnwxYw|S)zCheap memory snapshot: own RSS + system availability + swap. Pure ``/proc`` reads, Linux-only (returns ``{}`` elsewhere), never raises. Values in KiB to match the kernel's units. z/proc/self/statusutf-8encodingzVmRSS:rss_kibN>MemTotalSwapFree SwapTotal MemAvailablez /proc/meminfo:rr( mem_total_kibr+mem_available_kibr*r) swap_used_kib)open startswithintsplitOSError ValueError IndexErrorlen)samplefhlinememinfowantedkeys r sample_memoryr>Ms_  F % 8 8 8 B  ??8,,(+DJJLLO(<(45==5#4#4455<< tts5''A AAB"2/B"" B0/B0cLtt|}|r|ddkrdSt|d|drdS|d|d|dd} dd lm}t||}n#t $rd}YnwxYw|r|d |d <|d }t|tr}||d <|d}|d}t|tr9|tks)t|tr|dkr||z tkrd|d<|S)uInspect the previous life's sentinel; return an evidence dict when it died uncleanly, else ``None``. Read-only — does not rewrite the sentinel. phaserunningNrdre started_at) prior_pidprior_started_atprior_start_timer)get_loop_heartbeat_path updated_atlast_heartbeat_atmemlast_heartbeat_memr-r.T suspected_oom) rHr rrpgateway.shutdown_watchdogrxrVrErFr2_LOW_MEM_AVAILABLE_KIB_LOW_MEM_AVAILABLE_FRACTION)rsentinelevidencerxhbr{totalavails rdetect_unclean_exitrs5d;;<rHrYrcrprrrrrrrs$$L#"""""  ''''''''&&&&&&&&&&  8 $ $97 #"///// !!!!H4444JJJJ L L L L!!!!H(((((V,,,,,` $%PPPPPBr