yj/f UdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl mZddlmZmZmZddlmZmZejeZgdZdaeeed<ejd Zd Zd e edzd e efd Z!de"dzd e"eeffdZ#d e"eeffdZ$ejdZ%ded efdZ&d efdZ'ddddde(dedzdedzd e(fdZ)dedefdZ*d eefdZ+gdZ,dZ-d Z.d!e d e/fd"Z0d#Z1d$Z2gd%Z3d e4e ee"eefe effd&Z5d'e ed(e"eefd)e ed efd*Z6d>d,e/d e/fd-Z7d(e"eefd e8efd.Z9d!e ed/e8ed e efd0Z:d?d2e/d3e/d e efd4Z;ded5ed e/fd6Zee/ed8<da?ee/ed9<d5ed e/fd:Z@d@d;ZAGd<d=eZBdS)AzDocker execution environment for sandboxed command execution. Security hardened (cap-drop ALL, no-new-privileges, PID limits), configurable resource limits (CPU, memory, disk), and optional filesystem persistence via bind mounts. N)Path)Optional)BaseEnvironmentEnvironmentConnectionError _popen_bash)_HERMES_PROVIDER_ENV_BLOCKLIST_is_hermes_internal_secret)z/usr/local/bin/dockerz/opt/homebrew/bin/dockerz6/Applications/Docker.app/Contents/Resources/bin/docker_docker_executablez^[A-Za-z_][A-Za-z0-9_]*$z hermes-egress forward_envreturncg}t}|pgD]}t|tstd|3|}|sJt |std|||vr||| ||S)z?Return a deduplicated list of valid environment variable names.z0Ignoring non-string docker_forward_env entry: %rz-Ignoring invalid docker_forward_env entry: %r) set isinstancestrloggerwarningstrip_ENV_VAR_NAME_REmatchaddappend)r normalizedseenitemkeys ?/home/ragecks/.hermes/hermes-agent/tools/environments/docker.py_normalize_forward_env_namesr0sJUUD!r$$$  NNMt T T T jjll  %%c**  NNJD Q Q Q  $;;   # envc^|siSt|tstd|iSi}|D]\}}t|t r,t |std|b|}t|t sOt|tttfrt |}ntd|||||<|S)zValidate and normalize a docker_env dict to {str: str}. Filters out entries with invalid variable names or non-string values. zdocker_env is not a dict: %rz#Ignoring invalid docker_env key: %rz/Ignoring non-string docker_env value for %r: %r) rdictrritemsrrrrintfloatbool)rrrvalues r_normalize_env_dictr'Is  c4 5s;;; !#Jiikk   U#s## +;+A+A#))+++N+N  NN@# F F F iikk%%% %#ud!344 E PRUW\]]] 3 rcL ddlm}|piS#t$ricYSwxYw)zDLoad ~/.hermes/.env values without failing Docker command execution.rload_env)hermes_cli.configr* Exceptionr)s r_load_hermes_env_varsr-gsN......xzzR  s  ##z[^A-Za-z0-9_.-]r&ct|tr|sdStd|}|ddpd}|S)uCoerce *value* into a Docker label-safe form (alnum + ``_.-``, ≤63 chars). Empty or all-invalid inputs collapse to ``"unknown"`` so the resulting label is always queryable. Used at container-create time; never round-trip a sanitized value back into application logic. unknown_N?)rr_LABEL_VALUE_OK_REsub)r&cleaneds r_sanitize_label_valuer5wsN eS ! !y $$S%00Gcrcl'iG NrcJ ddlm}|pdS#t$rYdSwxYw)a Return the active Hermes profile name, or ``"default"`` on any error. Resolved at container-create time so a single container is permanently tagged with the profile that created it. Profile switches inside the same process don't retroactively relabel running containers. rget_active_profile_namedefault)hermes_cli.profilesr8r,r7s r_get_active_profile_namer;sO??????&&((5I5 yys  ""iX)max_age_secondsprofile_filter docker_exer<r=r>c |ptpd}gd}|r'|ddt|g tj|ddg|ddd d d d d d tj}n?#tjtf$r&}t d|Yd}~dSd}~wwxYw|j dkr:t d|j |j dSd|j D}|sdSddl}|j|jj} d} |D]"} t'|| } | | | z } | |kr4 tj|dd| gd d d d dtj}|j dkr7| dz } td| ddt-| n;t d| dd|j #tjtf$r/}t d| dd|Yd}~d}~wwxYw| S)u{Remove stale hermes-tagged containers left behind by prior processes. Targets containers that match all of: * ``label=hermes-agent=1`` (created by this codebase) * ``status=exited`` (running containers are NEVER reaped — they may belong to a sibling Hermes process whose reuse path will pick them up; killing them would crash the sibling mid-command) * (optional) ``label=hermes-profile=`` (sweep only the caller's profile by default; a hermes process in profile A must not tear down profile B's containers) * ``State.FinishedAt`` older than *max_age_seconds* ago (so a sibling process that just exited and is about to be replaced doesn't get its container yanked out from under it) Returns the number of containers removed. Best-effort: any failure (docker daemon unreachable, slow inspect, parse error) is logged at debug level and the function returns whatever it managed before the failure. Safe to call repeatedly; idempotent. Issue #20561 — this is the safety net for SIGKILL / OOM / crashed terminal exits that bypass the ``atexit`` cleanup hook. Without it, even with the cleanup-fix in the prior commit, a hard-killed Hermes process leaves its container behind permanently because there's no subsequent Hermes process scheduled to reuse that exact (task, profile) pair. docker)--filterlabel=hermes-agent=1rAz status=exitedrAlabel=hermes-profile=ps-a--formatz{{.ID}}Tutf-8replaceFcapture_outputtextencodingerrorstimeoutcheckstdinz"orphan reaper docker ps failed: %sNrz'orphan reaper docker ps returned %d: %sc^g|]*}||+Sr.0lns r z*reap_orphan_containers..s-TTTBTRXXZZTTTrrm-frKrLrMrNrOrQz2Reaped orphan container %s (exited %d seconds ago) docker rm -f %s failed: %sz%orphan reaper docker rm %s failed: %s) find_dockerextendr5 subprocessrunDEVNULLTimeoutExpiredOSErrorrdebug returncodestderrrstdout splitlinesdatetimenowtimezoneutc_container_finished_at total_secondsinfor#)r<r=r>r@filterslistinge candidate_idsrlrmremovedcid finished_atageresults rreap_orphan_containersr|sB 4;== 4HFOOOGf $cr}r{ruraw_rerls rrprps J0G VdWYXZbg$     %w / Atj|tjr|atd||Stj d}|r|a|Stj d}|r|atd||StD]a}tj|r@tj|tjr!|atd||cSbdS)uLocate the docker (or podman) CLI binary. Resolution order: 1. ``HERMES_DOCKER_BINARY`` env var — explicit override (e.g. ``/usr/bin/podman``) 2. ``docker`` on PATH via ``shutil.which`` 3. ``podman`` on PATH via ``shutil.which`` 4. Well-known macOS Docker Desktop install locations Returns the absolute path, or ``None`` if neither runtime can be found. NHERMES_DOCKER_BINARYz'Using HERMES_DOCKER_BINARY override: %sr@podmanz%Using podman as container runtime: %sz%Found docker at non-PATH location: %s) r osgetenvpathisfileaccessX_OKrrrshutilwhich_DOCKER_SEARCH_PATHS)overridefoundrs rr`r`s7%!!y/00HBGNN8,,8RW1M1M% =xHHH L " "E "  L " "E " ;UCCC % 7>>$   BIdBG$<$< !%  KK? F F FKKK 4r)z --cap-dropALL --cap-add DAC_OVERRIDErCHOWNrFOWNERz--security-optzno-new-privileges--tmpfsz/tmp:rw,nosuid,size=512mrz#/var/tmp:rw,noexec,nosuid,size=256m2561g extra_argsc8td|pgDS)zTrue when user-supplied docker_extra_args already set ``--shm-size``. In that case we skip our default so the user's value is unambiguous (rather than relying on flag-ordering / last-wins behavior). c3vK|]4}t|to|dkp|dV5dS) --shm-sizez --shm-size=N)rrr)rVas r z+_extra_args_set_shm_size..ws[  1cQ\ 1 PQ\\-5P5Pr)any)rs r_extra_args_set_shm_sizerqs7 "  r)rz/run:rw,noexec,nosuid,size=64m)rz/run:rw,exec,nosuid,size=64m)rSETUIDrSETGIDc  ddlm}ddlm}n7#t$r*}t d|gigfcYd}~Sd}~wwxYw|}|dpi}|dsgigfS|}t|dd }|j s3d }|rt|t d |gigfS|j r|js`` per minted mapping * ``host_args`` — extra ``--add-host`` flags so the container can reach the host-side proxy (Linux needs ``host.docker.internal:host-gateway``; Docker Desktop populates this automatically on macOS/Windows) Returns three empty containers when the proxy is disabled, not yet set up, or not currently running. If ``proxy.enforce_on_docker`` is true and the proxy is enabled-but-not-running, raises ``RuntimeError`` so the docker backend refuses to start the sandbox. r load_config iron_proxyz%Egress proxy plumbing unavailable: %sNproxyenabledenforce_on_dockerTzvproxy.enabled is true but iron-proxy is not configured. Run `hermes egress setup` to mint tokens and write proxy.yaml.u:%s — continuing without proxy (enforce_on_docker=false).z.iron-proxy is enabled but not running on port z&. Start it with `hermes egress start`.z!iron-proxy CA cert vanished from z0. Re-run `hermes egress setup` to regenerate it.ziron-proxy is configured but mappings.json is empty or corrupt. Re-run `hermes egress setup` to mint provider tokens before starting a sandbox.z#/etc/ssl/certs/hermes-egress-ca.crt-v::rozhttp://host.docker.internal:r]z127.0.0.1,localhost,::11--use-openssl-ca) HTTPS_PROXY https_proxy HTTP_PROXY http_proxyNO_PROXYno_proxyREQUESTS_CA_BUNDLE SSL_CERT_FILECURL_CA_BUNDLENODE_EXTRA_CA_CERTSHERMES_EGRESS_PROXY"_HERMES_EGRESS_NODE_OPTIONS_APPENDHERMES_PROXY_TOKEN_alias_env_namesrSz --add-hostz!host.docker.internal:host-gateway)r+ragent.proxy_sourcesr ImportErrorrrgget get_statusr% configured RuntimeErrorrpid listening tunnel_port ca_cert_pathexists load_mappings proxy_token real_env_namegetattr)ripexccfg proxy_cfgstatusenforcemsgmappings container_ca volume_args proxy_urlplain_http_url env_overridesmalias host_argss r_egress_proxy_args_for_dockerrs41111118888888  Q 1266<" 1 1E#$=M%  1 )*MNI  22s  A>AArrrc|s|s|sdStj|||ddd}tj|dddS) z@Stable Docker-label value for the egress posture of a container.off)rrrT),r) sort_keys separatorsrGN)jsondumpshashlibsha256encode hexdigest)rrrpayloads r_egress_reuse_fingerprintr.s =Iuj&*"  G >'..11 2 2 < < > >ss CCrTr9c ddlm}t|dpid|S#tt f$r|cYSt $r|cYSwxYw)z7Read proxy.enforce_on_docker with fail-safe defaulting.rrrr)r+rr%rrrfr,)r9 _load_cfgs r_egress_enforce_on_dockerrBs>>>>>>YY[[__W--3889LgVVWWW  ! sAAA+ A+*A+cLhd}|d|D|S)zDEnv names that would weaken or bypass enforced egress if overridden.> rrrrrr NODE_OPTIONSrrrrc3nK|]0}|ds|d,|V1dS)_API_KEY_TOKENN)endswith)rVrs rrz-_critical_egress_env_names..Vs\ << # #'*||H'='= r)update)rcriticals r_critical_egress_env_namesrNsNH  OO$ Orcritical_namescNg}hd}ddh}d}|t|kri|||dzt|kr ||dznd}|vrWdkr|n5|ddd}||vr|||d z }tfd |Dr{d r|dndddddd}||vr||n4|vstfd |Dr||dz }|t|kit t |S) zCReturn docker_extra_args entries that can override egress controls.>--env-e --env-filez --networkz--netrr]r=c3HK|]}|dVdSrNrrVflagargs rrz0_extra_args_egress_collisions..qs5@@ds~~jjj))@@@@@@rz --env-file=c3HK|]}|dVdSrrrs rrz0_extra_args_egress_collisions..xs5(^(^4 )C)C(^(^(^(^(^(^r)lenrsplitrrsortedr) rr collisions env_flags network_flagsinxtnamer s @r_extra_args_egress_collisionsr]sJ---I '*M A c*oo  m#$q53z??#:#:jQ )  l""!!#&&&&yya((+>))%%d+++ FA  @@@@i@@@ @ @ #~~m,, ,!!,////yya((+11#q99!<>))%%d+++ M ! !S(^(^(^(^P](^(^(^%^%^ !   c " " " Q+ c*oo  , #j// " ""rFrun_as_host_userrun_execct|rtnt}tt|z}|r|S|ttzS)aIReturn the security/cap/tmpfs args tailored to the privilege mode. ``run_exec`` mounts ``/run`` with ``exec`` instead of the hardened ``noexec`` default. This is required for s6-overlay images whose ``/init`` entrypoint execs ``/run/s6/basedir/bin/init`` during startup; see ``_image_uses_init_entrypoint``. )list_RUN_TMPFS_EXEC_RUN_TMPFS_NOEXEC_BASE_SECURITY_ARGS_PRIVDROP_CAP_ARGS)rr run_tmpfsargss r_build_security_argsr ~sOG__6GHHI # $ $y 0D $)** **rimagec  tj|dd|ddgdddddtj }n@#tjtf$r'}t d ||Yd }~d Sd }~wwxYw|jd kr;t d||j|j d S|j pd }|r|dkrd S tj |}n#ttf$rYd SwxYwt|t r|g}t|t"r|sd St!|d  }|dvS)aReturn True if ``image``'s entrypoint is the s6-overlay ``/init``. Such images (e.g. anything built on ``s6-overlay``, including ``hermes-agent:latest``) already provide their own PID-1 init and execute ``/run/s6/basedir/bin/init`` during stage0 startup. They are incompatible with Docker's ``--init`` (two competing PID-1 inits) and with a ``noexec`` ``/run`` mount. Detection is best-effort: on any inspection failure we return False and keep the hardened defaults. r!rrFz{{json .Config.Entrypoint}}TrGrHrIr\z/Docker: could not inspect entrypoint for %s: %sNFrz4Docker: image inspect for %s returned %d (stderr=%s)rnull)z/initz&/package/admin/s6-overlay/command/init)rbrcrdSubprocessErrorrfrrgrhrirrjrloadsr TypeErrorrrr)r>r!r{rur entrypointfirsts r_image_uses_init_entrypointr)s  )U 6 8 $      & 0 FqQQQuuuuuA  B 6$fm&9&9&;&;   u = B % % ' 'C #--uZ__  "uu*c""" \ j$ ' 'zu  1   $ $ & &E G GGs'+.A+A&&A+C//DDcttdd}ttdd}||dS |d|S#t$rYdSwxYw)auReturn ``:`` for the current host user, or ``None`` on platforms where this is not meaningful (e.g. Windows without posix ids). We intentionally read ``os.getuid()``/``os.getgid()`` directly rather than going through ``getpass``/``pwd`` so this stays cheap and never raises on nameless UIDs (nss lookups can fail inside sandboxed launchers). getuidNgetgidr)rrr,)get_uidget_gids r_resolve_host_user_specr/s{b(D))Gb(D))G'/t'))))ggii))) ttsA AA_storage_opt_ok_cgroup_limits_okcttSt}|r|sdadS tj|ddddddd d |d d g d d dddtj}|jdkatsr{rus r_cgroup_limits_availabler;s1"$  J U!uV  uj% GS "  $   #-2   NN# $"++--dsd3     VVV!QSTUUUUUUUUV sA?B&& C0CCc t}|s+tdtdd t j|dgddddd tj }|jd krJtd ||j|j td ddS#t$r/td|dtddtj $r/td|dtddt$rtddwxYw)zBest-effort check that the docker CLI is available before use. Reuses ``find_docker()`` so this preflight stays consistent with the rest of the Docker backend, including known non-PATH Docker Desktop locations. zDocker backend selected but no docker executable was found in PATH or known install locations. Install Docker Desktop and ensure the CLI is available.z|Docker executable not found in PATH or known install locations. Install Docker and ensure the 'docker' command is available.zNInstall Docker (or fix PATH) and retry, or switch terminal.backend to 'local'.) retry_hintversionTrGrHr\rzIDocker backend selected but '%s version' failed (exit code %d, stderr=%s)zXDocker command is available but 'docker version' failed. Check your Docker installation.zaThe Docker daemon may be down or the current user lacks permission (docker group). Fix and retry.zVDocker backend selected but the resolved docker executable '%s' could not be executed.)exc_infozHDocker executable could not be executed. Check your Docker installation.z)Repair the Docker installation and retry.zYDocker backend selected but '%s version' timed out. The Docker daemon may not be running.zHDocker daemon is not responding. Ensure Docker is running and try again.znStart the Docker daemon (e.g. `systemctl start docker` or launch Docker Desktop), then retry the same command.z4Unexpected error while checking Docker availability.N) r`rerrorrrbrcrdrhrirFileNotFoundErrorrer,)r>r{s r_ensure_docker_availablerCs J        ) K/     7  # $    N   ! ! LL,! ##%%    -2@   " !A            ) VB      $      4     ) VG      B      s 'B;;BEc%eZdZdZdZdeedffdZdddddd d d d d dd d d d defd ed ede de de de de dede de ed zde d zde deede de de de def$fd Zde efdZde eeffd Zdee eefeeffd!Zdee eeedfffd"Zde efd#Zd d$d d%d&ed'e de d(ed zdejf d)Zd*Zd+ede fd,Zde fd-Zd=d/ed ede ffd0 Zede fd1Zd2edeefd3Zd4ed5ed6edeeeeffd7Z d d8d9e fd:Z!d>de de fd<Z"xZ#S)?DockerEnvironmentuHardened Docker container execution with resource limits and persistence. Security: all capabilities dropped, no privilege escalation, PID limits, size-limited tmpfs for scratch dirs. The container itself is the security boundary — the filesystem inside is writable so agents can install packages (pip, npm, apt) as needed. Writable workspace via tmpfs or bind mounts. Persistence: when enabled, bind mounts preserve /workspace and /root across container restarts. Tr .c*t|jS)z@Keep explicit docker_forward_env values out of shared snapshots.)tuple _forward_envselfs r,_additional_profile_scoped_passthrough_namesz>DockerEnvironment._additional_profile_scoped_passthrough_namesbsT&'''r/rootr:rFr9Nr!cwdrOcpumemorydiskpersistent_filesystemtask_idvolumesr rnetworkhost_cwdauto_mount_cwdrrpersist_across_processesshm_sizec  !QRSTU|dkrd}t|||_|_d_|_t | _t| _ d_ d_ i_ d_ d_d_g_t"d| | 2t'| t(st"d| g} t-g}|d kr3t/|r$|d t3|g|d kr)t/|r|d |d gt/|r|d t4gt3|pd}|r,|dkr&t9|s|d|g|d krZt:jdkrJr|dd|d gnt"d| s| dd dl!m"}g}d}| pgD]}t'|t2st"d|3|}|sJd|vr|d|gd|vrd}lt"d|| rd}(hd?}/tS d d@lBmC}0dA|0DDSn#th$rYnwxYw|/Sz}1tsSUfdB|1D}2|2rd}4n&#t|t~f$rd}4Ynth$rd}4YnwxYw|4r,Ur*tj }5|5FUn)tU}5|5Fj |5GdFd}6|6r|5=dGd}7|7H}8dHdIhQ|6TTQvr|>rt"dS|t|otO|<|>T}?t"dU|g}@|pgD]H}At'|At2st"dV|A3|@ |AIUrNt|@R}B|Br_|C_dadbdc|Fdd|Et|'i_ d}H|rV|F|E|'}I|I|I\}J}Kd}Ld}M| sW|J}M|Mdek}L|Lrt"df|Jddg|Mpdh|F|E tjYjLdidj|JgdddkdldmdtjZnnG#tj[t~f$r.}$t"do|Jddg|$Yd}$~$nd}$~$wwxYwd}I|I|I\}J}K|J_ |Kdpkr tjYjLdq|JgdddkdldmdtjZnnT#tj\tj[f$r6}$t"dr|Jddg|K|$d_ Yd}$~$nd}$~$wwxYwj r(t"ds|Jddg|F|E|Kd}H|Hs-|>rgndtg}NjLdudvg|Ndw|D|Gdx||C|dydz}Ot")d{dMI|O tjY|Odddkdld|dtjZn}Pnk#tj\tj[f$rM}$t"d}|D|$tjYjLdidj|Dgdd~tjZd}$~$wwxYw|Pj]_ t"d|Dj ddg^__`dS)N~rL)rMrOFrSrzDockerEnvironment volumes: %sz'docker_volumes config is not a list: %rrr3r4rr6r9rdarwin --storage-optzsize=zDocker storage driver does not support per-container disk limits (requires overlay2 on XFS with pquota). Container will run without disk quota.z--network=none)get_sandbox_dirz'Docker volume entry is not a string: %rrrz :/workspaceTz*Docker volume '%s' missing colon, skippingz@Skipping docker cwd mount: host_cwd is not a valid directory: %sr@home)exist_okz:/root workspacerz/workspace:rw,exec,size=10g)rz/home:rw,exec,size=1grz/root:rw,exec,size=1gz.Mounting configured host cwd to /workspace: %szDSkipping docker cwd mount: /workspace already mounted by user config)get_credential_file_mountsget_skills_directory_mountget_cache_directory_mounts host_pathugDocker: skipping credential mount — source is a directory (likely Docker-in-Docker auto-creation): %su:Docker: skipping credential mount — source not found: %scontainer_pathrz$Docker: mounting credential %s -> %su?Docker: skipping skills mount — source is not a directory: %sz$Docker: mounting skills dir %s -> %su>Docker: skipping cache mount — source is not a directory: %sz#Docker: mounting cache dir %s -> %sz1Docker: could not load credential file mounts: %sc3$K|] }|v|V dSNrS)rVr_critical_egress_namess rrz-DockerEnvironment.__init__..Ps7))C;Q4Q4Q4Q4Q4Q4Q))rz@docker_forward_env would inject real egress-protected variables z; enforce_on_docker is rdisabled.zi Remove these names from docker_forward_env or disable enforce_on_docker to opt out of egress isolation.zC%s Explicit docker_forward_env values will override egress tokens.rrz:Could not read proxy config for egress collision check: %sr> rrrrrrrrrrrch|] }|j SrS)r)rVrs r z-DockerEnvironment.__init__..s'+++()AO+++rc3K|]I}|jvr>|vsj||kr#|vs|vrj||kE|VJdSrg)_env)rVk_critical_provider_keysegress_env_overridesrJs rrz-DockerEnvironment.__init__..s!! >>111y|';A'>>>000111 IaL,@,CCCDCCC!!rz;docker_env in config.yaml overrides egress-proxy variables z` Remove these keys from docker_env or disable enforce_on_docker to opt out of egress isolation.zX%s Falling back to docker_env values; sandbox traffic will NOT route through the proxy.rrrz--use-bundled-cac(g|]}|v|k |SrSrSrVt_CA_MODE_FLAGS append_tokens rrXz.DockerEnvironment.__init__..s/cccn9L9LQRVbQbQb1QbQbQbrzOverriding conflicting NODE_OPTIONS CA-mode flag(s) %s with egress-required %s to keep Node routed through the egress CA store.c(g|]}|vs|k |SrSrSrss rrXz.DockerEnvironment.__init__..s/"n"n"n.AXAX\]am\m\m1\m\m\mr rrz--userz)Docker: running container as host user %szdocker_run_as_host_user is enabled but this platform does not expose POSIX uid/gid; container will start as its image default user.ugDocker: image %s uses /init (s6-overlay) as entrypoint — skipping --init and mounting /run with exec.)rzDocker volume_args: %sz/Ignoring non-string docker_extra_args entry: %rz7docker_extra_args would override egress-proxy controls zP Remove these args or disable enforce_on_docker to opt out of egress isolation.z2%s Extra Docker args may bypass egress isolation.zDocker run_args: %shermes---labelzhermes-agent=1zhermes-task-id=zhermes-profile=z hermes-agentrhermes-task-idhermes-profilenoneuExisting container %s has NetworkMode=%s but docker_network=false requests an air-gapped container — removing it and starting fresh (task=%s, profile=%s).r^r/rYrZrGrHr[rJz,Failed to remove mismatched container %s: %srunningstartu[Failed to start existing container %s (state=%s): %s — falling back to a fresh container.z:Reusing container %s (task=%s, profile=%s, prior state=%s)--initrc-d--name-wr8infinityzStarting container: %sxz@*,0')  $&). (* 3W===  z'4'@'@  NNDg N N NG !"""  77/667  (CHH!5 6 6 6 A::2599:  *lll!; < < < #E * * H  .2E!F G G G(.b!!''))  63#::&>z&J&J:  ,!4 5 5 5 !88 00**,, $$ot%GHHHHe 3  !1 2 2 2 <;;;;; ',$Mr R RCc3'' H#NNN))++C czz""D#;/// C''370KSQQQQHPXrwrw'9'9('C'CDDDVX  1\"" 1 l++ 110   gh grw}}\/J/J g LL[]e f f f-1(,   %o''(2WN2OTTT$ : , 01!; : < <   < 455zz||NNY""#K0VV<@P3QVVV$ : - !12 :9;;   ;{344zz||NNX""";/TT+>N2OTTT$ 9 , 01 " Q Q Q LLLa P P P P P P P P Q * + + C02B1  46F  455!;>2Et#L#LMMO''' #14 # NNNNNN++-=-K-K-M-M+++''    /2III !!!!!!$!!!K" G!,GG$3CyyGGG #&%%% 89= ) Q Q Q Q Q Q$())++//88>B($//%% ! !W% ) ) )$( ! ! ! ) ) )$( ! ! ! ) ! )%9 )diJ   2 3 3 3 3233J   di ( ( ()nn 0$    5&NN>2>>M+1133O12DEN.4466L~--cccccocccNN+,3\ #o"n"n"n"no"n"n"n?22&&|444),/)B)B)H)H)J)JJ~ &n- 5~t444*%% @ @C OOTc#=#=JsO#=#=> ? ? ? ? "  /11I$%y1  GSSSS*'==4H99I5QQ   KK?    -  0i'    ,k:::$" ( (Cc3'' PRUVVV  " "3 ' ' ' '   =!7!! ! G(GG$3CyyGGG #&:::H$                )<888:4:<<#3BQB#799--E-G-GHH *733 ' 555 777 +<">|"L"LK$/6$9M $NN1%SbS);+C)"L  m"!-tT<H+/!% $&"'","4 '5w?mmm'UWcdgegdgWhjkllllllllm#H#&.# e%1"I%%2"!-w E+/!% $&"&","4 '9:;TU222H("-ua .2****** 2%"KKT$SbS):|U"F, ^1@xjI %   )          $ G LL1388G3D3D E E E ##' $, 1:3LM   R"A%tT>B#'$, # $"(!4!4!6!6D  KK3^TEWX[Y[X[E\ ] ] ]#7799 s-H#\ ]\;;]<%`""a<7 a<aa7 a a77a<5$c c'&c' Aff5' f54f5'/xy-$yy5.z$${5?,{00{5>&~%%A@ AA@@A@ cj|\}}t|j}|||D]}||dt t ||_g}t |D]$}|d|d||g%|S)zBuild -e KEY=VALUE args for injecting host env vars into init_session. These are used during init_session() so that export -p captures the configured environment and the current profile's forwarded values. Nrr) _resolve_passthrough_envr!rnrrrGrrra)rJpassthrough_env unset_namesexec_envrrrs rrz&DockerEnvironment._build_init_env_argss (,'D'D'F'F$#' ??((( % %D LLt $ $ $ $-26+3F3F-G-G*(## : :C KK#77 778 9 9 9 9 rc6|dS)zBResolve forwarded host variables through the active profile scope.r)rrIs r_build_passthrough_envz(DockerEnvironment._build_passthrough_envs,,..q11rcvi}t|j}t}d}d}d} ddlm}m}ddlm}m} |}| }t|}n#t$rYnwxYwd|D} || tz z} | rtni} t} t| D]}}tj |p| |}| |||}||||<A|r:||s/t|r| |~|| fS)zz.serr)get_all_passthroughresolve_passthrough_value)_is_global_envis_multiplex_activec0h|]}t||SrS)r )rVros rrlz=DockerEnvironment._resolve_passthrough_env..$s6   /I!/L/L    r)rrHtools.env_passthroughrragent.secret_scoperrr,rr-rrrrr fullmatchr)rJrexplicit_forward_keyspassthrough_keysrmultiplex_active is_global_envrr_is_multiplex_active_implicit_forward forward_keys hermes_envrrr&s rrz*DockerEnvironment._resolve_passthrough_env s#% #D$5 6 6%(UU$(! ++           g f f f f f f f*M3355 "#6#6#8#899      D   '   -0ADb0bc 0<D*,,," # ,'' % %CIcNN9jnnS&9&9E(411#u== % ! %--*<*< %AQA[A[\_A`A` %$$$$$s3A!! A.-A.c|\}}g}t|D]$}|d|d||g%|tt|fS)zFBuild runtime forwarding args plus names absent from the active scope.rr)rrrarG)rJrrrrs r#_build_runtime_env_args_with_unsetsz5DockerEnvironment._build_runtime_env_args_with_unsets4s~'+'D'D'F'F$/** A AC KK#>>(<>>? @ @ @ @U6+..////rc6|dS)z|js Jd|jdg}||dd}|r||jn3|jr,|\}}|||rt|dd}|r'dd|D}d |d |}||jg|r|d d d |gn|d d |gt||S)z1Spawn a bash process inside the Docker container.zContainer not startedexecNz-irSrrxc3>K|]}tj|VdSrg)shlexquote)rVrs rrz.DockerEnvironment._run_bash..Vs,#N#N$EK$5$5#N#N#N#N#N#Nrzunset z 2>/dev/null || true bashz-lz-c) rrrrar_profile_scoped_passthroughrrrr) rJrrrOrcmdr runtime_args quoted_namess r _run_bashzDockerEnvironment._run_bash@s[!::#:::!(  ! JJt    (*  % JJt* + + + +  - %(,(P(P(R(R %L+ JJ| $ $ $  M!$(GLLK  S88#N#N+#N#N#NNNLR,RRjRRJ D&'(((  3 JJdJ7 8 8 8 8 JJj1 2 2 23 +++r)zNo such containerzis not runningzno such containeroutputcDtfd|jDS)zCReturn True if the output indicates the container no longer exists.c3 K|]}|vV dSrgrS)rVprs rrz7DockerEnvironment._is_container_gone..ns'DD11;DDDDDDr)r_NO_CONTAINER_PATTERNS)rJrs `r_is_container_gonez$DockerEnvironment._is_container_gonels)DDDD(CDDDDDDrc |jpddd}td|d|_|jdd}|jdd}||||jt d}||\}}|dkr+||_td |ddn tj |j d |gd d d d dd tj ||_td|ddnL#tj tj f$r.}td|dd|Yd}~nd}~wwxYw|js|jstddS ddl}d|jdd} |jrgndg} g} |jD]!\} } | d| d| g"|j ddg| d| | d|j|j|jdd}tj |d d d d d d tj }|j|_| |_td!| |jddnJ#tj tj t6f$r&}td"|Yd}~dSd}~wwxYw d|_|n3#t<$r&}td#|Yd}~dSd}~wwxYwtd$|jpdddd S)%a9Recreate the container after it was removed out-of-band. Tries label-based reuse first; if no existing container is found, starts a fresh one with the same image and run-args. Returns True on success, False if recreation fails (caller should surface the original error). rNr^u7Container %s appears to be gone — attempting recoveryr|r}rrz&Recovery: reusing running container %srTrGrHr[rJz Recovery: restarted container %sz*Recovery: failed to start container %s: %sz8Recovery: no saved image name, cannot recreate containerFrryrzrr{rrcrrrr8rrz)Recovery: created fresh container %s (%s)z,Recovery: failed to create new container: %sz2Recovery: init_session failed in new container: %su(Recovery successful — new container %s)rrrrrrrrrrbrcrrdrrerrArrrrr"rarMrrjrrrf_snapshot_readyrr,)rJold_idr profile_labelrrxrru_uuidnew_namerrrovrr{s r_recreate_containerz%DockerEnvironment._recreate_containerpsO$*CRC0 Ev   "\%%&6;;  (()92>> 00  t|'7'78I5'Q'Q    !JC !!%(" Dc#2#hOOOO ^N)7C8'+$QZdfnr(0 *-D&KK BCHMMMM"5z7PQ^^^NN#OQTUXVXUXQY[\]]]]]]]]^!! ;  WXXXu $$$$>>>$eT   '     (  ' K  ( $DtgV_iltx$,&,]%8%8%:%:"'/$ ?d0"51:3LgV    KQOOOuuuuu   #(D           LLMq Q Q Q55555   >ASAYWY[^\^[^@_```tsJAD11E: $E55E:)DJ<>LL## M-MMrcommandc 0tj||fi|}|dddkr_||ddr6|jr/|rtj||fi|}|S)a Execute a command, auto-recovering from dead containers. If the container was removed out-of-band (idle reaper, docker prune, OOM kill, daemon restart), detect the error and recreate the container transparently before retrying once. rhrrr)rexecuterrrr)rJr rMkwargsr{rs rr"zDockerEnvironment.executes!#8888 JJ|Q ' '1 , ,'' 8R(@(@AA -. -'')) A(#@@@@ rc dttS tpd}tj|dddgddddd tj }|j}|d krd ad Stj|d dddgdddddtj }|jdkrC|j}|r%tj|d|gddtjdand an#t$rd aYnwxYwt dttS)zCheck if Docker's storage driver supports --storage-opt size=. Only overlay2 on XFS with pquota supports per-container disk quotas. Ubuntu (and most distros) default to ext4, where this flag errors out. Nr@rrrFz {{.Driver}}TrGrHrr\overlay2Fcreater\zsize=1mz hello-worldrIrrYr?rz Docker --storage-opt support: %s) r0r`rbrcrdrjrlowerrhr,rrg)r@r{driverprober}s rrz(DockerEnvironment._storage_opt_supportedsk  &" " $ ]].hF^];#$\^ (F ]((**0022F##"'uN?I}M#$\^ (E 1$$$|1133 =NFD,#?26)3);====#'"' $ $ $#OOO $ 7IIIsA,C9>A:C99 DDr}c  tj|jddd|gddddddtj }n?#tjt f$r&}t d |Yd }~d Sd }~wwxYw|jd kr:t d |j|j d S|j }|pd S)aReturn the container's ``HostConfig.NetworkMode`` (e.g. ``bridge``, ``none``, ``host``), or ``None`` when inspection fails. Used by the reuse path to make sure a persisted container's network mode still matches the operator's ``docker_network`` setting; callers treat ``None`` (unknown) as a mismatch when lockdown was requested, so a failed inspect fails closed rather than open. rrFz{{.HostConfig.NetworkMode}}TrGrHrFrJz%docker inspect NetworkMode failed: %sNrz*docker inspect NetworkMode returned %d: %s) rbrcrrdrerfrrgrhrirrj)rJr}r{rumodes rrz)DockerEnvironment._container_network_modes ^$i =  $GI (   FF)73    LL@! D D D44444    ! ! LL<!6=#6#6#8#8   4}""$$|ts03A/ A**A/rrrc * dddd|dd|g}|dkr%|ddtd|gd}n d tzd z}tj|jd d g|d |ddddddtj}n?#tjtf$r&}t d|Yd}~dSd}~wwxYw|j dkr:t d|j |j dSd|j D}|sdSd} d} |D]} |dkr|| dd} t!| dkr2| d| d| d}}} |dvr#t d| t|nL| dd} t!| dkr| d| d}} | | |f} |dkr| | |f} | p| S) uLook for an existing container labeled for this (task, profile). Returns ``(container_id, state)`` on hit, ``None`` on miss / on any failure (including ``docker ps`` itself failing). State is one of the values Docker reports via ``{{.State}}`` — e.g. ``running``, ``exited``, ``created``, ``paused``, ``restarting``, ``dead``. The caller decides whether the state warrants ``docker start`` before reuse. Restricted to the docker-stored label set this class creates; never matches containers that happened to be named ``hermes-*`` but were started by some other tool. rArBzlabel=hermes-task-id=rCrzlabel=rz{{.ID}} {{.State}}z{{.ID}} {{.State}} {{.Label "z"}}rDrErFTrGrHrFrJu;docker ps probe failed: %s — will start a fresh containerNru@docker ps probe returned %d: %s — will start a fresh containerc:g|]}||SrSrTrUs rrXz>DockerEnvironment._find_reusable_container..[s%GGGBHHJJGGGGr rr])rz rz7skipping container %s for egress=off reuse: label %s=%rr)rarrbrcrrdrerfrrgrhrirrjrkr r r')rJrrrrsfmtr{rulinesrr(rWpartsrxr egress_vals rrz*DockerEnvironment._find_reusable_container"s$ 2@J@@CMCCG u$$ ,W5F,W,W,W,WXYYY+8:KKeS^$dD!$ $GI (   FF)73    LLVXY Z Z Z44444    ! ! LLR!6=#6#6#8#8   4GGfm6688GGG 4  ' 'Bu$$q))u::>>).q58>>3C3CU1XJU%>>>LL&'*,=z ?q))u::??"1XuQx~~'7'7U}e  !!go,%sA:A==B9B44B9) force_remover4c |js2|js)|j|jfD]}|rt j|ddS|rd d n|jr d|_dSd d |jddd  fd }ddl}| |dd }| ||_ d|_ r0|js+|j|jfD]}|rt j|ddSdSdS) u} Tear down the container according to persist mode and *force_remove*. Persist-mode (``persist_across_processes=True``, the default) leaves the container **running** untouched. The docs promise "ONE long-lived container shared across sessions" and stopping it on every Hermes exit breaks that promise: * Background processes inside the container (``npm run dev``, watchers, long-running pytest) get killed every time the user runs ``/quit``. * Every reuse requires ``docker start`` + waiting for the container to come back up, adding 1–2s to the first tool call of the new session. * The user-visible difference between "ONE long-lived container" and "a new container that happens to share state" is exactly this: processes survive in the former, die in the latter. Resource reclamation for the persist-mode case lives in the ``reap_orphan_containers()`` path (see issue #20561 commit 3): if no Hermes process touches a labeled container for ``2 × lifetime_seconds`` it gets ``docker rm -f``'d at the next Hermes startup. That covers the SIGKILL / OOM / abandoned-laptop cases without us needing to stop the container on every graceful exit. Opt-out mode (``persist_across_processes=False``) still does ``docker stop`` + ``docker rm -f`` on every cleanup, matching the pre-PR behavior for users who explicitly want per-process isolation. ``force_remove=True`` overrides persist mode and always tears the container down (``docker stop`` + ``docker rm -f``). This is the explicit-teardown path for ``/reset``, ``cleanup_vm(task_id)``-driven resets, or any caller that wants a guaranteed fresh container on next ``DockerEnvironment(task_id=...)``. No current caller passes ``force_remove=True``; the parameter is here so the explicit-teardown semantics can be wired up later without changing this method's signature. Cleanup runs on a daemon thread with bounded ``subprocess.run`` calls (not the racy ``Popen(... &)`` pattern from before PR #33645). The atexit hook in ``tools/terminal_tool.py`` waits up to 15s for the thread to finish before the interpreter exits, so ``docker stop`` / ``docker rm`` actually completes when we do trigger it. T) ignore_errorsNr^r crh tjdddgddtjn?#tjtf$r&}t d|Yd}~nd}~wwxYwri tjdd gddtjdS#tjtf$r'}t d |Yd}~dSd}~wwxYwdS) Nstopz-t10Tr[rz%docker stop %s timed out / failed: %srYrZr_)rbrcrdrerfrr)rur}r>log_id should_remove should_stops r _do_cleanupz.DockerEnvironment.cleanup.._do_cleanupsH WWN#VT4F'+R(0 #17;WWWNN#JFTUVVVVVVVVW LLN#T4>'+R(0 #17;LLLNN#?KKKKKKKKKL L Ls-'-A)A$$A)/&BC-CCrzhermes-cleanup-)targetdaemonrr N) rrrrrrmtreerr threadingThreadr_cleanup_thread) rJr4dr=rBrtr}r>r:r;r<s @@@@@rcleanupzDockerEnvironment.cleanup~sT)  # =-t~>==A= at<<<< F  !K MM  + !"&D  FK M% crc" L L L L L L L L L L4    KC]U[C]C]  ^ ^   !  9!1 9)4>: 9 99M!48888 9 9 9 9 9 9r>@ct|dd}||sdS||| S)uBlock up to *timeout* seconds for the cleanup worker thread. Returns ``True`` if the thread finished (or no thread was started), ``False`` on timeout. The atexit hook in terminal_tool.py calls this on every active environment so docker stop/rm actually completes before the Python process exits — without this, ``hermes /quit`` races the interpreter shutdown and leaves stopped containers behind. rDNT)rO)ris_aliver)rJrOthreads rwait_for_cleanupz"DockerEnvironment.wait_for_cleanupsU0$77 >!2!2>4 G $$$??$$$$r)r)rG)$__name__ __module__ __qualname____doc__r rGrrK_DEFAULT_SHM_SIZEr#r$r%rr!rrrrrrrrrbPopenrrrrr" staticmethodrrrrFrK __classcell__)rs@rrErETs'  #'(eCHo((((&+ (,"&$!&)-)'N N N N  N   N   N N  $N N N #Y%N D[N N 3-N N  !N "#N $#'%N &'N N N N N N `d3i$2S#X2222&%%S#XC0H*I&%&%&%&%P0U49eCQTHo;U5V0000=c====;@!$+/ , , ,C ,4 , ,!Dj ,4>4D , , , ,L EEEEEERTRRRRhs"(D(((\(T C HSM    DZ Z Z  Z %S/ " Z Z Z Z x/4u9u9u9tu9u9u9u9n % % % % % % % % % % %rrE)T)Fr@)CrOrrloggingrrr rrbrrpathlibrtypingrrrrrtools.environments.localrr getLoggerrLrrr r__annotations__compilerrrrr!r'r-r2r5r;r#r|rpr`rrrPr%rrrrrGrrrrrrr r)r/r0r1r;rCrErSrrr[se      8 $ $  %)HSM(((2:9::#d3i$.>492TD[T#s(^<tCH~ RZ 233       #    "!%! [[[[$J[d [  [[[[| s #    F*Xc]****D  $    @;  ^3uT#YS#XS -Q'R^3^3^3^3BDcDS>DCyD DDDD(  t t     d38n S    #S #+.s8# #Y####B + +4 +4 +DQTI + + + +*HC*H*H*H*H*H*HZ#$#'$&&&$(8D>(((1C1D1111hMMMM`n%n%n%n%n%n%n%n%n%n%r