yjTd UdZddlZddlZddlZddlZddlmZddlZddlZddl Z ddl m Z ddl m Z mZmZmZddlmZddlZdae ed<dae ed<dd Zdd lmZdd lmZdd lmZddlZeje Z!eddZ"d e#fdZ$e$Z%dZ&ddl'Z'd e(fdZ)d e(fdZ*e*Z+ee+dZ,dZ-de.d e/fdZ0de.d e/fdZ1de.d e/fdZ2de3d ee.fdZ4e5hdZ6de d e d e/fd!Z7d"e d#e.d e8ee ee.ee.ffd$Z9d%e:d e/fd&Z;d'd(de.d)e d*e(d+ed"e d e.fd2Z?dd"e d3ee.d e.fd4Z@d5ZAd6ZBd7ZCd8ZDd%e:d e/fd9ZEd"e d:e(d e/fd;ZF dd"e dZGd?ee<d@ee<d ee.fdAZHdeDddfd"e d3ee.dBe(d:ee(dCee<d e.f dDZIdEe.dFe.d e/fdGZJd e/fdHZK dd/e.dIe.dJe.dKe(dLee.d ee.e ff dMZLejMdNZN dd/e.dIe.dOee.d@) osgetenvstripfloat ValueErrorhermes_cli.configrr Exception)env_valrrcfgvals r_resolve_download_timeoutr-Msi8"==CCEEG >> !    D  ::::::::kmmgc;2DEE ?::        4s!: AA 0A== B  B i c tdttjdS#tt f$r&tdtjpdcYSwxYw)uBest-effort host CPU count, honoring cgroup/affinity limits when set. Prefers ``os.sched_getaffinity`` (the CPUs this process may actually run on — respects container/cpuset pinning) and falls back to ``os.cpu_count()``. Returns at least 1. r)maxlenr#sched_getaffinityAttributeErrorOSError cpu_countrr_detect_host_cpusr7sk+1c".q1122333 G $+++1blnn)*****+s.14A('A(cXtjdd}|r) t|}|dkr|Sn#t$rYnwxYw ddlm}m}|}||ddd}|t|}|dkr|Sn#t$rYnwxYwtS) uResolve how many vision encode/resize bursts may run concurrently. Defaults to the host's usable core count (``_detect_host_cpus``) — no fixed ceiling, because the cap tracks the actual exhausted resource (CPU cores), not a magic number. The LLM call is NOT covered by this limit, so legitimate multi-image fan-out keeps full request concurrency; only the simultaneous CPU bursts are bounded so the event loop always keeps a core. Resolution order: HERMES_VISION_MAX_CONCURRENCY env → config.yaml auxiliary.vision.max_concurrency → host core count. Any value that parses to < 1 is ignored in favor of the next source so the cap can never be disabled into an unbounded encode storm. HERMES_VISION_MAX_CONCURRENCYrr/rrr r!max_concurrency) r#r$r%intr'r(rrr)r7)r*parsedrrr+r,s r_resolve_vision_cpu_workersr=si7<<BBDDG \\F{{     D   ::::::::kmmgc;2CDD ?XXF{{        s#A AA8B BBz vision-encode) max_workersthread_name_prefixcKddl}tj}|t|j|g|Ri|d{VS)uRun a sync encode/resize callable on the bounded vision CPU executor. Offloads CPU-bound image work to :data:`_vision_cpu_executor` so it (a) never runs on the caller's event-loop thread and (b) is bounded to the host's usable core count process-wide. Excess encodes queue on the executor instead of all running at once, leaving cores free for the loop. The LLM call must NOT be routed through here — only the encode/resize. rN) functoolsasyncioget_running_looprun_in_executor_vision_cpu_executorpartial)fnargskwargsrAloops r_run_encode_on_cpu_executorrKs{  # % %D%%/i/DTDDDVDD      rurlc|rt|tsdS|dsdSt|}|jsdSdS)z2HTTP(S) shape check only (scheme, netloc). No DNS.F)http://https://T) isinstancestr startswithr netloc)rLr<s r_image_url_shape_okrTs[ jc**u >>1 2 2uc]]F =u 4rcFt|sdSddlm}||S)zHValidate image URL for sync callers and tests (SSRF via sync DNS check).Fr) is_safe_url)rTtools.url_safetyrV)rLrVs r_validate_image_urlrXs9 s # #u,,,,,, ;s  rcVKt|sdSddlm}||d{VS)zAValidate remote image URL without blocking the event loop on DNS.Fr)async_is_safe_urlN)rTrWrZ)rLrZs r_validate_image_url_asyncr[sP s # #u222222""3'' ' ' ' ' ' ''rdatac4|dd}|drdS|drdS|drdS|d rd St|d kr|dd d kr|dd dkrdSdS)u2Magic-byte MIME sniff on raw bytes (authoritative; no extension trust). Returns ``None`` for anything without a recognized image header — including SVG, which has no magic bytes. The resolver special-cases SVG (sniffs ``rar_r`rfsvg_pathout_pathc ddl}|t|t||o|jdkS#t $rYnwxYw ddlm}ddl m }|t|}|V| |t|d|o|jdkSn#t $rYnwxYwddl }ddl }dd t|t|gd t|d d |gfD]}||dre ||d d d|j|r |jdkrd Sr#t $rY~wxYwdS)uBest-effort SVG → PNG rasterization. Returns True on success. Tries, in order: cairosvg, svglib+reportlab, then system rasterizers (rsvg-convert, inkscape). All are soft dependencies; if none is available we return False and the caller rejects the image with an actionable error rather than embedding an unsupported media_type that would wedge the session. rN)rLwrite_to)svg2rlg)renderPMPNG)fmtz rsvg-convertz-oinkscapez--export-type=pngz--export-filename=T)checkcapture_outputtimeoutstdinF)cairosvgsvg2pngrQexistsstatst_sizer) svglib.svglibrmreportlab.graphicsrn drawToFileshutil subprocesswhichrunDEVNULL) rirjrwrmrndrawing_shutil _subprocesscmds r_rasterize_svg_to_pngrs2 S]]S]]CCC  @X]]__%)``. SVG is rasterized to PNG (best-effort, soft deps). Other raster formats Pillow can read (BMP, TIFF, etc.) are re-encoded to PNG. This runs BEFORE the image is base64-embedded into conversation history, so an unsupported media_type can never reach the provider and wedge the session. N cache/visiontemp_vision_imagesTparentsexist_ok converted_.png image/svg+xmlr_)NNu8This is an SVG, which vision models cannot read directly, and no SVG rasterizer is installed (tried cairosvg, svglib, rsvg-convert, inkscape). Convert the SVG to PNG first — e.g. open it in a browser and screenshot it, or install a rasterizer (`pip install cairosvg`) — then re-run vision_analyze on the PNG.rImage)RGBRGBALrroformatz'Failed to normalize %s image to PNG: %sz Image format z is not supported by the vision API and could not be converted to PNG (install Pillow for raster conversion). Convert it to PNG or JPEG and try again.) _ANTHROPIC_SUPPORTED_MEDIA_TYPESrmkdiruuiduuid4rPILropenmodeconvertsaveryrzr{r)loggerwarning)rrout_dirrj _PILImage_img_excs r_normalize_to_supported_imagerDs 888=$..^-ABBG MM$M...8djll8888H'' X 6 6 /[$. .   ,****** ^^J ' ' .4y 444||F++ IIhuI - - - . . . . . . . . . . . . . . . ??   /!81!._ssrf_redirect_guards VUUUUUUU44X>>  &7&7 &E&E E E E E E E OOO     rrcreate_ssrf_safe_async_clientrrufollow_redirects event_hooksoMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36zimage/*,*/*;q=0.8z User-AgentAcceptrrrr/z-Image download failed after %s attempt(s): %sdrz)Image download failed (attempt %s/%s): %s2zRetrying in %ss...zB_download_image exited retry loop without attempting (max_retries=r)rBparentrrangerrrWr_VISION_DOWNLOAD_TIMEOUTr_VISION_MAX_DOWNLOAD_BYTESr)rrrrQrsleep RuntimeError) rrrrBr last_errorattemptrrre wait_times r_download_imagersNNNTD999   J%%3+3+2 +*955G :%gi&8999 F F F F F F540!%'*>)?@        ..'X"5!(                                      "    + + +J022 gq6P6P CaKFF4C4L!  gk*I NNFRS U`befgbhbhiljlilbm n n n NN/ ; ; ;'- ** * * * * * * * * * * * *' +. [[[[[  s=AC$B:(C: C CC C F#B?FF#ct|j}dddddddd}||dS)z Determine the MIME type of an image based on its file extension. Args: image_path (Path): Path to the image file Returns: str: The MIME type (defaults to image/jpeg if unknown) r`r_rarbrfr)z.jpgz.jpegrz.gifz.bmpz.webpz.svg)suffixlowerr)r extension mime_typess r_determine_mime_typer?sO!''))IJ >>)\ 2 22r mime_typec|}tj|d}|pt |}d|d|}|S)a0 Convert an image file to a base64-encoded data URL. Args: image_path (Path): Path to the image file mime_type (Optional[str]): MIME type of the image (auto-detected if None) Returns: str: Base64-encoded data URL (e.g., "data:image/jpeg;base64,...") asciidata:;base64,) read_bytesbase64 b64encodedecoder)rrr\encodedmimedata_urls r_image_to_base64_data_urlr Vsf  " "Dt$$++G44G  8,Z88D/t..W..H Ori@i@iiPc|t|tfddDS)z;Detect if an API error is related to image or payload size.c3 K|]}|vV dSNr6.0hinterr_strs r z'_is_image_size_error..s74twr) too largepayload413content_too_largerequest_too_largerinvalid_requestexceeds size limit)rQrany)rrs @r_is_image_size_errorrsT%jj  G ,  r max_dimensionc ddlm}||5}t|j|kcdddS#1swxYwYdS#t $rYdSwxYw)uTrue if the image's longest side exceeds ``max_dimension`` px. Anthropic enforces an 8000px per-side cap independently of the 5 MB byte cap, so a tall small-byte screenshot can pass every byte check yet trip a non-retryable 400. Returns False (don't force a resize) when Pillow is unavailable or the file can't be read as an image — the byte-based checks still apply, and we never want a missing soft dependency to break the embed path. rrNF)rrrr0sizer))rrrrs r_image_exceeds_dimensionr"s****** ^^J ' ' 24ty>>M1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 uus3AA AAA A A AAregion offset_outcR ddlm}n#t$rYdSwxYwt|tt fr,t |dkstd|DsdS ||5}|j \}}d|D\}}} } tdt||} tdt||} tdt| |} tdt| |}| | ks|| kr*ddd |d |d | d | d |d |d |d |dfcdddS| | | | |f}|| |d <| |d<| | z |d<|| z |d<| |jdtjjddd}|jdvr|d}||d|ddfcdddS#1swxYwYdS#t*$r}ddd|fcYd}~Sd}~wwxYw)uCrop ``image_path`` to ``region`` = [x1, y1, x2, y2] (original-image pixels). Applied BEFORE :func:`_resize_image_for_vision` so the cropped area gets the full downscale resolution budget — a "zoom" into a detail region. Coordinates are clamped to the image bounds; a region that clamps to zero area (or is inverted/malformed) is rejected with an error naming the actual image dimensions so the caller can retry with sensible values. Ported from: QwenLM/qwen-code zoom-image.ts (Apache-2.0). Returns: (cropped_temp_path, out_mime, None) on success — the caller owns cleanup of the temp file — or (None, None, error_message) on failure. rr)NNz[region cropping requires Pillow (`pip install Pillow`); retry without the region parameter.rdc3zK|]6}t|ttfot|t V7dSr)rPr;r&boolrvs rrz%_crop_image_region..sB[[q:a#u..Jz!T7J7J3J[[[[[[r)NNzdInvalid region: expected [x1, y1, x2, y2] as four numbers (pixel coordinates in the original image).c34K|]}t|VdSr)r;r(s rrz%_crop_image_region..s(55c!ff555555rNzInvalid region [, zG]: crops to zero area after clamping to the image bounds. The image is xu+ px — pick x1r2rr)UFrrwhr cd*||fkr$dd<dd<|d<|d<dSdSdS)z;Publish the downscale into ``scale_out`` when dims changed.NrrIr/rJrKrLr6)rcrd orig_dimsrZs r _record_scalez/_resize_image_for_vision.._record_scale|s[  aVy%8%8&/lIl #'0|Im $%&Ik "&'Il # # # ! %8%8rc4dSt||kS)z3True if both pixel dimensions are within the limit.NT)r0)rcrdrs r_dims_okz*_resize_image_for_vision.._dims_oks!  41ayyM))rg?r^zResized to %dx%d (attempt %d)rqualityrrrz4Auto-resized image fits: %.1f MB (quality=%s, %dx%d)z=Auto-resize could not fit image under %.1f MB (best: %.1f MB))!rzr{rrrr0r!r)r r1ior3tools.lazy_depsr]rinforrrr.r/r;r'rresizeLANCZOSBytesIOrrrgetvaluerr)#rrrYrrZ file_size estimated_b64needs_resize_for_bytesneeds_resize_for_dims _PILQuick _quick_imgr r_io _ensure_depr  pil_formatout_mimer:rD quality_steps prev_dims candidatergrirscalenew_wnew_heffective_scaleqbuf save_kwargsr rfs# `` @r_resize_image_for_visionrs(!!)I]q(3.M*-=="   . . . . . . ++ 1zz''-77,0) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1    D  "*?,Z9MMM x==, , ,O -   = = = = = = K e 4 4 4 4 ! ! ! ! ! !         KKU V V V4Z9UUUOOOOO    ( KKu[)=K+H K0-AAA  8,Z88D+--6J(E11{{|Hjj$$  ?EEE  0yQQQH  VM 9 9kk%   %/&$8$8LLgMCJ'ICJ'II((((((((((*C*C*D****** 88#!#! Q;;ECI-..33ECJ.//44E{{sy1}}"$sy.C _ <==rBB"a"$sz/C O ;< google-vertexanthropic-vertexnousvertexbedrock openrouterT>anthropic-directclaude anthropic> openai-chat azure-openai openai-codexopenai> google-geminigoogle-vertex-geminigeminigooglezgemini-3z gemini-pro-3zgemini-flash-3r)get_provider_profile)rPrQr%r providersrsupports_visionr))rrp _AGGREGATORSmrprofiles r_supports_media_in_tool_resultsrsT( h $ $u  A u L Lt 777t EEEt III%%% 5 KKMM   ! ! ??n115E5J5J4u  222222&&q))  7#: 4      5s*C CCc6 ddlm}m}ddlm}m}ddlm}|}|}|}||||dkrdSt||p||||duS#t$r&}t d|Yd }~dSd }~wwxYw) aHWhether vision tools should attach the image to the main model directly instead of routing through the auxiliary vision LLM. True when image routing resolves to ``native`` AND either the provider is known to accept images inside tool results, or the user explicitly declared the model vision-capable via the ``model.supports_vision`` config override. The override is the escape hatch for custom/local providers that aren't in the static allowlist. Best-effort: any resolution failure returns False so the caller falls back to the legacy aux-LLM path. r)_read_main_provider_read_main_model)decide_image_input_mode_lookup_supports_vision)rnativeFTz(Native vision fast-path check failed: %sN) rrragent.image_routingrrr(rrr)rr) rrrrrrrr+rDs r#_should_use_native_vision_fast_pathrsPPPPPPPPXXXXXXXX111111&&((  ""kmm " "8UC 8 8H D D5 +He < < E&&x<<D   ?EEEuuuuusAA( A(( B2BBquestionimage_data_urlimage_size_bytes scale_notecd}t|tr.|r|d|z }|r|d|z }d|dz dd}dd |d d d |id g||dd|dddS)aBuild the multimodal tool-result envelope returned by the fast path. Shape: { "_multimodal": True, "content": [ {"type": "text", "text": ""}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}} ], "text_summary": "", "meta": {"image_url": ..., "size_bytes": N}, } The text part exists for two reasons: (1) it gives the model an instruction to act on now that the pixels are in context, and (2) providers that don't support multimodal tool results can fall back to ``text_summary``. ulImage loaded into your context — you can see it natively now. Use your built-in vision to answer the user.z Question: z Note: z,Image attached natively for the main model (.1fz# KB). Answer using built-in vision.TtexttyperrrLrrN)r size_bytes native_vision) _multimodalcontent text_summarymeta)rPrQr%)rrrrr text_partsummarys r _build_native_vision_tool_resultr-s: 7(C  9X^^%5%598hnn&6&6888 /.*...  ( t #/ ( ( ( Y / / ~/F G G  "4C4*!     rcKdWVdS)aDeprecated no-op shim kept for backward compatibility. The fan-out cap was narrowed to the CPU-bound encode/resize burst only (see :data:`_vision_cpu_executor` / :func:`_run_encode_on_cpu_executor`). Holding a slot across the whole analysis serialized legitimate multi-image workflows behind the slow LLM call, which is exactly what we don't want. This context manager no longer gates anything; encode/resize is bounded where it actually runs. Retained only so any external caller importing it keeps working. Nr6r6rr_vision_concurrency_slotrgs EEEEEErtask_idc n Kt|tr|stddSd}d} ddlm}|rQtdd|r=|< |r|SS#t$rYSwxYwSSddl m }m }m }  | ||| d{V} np#|$rh} tt| dcYd} ~ |r=|< |r|SS#t$rYSwxYwSSd} ~ wwxYw| j } t| j} t!d d }|d d |dt%jdz }t)j|j| jd{Vd }t)jt.|| d{V\}} }|s|St|pdd|r=|< |r|SS#t$rYSwxYwSS||krY|r:|r& |n#t$rYnwxYw|}d }|j} i}i}|t)jt4|||d{V\}}}|s|St|pdd|r=|< |r|SS#t$rYSwxYwSS|r:|r& |n#t$rYnwxYw|}|} d }|j} t7t8|| d{V}t|t:k}t7t<|t>d{V}|s|rt7t@|| t:t>|d{V}t|tBkrrtdt|dz ddtBdz ddd|r=|< |r|SS#t$rYSwxYwSStE|||| tG|pd|pd|r=|< |r|SS#t$rYSwxYwSS#t$ry} tH%d| td| dcYd} ~ |r=|< |r|SS#t$rYSwxYwSSd} ~ wwxYw#|r=|< |r|ww#t$rYwwxYwwwxYw)a.Fast path for vision-capable main models. Loads the image (data: / http(s) / file:// / local path / sandbox-container path) via the unified resolver, base64-encodes it, and returns a multimodal tool-result envelope. The agent loop unwraps it; provider adapters serialize it into the right tool-result-with-image shape for each backend. Returns: A ``_multimodal`` envelope dict on success. A JSON error string on failure (matches the existing tool-result contract so the agent loop displays errors normally). zimage_url is requiredFsuccessNris_interrupted InterruptedImageResolutionErrorResolveContextresolve_image_sourcerrrTr temp_image_.imgImage normalization failed.r$Region crop failed.r\)rrYrrZ2Image too large for vision API: base64 payload is r_r MB (limit .0fw MB) even after resizing. Install Pillow (`pip install Pillow`) for better auto-resize, or compress the image manually.)rrrrrz"Native vision fast path failed: %szNative vision failed: )&rPrQr% tool_errortools.interruptrryrr)tools.image_sourcerrrr r1r\rrrrrB to_thread write_bytesrrzr{rErKr _EMBED_TARGET_BYTESr"_EMBED_MAX_DIMENSIONr_MAX_BASE64_BYTESrrXrr)rrrr#temp_image_pathshould_cleanuprrrrresolvedrDdetected_mime_typertemp_dirnormalized_path _norm_err _crop_offset _scale_info cropped_path cropped_mimecrop_errr _over_bytes _over_dimss r_vision_analyze_nativervs$ i % %BY__->->B15AAAA&*ONC222222 >   <mU;;;v  o9 "))++-#**,,,,-      9k           711)^^T[=\=\=\]]]]]]]]HH# 7 7 7c#hh666 6 6 6 6 6X  o9 "))++-#**,,,,-      9[ 7&]x}--!.2FGGtd333"%E4:<<%E%E%EE ;X]KKKKKKKKK@G?P )?(+=!42% $$$N>""%666!7>**k:D77/;?J777 "2  o9 "))++-#**,,,,-      90)-(#t\%9T      o9 "))++-#**,,,,-      9 III;SAAA8388%HHHHHHHH  o9 "))++-#**,,,,-      9 I  o9 "))++-#**,,,,-      9se S,'(B BB# S,.C S, D8D3-D8.S,7(D!! D.-D.3D88CS,>(H(( H54H5:S,I,+S,, I96S,8I99A#S,!(L  LLS,4M S, MS,MC/S, (Q44 RR%S,0(S S'&S', U/6.U*$U/%U2.(U U%$U%*U//U22V48(V" V4" V/,V4.V//V4 user_promptc/Kt|ts|t|nd}|t|dkr |dddzn||dddd|dd}d}d }d} dd lm} | rt d d |rz|ry|rf |t d S#t$r'} t d| d Yd} ~ Sd} ~ wwxYwSSSt d|ddt d|ddddl m} m} m}  | || |d{V}n)#| $r!}t#t|d}~wwxYw|j}t'dd}|d d |dt+jdz }t/j|j|jd{Vd }t|j}|dz }t d|t/jt6||d{V\}}}|s|t#|pd||kr@|r:|r& |n#t$rYnwxYw|}d }i}i}|~t/jt8|||d{V\}}}|s|t#|pd|r:|r& |n#t$rYnwxYw|}|}d }t d t;t<||!d{V}t|dz }t d"|t|t>krft;t@|||#d{V}t|t>kr0t#d$t|d%z d&d't>d%z d(d)||d*<|}d+d,|d-d.d/|id0gd1g}t d2d3}d4} dd5l!m"} m#}!|!}"| |"d6d7i8}#|#$d9}$|$tK|$}|#$d:}%|%tK|%}n#t$rYnwxYwd7||d;|d<}&|r||&d=<tM tOdZi|&d{V}'n#t$r}(tQ|(rt|tRkrt d>t|d%z tRd%z t;t@|||#d{V}||dd?d@d.d/<tOdZi|&d{V}'nYd}(~(nd}(~(wwxYwtU|'})|)s;t dAtOdZi|&d{V}'tU|'})t|)}*t dB|*|)pdC})tW|pd|pd}+d |+rdD|+dE|)n|)dF},|+r|+|,dG<d |dH<|*|dI<tX-dJ|tX.t_j0|,dKdL|rz|ry|rf |t d S#t$r'} t d| d Yd} ~ Sd} ~ wwxYwSSS#t$r}-dMt|-}.t1dN|.d t|-2/tg/fdOdPDrdQ|-})n6tg/fdRdSDr|dT|-})ndU/vsd./vrdV|-})ndW|-})d|.|)dX},|.|dY<tX-dJ|tX.t_j0|,dKdLcYd}-~-|rz|ry|rf |t d S#t$r'} t d| d Yd} ~ Sd} ~ wwxYwSSSd}-~-wwxYw#|rz|ry|rf |t d w#t$r'} t d| d Yd} ~ wd} ~ wwxYwwwwxYw)[a Analyze an image from a URL or local file path using vision AI. This tool accepts either an HTTP/HTTPS URL or a local file path. For URLs, it downloads the image first. In both cases, the image is converted to base64 and processed using Gemini 3 Flash Preview via OpenRouter API. The user_prompt parameter is expected to be pre-formatted by the calling function (typically model_tools.py) to include both full description requests and specific questions. Args: image_url (str): The URL or local file path of the image to analyze. Accepts http://, https:// URLs or absolute/relative file paths. user_prompt (str): The pre-formatted prompt for the vision model model (str): The vision model to use (default: google/gemini-3-flash-preview) Returns: str: JSON string containing the analysis results with the following structure: { "success": bool, "analysis": str (defaults to error message if None) } Raises: Exception: If download fails, analysis fails, or API key is not set Note: - For URLs, temporary images are stored under $HERMES_HOME/cache/vision/ and cleaned up - For local file paths, the file is used directly and NOT deleted - Supports common image formats (JPEG, PNG, GIF, WebP, etc.) Nrr...)rrrFr) parametersrranalysis_length model_usedrTrrrzCleaned up temporary image file#Could not delete temporary file: %srzAnalyzing image: %s<User prompt: %srrrrrrrrrzImage ready (%.1f KB)rrrzConverting image to base64...r\z#Image converted to base64 (%.1f KB))rrZrr_rrrrruserrrrrLrrolerz%Processing image with vision model...g^@皙?rr r!defaultru temperatureitaskmessagesr max_tokensrurzYAPI rejected image (%.1f MB, likely too large); auto-resizing to ~%.0f MB and retrying...rr/z0Vision LLM returned empty content, retrying oncez(Image analysis completed (%s characters)zIThere was a problem with the request and the image could not be analyzed.[z] ranalysisrrrvision_analyze_toolrindent ensure_asciizError analyzing image: %sc3 K|]}|vV dSrr6rs rrz&vision_analyze_tool..$7  4tw      r402 insufficientzpayment requiredcreditsbillinghInsufficient credits or payment required. Please top up your API provider account and try again. Error: c3 K|]}|vV dSrr6rs rrz&vision_analyze_tool..+s7  T      r)does not supportznot support imagecontent_policy multimodalunrecognized request argumentz image inputzO does not support vision or our request was not accepted by the server. Error: rzThe vision API rejected the image. This can happen when the image is in an unsupported format, corrupted, or still too large after auto-resize. Try a smaller JPEG/PNG and retry. Error: zQThere was a problem with the request and the image could not be analyzed. Error: rrrrr6)4rPrQr1rrrryrrrr)rrnrrrrr'r rrrrrBrrr\rrErKr rrr(rrrr&rr r_RESIZE_TARGET_BYTESr rX_debuglog_callrjsondumpsrrr)0rrrrr#debug_call_datarrrr cleanup_errorrrrrrDrr image_size_kbrrrrrrrr data_size_kbcomprehensive_promptrvision_timeoutvision_temperaturerr_cfg _vision_cfg_vt_vtemp call_kwargsr_api_errrrrresultr error_msgrs0 @rrrs N k3 ' 'J*5*Ac+&&&r #8;K8H8H38N8N;tt,u44T_    OONH222222 >   <mU;;;|  o /2H2H2J2J  &&((( >????   9=SW     y  )9SbS>::: %{4C4'8999            '11)^^T[=\=\=\]]]]]]]]HH# ' ' 'SXX&& & '&]!.2FGGtd333"%E4:<<%E%E%EE ;X]KKKKKKKKKx}--(4/  +];;;@G?P )?**T1  9<HHH ~  !2 2 2#>(+=%$'$'$'''''''N>""%666 7>**k:D77/;?J777/?*+ + !' 4 !,!>&    $  ;<<<    > > > > > > > >;==D!'$ XrJJJK//),,C!&s __]33F!%*6]]"    D  -%     )#(K     +::k::::::::HH   $X.. N++.BBB @'';7(K8  (C,#/A)(+(+(+"+"+"+"+"+"+AO I&q)+6u=!/!>!>+!>!>>>>>>> &099 > NNM N N N+::k::::::::H3H==Hh-- >PPPjj&  4!5  8BP4J44(444    .#-F< %) "-<)* -??? z&???j  o /2H2H2J2J  &&((( >????   9=SW     g /@/@/@6c!ff66  T9t 444a&&,,..     ,       B>?BB H    .        662366 H' ) )[G-C-C H+'(++     $- -??? z&????????  o /2H2H2J2J  &&((( >????   9=SW     g/@f  o /2H2H2J2J  &&((( >????   9=SW     sj \<.C C9C44C9?A\<E-,\<-F2FFC<\<J%$\<% J2/\<1J22A\<L'&\<' L41\<3L44D \<A-S\< S \<S \<1T\< WB*V=8\<=WC;\<.\ \6\11\6< c CImage URL (http/https), local file path, or data: URL to load.)r descriptionz{Your specific question or request about the image. Optional context the model uses on the next turn after seeing the image.arrayrintegerrdaFOptional [x1, y1, x2, y2] crop region in pixel coordinates of the ORIGINAL image, applied before any downscaling so the region keeps full resolution. Intended flow: load the full image first, then call again with a region to zoom into a detail (small text, UI element, fine print). Coordinates are clamped to the image bounds.)ritemsminItemsmaxItemsr8)rrr#)r propertiesrequired)rr8rrHkwczK|dd}|dd}|d}|d}tr3tdt ||||d{VSd|}d} d d lm}m} | } || d d d } | r#t|  pd}n#t$rYnwxYw|s)tj dd pd}t|||||d{VS)Nrrrr#rz vision_analyze: native fast path)rr#z]Fully describe and explain everything about this image, then answer the following question: rrr r!rAUXILIARY_VISION_MODEL)rrrrnrr(rrrQr%r)r#r$r) rHr@rrr#r full_promptrrrr%_vmodels r_handle_vision_analyzerEsb))Ixx B''H XXh  FffYG+,,a 6777+IxY_`````````` -"* - - E ::::::::{}}'$ Xw??  1LL&&((0DE      H 2B77==??G4$Y UG\bccc c c c c c ccs"AC(( C54C5r!Tu👁️)rtoolsetschemahandlercheck_fnis_asyncemoji video/mp4z video/webmz video/movz video/mpeg).mp4z.webmz.movz.aviz.mkvz.mpegz.mpg video_pathch|j}t|S)zIReturn a video MIME type based on file extension, or None if unsupported.)rr_VIDEO_MIME_TYPESr)rNexts r_detect_video_mime_typerR s*   ! ! # #C   % %%rc|}tj|d}|p1t|jd}d|d|S)z2Convert a video file to a base64-encoded data URL.rrLrr)rrrrrPrrr)rNrr\r r s r_video_to_base64_data_urlrT&sq  " "Dt$$++G44G  U)--j.?.E.E.G.GUUD *4 * * * **rc|tjdd}|dvS)N TERMINAL_ENVlocal)rrW)r#r$r%r)backends r_terminal_backend_is_localrY.s6i006688>>@@G m ##rvaluec|pd}|sdS|d S)NrF)rNrOr)r%rrR)rZlowereds r_is_path_like_video_sourcer]3sH{!!##))++G u!!"BCC CCr video_sourcec Kddlm}m}m}|}|dr|t dd}t |j}|tvrLtd|dd tt  ||||d d{V}n #|$r}td ||d}~wwxYwtd d } | dd| dt!j|z } | |j| S)u6Read a path via the shared media resolver into a local temp video file. Routes through :func:`tools.image_source.resolve_image_source` with ``permitted=("video",)`` so terminal-backend video reads get the exact pipeline vision_analyze uses: media-cache host reads (gateway-downloaded videos live on the host, not in the sandbox), bounded in-sandbox exec-read (``head -c`` cap — no unbounded base64 stream, no python3 dependency in the sandbox image), lazy env bring-up (#62825), the credential-read guard, and the 50MB ingest cap. rrfile://NUnsupported video format: ''. Supported: r+r)video) permittedz,Could not read video from terminal backend: cache/videotemp_video_filesTrterminal_video_)rrrrrRr1rrrrPr'rOsortedkeysrrrrrr\) r^rrrrsourcerrrDr temp_paths r(_materialize_video_from_terminal_backendrl:s^]]]]]]]]] F ##)I( &\\ & & ( (F &&& H& H H))F+<+A+A+C+C$D$DEE H H   X-- ..999Z          XXXMMMNNTWWXm-?@@H NN4$N///CTZ\\C6CCCI (-((( s7CC3C..C3 video_urlc Kddl}|jddd}d}t|D];} t |}|rt |dddlm}|ddd |gi 4d{V } t| ||td d d dd{Vdddd{Vn#1d{VswxYwY|cS#t$r} | }||dz krTd|dzz} t d|dz|t| dd|j| d{Vn3td|t| dddYd} ~ 5d} ~ wwxYw|t!d|d|)z7Download video from URL with SSRF protection and retry.rNTrc~Kddlm}m}||}|r#||d{Vstd|dSdS)Nrrrrrs rrz-_download_video.._ssrf_redirect_guardesUUUUUUUU44X>>  &7&7 &E&E E E E E E E OOO     rrrgN@rrrzvideo/*,*/*;q=0.8rVideorr/rz)Video download failed (attempt %s/%s): %srz+Video download failed after %s attempts: %srrzB_download_video exited retry loop without attempting (max_retries=r)rBrrrrrrWrr_MAX_VIDEO_BASE64_BYTESr)rrrQrrr) rmrrrBrrrrrrrrs r_download_videorr_sNNNTD999J%%$$# *955G :%gi&8999 F F F F F F44!%'*>)?@        .+'X"5!(                                      "      Jq(('A+. JGVWKYdfijkflflmpnpmpfqrrr#gmI.......... AQ  _Q\ _ _ _    s=AC ?$B5#C 5 B? ?C B? C  E1BE,,E1c r#Kt|ts|t|nd}|t|dkr |dddzn||dddd|dd}d}d } dd lm}|rt d d |rz|ry|rf |t d S#t$r'}t d|d Yd}~Sd}~wwxYwSSSt d|ddt d|dd|} | dr| tdd} ttj| } t%sCt'|r4t d|t)||d{V}d }n| r>ddlm} | t| t d|| }d}nt1|d{Vrit3|} | rt5| dt7dd} | dt9jdz }t=||d{Vd }nt?d| j!}|dz }t d|tE|}|sQt?d |j#d!d"$tKtL'|tPkrt d#|tS||$}t|dz }t|tTkr t?d%|d&d'tTdz d(d)||d*<d+d,|d-d.d/|id0gd1g}d2}d3} dd4l+m,}m-}|}||d5d6i7}|.d8}|t_ta|d2}|.d9}|ta|}n#t$rYnwxYwd6||d:|d;}|r||d<<tctedTi|d{V}tg|}|s;t d=tedTi|d{V}tg|}|rt|nd}t d>|d |pd?d@} d |dA<||dB<th5dC|th6toj8| dDdE|rz|ry|rf |t d S#t$r'}t d|d Yd}~Sd}~wwxYwSSS#t$r}!dFt|!}"t9dG|"d t|!:#tw#fdHdIDrdJ|!}nGtw#fdKdLDrdM|!}n&tw#fdNdODrdP|!}ndQ|!}d|"|dR} |"|dS<th5dC|th6toj8| dDdEcYd}!~!|rz|ry|rf |t d S#t$r'}t d|d Yd}~Sd}~wwxYwSSSd}!~!wwxYw#|rz|ry|rf |t d w#t$r'}t d|d Yd}~wd}~wwxYwwwwxYw)UzEAnalyze a video via multimodal LLM. Returns JSON {success, analysis}.Nrrr)rmrrFr)rrrrrvideo_size_bytesTrrrzCleaned up temporary video filerrzAnalyzing video: %srrrr`z-Reading video source via terminal backend: %s)raise_if_read_blockedzUsing local video file: %srrerf temp_video_rMzKInvalid video source. Provide an HTTP/HTTPS URL or a valid local file path.r_zVideo ready (%.1f MB)rarbr+u,Video is %.1f MB — may be slow or rejectedr\z+Video too large for API: base64 payload is rrrz+ MB). Compress or trim the video and retry.rtrrrrmrL)rrmrgf@rrr r!rrurirrz#Empty video response, retrying oncez(Video analysis completed (%s characters)zIThere was a problem with the request and the video could not be analyzed.rrrvideo_analyze_toolrrzError analyzing video: r c3 K|]}|vV dSrr6rs rrz%video_analyze_tool..;r rr rc3 K|]}|vV dSrr6rs rrz%video_analyze_tool..Bs7  T      r)rznot support videorrrz video inputrmzThe model does not support video analysis or the request was rejected. Ensure you're using a video-capable model (e.g. google/gemini-2.5-flash). Error: c3 K|]}|vV dSrr6rs rrz%video_analyze_tool..Ms7  T      r)rrrrrrrz_The video is too large for the API. Try compressing or trimming the video (max ~50 MB). Error: zQThere was a problem with the request and the video could not be analyzed. Error: rrr6)   <mU;;;b  o /2H2H2J2J  &&((( >????   9=SW     _  )9SbS>::: %{4C4'8999!  " "9 - - 9'I8L"',,\::;; )++ 0J90U0U  KKG S S S$LYX_$`$```````O!NN    ! !  ? ? ? ? ? ? ! !#j// 2 2 2 KK4i @ @ @(O"NN,Y77 7 7 7 7 7 7 *955G :%gi&8999%m5GHHH&)Itz||)I)I)IIO!)_== = = = = = = =!NN] +//119(K8  +];;;/@@  Lo.DLL"ii/@/E/E/G/G(H(HIILL  4 4 4 NNI= Y Y Y2?m\\\>**k: ~  !8 8 89lT991[AM999  /?*+!' + !,!>&    $   > > > > > > > >;==D!'$ XrJJJK//),,C!$U3ZZ!7!7 __]33F!%*6]]"    D  -%     )#(K    '66+66666666/99 > NN@ A A A+::k::::::::H3H==H+3:#h--- >PPP o$o   &* "-<)*,o>>> z&???h  o /2H2H2J2J  &&((( >????   9=SW     e /@/@/@6c!ff66  T9t 444a&&,,..     ,       B>?BB H    .       >:;>> H     .       62366 H +'(++      $- ,o>>> z&????????  o /2H2H2J2J  &&((( >????   9=SW     e/@d  o /2H2H2J2J  &&((( >????   9=SW     s X.C C7C22C7=LXA;Q?>X? R  X R  DX,.W X %XX  ^3D ^.*^3+^6.]77 ^(^##^(.^33^66`6._?>`6? `0 `+&`6+`00`6 video_analyzeu?Analyze a video from a URL or local file path using a multimodal AI model. Sends the video to a video-capable model (e.g. Gemini) for understanding. Use this for video files — for images, use vision_analyze instead. Supports mp4, webm, mov, avi, mkv, mpeg formats. Note: large videos (>20 MB) may be slow; max ~50 MB.z5Video URL (http/https) or local file path to analyze.zpYour specific question about the video. The AI will describe what happens in the video and answer your question.)rmrc "|dd}|dd}d|}d} ddlm}m}|}||ddd p ||dd d } | r#t | pd}n#t $rYnwxYw|sPtjd dp(tjd dpd}t||||d S)NrmrrzFully describe and explain everything happening in this video, including visual content, motion, audio cues, text overlays, and scene transitions. Then answer the following question: rrr rcrr!AUXILIARY_VIDEO_MODELrBrr) rr(rrrQr%r)r#r$rw) rHr@rmrrCrrrr%rDs r_handle_video_analyzersRb))Ixx B''H J?G J J E ::::::::{}}'$ Wg>>o''$P[]egnBoBo  1LL&&((0DE      z 1266<<>>y")LdfhBiBiBoBoBqBqyuy ieRVVIEVEV W W WWsAB BBrcu🎬)r N)rr)NN)NNN)j__doc__r contextlibrBrconcurrent.futuresrloggingr#rpathlibrtypingrrrr urllib.parser rr __annotations__r rhermes_constantsrtools.debug_helpersrtools.website_policyrsys getLogger__name__rrr&r-rr threadingr;r7r=_VISION_CPU_WORKERSrErKrQr'rTrXr[bytesrh frozensetrrr5rr)rdictrrrr rrrrrr"rErXrrrrasynccontextmanagerrr4rrr2print api_availableexitactive session_idtools.registryr4rVISION_ANALYZE_SCHEMArEregisterrPrqrrRrTrYr]rlrrrwVIDEO_ANALYZE_SCHEMArr6rrrs :  111111 111111111111!!!!!! $(c((( 0 0 0 0,+++++,,,,,,555555  8 $ $ n.B C C C 5"5466.@ +3 + + + + S    F2133*)#&    S T    ST((((((Ux}6$-9:::$$ ,D,D,T,,,,^77%(7 8D>8C=(3- 787777t d>DDD DD D  DD DDDDN__S_t_#_VZ____D3T3c3333.$8C=TW8%&' dcd*"&C;C;C; C;C; 8D>8C=(3- 78 C;C;C;C;L**$*c]****ZKO6J=A:>\R\R\R(3-\R03\R-5c]\R*2$\RDG\R\R\R\RVEcE#E$EEEEPTF!% 77777 7  7  #s(^ 7777t       ""! ZZZZc]Z TN Z  ZZZZ@!! DDDD Dc] D TN D  DDDDN 46 z E '((( E(OOO.-//M , 7888 G H H H  *+++ E /000}R Jv7HJJKKK c6K\cccdddd PQQQ E  E :;;; E  E"III E     E 5666 E @AAA E BCCC E) E     E !""" E  E <=== E >??? E 7888 E 5666 E 4555 E/ E $%%% E ,--- E HIII E BCCC 0///////  &!_ ! ]   ),C    0!*-5((V&dtCH~&dS&dS&d&d&d&dR   " &  "       +)&&#&&&& ++$+8C=+TW++++$D$$$$ DcDdDDDD""xX[}"ae""""J99S9t9#9VZ9999~! TTTT Tc] T  TTTTp  ?!V ! R   !*-  4XS#XXcXinXXXX0    ! &  r