Преглед изворни кода

Stop shadowing the host's pinned dependencies

setup_vendor_path() put the whole vendored tree on sys.path[0], so every
module the Hermes agent imported after loading this plugin got our
copies of packages Hermes ships and pins itself — httpx, certifi, anyio,
httpcore, h11, idna, typing_extensions. Hermes pins certifi exactly
(2026.5.20 against our 2026.7.22), so the plugin was substituting the CA
bundle for the entire process.

Append vendor/common instead: the host's versions win and ours remain a
fallback, which keeps out-of-process use (the cron sender) working
without Hermes. vendor/platform/<tag> stays prepended — those compiled
extensions have to match the vendored chattolib, and falling back to a
host protobuf of another version would break the generated _pb code.

Verified both directions: with the host providing httpx/certifi they are
used while pyqwest and chattolib still come from vendor/; without a host
httpx the vendored one is picked up and ChattoClient still constructs.

Keeping the overlapping packages vendored is deliberate — dropping them
would save ~2.7 MB of ~59 MB (pyqwest, which Hermes does not ship, is
~15 MB per platform) in exchange for a hard dependency on Hermes' pins.
Paul Klumpp пре 1 недеља
родитељ
комит
b169111593
2 измењених фајлова са 53 додато и 15 уклоњено
  1. 25 4
      VENDORING.md
  2. 28 11
      vendor_path.py

+ 25 - 4
VENDORING.md

@@ -33,10 +33,31 @@ vendor/
 ```
 ```
 
 
 `vendor_path.py` derives the platform tag from `platform.system()` /
 `vendor_path.py` derives the platform tag from `platform.system()` /
-`platform.machine()` at import time and prepends
-`vendor/platform/<tag>` and `vendor/common` to `sys.path` — platform directory
-first, so the right binaries win. Both `adapter.py` and `platform_config.py`
-call `setup_vendor_path()` before importing anything from `chattolib`.
+`platform.machine()` at import time and puts both directories on `sys.path`.
+Both `adapter.py` and `platform_config.py` call `setup_vendor_path()` before
+importing anything from `chattolib`.
+
+### sys.path order — and why it differs per directory
+
+| Directory | Position | Why |
+| --- | --- | --- |
+| `platform/<tag>` | **prepended** | The compiled extensions must match the vendored chattolib. A host copy of `protobuf` at a different version would break the generated `_pb` code. |
+| `common` | **appended** | The Hermes agent ships most of this itself — `httpx` (same pin, 0.28.1), `certifi`, `anyio`, `httpcore`, `h11`, `idna`, `typing_extensions`. A plugin must not shadow the host's pinned versions; Hermes pins `certifi` exactly, and prepending would substitute our CA bundle process-wide. |
+
+So the host's pure-Python packages win, and our copies act as a fallback —
+which keeps out-of-process use (the cron sender, `hermes_standalone_sender_fn`)
+working in an environment without Hermes.
+
+The packages Hermes does *not* provide — `chattolib`, `connectrpc`, `pyqwest`,
+`protobuf`, `protobuf-py`, `protobuf-py-ext` — always come from `vendor/`. They
+are also the bulk of it: `pyqwest` alone is ~15 MB per platform, so dropping
+the packages Hermes already ships would save only ~2.7 MB of ~59 MB and buy a
+dependency on Hermes' exact pins. Not worth it.
+
+chattolib requires `httpx>=0.27` and `protobuf>=5.28`; connectrpc requires
+`protobuf-py==0.1.1` and `pyqwest>=0.5.1`. If a future Hermes release pins
+`httpx` below 0.27, the appended fallback no longer helps — the host copy would
+be found first and be too old.
 
 
 If the current platform was not vendored, the import fails with an explicit
 If the current platform was not vendored, the import fails with an explicit
 message naming the available platforms — rather than a cryptic `dlopen` error
 message naming the available platforms — rather than a cryptic `dlopen` error

+ 28 - 11
vendor_path.py

@@ -11,9 +11,20 @@ protobuf-py-ext) ship compiled extension modules::
         linux-aarch64/
         linux-aarch64/
         macos-arm64/
         macos-arm64/
 
 
-Both directories are prepended to ``sys.path`` (platform first), so the
-vendored copies win over anything installed system-wide. Run
-``./vendor_chattolib.sh`` to (re)build the tree.
+The two directories enter ``sys.path`` differently, on purpose:
+
+* ``platform/<tag>`` is **prepended**. It holds the compiled extensions that
+  must match the vendored chattolib (pyqwest, protobuf, protobuf-py-ext);
+  falling back to a host copy of a different version would break the generated
+  protobuf code.
+* ``common`` is **appended**. Most of it (httpx, anyio, httpcore, h11, idna,
+  certifi, typing_extensions) is also shipped by the Hermes agent itself, and
+  a plugin has no business shadowing the host's pinned versions — Hermes pins
+  certifi exactly, and we would otherwise substitute our own CA bundle for the
+  whole process. The host's copies win; ours only serve as a fallback, which
+  keeps out-of-process use (the cron sender) working on its own.
+
+Run ``./vendor_chattolib.sh`` to (re)build the tree.
 """
 """
 
 
 from __future__ import annotations
 from __future__ import annotations
@@ -68,11 +79,13 @@ def available_platforms() -> List[str]:
 
 
 
 
 def setup_vendor_path() -> List[str]:
 def setup_vendor_path() -> List[str]:
-    """Prepend the vendored packages for this platform to ``sys.path``.
+    """Put the vendored packages for this platform on ``sys.path``.
 
 
-    Returns the directories that were added. Raises ImportError when this
-    platform was not vendored — a clear message beats the dlopen error you
-    would otherwise get from a foreign-architecture .so file.
+    The compiled extensions are prepended, the pure-Python packages appended;
+    see the module docstring for why. Returns the directories that were added.
+    Raises ImportError when this platform was not vendored — a clear message
+    beats the dlopen error you would otherwise get from a foreign-architecture
+    .so file.
     """
     """
     tag = platform_tag()
     tag = platform_tag()
     platform_dir = VENDOR_DIR / "platform" / tag
     platform_dir = VENDOR_DIR / "platform" / tag
@@ -81,7 +94,7 @@ def setup_vendor_path() -> List[str]:
     # Legacy flat layout (single platform, everything directly in vendor/).
     # Legacy flat layout (single platform, everything directly in vendor/).
     if not platform_dir.is_dir() and not common_dir.is_dir():
     if not platform_dir.is_dir() and not common_dir.is_dir():
         if (VENDOR_DIR / "chattolib").is_dir():
         if (VENDOR_DIR / "chattolib").is_dir():
-            paths = [VENDOR_DIR]
+            prepend, append = [VENDOR_DIR], []
         else:
         else:
             raise ImportError(
             raise ImportError(
                 f"Chatto: no vendored dependencies found in {VENDOR_DIR}. "
                 f"Chatto: no vendored dependencies found in {VENDOR_DIR}. "
@@ -94,13 +107,17 @@ def setup_vendor_path() -> List[str]:
             f"Run 'PLATFORMS={tag} ./vendor_chattolib.sh' to add it."
             f"Run 'PLATFORMS={tag} ./vendor_chattolib.sh' to add it."
         )
         )
     else:
     else:
-        # Platform-specific extensions must shadow anything in common/.
-        paths = [platform_dir, common_dir]
+        prepend, append = [platform_dir], [common_dir]
 
 
     added = []
     added = []
-    for path in paths:
+    for path in prepend:
         entry = str(path)
         entry = str(path)
         if entry not in sys.path:
         if entry not in sys.path:
             sys.path.insert(0, entry)
             sys.path.insert(0, entry)
             added.append(entry)
             added.append(entry)
+    for path in append:
+        entry = str(path)
+        if entry not in sys.path:
+            sys.path.append(entry)
+            added.append(entry)
     return added
     return added