فهرست منبع

Stop shadowing the host's pinned dependencies

setup_vendor_path() put the whole vendored tree on sys.path[0], so every
module the Hermes agent imported after loading this plugin got our
copies of packages Hermes ships and pins itself — httpx, certifi, anyio,
httpcore, h11, idna, typing_extensions. Hermes pins certifi exactly
(2026.5.20 against our 2026.7.22), so the plugin was substituting the CA
bundle for the entire process.

Append vendor/common instead: the host's versions win and ours remain a
fallback, which keeps out-of-process use (the cron sender) working
without Hermes. vendor/platform/<tag> stays prepended — those compiled
extensions have to match the vendored chattolib, and falling back to a
host protobuf of another version would break the generated _pb code.

Verified both directions: with the host providing httpx/certifi they are
used while pyqwest and chattolib still come from vendor/; without a host
httpx the vendored one is picked up and ChattoClient still constructs.

Keeping the overlapping packages vendored is deliberate — dropping them
would save ~2.7 MB of ~59 MB (pyqwest, which Hermes does not ship, is
~15 MB per platform) in exchange for a hard dependency on Hermes' pins.
Paul Klumpp 1 هفته پیش
والد
کامیت
7f5c36cab3
2فایلهای تغییر یافته به همراه53 افزوده شده و 15 حذف شده
  1. 25 4
      VENDORING.md
  2. 28 11
      vendor_path.py

+ 25 - 4
VENDORING.md

@@ -33,10 +33,31 @@ vendor/
 ```
 
 `vendor_path.py` derives the platform tag from `platform.system()` /
-`platform.machine()` at import time and prepends
-`vendor/platform/<tag>` and `vendor/common` to `sys.path` — platform directory
-first, so the right binaries win. Both `adapter.py` and `platform_config.py`
-call `setup_vendor_path()` before importing anything from `chattolib`.
+`platform.machine()` at import time and puts both directories on `sys.path`.
+Both `adapter.py` and `platform_config.py` call `setup_vendor_path()` before
+importing anything from `chattolib`.
+
+### sys.path order — and why it differs per directory
+
+| Directory | Position | Why |
+| --- | --- | --- |
+| `platform/<tag>` | **prepended** | The compiled extensions must match the vendored chattolib. A host copy of `protobuf` at a different version would break the generated `_pb` code. |
+| `common` | **appended** | The Hermes agent ships most of this itself — `httpx` (same pin, 0.28.1), `certifi`, `anyio`, `httpcore`, `h11`, `idna`, `typing_extensions`. A plugin must not shadow the host's pinned versions; Hermes pins `certifi` exactly, and prepending would substitute our CA bundle process-wide. |
+
+So the host's pure-Python packages win, and our copies act as a fallback —
+which keeps out-of-process use (the cron sender, `hermes_standalone_sender_fn`)
+working in an environment without Hermes.
+
+The packages Hermes does *not* provide — `chattolib`, `connectrpc`, `pyqwest`,
+`protobuf`, `protobuf-py`, `protobuf-py-ext` — always come from `vendor/`. They
+are also the bulk of it: `pyqwest` alone is ~15 MB per platform, so dropping
+the packages Hermes already ships would save only ~2.7 MB of ~59 MB and buy a
+dependency on Hermes' exact pins. Not worth it.
+
+chattolib requires `httpx>=0.27` and `protobuf>=5.28`; connectrpc requires
+`protobuf-py==0.1.1` and `pyqwest>=0.5.1`. If a future Hermes release pins
+`httpx` below 0.27, the appended fallback no longer helps — the host copy would
+be found first and be too old.
 
 If the current platform was not vendored, the import fails with an explicit
 message naming the available platforms — rather than a cryptic `dlopen` error

+ 28 - 11
vendor_path.py

@@ -11,9 +11,20 @@ protobuf-py-ext) ship compiled extension modules::
         linux-aarch64/
         macos-arm64/
 
-Both directories are prepended to ``sys.path`` (platform first), so the
-vendored copies win over anything installed system-wide. Run
-``./vendor_chattolib.sh`` to (re)build the tree.
+The two directories enter ``sys.path`` differently, on purpose:
+
+* ``platform/<tag>`` is **prepended**. It holds the compiled extensions that
+  must match the vendored chattolib (pyqwest, protobuf, protobuf-py-ext);
+  falling back to a host copy of a different version would break the generated
+  protobuf code.
+* ``common`` is **appended**. Most of it (httpx, anyio, httpcore, h11, idna,
+  certifi, typing_extensions) is also shipped by the Hermes agent itself, and
+  a plugin has no business shadowing the host's pinned versions — Hermes pins
+  certifi exactly, and we would otherwise substitute our own CA bundle for the
+  whole process. The host's copies win; ours only serve as a fallback, which
+  keeps out-of-process use (the cron sender) working on its own.
+
+Run ``./vendor_chattolib.sh`` to (re)build the tree.
 """
 
 from __future__ import annotations
@@ -68,11 +79,13 @@ def available_platforms() -> List[str]:
 
 
 def setup_vendor_path() -> List[str]:
-    """Prepend the vendored packages for this platform to ``sys.path``.
+    """Put the vendored packages for this platform on ``sys.path``.
 
-    Returns the directories that were added. Raises ImportError when this
-    platform was not vendored — a clear message beats the dlopen error you
-    would otherwise get from a foreign-architecture .so file.
+    The compiled extensions are prepended, the pure-Python packages appended;
+    see the module docstring for why. Returns the directories that were added.
+    Raises ImportError when this platform was not vendored — a clear message
+    beats the dlopen error you would otherwise get from a foreign-architecture
+    .so file.
     """
     tag = platform_tag()
     platform_dir = VENDOR_DIR / "platform" / tag
@@ -81,7 +94,7 @@ def setup_vendor_path() -> List[str]:
     # Legacy flat layout (single platform, everything directly in vendor/).
     if not platform_dir.is_dir() and not common_dir.is_dir():
         if (VENDOR_DIR / "chattolib").is_dir():
-            paths = [VENDOR_DIR]
+            prepend, append = [VENDOR_DIR], []
         else:
             raise ImportError(
                 f"Chatto: no vendored dependencies found in {VENDOR_DIR}. "
@@ -94,13 +107,17 @@ def setup_vendor_path() -> List[str]:
             f"Run 'PLATFORMS={tag} ./vendor_chattolib.sh' to add it."
         )
     else:
-        # Platform-specific extensions must shadow anything in common/.
-        paths = [platform_dir, common_dir]
+        prepend, append = [platform_dir], [common_dir]
 
     added = []
-    for path in paths:
+    for path in prepend:
         entry = str(path)
         if entry not in sys.path:
             sys.path.insert(0, entry)
             added.append(entry)
+    for path in append:
+        entry = str(path)
+        if entry not in sys.path:
+            sys.path.append(entry)
+            added.append(entry)
     return added