|
@@ -13,25 +13,26 @@ dependency tree.
|
|
|
|
|
|
|
|
## Layout
|
|
## Layout
|
|
|
|
|
|
|
|
-Three of the vendored packages contain compiled extension modules and are
|
|
|
|
|
-therefore platform-specific: `pyqwest` (~15 MB, the HTTP core used by
|
|
|
|
|
-`connectrpc`), `protobuf` and `protobuf-py-ext`. Everything else is pure
|
|
|
|
|
-Python. The tree is split accordingly, so the shared part is stored once:
|
|
|
|
|
|
|
+Two of the vendored packages contain compiled extension modules and are
|
|
|
|
|
+therefore platform-specific: `protobuf` (its `google/_upb` runtime) and
|
|
|
|
|
+`websockets` (its `speedups` mask helper). Everything else is pure Python. The
|
|
|
|
|
+tree is split accordingly, so the shared part is stored once:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
|
vendor/
|
|
vendor/
|
|
|
├── __init__.py
|
|
├── __init__.py
|
|
|
-├── common/ # pure-Python packages (~8 MB), all platforms
|
|
|
|
|
|
|
+├── common/ # pure-Python packages (5.8 MB), all platforms
|
|
|
│ ├── chattolib/
|
|
│ ├── chattolib/
|
|
|
-│ ├── connectrpc/
|
|
|
|
|
│ ├── httpx/ httpcore/ h11/ anyio/ idna/ certifi/
|
|
│ ├── httpx/ httpcore/ h11/ anyio/ idna/ certifi/
|
|
|
-│ └── opentelemetry/ protobuf/ typing_extensions.py
|
|
|
|
|
-└── platform/ # compiled extensions, ~16 MB per platform
|
|
|
|
|
- ├── linux-x86_64/ # pyqwest/, protobuf_ext/, google/
|
|
|
|
|
|
|
+│ └── typing_extensions.py
|
|
|
|
|
+└── platform/ # compiled extensions, 2.3–3.9 MB per platform
|
|
|
|
|
+ ├── linux-x86_64/ # google/ websockets/
|
|
|
├── linux-aarch64/
|
|
├── linux-aarch64/
|
|
|
└── macos-arm64/
|
|
└── macos-arm64/
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
|
|
+Total: 14.4 MB on disk (~9.6 MB tracked by git, the `.so` files compress well).
|
|
|
|
|
+
|
|
|
`vendor_path.py` derives the platform tag from `platform.system()` /
|
|
`vendor_path.py` derives the platform tag from `platform.system()` /
|
|
|
`platform.machine()` at import time and puts both directories on `sys.path`.
|
|
`platform.machine()` at import time and puts both directories on `sys.path`.
|
|
|
Both `adapter.py` and `platform_config.py` call `setup_vendor_path()` before
|
|
Both `adapter.py` and `platform_config.py` call `setup_vendor_path()` before
|
|
@@ -48,16 +49,29 @@ So the host's pure-Python packages win, and our copies act as a fallback —
|
|
|
which keeps out-of-process use (the cron sender, `hermes_standalone_sender_fn`)
|
|
which keeps out-of-process use (the cron sender, `hermes_standalone_sender_fn`)
|
|
|
working in an environment without Hermes.
|
|
working in an environment without Hermes.
|
|
|
|
|
|
|
|
-The packages Hermes does *not* provide — `chattolib`, `connectrpc`, `pyqwest`,
|
|
|
|
|
-`protobuf`, `protobuf-py`, `protobuf-py-ext` — always come from `vendor/`. They
|
|
|
|
|
-are also the bulk of it: `pyqwest` alone is ~15 MB per platform, so dropping
|
|
|
|
|
-the packages Hermes already ships would save only ~2.7 MB of ~59 MB and buy a
|
|
|
|
|
-dependency on Hermes' exact pins. Not worth it.
|
|
|
|
|
-
|
|
|
|
|
-chattolib requires `httpx>=0.27` and `protobuf>=5.28`; connectrpc requires
|
|
|
|
|
-`protobuf-py==0.1.1` and `pyqwest>=0.5.1`. If a future Hermes release pins
|
|
|
|
|
-`httpx` below 0.27, the appended fallback no longer helps — the host copy would
|
|
|
|
|
-be found first and be too old.
|
|
|
|
|
|
|
+Note that Hermes' pins drift from ours (`anyio` 4.12.1 vs 4.15.1, `idna` 3.18
|
|
|
|
|
+vs 3.20, `certifi` 2026.5.20 vs 2026.7.22) — the append order is exactly what
|
|
|
|
|
+keeps those differences harmless.
|
|
|
|
|
+
|
|
|
|
|
+The packages Hermes does *not* provide — `chattolib` and the compiled
|
|
|
|
|
+`protobuf`/`websockets` — always come from `vendor/`.
|
|
|
|
|
+
|
|
|
|
|
+The rest (`httpx`, `httpcore`, `h11`, `anyio`, `idna`, `certifi`,
|
|
|
|
|
+`typing_extensions`, ~4.2 MB) Hermes *does* provide, so it is fair to ask why
|
|
|
|
|
+they are vendored at all. Because of the cron sender:
|
|
|
|
|
+`hermes_standalone_sender_fn` delivers scheduled messages out of process, in no
|
|
|
|
|
+Hermes venv, and there `vendor/common` is the only thing on `sys.path`. The
|
|
|
|
|
+append order means those copies are inert wherever Hermes runs and
|
|
|
|
|
+load-bearing where it does not.
|
|
|
|
|
+
|
|
|
|
|
+chattolib requires `httpx>=0.27`, `protobuf>=5.28` and (via the `[realtime]`
|
|
|
|
|
+extra) `websockets>=13.0`. Two of those floors are worth knowing about:
|
|
|
|
|
+`protobuf>=5.28` is wrong in practice — chattolib b13 ships gencode 7.35.1, and
|
|
|
|
|
+protobuf ≥ 5.27 refuses at import time to load gencode newer than its own
|
|
|
|
|
+runtime, so a host protobuf older than 7.35.1 raises `VersionError` rather
|
|
|
|
|
+than degrading. And if a future Hermes release pins `httpx` below 0.27, the
|
|
|
|
|
+appended fallback no longer helps — the host copy would be found first and be
|
|
|
|
|
+too old.
|
|
|
|
|
|
|
|
If the current platform was not vendored, the import fails with an explicit
|
|
If the current platform was not vendored, the import fails with an explicit
|
|
|
message naming the available platforms — rather than a cryptic `dlopen` error
|
|
message naming the available platforms — rather than a cryptic `dlopen` error
|
|
@@ -77,9 +91,19 @@ PLATFORMS="linux-x86_64 linux-aarch64 macos-arm64 windows-amd64" ./vendor_chatto
|
|
|
|
|
|
|
|
# Pin a specific chattolib version (keep the [realtime] extra — without it
|
|
# Pin a specific chattolib version (keep the [realtime] extra — without it
|
|
|
# websockets is not vendored and the event stream cannot start)
|
|
# websockets is not vendored and the event stream cannot start)
|
|
|
-CHATTOLIB_SPEC="chattolib[realtime]==0.5.0a5.post1" ./vendor_chattolib.sh
|
|
|
|
|
|
|
+CHATTOLIB_SPEC="chattolib[realtime]==0.5.0b13" ./vendor_chattolib.sh
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
|
|
+The script passes `--only-binary :all: --no-binary chattolib`. Everything must
|
|
|
|
|
+come as a prebuilt wheel, because a source distribution of `pyqwest`,
|
|
|
|
|
+`protobuf` or `websockets` could not be built for a *foreign* platform — uv
|
|
|
|
|
+would build it for the host and silently drop host-architecture binaries into
|
|
|
|
|
+`vendor/platform/<other-tag>/`. chattolib itself is the documented exception: it
|
|
|
|
|
+is pure Python, so building it from an sdist yields identical,
|
|
|
|
|
+platform-independent files on any host. That exception is load-bearing, not
|
|
|
|
|
+cosmetic — upstream stopped publishing wheels after 0.5.0b4, so without it the
|
|
|
|
|
+resolver falls back to b4 and a refresh silently changes nothing.
|
|
|
|
|
+
|
|
|
The script wipes `vendor/`, installs the dependency tree once per platform via
|
|
The script wipes `vendor/`, installs the dependency tree once per platform via
|
|
|
`uv pip install --python-platform …`, splits the result with `vendor_split.py`,
|
|
`uv pip install --python-platform …`, splits the result with `vendor_split.py`,
|
|
|
and verifies that `import chattolib` works on the machine you ran it on.
|
|
and verifies that `import chattolib` works on the machine you ran it on.
|
|
@@ -98,19 +122,33 @@ Afterwards: `git add vendor/`.
|
|
|
| `windows-amd64` | `x86_64-pc-windows-msvc` | — |
|
|
| `windows-amd64` | `x86_64-pc-windows-msvc` | — |
|
|
|
|
|
|
|
|
All seven are available as binary wheels upstream; only the default three are
|
|
All seven are available as binary wheels upstream; only the default three are
|
|
|
-committed, to keep the repository at a reasonable size (~59 MB for `vendor/`).
|
|
|
|
|
|
|
+committed, to keep the repository at a reasonable size (14.4 MB for `vendor/`).
|
|
|
|
|
|
|
|
### Python versions
|
|
### Python versions
|
|
|
|
|
|
|
|
The script resolves wheels for **Python 3.11** (`PYTHON_VERSION`), which is the
|
|
The script resolves wheels for **Python 3.11** (`PYTHON_VERSION`), which is the
|
|
|
-project minimum. At that version the three binary packages still provide
|
|
|
|
|
-`cp310-abi3` wheels, and the stable ABI is forward compatible — so one build
|
|
|
|
|
-covers 3.11 through 3.14. Resolving for 3.12+ would instead pull
|
|
|
|
|
-version-specific wheels (`cp312-…`) that only work on that exact version.
|
|
|
|
|
|
|
+project minimum. What that buys differs per package, and the difference matters:
|
|
|
|
|
+
|
|
|
|
|
+| Package | Tag at 3.11 | Consequence |
|
|
|
|
|
+| --- | --- | --- |
|
|
|
|
|
+| `protobuf` | `abi3` (`_message.abi3.so`) | Stable ABI, forward compatible — one build covers 3.11 through 3.15. |
|
|
|
|
|
+| `websockets` | `cp311` (`speedups.cpython-311-…so`) | Version-specific. |
|
|
|
|
|
+
|
|
|
|
|
+So on Python 3.12+ the protobuf runtime still loads, and `chattolib` imports
|
|
|
|
|
+fine — but `websockets.speedups` is *not* importable and websockets silently
|
|
|
|
|
+falls back to its pure-Python `apply_mask` (it wraps the import in
|
|
|
|
|
+`try: … except ImportError:`). Verified on 3.12 and 3.14: chattolib and its
|
|
|
|
|
+realtime module import, `apply_mask` resolves to `websockets.utils`.
|
|
|
|
|
+
|
|
|
|
|
+That is a performance fallback, not a breakage — masking is a small part of
|
|
|
|
|
+frame handling. It is the price for keeping one build instead of one per minor
|
|
|
|
|
+version, and the reason `PYTHON_VERSION` must stay at the project minimum:
|
|
|
|
|
+raising it would pull `cp312-…`-style wheels and make the whole tree
|
|
|
|
|
+version-specific.
|
|
|
|
|
|
|
|
-**If you raise `PYTHON_VERSION`, check that the resulting `.so` files are still
|
|
|
|
|
-named `*.abi3.so`.** If they are not, the vendored tree silently becomes
|
|
|
|
|
-Python-version-specific.
|
|
|
|
|
|
|
+**If you raise `PYTHON_VERSION`, check the resulting `.so` names.** `*.abi3.so`
|
|
|
|
|
+for protobuf; anything under `websockets/` is tied to that exact CPython minor
|
|
|
|
|
+version regardless.
|
|
|
|
|
|
|
|
## Development without vendoring
|
|
## Development without vendoring
|
|
|
|
|
|
|
@@ -119,7 +157,9 @@ pip install chattolib
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
A system-installed chattolib is only used if `vendor/` is absent — the vendored
|
|
A system-installed chattolib is only used if `vendor/` is absent — the vendored
|
|
|
-copy is deliberately prepended to `sys.path` and therefore wins.
|
|
|
|
|
|
|
+copy is deliberately prepended to `sys.path` and therefore wins. Keep the
|
|
|
|
|
+installed version in step with `vendor/common/chattolib-*.dist-info/`, or tests
|
|
|
|
|
+will pass against one chattolib while the plugin ships another.
|
|
|
|
|
|
|
|
## License
|
|
## License
|
|
|
|
|
|