Browse Source

Track Chatto 0.4.3

Chatto HQ is now serving 0.4.3. Per the version-tracks-Chatto policy,
chattolib bumps to match.

Only upstream .proto delta is a docstring change on
chatto.api.v1.ListRoomMembersRequest (default page size doc raised
from 20 to 250 — no wire change). buf/validate/validate.proto also
had comment-only tweaks. Client code and generated bindings are
unchanged in shape.

Live-verified against chat.chatto.run (0.4.3) with the RoboChatto
account: login, me(), list_rooms(), get_motd(), list_roles() all
decode correctly. ruff + mypy --strict + pytest all clean.

Also lands the scrapped tokio-leak-investigation bean
(chattolib-vel8): after measurement, RSS plateaus around 85-91 MiB
after ~2500 RPCs and dips back down — bounded tokio pool footprint,
not an unbounded leak.
TheCodemancer 3 months ago
parent
commit
0a3a62511c

+ 20 - 0
.beans/chattolib-jbw2--update-chattolib-to-chatto-043.md

@@ -0,0 +1,20 @@
+---
+# chattolib-jbw2
+title: Update chattolib to Chatto 0.4.3
+status: completed
+type: task
+priority: high
+created_at: 2026-07-09T15:50:04Z
+updated_at: 2026-07-09T15:51:27Z
+---
+
+Chatto server on chat.chatto.run is now at 0.4.3. Regenerate the proto bindings from upstream, diff the API surface for breaking or additive changes, adapt the client if needed, bump chattolib to 0.4.3, run the full check + live-verify, then publish and push.
+
+## Summary of Changes
+- Reran `scripts/generate_pb.sh` against upstream main. Regenerated Python bindings picked up the two upstream proto changes:
+  - `chatto.api.v1.ListRoomMembersRequest`: default page size doc-comment changed from 20 to 250 (no wire/API change).
+  - `buf/validate/validate.proto`: comment/example updates only.
+- No client code changes required; every method, dataclass, and enum still matches the current server.
+- Bumped `pyproject.toml` `version` from `0.4.2.post2` to `0.4.3` per the [[feedback-versioning-tracks-chatto]] policy (chattolib version tracks Chatto server X.Y.Z).
+- Full check: `ruff`, `mypy --strict`, `pytest` (48 passed, 4 skipped) all clean.
+- Live-verified against Chatto HQ (now serving v0.4.3) with the RoboChatto account: login, viewer, list_rooms, get_motd, list_roles all decode correctly.

+ 71 - 0
.beans/chattolib-vel8--investigate-tokio-related-memory-leak-in-connect-t.md

@@ -0,0 +1,71 @@
+---
+# chattolib-vel8
+title: Investigate tokio-related memory leak in Connect transport
+status: scrapped
+type: bug
+priority: high
+created_at: 2026-07-09T15:34:48Z
+updated_at: 2026-07-09T15:40:03Z
+---
+
+Reported: memory usage grows over time when running chattolib against a live Chatto instance. Suspect the Rust runtime underneath connectrpc (pyqwest wraps reqwest which runs on tokio). Root-cause and mitigate — either through client-lifecycle changes in ChattoClient, an upstream connectrpc/pyqwest bug report, or a workaround.
+
+## Investigation
+
+**Setup.** Used psutil (RSS) and tracemalloc against a live Chatto HQ instance. Warmed up before measurements to skip cold-start allocations. gc.collect() before each reading.
+
+### Reproduction
+
+**Sustained RPC load on one ChattoClient** — the scenario that first looked like a leak:
+
+| RPCs | RSS |
+|---|---|
+| 500 | 78.8 MiB |
+| 1000 | 82.9 MiB |
+| 2000 | 88.1 MiB |
+| 3000 | 90.0 MiB |
+| 3500 | **84.2 MiB (dropped)** |
+| 5000 | 87.9 MiB |
+| 7000 | **86.8 MiB (dropped)** |
+| 10000 | 87.2 MiB |
+
+RSS **plateaus around 85–91 MiB** after ~2500 RPCs and drops back down at times. Not unbounded growth — it's the steady-state footprint of the tokio HTTP/2 connection pool + stream state.
+
+### Confirming: not Python
+
+tracemalloc across 500 RPCs after warm-up:
+
+- 372 KiB in `google.protobuf.json_format` (MessageToDict)
+- 114 KiB in `chattolib.types` (dataclass parsers)
+- 140 KiB in `asyncio`
+- ~750 KiB total Python growth, but RSS grew 11 MiB.
+
+So ~93% of the growth is in the native (Rust/pyqwest/reqwest/tokio) layer, and the Python side is well-behaved.
+
+### Ruled out: pyqwest client multiplication
+
+I hypothesised the 24-services-per-client shape (each service instantiates its own pyqwest `Client`) was the culprit. Measured directly:
+
+- 200 ChattoClient create/close cycles, current impl (24 pyqwest each): 58.04 MiB
+- 200 ChattoClient create/close cycles, one shared pyqwest across all services: 58.06 MiB
+- **Δ: +0.02 MiB — no material improvement.**
+
+Reason: `pyqwest.Client(transport=None)` uses `get_default_transport()` — a process-wide shared tokio transport. The 24 Python-side `Client` objects are thin wrappers around the same native transport, so sharing is already happening under the hood.
+
+### Realtime WebSocket
+
+30 seconds of live event streaming from Chatto HQ (27 events): RSS grew from 62.92 → 62.93 MiB. **Fully stable.**
+
+### Conclusion
+
+**No leak.** What looked like a leak is the tokio connection pool reaching its steady-state footprint over the first ~2000-3000 request/response RPCs. Memory both grows and shrinks within that window and does not exceed ~90 MiB even at 10 000 RPCs.
+
+If a production process is showing **genuinely unbounded** growth, the more likely culprits are:
+- Application code holding references to response messages
+- Realtime event handlers accumulating state (chattolib's stream is fine, but a user's async task keeping refs isn't)
+- A configured OpenTelemetry exporter (pyqwest emits OT metrics; without an exporter they no-op, but a badly-configured batching exporter could accumulate)
+- Something outside chattolib
+
+## Reasons for Scrapping
+
+Closing as not-a-bug. Followed up with detailed telemetry so future investigations start with data. If the user hits a real reproducer, reopen with the specific scenario and I'll instrument further.

+ 9 - 5
proto/buf/validate/validate.proto

@@ -4458,7 +4458,7 @@ message DurationRules {
   // ```proto
   // message MyDuration {
   //   // value must equal 5s
-  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.const = "5s"];
+  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.const = { seconds: 5 }];
   // }
   // ```
   optional google.protobuf.Duration const = 2 [(predefined).cel = {
@@ -4473,7 +4473,7 @@ message DurationRules {
     // ```proto
     // message MyDuration {
     //   // must be less than 5s
-    //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.lt = "5s"];
+    //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.lt = { seconds: 5 }];
     // }
     // ```
     google.protobuf.Duration lt = 3 [(predefined).cel = {
@@ -4490,7 +4490,7 @@ message DurationRules {
     // ```proto
     // message MyDuration {
     //   // must be less than or equal to 10s
-    //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.lte = "10s"];
+    //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.lte = { seconds: 10 }];
     // }
     // ```
     google.protobuf.Duration lte = 4 [(predefined).cel = {
@@ -4611,7 +4611,9 @@ message DurationRules {
   // ```proto
   // message MyDuration {
   //   // must be in list [1s, 2s, 3s]
-  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.in = ["1s", "2s", "3s"]];
+  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration = {
+  //     in: [{ seconds: 1 }, { seconds: 2 }, { seconds: 3 }]
+  //   }];
   // }
   // ```
   repeated google.protobuf.Duration in = 7 [(predefined).cel = {
@@ -4627,7 +4629,9 @@ message DurationRules {
   // ```proto
   // message MyDuration {
   //   // value must not be in list [1s, 2s, 3s]
-  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration.not_in = ["1s", "2s", "3s"]];
+  //   google.protobuf.Duration value = 1 [(buf.validate.field).duration = {
+  //     not_in: [{ seconds: 1 }, { seconds: 2 }, { seconds: 3 }]
+  //   }];
   // }
   // ```
   repeated google.protobuf.Duration not_in = 8 [(predefined).cel = {

+ 1 - 1
proto/chatto/api/v1/member_directory.proto

@@ -85,7 +85,7 @@ message ListRoomMembersRequest {
   string room_id = 1 [(buf.validate.field).string.min_len = 1];
   // Optional case-insensitive search against login and display name.
   string search = 2 [(buf.validate.field).string.max_len = 100];
-  // Page request. Defaults to 20 results when absent or limit is zero.
+  // Page request. Defaults to 250 results when absent or limit is zero.
   PageRequest page = 5;
 }
 

+ 1 - 1
pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "chattolib"
-version = "0.4.2.post2"
+version = "0.4.3"
 description = "Async Python client library for the Chatto webchat Connect API"
 requires-python = ">=3.11"
 license = "MIT"