Task: Create Performance Comparison Tests for Capnweb vs JSON-RPC
Context
We have a Cloudflare Workers project at /home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions that uses capnweb for RPC. We want to:
- Create visibility into the serialization process - understand what capnweb does internally
- Benchmark capnweb vs JSON-RPC - compare performance across different scenarios
- Test remote worker ↔ local bun process communication
Current Implementation
Server-side (packages/sessions/src/rpc.ts):
import { RpcTarget } from "capnweb"
export class ShareRpcImpl extends RpcTarget {
async createShare(sessionID: string): Promise<SyncInfo> { ... }
async syncShare(shareID: string, secret: string, data: SyncData[]) { ... }
probeValue(input: ProbeValueInput): ProbeValueOutput {
return {
when: input.when.toISOString(),
bytes: Array.from(input.bytes),
size: input.bytes.byteLength,
nested: input.nested,
}
}
async probeCallback(cb: ProbeCallback): Promise<string> {
return await cb("server-called")
}
}
RPC Contract (packages/sessions/src/rpc-contract.ts):
export type ProbeValueInput = { when: Date; bytes: Uint8Array; nested: { x: number } }
export type ProbeValueOutput = { when: string; bytes: number[]; size: number; nested: { x: number } }
export type ProbeCallback = (msg: string) => string | Promise<string>
export interface ShareRpc extends RpcTarget {
createShare: (sessionID: string) => Promise<SyncInfo>
syncShare: (shareID: string, secret: string, data: SyncData[]) => Promise<{ success: boolean; syncCount: number }>
probeValue: (input: ProbeValueInput) => ProbeValueOutput
probeCallback: (cb: ProbeCallback) => Promise<string>
}
Worker entry (packages/sessions/src/index.tsx):
import { newWorkersRpcResponse } from "capnweb"
app.all("/rpc/share", async (c) => {
if (!isAuthorizedRpcRequest(c)) {
return c.json({ error: "Unauthorized" }, 401)
}
return newWorkersRpcResponse(c.req.raw, new ShareRpcImpl(c.env))
})
Client-side (packages/opencode/src/share/share-next.ts):
import { newHttpBatchRpcSession } from "capnweb"
const result = await newHttpBatchRpcSession<ShareRpc>(
new Request(`${await url()}/rpc/share`, { headers: rpcHeaders() }),
).createShare(sessionID)
Requirements
Create a test suite in packages/sessions/src/bench/ that:
Serialization Visibility Tests
- Log/capture the raw bytes being sent/received by capnweb
- Compare serialized size vs JSON.stringify for the same data
- Show the wire format differences
Performance Benchmarks
- Measure round-trip time for various payload sizes
- Compare capnweb vs JSON-RPC (implement a simple JSON-RPC endpoint for comparison)
- Test scenarios:
- Small payloads (simple strings, numbers)
- Medium payloads (session objects with messages)
- Large payloads (sessions with many messages/parts)
- Binary data (Uint8Array)
- Callback round-trips
Remote Communication Tests
- Create a bun test that calls the deployed worker at
https://opencode.j9xym.com/rpc/share - Compare local wrangler dev vs remote production performance
- Measure serialization time separately from network time
- Create a bun test that calls the deployed worker at
Test Structure
packages/sessions/src/bench/ serialization.test.ts # Visibility into wire format performance.test.ts # Timing comparisons json-rpc-server.ts # Simple JSON-RPC implementation for comparison fixtures.ts # Test data generators
Implementation Notes
- Use
bun testfor the test framework - Use
performance.now()orBun.nanoseconds()for timing - For JSON-RPC, create a simple endpoint that does the same operations
- Include warmup iterations to avoid cold start skew
- Report results in a readable format (table or summary)
- The probeValue and probeCallback methods are specifically designed for testing serialization
Expected Output
After running tests, we should see output like:
Serialization Comparison (1000 iterations):
Small payload: capnweb 45 bytes vs JSON 82 bytes (45% smaller)
Medium payload: capnweb 1.2KB vs JSON 2.8KB (57% smaller)
Binary data: capnweb 512 bytes vs JSON 2.1KB (76% smaller)
Round-trip Time (local):
Small: capnweb 0.8ms vs JSON-RPC 1.2ms
Medium: capnweb 2.1ms vs JSON-RPC 4.5ms
Large: capnweb 8.3ms vs JSON-RPC 22.1ms
Round-trip Time (remote):
Small: capnweb 45ms vs JSON-RPC 52ms
Medium: capnweb 48ms vs JSON-RPC 68ms
Files to Read for Context
/home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions/src/rpc.ts/home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions/src/rpc-contract.ts/home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions/src/index.tsx/home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions/src/types.ts/home/jm/data/code/opencode-trees/capnweb-session-transport/packages/sessions/AGENTS.md
Verification
Run bun test src/bench/ to execute the tests and verify they work correctly.