Meta · Menlo Park, CA

Thrift / ServiceRouter · RPC Infrastructure

Admission Control & Fault Tolerance

• Redesigned ServiceRouter's error-throttling system and drove it to 100% global, default-on — the admission-control layer that caps traffic at K×goodput during overload, now shedding ~7.8T requests/day across ~14K services.
• Added per-tenant (clientId) isolation so noisy neighbors are throttled without harming co-tenants; validated during production SEVs where the system absorbed 8–20× overload spikes while preserving goodput for well-behaved callers.
• Designed and shipped appErrorsMap, a per-error fault-tolerance configuration replacing a decade-old single-list design — giving owners independent control of retry, markdown, throttling, caching, and failover per error code. Deployed on ~15K services.

Connection Liveness & Host Health

• Designed the keepAliveTimeout mechanism (RSocket KeepAlive frames via KeepAliveWatcher) that detects and kills slow/dead connections, protecting servers from OOM and connection exhaustion — adopted by ~615 services handling 159M req/day. Generalized a customer-specific fix into a fleet-wide Thrift primitive.
• Drove async-probe health-check rollout to 100%: HC latency P99 268ms → 0ms, success P99 648 → 120ms (5.4×), error path 6.9s → 625ms (11×). Authored the jitter-by-default fix preventing coordinated distributed health-check storms, and an aggregated jitter cache yielding −31% CPU on selection status.

Quota & Resource Attribution

• Launched Thrift Quota Management (TQMS) — a server-side, one-config CPU-time/QPS quota system with fail-open semantics, ~5s reaction time, and dry-run mode. Adopted by ~430+ services with CPU-time reporting.
• Thrift-side technical lead for Platform Quota Management (PQMM), a cross-team (4-org) module providing per-platform resource attribution with zero per-service instrumentation. Rolled out to 100% of a major serving tier and validated across 50+ services.

Streaming Observability & Bidirectional Streaming

• Led the Streaming Visibility program end-to-end: built unified observability for all Thrift streaming shapes (stream, sink, bidi, interactions) covering TTFT, chunk intervals, backpressure/credits, lifecycle, and active-stream gauges — the foundational instrumentation GenAI streaming services depend on.
• Built ServiceRouter bidirectional streaming support and the entire bidi observability/metrics layer. Production customers include GenAI token-streaming inference and LLM prediction pipelines.
• Led logging-efficiency work that reduced streaming telemetry volume by ~1000× while adding signal — cutting raw samples from ~800M to ~70M per window.

iT&X Data LLC · Blacksburg, VA

• Bootstrapped and scaled a specialized data infrastructure consultancy, designing custom high-performance data pipelines and distributed storage engines for enterprise clients.
• Architected lock-free concurrent execution engines in C++ and Rust, achieving orders-of-magnitude performance improvements over traditional thread-pool architectures for real-time data ingestion.

Virginia Tech · Blacksburg, VA

Tsinghua University · Beijing, China