Gateway Engineering / Full-time
Senior Software Engineer, Gateway Data Path
Build the request path that resolves jurisdiction, evaluates policy, pseudonymizes PII and streams completions, inside an 8 ms p99 overhead budget.
Team
Gateway Engineering
Location
Chennai
Type
Full-time
Posted
15 Oct 2026
- Work on the gateway's hot path in Go and Rust: request parsing, OpenAI-compatible and Anthropic-compatible APIs, server-sent event streaming and upstream connection pooling.
- Integrate embedded OPA evaluation, PII detection and receipt minting so that each stage has a measured latency budget and a regression test in CI.
- Keep prompts and completions in memory only: no logging of bodies, no disk spill, and tests that prove it.
- Implement upstream TLS verification that captures the certificate SHA-256 thumbprint for the receipt and pins expected endpoints from the registry.
- Profile under production-shaped load, read flame graphs with the team, and turn findings into small, reviewable changes.
- Pair with Customer Engineering on difficult integrations, such as SDKs with unusual retry behaviour or very long streamed responses.
- Five or more years building high-throughput network services in Go or Rust, with at least one system where p99 latency was a release criterion.
- Deep understanding of HTTP/1.1, HTTP/2, TLS 1.3 and streaming responses, and of what goes wrong with each at scale.
- Experience with profiling and benchmarking tools such as pprof, perf or criterion, and with load testing that reflects real traffic.
- Good judgement about memory handling and data lifetime in code that touches sensitive payloads.
- Clear code review comments and design documents.
- Contributions to Envoy, a Go or Rust HTTP stack, or an open-source proxy.
- Experience embedding OPA or another policy engine in a request path.
- Familiarity with LLM provider APIs, tokenization and streaming formats.
- 01Application review
- 02Intro call
- 03Paid work sample
- 04Panel
- 05Offer