The next bottleneck for AI agents isn't intelligence, it's infrastructure. When a coding agent plans, calls tools, and reasons over a million-token context, the cost and latency of each API call become the primary constraints on what it can do. CoralBricks, a Seattle-based inference startup founded in June 2026, is building its platform on the premise that the existing cloud for chatbots is the wrong shape for this new workload.
Its product is an OpenAI-compatible API, but the underlying mechanics are tuned for a different rhythm. Instead of optimizing for short, bursty conversations, CoralBricks promises high token throughput, near-zero rate limits, and a critical twist: free cached input tokens [PERPLEXITY SONAR PRO BRIEF]. For developers orchestrating long-running agents, that last feature changes the unit economics of iterative reasoning, where the same prompt context might be reused across dozens of steps.
The infrastructure wedge
CoralBricks is not selling a novel model. It serves open-source models like Kimi, GLM, and gpt-oss, all supporting up to 1 million tokens of context [PERPLEXITY SONAR PRO BRIEF]. The differentiation is in the delivery layer. The company claims its infrastructure delivers "multiple times the tokens per second of a typical provider" [PERPLEXITY SONAR PRO BRIEF]. While independent benchmarks are not yet public, the architectural focus is clear: eliminate the throttling and per-token friction that makes agentic workflows prohibitively slow or expensive.
The platform's economics are built around three technical promises:
- High token throughput. A focus on raw tokens per second, which directly translates to how quickly an agent can process long contexts and generate complex outputs.
- Near-zero rate limits. Designed for sustained, high-volume workloads from autonomous systems, not human-in-the-loop chat.
- Free cached input. Perhaps the most significant lever, this makes the often-repeated prompt portion of an agent's operation cost-zero, attacking a major line item in iterative tasks.
A pivot toward inference
The company's current positioning represents a strategic shift. Earlier in 2026, co-founder Hitesh Jain described CoralBricks as developing a commerce-specific embedding model and retrieval foundation, targeting 30 ms p95 latency for search and copilots [PERPLEXITY SONAR PRO BRIEF]. By September, the company page had pivoted to center squarely on inference infrastructure for agents [PERPLEXITY SONAR PRO BRIEF]. This evolution suggests a rapid market test, converging on the high-throughput agent problem as a clearer wedge. The company's open-source 'reef' framework on GitHub, which includes a finance-focused agent instance, signals a commitment to this developer-centric, agent-harness world [GitHub, March 2026].
The Meta and AWS engineering pedigree
The technical bet is backed by a founding team with relevant infrastructure pedigree. CEO Hitesh Jain was a Principal Engineer in generative AI at Meta, and co-founder Divy Vasal is formerly of AWS [PERPLEXITY SONAR PRO BRIEF]. This background in scaling complex systems at major cloud and AI players informs the platform's ambitions. Jain is also an early-stage investor in several AI startups, including Cielara AI and Byteroll, giving him a lens into the broader tooling ecosystem [PERPLEXITY SONAR PRO BRIEF].
External validation points include acceptance into the NVIDIA Inception Program in March 2026 and presenting at the Seattle Startup Summit in April 2026 [PERPLEXITY SONAR PRO BRIEF]. The company is backed by Afore Capital and Foundations Accelerator, and was featured at an Afore Capital event in August discussing the "cost curve of a coding agent" [PERPLEXITY SONAR PRO BRIEF]. A hiring push for a Founding Developer Advocate role underscores a focus on winning over the developer community through integrations and workshops [PERPLEXITY SONAR PRO BRIEF].
| Founder | Role | Key Background |
|---|---|---|
| Hitesh Jain | Co-founder & CEO | Principal Engineer, Generative AI at Meta; IIT Roorkee alumnus [PERPLEXITY SONAR PRO BRIEF] |
| Divy Vasal | Co-founder & Head of Engineering | Formerly of AWS [PERPLEXITY SONAR PRO BRIEF] |
The scale and competition question
The technical breakdown is compelling, but the real test is operational scale. CoralBricks is betting that a performance-optimized gateway for open models can capture a segment of the market that outgrows generic inference services. The primary risk is that this segment, while growing, may not be large or defensible enough. Larger cloud providers could replicate the caching and rate-limit features once agent workloads become a standard offering. Furthermore, the company's reliance on upstream open-source model providers like Kimi and GLM introduces a layer of dependency; performance and cost advantages could be eroded by changes at that layer.
The company's answer appears to be a combination of deep technical optimization and community lock-in. By providing the 'reef' harness framework and aggressively courting developers, CoralBricks aims to become the default integration for teams building agentic systems. The next twelve months will be about proving that the performance claims hold under real customer load and that the developer traction converts into sustained, paid usage. For now, the bet is clear: the infrastructure for the next generation of AI needs to be rebuilt, not borrowed.
Sources
- [GitHub, March 2026] Coral Bricks AI 'reef' repository | https://github.com/Coral-Bricks-AI