---
title: "Nexus"
tagline: "AI gateway"
language: "Go"
license: "Apache-2.0"
since: "2026"
canonical: "https://xraph.com/work/nexus"
---

# Nexus

A gateway for LLM traffic that you compile into your own binary. Route across providers by cost, latency or priority; cache responses with optional semantic matching; redact PII and block prompt injection on the way through; enforce per-tenant budgets. It also exposes an OpenAI-compatible endpoint, so existing SDKs point at it unchanged.

## Install

```bash
go get github.com/xraph/nexus
```

## What it is

Nexus is an AI gateway you compile into your own binary. Route across providers, cache, guard and observe LLM traffic without running a separate SaaS in the path.

## What it does

- **Multi-provider routing** to OpenAI, Anthropic or any OpenAI-compatible API, with priority, cost-optimised, round-robin and latency-based strategies.
- **Guardrails**: PII detection, prompt injection blocking and content filtering, each with block, redact or warn actions.
- **Caching** with deterministic keys and optional semantic matching, over memory or Redis.
- **Multi-tenant isolation**: scoped requests, API keys, rate limits, usage tracking, budget enforcement and per-tenant model aliases.
- **Model aliases** mapping a virtual name to a provider target, with per-tenant overrides and weighted routing, so "fast" can mean something different for each customer.
- **Transforms**: system prompt injection, RAG context augmentation, anonymisation and output normalisation.
- **Streaming**: reasoning deltas, tool-call streaming, multi-modal chunks, and four wire formats: OpenAI-envelope SSE, nexus-native SSE with named events, NDJSON, and a bidirectional WebSocket at /v1/realtime. Record-and-replay caching is idempotent across all of them.
- **OpenAI-compatible proxy**, so existing SDKs point at your gateway unchanged.

## Why not a hosted gateway

Every hosted AI gateway puts a third party in the path of your prompts, which is a data-governance conversation before it is an architecture one. Nexus is fifteen lifecycle hooks and three storage backends in your own process. The trade is that you operate it, and you already operate the process it lives in.

---

*Unpublished draft. Not peer reviewed and not submitted to a venue.*


## Anatomy

- **Routing**: Multi-provider with aliases and fallbacks.
- **Guardrails**: Checks applied before and after the model call.
- **Caching**: Response reuse keyed on the full request.
- **Proxy**: OpenAI-compatible surface so clients need no changes.

## Constraints and trade-offs

- (decision) Not a hosted gateway: prompts and completions are the most sensitive data a product has.
- (constraint) Multi-tenant isolation is enforced at the gateway, not left to the caller.