Nearshore Platform Notes

How we build and run cloud-native platforms and AI infrastructure for US teams. Real numbers, real configs, anonymized clients.

Self-hosted LLM inference on Kubernetes with LightLLM

· 4 min read

Serving our own models behind the MCP gateway: GPU node pools, continuous batching with LightLLM, and autoscaling on queue depth instead of CPU.

Running an MCP gateway on Kubernetes for AI agents

· 5 min read

One front door for AI agents: MCP servers behind a Kubernetes gateway with auth, per-tool authorization, session routing and a per-call audit log.