Skip to main content
TACAVAR
•AI Ops

Is Your AI Gateway or Inference Server Exposed? A Hardening Checklist

Self-hosted AI gateway and inference server checklist: GPUStack, SGLang, Kong and key-theft lessons, and what to lock down first.

If an AI gateway or inference server is reachable from an untrusted network, treat it as exposed: the November 2026 bugs below turn a reachable admin or usage API into stolen provider keys or code execution. Inference server security comes down to trust boundaries: upgrade GPUStack past the KEV'd auth bypass, patch SGLang and Kong, and put real authentication in front of every management endpoint.

In one recent week, four separate reports landed on the same failure: a management API, usage counter, or admin interface reachable from an untrusted network, with a default trust assumption doing the rest. If you run GPUStack, SGLang, llama.cpp, an Ollama-style runtime, or a gateway like Kong AI Gateway or LiteLLM, this is your checklist — not a CVE encyclopedia.

What "exposed" means for an inference server or AI gateway

An inference server or LLM gateway is exposed when something designed for a local, trusted context is reachable from an untrusted one. The pattern showed up four ways:

  • Loopback treated as identity. GPUStack's auth bypass (CVE-2026-102895) promoted any request appearing to come from 127.0.0.1 to the built-in administrator. Behind the standard same-host reverse proxy, every remote request arrives with a loopback TCP peer — a trust signal meant for local convenience becomes a network-facing login bypass.
  • A client-supplied value reaching a privileged code path. Kong's CVE-2026-92544 was SQL injection through a client-supplied model name written into an LLM usage-counter table — a field any API client controls flowed into a query.
  • A bug in the engine itself. NEAR AI's incident started with a remote code execution vulnerability in SGLang — one server, then lateral movement. Confidential computing did not prevent it: the vulnerable code was running inside the enclave.
  • Someone else's automation hunting all of the above. Amazon's MadPot decoy sensors captured requests from offensive AI agents carrying records of credential-theft campaigns against internet-exposed AI applications — campaigns that survived by finding gateways and app builders that aggregate provider keys and pulling the keys out.

Amazon's post says none of the techniques in these campaigns were new — what agents change is speed and persistence. What matters for a self-hosting team is the target shape: Amazon's named targets were LLM gateways and proxies including One API, New API, Veloera, Sub2API, and LiteLLM, and AI application builders including Dify, FastGPT, Langflow, RAGFlow, Open WebUI, LobeChat, and Ollama. Amazon assessed the campaigns with moderate confidence as ARTEX-attributed and Chinese-nexus, and states it did not observe the activity as it happened and cannot verify the outcomes the agents reported. It did not name GPUStack, SGLang, or Kong — those findings are below. Earlier ARTEX reporting, on attacks against South Korean financial firms, is covered in our piece on Claude as an autonomous hacking agent.

What landed in one week (four pegs, attributed separately)

GPUStack: an auth bypass now on CISA's KEV list

CISA added CVE-2026-102895 — "GPUStack Improper Authentication Vulnerability" — to the Known Exploited Vulnerabilities catalog on November 9, 2026, citing evidence of active exploitation. Per the GitHub advisory (GHSA-gj9m-f8ph-pmhr, CVSS 9.8 Critical), in GPUStack before 2.2.0 a request whose source appears to be 127.0.0.1 is promoted to the built-in administrator when force_auth_localhost is false — the default. The source can be a same-host reverse proxy's TCP peer or a forged X-GPUStack-Real-IP header. The vendor reports exploitation in the wild against a v2.1.1 deployment, with impact including enumerating and modifying users, models, and workers; minting persistent admin API keys; disclosing secrets; and code execution through an admin-only endpoint.

What to do, and who says it:

  • The GHSA says the fix is in 2.2.0; upgrade to 2.2.0 or later.
  • Ostorlab (updated November 11) recommends 2.2.2 or later as the safe floor, because 2.2.2 also closes a separate SAML-callback bug (CVE-2026-102894) affecting 0.7.1–2.2.1. Ostorlab's two points teams skip: recreate the container rather than only pulling the image, and rotate worker and cloud-provider credentials — an upgrade does not revoke API keys an attacker already created.
  • The advisory's workarounds — force_auth_localhost=true, stripping client-supplied real-IP headers at the edge, keeping the API off untrusted networks — reduce exposure to CVE-2026-102895 only.
  • One caution on deadlines: CISA's KEV entry carries a November 12 remediation due date, but that applies to federal civilian agencies under Binding Operational Directive 26-04. It is not a deadline for your company — it is a signal that this bug is being exploited now.

We don't reproduce exploit steps or payloads here.

NEAR AI: an SGLang RCE on an inference server

NEAR AI's incident report, dated November 9, 2026, describes an attacker exploiting a remote code execution vulnerability in SGLang on their servers; code execution on one inference server led to broader infrastructure access. Per NEAR AI's own numbers and caveats: account details (email, name, and for some accounts plan and balance data) were downloaded for approximately 100 users; 18 agents on a legacy deployment outside NEAR AI's confidential computing (TEE) environment were reached, including chat history, settings, and logs. NEAR AI found no indication stored Private Chats or uploaded files were accessed, and two Cloud customers' API keys that appeared in logs were revoked.

NEAR AI's own lesson: confidential computing protects data from the operator and the host — it does not protect against a vulnerability in the software running inside the environment. The engine is code, and code has bugs. They reported the vulnerability to the SGLang project. NEAR AI has not published a CVE or a fixed SGLang version, so watch the SGLang project's releases and advisories. The SGLang upgrade-and-check list in our self-hosted AI sovereignty guide was written for an earlier, unrelated SGLang bug, but it is still a sound baseline.

Kong AI Gateway: a model name that runs SQL

Kong's AI Gateway changelog for 2.2.1 (released November 10, 2026) states: "Security: Fixed CVE-2026-92544 in the LLM usage counters by escaping every value before running the SQL statement, preventing SQL injection when a client-supplied model name is written to the license_llm_data table." The same fix appears without the CVE id in the 2.3.0 changelog (November 4, 2026), and Kong Gateway Enterprise lists CVE-2026-92544 fixes in 3.16.0.2, 3.15.0.7, 3.14.0.17, 3.13.0.12, and 3.10.0.20, all dated November 10. Check the live changelogs against your version before you patch — we are not attaching a CVSS score because no Kong primary states one.

The lesson travels beyond Kong: telemetry surfaces — usage counters, license reporting, request logs — are attack surface too, and a field as innocuous as a model name can carry an injection payload.

Amazon: agents hunting exposed gateways for provider keys

The AWS Security Blog post from CJ Moses (November 13, 2026) describes MadPot decoy sensors, between October 3 and November 5, 2026, capturing requests from AI agents that included operating memory from offensive campaigns run on ARTEX, an open-source agentic penetration-testing framework. The most prominent campaign was large-scale theft of credentials for AI models — organized so stolen keys paid for the inference powering further credential hunting. The shared method: find internet-exposed AI applications, extract the upstream provider keys they hold, validate those keys, and pool them.

Amazon's defender takeaway: bring AI applications, model gateways, and inference endpoints under the same controls as other production systems, and watch for unexpected spend or unusual usage.

A self-hosted LLM security checklist

Vendor-agnostic, in the order most teams should work it:

  1. Bind management APIs to loopback or a private network. Admin panels, provider-configuration endpoints, and anything that can mint keys should not sit on a public interface. If it must be reachable, put it behind a VPN or an authenticating edge — not a bare port.
  2. Put real authentication in front of every admin and provider-config endpoint. GPUStack's CVE-2026-102895 is the case study: loopback origin and client-supplied headers are network topology, not identity. If your server has a "trust localhost" default, see what a same-host reverse proxy does to every peer address, and turn it off.
  3. Set and rotate an API key before exposing server tools or MCP off loopback. The llama.cpp v1.0.0 server README documents --allow-unauthenticated-tools as allowing server tools, MCP servers, or an MCP proxy on a non-loopback address without an API key, "only for trusted networks (default: disabled)." That default is right; set --api-key if there's any doubt.
  4. Disable or change default administrator and master keys. Amazon's post names default master keys on gateway platforms as one playbook entry agents used; our LiteLLM 1.107.0 compromise piece also covers a separate LiteLLM Enterprise master-key leak advisory.
  5. Do not leave provider keys where clients can reach them. Client-side JavaScript, source maps, debug routes, and environment files served from the web root are the classic leak paths — see our leaked AI API keys piece.
  6. Patch the inference engine and the gateway on production cadence. SGLang had an RCE; Kong had SQL injection in a usage counter; GPUStack had an auth bypass exploited in the wild. The KEV listing is your cue that "when we get to it" is not a schedule.
  7. After a suspected exposure: revoke, then review. Revoke provider keys at the provider, revoke locally minted admin and API keys (an upgrade does not revoke them for you), review usage and billing for the exposure window, and check administrative state for changes you didn't make. Our leaked AI API key first-hour checklist covers the revoke-and-review order.

Where this becomes a standing function

Everything above is a point-in-time fix. The failure behind all four pegs — trust defaults nobody re-examined, management surfaces treated as internal, keys pooled without governance — recurs every time you add a model, a provider, or a new agent. Turning that into a standing function — who owns exposure review, which endpoints require real authentication, how key rotation is triggered — is an operations practice, not a one-time patch. Our buying guide is the shorter version.

If you want a first read on how visible and citable your AI-facing surfaces are from the outside, start with the free GEO scan.

FAQ

Is my AI gateway or inference server exposed? Check three things: whether any admin, management, or provider-configuration endpoint is reachable from an untrusted network; whether the server has a "trust localhost" or default-credential behavior that a reverse proxy or a default install turns into an auth bypass; and whether provider keys sit anywhere client-reachable. GPUStack's KEV-listed bypass is the clearest recent example of the second. If a management API answers without real authentication, treat the server as exposed now.

What should I do if my GPUStack or SGLang server was reachable from the internet? Upgrade past the vulnerable versions (GPUStack 2.2.0 per the GitHub advisory; Ostorlab recommends 2.2.2 as the floor, and recreate the container rather than just pulling the image), update SGLang to its latest release (NEAR AI has not named a CVE or fixed version), then revoke provider keys at the provider, revoke locally minted admin API keys — upgrading does not revoke keys an attacker already created — and review usage and billing for the exposure window. CISA's KEV listing reflects active exploitation, so treat a reachable pre-2.2.0 GPUStack as compromised until reviewed.

Did Amazon observe attackers stealing keys from Kong, GPUStack, or SGLang? No. Amazon's Threat Intelligence post described agents hunting internet-exposed AI gateways and app builders — its named list includes One API, New API, Veloera, Sub2API, LiteLLM, Dify, FastGPT, Langflow, RAGFlow, Open WebUI, LobeChat, and Ollama — but it did not name GPUStack, SGLang, or Kong as targets. Amazon also states moderate confidence on its ARTEX attribution and says it cannot verify the outcomes the agents reported.

Does self-hosting or confidential computing make an inference-server bug go away? No. NEAR AI's SGLang incident ran inside an inference server, and their report is explicit that confidential computing protects data from the operator and the host but not against a vulnerability in the software running inside the environment — the attacker could even observe confidential-mode requests in processing on the accessed server. Self-hosting changes who operates the software, not whether the software has remotely exploitable bugs.