Part ofAI API Hub

AI Agents Are Mass-Harvesting Keys from Self-Hosted AI Gateways: A Hardening Checklist for LiteLLM, One API, Dify, and More

1 viewsAPI keysAI gatewayAPI relayLiteLLMOne APIDifySelf-hosting security

On November 13, AWS reported attackers using AI agents to scan internet-exposed LiteLLM, One API, New API, Sub2API, Dify, Open WebUI, Ollama, and similar services, pull out the upstream model keys stored inside, and use the stolen credit to fund the next round of scanning. Following the entry points AWS lists, this guide gives a five-minute self-check, product-by-product hardening, a deployment layout that separates the admin side from the call side, and what to do in what order if a key has already been abused.

If you run LiteLLM, One API, New API, or Sub2API on a cloud server, or self-host AI services such as Dify, Open WebUI, or Ollama, this is worth reading now.

In a security blog post published on November 13, AWS reported that between October 3 and November 5, MadPot, its global network of decoy sensors, captured a large volume of requests from attackers' AI agents. The sensors posed as common large language model inference APIs, and the attackers' agents treated them as their own model backend, sending along whole task records, system prompts, and tool-call histories. The largest campaign visible in those records was the mass theft of AI model API keys: agents scanned internet-exposed gateways and AI apps, pulled out the upstream keys stored inside, checked that they worked, added them to a shared pool, and then used the stolen credit to power more agents that kept scanning.

The short version: Never expose admin consoles, setup pages, or inference ports directly to the internet. Replace default passwords and sample master keys today. Give the gateway its own key for each upstream provider, with a spend limit set in that provider's console. If you deploy with Docker, make sure ports are bound only to 127.0.0.1. Those four steps close most of the entry points AWS lists.

We've published several guides to running your own gateway, including setting up your own API relay, deploying a LiteLLM gateway, and deploying Sub2API. This is the security companion to those: what AWS saw, then self-checks and hardening steps you can follow.

What AWS saw

The opening of the AWS Security Blog post from November 13, 2026, titled as Amazon Threat Intelligence observing threat actors running cybercrime campaigns with ARTEX AI agents

The opening of the AWS Security Blog post from November 13, 2026, titled as Amazon Threat Intelligence observing threat actors running cybercrime campaigns with ARTEX AI agents

Figure 1: The opening of the original AWS Security Blog post, published by CJ Moses on November 13, 2026.

AWS assesses with moderate confidence that the campaigns used ARTEX, an open-source agentic penetration testing framework: a planner agent splits the work, worker agents run conventional attack tools, and verification and compression agents check and condense the results, all sharing a persistent knowledge graph. AWS is also clear about the limits: much of the recorded activity took place on systems MadPot doesn't monitor, so the outcomes the agents reported can't be verified.

What matters for self-hosters is which targets and entry points the post names.

Products named. LLM gateways and proxies: One API, New API, Veloera, Sub2API, LiteLLM. AI app builders: Dify, FastGPT, Langflow, RAGFlow, Open WebUI, LobeChat, Ollama.

Entry points used. None of the techniques AWS lists is new:

  • Installs that hadn't finished setup and could be claimed by whoever got there first;
  • Management and provider-configuration endpoints reachable without signing in;
  • Default admin credentials, with the LiteLLM default master key named specifically;
  • Known vulnerabilities, with Langflow's CVE-2025-3248 and RAGFlow's CVE-2026-28797 named;
  • An authentication bypass via a request header in One API family systems, for which AWS gives no fixed version;
  • Server-side requests through "fetch a URL" features, used to reach cloud metadata services;
  • Secrets in front-end JavaScript, source maps, debug routes, and environment files.

Where targets were found. Internet scanning services, certificate transparency logs, GitHub repositories and event archives, npm and PyPI packages, container image layers, notebooks, model-hosting spaces, paste sites, and Telegram channels.

One detail stands out: the attackers' instructions told agents to treat credentials that had been publicly exposed for more than 30 days as low-value inventory, since they'd likely already been used or revoked. In other words, they're after freshly leaked keys. A .env file you accidentally pushed to GitHub today will be found faster than you think.

Why gateways and AI app platforms are prime targets

Break into an ordinary web server and you get that machine. Break into an API relay and you get every upstream key behind it. AWS says the attackers preferred apps that hold many keys at once, because a single weak management interface can expose many downstream accounts.

The attackers' loop: find targets, get in, pull keys, validate them, pool them, and use the stolen credit to keep scanning

The attackers' loop: find targets, get in, pull keys, validate them, pool them, and use the stolen credit to keep scanning

Figure 2: The loop as AWS describes it. Stolen keys aren't just sold; they directly pay for the inference behind the next round of scanning, billed to the original owner.

For solo operators and small teams, the most direct loss is money: the upstream provider bills real usage, and by the time you see it at the end of the month, the credit may be gone. The second is data: gateway logs often hold full requests and responses.

A five-minute self-check: find out what you've exposed

These checks only read; they don't change anything. Run through them first, then decide what to fix.

1. See which ports the machine is listening on. On a Linux server, run:

sudo ss -tlnp

Look at the local address column. 127.0.0.1:port is reachable only from the machine itself; 0.0.0.0:port or [::]:port means any network interface can reach it. Common ports include 3000 for One API and New API, 4000 for LiteLLM, 11434 for Ollama, and the web ports for Dify and Open WebUI.

2. See where Docker published your ports.

docker ps --format '{{.Names}}\t{{.Ports}}'

Output like 0.0.0.0:3000->3000/tcp means the service is open to the whole internet. A common misconception: plenty of people assume a server is safe because ufw is turned on. Docker's documentation says that when you publish a container's ports, traffic is diverted before it reaches the ufw rules, which effectively bypasses your firewall configuration. Source

3. Try it from outside. From a phone on mobile data or a computer on another network, open your server's public IP and port in a browser, for example http://203.0.113.10:3000 (with your own IP and port). If you see a login page, a setup wizard, or an API response, it's on the public internet.

4. Check whether default credentials were changed. One API's README says the initial account is root with password 123456 and says you must change it after first logging in; for New API and other forks, check their own docs. LiteLLM's docs and examples long used sk-1234 as the sample master key. In a February 2026 scan, Wiz found 3,074 internet-reachable LiteLLM instances; 294 of them (9.6%) accepted sk-1234, and 191 required no authentication at all. Source

5. Check usage in your upstream consoles. Open the usage and billing page for every model provider whose keys sit in your gateway, and look for unexplained spikes, unfamiliar models, or steady overnight calls in the past month.

Product by product

These are grouped by the products AWS names. Settings change between versions, so the table gives the direction and the official guidance we checked; confirm against the docs for your version before changing anything.

ProductCheck firstHow to tighten
LiteLLMIs the master key sk-1234 or another sample value? Is one set at all?Replace it with a long random value (the docs require it to start with sk-), injected through the LITELLM_MASTER_KEY environment variable. Wiz notes the master key is also used to sign login sessions, so a default value lets someone forge sessions. Upgrade to a release with the relevant fixes
One API, New API, VeloeraIs the root account still on the default password? Can the admin console be reached from the internet?Change the password now. Because AWS gives no fixed version for the header-based auth bypass, keep the admin console off the internet and reach it only over a VPN or SSH tunnel
Sub2APIWho can reach the admin console? Which upstream accounts are in the pool?Set admin credentials as in our deployment guide and reach the console only on a private network or through a tunnel; treat the attached accounts as high-value credentials
Dify, FastGPT, Langflow, RAGFlowAny instance installed but not yet set up? Are you patched for the named CVEs?Finish admin setup immediately after deploying; upgrade Langflow and RAGFlow to versions that fix those CVEs; restrict outbound traffic from containers with "fetch a URL" features and block cloud metadata addresses
Open WebUI, LobeChatDid you create the first admin account yourself? Is sign-up open?Open WebUI's docs say the first account registered on a fresh install becomes admin automatically and list that window as an accepted risk; you can pre-create the admin on first start with WEBUI_ADMIN_EMAIL and WEBUI_ADMIN_PASSWORD. Source
OllamaHas OLLAMA_HOST been changed to 0.0.0.0?The docs say Ollama binds 127.0.0.1:11434 by default, and they describe no authentication for the local server; if you need remote access, put it behind a reverse proxy that requires authentication. Source

Keys in front ends and repositories. This applies whatever product you use. Anything a browser can see, an attacker can see: don't put upstream keys in front-end code, build output, or source maps; add .env to .gitignore; and don't COPY secrets into image layers at build time. Inject them at runtime instead.

Deployment layout: separate the admin side from the call side

However you configure each product, it comes down to one principle: the entry point that calls models can face the internet; the entry point that changes settings or shows keys can't.

A recommended self-hosted gateway layout: callers reach a locally bound gateway through a reverse proxy, admins reach the console over a VPN or tunnel, and the gateway uses a dedicated, spend-limited key for each upstream

A recommended self-hosted gateway layout: callers reach a locally bound gateway through a reverse proxy, admins reach the console over a VPN or tunnel, and the gateway uses a dedicated, spend-limited key for each upstream

Figure 3: Call traffic and admin traffic take separate routes. The internet only sees the call endpoints the reverse proxy allows.

In practice:

  1. Bind services to the local machine only. Write Docker ports as -p 127.0.0.1:3000:3000 instead of -p 3000:3000. Docker's docs say that without a host address, ports are published to all host addresses (0.0.0.0 and [::]); you can also change the daemon's default binding to 127.0.0.1. Source
  2. Expose only the reverse proxy. Serve HTTPS with Nginx or Caddy, allow only the paths callers need (for example /v1/), and reject admin consoles, setup pages, and debug routes at the proxy.
  3. Reach the admin console through a tunnel. When you need to change settings, connect over a VPN, ssh -L port forwarding, or a zero-trust access tool, and disconnect when you're done.
  4. One downstream key per caller. Issue a separate downstream key to each caller (each app, each colleague's Claude Code or Codex CLI) with its own quota and expiry, so you only revoke one when something goes wrong.
  5. Turn on logging and alerts. Watch at least three things: spending spikes, channels or tokens in the console that you didn't create, and admin logins from unfamiliar IPs.

Limit the upstream keys themselves

However well you guard the gateway, assume it will be breached someday, and configure upstream keys as though they'll leak:

  • One key per gateway. Don't share it with local development, CI, or other projects, so the blast radius is clear and revoking it doesn't break anything else.
  • Set spend limits in the provider console. In Claude Console, for example, you can give each project its own workspace with its own spend limit; most other providers offer project-level budgets or limits too, configured in their own consoles.
  • Rotate regularly, and confirm afterward that the old key really stops working.
  • Don't put subscription accounts behind a relay. Plugging a personal subscription into a shared relay can itself violate the terms of service, and if it leaks you lose the whole account. For how subscriptions and API billing differ, see how to pay for Claude and Codex.

If a key has already been abused

When you see unusual upstream billing or unfamiliar changes in the console, work in this order:

  1. Revoke the key in the upstream provider's console first. It's the fastest way to stop the loss; don't wait until you've finished investigating.
  2. Take the service off the internet: close the public port or stop the reverse proxy so the attacker can't come back with other credentials.
  3. Replace all admin credentials: admin passwords, the master key, database passwords, and any cloud access keys readable on that server.
  4. Review the console's change history: new channels, tokens, and users, and any modified model mappings. In AWS's report, the attackers graded and recorded the access they obtained, so places they got into may be reused.
  5. Decide whether to rebuild the server. If the way in was a remote code execution bug (such as the Langflow and RAGFlow issues named above), changing passwords isn't enough; redeploy on a clean system and restore your data.
  6. Contact the upstream provider's support with the time window of the abuse. Whether charges are waived is up to the provider, but reporting early is better than not reporting at all.

Who should act, and the alternatives

Act now if you self-host any of the services above on an internet-facing server, especially if you've opened a relay to friends or a team, or deployed with a one-click Docker setup and never changed the ports or default passwords.

Lower risk: you only run Ollama on your own computer and the port is still the default 127.0.0.1, or you use official APIs directly without any middle layer. You should still check your repositories and front ends for leaked keys.

If you'd rather not maintain the security yourself, use the provider's official API directly, or a hosted model aggregator such as OpenRouter, and leave gateway operations to the service; the trade-off is less flexibility and cost control than self-hosting. For a comparison of the two approaches, see OpenRouter vs. official APIs. For where a gateway sits in the architecture, start with the wiki entries on AI gateways and API keys.

Frequently asked questions

My server runs ufw. Is a gateway deployed with Docker still exposed?

It can be. Docker's documentation says that when you publish container ports, traffic is diverted before it goes through the ufw rules, effectively bypassing your firewall configuration. Writing the port as 127.0.0.1:port:port and exposing it through a reverse proxy is the reliable approach.

I've already changed the default password. Do I still need to take the admin console off the internet?

Yes. AWS mentions a header-based authentication bypass in One API family systems with no fixed version given, and Langflow and RAGFlow have named vulnerabilities too. A password only stops the "guessed the password" route; keeping the admin console off the internet stops the others.

Do I need to worry if I only run Ollama on my own machine?

By default the risk is low, because Ollama binds only to 127.0.0.1:11434. If you changed OLLAMA_HOST to 0.0.0.0 so your phone or LAN can reach it, and the port is mapped to the internet, put it behind a reverse proxy with authentication, since the docs describe no authentication for the local server.

How can I tell whether someone else has used my upstream key?

Check the usage details in the provider's console: filter by date, model, and key, and look for spikes you can't explain, models you've never used, or steady calls during hours you don't work. If in doubt, just rotate it; that costs less than investigating.

Summary

The most useful line in the AWS report is that the agents succeeded where basic security hygiene was missing: default or hardcoded credentials, keys left in code and public artifacts, unauthenticated interfaces and inference endpoints, and unpatched software. AI agents make scanning faster and tireless, but they still exploit the same old problems. For a self-hosted gateway, taking the admin side off the internet, replacing default credentials, and putting spend limits on upstream keys blocks most of the entry points the report describes.

References

Report incorrect information

Choose an issue below. You do not need to sign in or leave contact details.