Field notes · 29 October 2026 · AI

The AI supply chain: models, plugins and tool servers as third parties

In plain words: An AI helper is built from parts made by other people, and any of those parts can be swapped for a bad one, so you need to know who made each part and what it can touch. AI, explained simply

In short

• An AI system is not one product. It is a hosted model, plus weights pulled from a hub, plus the libraries, editor plugins, tool servers and SaaS connectors that let it read and act. Each of those is a third party.
• The real incidents so far have come through the small parts, not the big model providers: a fake tool server that copied every email it sent, a coding assistant release with a wiping prompt in it, an AI chat app whose tokens opened hundreds of customers' CRM systems.
• The controls are not new. Know which parts you run, where each came from and what it can reach; pin versions; give each part its own narrow credentials; and put the ones that hold tokens into the same supplier register as everything else.

Five kinds of supplier

When an organisation reviews an AI supplier, it usually means the company whose model it pays for. That review matters, but it covers one part of five.

1. Model providers. The hosted model behind an API. This is the part procurement already sees: a contract, a data processing agreement, an assurance report.
2. Model weights. Open models downloaded from a hub and run in-house. A model file is code as much as data: the common PyTorch format is built on Python's pickle, which runs instructions as it loads.
3. Libraries and plugins. The Python packages, coding assistants and editor extensions that developers install, often without anyone else knowing.
4. Tool servers. The small programs, now mostly using the Model Context Protocol (MCP), that give an agent its hands: send an email, query a database, open a ticket. They run with whatever credentials they are given.
5. SaaS connectors. AI features inside other products, joined to your systems with OAuth tokens that outlive the meeting where someone clicked "Allow".

The OWASP Top 10 for LLM applications puts supply chain third in its 2025 list for this reason, and the joint guidelines for secure AI development from the UK's NCSC, the US's CISA and partner agencies name the parts outright: "models, data, software libraries, modules, middleware, frameworks, and external APIs". Their advice is plain: hold AI suppliers to "the same standards your own organisation applies to other software".

What has already gone wrong

Each of these came through a part that rarely appears in a supplier review.

• A tool server that copied mail. In September 2025 Postmark warned that an npm package called postmark-mcp was not theirs. It had copied their name and published fifteen honest versions; version 1.0.16 added one line that blind-copied every email it sent to an outside address. Koi Security, who found it, counted about 1,500 downloads a week.
• A coding assistant with a wiping prompt. In July 2025 a GitHub token with too much access, left in a build configuration, let someone push code into the Amazon Q Developer extension for VS Code. Version 1.84.0 shipped with a prompt telling the agent to delete local files and cloud resources. AWS says the code did not run because of a syntax error, and replaced the release with 1.85.0.
• An AI chat app's tokens. In August 2025 attackers stole the OAuth tokens that Salesloft's Drift chat agent used to write into its customers' Salesforce systems. Google's threat intelligence team says they used them from 8 to 18 August to pull data from many companies, searching support cases for passwords and cloud keys. The customers had not been breached in any usual sense; a connector they had approved was used as designed, by someone else.
• Model files that run code. In February 2025 ReversingLabs described two models on Hugging Face whose files opened a remote shell when loaded. They had been packed so that the hub's scanner stopped at a deliberate error after the payload had already run. Hugging Face removed them within a day and changed the scanner.
• Packages that do not exist yet. A study presented at USENIX Security 2025 generated 576,000 code samples from 16 models and found that, on average, 5.2% of the packages commercial models suggested, and 21.7% from open models, did not exist. Anyone can register one of those names and wait for an assistant to recommend it.

Why the usual review misses them

My own view is that three habits leave the gap.

The first is that most of these parts are free. Supplier management is driven by contracts and invoices; an MCP server installed from npm or a model pulled from a hub produces neither, so it never reaches the register. The DORA third-party register asks for ICT services by contract, and the same blind spot carries into any sector's version of it.

The second is that they are installed by individuals. A developer adds an editor extension; an analyst connects a chat app to the CRM; a team tries a tool server on a Friday. Each is reasonable on its own, and none is visible to whoever owns supplier risk.

The third is that their access is borrowed. A tool server or connector usually runs with a person's token or a shared key, so it looks like that person in the logs. That is the same problem as an agent acting with borrowed credentials, one level down.

What to do on Monday

• List the parts. For each AI feature, write down the model, where any weights came from, and every plugin, tool server and connector it uses. CycloneDX has a machine-learning bill of materials (ML-BOM) format for this, which can sit beside the software bill of materials you may already keep.
• Pin and verify. Install tool servers and plugins from their publisher's own repository, pin the version, and review changes before updating. Prefer model formats that hold weights only, such as safetensors, over ones that run code when loaded.
• Give each part its own key. A tool server or connector should have its own narrow, short-lived credential, not a person's token, so that you can see it in the logs and revoke it alone.
• Put token holders in the register. Anything that holds a token into your systems is a supplier, paid or not. Add a line for it, with an owner and a date to review it.
• Ask your model providers the same questions as everyone else: where data goes, how long it stays, which subprocessors they use, and how they will tell you when something changes.

None of this needs an AI-specific programme. It needs the existing third-party and software supply chain work to notice that the parts are now smaller, free and installed one person at a time.

Sources

• OWASP GenAI Security Project, LLM03:2025 Supply Chain (2025).
• NCSC, CISA and partners, Guidelines for secure AI system development: secure development (November 2023).
• The Register, Fake Postmark MCP server hijacked to steal emails (29 September 2025), reporting Koi Security's findings.
• AWS, Security bulletin AWS-2025-015: Amazon Q Developer for VS Code (CVE-2025-8217) (23 July 2025).
• BleepingComputer, Salesloft breached to steal OAuth tokens for Salesforce data-theft attacks (August 2025), reporting Google Threat Intelligence and Salesloft.
• ReversingLabs, Malicious ML models discovered on Hugging Face platform (February 2025).
• Spracklen and others, We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs (USENIX Security 2025).
• OWASP CycloneDX, Machine Learning Bill of Materials (ML-BOM).

• Earlier on this topic: Jev, the model that only decides: where it fits in security work

Follow with the feed

Personal work and my own view, not my employer's. Sources are public.