A vendor can switch on AI decision-making through a routine software update that sets off none of the usual diligence gates: no purchase, no contract event, nothing for a reviewer to flag. Technology executive Robin Horstmann argues the release note itself is a third-party risk event to which you need to be paying closer attention.
You would never let a vendor walk into your operation and start deciding on their own which product problems are serious enough to stop a shipment. But that is pretty much what happens when a routine software update switches on AI to do exactly that — and in most companies, nothing in a review process will catch it.
For argument’s sake, consider a wellness supplement plant running a quality management system. Software logs every problem on the line — an ingredient weighed wrong, a blend mixed too briefly, a contaminant found at a check — and routes each one to a person who decides whether it’s serious enough to stop the shipment. One day, the software vendor sends out an update with a note reading: “AI-assisted deviation triage, enabled by default.”
Does anybody have real insight into that change? Legal doesn’t, because the contract already allowed the vendor to add features. Procurement doesn’t, because nothing was bought. The change approver clears it as routine, because there’s no new system to connect and nothing new to install. And the risk assessment sitting in the file still describes the old version — the one that sent every problem to a person. The update was cleared, but no one checked what the new feature actually did — that a decision a person used to make was now being made by software.
This isn’t just a manufacturing problem: The same concern applies to an HR system or a procurement tool.
The vendor in our made-up scenario did nothing wrong. Most SaaS agreements contain a unilateral modification clause that lets the vendor add or alter features at its discretion, and it typically requires notice only when a change removes functionality. Adding an AI capability removes nothing, so no notice is owed. The right to train on customer data usually lives elsewhere, in the services agreement’s language about improving or developing the services.
The place to fix this is the next renewal. A notice obligation tied to “material change” is weak, because the vendor decides what counts as material. The obligation should name the capability: written notice before the vendor activates a feature that lets its software make decisions, generate content or score customer data on its own. It should come with the right to turn that capability off for the whole organization rather than user by user as well as a documentation requirement — which model and version, where it runs, how long inputs and outputs are kept and whether the vendor trains on customer data.
US Finalizes CMMC Rule: Cybersecurity Verification Now Determines Contract Eligibility for Defense Contractors
New requirements expected to affect more than 300,000 defense contractors by the time they are fully phased in
Read moreDetailsYou can’t turn the update down, but you can catch it
Cloud software takes away the usual chance to say no. The vendor updates every customer at once on its own schedule; there’s no version to approve and no upgrade to decline. The change arrives whether anyone reviews it or not, which means the only real control left is noticing it in time to do something about it.
That job lands on whoever owns the system in quality or in IT. They receive the vendor’s release announcements and open a change ticket for each update. That ticket is the one reliable place to ask the question that separates a routine release from a third-party risk event: Does this update add or change anything that lets the software make decisions, generate content or score company data on its own?
When the answer is yes, the ticket should go to whoever owns third-party risk, who decides whether the change needs a fresh look. That look is far narrower than the review that precedes signing a vendor — it covers a single new capability, not the whole platform — and it comes down to two questions. Does anyone now act on what the software decides without a person in the loop? And would the record it produces survive a regulator asking how the decision was made?
Not every vendor is worth this attention. The vendors deserving this scrutiny are those that hold your most sensitive data and touch your most consequential decisions. Rank vendors on those two measures, take the top tier and record a named owner against each in the vendor file. For that group, a release announcement is too easy to miss in an inbox, so put a standing review on the calendar every quarter with the person who owns the relationship.
The questions to ask whenever AI shows up
The same questions work whether a capability arrived in last week’s update or came bundled with the original contract. Most can be answered by the vendor’s account team, as long as someone technical is on the call.
- Which model is it, whose infrastructure does it run on and in what region?
- Does the vendor send company data to the model provider, and is that provider named as a sub-processor under the current terms?
- Does the vendor train or fine-tune on customer data? What’s the default, and how is a change to that confirmed in writing?
- Can the capability be switched off for your organization alone — and is that a supported setting or a workaround the vendor might remove later?
- What do the logs actually capture: inputs, outputs, the confidence or ranking score, who accepted or overrode the result, and how long each is kept?
- What evidence does the vendor hold that the capability is accurate for the specific decision it’s making, and what did it test against?
- How does the vendor tell customers when it changes the model version — and will it commit to that in writing?
- How does the vendor classify itself under the EU AI Act for this capability, and what does it treat you as?
The answers decide whether the reassessment ends in paperwork or in a finding, and whether the capability stays switched on while the question is open.
One question deserves settling before any of this comes up: who inside the organization is allowed to shut the capability down or override it? The gap here is wide. In an ISACA survey this year of more than 3,400 digital trust professionals, 39% didn’t know whether their organization had a documented process for shutting down or overriding an AI system, and 56% didn’t know how long halting one would take after a security incident.
The release note you’ll never see
Every bit of this assumes the vendor is yours. Take one step outside the company and that falls apart. A co-manufacturer runs its own quality management system, licensed from its own vendor. The brand owner has no contract with that vendor, no access to its release notes and no standing to ask it anything. The assessment on file describes the co-manufacturer, not the software it runs — so the only place to reach the risk is the co-manufacturing agreement itself.
That means writing the protections in as flow-down obligations: notice when the capability of any system touching your product changes and the same documentation the platform vendor gives its own customers. If a co-manufacturer’s AI releases a batch that should have been held, the recall notice carries the brand owner’s name, not the co-manufacturer’s.
The exercise that surfaces all of this takes an afternoon. Take the 10 vendors holding your most sensitive data. For each, compare the date of the most recent assessment against the software version running in production now and read the release notes for everything in between. Then ask every co-manufacturer the same question about the system on its floor. The result is either a clean answer or a list of AI capabilities already making decisions that no one has assessed, which is exactly the list worth having.
Robin Horstmann is a technology leader with 30 years in enterprise IT for manufacturers and distributors, more than 25 of them in consumer goods. She holds the AIGP, AAISM, CRISC, CISM, CDPSE and PMP certifications and wrote “The Governed Enterprise: An AI Governance Playbook for Consumer Goods.” 
