Using a third party’s AI does not relieve an organization of its duties to consumers and employees affected by it. Angela Juneau of Pashman Stein Walder Hayden lays out the questions organizations should ask before they buy — from how a model was trained to who owns its outputs — so that legal, compliance and business teams can weigh the risks before deployment rather than after.
Organizations are using AI in virtually every aspect of their business: filtering job applicants, generating content, analyzing data, streamlining internal investigations, responding to customer service inquiries and generally accelerating business operations.
But businesses that adopt AI without understanding how a model was developed, how it operates and how it handles data may expose themselves to risks that do not become apparent until problems arise. As regulators, consumers, employees and business partners increasingly scrutinize AI use, organizations should approach AI procurement with the same diligence they would apply to any critical technology investment.
The following questions can help organizations evaluate third-party AI before deployment and establish a framework for evaluating AI-related risk.
Data lineage
Where did the AI’s training data come from? Was the model trained on data that was lawfully obtained and properly licensed? Did the developer take steps to minimize the use of personal information, copyrighted material or other sensitive data that could create legal or compliance risks?
Data quality
Can you trust the quality of the data behind the model? The quality of an AI system depends on the quality of the data used to train it. What steps has the vendor taken to ensure data accuracy, reduce bias and improve data quality?
Intended use
Is the AI suitable for your intended use case? An AI tool that performs well in one context may perform poorly in another. Has the model been tested for the specific business purpose for which it will be used?
At AI’s Inflection Point, How Do You Go From Experimentation to Enterprise Value?
Capturing enterprise value from AI now depends less on the technology itself than on how rigorously executives and boards govern, oversee and steer it
Read moreDetailsReliability
How accurate and reliable are the outputs? What testing has been performed to evaluate accuracy, consistency and error rates? How will your organization verify AI-generated content, recommendations or conclusions before relying on them?
Output ownership
Who owns the outputs and related intellectual property? Review the vendor’s terms carefully. Does your organization own the content generated by the AI or does the provider retain certain rights? Are there restrictions on how outputs may be used or shared?
Human oversight
What human oversight exists? Who is responsible for reviewing AI-generated outputs or AI-assisted decisions? If the AI influences employment, customer, financial or operational decisions, what mechanisms exist for review, escalation or correction?
Privacy and security
How does the AI handle sensitive information? What data is collected, stored, retained or shared? If employees, applicants, customers or proprietary business information are involved, what safeguards protect that information?
Transparency
How transparent is the system? Do users know when they are interacting with AI? Can the vendor explain how the system reaches conclusions, makes recommendations or generates outputs to the extent necessary for business and regulatory requirements?
Governance and monitoring
How is the AI monitored and governed over time? AI performance can change as data, business conditions and user behavior evolve. How does the vendor monitor performance, address model drift, respond to incidents and implement updates?
Business continuity
What happens if the system fails or becomes unavailable? Does your organization have a contingency plan if the tool becomes unavailable, produces harmful outputs, experiences a security incident or no longer meets business needs? What happens to your data if you terminate the relationship?
Organizations are responsible for their AI use and its outputs. Using third-party AI does not relieve an organization of its legal and regulatory duties to consumers and employees. Effective governance requires collaboration among legal, compliance, privacy, security, technology, human resources and business stakeholders to evaluate risks and establish appropriate safeguards.
The questions above are not intended to discourage innovation. Rather, they provide a practical framework for identifying risks before they affect operations, compliance or business objectives. Organizations that conduct due diligence and implement effective governance measures will be better-positioned to realize AI’s benefits while managing the legal, operational and reputational risks that accompany its use.
Angela Juneau is an attorney at Pashman Stein Walder Hayden and is certified as an AI Governance Professional (AIGP) by the International Association of Privacy Professionals (IAPP). She advises organizations on the legal, compliance, governance and operational risks associated with implementing AI. 
