Procurement checklist
AI vendor security questionnaire: 40 questions to ask before you sign
An AI vendor security questionnaire tests what a standard security review misses: where prompts and outputs go, which model providers see your data, whether your data trains their models, how prompt injection is handled, and who answers for a wrong decision. Send these 40 questions with the RFP or before a pilot, and ask for evidence, not a yes or no.
AIDOLS Research Team · Updated · Editorial standards
Free to use and adapt. Opens in Excel, Numbers or Google Sheets, with blank columns for the vendor's answer, the evidence provided and your assessment.
How to use it
Step 1
Send it early
Send it with the RFP or before the pilot. Answers on data use, residency and incident notice decide whether a vendor is eligible at all, and a gap found after a pilot costs weeks.
Step 2
Ask for evidence
For each answer, ask for the document behind it: the report, the contract clause, the sub-processor list, the test results. A yes without evidence is not an answer.
Step 3
Score every answer
Mark each one good, partial or missing against the 'good answer' column. Compare vendors on the same sheet.
Step 4
Decide what is a stop
Agree before the answers arrive which gaps end the evaluation. For most buyers that includes training on your data by default and no written incident notice window.
Section 1 · 7 questions
Data handling and residency
DH-01
What data does the system receive, store and generate, and how long is each kind kept?
- Why it matters
- You cannot assess risk on data you have not inventoried, and retention decides how much is exposed in a breach and what has to be deleted later.
- What a good answer looks like
- A data-flow diagram listing each data category (inputs, prompts, outputs, logs, embeddings), where it is stored and a retention period for each, with retention you can configure.
Maps to: NIST AI RMF: Map
DH-02
Is our data used to train, fine-tune or improve any model, yours or a third party's?
- Why it matters
- Data used for training can resurface in answers to other customers and cannot easily be withdrawn once it is in a model.
- What a good answer looks like
- No by default, written into the contract, and covering the vendor and every model provider it calls. Any opt-in is separate, specific and in writing.
DH-03
In which countries and cloud regions is our data stored and processed, including backups, logs, support access and model inference?
- Why it matters
- Residency commitments often cover storage but not the model calls, backups or support staff in other countries.
- What a good answer looks like
- A region list per data category (storage, inference, backups, logs, support access), an option to keep data in Canada or your required region, and contractual notice before any change.
Maps to: Quebec Law 25; GDPR Chapter V
DH-04
How is our data separated from your other customers' data?
- Why it matters
- Shared retrieval indexes, caches and fine-tuned models can leak one customer's documents into another customer's answers.
- What a good answer looks like
- Isolation per customer for storage, retrieval indexes and caches, and tests showing one customer cannot retrieve another's content.
Maps to: OWASP LLM08:2025 (vector and embedding weaknesses)
DH-05
Who can read our prompts, outputs and logs, and do your model providers keep copies?
- Why it matters
- Prompts and outputs often hold the most sensitive data in the system, and provider-side retention for abuse monitoring is easy to miss.
- What a good answer looks like
- Access limited to named roles and logged; provider retention stated in writing, including any zero-retention arrangement and its exceptions.
DH-06
How do we export our data, and how do you delete it when the contract ends?
- Why it matters
- Exit is where lock-in and forgotten copies appear. Deletion has to reach backups, indexes and anything derived from your data.
- What a good answer looks like
- Export in an open format at no extra charge; deletion within a stated number of days covering backups, embeddings and fine-tuned weights, confirmed in writing.
DH-07
Does the system handle personal, health or payment card data, and how is that data minimized?
- Why it matters
- Each category brings its own legal obligations, and minimizing the data shrinks the scope of every other answer in this questionnaire.
- What a good answer looks like
- A classification of the data the system touches, masking or redaction before data reaches a model where possible, and a stated legal basis or consent model for personal information.
Maps to: PIPEDA Principle 4.4 (limiting collection)
Section 2 · 5 questions
Model providers and sub-processors
SP-01
Which third-party model providers and AI services does the product call, and for what?
- Why it matters
- Your data travels through every model API the vendor uses, and each of those providers is a sub-processor.
- What a good answer looks like
- A named list (provider, model family, purpose, region), kept current on a public page or in the contract.
SP-02
Who are all your sub-processors, and how will you tell us about changes?
- Why it matters
- You stay accountable for personal information you hand to a vendor, including what its own suppliers do with it.
- What a good answer looks like
- A complete sub-processor list with location and function, written notice before any change, and a right to object.
Maps to: PIPEDA Principle 4.1.3; GDPR Article 28
SP-03
What happens when a model provider changes, deprecates or withdraws the model you use?
- Why it matters
- Providers retire model versions on their own schedule, and a silent swap can change accuracy, cost and behaviour.
- What a good answer looks like
- Version pinning, notice before a model change, re-testing against your acceptance cases before the switch, and a fallback model.
SP-04
Do the terms you hold with your model providers support every commitment you make to us?
- Why it matters
- A vendor cannot promise you more than its own upstream contract allows on training, retention and residency.
- What a good answer looks like
- Confirmation that the upstream terms match each commitment made to you, with the relevant clauses available on request.
SP-05
How do you vet open-source models, datasets and libraries, and track their licences and vulnerabilities?
- Why it matters
- Model weights and AI libraries carry licence restrictions and supply-chain risk like any other software.
- What a good answer looks like
- A software bill of materials that includes models and datasets, a licence review, and a process for patching vulnerable components.
Maps to: OWASP LLM03:2025 (supply chain)
Section 3 · 7 questions
Security controls
SC-01
What independent security assurance can you share (for example a SOC 2 Type II report or an ISO/IEC 27001 certificate), and does its scope include the AI features?
- Why it matters
- A report that scopes out the AI product, or covers only corporate systems, says little about the system you are buying.
- What a good answer looks like
- A current report or certificate whose scope names the product and its AI components, plus a bridge letter for any gap since the report period ended.
Maps to: AICPA SOC 2; ISO/IEC 27001
SC-02
How do you defend against prompt injection, including instructions hidden in documents, emails or web pages the system reads?
- Why it matters
- Prompt injection is the first risk on the OWASP list for LLM applications, and any system that reads untrusted content is exposed to it.
- What a good answer looks like
- Layered controls (instructions kept apart from retrieved content, input and output filtering, least-privilege tool access) and regular adversarial testing with results you can see.
Maps to: OWASP LLM01:2025 (prompt injection)
SC-03
Which tools, APIs and write actions can the system use on its own, and with whose permissions?
- Why it matters
- An AI agent with broad permissions turns a wrong answer into a wrong action.
- What a good answer looks like
- A list of every action the system can take, least-privilege service accounts, human approval for irreversible or high-value actions, and a log entry for each action.
Maps to: OWASP LLM06:2025 (excessive agency)
SC-04
How are sign-in, access and administrative rights controlled?
- Why it matters
- Stolen credentials and excess admin rights remain common ways into business systems.
- What a good answer looks like
- Single sign-on with your identity provider, multi-factor authentication enforced, role-based access, and admin actions logged where you can review them.
SC-05
Is data encrypted in transit and at rest, and who controls the keys?
- Why it matters
- Encryption is standard; who controls the keys decides who can actually read the data.
- What a good answer looks like
- TLS 1.2 or later in transit, strong encryption at rest, and an option for customer-managed keys where the risk warrants it.
SC-06
When were the product and its AI features last penetration-tested or red-teamed, and by whom?
- Why it matters
- AI features add attack surface a standard web application test misses: prompt injection, data extraction and jailbreaks.
- What a good answer looks like
- An independent test in the last 12 months that covered the AI components, with a summary of findings and their remediation status.
SC-07
Can the system reveal its system prompt, other users' data or restricted documents in its answers, and how have you tested that?
- Why it matters
- Retrieval systems return documents a user should not see unless permissions are enforced at the moment of the query.
- What a good answer looks like
- Document-level permissions enforced at retrieval, output checks for sensitive data, and test cases that try to extract restricted content.
Maps to: OWASP LLM02:2025 and LLM07:2025
Section 4 · 6 questions
Privacy: PIPEDA, Quebec Law 25, GDPR
PV-01
Will you sign a data processing agreement that covers our obligations under PIPEDA, Quebec Law 25 and, where it applies, GDPR?
- Why it matters
- Under each of these laws you remain accountable for the personal information a vendor processes for you.
- What a good answer looks like
- A data processing agreement naming the purposes, sub-processors, security measures, incident notice, help with access requests, and deletion at the end of the term.
Maps to: PIPEDA Principle 4.1.3; Quebec Law 25; GDPR Article 28
PV-02
Does the system make, or feed, decisions about individuals based exclusively on automated processing?
- Why it matters
- Quebec Law 25 requires telling the person and offering a review by a human when a decision is based exclusively on automated processing; GDPR Article 22 restricts such decisions.
- What a good answer looks like
- A clear account of where the system decides and where it only recommends, a human review path, and the information needed to explain a decision to the person affected.
Maps to: Quebec Law 25; GDPR Article 22
PV-03
Can you give our privacy team what it needs for a privacy impact assessment of this system?
- Why it matters
- Quebec Law 25 requires a privacy impact assessment for new information systems involving personal information and before personal information leaves Quebec. A vendor that cannot supply the facts stalls yours.
- What a good answer looks like
- Documentation of data flows, purposes, retention, safeguards and transfers, in a form your privacy team can use directly in its assessment.
Maps to: Quebec Law 25
PV-04
How do you handle access, correction and deletion requests that involve data inside the AI system, including logs and embeddings?
- Why it matters
- People's rights over their personal information apply wherever it sits, including copies and derived data.
- What a good answer looks like
- A documented process and timeline, tooling that finds one person's data across every store, and support for your response deadlines.
Maps to: PIPEDA Principle 4.9 (individual access); GDPR Articles 15-17
PV-05
Is personal information transferred outside Canada or the EEA, and under what safeguards?
- Why it matters
- PIPEDA expects openness about processing in other countries, Quebec Law 25 requires an assessment before a transfer outside Quebec, and GDPR requires a transfer mechanism.
- What a good answer looks like
- A list of destination countries, the legal mechanism for each (for GDPR, for example Standard Contractual Clauses), and notice before any new transfer.
Maps to: PIPEDA; Quebec Law 25; GDPR Chapter V
PV-06
Does the product tell people when they are dealing with an AI system or AI-generated content?
- Why it matters
- The EU AI Act sets transparency duties for systems that interact with people or generate synthetic content, and customers expect the same elsewhere.
- What a good answer looks like
- Configurable AI disclosure in the interface and labelling of AI-generated content, documented so you can show it to a regulator or a customer.
Maps to: EU AI Act Article 50
Section 5 · 6 questions
Governance: NIST AI RMF, ISO/IEC 42001, EU AI Act
GV-01
Do you run an AI risk management program, and how does it map to the NIST AI RMF or ISO/IEC 42001?
- Why it matters
- A named framework shows the vendor manages AI risk as an ongoing process, not as a one-off review before launch.
- What a good answer looks like
- A documented program with an owner, a risk register for this product, and a mapping to the NIST AI RMF functions (Govern, Map, Measure, Manage) or to ISO/IEC 42001, with its certificate status stated plainly either way.
Maps to: NIST AI RMF 1.0; ISO/IEC 42001:2023
GV-02
What is the system's intended use, and which uses do you prohibit or advise against?
- Why it matters
- Risk depends on use. A written intended use is the basis for every other control in this questionnaire.
- What a good answer looks like
- A model card or system card stating intended use, known limitations and out-of-scope uses, updated with each major release.
Maps to: NIST AI RMF: Map
GV-03
How do you measure accuracy, bias and failure rates for our use case, before and after go-live?
- Why it matters
- Vendor benchmarks rarely match your data. Without acceptance tests on your own cases you cannot tell whether the system works, or when it starts to drift.
- What a good answer looks like
- Agreed acceptance test sets built from your data, reported metrics with thresholds, and ongoing monitoring with alerts when results move.
Maps to: NIST AI RMF: Measure
GV-04
Under the EU AI Act, is this product, or our use of it, a high-risk AI system, and what will you provide as its provider?
- Why it matters
- High-risk classification brings obligations for providers and for deployers, and you need to know which ones land on you.
- What a good answer looks like
- The vendor's own classification with its reasoning, the documentation it supplies as provider (technical documentation, instructions for use, logging), and what it expects of you as deployer.
Maps to: EU AI Act Article 6 and Annex III; Article 26
GV-05
How are model, prompt and configuration changes approved, tested and announced?
- Why it matters
- In an AI system a prompt edit or a model swap is a production change and can alter answers as much as new code.
- What a good answer looks like
- Change management that covers prompts, models and retrieval settings, regression tests against your acceptance cases, and release notes to customers.
Maps to: ISO/IEC 42001:2023
GV-06
Who is accountable for AI risk at your company, and how is it reported to leadership?
- Why it matters
- Accountability needs a name and a reporting line, not only a policy page.
- What a good answer looks like
- A named role with authority to stop a release, and regular reporting to executive leadership or the board.
Maps to: NIST AI RMF: Govern; ISO/IEC 42001 clause 5 (leadership)
Section 6 · 5 questions
Monitoring and incident response
IR-01
How quickly will you notify us of a security or confidentiality incident affecting our data, and what will the notice contain?
- Why it matters
- Your own deadlines start when you learn of an incident: GDPR gives controllers 72 hours to notify the regulator, and PIPEDA and Quebec Law 25 require prompt reports when people are at risk of serious harm.
- What a good answer looks like
- A fixed notice window in hours, written into the contract, with a named contact and the facts you need for your own reports.
Maps to: GDPR Article 33; PIPEDA section 10.1; Quebec Law 25
IR-02
What counts as an AI incident for you (harmful output, data exposed through answers, an unsafe action), and do you have a runbook for it?
- Why it matters
- Many incident plans cover breaches but not a model that starts giving wrong or harmful answers at scale.
- What a good answer looks like
- An AI-specific incident definition and runbook, and the authority and means to switch the feature off quickly.
Maps to: NIST AI RMF: Manage
IR-03
What do you log, how long do you keep it, and can we get the logs?
- Why it matters
- Without inputs, outputs and actions on record you cannot investigate a complaint or reconstruct a decision.
- What a good answer looks like
- Logs of prompts, retrieved sources, outputs and actions, kept for an agreed period and exportable to your own security tools.
Maps to: EU AI Act Article 12 (record-keeping, high-risk systems)
IR-04
How do you monitor the system in production for drift, misuse and unusual behaviour?
- Why it matters
- AI systems degrade quietly as data and usage change.
- What a good answer looks like
- Dashboards and alerts on quality, cost, error rates and misuse patterns, with a named owner who acts on them.
Maps to: NIST AI RMF: Measure
IR-05
Can you walk us through your most recent incident and what changed afterwards?
- Why it matters
- How a vendor describes a past incident tells you more than its written policy does.
- What a good answer looks like
- A candid, anonymized account of a real incident, its root cause and the fix, or a credible explanation if there has been none.
Section 7 · 4 questions
Commercial and contract terms
CT-01
Who owns the outputs, fine-tuned models, prompts and configurations built for us?
- Why it matters
- Work built on your data can become part of the vendor's product if the contract is silent.
- What a good answer looks like
- You own the outputs and anything trained on your data; the vendor keeps its pre-existing platform. Both are stated in the contract.
CT-02
What liability and indemnity do you offer for data protection breaches and for third-party IP claims arising from outputs?
- Why it matters
- Liability caps tied to fees paid can sit far below the cost of a breach or an IP dispute.
- What a good answer looks like
- A separate, higher cap for data protection breaches, and an IP indemnity that covers outputs produced through normal use of the product.
CT-03
How is pricing calculated, and what happens to our costs if usage or model prices change?
- Why it matters
- Usage-based AI pricing can grow faster than the value delivered, and upstream model price changes are often passed through.
- What a good answer looks like
- A clear pricing unit, spending caps or alerts, notice before price changes, and usage reports you can check.
CT-04
What service levels, support and exit assistance are in the contract?
- Why it matters
- An AI feature your team depends on needs the same availability and support commitments as any production system, and a way out.
- What a good answer looks like
- Uptime and response-time commitments with remedies, named support contacts, and exit assistance (data export, a transition period) on defined terms.
Answers that should stop the evaluation
- "We're compliant" with no report, certificate or contract clause behind it.
- No list of the model providers and sub-processors that see your data.
- Your data trains or improves models by default, or the answer depends on a plan tier.
- No incident notice window in the contract, only "promptly" or "as required by law".
- No way to switch the AI feature off quickly if it starts giving wrong answers.
- Accuracy shown only on the vendor's benchmark, never on your own cases.
This questionnaire is a starting point for your own due diligence, not legal advice. The laws and frameworks it names (PIPEDA, Quebec Law 25, GDPR, the EU AI Act, the NIST AI RMF, ISO/IEC 42001 and the OWASP Top 10 for LLM Applications) apply differently by sector and jurisdiction; confirm with counsel which ones bind you.
Frequently asked questions
What is an AI vendor security questionnaire?
A list of due-diligence questions a buyer sends an AI vendor before signing. It covers what a standard security questionnaire covers (access control, encryption, incident response) plus the parts specific to AI: model providers and sub-processors, use of your data for training, prompt injection, what the system can do on its own, accuracy testing, and accountability for automated decisions.
How is it different from a standard vendor security questionnaire?
A standard questionnaire assumes software that does what it is coded to do. An AI system reads untrusted content, calls third-party models, keeps prompts and outputs, and produces answers that can be wrong. Questions about training data, prompt injection, agent permissions, model changes and drift have no equivalent in a traditional review.
Which Canadian privacy laws apply when an AI vendor processes our data?
For most private-sector organizations, PIPEDA federally and, for personal information about people in Quebec, Law 25, which amended Quebec's private-sector privacy act. Alberta and British Columbia have their own private-sector privacy acts. Under each, you remain accountable for personal information a vendor processes for you, so the vendor's answers become part of your own compliance record. This page is not legal advice; confirm with counsel which laws apply to you.
Does a SOC 2 report cover AI risks?
Not by itself. A SOC 2 report tests controls within a scope the vendor chooses, and that scope can leave out the AI features entirely. It does not test prompt injection, use of your data for training, or the accuracy of answers. Check the scope first, then ask the AI-specific questions separately.
Can we use and edit this questionnaire?
Yes. Download the CSV, remove the questions that do not apply, and add your own. It has blank columns for the vendor's answer, the evidence provided and your assessment. All 40 questions are free to use.
When in the buying process should we send it?
With the RFP or before a pilot starts. Answers on data use, residency and incident notice decide whether a vendor is eligible at all; the governance and contract questions can follow during negotiation.
Before the questionnaire: does the workflow need AI at all?
Many workflows that reach procurement as an AI purchase need ordinary software or a process change instead. See AI vs automation vs software for how to tell, or have one workflow mapped free and get a written verdict.