# Evaluate an AI SDLC tool 29 questions to put to a vendor. Open https://aisdlclandscape.org/evaluate/ to see what a tool's vendor has published for each. ## Data Where code and prompts go, who processes them and what is kept. 1. Which model providers and hosting providers process our code and prompts, model by model? Why: One vendor documents that prompts and code context go to several outside model providers, and that its own models may add inference providers. Source: https://cursor.com/docs/enterprise/privacy-and-data-governance 2. Does zero data retention apply to every model and every preview feature, or only some? Why: One product's models run on several hosts, and retention terms and exceptions differ between them. Source: https://docs.github.com/en/copilot/reference/ai-models/model-hosting 3. Is our code used to train any model, yours or a provider's, and where is that written down? Why: Buyers name uncertainty about what models learn as a reason not to buy. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ 4. What happens to our terms when a provider changes its own? Why: A contract that spans several providers can change under the buyer without the vendor changing anything. Source: https://docs.github.com/en/copilot/reference/ai-models/model-hosting 5. Can the tool run in our cloud account, on premises or in a single tenant? Why: Where code goes decides what a security team will sign. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ ## Continuity What happens if the vendor, its owner or its model supplier changes. 1. Who owns the vendor today, and has it been acquired or announced a sale? Why: A change of owner changes terms, roadmap and support. The market moves page lists the recorded ones. 2. Does the product depend on one model provider, and what happens if that supplier cuts access? Why: Some tools route to several providers and some to one. Many vendors do not publish which. Source: https://cursor.com/docs/enterprise/privacy-and-data-governance 3. What can we export on exit: repository context, prompts, policy configuration, audit logs and usage history? Why: We found no public standard for exit and portability terms of AI coding tools, so the answer has to come from the vendor. 4. Is any part open source, so we can keep running it if the vendor stops? Why: Published source is a hedge against the vendor failing or being bought. 5. Which companies of our size use it in production, and will they talk to us? Why: Teams revert to manual process when they do not trust a tool, so references from peers carry weight. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ ## Pricing What it costs at your scale, beyond the seat. 1. Is pricing per seat, per usage, a subscription or a mix? Why: Seat price alone understates cost: total cost can include token overages, premium model tiers, agentic compute and governance infrastructure. Source: https://getdx.com/blog/ai-coding-assistant-pricing/ 2. What triggers overage charges, and is there a cap we can set? Why: The same analysis says overages can multiply the headline price for heavy agentic use. Source: https://getdx.com/blog/ai-coding-assistant-pricing/ 3. Which models sit in premium tiers, and who decides which one a developer uses? Why: Premium model tiering is a named cost driver. Source: https://getdx.com/blog/ai-coding-assistant-pricing/ 4. What does it cost us per accepted change at today's usage, before we expand? Why: The analysis recommends a measured baseline before rollout. We found no public benchmark for companies of 50 to 2,000 engineers that separates productivity gain from added review and governance cost. Source: https://getdx.com/blog/ai-coding-assistant-pricing/ ## Integration How it fits the tools you already run. 1. Which source control, CI/CD and issue tracker does it connect to, and how? Why: Evaluation advice is to prefer assistants that integrate with the repository and the other development tools. Source: https://thenewstack.io/ai-coding-assistants-12-dos-and-donts/ 2. Which other tools on this map does it pair with, and which does it replace? Why: The records name which tools integrate, ship inside one another or compete. 3. Can our policies decide who may use which feature, per team and repository? Why: Policy permission is on the list of things to test in a trial. Source: https://thenewstack.io/ai-coding-assistants-12-dos-and-donts/ 4. What integration commitments do you publish, and where? Why: We found no comparable published integration commitments across IDEs, source control, CI/CD, identity and scanners. ## Security Controls, assurance and what you can show an auditor. 1. Which compliance attestations does the vendor hold, and for which products? Why: Buyers cite the inability to prove output reliability to auditors as a reason not to buy. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ 2. Do you keep audit logs of prompts, outputs and accepted changes, and for how long? Why: Governance built for traditional software often does not map to AI workflows. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ 3. Do you answer a standard security questionnaire for AI-generated code? Why: One security vendor publishes a template covering scanning, fix accuracy, policy alignment and scale. It is a vendor's own, so treat it as a starting point. Source: https://snyk.io/lp/genai-coding-security-rfp/ 4. What copyright protection do you give us for generated code? Why: Copyright protection is on the list of things to evaluate. Source: https://thenewstack.io/ai-coding-assistants-12-dos-and-donts/ 5. Can we start on low-stakes work and widen the scope as we build trust? Why: The published rollout pattern starts with boilerplate, tests and documentation, keeps people accountable for generated code, then widens. Source: https://stackoverflow.blog/2026/02/18/closing-the-developer-ai-trust-gap/ ## Acceptance What the tool gives the person who must answer for the changes it produces. These come from the maintainer's thesis, and few vendors publish them. 1. Can a named person accept or refuse each AI-generated change, and can an administrator set who holds that authority? Why: Acceptance that cannot be refused is not acceptance, so the authority and the right to refuse come first. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance 2. What does the reviewer see before accepting: the change's intent, its footprint, what changed in the system meanwhile, the evidence and its gaps? Why: A person can only decide from a brief that is short enough to read and complete enough to trust. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance 3. Can we set a boundary below which changes are accepted by policy, so only consequential changes reach a person? Why: The number of decisions each person faces has to stay within what they can judge. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance 4. Does the tool record who accepted which change, on what evidence, and can we export that record? Why: A decision has to be reconstructable before anyone can answer for it. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance 5. After acceptance, can a person halt, roll back or disable an AI-generated change, and is doing so safe by design? Why: Authority without a way to stop and reverse is not control. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance 6. Does the tool show a person how the changes they accepted behaved in production? Why: People keep their judgment accurate only when they see how past decisions turned out. Source: https://aisdlclandscape.org/ownership#9-accountable-acceptance