The code scan comes back green, the open source licenses look clean, and the security report has no headline problem. That still does not tell you whether the AI product works. AI due diligence adds a second layer: eval quality, data rights, model dependency, and wrapper depth.
Traditional software diligence asks whether the machine is well built. AI diligence asks whether the machine makes good decisions under messy conditions. That difference matters because an AI product can have a tidy repository and still give wrong answers, use data it cannot prove it owns, or vanish as a paid feature when a model provider ships the same thing.
Sema says its codebase scan dataset is based on $1T of enterprise software organizations analyzed.
That is not a criticism of code scans. The standard software checklist still matters. If the code is full of known vulnerabilities, license problems, and unmaintainable modules, the AI layer will not save the investment. The problem is category fit. A metal detector is useful at an airport. It does not test whether the pilot can land in fog.
Two free tools put part of this into practice: wrapper-test scores whether an AI product is a thin wrapper, and model-rot flags a repo that depends on an LLM model already scheduled for retirement. Both run locally, with no API keys.
Software diligence inspects the repository
Software diligence is a repository inspection first. It checks whether the code can be maintained, whether third-party components create legal risk, whether security practices are reasonable, and whether the engineering process can survive after a financing or acquisition.
Sema’s codebase scan page lists code quality, GenAI code use, intellectual property risk, code and cyber security, process quality, team signals, and cloud cost as areas it can assess. That is a broad and useful map of ordinary software risk. It tells you whether the kitchen wiring is dangerous, whether the lease is clean, and whether the cookbooks belong to someone else.
But AI product risk often sits outside that map. The repository may show API calls, prompt templates, and a queue processor. It will not show whether the answers are right on real customer tasks unless the team has built measurement around those tasks. It will not prove data rights unless contracts and consent records exist outside the codebase.
AI diligence inspects behavior
AI diligence starts with behavior. The core question is not “is the code clean?” but “does the system reliably do the valuable thing customers pay for, and can the company prove it without a staged demo?”
OpenAI’s evaluation best-practices guide says evals are structured tests for accuracy, performance, and reliability despite the variable nature of generative AI. That one sentence explains why a normal test suite is not enough. Traditional tests ask whether a function returns the same answer. AI evals ask whether a messy answer is good enough across many real cases.
A software scan can confirm that a billing service has tests. An AI diligence review asks whether the claims assistant was graded on denied claims from March, renewal emails from April, and the weird handwritten PDF one broker uploads every Tuesday. That is the difference between checking a scale and tasting the soup.
Gap 1: eval quality is invisible to code scans
Eval quality is invisible unless the startup keeps a real scoreboard. A code scan may find a folder named evals, but it cannot tell whether those tests represent customer work, whether humans calibrated the grader, or whether the team tracks regressions before release.
The good evidence is boring: test cases from production-like inputs, expected answers, pass rates over time, failure categories, and release gates. The bad evidence is a public benchmark number copied from a model card. That benchmark may be useful for comparing models in isolation. It does not tell you whether the startup’s contract-review product catches the clause customers sue over.
Ask to see the last three failures. Strong teams can show them. Weak teams say the product is improving and offer another demo. In software diligence, a failing unit test is an engineering issue. In AI diligence, no failing examples at all may be the larger problem.
Gap 2: data provenance lives in contracts
Data provenance means the paper trail for the data used to train, fine-tune, test, or personalize the AI system. The code repository rarely contains that paper trail, because the real evidence is in customer agreements, data processing terms, vendor contracts, upload logs, and access controls.
Ask where the data came from and what the startup is allowed to do with it. If the answer is “public data,” slow down. Publicly visible is not the same as permitted for training, resale, fine-tuning, or customer-specific automation. A menu in a restaurant window is public. That does not give you the right to copy the chef’s supplier book.
This is where standard software diligence can produce false comfort. The dependency licenses might be perfect while the AI training data is undocumented. Later investors will ask the same question. So will acquirers. So might customers in regulated industries.
Gap 3: model dependency is a hidden landlord
Model dependency turns the provider into a hidden landlord. If the startup depends on one model, one API policy, one price sheet, and one safety filter, then changes by that provider can alter margins and product behavior without a line of startup code changing.
This is not theoretical. Model providers publish deprecation schedules and replacement recommendations as normal platform hygiene. For a startup without evals and fallback paths, that kind of notice is a sudden renovation notice taped to the shop door.
AI diligence asks whether the team has tested a second model, not whether it prefers one. The good answer is, “Model B is 6 points worse on extraction but cheaper on classification, and the swap takes a week.” The bad answer is, “The current model is the best.” Best today is not a plan.
Gap 4: wrapper depth changes the investment case
Wrapper depth asks whether the product is a thin interface over someone else’s model or a workflow customers would miss if the model provider copied the surface feature. This is a business-quality question with technical evidence behind it.
A shallow wrapper has prompts, a chat box, and maybe a dashboard. A deeper product has customer-specific context, integrations into daily tools, permissioning, audit trails, evals, and data that improves through use. The first is a label on a jar. The second is the pantry, recipe book, delivery route, and weekly grocery list.
Standard software diligence can inspect both codebases and see healthy repositories. AI diligence asks what survives if the provider ships the headline capability. That question belongs beside valuation, not in a footnote.
Angels are left with network diligence
Angel investors face the worst version of this gap because the classic tooling is priced and scoped for larger checks. The public advice often sends angels back to their network, advisors, customer calls, and investor group contacts instead of an affordable AI-specific review.
The Angel Capital Association’s due diligence guide tells angels to use their networks for competitor information and notes that one New York Angels process expects around 30 conversations. That is useful human diligence. It is not an eval review.
So the practical answer is layered. For small checks, run a strict founder script yourself: ask for evals, data rights, model fallback, per-query cost, security boundaries, and what survives a provider feature launch. For larger checks, bring one engineer for a day. Do not buy a full acquisition audit for a small angel check. Do not wire a serious seed check after only watching a demo.
The replacement playbook
The better playbook keeps software diligence and adds AI diligence after it. First, scan the repository for ownership, security, quality, dependency, and process risk. Second, inspect the AI evidence: evals, data provenance, model dependency, cost per unit, and wrapper depth. The deeper version lives in the technical due diligence checklist for AI startups.
Ask for artifacts, not opinions. Eval dashboard. Data source list. Model fallback notes. Last incident write-up. Prompt injection test. Customer workflow map. Unit cost estimate. These are the drawer handles you can pull. A founder who has built the product for real will have most of them within reach. If the product touches sensitive customer data, add the questions in what to ask an AI vendor about security.
The result is not a perfect prediction. No diligence process gives that. It is a filter. Software diligence tells you whether the code can be owned and maintained. AI diligence tells you whether the behavior can be trusted and defended. In AI investing, you need both, because clean code around unmeasured behavior is just a neat box with smoke inside.