The DDQ evidence-attachment API

DDQs increasingly require evidence attachments alongside narrative answers. “Provide a copy of your most recent SOC 2 Type II report.” “Attach your data-protection impact assessment.” “Upload your business-continuity test results.” Buyers want documents, not just words.

This is a workflow we shipped earlier this year. The DDQ evidence-attachment API matches buyer-side evidence-request fields to documents in the customer’s KB evidence vault, surfaces the candidate matches in-app, and lets the user confirm or override before any document is sent. The human-in-the-loop step is the part of the design we are most opinionated about.

The schema

The evidence vault is a separate part of the KB. Documents live in:

export const evidenceDocs = pgTable("evidence_docs", {
  id: uuid("id").primaryKey(),
  companyId: uuid("company_id").notNull(),
  title: text("title").notNull(),
  category: text("category").notNull(), // "soc2-report", "pen-test", "policy", ...
  effectiveDate: date("effective_date"),
  expiresAt: date("expires_at"),
  storageRef: text("storage_ref").notNull(), // pointer to encrypted blob
  embedding: vector("embedding", { dimensions: 3072 }),
  approvedAt: timestamp("approved_at"),
  approvedByUserId: uuid("approved_by_user_id"),
});

Each document has a category, an effective date, and an expiration date. The category vocabulary is controlled — about 40 categories cover the common buyer requests. Documents that do not fit a category get tagged other and surface less reliably; we are working on the long tail separately.

Evidence requests on the buyer side come in via DDQ field metadata:

type EvidenceRequest = {
  questionId: string;
  requestedCategory: string | null; // sometimes inferred, sometimes explicit
  requestedAsOfDate: string | null; // "current" or a specific cutoff
  requestText: string;
};

The requestText is the buyer’s natural-language description of what they want. The requestedCategory is sometimes set explicitly by the questionnaire format (some structured DDQ formats use controlled vocabularies); more often it has to be inferred from the request text.

The matching logic

The matching logic runs in three passes.

Pass 1 — category match. If the buyer’s request has an explicit category, look up evidence documents in that category. If the category is inferred from the request text, the inference uses a small classifier trained on labeled DDQ field examples; the classifier outputs a category and a confidence score. Confidence below 0.7 falls through to pass 2.

Pass 2 — semantic match. Embed the request text and search the evidence-vault embeddings for nearest neighbors. The top three candidates surface for review. This catches cases where the buyer’s request does not map cleanly to the controlled category vocabulary — e.g., a request for “documentation of your secure development lifecycle” does not have a category named exactly that, but the closest document is the secure-SDLC policy.

Pass 3 — recency filter. Among candidates from passes 1 and 2, filter to documents whose effective date is current (or within the buyer’s requestedAsOfDate if specified) and whose expiration date is in the future. Expired documents are surfaced separately with a “this document expired on [date]” warning rather than silently excluded — sometimes a buyer accepts an expired document with an updated cover letter, and the user needs to make that call.

The output is a ranked list of candidate documents per evidence request, with metadata: category match, semantic similarity, recency status, and approval status.

The human-in-the-loop step

The candidate list surfaces in the UI as a per-question evidence panel. The user reviews each candidate, accepts the top match (which is the default), or selects a different document. Nothing is attached to the response without explicit confirmation.

We have been asked to ship an “auto-attach” mode that bypasses confirmation when the top match is high-confidence. We will not. The reasons mirror our broader stance on grounded AI:

Buyer-side evidence requirements are fact-bearing. A wrong document attached to a DDQ is worse than no document — it is a substantive misrepresentation that can affect contract terms, compliance posture, and downstream audit risk. The cost of a wrong attachment is meaningfully higher than the cost of a manual confirmation step.

Document context matters in ways the matcher cannot see. A SOC 2 report from one subsidiary is not interchangeable with a SOC 2 report from another. A policy that is current at the parent company may not apply to the subsidiary the contract is being signed under. The user has context about the relationship between the customer and the buyer that the matcher does not.

The confirmation step is fast. In the typical workflow, confirmation takes two to four seconds per evidence request. A 200-question DDQ with 30 evidence-bearing fields requires roughly two minutes of confirmation time across the response. That is a small tax against meaningful protection.

Arphie’s analysis of security questionnaire automation names the same trade-off — automation accelerates the work, but the time lag between document update and questionnaire submission means responses can be outdated before they are even reviewed. The human confirmation step is what catches that.

The expiration and refresh workflow

Evidence documents expire. SOC 2 reports cover specific audit periods; pen-test results cover specific test windows; policies are revised on annual or biannual cycles. The vault tracks expiration dates and surfaces upcoming expirations in two ways.

On the response side, evidence requests that match an expiring document trigger a warning at attachment time. The user sees “this SOC 2 report expires in 14 days; you may want to attach the most recent version if available” before confirming the attachment. If a newer version is in the vault, it is suggested as an alternative.

On the vault side, a weekly digest surfaces all documents expiring in the next 60 days, grouped by category and assigned to the document owner. Owners get a Slack notification with a link to the document and a one-click action to mark it “in renewal” or “no replacement coming.” The replacement workflow is where most of the operational discipline lives — without it, the vault becomes a graveyard of expired documents that the matcher confidently surfaces and the user confidently attaches.

Safe Security’s research on vendor questionnaires names this exact failure pattern — vendors recycling outdated answers, accumulating “yes” responses that do not reflect real security posture. The expiration tracking is the cheapest defense we know how to build.

What is in scope and what is not

The API today handles the most common evidence categories (SOC 2, ISO 27001 certificates, pen-test summaries, business-continuity test results, data-protection impact assessments, security policies, encryption attestations, and roughly 30 others). It does not handle:

• Bespoke buyer-defined evidence formats. Some buyers ask for custom artifact formats — a specific spreadsheet template, an internal questionnaire response. These do not match the vault’s category vocabulary; the user fills them in manually.
• Live evidence generation. Some buyers ask for runtime artifacts — a current backup log, a current uptime dashboard screenshot. The vault is for static, periodically-refreshed documents; live evidence is a separate workflow.
• Cross-vault dependency tracking. A SOC 2 report that references a specific data-flow diagram from a different vault entry does not get its dependency tracked automatically. We surface dangling references in red-team review but the cross-vault graph is on the backlog.

For the broader DDQ-response discipline, see our DDQ playbook. The evidence-attachment API is one part of that playbook; the rest is upstream work in the KB and downstream work in the review cadence.