Multi-tenant DDQ templates across customer accounts

A DDQ is two things at once. It’s a questionnaire with a structure — the 270 CAIQ questions arranged in their canonical domains, or the SIG Lite’s 120 questions in theirs. And it’s a response from a specific vendor, with tenant-specific facts — this vendor’s SOC 2 auditor, this vendor’s data-center regions, this vendor’s encryption key rotation policy.

The product problem: let every customer tenant reuse the questionnaire structure (the hard-won work of mapping 270 questions to 270 answer shapes) without letting tenant-specific facts from one account leak into another.

This post is how we split those two things apart.

The template vs. the answer

A PursuitAgent DDQ template carries three kinds of content:

1. The question structure — the ordered list of questions, their identifiers (CSA-IAM-02 or SIG-NET-15, for example), their domain classifications, their expected response type (free text, yes/no, evidence attachment).
2. The answer shape — a per-question specification of what a good answer contains. For a CAIQ question about encryption key rotation, the shape specifies: policy reference, rotation interval, key management system, compensating controls if any.
3. Answer pointers — references to KB blocks that, when filled with tenant facts, produce the answer.

The template is tenant-agnostic. The CAIQ v4 template for “IAM-02: How are privileged user accounts managed?” has an answer shape that asks for the named privileged-access-management tool, the approval workflow, the review cadence, and the evidence pointer. The shape doesn’t know or care which tenant will fill it.

The answer is tenant-specific. When tenant A fills the shape, the named tool is their PAM tool; the workflow is their workflow; the evidence pointer resolves to their SOC 2 report. Tenant B fills the same shape with different facts. The shape is shared; the content is isolated.

Where isolation lives in the stack

Every KB block in PursuitAgent carries a tenant ID. Every retrieval query filters by that tenant ID at the SQL layer before any vector search runs. The retrieval code path is:

select id, text, embedding <=> $1 as distance
from kb_blocks
where tenant_id = $2
  and deleted_at is null
order by embedding <=> $1
limit 20;

The tenant filter is a hard where clause on a non-nullable column. It sits ahead of the vector operator in the query plan and gets index-prefix treatment. A query without the filter doesn’t compile; our query builder refuses to emit a block-retrieval query without a tenant ID. The filter is not “we hope retrieval remembers” — it’s “retrieval cannot be constructed without it.”

Templates, by contrast, live in a shared table with a null tenant ID. They are readable by every tenant. Mutations to templates are gated behind a PursuitAgent-internal role; a customer cannot write to the template table.

The answer-binding step

When a tenant resolves a template into a response, the system walks each question’s answer shape and attempts to bind each slot to a KB block from the tenant’s namespace. A shape slot for “SOC 2 report reference” binds to whichever block in the tenant’s KB is tagged as the current SOC 2 report. A shape slot for “encryption-at-rest policy statement” binds to the tenant’s current version of that policy.

Binding is where the split matters most. The shape says “put a SOC 2 reference here.” Tenant A’s binding finds tenant A’s SOC 2. Tenant B’s binding finds tenant B’s SOC 2. Neither binding can reach across tenants, because the retrieval that resolves the binding filters by tenant ID before it does anything else.

If a tenant has no block that matches a shape slot, the binding fails closed. The question comes back to a human with “no KB block available” — never with a guess from another tenant’s corpus, never with a best-effort answer stitched from the model’s training data.

Two places this gets hard

Template versioning across tenants. CAIQ goes from v4.0 to v4.0.2. The template updates. Every tenant’s already-bound answers need a path forward: accept the new template and re-bind, or stay on the old template until the tenant explicitly upgrades. We default to “stay on the old template” for in-flight DDQs; upgrading mid-DDQ creates more confusion than it solves. Tenants can opt into auto-upgrade on completion for future DDQs.

Evidence attachments. An answer shape might say “attach the current SOC 2 PDF.” Each tenant has their own PDF. The binding attaches the tenant’s file; the evidence is never shared. The PDF itself lives in object storage under a tenant-scoped path (tenants/{tenant_id}/evidence/{id}), and the pre-signed URL we emit on attachment checks the requesting user’s tenant against the path. A request for a cross-tenant URL returns 404, not 403 — we don’t even acknowledge that another tenant’s file exists.

What this gets us

Three things worth naming:

1. New customers inherit the template corpus on day one. They don’t have to rediscover which CAIQ question maps to which answer shape; that work was done once and is shared. See the DDQ response playbook for how the intake side of this reuses the shared structure.
2. Template improvements compound across tenants. When an engineer improves an answer shape — adding a new slot for a compensating control, for example — every tenant benefits on the next DDQ without a migration.
3. Tenant facts stay in the tenant’s corpus. A leak between tenants is not a “we didn’t think of it” risk; it’s a class of bug that the retrieval code path is constructed to prevent.

What we haven’t built yet

Cross-industry templates. A fintech SaaS vendor and a healthcare SaaS vendor use overlapping but not identical DDQ sets; an insurance vendor’s DDQ overlaps both but with domain-specific additions. We have one canonical CAIQ v4 template and one canonical SIG Lite template; industry-specific variants are on the roadmap but not shipped. Customers in regulated industries currently layer custom questions onto the canonical template at the tenant level, which works but doesn’t benefit from cross-tenant compounding.

Related

• The DDQ response playbook, end to end — the canonical playbook.
• The evidence vault architecture — where SOC 2 PDFs actually live.
• Shipped: auto-attachment of evidence — the binding step, productized.