← The Playbook

Shoals · Shoal-level skill

Data Request Fulfillment

A judgment-led, two-phase workflow for fulfilling customer dataset requests — AI-driven search and generation, with adversarial loops on both data and documentation before secure airlock publish.

datahealthcareanalyst-workflow

Data Request Fulfillment is the workflow a Secure Data Analyst follows when a customer requests a dataset. Requests arrive by email, CRM submission, or manually transcribed from a phone call. All activity — from first contact to final handoff — is recorded in the CRM as the system of record for both internal and external customer interactions.

This is a two-phase process. Phase 1 establishes whether the request is feasible and what data is available before any contract is signed. Phase 2 produces, validates, and delivers the final dataset once the customer confirms they want to proceed. AI carries the search, generation, and review weight throughout. The analyst provides the oversight that makes the output trustworthy.

Workflow

  1. Request intake. The request lands in the CRM — via direct submission, email intake, or a manual entry from a phone conversation. At this point, the CRM automatically triggers the AI Feasibility Skill as a standalone agent process. The analyst does not initiate this manually; it is a workflow-level automation.

  2. AI Feasibility Skill. The skill runs autonomously against the data estate and produces a structured Markdown output file, stored at:

    projects/<customername>-<customerid>/<RequestID>/feasibility.md

    The output contains:

    • Query observations — what the AI understood from the original request and any ambiguities it identified
    • Assumptions — the constraints and interpretations applied during search
    • Recommendations — how to proceed, what is available, and what is not
    • Underlying datasets — ordered by medallion tier: Gold first, then Silver, then Bronze. Each dataset entry includes the schema subset that matches the request.
    • SQL queries — starting-point queries likely to produce the required outputs, ready for analyst review and refinement
    • Cohort metrics — a high-level summary of volume and distribution to support the originating query

    Token usage is tracked across all AI agent activity at this stage. This data accumulates over time and forms the cost baseline for future automation decisions.

  3. CRM update. Once the skill completes, the CRM workflow advances automatically. A clean rich-text summary of the feasibility output is posted to the ticket, and the request status moves to Human Agent Review.

  4. Human agent review. The analyst reviews the AI output. This is a judgment step — not a sign-off formality. If the feasibility output is acceptable, the agent confirms and the summary is returned to the customer for review and next steps. If the output requires correction, the agent ticks the Human Override checkbox and records a comment explaining what was wrong. This feedback is routed back as a learning signal for the AI agent.

  5. Customer decision gate. The customer reviews the feasibility summary and decides whether to proceed to contract. No production work begins until confirmation is received.

    Future capability — customer-led automation

    As the organisation builds confidence in AI agent performance — measured against both token cost and success rate against a defined tolerance — the CRM workflow supports a Customer-Led AI Full Automation parameter. When enabled for registered customers, the feasibility study is delivered directly from their request portal without entering the human review stage. Phase 1 becomes fully automated for trusted request types.

Human Maturity

SFIA level 4 minimum. At SFIA 4, the analyst can execute both phases with autonomy, assess the AI feasibility output with genuine judgment, and manage the adversarial refinement loop without guidance. Below SFIA 4, the analyst can participate in the workflow but should not own the feasibility sign-off or the loop exit decision unsupervised.

The limiting factor in this workflow is not technical skill. It is the quality of judgment the analyst brings to the review steps. An analyst who treats the human override gate as an administrative formality, or who exits the code refinement loop before the specification is genuinely met, will produce outputs that pass the process but fail the customer. The workflow is designed to surface those failures — but only if the analyst engages each gate as a real decision.

Model Maturity

L3 (semi-autonomous search, generation, and governance review with analyst oversight). The AI handles the feasibility search, first-pass code generation, adversarial code review, and information governance check. The analyst drives intake framing, specification confirmation, loop exit decisions, and final documentation sign-off.

At higher model maturity levels, Phase 1 becomes more autonomous and the customer-led automation parameter becomes viable at a broader class of requests. At L4, the AI could own the Phase 1 output end-to-end for well-defined request types, with the analyst reviewing rather than guiding. The governance review and CRM workflow boundaries remain analyst-owned regardless of model maturity; those boundaries are architectural, not maturity-dependent.

Benefits

Three gains compound over time when this workflow operates consistently.

Throughput. A feasibility assessment that previously required days of manual data exploration can be completed in hours. The AI search covers the data estate faster and more systematically than manual querying, and the structured loop compresses the refinement cycle. More requests reach a decision point sooner.
Accelerator effect. For both Phase 1 and Phase 2, the AI agent produces a structured first draft — a feasibility output and an extraction code base — that the analyst works from rather than starts from scratch. Each activity has a generated accelerator, which compresses the time-to-quality curve and allows the human to focus on judgment rather than origination.
Auditability. The output bundle — dataset, data dictionary, summary document, IG review — is a by-product of how the workflow is structured, not an afterthought. Every delivered dataset arrives with a complete reproduction record. In regulated data environments, that is not marginal — it is the difference between a dataset that can be defended and one that cannot.
Trust compounding. Customers who receive consistent, clean, well-documented output bundles become repeat requesters. The adversarial loop and IG review catch errors before they reach the customer. Over time, a team operating this workflow builds a reputation for deliverables that hold up — which changes the nature of the commercial relationship.

Risks

Three risks are worth naming plainly.

Gate compression. The workflow has four explicit gates: feasibility confirmation, human override review, adversarial loop exit, and IG sign-off. Under time pressure, analysts compress gates into formalities. A human override checkbox ticked in thirty seconds is not a review — it is a liability transfer. Each gate exists because a specific failure mode was identified at that point.
Specification drift. The adversarial loop can only test against the specification it was given. If the specification changes during production — because the customer clarified their request or the analyst reinterpreted scope — the loop is testing against the wrong standard. Any change to the request specification after feasibility confirmation must restart Phase 1, not continue into production.
Automation over-trust. The customer-led automation parameter is a maturity milestone, not a cost-cutting shortcut. Enabling full automation before the success rate meets the defined tolerance means removing the human review gate while the AI output is still unreliable. The tolerance threshold must be set deliberately, based on real performance data — not aspirationally.

Mitigations

Gate compression — make the gate criteria explicit and written before the phase begins. The human override checkbox should reference a written checklist, not an instinct. The loop exit should reference the specification agreed at Phase 1. Written criteria survive time pressure; instincts do not.
Specification drift — treat any post-confirmation scope change as a new request. It restarts Phase 1, not Phase 2. This is not a policy preference; it is a correctness requirement. A change that appears minor at the specification level can materially alter the dataset and invalidate the IG review.
Automation over-trust — use the token cost and success rate data that accumulates from Phase 1 activity to set the automation threshold deliberately. The transition to customer-led automation should be a measured governance decision made with real evidence, reviewed periodically as the AI agent's capability evolves.

Business Area Impact

When Data Request Fulfillment operates at scale, the workload distribution shifts in two directions.

Request capacity increases without proportional headcount increase. A single analyst working this workflow can handle more concurrent requests — the AI feasibility search, code generation, and governance review steps run in parallel to analyst review rather than replacing them sequentially. This changes the unit economics of data-as-a-service offerings.

The roles most directly affected are data coordinators and junior analysts whose primary function was manual data extraction and formatting. These functions do not disappear — but their task profile narrows toward specification quality, criteria definition, and exception handling, where domain knowledge has the highest leverage. The organisations that handle this well invest those individuals in the phases where human judgment is irreplaceable.

Handoff

Data Request Fulfillment produces a structured output bundle: dataset, data dictionary, summary document, and IG review record. These are delivered as a unit via the CRM. A dataset without its documentation is not a completed fulfillment — it is an unfinished one.

The handoff point is the CRM notification to the customer. At that moment, analyst ownership ends and the receiving system handles customer access provisioning, secure storage, and any further processing appropriate to the access tier.