StudioAI agents for data onboarding
Bringing a new client's data into the platform took engineers about three weeks of manual work. AI agents could do most of it. The design problem was making that work something people could check, overrule and undo.
Result
3 wks → 2–3 days
to onboard a new client's data
- RoleDesign Lead
- Timeline10 weeks
- WhenNov 2025 – Jan 2026
- Team5 disciplines
- UsersData & security engineers, platform admins
- PlatformWeb app · internal + admin
DisagreementsThe agents disagree about where “Owner” belongs. Waiting for you.
Does “Owner” mean a person or a team?
What will change
Resolve 1 disagreement first- Added
- +47
- Changed
- 12
- Connected
- —
- Removed
- 0
Working recreation. Press L for the decision behind each part.Open “Design notes” in the footer for the decision behind each part.
NDA Interactive recreation with sample data. These are not production screens or customer data.
In 30 seconds
- 1
The problem
Bringing a new client's data into the platform took engineers about three weeks of manual work. When a source tool changed, things broke without anyone noticing.
- 2
What I did
I designed how people supervise AI agents doing that work: how sure each agent is, what happens when agents disagree, a preview before anything changes, and one-click undo. A person always makes the final call.
- 3
The outcome
Onboarding went from about 3 weeks to 2–3 days, with 25+ data sources supported at launch and 4 hidden breakages caught in the first month.
Jump toKey decisions Results
01The problem
Every product depends on this data
Prevalent's products all read from one shared map of a company's security data. Every tool a client uses (Qualys, Okta, CrowdStrike, ServiceNow) describes things in its own way, and each one has to be translated into Prevalent's model before any product can use it.
The ML team was already building agents that could do that translation. What nobody had designed was how people would supervise them. I owned the experience end to end: research with engineers, how people work alongside the agents, what happens when agents disagree, and how changes are previewed, approved and undone.
I worked out the agents' roles with the ML team before drawing any screens, so the interface only promised what the agents could really explain.
Constraints
- Everyone shares the data
- One onboarding can change thousands of records that every other product relies on.
- AI isn't always sure
- The same input can give different answers. The design had to be honest about uncertainty, not hide it.
- Two audiences, one tool
- Data engineers want every detail; security engineers want a decision they can make in seconds.
- Client privacy
- What the agents learn from one client can't be used for another.
Onboarding took weeks, broke silently, and only engineers could do it
Every new data source meant an engineer reading through it, writing the translation by hand and checking it several times. Onboarding a client took about three weeks. Security teams waited in a queue for data they needed now.
Worse, when a source tool changed how it sent data, things broke without warning. The damage showed up weeks later as a dashboard that looked wrong, and undoing it meant an engineering fix.
Agents could make the work fast. But fast, unchecked changes to data every product relies on are a bigger risk than slow ones. The brief had to change.
The briefAutomate the onboarding work.
The brief, rewrittenMake the agents' work easy to check, so people can trust it at the speed it runs.
02Research
Checking took longer than doing
Research ran alongside the ML team's early agents, so I could design around what the agents could actually explain, not what we wished they could.
How
- Shadowing engineers
- Sat with data engineers while they onboarded real data sources, noting where the time actually went.
- Reviewing past breakages
- Went through old tickets to see how things broke and how late people found out.
- Sessions with the ML team
- Worked out what each agent could show people: how sure it is, why it decided, what else it considered.
- Concept tests
- Tested versions of the disagreement card with engineers, using real cases from past onboardings.
What we learned
- 01
Engineers spent their time checking
Engineers spent more time checking the work than doing it. A faster agent alone wouldn't save the time.
Which led to Confidence on everything the agents do, and a preview before anything changes.
- 02
Silent failures cost the most
The expensive breakages were the ones nobody noticed until a dashboard looked wrong weeks later.
Which led to Ongoing checks that flag when a source tool changes, long after onboarding.
- 03
One big check was too much
Checking everything from an onboarding in one sitting was heavy, and it was the part most likely to be rushed.
Which led to Review split into smaller sessions: data sources, onboarding and versions.
03Approach
Principles for this product
- Show the work
- If an agent can't say what it did and how sure it was, its work doesn't go through.
- Disagreement is useful
- When agents disagree, it tells you something about the data. It isn't an error to hide.
- Everything can be undone
- Speed is only safe when every change can be previewed before and undone after.
I designed how the agents work together before the screens
My first artefact was a diagram of the agents, before any wireframe: what each one does, what it hands on, and what it can explain. Product and ML agreed it with me.
Once that was clear, the interface got simpler: each screen shows one step of the work, and each button maps to something real a person can do: check, choose, approve or undo.
Tried and set aside
- One "AI is working" status
- Simple to show, but impossible to tell which part to trust.
- Pass / fail checks
- Hides the difference between something the agent is 65% sure of and 97% sure of.
- Reports after the fact
- Telling people what changed after it had already changed.
Sketches
Redrawn for this portfolio. The original work is under NDA.

When agents disagree, both options get equal weight and nothing is pre-selected. Approve stays locked until a person decides.
04Decisions
Decision 01Agents do the work, people decideWhat I chose Agents do the preparation. The final step, making the change, is always a person's.
- What I turned down
- Letting agents make changes on their own when they're confident enough.
- Why
- Every product relies on this data. One bad automatic change spreads everywhere before anyone sees it. A person approving each change is the simplest place to catch it.
- What it cost
- A person in every step caps the speed. Onboarding takes days rather than hours.
Decision 02Always show how sure the agent isWhat I chose Everything an agent does shows how confident it is. Low-confidence items are checked one by one; the rest are approved together.
- What I turned down
- A simple pass or fail.
- Why
- 65% sure isn't the same as 97% sure. Showing it points people at what actually needs attention; pass / fail gives false comfort.
- What it cost
- People can over-read numbers, so the cut-off line, not the number itself, decides what needs a closer look.
Decision 03When agents disagree, a person choosesWhat I chose Disagreements appear as a card showing both options, each agent's reasoning and what would change. Nothing is pre-selected.
- What I turned down
- Quietly picking the more confident option.
- Why
- Disagreement means the data really is unclear. Hiding it behind a best guess just moves the mistake somewhere harder to find.
- What it cost
- Every disagreement takes a person's attention.
Decision 04Preview before anything changesWhat I chose Before approving, people see exactly what will be added, changed and connected, and every change can be undone in one step.
- What I turned down
- A report after the change is made.
- Why
- Every product uses this data, so people need to see the impact before they approve, not after.
- What it cost
- One more step in every review, in exchange for never finding a bad change later.
Agents do the work; people keep the final say
- User needs & defining success
Where the line sits is a design decision
The confidence cut-off decides how many items people check one by one and how many they approve together. It's a trade between care and effort.
- Mental models
Review follows how the work is really done
Engineers already think in data sources, onboarding and versions. Review is split the same way, so each check is a focused task.
- Explainability & trust
Every decision comes with its reasoning
Everything shows how sure the agent was. Every disagreement shows both options and why. Every record shows where it came from and who approved it.
- Feedback & control
Approve, overrule, undo
People approve every change, choose when agents disagree, set the confidence cut-off, and undo any change with a preview of what will be reversed.
- Errors & graceful failure
Problems are shown, not hidden
Disagreements become a card instead of a silent guess, and changes in a source tool are flagged before they break anything.
- Data & evaluation
Designed around what the agents can really explain
Sessions with the ML team mapped what each agent could actually show before any screen promised it.
Grouped by the themes of Google’s People + AI Guidebook.
Deliberately left out
- Agents acting on their own
- The agents can do the work, but they don't make the final change. No automation without human approval is a product rule, not a launch compromise.
- Learning across clients
- Easy to build, hard to do responsibly. Held back for client privacy.
- Free-text instructions to agents
- Powerful but unpredictable. Replaced with structured choices.
Where data came from, and undo
| Record | Came from | Prepared by | How sure | Approved |
|---|---|---|---|---|
| DEVICE Payments gateway | Qualys | Clean-up | 97% | ○ Approved together |
| RISK Remote access flaw | Tenable | Clean-up | 91% | ○ Approved together |
| POLICY Patch within 30 days | ServiceNow | Matching | 74% | ◆ Checked one by one |
| ACCOUNT Database service account | Okta | Matching | 88% | ○ Approved together |
| DEVICE Customer database | CrowdStrike | Matching | 83% | ○ Approved together |
Every record shows where it came from, which agent prepared it, how sure it was and who approved it. Any change can be undone.
05Testing
One big check was too much, so review was split up
Because every change needs a person, the review itself became the bottleneck. The iteration was about making that human step manageable, not removing it.
How review is structured
- First version
One complete check of everything an onboarding produced.
- What testing showed
Checking everything at once was too much to do well in one sitting.
- What shipped
Review split into smaller sessions (data sources, onboarding and versions), each a focused check that ends with a person approving.


Drag to compare. Sketches redrawn for this portfolio; the original work is under NDA. - First version
06Results
From a three-week queue to a few days
- 3 wks → 2–3 daysClient onboarding
Time to bring a new client's data into the platform, down from about three weeks.
- 25+Data sources at launch
Supported through the same agent flow from day one.
- 4Hidden breakages caught
Changes in source tools caught in the first month that would have broken data silently before.
What I’d measure next
- How often people overrule agents
- Shows where the agents need to get better.
- Quality of review
- Whether people are really checking, or just clicking approve.
Keeping people in the loop is a design problem
- Design the system before the screens
- Agreeing how the agents work with ML first made the interface simpler.
- Design the human step, don't just require it
- "No automation without approval" is easy to write as a rule. Making it work meant splitting review into sessions people could actually finish.
- Measure review from day one
- I'd track how people review from launch, so the confidence cut-off is set with data, not guesswork.
- Next: fix breakages in one step
- When a source tool changes, suggest the fix and show its impact together, so it's one review instead of a ticket.