From a Code Assistant to Robotic Process Optimization β€” the same AI that drives your browser can observe, remember, and optimize how work really flows
FROM DIGITAL TWIN TO PHYGITAL TWIN

From a Digital Twin to a Phygital Twin.

How to codify process knowledge with a Code Assistant into Agentic Robotic Process Optimization — observe the real work in the browser, reason about it on the spot, then replay, simulate, and optimize it.

A digital twin models a system from clean, structured data. But most real work doesn't live in clean data — it lives in how people move across real apps, screens, and spreadsheets, full of judgment no system ever logged. A phygital twin is stitched from exactly that: a Code Assistant quietly observes the actual work, reasons about what's on the screen, remembers every decision in one place, and turns it into something you can replay, simulate, and optimize.

πŸ‘€ Observe  real work across apps 🧠 Remember  every decision in one memory βš™οΈ Optimize  the whole process, not one task
πŸŽ₯

The way I picture it: a flight recorder for knowledge work. Each observer notes what someone did, in which app, what the screen was showing, and why. Stitch those recordings together and you finally get to see the whole decision journey β€” and once you can measure it, you can improve it like any other system.

First, where this fits

Before you can optimize a process, you have to discover it.

If you've spent any time around enterprise automation, you already know the big names β€” UiPath, Automation Anywhere, Blue Prism on the RPA side, and Celonis on the process-mining side. They're genuinely powerful, and I'm not here to knock them.

But here's the thing I keep running into. RPA is brilliant once you already know the steps β€” you hand it a defined procedure and it runs it tirelessly. Process mining is brilliant once your work already lives in clean system logs β€” it reads an ERP's event table and reconstructs the flow. Both of them quietly assume the hard part is already done: that somebody, somewhere, has written the process down.

In most companies, nobody has. The real workflow doesn't live in one tidy system β€” it's scattered across a dozen web apps, a pile of spreadsheets, a thread of emails, and a ton of tribal knowledge that only exists in people's heads. The analyst who knows to re-check margin before pricing goes out. The planner who always pings product when a supplier slips. None of that is documented anywhere. There is no process map β€” and you can't mine logs for a process that was never logged in the first place.

So before optimization, simulation, or any of the fun stuff, there's an unglamorous first step almost everyone skips: discovery. You have to surface what the process actually is β€” bottom-up, from how work really happens β€” instead of pretending a top-down diagram already exists.

πŸ”Discoverβ†’ βš™οΈOptimizeΒ· you can't optimize what you can't see

That discovery step is exactly where the big platforms get expensive and heavy β€” long deployments, connectors, six-figure licenses. And that's the gap I want to talk about. Because the browser the whole company already works in turns out to be the perfect place to watch real work unfold — which means we can now build a much lower-cost, custom alternative that discovers the process bottom-up, then opens the door to optimizing it. That's what the rest of this post is about.

How I think about it

We already automate tasks. What I really want is to mine the process.

Driving Chrome through the DevTools Protocol β€” CDP, the browser-control interface, not a customer data platform β€” to click and type is old news at this point. What I think we've been missing is a cheap way to turn that exact same capability into real process intelligence. It's a small change to build β€” but it completely changes what the AI is for.

πŸ’»
The part everyone already accepts

AI + Chrome DevTools Protocol automates a task

One assistant hops into a live, signed-in browser and finishes a single job, start to finish.

  • Opens the same screens and forms a person uses
  • Types values, clicks buttons, submits the work
  • All about getting one task β€œdone”
Familiar today
πŸ”­
The low-cost extension I'm excited about

AI + Chrome DevTools Protocol mines the process

A bunch of assistants quietly watch instead. Every time someone makes a real decision, the assistant reads the screen, reasons about what it means, and drops a record into one shared memory — not just the click, but the context and the why.

  • Logs three layers: the event, the on-screen context, and a short reason
  • Stitches those records into a cross-persona journey
  • All about making the whole process better
New: Agentic Robotic Process Optimization
πŸ‘€Observe+ 🧠Remember= a system you can replay Β· simulate Β· optimize

How it works

How it actually works, start to finish

These five steps work for basically any process. To keep it concrete, I'll run one example all the way through β€” a company deciding whether to launch a new product.

1 Β· Event

What happened — the action taken and the decision made, in a clean record.

2 Β· Context

What the UI/UX was actually showing at that moment — the numbers, flags, and screen the person was looking at.

3 Β· Reasoning

A short why, inferred on the spot — the assistant's read of what the context meant for the decision.

Why this isn't classic RPA: traditional RPA and process mining log only step 1 — the bare event. The difference here is steps 2 and 3. Because the observer can read the screen and reason in the moment, the memory holds event + context + reasoning, not just a trail of clicks. That's what makes the captured process actually understandable later.
1
πŸ‘οΈ

Observe & read the screen

In plain terms: the observers ride along inside the apps people already use — reading the screen, the data on it, and what the person is doing, not just logging clicks.

Our exampleAn analyst reviews a demand chart; the observer sees the +18% trend on screen.

➜
2
🧾

Capture event, context & reasoning

In plain terms: each meaningful action becomes one record with three layers — the event, the on-screen context it happened in, and a short reason the assistant infers on the spot.

Our exampleβ€œRecommended $199 — rival at $179, but the 35% margin target is met at higher volume.”

➜
3
πŸ—„οΈ

Store it centrally

In plain terms: every observer streams into one shared memory, all timestamped and linked.

Our exampleMarket, product, pricing, supply and finance events land together.

➜
4
🧡

Re-sequence the journey

In plain terms: the events get reconnected by the thing they're about, and the real branching path shows up.

Our exampleOne launch decision, stitched across four core roles over nine business days.

➜
5
βš™οΈ

Simulate & optimize

In plain terms: replay what happened, try what-ifs, and point out where to take friction out.

Our exampleβ€œSupply review is your most common stall β€” start it earlier.”

Here's the part I like: you don't reorganize anything up front. The browser just remembers what already happens, and that recorded past turns into a simulator for a better future.

The architecture, illustrated

A few AI observers, one shared decision memory

Nothing gets ripped out or replaced β€” the observers just ride on top of the apps people already use. Once all the events live in one place, three things open up: replay, simulation, and optimization.

AI OBSERVERS SHARED DECISION MEMORY INTELLIGENCE ENGINES πŸ”­ Market AnalystBI Β· Research Portal πŸ”­ Product ManagerPLM Β· Requirements πŸ”­ Commercial AnalystPricing Β· Fin. Model πŸ”­ Supply & FinanceERP Β· Budget Β· Approval 🧠 Central Decision Memory One timestamped event stream Linked by user, app & business object Re-sequenced into decision journeys πŸ” Replaywhat happened πŸ§ͺ Simulatewhat-if paths βš™οΈ Optimizeremove friction

Four observers on the left, one shared memory in the middle, three intelligence engines on the right.

The real journey Β· interactive

Four roles, lots of steps, even more events

Real decisions branch, run in parallel, and loop back β€” they're almost never a clean straight line. Each persona gets a lane with a few real steps, and every step becomes its own event. Tap or hover any step to see the role, the app, and the event it drops into memory.

πŸ”Ž Market AnalystBI Β· Research Portal πŸ“¦ Product ManagerPLM Β· Requirements πŸ’² Commercial AnalystPricing Β· Fin. Model 🏭 Supply & FinanceERP Β· Budget Β· Approval Scan signalsResearch Portal Analyze demandBI Dashboard Compare rivalsExcel strong? no β†’ re-research yes β†’ handoff Draft conceptPLM Prioritize reqsRequirements Validate scopeCollaboration ready? no β†’ refine concept yes β†’ handoff Estimate demandFinancial Model Model pricingPricing Platform Estimate marginExcel margin? no β†’ re-model price yes β†’ handoff Check capacityERP Validate ROIBudget System ConsolidateApproval Flow approve? βœ“ Launch Approvedjourney complete no β†’ defer / re-scope
Step (role + app)
Decision branch
Margin gate
Rework loop
Outcome

Tap a step to inspect it

Each step names the role, the web app, the activity, and the decision event written to memory.

Each lane is one persona; the multiple steps inside a lane are exactly the events RPM captures within that role and app.

A complete example, told like a story

One launch decision, across multiple roles over multiple days

Here's that same launch decision in plain language β€” and the event each moment quietly writes to memory. Nobody changed how they work; the observers just remembered.

πŸ”Ž

Market Analyst β€” Mon, Mar 3 Β· 9:05 AM

β€œThis segment is finally worth a look.”

She digs into category growth, the demand curve, and three competitors, then says yes β€” this one's worth chasing.

event: opportunity_identified
πŸ“¦

Product Manager β€” Tue, Mar 4 Β· 10:15 AM

β€œLet's make it the premium option.”

He sketches the concept in PLM, locks down the feature set, and walks the scope past stakeholders.

event: product_concept_approved
πŸ’²

Commercial Analyst β€” Thu, Mar 6 Β· 11:30 AM

β€œFirst price misses margin β€” re-model it.”

Her first scenario doesn't clear margin, so she loops back, re-models, and lands on $199 where the numbers work.

event: pricing_recommendation_created
🏭

Supply Planner β€” Mon, Mar 10 Β· 2:20 PM

β€œA supplier constraint means we slip two weeks.”

Capacity looks fine until a supplier alert pops up; the date slips and Product gets pinged to adjust scope.

event: supply_plan_updated Β· rework
🧾

Finance Manager β€” Tue, Mar 11 Β· 3:45 PM

β€œROI holds on the new date.”

She checks the revised cash-flow plan, the ROI still holds, and she signs off β€” clearing the path to the exec.

event: financial_approval
βœ…

Executive Sponsor β€” Wed, Mar 12 Β· 4:10 PM

β€œApproved β€” let's launch.”

One packet, multiple teams, a dozen steps, and two rework loops β€” all of it rebuildable from events alone.

event: launch_approved

The data behind the picture

What actually lands in the database — events, context & reasoning

This is the part that makes it more than logging. Each row doesn't just record what happened — it also stores the on-screen context the decision was made in and a short reason the assistant inferred on the spot. Multiple rows per role, all sharing one business object id, are what rebuild the whole launch journey.

event_iduser_idappobserved_eventui_context (what the screen showed)reasoning (inferred on the spot)timestampdecision
πŸ”Ž Market Analyst Β· u_11827
evt_90001u_11827Research PortalScanned category growth signals, bookmarked segmentGrowth chart trending +18% YoYSegment momentum looks real — worth a deeper look2026-03-03 09:02β€”
evt_90008u_11827BI DashboardAnalyzed 12-month demand curveDemand curve steady, low seasonalityStable demand lowers the launch risk2026-03-03 09:18β€”
evt_90014u_11827ExcelCompared three competitor offersRivals priced $169–$189, none premiumThe premium tier is wide open2026-03-03 09:35Recommend opportunity
πŸ“¦ Product Manager Β· u_24511
evt_90040u_24511PLMDrafted concept briefMarket opportunity brief attachedAnchor the concept to that premium gap2026-03-04 10:05β€”
evt_90047u_24511RequirementsPrioritized premium feature setBacklog ranked by value vs. effortLead with the three differentiating features2026-03-04 10:22β€”
evt_90052u_24511CollaborationValidated scope with stakeholdersComments resolved, scope signed offScope is stable enough to price2026-03-04 10:41Concept approved
πŸ’² Commercial Analyst Β· u_39003
evt_90100u_39003Financial ModelEstimated demand at three price pointsElasticity table across $179/$199/$219$199 balances volume and margin2026-03-06 11:15β€”
evt_90106u_39003Pricing PlatformRan elasticity model, saved scenario P2Scenario P2 margin = 31% on screenBelow the 35% target — needs rework2026-03-06 11:29β€”
evt_90111u_39003ExcelEstimated margin; first pass below targetMargin cell flagged red vs. targetCan’t recommend yet; re-model the price2026-03-06 11:48Re-model price
evt_90119u_39003Pricing PlatformRe-modeled at $199, margin metRevised margin = 36%, above targetNow defensible at the premium tier2026-03-07 12:10Recommend $199
🏭 Supply & Finance · u_17388 / u_28450
evt_90160u_17388ERPChecked capacity, found supplier constraintSupplier lead-time flag red on PO screenCapacity gap forces a date slip2026-03-10 14:20Delay 2 weeks
evt_90171u_28450Budget SystemValidated ROI on revised dateROI 22% on the two-week-later planStill clears the hurdle rate2026-03-11 15:05Approve investment
evt_90180u_28450Approval FlowConsolidated packet for executiveAll approvals green in workflow viewPacket is ready for the exec decision2026-03-12 15:45Launch approved

Every row carries three layers — the event, the ui_context it happened in, and a short reasoning — all sharing business_object_id = launch_sku_family_x, the thread that reconstructs the full cross-role journey.

Why it's optimization, not just mining

Why I think this is optimization, not just mining

Once the journey is just data, the AI can answer questions no single app ever could β€” and then actually do something about them.

πŸ” Replay

Walk back through exactly how one launch decision moved across roles and apps β€” every branch and rework loop, in order.

πŸ§ͺ Simulate

Try what-ifs against the real history: β€œif supply review started earlier, how many of these two-week slips just go away?”

βš™οΈ Optimize

Suggest concrete moves β€” reorder steps, pre-fetch the evidence, or flag the role that keeps triggering rework.

Bottom line for me: AI + the Chrome DevTools Protocol is already familiar territory. Pushing it into event-first process optimization is a practical, low-cost way to actually see, replay, and optimize how decisions really flow β€” long before anyone signs up for a heavyweight transformation platform.