03 / Enterprise workflow modernization
Azure DevOps Testing & Client Onboarding
A client testing portal on Azure DevOps: Excel-based UAT moved into Test Plans, external client users onboarded as Entra ID guests into a repo-less project, and an event-driven pipeline — service hooks → Azure Function → Service Bus → on-prem receiver — that keeps client-facing bugs, internal work items, test cases, and legacy CM cases correlated by ID. A systems design problem wearing a configuration exercise's clothes.
- Role
- Workflow design, automation, permissions architecture, migration strategy
- Period
- Ongoing
- Technologies
- Azure DevOps Test Plans · Service hooks · Azure Functions · Azure Service Bus · Microsoft Entra ID · Microsoft Graph B2B · Oracle Database · ADO REST API
What was actually broken
Client UAT lived in spreadsheets: test plans as Excel files, results emailed around, client validation tracked by hand, and defects re-keyed into a separate legacy case-management system. Nothing linked a failing test to a bug to an internal case to a release. Each client onboarding repeated the same manual choreography, and no one could answer “what exactly did this client validate, and when?” without archaeology.
- External clients must participate without any access to internal projects or repositories
- Entra ID guest access and Azure DevOps license/permission models bound what's possible
- Client-facing bugs must reach the on-prem legacy CM Oracle database — the internal system of record
- Cloud-to-on-prem delivery must survive retries and out-of-order events without duplicating cases
- The legacy issue-management process can only be replaced gradually, not switched off
What I owned
- Map the existing Excel-based UAT workflow end-to-end before proposing the integrated one
- Design the repo-less Azure DevOps project with Test Plans mirroring real validation stages
- Automate external-user onboarding via Microsoft Graph B2B guest invitations with least-privilege permissions
- Build the event pipeline: ADO service hooks → Azure Function App → Azure Service Bus → on-prem receiver → legacy CM Oracle DB
- Rebuild test-plan summary dashboards inside VisionHub from the ADO REST API
- Plan migration and adoption in stages that never leave testers stranded
Surface & System
Surface
From spreadsheet choreography to one traceable flow
In the original workflow, a test plan was a file — copied, renamed, emailed, and eventually contradicted by its own siblings. In the redesigned workflow, a test plan is a live structure in Azure DevOps: suites map to validation stages, cases carry their history, and a client tester works the same board as an internal one, just with a narrower view.
The client experience was designed deliberately: a guest signs in with their own organization's identity, sees exactly the test suites assigned to them, records outcomes in place, and files defects that are born linked to the failing case — no re-keying, no email archaeology.
- Test plans, suites, and cases replacing versioned Excel files
- Client validation recorded in place, linked to builds and work items
- Guest testers see only their assigned suites — least-privilege by design
- Defects created from failing tests, already linked to their evidence
System
An event pipeline from cloud to on-prem Oracle
The invisible half is the integration: client testers are invited through Microsoft Graph B2B guest invitations into a deliberately repo-less Azure DevOps project, with permissions enforcing the narrow view. When a client files or updates a bug, an ADO service hook fires into an Azure Function App, which publishes to Azure Service Bus; an on-prem receiver consumes the queue and inserts linked cases into the legacy CM Oracle database — so client-facing bugs, internal work items, test cases, and CM cases stay correlated by ID across the cloud/on-prem boundary.
That boundary is where reliability lives or dies. Hooks can fire out of order, functions retry, and a naive pipeline will happily duplicate cases forever. The receiver is built idempotent, the queue absorbs on-prem downtime, and the automation boundaries are explicit: what the system syncs automatically, what it flags for humans, and what it refuses to touch. Test-plan summary dashboards were rebuilt inside VisionHub from the ADO REST API, closing the loop back into the internal platform.
- Graph API B2B guest invitations — automated, auditable client onboarding
- Repo-less ADO project: Test Plans and boards without code exposure
- Service hooks → Azure Function App → Azure Service Bus → on-prem receiver
- Idempotent inserts into the legacy CM Oracle DB, correlated by ID
- Queue buffering absorbs on-prem downtime; retries never duplicate cases
- Test-plan dashboards rebuilt in VisionHub via the ADO REST API
Client testing & case-sync pipeline
Internal QA and external client testers authenticate through Entra ID — clients invited as B2B guests via the Microsoft Graph API — into a repo-less Azure DevOps project with Test Plans. Bugs filed from failing tests trigger service hooks into an Azure Function App, which publishes messages to Azure Service Bus. An on-prem receiver consumes the queue and performs idempotent inserts into the legacy CM Oracle database, keeping client bugs and internal cases correlated by ID. VisionHub dashboards read test-plan summaries back from the ADO REST API.
Decisions that mattered
Model the real validation stages, not the tool's defaults
The suite structure mirrors how internal QA and client validation actually proceed, so adoption meant recognizing your own process in the tool rather than learning a foreign one.
Guests, not shadow accounts
Client testers are invited as Entra ID B2B guests with their own identities — auditable, revocable, and honest — into a project that contains no repositories at all. The blast radius of any misconfiguration is bounded by design.
A queue between cloud and on-prem, not a direct call
The Function App publishes to Service Bus instead of calling on-prem directly: the queue absorbs downtime, smooths bursts, and decouples the cloud pipeline from the legacy database's availability — while the idempotent receiver guarantees retries never create duplicate cases.
Where it landed
An in-progress staged modernization: Excel-based UAT is moving into Azure DevOps Test Plans, client guests onboard through an automated Graph invitation flow, and the cloud-to-on-prem event pipeline keeps client bugs and legacy CM cases correlated — designed for gradual adoption rather than a disruptive cutover.
What I'd explore next
Extending automation coverage as confidence grows, and completing the retirement of the legacy issue-management process.
- Conservative automation means some manual steps remain, in exchange for a sync that can be trusted
- Guest access adds onboarding friction per client, in exchange for real security and auditability
- Running old and new issue processes in parallel during migration costs effort but protects delivery