Technical assurance
What evidence should exist before a Microsoft 365 control is changed
Identity and access controls fail loudest at the moment somebody changes them. The difference between a routine change and an incident is usually the evidence gathered before anyone touched anything.
Changes to identity and access controls sit among the highest blast-radius changes an organisation makes. Conditional Access, MFA enforcement, sharing policies and legacy authentication settings shape who can reach what, everywhere, at once. They are also, mechanically, some of the easiest changes in the estate: a settings page, a toggle, a save button.
That mismatch is the whole problem. Commonly the configuration can be changed faster than its consequences can be understood, and the platform will not stop you. The discipline has to come from the evidence, not the interface.
Why control changes go wrong
The failure modes repeat across estates:
- The affected set was assumed, not enumerated, and the policy reached further than anyone believed.
- Service accounts, devices or integrations were quietly relying on the behaviour being removed.
- Exceptions existed only in someone's memory, so the change rediscovered them the hard way.
- Break-glass access was caught by the new rule, or had never been tested at all.
- Rollback meant turning it off again, with no definition of what turning it off would restore.
- Validation ended at nobody has complained yet.
None of these failures announces itself at change time. The anatomy is always the same: the change lands quietly, the affected people discover it as an outage, the service desk discovers it as a pattern, and the investigation discovers that nobody could say precisely what the control did before. The cost of the incident is set by the quality of the record, and the record can only be written beforehand.
The evidence that should exist first
None of this is exotic. It is the same evidence discipline that makes any platform change defensible, applied to controls where the blast radius is identity itself.
Exceptions are the real inventory
Every long-lived control accumulates exceptions, and the exceptions are where changes break. An exception that lives only in someone's memory is a dependency waiting to be rediscovered as an incident. Treat the exception list as part of the control itself:
- Every exception has a named owner and a written reason.
- Every exception has an expiry or review date. An exception without one is policy by accident.
- Exceptions are re-confirmed when the control changes, not assumed to carry over.
- The exception list is stored with the control's change record, where the next engineer will look.
Rollback is a definition, not an undo
Turning a control off again is not rollback; it is a second, unplanned change. Rollback means restoring the captured state: the exported configuration, the enumerated scope, the exceptions as they stood. Without the export taken in the first step, there is nothing to restore, only a memory of roughly how things were. That distinction is the difference between a five minute recovery and a reconstruction project.
The change itself
With the evidence in place, the change becomes a sequence rather than an event.
An evidence-led control change
- 01Capture current stateExport the control as it stands, so rollback has a definition.
- 02Enumerate the affected setIdentities, workloads and dependencies listed from the platform's own data, not from recollection.
- 03Surface exceptions and ownersEvery exception written down with its reason and owner, before it can surprise anyone.
- 04Define the change and its rollbackWhat changes, for whom, what rollback restores, and who approves it. In writing.
- 05Observe or pilotReport-only where available, a pilot group where not, with break-glass access tested explicitly.
- 06Enforce with validationEnforce, verify the expected effect against the enumerated set, and keep the whole record.
A control-specific example: Conditional Access
Conditional Access is the sharpest version of this problem, and the platform knows it: the tooling for evidence-led change is unusually good, and routinely unused. The sequence above maps onto it directly.
The same sequence, on a real control
- 01Capture the policyExport the policy and its assignments as they stand, so rollback restores a known state instead of a memory.
- 02Enumerate who it touchesAffected users, groups, applications and sign-in conditions listed from the directory, not from the policy's description.
- 03Record exclusions and ownersEvery excluded account and group written down with its reason, including the break-glass accounts.
- 04Simulate with What IfRun the defined scenarios through the What If tool to see which policies would apply to a given user, application and set of sign-in conditions.
- 05Observe in report-only modeDeploy the change as a report-only policy and read the sign-in logs: every real sign-in shows how the policy would have evaluated, without anyone being blocked.
- 06Test the way backProve the captured policy state can be restored before anyone needs it restored.
- 07Enforce with sign-offMove from report-only to enforced once the evidence, the owner and the approval agree.
The nuance is that these tools answer different questions. What If is a simulation: it tells you which policies would apply to the conditions you describe, and nothing about the conditions you forgot to describe. It also does not test service dependencies, so a policy tested against Microsoft Teams can still surprise you through Exchange Online. Report-only mode is observation: it evaluates most policies against real sign-ins without enforcing them, though not everything, such as controls in the user actions scope. Sign-in logs are the record: each event shows which policies applied, evaluated in report-only, or did not apply, and why.
Used together, the combination is strong. A copy of an enforced policy, modified and set to report-only, shows exactly how a proposed change would diverge from current behaviour on the same live sign-ins. That comparison, kept with the change record, is the difference between believing a change is safe and being able to show it. None of these tools replaces the ownership, exception review or sign-off described above; they make each of those decisions better informed.
A useful test
A useful test is whether the change record would satisfy the person investigating the incident the change might cause. Not a description written afterwards, but the record as it existed at the moment of enforcement.
“The settings page makes every control change look the same size. The evidence tells you which ones are not.”
Enumerating affected identities, dependencies and exceptions across a large tenant is exactly the kind of broad, tedious work that AI-assisted analysis genuinely accelerates, provided the decision and the enforcement stay human.
