A result only counts if we proved it was broken first.
The live-proof harness stands up an ephemeral Active Directory forest in AWS or Azure, registers real SPNs through the product's own execute path, and reads back a remote client's actual Kerberos mechanism. The verdict rules are strict on purpose.
We read the mechanism before the change and after it, and a second independent check has to agree.
Record the mechanism before the change. Make the change through Invoke-SpnExecutionEngine, not by hand. Record the mechanism after. Ask an independent second oracle. Only NTLM to Kerberos with agreement is a Pass.
If the baseline was already Kerberos, an SPN may have pre-existed and the run proved nothing about the product. That is Inconclusive, not Pass. A check that only reads the after-state cannot tell the two apart.
| baseline | after | oracle 2 | verdict | meaning |
|---|---|---|---|---|
| NTLM | KERBEROS | agrees | PASS | the change caused Kerberos |
| KERBEROS | KERBEROS | agrees | INCONCLUSIVE | nothing was proven |
| NTLM | NTLM | agrees | FAIL | the change did not take |
| NTLM | KERBEROS | disagrees | FAIL | oracles conflict; not trusted |
Fifteen provider scenarios are exercised per cloud. Not every scenario passes on every run and the matrix says which did not. The numbers above are best results to date, and the evidence pack for the run is what substantiates them.
The AWS best run was 2026-08-27; the Azure best run was 2026-09-04. Pack downloads are placeholders until publication: {{EVIDENCE_PACK_URL}}.
Provisioning log, per-host transcripts, oracle readings, a results matrix, and a teardown log, signed as a unit. The signing key and verification instructions belong on the trust center →.
File names above illustrate the structure; the pack format is described in its own README inside each pack.
How we back this up.
How we know the tests are worth having.
You will not find a test count or a percentage on this page. Those numbers change every week, and one that is out of date is worse than none, because the first thing a careful reader does is check it. Here is what we do instead, and why.
Every test has to be capable of catching a mistake.
That sounds obvious. It is not. It is easy to write a test that passes no matter what the software does, and it looks exactly like a real one from the outside. It gets counted, it turns the dashboard green, and it protects nobody. When we find one, we rewrite it or throw it away.
We plant a mistake in a copy of the code, and the tests have to catch it.
Nothing that ships is ever altered. We take a throwaway copy of the code, change one small thing in it - swap a comparison, remove a check - and run our own tests against that copy. The tests are what has to object. If they all still pass, then nothing was actually watching that part of the program, and we have found the gap ourselves rather than leaving it for a customer to find. The copy is discarded. Only the gap is kept, and it tells us exactly where to write a better test.
A test is not allowed to mark its own homework.
If you hand a test the answer and then ask it to check the answer, it will always agree with you. So any test covering how the software deals with a real system - your directory, your DNS, your SQL Server - has to be given something other than what it expects, or be run against the real thing. Occasionally a planted mistake changes nothing, because both versions of the code genuinely do the same job. We write down why, rather than leaving it looking like a hole.
Did it ask the right question, not just did it give an answer.
If the software asks your directory the wrong question, it gets nothing back. And "nothing found" looks identical to "nothing to find". So we check what it went and asked, not only what it reported afterwards. A clean report about a server nobody actually looked at is the most damaging thing a tool like this can hand you.
Four, because each one answers a different question.
Ordinary tests, run every time the code changes. The planted-mistake check above. Real proof, where we build a throwaway Windows domain in the cloud, confirm Kerberos is genuinely broken, fix it with the product, and confirm it genuinely works afterwards. And a security review of the parts that read data we do not control or make changes to your directory.
A check that tells you what it did not check.
Our own testing gate used to report all clear while quietly skipping a third of the files. That is the same mistake as calling a server clean when nobody looked at it, one level up, and we had been making it for weeks. It now says what it skipped and why. The goal is every scenario proven on a real domain, and as of 9 September every one of them is - including the last hold-out, DFS namespaces reached through an alias. That is the live matrix passing, which is not the same as the whole product being finished, and when something in it is failing again it gets written here by name. A gap you have to go looking for is a gap being hidden.
for the testing experts — the same claims in the usual vocabulary →
Nothing here is a stronger claim than the plain-English version above. It is the same six points with the field's own terms restored, for readers who would rather see them named.
- mutation testing
- "Plant a mistake in a copy of the code and see whether the tests catch it" is mutation testing. Stryker.NET covers the C# engine. PowerShell has no mainstream mutation tool, so the module is scored by a harness written for it, which generates the same operator, literal and statement mutations and runs Pester against each.
- surviving mutants
- A mutation that no test fails against is a survivor. Survivors are the work list: the suite is scored first and new tests are written against the specific survivors, rather than against an intuition about what looks untested.
- equivalent mutants
- "Sometimes we break the code and nothing changes" means the mutation produced a semantically identical program, so no test can distinguish it. These are proven equivalent by reasoning and recorded in the test file with that reasoning, never silently left in the survivor count.
- assertion independence
- "A test cannot mark its own homework" is the rule that an assertion is never fed its own expected value. A mock returning exactly what the test then asserts has tested the mock. Anything gated on an external system must vary the real-world value or be proven live.
- asserting the query
- "Did it ask the right question" means asserting the request, not only the result: the LDAP filter, the requested attributes, the account name, the service-name scope. A provider that queries for the wrong thing returns an empty set, which is indistinguishable from a clean finding at the result layer.
- negative control
- A live proof passes only when the baseline mechanism was NTLM, the post-change mechanism is Kerberos, and an independent second oracle agrees. A run that was already Kerberos before the change is Inconclusive, not Pass, because an SPN may have pre-existed. The verdict table is at the top of this page.
- the gate
- The mutation floor is enforced as a build gate rather than reported. It also enumerates what it did not score - files below full statement coverage cannot be meaningfully mutated - and warns when the coverage report it reads is older than any source file, because a file added since the last coverage run would otherwise be skipped in silence. That was a real defect in our own gate, not a hypothetical.