What 200 questionnaires could not show: continuous vendor monitoring at a hospital network

The answer, first

A security questionnaire is a self-assessment, written by the party being assessed, describing a moment that has already passed. The group's most recent cycle issued 214 questionnaires across roughly 200 vendors, received 168 back, averaged 41 days per response and took a median of eleven weeks to close a file. Nearly every answer was true on the day it was written. What the process was never built to answer is what happens in the other eleven months.

ShadowMap did not replace the questionnaire, and the group did not retire it. What changed is that the questionnaire stopped being the only evidence. ShadowMap assesses a vendor from the outside using the same methodology it runs against the customer's own estate — asset discovery, exposure analysis, credential exposure and a security rating — continuously, and without requiring the vendor's participation. In the first cycle it produced two things a questionnaire structurally cannot. A Tier 1 clinical vendor whose external posture had materially degraded four months after passing its annual review, with the next review seven months away. And the observation that 19 of the 64 monitored vendors served their customer-facing systems from the same sub-provider — a company the group had no contract with, had never assessed, and had no standing to send a questionnaire to.

Where the existing programme stopped

The group's third-party risk function was not weak. It had a documented tiering model, contractual security schedules, and two and a half people dedicated to running the cycle. It produced a defensible file for every vendor in scope. The limits were structural rather than operational.

A questionnaire measures what a vendor says about itself. A penetration test report measures a scoped moment, usually chosen by the vendor. A certification measures a management system, not an estate. None of these notices that a vendor acquired a smaller company in April and inherited its infrastructure, or that credentials belonging to that vendor's staff started appearing in stealer-log data in June. Nor does any of them describe a fourth party, because the register is built from contracts and there is no contract with a sub-provider.

The group's own external estate made the same point in miniature. ShadowMap resolved 640 responsive hostnames against an internal register of 470 — a 36% delta on an inventory the team had considered reliable. That was not the object of the engagement, but it reset expectations about what any list-based control can be relied on to cover, including a vendor list.

What the first cycle produced

Across the 64 vendors and the group's own estate, the first 30-day cycle generated roughly 9,800 raw signals. AI Review and analyst confirmation reduced that to 178 findings carried into the vendor register — of which 14 required contractual escalation inside a week, 52 were routed to vendors as confirm-and-fix, and the remainder went to monitoring. Roughly 55:1 from raw signal to register entry.

Two modules reported nothing, and said so. Code Repository Monitoring returned no confirmed exposure attributable to the group or its vendors in the first cycle — recorded explicitly as zero rather than omitted. Data Exposure found no publicly readable cloud storage under the group's own accounts, and two under vendors' accounts containing only marketing collateral, which were noted and closed.

The vendor that changed between reviews

The clinical transcription and coding vendor sat in Tier 1 because it processes discharge summaries. It passed its March review cleanly: current certifications, a returned questionnaire, no material changes declared.

Between April and July its external estate grew from 40 to 96 responsive hosts. The cause was ordinary — it had acquired a smaller coding firm and absorbed its infrastructure. Among the inherited hosts was a legacy coding workbench running on an unsupported application server, publicly reachable, with an administrative login exposed without multi-factor authentication or source restriction. Its Security Rating moved from B to D over the same period. The grade was not the finding; it was the reason someone looked.

In parallel, Dark Web monitoring surfaced 31 credential records for the vendor's corporate e-mail domain added in the preceding 90 days, nine of them on the domain used by the coding workbench's sign-on.

What was validated, and what deliberately was not

None of those 31 records was tested. ShadowMap holds authorisation to validate against the customer's estate, not a third party's, and vendor-side credentials were passed through as untested advisories with the standing instruction to verify and rotate. The report says which mode each finding is in, per finding, rather than implying a validation it did not perform.

Where authorisation did exist, CART was used. Nine of the exposed vendor-staff identities matched accounts held on the group's own contractor remote-access portal. Against those, in scope and with an audit identifier on every request so the group could reconcile the activity in its own logs: three confirmed working and moved to a working-credentials state, five confirmed already disabled, and one excluded from testing at the group's request because it was a registered break-glass account. Nothing was modified.

The same discipline ran across the group's own domains. The first scan returned 510 credential records; after de-duplication and matching against the current staff list, 46 belonged to current employees. Seven were confirmed working, 33 confirmed no longer valid, and six were not tested because they pointed at a third-party service the group did not control and had not authorised testing against.

Ownership was routed, never assumed. Seventeen hosts were discovered that referenced the group by name in vendor-controlled domains. They were surfaced as references your organisation — confirm before acting, not attributed. The group confirmed 11 as dedicated tenants it had not known existed, established that four were shared multi-tenant infrastructure not specific to it, and left two unresolved at the time of the report, recorded as unresolved.

The concentration question

Resolving hosting, DNS, mail, certificate and authentication dependencies across the 64 vendors changed the shape of the register.

Nineteen of the 64 served their customer-facing systems from the same regional managed-hosting provider — as did six of the group's own patient-facing services. Nine vendors used the same white-label patient-communication platform for appointment reminders and results notifications, meaning one third party held patient contact data through nine separate contracts, each individually assessed as low impact. And four of the five vendors nominated in the business-continuity plan as alternates for one another resolved to the same two data centres in the same metropolitan area, so the documented failover was not independent of the failure it was meant to survive.

None of that is a vulnerability. It is architecture, and it is normal. The difference is that it is now an owned fact with a number attached, reviewed quarterly, rather than an assumption embedded in a plan nobody had tested.

What changed

The questionnaire programme was re-tiered rather than replaced. Full-length questionnaires dropped from 200 vendors to 71, with 129 moved to a short attestation, because the group no longer needed a 180-question form to tell it which vendors deserved attention. The time that freed went into the fourteen escalations the first cycle produced. Median time from detecting a material vendor exposure to holding a written remediation plan from that vendor was six days.

Three contracts gained fourth-party disclosure clauses and a remediation SLA at renewal. The business-continuity plan was rewritten. Concentration exposure entered the quarterly risk committee pack as a standing item with a dated figure behind it.

The reporting change mattered as much as the findings. The board pack had previously reported 168 of 214 questionnaires returned — a measure of programme activity. It now reports what is observably true, on a stated date, about the 64 vendors holding 88% of vendor-held patient records, with the workflow state and audit trail behind every line. An auditor can reproduce the number.