Shadow AI discovery: prove each level of coverage

Verify shadow AI discovery through separate claims about software, network activity, account attribution and inspected interactions. Ask for evidence behind each claim on the browser, desktop and API paths you intend to cover. Installation records and domain connections answer different questions from a recorded policy decision about a prompt.

This guide proposes a controlled proof-of-coverage exercise. Microsoft documentation supplies examples of telemetry and its limits. No product or tenant was independently tested for this article.

Separate four evidence claims

Software present. An inventory entry can establish that a recognised application or extension is installed. Microsoft's software inventory documentation describes disk and registry evidence. That observation leaves actual use, sign-in and submitted content to be established separately.

Connection observed. A network event can establish contact with a destination. Microsoft's DeviceNetworkEvents schema includes destination and initiating-process fields. Background updates, a login page and an AI request can all generate connections to related infrastructure.

Account attributed. Telemetry may associate a connection with the account running the process. Assess shared sessions, service accounts and unattended processes before assigning an action to an individual person. Record the identity source and its limitations.

Interaction inspected. This claim needs evidence that the relevant AI request was examined at a defined control point. An app name, transaction count or volume of uploaded bytes does not establish content inspection.

These are independent questions. A browser-based service can be used without a dedicated desktop client. A discovered installation can remain unused. Use “not applicable” where a claim does not fit the path, and keep unsupported claims explicit.

Check where the signal comes from

Microsoft's Cloud Discovery overview explains that discovery analyses traffic logs against an app catalogue. Apps outside that catalogue are outside default discovery. Its traffic-log guidance also states that attributes missing from the input logs cannot be shown or analysed.

A perimeter log therefore depends on traffic reaching that collection path. A missing username field limits attribution. Destination URLs can improve identification compared with IP addresses, although shared infrastructure still needs careful interpretation.

The Defender for Endpoint integration can extend discovery beyond the corporate network using endpoint network-transaction logs. It requires the relevant licences, supported devices, onboarding and configuration. Microsoft documents platform-specific limits, including macOS TCP connection-close auditing and lack of UDP coverage for that integration.

Record those prerequisites before interpreting an empty report. An enabled integration and a reporting device are separate facts. Wait for the documented collection and processing interval before judging a test result.

Run a small comparison with harmless data

Use dedicated, authorised test devices and accounts. Choose fictional content with no customer information, working credentials or private business detail. Agree the expected evidence and collection interval before each action.

Start by opening an AI service's landing page without submitting a request. Then submit a harmless fictional task through the approved test account. Compare the records. If both appear as the same domain connection, the discovery data supports that level of detail.

Repeat through a desktop client and a dedicated API test client where those routes are in scope. Check whether initiating-process evidence distinguishes them and whether the relevant destinations are classified correctly. Keep API credentials inside the approved test setup and out of the evidence export.

Next, repeat one route from an onboarded device away from the corporate network. This tests the endpoint reporting path. A separately authorised, non-onboarded lab device using corporate egress can help identify what the perimeter path contributes.

Include a negative control: access a harmless non-AI feature on shared infrastructure where available. This tests whether the discovery label is more specific than the available evidence supports.

For every case, record the action, time, device, expected signal, observed signal, attribution source and remaining uncertainty. Capture a policy inspection or enforcement result only when a separate control actually supplies it.

Give local inference its own row

A fully local model may perform inference without an external cloud-service connection. Software or process telemetry could supply evidence of the runtime, depending on the product and configuration. Cloud-traffic discovery alone leaves that activity unmeasured.

Use an authorised local test with networking disabled if local inference is relevant to the buying decision. Record installation, execution and any available policy signals separately. Avoid equating an empty cloud-discovery result with an absence of AI activity.

Turn gaps into procurement decisions

Build a coverage record with one row per tested route. Give each evidence claim its own result: observed, unavailable, unsupported or not applicable. Add the reason and prerequisite, so an unexplained absence cannot quietly become a pass.

Ask the vendor to resolve the gaps that matter for the organisation's priority workflows. A missing signal may require configuration, a different collection source or a separate control. Confirm the remedy through another controlled test before expanding the coverage claim.

Our shadow AI overview introduces the discovery problem. The workplace governance guide explains what to do with an identified workflow. This proof-of-coverage exercise connects the two by showing which observations the organisation can actually rely on.

Map your shadow AI coverage with Promptective.