CrowdStrike explains update that crippled Windows environments

CrowdStrike has provided its first technical explanation for a file update that bricked Windows machines worldwide.

The vendor said in a blog post that a “sensor configuration update to Windows systems … triggered a logic error resulting in a system crash and ‘blue screen of death’ (BSOD) on impacted systems.”

It “corrected the logic error by updating the content” in the configuration file but indicated that a “thorough root cause analysis” was still needed “to determine how this logic flaw occurred.”

“This effort will be ongoing,” CrowdStrike said on Saturday.

“We are committed to identifying any foundational or workflow improvements that we can make to strengthen our process.”

CrowdStrike said it typically updated configuration files - known as “channel files” in the vendor’s ecosystem - for its Falcon sensors “several times a day”.

The update that went wrong had been intended to allow CrowdStrike Falcon sensors running on endpoints “to target newly observed, malicious named pipes being used by common C2 frameworks in cyberattacks.”

A named pipe is a mechanism that is “used to transfer data between processes that are not related processes, and between processes on different computers”, Microsoft documentation states.

CrowdStrike said that “systems running Falcon sensor for Windows 7.11 and above that downloaded the updated configuration from 04:09 UTC to 05:27 UTC – were susceptible to a system crash.”

Across Australia, airlines, airports, transportation networks, supermarkets, banks and enterprises had their device fleets crash from Friday afternoon AEST.

The federal government called an emergency meeting that involved CrowdStrike representation.

IT outages were then felt in other parts of the world.

The vendor has published a long list of actions and knowledgebase articles that IT administrators can use as part of remediation efforts.

CrowdStrike also used its technical explanation blog to dispute analysis on social media that suggested that blank or null values in the configuration file were part of the problem.

"This is not related to null bytes contained within [the offending] channel file, or any other channel file," the vendor said.