Are Your Windows Error Reports Leaking Data?

Websense® Security Labs™ recently processed a sample data set from the Websense ThreatSeeker® Intelligence Network to investigate the security risk from popular applications and services. We determined enterprise and public sector networks are inadvertently leaking information, which could be used by a threat actor as intelligence to craft specific attacks and compromise networks.

One troubling thing we observed is Windows Error Reporting (a.k.a. Dr. Watson) operating on Windows XP, Vista and 7 predominantly sends out its crash logs in the clear. These error logs could ultimately allow eavesdroppers to map out vulnerable endpoints and gain a foothold within the network for more advanced penetration. Here's more on why that's a concern:

• 80 percent of all network-connected PCs use it - that's more than one billion endpoints worldwide
• Dr. Watson reports information that hackers commonly use to find and exploit weak systems such as OS, service pack and update versions
• Crashes are especially useful for attackers since they may pinpoint a new exploitable code flaw for a zero-day attack
• Information is also sent for common system events like plugging in a USB device

- Alexander Watson, Director of Security Research, Websense, will present advanced findings related to this research at the 2014 RSA Conference in San Francisco. Join us for our session, "Use Anomalies to Detect Advanced Attacks Before Bad Guys Use It Against You" on Tuesday, February 25, 2014, at 4 p.m. PT. 

Windows Error Reporting (Code-named Watson)

Windows Error Reporting (a.k.a. Dr. Watson) is a crash reporting technology introduced by Microsoft with Windows XP. Microsoft's "Microsoft Error Reporting" privacy statement (http://windows.microsoft.com/en-US/Windows/microsoft-error-reporting-privacy-statement) describes the information that is sent up to Microsoft, and highlights the fact that applications with automatic error reporting enabled are (in some cases) not required to prompt the user before sending information to Microsoft.

Affected versions of Windows include Windows XP, Vista and Windows 7, as well as application crash reports from Microsoft applications on OS X. Windows 8 PCs enforce TLS encryption on all application telemetry to WER, following IT security best practices. In the online policy statement, Microsoft notes that administrators can implement fine-grained control over automated error reporting through pushing group policies to computers on the network. Here are instructions about how to do this: http://technet.microsoft.com/en-us/library/bb490841.aspx. However, our research indicates that by default many organizations are reporting (in clear-text) specific information about applications, services, and hardware through Microsoft Error Reporting. These application reports are not just limited to crashes, but also events such as failed application updates, USB device insertions, and in some cases even TCP Timeouts between computers on the network, a  large percentage of which is sent in HTTP clear text.

Let’s Dive In …

Many of us that use Windows-based computers in the workplace are familiar with the dialog box prompting you to voluntarily send information to Microsoft in the event of an application crash. However, it may be a surprise to learn what information is sent, and that a detailed report is sent (in clear text) each time that a new USB device (such as an iPhone, mouse or camera) is connected to a Windows computer. For example, below is a Wireshark capture associated with plugging a new Logitech wireless mouse to a computer running Windows 7. No interaction from the user is needed before it's sent to Microsoft.

(Click images for larger size)

As you can see, within seconds of connecting the new USB device to the computer, a report is sent to watson.microsoft.com in HTTP (clear text). This report includes a considerable amount of information that is URL encoded into the request. This information includes:

• Date
• USB Device Manufacturer
• USB Device Identifier
• USB Device Revision
• Host computer - default language
• Host computer - Operating system, service pack and update version
• Host computer - Manufacturer, model and name
• Host computer - Bios version and unique machine identifier

While this information is no doubt critical for Microsoft to debug application crashes and hardware configurations, it can represent a significant information leak when it leaves an organization without being encrypted. Due to the sensitivity of debugging information and metadata sent to Microsoft as part of Windows Error Reporting, Websense strongly recommends that organizations follow Microsoft's recommendations to redirect all Windows Error Reporting (WER) traffic on their network to an internal server using a Group Policy.

Below are two further examples of reports that can be sent to watson.microsoft.com through Windows Error Reporting. While hardware changes (such as the PnPGenericDriverFound message associated with plugging in a new USB device) can be sent up automatically, application crashes such as the second event described below require user approval before sending.

In the example above, we see a hardware change notification (PnPGenericDriverFound), which is reported to Watson.microsoft.com periodically when a device is plugged into a computer via USB, in some cases even if a driver exists on the system. After building lookup tables that map VID and PID codes to the respective vendor and product IDs (which can be found in Windows and Linux driver databases), it is possible to output a more human readable report describing the fact that an Apple iPhone 5 was plugged into a Sony Vaio notebook with Machine ID B293628 ... running Windows 7 SP1 with a BIOS version of R1090Y8.

The example above is a standard application crash report (signified by "StageOne") with Firefox version 21.0.0.4879 reporting an access violation in library Xul.dll, running on an Acer Aspire 1930 mid-tower running Windows 7 SP2.

Microsoft estimates that nearly 80 percent of Windows PCs participate in the Windows Error Reporting program, with an installed base of over one billion machines as of 2009. With the frequency of hardware changes (such as USB devices being plugged in), as well as automated messages such as device update failures and in some cases TCP timeouts, it is quite possible to quickly generate a representative model of a network. This includes network infrastructure, services, apps and versions. This information can be quite useful to an organization. It allows teams to understand applications and crashes on their own network, but could also pose significant risk if this information was intercepted by a hostile actor.

Below is an example report generated by parsing Watson Error Report logs to determine new USB devices that have been connected to computers within a sample data set. This information can be very useful to organizations. For example, to understand uptake of new BYOD policies and to identify potential security risks, but it is important that security best practices are used to protect this information. Microsoft's TechNet site details specific configurations that can be set for WER as part of a Group Policy, Problem Reports and Solutions, or direct editing of registry settings here.

Below is a sample data-set showing the top applications and versions reported in crash and update reports within a sample data set. Once again, knowledge of applications and versions deployed on a network can be a tremendous tool for IT security administrators to utilize. However, it's important that this information is protected and included in an organization's security configuration policies.

Conclusion:

We recommend services that report application telemetry and contain information about the security environment and underlying network infrastructure should be encrypted with SSL at a minimum, ideally using TLS 1.2 (http://tools.ietf.org/html/rfc5246). Applications that report this information without encrypting data risk leaking information at multiple points. This includes any upstream proxies, firewalls, and ISPs that are in between the corporate network and the destination as well as the application developer and their partner organizations.

In the case of Microsoft Error Reporting as well as other popular application telemetry reporting systems, Websense recommends that organizations set group policies (when possible) to force encryption on all telemetry reports and periodically audit their own network and applications for inadvertent leaking of information with security implications.

As part of a comprehensive security strategy, it is important that organizations understand the data contained in application telemetry reports and the level of controls used to protect metadata. All of this can impact an organization's security posture. Websense TRITON® web, email and data security provides multiple levels of protection against advanced threats including those tailored specifically to customer environments. If you are interested in learning more about the Seven Stages of Advanced Threats, take a few minutes to watch an archived webinar about the seven stages for advanced threats and data theft, why current defenses fail, and which defense layers you should use to protect your network, resources, and data.

In-Depth Presentation at 2014 RSA Conference:

Alexander Watson, director of security research, Websense, will be presenting advanced findings related to this research at the 2014 RSA Conference in San Francisco. Join us for our session, "Using Anomalies to Detect Advanced Attacks - Before It is Used Against You" on Tuesday, February 24, 2014, at 4 p.m. PT. In this discussion, we'll further explore the ramifications of this research, including targeting and exploit scenarios; identification/mapping of previously unknown zero-days through crash telemetry reports; and examples of how to use this research to identify zero-days that may exist on your network.

Updates: 1/3/2014 - Added clarification that affected versions of Windows include Windows XP, Vista and 7 and Microsoft application crashes on Apple OS X. Windows 8 enforces the recommended TLS encryption on all stages of telemetry reports to WER.