By now, you’ve likely heard of Heartbleed, a security bug in the OpenSSL cryptography library. The vulnerability is from improper input validation in the implementation code of the TLS heartbeat extension. The bug is in the heartbeat code, hence the bug’s name. Many applications and servers were affected, and VMware is no exception. VMware Heartbleed is a serious vulnerability: a bug in the security infrastructure itself leaves the systems open to full memory attacks. It is also widespread. Out on the open internet, 600k systems were originally found to be vulnerable. By late June, 300k were still vulnerable.
But what about core enterprise infrastructure? For example, are VMware ESX hosts susceptible?
The bad news: Indeed, VMware ESXi is susceptible to Heartbleed, as is VMware vCenter Server. There is a serious security advisory from VMware as well as a knowledge base article that explains it further.
The good news: Thankfully, security patches have been available from VMware for this critical vulnerability.
More bad news: Unfortunately, many VMware users have not been diligent at patching. According to CloudPhysics’ global data set, the world’s largest collection of machine metadata from virtualized datacenters, a lot of customers haven’t applied the VMware Heartbleed patches. How common is this problem? You might be surprised.
More than 50% of ESX and vCenter Servers are unpatched
According to our analysis, more than half the affected vCenter servers (57%) and ESXi hypervisor hosts (58%) are unpatched and remain vulnerable to VMware Heartbleed. This is a remarkably high percentage given that ESX run the majority of business critical VMs in the world. I speculate that IT teams are more lax about patching ESXi since those machines are typically behind the firewall and not easy to reach from the outside world. However, that laxity doesn’t make the delay in patching a good idea. For one thing, insider attacks continue to be a major source of breaches. Another consideration is that if outside attackers do manage to infiltrate a low privilege service inside your firewall, you have just given them carte blanche to attack your most sensitive data.
40% of orgs still vulnerable
At the macro level, we found 40% of organizations in our dataset are still susceptible to Heartbleed! That’s an incredibly high percentage three months after VMware identified and fixed the issue. In this case we counted organizations with at least one vCenter server or ESXi host running a vulnerable version.
The rate of patching is slowing down
How fast the patching gap is closing? Here we plot the percent of vCenter 5.5 and ESXi 5.5 servers globally (ignoring other versions that are not susceptible to Heartbleed) that are unpatched. While it’s encouraging that customers are moving in the right direction, the trend isn’t fast enough and is slowing down. Many security experts have predicted that it could take months to years to finally get rid of Heartbleed. Sadly, at least for VMware infrastructure, this seems to be true.
Three easy steps to rid yourself of Heartbleed
Is your organization part of the 40%? There’s no reason you should be. The fix is easy and the risk is not worth taking. And CloudPhysics is making it even easier: we’ve packaged up the VMware Heartbleed analytic we ran across our global data set, and it’s now available in our community (free) edition for users to run on their own VMware environments. What you can do:
- If you haven’t already, get CloudPhysics up and running in your datacenter (takes just a few minutes).
- Select and run the “Heartbleed Check.” You’ll find it in the Card Store. It will immediately show you precisely which ESXi hosts remain unprotected in your datacenter.
- Apply the patch(es). Here’s the table listing build numbers for the patches we’ve discussed here.
I hope you’ll take advantage of this free service. If you do, leave a comment below to let me know how it went or tweet me @virtualirfan!