Click here to close now.

Welcome!

Java Authors: Carmen Gonzalez, Elizabeth White, Liz McMillan, Pat Romanski, Yeshim Deniz

Related Topics: Security, Java, Microservices Journal, Linux, Open Source, Virtualization

Security: Article

The Definitive Guide to Heartbleed and Enterprise Mobility

There has been a lot of media attention on Heartbleed and as always that means a lot of bizarre and often conflicting advice

By Peter Rogers, Principal Mobility Architect, Cognizant

There has been a lot of media attention on Heartbleed and as always that means a lot of bizarre and often conflicting advice.  I sat down (for a very long time) to find the truth of the matter and bring it together in one single article for those involved in enterprise mobility.

What is Heartbleed?
"The [Heartbleed] bug is in the OpenSSL's implementation of the TLS/DTLS (transport layer security protocols) heartbeat extension (RFC6520). When it is exploited it leads to the leak of memory contents from the server to the client, and from the client to the server. The bug was introduced to OpenSSL in December 2011 and has been out in the wild since OpenSSL release 1.0.1 on 14th of March 2012. OpenSSL 1.0.1g released on 7th of April 2014 fixes the bug."
[http://www.heartbeat.com/]

"An attacker can retrieve a block of memory of the server up to 64kb. There is no limit on the number of attacks that can be performed. The attacker has no control over the memory region where the block is read from. Sensitive information that can be obtained is: SSL private keys; Basic authorization strings (username / password combinations); and source code [most important also cookies]. This bug affects both sides of the connection. Not only will client certificates not save you from having to update your server certificate, they can be read from the client (along with your username, password etc.) by any server you connect to."
[http://blog.fox-it.com/2014/04/08/openssl-heartbleed-bug-live-blog/]

"The Heartbleed bug is an example of Buffer Over-Read and that means that the offending OpenSSL heartbeat code does not check that this length is the actual length sent in the heartbeat request, allowing the request to ask for more data than it should be able to retrieve. The code then copies the amount of data indicated by the length from the incoming message to the outgoing message. If the length is longer than the incoming message, the software just keeps copying data past the end of the message. Since the length variable is 16 bits, you can request up to 65,535 bytes from memory. The data that lives past the end of the incoming message is from a kind of no-man's land that the program should not be accessing and may contain data left behind from other parts of OpenSSL."
[http://blog.cloudflare.com/answering-the-critical-question-can-you-get-private-ssl-keys-using-heartbleed]

The non-technical explanation from XKCD is slightly easier to follow (http://xkcd.com/1354/).

Here is a very good FAQ and an actual screen shot of a Heartbleed attack from Fox-IT.

The important thing to note is that not only can a client potentially read data from a server but the converse is also true. This impacts mobile clients too and although Apple claim they are safe there were allegedly concerns around Android 4.1.1. If a client application is vulnerable on a mobile or desktop device then equally the server can act as a honey trap to steal information from the client. Web browser software and IM clients are the biggest concern, so please make sure you don't use obscure and vulnerable software.

Who is stealing what and why?

The important question is how we got into this mess. The first thing to point out is that this is not a floor in SSL/TSP, instead it is an implementation problem in the open source OpenSSL library.  A major contributing factor has been that TLS versions 1.2 came with OpenSSL 1.0.1 and after the BEAST attack everyone rushed to implement TLS 1.2. The current theory doing the rounds is that the NSA had already ‘found ways around' SSL some time ago and this could have been by taking advantage of the Heartbleed bug. It is however noted that Man in the Middle (MITM) techniques are far easier, especially as some large companies act as their own Certificate Authority (CA). Edward Snowden leaked the existence of BULLRUN (a highly classified decryption program run by the NSA) and subsequently Edgehill (a similar program run by GCHQ) which subsequently prompted Google to double the size of their encryptions keys. We know that a number of companies knew about the vulnerability before it went public because they stated they had applied the patches at least a week ago. If you think about it a minute then OpenSSL is open source code which meant anybody could have worked it out and kept it a secret.

They say the exploit leaves no trace and so nobody actually knows if they have been targeted. Actually that statement isn't strictly true. A single heartbeat can potentially grab 64K of memory however an attacker can either keep reconnecting or during an active TLS connection keep requesting arbitrary number of 64 kilobyte chunks of memory content. There are now companies who are configuring their network monitoring software to watch for abnormal behaviour such as Fox-IT, who have developed a set of Snort signatures to detect successful exploitation of the bug.

The most important thing is what happens next and most hackers will be after what Heartbleed.com classes as primary key material.

"These are the crown jewels, the encryption keys themselves. Leaked secret keys allows the attacker to decrypt any past and future traffic to the protected services and to impersonate the service at will. Any protection given by the encryption and the signatures in the X.509 certificates can be bypassed. Recovery from this leak requires patching the vulnerability, revocation of the compromised keys and reissuing and redistributing new keys. Even doing all this will still leave any traffic intercepted by the attacker in the past still vulnerable to decryption. All this has to be done by the owners of the services."
[http://www.heartbeat.com/]

Let's imagine that a service provider has been using a vulnerable version of Open SSL 1.0.1 and the types of hacker that may target them based on time ranges:

  1. Soon after the vulnerability was released (14th of March 2012)
  2. Two or three weeks before the announcement was made public but was still being discussed in hushed circles
  3. Ever since the announcement (April 7th)
  4. Ever since the Heartbleed proof of concepts went live (April 8th)

The first group are likely to be very serious hackers or even possibly Government sponsored activities which are nothing to worry about unless you are of significant danger to anyone. The second group are likely to be serious hackers or service providers testing out vulnerabilities in their systems (and potentially partner solutions) after they learn of the issues. The third group is the rest of the world and depends on the general availability of OpenSSL vulnerability scanning software at that date. The fourth group covers hobbyist hackers because the proof of concepts are open source and highly available.

Unfortunately as of April 8th then anyone in the world can now test for this vulnerability using this widely shared Python Script (https://gist.github.com/takeshixx/10107280) and that puts it in the domain of hobbyists. Anyone can now dump a bit of RAM from a vulnerable server. It would definitely take a professional who can write their own script to make repeated heartbeats attacks and then got lucky with the timing of a server restart on a vulnerable operating system to be able to pull out a private encryption key. However, even an amateur can use Session Hijacking which means you look for cookies containing session IDs. In a single grab of memory then you are equally as likely to obtain a session ID or a username and password combination, and this is slightly harder to track for abnormal behaviour.

An example of which is running the Python proof of concept against a vulnerable JIRA ticketing system and pulling out a JSESSIONID (which is JIRA's way of tracking your HTTP session). If the system requires authentication then you can just insert the stolen cookie into the browser and become that user on the JIRA installation. (https://www.mattslifebytes.com/?p=533)

Can you actually steal a private key?

The Heartbleed bug is an example of buffer over-read and here is the offending code for the programmers amongst us:

p = &s->s3->rrec.data[0]

[...]

hbtype = *p++;
n2s(p, payload);
pl = p;

[...]

buffer = OPENSSL_malloc(1 + 2 + payload + padding);
bp = buffer;

[...]

memcpy(bp, pl, payload);

In NGINX, the keys are loaded immediately when the process is started, which puts the keys very low in the memory space. This makes it unlikely that incoming requests will be allocated with a lower address space. If NGINX is reloaded, it starts a new process and loads the keys right away, putting them at a low address. On NGINX you can therefore retrieve user credentials, cookies and even public parts of the certificate but so far not the private part. It all depends therefore on the operating system and how often the servers are restarted. CloudFlare have even issued a challenge by leaving a server running nginx-1.5.13 linked against OpenSSL 1.0.1.f on Ubuntu 13.10 x86_6

That said there appears to be a proof of concept working on Apache for a first request which corresponds with the server being rebooted.

The Certificate Revocation Tsunami
There is a genuine importance to the CloudFlare challenge and that is that if private keys can be stolen then there will be a whole lot of companies requesting certificates all at once leading to a so called Certificate Revocation Tsunami. Certificate Authorities are supposed to revoke certificates within 24 hours if there is any evidence of a key compromise. However, even if all of the affected certificates were to be revoked, contemporary web browser software handles certificate revocation poorly. For example, some browsers only perform OCSP revocation checks for Extended Validation certificates, while others ignore certificate revocation lists completely. The other concern is the rush to create new certificates will lead to incorrect certificates being generated which in turn leads to Man In The Middle (MITM) exploits down the line.

"The certificate authority infrastructure was never built to do a mass revocation of this many certificates and because of the way the infrastructure is built, if you did do a mass revocation of millions of certificates it would significantly slow down the performance of the Internet itself, which is potentially very, very bad."
[http://www.washingtonpost.com/blogs/the-switch/wp/2014/04/11/no-joke-this-company-wants-to-be-hacked-with-heartbleed/]

"Such is the haste to fix the fallout of the Heartbleed bug, some certificate authorities and website administrators have been making careless mistakes. PayPal's Hosted Message Applications, such as the one at https://view.paypal-communication.com, are now using Extended Validation certificates issued by VeriSign on 10 April 2014. The CAB Forum requires certificate authorities to adhere to a stringent set of guidelines [pdf] when issuing EV certificates, and it is the CA's responsibility to verify the accuracy of the information in the certificate. In particular, they must verify that the legal name of the subject in an EV certificate matches the name which appears on official government records. However, this verification does not appear to have been performed correctly in the case of these certificates."
[http://news.netcraft.com/archives/2014/04/11/heartbleed-certificate-revocation-tsunami-yet-to-arrive.html]

How can I protect myself?

Here is a list of all the password changing advice from the various companies and the results of a vulnerable scan performed on April 8, 12:00 UTC.

"Only 30,000 of the 500,000+ SSL certificates affected by the Heartbleed bug have been reissued up until today, and even fewer certificates have been revoked. Some of the first sites to deploy newly issued certificates in response to the OpenSSL vulnerability included Yahoo, Adobe, CloudFlare, DuckDuckGo, GitHub, Reddit , Launchpad, PayPal, Netflix and Amazon's CloudFront content delivery network."
[http://news.netcraft.com/archives/2014/04/11/heartbleed-certificate-revocation-tsunami-yet-to-arrive.html]

Fox-IT have actually listed a set of vulnerability tests, along with advice of the next steps and have developed a set of Snort signatures to detect successful exploitation of the bug.

Vendors are recommended to perform the following operations:

  • Upgrade the OpenSSL version to 1.0.1g (you can also recompile your OpenSSL 1.0.1 with the compile time option -DOPENSSL_NO_HEARTBEATS)
  • Request revocation of the current SSL certificate
  • Regenerate your private key
  • Request and replace the SSL certificate
  • Restart your services

Upgrading to OpenSSL 1.0.1g  does not automatically restart your services and likewise certificate revocation and replacement is a whole different step from ‘patching your server'.

  • If a vendor does not restart their services then their sessions are still vulnerable
  • Even if a vendor does revoke their certificates then some web browsers ignore the revocation
  • If a vendor does not replace their certificates then if their private key was stolen then they are wide open

If we look at the situation with WordPress then it appears they applied the patches but may not have replaced their certificates yet. I can also see smaller companies forgetting to restart their services after apply the OpenSSL 1.0.1g upgrade.

Should I change my password?

If you look at this list of vendors and their statements then the following rules apply:

  • You are safe if a vendor never used OpenSSL (remember SSL isn't broken itself, just one implementation of it including heartbeats)
  • You are safe if a vendor used an earlier version of OpenSSL (0.9.8 / 1.0.0)
  • You are moderately safe if a vendor upgraded to OpenSSL 1.0.1g, restarted their services and issued new certificates, before the public announcement on April 7th (apart from in the case of serious hackers)
  • You are at risk if a vendor upgraded to OpenSSL 1.0.1g, restarted their services and issued new certificates, sometime after April 7th (the longer the gap the higher the risk) and should change your passwords
  • You are at risk if a vendor upgraded to OpenSSL 1.0.1g, restarted their services and has not issued new certificates. You have a personal decision to make regarding changing passwords.

o   There is a small risk a private key could have been stolen and so even if you change your password then the site could still be vulnerable and you just gave your new password away

o   That said if the key did not get stolen then changing passwords prevents a hacker using your stolen details in the future.

  • You are at risk if a vendor either did not upgrade to OpenSSL 1.0.1g across all their services or forgot to restart their server. It is probably pretty pointless doing anything yet if that is the case.
  • You are at risk if a vendor made a nebulous statement like "We patched our servers" but was not drawn about restarting their services or issuing new certificates and should look to Internet forums
  • You are risk if a vendor has revoked their certificate but not issues a new certificate yet. Your web browser may ignore the certificate revocation and be open to MITM attacks.
  • You are at risk if you used client side software using a vulnerable version of OpenSSL and could have leaked confidential information from your desktop or mobile device

o   Double check the application that you use on Internet forums especially web browsers and IM clients

  • Most vendors are going to tell you to change your passwords anyway but check the details as above
  • Making your password really long is good practise but is no deterrent whatsoever from Heartbleed and you run the obvious risk of forgetting it or revealing it to hackers waiting for all the password resets

Imagine a hacker did target Vendor X, based on watching the charts of companies requesting new certificates and targeting those who haven't yet. Let us presume they were running Operating System X (all we know is that NGINX probably isn't vulnerable and Apache potentially is - but it would require specific timing) and just as a server rebooted the private key was added to the top of the OpenSSL memory stack and at that precise moment a hacker somehow managed to make a first request with an OpenSSL heartbeat and steal the private key. If we imagine that Vendor X are rebooting their servers regularly and maybe there was an insider involved then it becomes all the more probable. Let us now imagine the servers have been upgraded to OpenSSL 1.0.1g and all the services restarted but that the old certificates are still up there. That means if you reset your passwords then it makes no difference whatsoever because the hacker can decrypt all the traffic using their stolen private key. Changing your password actually means that the hacker now has your new password and you think you are actually safe. The hacker may have already decrypted everything on the site using their private key and have all your confidential information but let's assume they haven't. Vendor X then issue new certificates and does not bother going back through the last two years of logs to look for Heartbleed attacks. You have absolutely no idea that the hacker stole your new password and you carry on using the service and update it with your new credit card details...you are completely surprised that your account has been hacked even though Vendor X patched their servers.

That said, I would actually be more worried by the news that the majority of the traffic performing heartbeat vulnerability scans is apparently coming from various Government (thankfully not ours) sponsored cybercrime activity around the world.

I would like to end by thanking all the Blogs that I have referenced in bringing together this hopefully definitive guide to Heartbleed. I am happy to hear your views on the matter so please feel free to email me.

Peter

************************************************************************

Kevin Benedict Senior Analyst, Digital Transformation, EBA, Center for the Future of Work Cognizant View my profile on LinkedIn Learn about mobile strategies at MobileEnterpriseStrategies.com Follow me on Twitter @krbenedict Browse the Mobile Solution Directory Join the Linkedin Group Strategic Enterprise Mobility Join the Google+ Community Mobile Enterprise Strategies Recommended Strategy Book Code Halos Recommended iPad App Code Halos for iPads

***Full Disclosure: These are my personal opinions. No company is silly enough to claim them. I am a mobility and digital transformation analyst, consultant and writer. I work with and have worked with many of the companies mentioned in my articles.

More Stories By Kevin Benedict

Kevin Benedict is the Senior Analyst for Digital Transformation at Cognizant, a writer, speaker and SAP Mentor Alumnus. Follow him on Twitter @krbenedict. He is a popular speaker around the world on the topic of digital transformation and enterprise mobility. He maintains a busy schedule researching, writing and speaking at events in North America, Asia and Europe. He has over 25 years of experience working in the enterprise IT solutions industry.

@ThingsExpo Stories
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, and physical persons. In the IoT vision, every new "thing" - sensor, actuator, data source, data con...
Container frameworks, such as Docker, provide a variety of benefits, including density of deployment across infrastructure, convenience for application developers to push updates with low operational hand-holding, and a fairly well-defined deployment workflow that can be orchestrated. Container frameworks also enable a DevOps approach to application development by cleanly separating concerns between operations and development teams. But running multi-container, multi-server apps with containers is very hard. You have to learn five new and different technologies and best practices (libswarm, sy...
SYS-CON Events announced today that DragonGlass, an enterprise search platform, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. After eleven years of designing and building custom applications, OpenCrowd has launched DragonGlass, a cloud-based platform that enables the development of search-based applications. These are a new breed of applications that utilize a search index as their backbone for data retrieval. They can easily adapt to new data sets and provide access to both structured and unstruc...
There's Big Data, then there's really Big Data from the Internet of Things. IoT is evolving to include many data possibilities like new types of event, log and network data. The volumes are enormous, generating tens of billions of logs per day, which raise data challenges. Early IoT deployments are relying heavily on both the cloud and managed service providers to navigate these challenges. In her session at Big Data Expo®, Hannah Smalltree, Director at Treasure Data, discussed how IoT, Big Data and deployments are processing massive data volumes from wearables, utilities and other machines...
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you'll have no problem fil...
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
The worldwide cellular network will be the backbone of the future IoT, and the telecom industry is clamoring to get on board as more than just a data pipe. In his session at @ThingsExpo, Evan McGee, CTO of Ring Plus, Inc., discussed what service operators can offer that would benefit IoT entrepreneurs, inventors, and consumers. Evan McGee is the CTO of RingPlus, a leading innovative U.S. MVNO and wireless enabler. His focus is on combining web technologies with traditional telecom to create a new breed of unified communication that is easily accessible to the general consumer. With over a de...
Disruptive macro trends in technology are impacting and dramatically changing the "art of the possible" relative to supply chain management practices through the innovative use of IoT, cloud, machine learning and Big Data to enable connected ecosystems of engagement. Enterprise informatics can now move beyond point solutions that merely monitor the past and implement integrated enterprise fabrics that enable end-to-end supply chain visibility to improve customer service delivery and optimize supplier management. Learn about enterprise architecture strategies for designing connected systems tha...
Cloud is not a commodity. And no matter what you call it, computing doesn’t come out of the sky. It comes from physical hardware inside brick and mortar facilities connected by hundreds of miles of networking cable. And no two clouds are built the same way. SoftLayer gives you the highest performing cloud infrastructure available. One platform that takes data centers around the world that are full of the widest range of cloud computing options, and then integrates and automates everything. Join SoftLayer on June 9 at 16th Cloud Expo to learn about IBM Cloud's SoftLayer platform, explore se...
SYS-CON Media announced today that 9 out of 10 " most read" DevOps articles are published by @DevOpsSummit Blog. Launched in October 2014, @DevOpsSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce softw...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
15th Cloud Expo, which took place Nov. 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, expanded the conference content of @ThingsExpo, Big Data Expo, and DevOps Summit to include two developer events. IBM held a Bluemix Developer Playground on November 5 and ElasticBox held a Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete between sessions.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, deploy, and manage applications integrating voice, video and data. He is the co-founder of TeleStax, a...
The Internet of Things (IoT) promises to evolve the way the world does business; however, understanding how to apply it to your company can be a mystery. Most people struggle with understanding the potential business uses or tend to get caught up in the technology, resulting in solutions that fail to meet even minimum business goals. In his session at @ThingsExpo, Jesse Shiah, CEO / President / Co-Founder of AgilePoint Inc., showed what is needed to leverage the IoT to transform your business. He discussed opportunities and challenges ahead for the IoT from a market and technical point of vie...
Grow your business with enterprise wearable apps using SAP Platforms and Google Glass. SAP and Google just launched the SAP and Google Glass Challenge, an opportunity for you to innovate and develop the best Enterprise Wearable App using SAP Platforms and Google Glass and gain valuable market exposure. In his session at @ThingsExpo, Brian McPhail, Senior Director of Business Development, ISVs & Digital Commerce at SAP, outlined the timeline of the SAP Google Glass Challenge and the opportunity for developers, start-ups, and companies of all sizes to engage with SAP today.
The 3rd International @ThingsExpo, co-located with the 16th International Cloud Expo – to be held June 9-11, 2015, at the Javits Center in New York City, NY – is now accepting Hackathon proposals. Hackathon sponsorship benefits include general brand exposure and increasing engagement with the developer ecosystem. At Cloud Expo 2014 Silicon Valley, IBM held the Bluemix Developer Playground on November 5 and ElasticBox held the DevOps Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of...
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information. In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focused on understanding how industrial data can create intelligence for industrial operations. Imagine ...