How To Fix POODLE (And Why You’re Probably Still Vulnerable)

The internet has been in an uproar over the past few days as a result of Google’s announcement of the POODLE vulnerability, which effectively breaks SSLv3 completely. We’ve scanned every single site that has passed verification with Tinfoil Security (that is, signed up and verified ownership) using our free testing tool, and sent emails to all those customers that have vulnerable sites. Our anonymized statistics of websites that have been tested show that 68%—that is, more than two-thirds—of the internet is vulnerable as I write this.

There has been a lot of misinformation flying around the internet over the past few days, and a lot of overly technical (or not technical enough) “how to” guides that are actually incorrect. This isn’t because the internet intends to mislead, but simply because cryptography of any sort, but especially TLS/SSL, is hard. We don’t claim to be experts, but we try really hard to be well-versed. With that in mind, this is our guide on how to protect your servers and what mistakes you’ve probably (unintentionally) made.

How Everyone Is Messing Up Cipher Disabling

The simplest thing to think is “OK, so it looks like if I disable any cipher suite that has CBC in the name, I’ll be fine.” This is wrong, and everyone is doing it, because it is what everyone is recommending.

There are a number of cipher suites, specifically AES128-SHA, AES256-SHA, and similar, that do not contain CBC in the OpenSSL cipher suite name. In reality, these cipher suites are named TLS_RSA_WITH_AES_128_CBC_SHA and TLS_RSA_WITH_AES_256_CBC_SHA, respectively. These cipher suites do work in CBC mode, no matter what OpenSSL chooses to call them!

We’ve disclosed this exact vulnerability to many companies at this point, including some large, extremely engineering-focused companies I guarantee you've heard of, all of whom have mitigated it now by disabling these last few ciphers. Given that we continue to see this exact mistake across most of the websites we’re testing, you should make sure you check it on yours too.

Your best option

Disable SSLv3 completely. Do not support it as a protocol at all, and reject any clients which require it. Unfortunately, this proves hard to do for many companies, as they have legacy clients whom they need to support that only support SSLv3 (specifically, IE6 on Windows XP and older Android clients, usually).

Our recommendation is simple: if, in any way, you can completely disable support for older clients that require SSLv3 and disable support for that protocol entirely, please do it.

Your next best option

If you must maintain support for SSLv3, your next best option is to enable the TLS_FALLBACK_SCSV cipher suite value. This cipher is a patch submitted to OpenSSL by Google (the same guys who found the exploit in the first place). This cipher solves the issue of retrying failed connections, thus preventing attackers from forcing browsers to use SSLv3. Additionally, it may also help prevent future exploits, as it prevents downgrading from TLS 1.2 to 1.1 or 1.0 as well.

Our recommendation: Make sure that you support the TLS_FALLBACK_SCSV cipher suite value. If you use OpenSSL 1.0.1, make sure you upgrade to 1.0.1j. Similarly, if you use 1.0.0, make sure to upgrade to 1.0.0o, and if you use OpenSSL 0.9.8, upgrade to 0.9.8zc.

Your last resort

If you cannot disable SSLv3, and if you cannot upgrade OpenSSL to support TLS_FALLBACK_SCSV for some reason, you have one additional option left: carefully disable all cipher suites for SSLv3 that run in CBC (cipher-block chaining) mode. Really, you should do this anyway, since TLS_FALLBACK_SCSV doesn’t actually resolve POODLE for anybody using SSLv3, it just prevents any newer clients from downgrading to SSLv3 and thus becoming vulnerable, limiting the number of clients that are affected.

This means that if you have to use SSLv3, your only real option left is to use some form of an RC4 cipher. This also isn’t perfect (there is a known attack on it by way of Royal Holloway, but it requires 224 connections), but is better than using any of these CBC ciphers which can be broken much faster.

If your configuration doesn't allow you to disable these CBC ciphers only for SSLv3 (for example, if you run nginx), you should consider using something like HAProxy to terminate SSL traffic in front of it and disabling / redirecting around it there. This, however, is hard and prone to error so you can go with the next recommended option: make sure your preferred cipher is set to RC4, rather than any CBC cipher.

Our recommendation: The following cipher suites that SSLv3 is able to use must be disabled in order to effectively prevent POODLE:

IDEA-CBC-SHA, EXP-DES-CBC-SHA, DES-CBC-SHA, DES-CBC3-SHA, EXP-DH-DSS-DES-CBC-SHA, DH-DSS-DES-CBC-SHA, DH-DSS-DES-CBC3-SHA, EXP-DH-RSA-DES-CBC-SHA, DH-RSA-DES-CBC-SHA, DH-RSA-DES-CBC3-SHA, EXP-DHE-DSS-DES-CBC-SHA, DHE-DSS-CBC-SHA, DHE-DSS-DES-CBC3-SHA, EXP-DHE-RSA-DES-CBC-SHA, DHE-RSA-DES-CBC-SHA, DHE-RSA-DES-CBC3-SHA, EXP-ADH-DES-CBC-SHA, ADH-DES-CBC-SHA, ADH-DES-CBC3-SHA, EXP-RC2-CBC-MD5, IDEA-CBC-SHA, EXP-DES-CBC-SHA, DES-CBC-SHA, DES-CBC3-SHA, EXP-DHE-DSS-DES-CBC-SHA, DHE-DSS-CBC-SHA, DHE-DSS-DES-CBC3-SHA, EXP-DHE-RSA-DES-CBC-SHA, DHE-RSA-DES-CBC-SHA, DHE-RSA-DES-CBC3-SHA, ADH-DES-CBC-SHA, ADH-DES-CBC3-SHA, AES128-SHA, AES256-SHA, DH-DSS-AES128-SHA, DH-DSS-AES256-SHA, DH-RSA-AES128-SHA, DH-RSA-AES256-SHA, DHE-DSS-AES128-SHA, DHE-DSS-AES256-SHA, DHE-RSA-AES128-SHA, DHE-RSA-AES256-SHA, ADH-AES128-SHA, ADH-AES256-SHA

Check, and check again.

We’ve been updating our tester constantly and improving its detection and results. We’ve added even more information regarding supported ciphers and the like. When you think you’ve fixed your server against POODLE, you should run another check.

We will continue to keep our eyes on this and attempt to update this post with more ways companies are making common mistakes in mitigating this attack, but if you follow the above directions, you should be OK. Scan yourself for even more vulnerabilities using the button below.

Feel free to chat with us or email us at any time.


Michael "Borski" Borohovski

Michael Borohovski is cofounder and CTO at Tinfoil Security. He got his start in security when he was just 13 years old, and has been programming for longer than he can remember. When he's not busy breaking software or building it, he also loves singing, juggling, and magic tricks. Yes, magic tricks.

Tags: browser security SSL poodle 0day


Scanning Your Private Sites with the Bifrost Tunnel

Thor Foilz

Today we're excited to introduce our newest enterprise feature: tunneled scans. Private tunnels are a secure, encrypted connection between your private web application and Tinfoil's scanners. We've simplified the popular open-source tool Ngrok to make it as easy as possible to scan your sites with Tinfoil, without having to host and manage an extensive on-premise solution. With the Bifrost Tunnel you can perform security scans on:

  • Sites inside your private network
  • Development machines as you're coding a new feature
  • Continous integration systems as an acceptance test
  • And more! Let us know what you're using it for.

Get started by installing and activating a simple client application, available on all major platforms. Currently the Bifrost Tunnel is running in a closed beta for our enterprise customers. Interested in trying it out? 

Sign Scanning For Free Today


Ben Sedat

Ben Sedat is the Engineering Wizard of Tinfoil Security. He's a bit of a blend between a traditional software engineer (builder) and security engineer (breaker). He spends a lot of time thinking about security: both detection as well as creating solutions for the security issues that exist in software and the internet. He also plays lots of video games. Lots.

Tags: features


Shellshock Scanner Free for Everyone

We care a lot about our customers’ security and every time a new major vulnerability comes out, we have your back. With that, we want to let you know that we added a suite of tests for the recent Bash vulnerability known as Shellshock to the Tinfoil web application scanner last week.

Shellshock is a high-severity vulnerability that allows an attacker to run arbitrary code on a vulnerable server. It’s important that you run a full scan on your sites and update Bash on any that are vulnerable. If you are a customer on any of our paid plans, recent and future scans include these tests. If you’re using our free XSS-only plan, we’ve added the Shellshock tests to run with any of your scans through the end of the year.

For non-customers, feel free to sign up for a Tinfoil Security account. You’ll automatically be enrolled in a free 30 day trial of our Standard plan and post-trial you can always keep scanning your website for XSS for free! Once in a trial, any scan run will test your website for the Shellshock vulnerability. Please note that this does not include scans run from our homepage.

If you’re having any issues running scans or have any questions, we welcome your questions and feedback. Feel free to chat with us or email us at any time.


Angel Irizarry

Angel Irizarry is the Software Samurai of Tinfoil Security, and a self-proclaimed software purist. All he needs to do his best work is a plain Linux machine with Git and Emacs installed. He loves everything about front-end development, like making pages interactive and super fast, even if that means digging in and optimizing some SQL. When he's not writing code, which isn't very often, you'll find him on his iPad scouring his RSS feeds for news and rumors of cool new gadgets.


The 'Shellshock' Bash Bug in Plain English

I’ve seen a ton of scary articles about a newly discovered Bash vulnerability that has been affectionately named Shellshock by the security community. People are saying Shellshock is bigger than Heartbleed, and that it can affect not just millions of web servers, but also routers, smartphones, and even light bulbs.

 

These articles all follow the same basic template. They say that there is a bug in Bash that can allow a remote attacker to execute any code they want on a vulnerable machine. Then they say that millions of computers run Bash, and as a result we are all doomed. The ones that lean more on the technical side present you with a snippet of Bash code that, when you run it, prints out something menacing like “You’ve been hacked!” if your version of Bash is vulnerable. As a developer, it’s been frustrating that these articles, in an effort to not confuse and lose the attention of their reader base, have shied away from going into the technical details of the bug. I think the details of Shellshock are instructional, and they’re way more interesting to read about than statistics on the pervasiveness of Bash. Yes, you should all patch your Bash right away. But let’s talk about the bug itself.

For fun, here’s that line of code that you can run to see if your version of Bash is vulnerable. Bash comes pre-installed in almost all Linux distributions, and it is the default shell in OS X Terminal. Windows users are safe, unless you manually installed Bash using Cygwin.

env foo='() { :; }; echo "Vulnerable!"' bash -c ':'

If your Bash version is vulnerable to Shellshock, it will print “Vulnerable!”, but why? That’s definitely not the intended behavior of this code. We are setting an environment variable foo to be the string '() { :; }; echo "Vulnerable!"' and then invoking a sub-shell that, in this case, does nothing. The end result should be that nothing is printed on the screen.

The problem stems from the funky way that Bash stores functions in environment variables. Let’s say you open up Bash and define a simple “Hello world” function.

$ function foo {
>   echo "Hello world";
> }
$

You can run the function all you want in this shell, but you can’t call it in programs that run inside it, which are also known as sub-shells.

$ foo
Hello world
$ bash -c 'foo'
bash: foo: command not found
​$

Let’s say you really want to run a sub-shell that uses the function foo. The standard way to do this is to use the export command to turn foo into an environment variable. The -f flag tells export that you are referring to a function.

$ export -f foo
$

You can also use the export command to make plain-old string variables into environment variables.

$ export bar="I am a string"
$

Using the env command, you can see all of the current environment variables as a list where each element is of the form <variable>=<value>. For functions, Bash uses special characters to distinguish it from the rest of the variables.

$ env
…
foo=() {  echo "hello world"
}
bar=I am a string
…

I’ve highlighted the special function characters so you can see them. This is where the vulnerability comes in. All Bash environment variables are strings, even when they represent functions. Bash uses the characters () { to distinguish a function string from a regular string. When a sub-shell is invoked, a copy of each environment variable is created and made available to the sub-shell. When Bash gets to an environment variable that starts with () {, it realizes this is a function string and evaluates the line in order to turn it into a real function. Unfortunately, up until a few days ago, Bash would just evaluate the entire string as code, blindly, with the same user permissions as given to the sub-shell, and without actually checking if the string is only a function definition. Therefore, it would not only run the function definition, but potentially any code, malicious or otherwise, that followed it. Let’s come back to the original one-line test.

env foo='() { :; }; echo "Vulnerable!"' bash -c ':'

Here, I’m using the () { characters to denote a function definition. However, I’m also ending the function definition and following it with more code. When I invoke the sub-shell using the bash command, the string inside of foo gets evaluated, and the echo is executed!

It gets worse. Exploiting this vulnerability on the web is shockingly (pun-intended) easy. Many web servers invoke Bash scripts in response to requests. One of the many ways that they can do this is by using the Common Gateway Interface, or CGI. It’s common for the web server to pass HTTP request information into the shell script, and the common way to do this is with environment variables. Things like the user agent string, cookies, and the GET parameters are stored in environment variables before running the sub-shell. Since users have access to all of these pieces of information, a malicious user could change their user agent string to be, say, '() { :; }; <malicious code>' and can force the web server to run any code they want.

Since this vulnerability was announced to the public last week, the Bash source code has gotten lots of new attention from security researchers. Many similar bugs in Bash have popped up, most of them similar to the original, and all allow unintended code execution. The latest version of Bash, version 4.3, has been patched three times in the last week to fix the discovered Shellshock variants, and there will likely be more variants discovered in the coming weeks. The best thing you can do is update Bash on all of your machines, even if they aren’t running network services. In addition, we’ll be updating the Tinfoil scanner in the next few days to scan for all of the known variants of Shellshock on your website. Tinfoil includes a free 30-day trial once you sign up, and in addition to the Shellshock update coming shortly, it scans for many more common web vulnerabilities.


Angel Irizarry

Angel Irizarry is the Software Samurai of Tinfoil Security, and a self-proclaimed software purist. All he needs to do his best work is a plain Linux machine with Git and Emacs installed. He loves everything about front-end development, like making pages interactive and super fast, even if that means digging in and optimizing some SQL. When he's not writing code, which isn't very often, you'll find him on his iPad scouring his RSS feeds for news and rumors of cool new gadgets.

Tags: plain english shellshock


What is DevOps?

The other week we announced our foray into combining DevOps and security into what we call DevSec. As we move forward with more posts on DevSec, I realize it’s important to lay out a good foundation for any company looking to become more agile with their security process. Today, we’ll take some time to understand what exactly DevOps is.

DevOps is a blend of development and operations, brought about a few years back when developers started pushing toward an agile development model. Agile development is just that: developing new technology quickly. Previously in web development, code and new versions of product and website would typically have a long hold between final development and public launch. During those weeks to months, testing and operation teams would make sure that no other piece of the product broke with the newly introduced features. Security teams would also use this time to make sure no vulnerabilities would become public as well. DevOps and cloud computing have changed this model to allow for quick deployment, where new product changes get pushed out weekly, daily, or sometimes even hourly.

One piece of the DevOps culture and mindset is the empowerment of developers to build and test against production-like systems, with automatic or semi-automatic deployment of the software. This is known as the concept of “infrastructure as code”. Things that only an enormous operations team used to be able to do are now being done with scripts in tools like Chef or Puppet, by a smaller operations team. Operations teams are getting more “dev” experience, and development teams are getting more “ops” experience. Over time, if all goes well, operations becomes more automated, speeding up deployment and getting software out to customers faster. This is the essence of DevOps; you can think of DevOps as a process for enabling continuous delivery of software.

DevOps introduces new practices to make sure that testing is done as code is written. These include unit tests (testing of a single point or feature to make sure it always works), integration tests (testing of complex inter-operations of your systems), and regression tests (testing done to ensure a bug previously found doesn’t reoccur), oftentimes written by the developers themselves. Continuous integrations systems, such as Jenkins or CircleCI, can ensure that tests previously run after a code-freeze are now run at every new commit of code. This automation of the testing process allows developers to quickly change and develop a product, increasing business agility, while not compromising on product quality.

Any company looking to become more agile (even non-tech companies) can benefit from creating a DevOps process. The process introduces automated tools to speed up all processes of development and IT, keeping the business moving forward. Our next post will outline some best practices on setting up a DevOps process, but as we explore the bridge between DevOps and security, we welcome all feedback and questions. Email or chat with us anytime.


Ainsley Braun

Ainsley Braun is the co-founder and CEO of Tinfoil Security. She's consistently looking for interesting, innovative ways to improve the way security is currently implemented. She spends a lot of her time thinking about the usability and painpoints of security, and loves talking with Tinfoil's users. She also loves rowing and flying kites.


Tinfoil Security Blog

Tinfoil Security provides the simplest security solution. With Tinfoil Security, your site is routinely monitored and checked for vulnerabilities using a scanner that's constantly updated. Using the same techniques as malicious hackers, we systematically test all the access points, instantly notifying you when there's a threat and giving you step-by-step instructions, tailored to your software stack, to eliminate it. You have a lot to manage; let us manage your website's security.