Tag Archives: Security

About security issues.

Why this blog uses HTTPS

Kerika, Technology

It’s not that we have sensitive stuff on the blog: quite the contrary. It’s just that we have implemented HTTP Strict Transport Security (HSTS) across the entire kerika.com domain.

HSTS s a security mechanism which helps to protect websites against protocol downgrade attacks and cookie hijacking.

It allows web servers to declare that web browsers should only interact with it using secure HTTPS connections, and never via the insecure HTTP protocol.

Since our blog is on a sub-domain of kerika (blog.kerika.com, to be precise), we needed to implement SSL and HTTPS for the blog as well.

Changing your Kerika password

Kerika, Technology, , ,

For folks who sign up directly with Kerika, we store the user password (in an encrypted form, of course), which means that these users can change their passwords directly from within the Kerika application by going to their My Account page at https://kerika.com/my-account:

Changing password for Google sign up
Changing password for Google sign up

For people who sign up using their Google or Box IDs, we rely upon Google/Box to manage their passwords: in fact, we never even see anyone’s Google or Box password, even for a second!

So, their My Account page looks a little different, like in this example of a Kerika+Google user:

Changing password for Google sign up
Changing password for Google sign up

 

Security within a Virtual Private Network

Kerika, Technology,

All of Kerika’s servers, which run on Amazon Web Services (AWS), operate within a Virtual Private Network (VPN), so they can be configured to only listen on local ports, e.g. ports like 10.0.0.1, etc.

This means that they cannot be accessed directly from the Internet: instead, all connections are routed through an Elastic Load Balancer (ELB), which is a special kind of AWS server that handles connections from all users.

The ELB is very secure: it implements SSL 2.0, and when vulnerabilities like Heartbleed and POODLE are discovered, it is relatively easy for us, with Amazon’s help, to quickly ensure that the the ELBs are patched.  Patching the ELBs quickly gives us breathing room to patch all the other servers involved, particularly if vulnerabilities are found at the platform level itself.

But, running a VPN isn’t enough: while it blocks people outside the Kerika server environment from directly accessing our database, there is still — at least a theoretical possibility — that an attacker can find his way inside the VPN, and then try to connect to our database server on a local port.

To avoid this scenario, we use SSL within the VPN as well, so that the connections from the load balancers to the database servers are also authenticated and encrypted.

Lenovo did something really rotten

Technology,

The news that Lenovo pre-installed adware on all consumer laptops sold in the US for the last three months of 2014 (yup, that would be the Thanksgiving through Christmas prime shopping season of the year) is being sadly under-reported by the mainstream press, although the tech press has a better idea of just how much mischief Lenovo did.

The really outrageous point here isn’t that adware came along with the other bloatware that all Windows users suffer from: it’s the fact that this adware was deliberately designed to undermine SSL, which underpins all security on the Internet.

Here’s how SSL is supposed to work: if you connect to Kerika, you are using a secure, encrypted connection to somebody that you genuinely believe is Kerika, Inc. of Issaquah, Washington, United States.

But how do you really know that it’s Kerika on the other end, and not someone pretending to be Kerika?

The only reliable way is to click on the lock icon shown in your browser (whenever you are on a secure SSL connection to any website), and your browser will then tell you who you are connected to, and more importantly, why the browser believes you are actually connected to Kerika and not somebody pretending to be Kerika.

Kerika SSL certificate
Kerika SSL certificate

The image above is the actual SSL certificate shown when you connect to Kerika, and then click on the lock icon in your browser.

It says, in effect, that a company called Symantec Corporation is the one that vouches for Kerika’s identity: in other words, it is Symantec Corp. that is assuring you that it really is Kerika that you are connected to, and not somebody pretending to be Kerika.

These SSL certificates could be issued by anyone, for example Facebook relies upon a company called DigiCert:

Facebook SSL
Facebook SSL

And Bank of America relies upon a company called Verisign:

Bank of America SSL
Bank of America SSL

Unless you happen to be using a Lenovo computer that you bought last Christmas, in which case there is a “man-in-the-middle” that you weren’t aware of:

Lennovo's fake SSL
Lennovo’s fake SSL

(Above image captured by security researcher Kenn White, @kennwhite)

On this Lenovo computer, an adware company called “Superfish” is the one that’s vouching for Bank of America, which isn’t right at all!

This is a classic “man-in-the-middle” attack scenario: most people would see the lock appear on the browser when connecting to a secure website, like Bank of America’s, and assume that they are safe. Instead, their communications is actually being intercepted by Superfish before it gets to Bank of America.

This is inexcusable, even for the “very minor compensation” that Lenovo got from Superfish.

(And, by the way, this is pretty much how most Windows PC manufacturers make money: there is so much price competition in the Windows market that they all resort to bloatware and adware to juice up their profit margins…)

And because the same piece of adware was distributed on literally thousands of machines, the same private encryption key is being used on all of these machines, which makes it easy for people to use these bogus SSL certificates to create man-in-the-middle attacks on any number of banks and other secure websites.

(And, of course, the private encryption key has already been decrypted, just hours after the news about Lenovo broke this morning.)

All this makes for appalling news for anyone who bought a Lenovo PC towards the end of last year. Merry Christmas.

Getting rid of a pesky “Mixed Content” warning

Agile & Scrum, Technology, Usability, , , , , , ,

When you first use Kerika, your browser has a reassuring sign that your connection to our servers is being encrypted:

No warning when you first use Kerika
No warning when you first use Kerika

But as soon as you open a card that contains any attachments, e.g. files stored in your Box account if you are using Kerika+Box, this reassurance would disappear, and instead you would see a warning about “Mixed Content”, which basically means that some of the data shown on your Kerika page was coming from a source that wasn’t using HTTPS.

Why there is a mixed content warning
Why there is a mixed content warning

This was because of a small bug in how we were dealing with the thumbnails we got for files stored in your Google or Box account: for faster performance we were caching these on our own Amazon S3 cloud storage (so we wouldn’t have to keep getting them from Google/Box every time you open the same card.)

It turns out that we weren’t fetching the thumbnails from S3 using HTTPS, which meant that as soon as you switched to the Attachment view of a card, your browser’s address bar would show the “mixed content” warning.

There was no real vulnerability resulting from this, but it did interfere with the user experience for that minority of users who like to keep a sharp eye on their browser’s address bar so we have fixed that with our latest release.

Now you should always have the warm reassurance of seeing the green secure site symbol on your browser when you open a card!

We have upgraded our SSL Security

Kerika, Technology,

We have upgraded the SSL certificate, used to secure your browser’s connection to kerika.com, from SHA-1 to SHA-2.

Kerika SSL
Kerika SSL

 

(SHA-2 is a cryptographic hashing algorithm developed by the National Institute of Standards and Technology (NIST) to replace SHA-1.)

This puts Kerika ahead of moves that Google and Microsoft will soon take, for the Chrome and Internet Explorer browsers, respectively, that will start showing warning signs when you visit a website that uses the older SHA-1 certificates.

If you are not sure whether your favorite secure site has upgraded to SHA-2, Symantec has a helpful tool you can use:

Kerika SSL check
Kerika SSL check

Changing passwords got a little easier

Usability, ,

Since we use OAuth 2.0 to let people sign up (and sign-in) using their Box or Google IDs, Kerika never actually sees any user’s password.

But, a lot of our users don’t quite understand how OAuth works, and they expect that when they go to the My Account screen in Kerika, they should be able to change their password right there.

Well, we aren’t going to move away from OAuth because we believe that’s a far more secure arrangement than having Kerika manage your password, but with our new release we are making it easier for people to figure out where they should go to change their passwords:

Change Password
Change Password

If you go to your Kerika account (http://kerika.com/my-account) and click on the Change Password button, it will take you to either Box or Google website where you can change your password.

A small “feature”, but one which we know will help smooth the way for at least some of our users 🙂

Kerika is secure against the SSL 3.0 fallback vulnerability

Kerika, Technology

You may have heard of the “Poodle” vulnerability in SSL, which allows the plaintext of secure connections to be calculated by a network attacker.

This vulnerability was discovered recently by Google engineers; here’s how it works:

  • Secure Internet connections used to be implemented with SSL 3.0, which is actually a pretty old protocol now. (About 18 years old, in fact, which means it dates back to the Netscape era 🙂
  • Over the years, SSL 3.0 was implemented by everyone who produced Web servers: e.g. Microsoft, Netscape, Apache, etc.
  • SSL 3.0 has since been supplanted with Transport Layer Security (TLS), which also comes in several flavors — TLS v1, v1.1 and v.1.2
  • And SSL was around for such a long time, everyone knew it worked. With TLS, however, bugs are sometimes found in different Web servers, depending upon who is producing (and maintaining) a particular brand of Web server.
  • In order to get around potential problems with the way a particular Web server had implemented TLS, browser clients (i.e. software that runs in a browser, like Kerika does) will also, very often, try to connect to the Web server using with SSL 3.0 as a fallback protocol.

Well, the good folks at Google found that SSL has a very fundamental vulnerability in it, that’s inherent in the protocol and cannot be patched: in an example attack called Padding Oracle On Downgraded Legacy Encryption (POODLE), an attacker can steal “secure” HTTP cookies or other bearer tokens such as HTTP Authorization header contents.

Angry Poodle
Angry Poodle

This problem is basically unfixable with SSL 3.0 because it uses RC4 ciphers for encryption, and RC4 is pretty darn old: it dates back to 1987!

(And, yet, according to Microsoft, even last year over 40% of Web connections were using RC4.)

The only way to secure against this vulnerability is to not allow SSL 3.0 as a fallback method for connecting to your Web server.

And that’s what Kerika does: we only support TLS connections.

Doing our bit to keep the Internet safe… 🙂

 

No, not Shellshocked

Kerika, Technology,

The announcement by CERT yesterday that there is a vulnerability in the Bourne Shell (more commonly known as “bash”) wasn’t great news for anyone running any variant of Unix, which includes Linux and MacOS.

Linux is very widely used for modern Web servers, particularly those running on Amazon Web Services — like Kerika does.

There are a number of variants of Linux out there, which makes things a little harder whenever a vulnerability is announced: you have to make sure your particular variant of Linux is patched quickly.

Luckily, this problem was fixed as fast as the notorious Heartbleed bug: within a couple of hours of the report of Shellshock, Amazon and Google (and, most likely, every other cloud services provider out there) started installing patches, and so the Software-as-a-Service (SaaS) world got back into good shape very quickly.

In our own case, we use Ubuntu Linux, and they were equally swift in issuing a patch for Shellshock which we installed yesterday.

On a side-note, we are less enthusiastic about Apple’s announcement that “the vast majority of users are not at risk“.

That’s true only in a literal sense: the vast majority of Mac users don’t ever use the Terminal program to access the shell, and a lot of permissions on Macs are locked down by default (and most users never bother exploring all their administrative privileges).

But, in a practical sense this bland statement from Apple understates the actual risk faced by Mac users: a significant majority of startups use Mac for their software development, which means a critical set of Mac users are still sitting exposed!

The sooner Apple fixes this bug, the better is will be for the startup world.

Heartbleed: no heartache, but it did prompt a complete security review

Kerika,

So, here’s how we dealt with the Heartbleed bug…

We learned about the bug just like you did: through news reports early on April 7th. Heartbleed was a “zero-day” bug, and the OpenSSL team put out an updated (patched) version of the OpenSSL protocol the same day, which meant that everyone, everywhere, had to scramble to get their systems patched as quickly as possible.

(And the bad guys, meanwhile, scrambled to grab sensitive information, with the Canadian tax authorities being among the first to report that they had been hacked. Of course, “first to report” isn’t the same as “first to actually get hacked”. Most people who got hacked either never found out, or never said anything…)

Kerika uses OpenSSL too, and our immediate concern was updating the Elastic Load Balancer that we use to manage access to our main Amazon Web Services (AWS) servers: the load balancers are where OpenSSL is installed; not on the Web servers that sit behind the load balancer.

Using Amazon Web Services turned out to be a really smart decision in this respect: Amazon just went ahead and patched all their load balancers one by one, without waiting for their customers to take any action. In fact, they patched our load balancer faster than we expected!

Patching the load balancer provided critical immediate protection, and gave us the time to do a more leisurely security review of all our operations. This was long overdue, it turned out, and so we went into “housecleaning mode” for over a week.

One part of this, of course, was updating all our Ubuntu Linux machines: Canonical Software was also very prompt in releasing a patched version of Ubuntu which we loaded onto all of our development, test, and production services. So, even though the OpenSSL vulnerability had been patched at the load balancer, we also applied patches on all our development, test and production servers even though these couldn’t be directly accessed from the Internet.

Next, we decided to clean up various online services that we weren’t actively using: like many other startups, we frequently try out various libraries and third-party services that look promising. We stick with some; others get abandoned. We had accumulated some API keys for services that we weren’t using any more (e.g. we had a YouTube API key that no one could even remember why we had gotten in the first place!), and we deactivated everything that wasn’t actively been used.

Closing unneeded online accounts helped reduce our “attack surface”, which adds to our overall security.

And, of course, we changed all our passwords, everywhere. All of our email passwords, all of our third-party passwords. All of our online passwords and all of our local desktop passwords. (On a personal level, our staff also took the opportunity to change all their banking and other online passwords, and to close unneeded online accounts, to reduce our personal attack surfaces as well.)

We got new SSL certificates: from Verisign for our production load balancer, and from GoDaddy for our test load balancer. Getting a new SSL certificate from Verisign took much longer than we would have liked; getting one from GoDaddy took just seconds, but on the other hand, Verisign does have a better reputation…

We reviewed our internal security policies and procedures, and found a few places where we could tighten things up. This mostly involved increased use of two-party authentication and — most importantly — further tightening up access to various services and servers within the Kerika team. Access to our production servers is highly restricted even within the Kerika team: we use AWS’s Identity & Access Management service to restrict access using roles and permissions, even within the small subset of people who have any access to the production server.

Finally, we are adding more monitoring, looking out for malicious activity by any user, such as the use of automated scripts. We have seen a couple of isolated examples in the past: not malicious users, but compromised users who had malware on their machines. Fortunately these attempts were foiled thanks to our robust access control mechanisms which manage permissions at the individual object level in Kerika — but, like every other SaaS company, we need to be vigilant on this front.

All of this was good housekeeping. It disrupted our normal product development by over a week as we took an “all hands on deck” approach, but well worth it.