[Resolved] Server problems; service inaccessible

April 18th, 2014  |  Published in Status  |  2 Comments

We’re having problems with an unresponsive central server which is affecting much of the Runbox email system and causing the email service to be inaccessible. We’re working to resolve it and expect to be operating normally within 30-45 minutes.

Update 01:15 CET: We are back online and investigating what happened. In any event this was a server that is soon to be replaced.

New IMAP servers deployed with Perfect Forward Secrecy

April 11th, 2014  |  Published in News, Security  |  5 Comments

Our new IMAP servers were successfully deployed today after upgrading the new ZFS based storage, which resolved an error that had previously caused problems. The technical details of this error can be found in the official bug report from the operating system distributor.

The combination of new, powerful IMAP servers and a modern, ZFS based SAN (Storage Area Network) should significantly improve IMAP performance in the coming days and weeks as we move email accounts to the new storage unit.

Perfect Forward Secrecy support for IMAP

Additionally, the new IMAP servers support Perfect Forward Secrecy on SSL (encrypted) connections, which prevents an unlikely eavesdropper to decrypt the communication between client and server.

You do not have to change anything in your email client to enjoy these new technologies, but do let us know if you experience any problems.

Tags: ,

[Resolved] “Heartbleed” SSL vulnerability

April 10th, 2014  |  Published in Security, Status  |  4 Comments

On April 8, it was revealed in the media that a vulnerability in the internet encryption standard OpenSSL had been discovered. This vulnerability could potentially allow someone to access additional parts of the memory of servers protected by the OpenSSL software.

As stated in the OpenSSL Security Advisory:

A missing bounds check in the handling of the TLS heartbeat extension can be used to reveal up to 64k of memory to a connected client or server.

This could potentially compromise sensitive data such as the secret keys used to identify the service providers and to encrypt the traffic, the names and passwords of users, and actual content.

Runbox’ servers are secured

Runbox immediately upgraded our installations of OpenSSL on April 8 upon learning about this vulnerability. We have also reissued and reinstalled all our SSL certificates for both Web, POP, IMAP, and SMTP services.

Additionally Runbox web services already supports Perfect Forward Secrecy, which issues unique SSL key pairs for each connection. This prevents an unlikely eavesdropper from retroactively decrypting communications between server and client even if they managed to get the private key.

What you can do

We have no indications that any information has leaked from our systems, and our assessment is that the risk of such leaks is very small. Client computers and software are not affected by this vulnerability.

However, we recommend that you change your Runbox password to be entirely certain that no one else can access your account. It’s a good idea to change your password regularly, and use different passwords for different services. Please see Tips for choosing and protecting passwords for some useful rules about password generation and usage.

More information about Heartbleed from the security company Codenomicon is available at http://heartbleed.com/.

Tags:

[Resolved] Transition to new servers and storage

March 27th, 2014  |  Published in Status  |  7 Comments

Runbox has seen a tremendous growth in our user base over the past months following the NSA revelations in the press. As a consequence of this we started executing our plans in January to acquire and install new and powerful virtualization servers and storage units.

Moving to the new servers

After substantial preparation of our server infrastructure we started moving data to the new ZFS based storage servers this week. The new storage servers are substantially faster, more reliable, and adds a lot more capacity than the current ones, and this process is moving forward steadily.

We are also deploying new, IMAP servers as an intermediate step towards completely replacing our  application server infrastructure. The IMAP servers we are currently deploying will improve IMAP performance while we complete the process of installing new, physical application servers that will replace both our current IMAP, POP, and web servers.

Some bumps in the road…

Some of our POP users started experiencing connection problems after being moved to the new storage servers. These users have now been moved back to the old storage servers until we resolve these problems. Update 13:00 CET 27.03.2014: This has probably been solved and we are waiting for feedback from everyone that was affected previously.

Additionally, the interaction between new storage, old storage and the new IMAP servers did not work exactly as predicted, so we rolled back the changes on Wednesday. We had done extensive testing over a long period of time before we deployed this solution, but with some differences (NIC, OS versions) We have now done further testing and will attempt deployment again shortly .

What we’re doing to resolve the problems

We have reviewed the process thus far in detail and uncovered the likely cause of the problems between the new and old servers. We are making the required system changes to ensure a smooth transition next time.

We would like to apologize to those of you who have experienced connection problems with Runbox recently with IMAP and POP, and assure you that we, along with our team of system administrators, will work to resolve these problems over the next few days so that we can provide fast and reliable services to everyone who cares about online privacy, security and sustainable services.

Update 01.04.2014:

We have gathered and analyzed data from the previous attempt at deploying the new servers and will make another attempt Wednesday (02.04.2014) morning CET, this time using a new set of virtualized servers. We will test new combinations of hardware and software between 8-10 AM CET until we have found the configuration that performs best. Meanwhile we have adjusted the configurations of the current IMAP servers to allow more concurrent connections and stop the connection errors some of our customers have seen throughout the day.

Update 03.04.2014: 

Generally IMAP should now operate normally. Between 9 and 11 AM CET when we carry out configuration work with the new IMAP servers some users may experience intermittent connection problems. This work will ensure that the new servers perform at their optimum reliability when we complete their configuration.

The new IMAP servers have performed perfectly during our test phase while emulating a large number of users, but something causes them to slow down when communicating with the new ZFS based storage units. We are working systematically to eliminate the causes and are excited about offering this superior storage technology to all our customers.

Update 08.04.2014:

After several days of testing we have narrowed down the problem to the new ZFS based storage units; not the IMAP servers as was indicated earlier. There are two main issues we are looking at and we expect to have a permanently deployed solution after a couple more days of work.

We plan to do the work outside of European and US business hours to avoid service disruptions for as many customers as possible. We are also looking at contingency plans in case this does not turn out as expected.

If you experience connection errors with Runbox IMAP, please contact Support as the symptoms can vary from account to account. We can then take steps to improve the situation for your account specifically.

Update 11.04.2014:

We have confirmed that the problem with the new ZFS storage was related to deadlocks in certain NFS threads in its operating system. A patch for this error was recently released, and after applying this upgrade the server has been operating perfectly for a full working day.

We therefore believe the problem to be resolved. We will continue to monitor its performance closely over the next few days.

The plan is then to continue moving user accounts to the new ZFS storage and our new IMAP servers, which is likely to improve IMAP performance for all our customers.

Tags: ,

[Resolved] Problems with payment processor

March 3rd, 2014  |  Published in Status

Our payment processor is currently having problems processing credit card payments. While they are working to resolve them you can use PayPal to pay with your credit card, even if you don’t have a PayPal account.

Just go to https://runbox.com/payment, select the desired product(s), and choose PayPal as your payment method on the next page.

Thank you, and our apologies for the inconvenience.

Tags:

Runbox mentioned in The New York Times

February 19th, 2014  |  Published in News  |  14 Comments

In an article in The New York Times discussing the challenges E.U. leaders face in protecting individuals’ data, Runbox is mentioned along with Deutsche Telekom as examples of companies providing services that increasingly protect their customers’ privacy.

Being firmly located in Norway, the Runbox email service is governed by strict privacy regulations and is a safe alternative to American email services as well as cloud-based services that move data across borders and jurisdictions.

Read the full article here: E.U. Leaders Seek Way to Protect Individuals’ Data

Tags:

Phishing scam alert

February 10th, 2014  |  Published in News

The following fraudulent email was sent to several runbox users today:

From: “Runbox Team” <runbox@nullrunbox.com>
Subject: This Important message is for you!!!

This is most likely a phishing scam trying to obtain logins, so please delete it immediately and DO NOT click on any links in the message. If you have already responded to this email in any way, please change your password immediately, from the main Account page in the webmail.

New feature: Tag management

January 26th, 2014  |  Published in News, Webmail  |  2 Comments

Many of our customers use message tags as an alternative to folders in order to organize and categorize their email.

You can now manage your message tags by clicking Tags in the left pane in Webmail, or by going directly to https://runbox.com/mail/tags. The Tag management screen lets you add and delete tags, and get an overview of the messages that are already tagged.

To add a tag to a message, just open the message, select the tag name (or [New tag] to enter a new tag name) and click “Add tag”.

Tags: ,

Max sub-accounts now available

January 15th, 2014  |  Published in News

By popular request we have added a new level of sub-accounts with similar quotas to the main Max account.

The Max Sub-account is now available for purchase from our Payment Options page and includes:

  • 15 GB email storage space
  • 2 GB file storage space
  • 10 GB email bandwidth per week
  • 5000 outgoing messages per day
  • 50 email aliases

Please see our Price Plans for details on all our products.

Tags: ,

Scheduled maintenance Jan 9; brief downtime

January 7th, 2014  |  Published in Status

Our sysadmins will replace two switches in our network Thursday January 9 at around 18:00 CET (12 noon EST). This will probably cause some short periods of downtime for the Runbox email services while traffic shifts from one path to the other.

No email will be lost or rejected as a consequence of this maintenance. Our US based web hosting services will not be affected.

Tags: