Regarding the issues that occurred on December 8th

Hello to all our valued customers,

AppFirst is going through very exciting times right now. We’ve successfully rolled out our log file data offering (hundreds of logs have already been uploaded in just a short time) and we’re constantly working to provide improved services to all of our customers.

As much as we’d like to share all of those successes today, instead we’re unhappily writing about a production problem we encountered yesterday that you may have noticed on your dashboards.

What’s Been Happening

Recently, there have been problems stemming from our Postgres database, which is where we currently store all of our process data. As a result of monitoring over 100 million processes, and given our current system architecture, we’ve come to a place where Postgres can’t handle this amount of data. As a result, the problems of slow page load times and data loading errors emerged.

How We’re Resolving These Issues For A Bright Future

Now that we have identified the problem, we’re improving our architecture to keep all the process data in HBase rather than Postgres. We’re confident this will resolve the current scaling issues and architectural problems we’re encountering. We’re taking a three-step course of action to achieve our goals:

Step 1: Earlier this morning, we stopped reporting process data to our free Developer users. We’re not happy that we had to do this, but already, our product is operating 5X faster than it previously was.

>> If you’re a Developer user and still need/want that process data, please contact us and we will re-enable it for you.

Step 2: Next week, we will begin moving our process data to HBase. During this time, process data will be stored in both HBase and Postgres. This will help keep load times down and everyone happy.

Step 3: After the New Year, we’re going to be moving all of our process data to HBase. We chose HBase because it’s a much more scalable solution. Postgres will continue to be used to store account info and things of that nature.

In the meantime, AppFirst continues to be grateful for your support and we offer our apologies for any inconveniences this may have caused. We’re working extremely hard to ensure that we remain a reliable solution for you.

We’re losing sleep so you don’t have to.

David Roth
Co-Founder and CEO