Tag Archives: Google Apps

Google Drive, Gmail, Google Docs, Google Contacts.

Google was flaky for 2 hours today!

Kerika, Technology

Between the hours of 9AM and 11AM PST, Google’s authentication service — which we used to sign in users of Kerika+Google — kept having problems that affected people at random.

It was a tough morning for us, dealing with the flood of “504 System Timeout” errors coming back from Google, and feeling helpless that we couldn’t provide the kind of high-quality user experience that is at the core of the Kerika brand.

The problems finally went away by themselves, but a total of 31 Kerika users were affected and we are reaching out to each of them individually to apologize for the inconvenience, and explain what happened.

This is one of those situations where Kerika cannot do anything to fix the problem: if you signed up as a Kerika+Google user, when you try to login to Kerika you are automatically redirected to Google’s authentication service, which then comes back to Kerika to give us your identity information.

Then, we use the identity information to log you into the correct Kerika account.

Normally, all this happens really fast: you click on the Sign In button at Kerika, Kerika redirects your browser to Google, Google responds immediately, and within a couple of seconds you are logged into Kerika.

It all happens so fast and smoothly, 99.999% of the time, that most people are completely unaware that their browser was even redirected to Google in the first place — it’s something you might notice only if you have a very slow WiFi connection, and you are paying close attention to your browser’s status bar.

But every once in a while, Google won’t respond when you get redirected there by Kerika. In that case we retry again several times, and then finally Kerika does a “timeout”: it gives up.

(This problem has happened before, and as software developers ourselves, we are, of course, very sympathetic to other software companies that experience occasional bugs and hiccups, but Google can be irksome in their lack of transparency.)

This happens so infrequently that we didn’t really have any special code in place to tell users why they were not able to login, but that’s going to change starting tomorrow: if Google’s servers are not responding fast enough, we will show a special page to the user explaining what’s happening, so they understand the situation better.

 

Google has been flaky all morning

Kerika

Between 9AM and 11AM Pacific Standard Time, Google’s authentication service threw up about about 60 errors, which gave many of our users a rough ride.

The problem was that Google’s authentication servers were timing out repeatedly, showing errors like this:

Request URL: https://kerika.com/authcallback/google?code=XXXXXXXX
Request Method: GET
Status Code: 504 Gateway Timeout

There isn’t anything we can do in this situation to help our Kerika+Google users, unfortunately, since Google’s authentication service needs to be up and running in order to log in to Kerika.

The silver lining in this cloud, for now, is that this is a relatively rare occurrence.

The new OAuth 2.0 integration with Google Apps

Technology

With our latest version, we will be using OAuth 2.0 uniformly across all the ways you could sign up for Kerika:

  • As a Google Apps for Business user, e.g. someone who has the premium (paid) version of Google Apps, and signs up at kerika.com or through the Google Apps Marketplace.
  • As a free or premium user of Box, who signs up at kerika.com or through the Box App Store.

Across all these, we are now using OAuth 2.0: the more modern, robust implementation of OAuth which lets you sign into Kerika using a Google or Box ID, without Kerika ever seeing your password!

(Background: while we were using OAuth 2.0 for people who signed up directly at Kerika.com, we had OAuth 1.0 in place for people who signed up through the Google Apps Marketplace, and we needed to make every pathway consistently work with OAuth 2.0 and completely get rid of OAuth 1.0)

This new version will affect all premium users of Google Apps:

With the old (OAuth 1.0) integration with Google, it was possible for individual users who had the premium version of Google Apps to sign up for Kerika.

With the new (OAuth 2.0) integration, the Google Apps Administrator for the domain (i.e. your company) to authorize Kerika for the entire domain.

In other words, if you are a premium user of Google Apps, your Google Apps Admin — typically someone in your IT department — will need to authorize Kerika for your domain before you can use Kerika.

This will affect you even if you are an existing user of Kerika.

The good news is that once your Google Apps Admin authorizes Kerika for your company’s domain, it won’t be necessary for individual users to authorize Kerika any more: it becomes much easier for your colleagues to sign up.

Why we are integrating with Box, Part 9: Final QA

Government, Kerika, Usability, , ,

(The ninth in a series of blog posts on why we are adding integration with Box, as an alternative to our old integration with Google Drive.)

We have been doing internal testing (“eating our own dogfood”) of Kerika+Box for the past three weeks, and the results have been much better than we expected!

We have found very few bugs so far, which is great — it’s feels like a huge vindication of our decision to invest several Sprints in improving our internal QA processes, clearing the backlog of old bugs, and generally improving our software development processes with code reviews across the board, for even the smallest changes.

In other words, we didn’t move fast and break things: we moved slowly and broke nothing. Which makes sense when you have paying customers who rely upon your product to run their businesses…

Since Kerika makes it really easy to have multiple backlogs in a single account, we put all the OAuth and infrastructure work in a separate backlog, allowing a part of the team to concentrate on that work somewhat independently of other, more routine work like bug fixes and minor usability updates.

And, as before, put every feature in a separate git branch, making it easy to merge code as individual features get done.

Here’s what our Box QA board looks like, right now:

Box QA board
Box QA board

The user interface for Kerika+Box is essentially the same as for Kerika+Google, with a few quirks:

Box requires more frequent logins: Google provided us with relatively long-lived refresh tokens, so a user could close a Kerika browser tab and reopen it a day later and log right back in.

With Box, you are going to see a login screen much more often, along with a screen asking you to re-authorize Kerika as a third-party app that can access your Box account.

This is kind of irritating, but apparently unavoidable: from what we have found on Stack Overflow, Bug views this as a feature rather than a bug.

The other, really big difference is that files are edited offline rather than in the browser itself: when you click on the Edit button, you will end up downloading a local copy of the file, using Microsoft Office for example, and then when you do a Save of that file, your latest changes are uploaded automatically to the cloud.

Here’s what you see when you open a file attached to a card on a Kerika board, when you use Kerika+Box:

Example of opening a file within Box
Example of opening a file within Box

This works great most of the time, except when two people are making changes simultaneously: in that situation, Google’s in-browser editing seems a lot more convenient.

On the other hand, downloading local copies of files means that you get the full power of Microsoft Office, and we know that’s very important for some of our users, e.g. consultants dealing with complicated RFPs or government users dealing with official documents.

Performance also seems a little less than Google Drive, although we would stress that this is highly variable: while Google Drive files generally open within 1-3 seconds in a new browser tab, they can take much longer if Google’s servers are slow.

Overall, we are very pleased with Kerika+Box: we are planning to do all of our new development with this new platform, to continue eating our delicious dogfood 😉

The full series:

Why we are integrating with Box; Part 8: Our experience with Box (so far)

Kerika, Technology,

(The eighth in a series of blog posts on why we are adding integration with Box, as an alternative to our old integration with Google Drive.)

We have finished our development work on the Kerika+Box integration, and have started internal testing this week!

Our philosophy all along has been to “eat our own dogfood”: every new feature that we have ever built has been used internally, for real, by the Kerika team for its own product development and team collaboration. Since we are ourselves a distributed team, spread out between Seattle and India, it makes perfect sense for us to use Kerika to build Kerika 😉

Here’s what our Box integration testing board looks like:

Box integration testing board
Box integration testing board

We haven’t run into many problems so far: we did find one small bug in the Java Library for the Box Collaboration API, and we have been extremely impressed with the responsiveness of the Box team.

  • We first logged the bug at Github, where we had assumed (incorrectly, as it turns out) Box would want their bugs reported.
  • Having logged the bug, we pinged Indy Sen at Box, by Twitter and by email.
  • Indy responded within 15 minutes, on both Twitter and email.
  • Indy got us in touch with the API team at Box, and asked us to log the bug on Stack Overflow, where Box does its platform support.
  • We did that later in the evening, at 11:04PM.
  • Within 4 minutes a support engineer confirmed that the bug existed.
  • Later in the evening we asked a follow-up question, to confirm that our workaround was sensible.
  • The next day, before noon, their support engineer confirmed that our workaround was fine.
  • A couple of hours later, Peter Rexer told us they had fixed the bug and merged the code onto Github.

Now, admittedly, the bug was very trivial — a single misplaced character — but the responsiveness of Box as an organization just blew us away!

Contrast this with the experience we have had in the past with Google Drive, when problems would mysteriously appear and then disappear just as mysteriously, either the same day or several weeks later…

The full series:

Why we are integrating with Box; Part 7: Disentangling from Google

Kerika, Technology,

(The seventh in a series of blog posts on why we are adding integration with Box, as an alternative to our old integration with Google Drive.)

Having decided to add a second cloud platform, we had to go about the cumbersome, risky process of disentangling Kerika from its close embrace of the Google stack.

This has taken us several months, if we are honest about counting the entire effort and duration.

The first part of this was to create a more layered architecture: wherever there was a direct interface to a Google service, we had to create an intermediate shim that allow us to substitute the Google service with another.

And to be frank, when we first started work on Kerika in 2010, we hadn’t seriously considered the possibility that we would use any other cloud service so our architecture had become, over the years, more Google-focused than we had realized.

Disentangling from Google Contacts was the easiest step, and in fact we had already taken that break some time ago in response to users being fearful about Kerika accessing their Google Contacts.

In terms of login, however, things got much more complicated: we had originally integrated with Google using OAuth 1.0, and then later upgraded to OAuth 2.0 as OAuth 1.0 started to get deprecated.

In theory this meant that we could easily integrate with any other OAuth provider, e.g. Box, Facebook, Yahoo, etc.

But as we developed our long-term strategy and talked to prospective customers, it became clear that sooner or later Kerika would have to provide direct signup/sign-in, in addition to OAuth: a significant contingent of folks wanted to be able to sign up directly at Kerika.com rather than use existing credentials from Google, Box, etc.

(We are not quite sure why this is really advantageous for anyone, but the sentiment is real nonetheless.)

So, we decided to implement our own OAuth server, which would act as a gateway to OAuth servers from Google or Box.

This had to be implemented such that the entire sign-up and sign-in process remains unchanged: someone coming to Kerika.com would get redirected automatically to the correct authentication service, which could then redirect the user’s browser back to the Kerika application server.

And, obviously, the entire process had to be smooth, fast and completely transparent to the end-user: how we implemented multiple OAuth sign-ins wasn’t the user’s concern! This had to work really well for both free users of Google, e.g. folks with just a regular Gmail account, and premium users who had signed up for Google Apps for Business or any of its variants (for Government, Education, and Nonprofits).

Fortunately, we already gained experience in dealing with the complexities of browser redirects in enterprise environments, where any sloppiness in terms of using HTTPS, for example, can trip up security alarms in large organizations.

And, we had learned the hard way to look for various edge cases, like restricted domains — if someone from within a restricted domain tries to add someone from outside that domain, we need to trap that condition so that the user experience remains excellent, rather than landing anyone on some scary error page from Google!

The other major complexity was to normalize the service we provide, so that the user experience is equivalent for both Google and Box users.

An example of a “normalized user experience”: an existing Kerika, who uses Google Drive, decides to invite someone to join a project. At that time, we need to check whether the invitee is already registered as a Kerika user or not. If the invitee is already a Kerika user, we need to see whether that user is also using Google Drive.

If it turns out that the invitee has been using Box, we need to make sure the invitee is correctly prompted to switch from using a Google-based Kerika account to a Box-based account, so that she has all the proper access to the new project’s files. And this switch between accounts needs to be a single-click, to keep the Kerika user experience excellent!

Other examples: exceptions and error conditions as reported by Google and Box are very different, reflecting the peculiarities of each service. We need to make sure we capture all these and try to keep the end-user experience as smooth as possible. Fortunately, we had developed a great deal of experience managing exceptions generated by Google Apps, so we could bring all this to bear on our Box implementation.

To get all this right, we had to carefully plan a series of milestone releases that were done in parallel with our regular development: although the work was going to take several months to finish, we needed to keep supporting our existing users with critical updates, bug fixes and limited new product enhancements.

Fortunately Kerika lets you have multiple Scrum Backlogs in the same account, and that’s what we did: we created a separate, smaller Product Backlog consisting of just the OAuth work, and let that development run in parallel while we continued to draw down on our Main Product Backlog.

The OAuth work finally got done in four Sprints, each of which was organized as a separate Scrum Board.

As we came to our regular release cycles, we would pick up items that were complete, from either the OAuth Backlog or the Main Backlog, and merge these to create a new release package.

To get this right, we used git to the fullest, putting each feature into a separate branch so that we could retain the maximum flexibility in terms of what made it into a particular release.

Without this use of multiple git branches, our release cadence would have been tough to maintain.

The full series:

 

Why we are integrating with Box; Part 6: The Box Option

Kerika, Technology, ,

(The sixth in a series of blog posts on why we are adding integration with Box, as an alternative to our old integration with Google Drive.)

And this brings us to Box…

We first heard of Box through cloudPWR, a Kerika partner and long-time Box reseller: Shadrack White, Dennis Brooke, and Cullen Hower suggested to us (a year ago!) that we consider integrating with Box.

Shad and Dennis were both enthusiastic proponents of Box: they had done several implementations of Box, including a very interesting use-case for Washington State’s Liquor Control Board which found itself in the business of regulating medical marijuana this year.

And Dennis in particular has been a great proponent of Kerika: he had introduced it to some of his clients.

As we looked at Dropbox and OneDrive as possible alternatives to Google Drive, Box came up repeatedly in the conversations we were having with enterprises.

It was clear that Box was treated very seriously by some folks that we considered very smart and knowledgeable — a senior director at Amazon, for example, told us (off the record) that he considered Box to be the most enterprise-ready cloud storage platform — and so we decided to take a closer look at the Box platform.

We attended Boxworks in San Francisco last summer, and were immediately struck by the differences in tone and substance between Boxworks and DBX.

While Dropbox is a consumer-oriented company with a newly developed interest in the enterprise, Box is a very enterprise-focused company (with little or no interest in consumers).

We took a close look at the Box API, and were very pleased with what we found: Box’s API was very close to what we were getting from Google Drive, which meant that a Kerika+Box integration could offer a really good user experience:

  • If you add a file to a card or canvas on a Kerika project board, it will automatically get shared with everyone who is part of the project team.
  • People’s access to project files will be automatically controlled (and updated in real-time) based upon their roles: Team Members will get read+write access; Visitors will get read-only access.
  • There will be no need for users to manually adjust or manage permissions on any files: Kerika will provide a great contextual layer on top of the cloud storage platform.

Box has another great advantage: it doesn’t have its own proprietary format for storing Word, Excel, etc.  This is a big issue for many enterprises who would like to use Kerika, but don’t want to move away from Microsoft Office format.

If we can the great Kerika user experience, with an enterprise-class cloud service, and the convenience of Microsoft Office, we think we will have a winner on our hands!

So, from a technology perspective a Kerika+Box integration makes a lot of sense. But what about from a market perspective?

When we polled some of current and prospective customers, however, the reaction was somewhat mixed:

  • Folks who were knowledgeable about Box were very supportive. (Good!)
  • Folks who were already using Box were very enthusiastic. (Excellent!)
  • Unfortunately, too many people still haven’t heard about Box… (Not so good…)

Box’s biggest challenge at the moment is name-recognition: far too many folks we talked to confuse Box with Dropbox.

The name confusion vis-a-vis Dropbox is a pretty big issue that we are betting that Box can ameliorate on its own, and Box’s pending IPO should certainly help with gaining greater name recognition and a more distinctive personality in the marketplace.

We are also hoping to build good partnerships with Box resellers, like our friends at cloudPWR, who have long-standing relationships with enterprises that would be great candidates for Kerika’s work management tools.

The full series:

 

Why we are integrating with Box; Part 5: The OneDrive Option

Kerika, Technology, ,

(The fifth in a series of blog posts on why we are adding integration with Box, as an alternative to our old integration with Google Drive.)

This post is shorter than the others in this series for one simple reason: the OneDrive API has no way for third-party apps to manage permissions on folders, and that’s a show-stopper for us in terms of providing a really good user experience.

This limitation was probably more frustrating for us than the case with Dropbox:

  • We are next door to Microsoft: Issaquah is just 15 minutes away from Redmond, which means its easy for us to find people to talk to at Microsoft.
  • Microsoft has a great history of working with third-party developers and a very robust partner program.
  • Nearly all enterprises who are interested in Kerika are long-standing users of the Microsoft stack (SharePoint/Project/Exchange); a Kerika+OneDrive solution would have been a relatively easy sale in terms of internal IT politics within most enterprises.

We hadn’t taken OneDrive very seriously when it was available only as SkyDrive, i.e., when it was available only as part of the full Microsoft stack, but once Satya Nadella became CEO and (coincidentally?) Microsoft decided to un-bundle their cloud platform and create OneDrive, the platform became much more interesting for us.

(We had seen only lukewarm enthusiasm, at best, for the full SkyDrive package, but OneDrive as a standalone alternative to Google Drive, Dropbox, Box, etc. was more interesting.)

Interestingly, the feedback we got also pointed to why adoption of OneDrive might be slower than Microsoft would like: OneDrive is being marketed as part of the Office 365 solution set, and many of organizations we were talking to seem slow to adopt Office 365.

One big reason for dragging their feet is that few enterprises that we talk to are enthusiastic about the implied upgrade to Windows 8.x.

(Upgrading to Win 8.x isn’t a technical requirement, but the sales push from Microsoft is for the full Win 8.x/O365 deal.)

One cause for this, of course, is that there isn’t any great love out there for the new Win 8.x user interface: few end-users seem enthusiastic, and IT folks are very worried about training and support for a radically different user experience.

Even within Washington State’s government agencies — where one might expect Microsoft to have a home field advantage — we haven’t seen any real enthusiasm for OneDrive; in fact, Box is the only cloud service provider to have a master services contract with Washington!

The full series: