Enthusiasm never stops

Google App Engine – Datastore performance, and Memcache behavior

Leave a comment

Ever since I’ve been working with Google App Engine, there are two issues which bothered me a lot:

  1. Datastore performance – lots of people have already written about it (see links #1, #2, and #3). Currently, when working with small datasets, it’s far from being comparable even with a slow MySQL database, and you may also occasionally get internal errors, as well as increased latencies. I contacted Google about this, and asked them if the Business customers of GAE who pay for it would get better Datastore performance. Here is what I got as an answer from Nick Johnson, a GAE developer:

    Business customers will receive paid support, which is prioritized, as well as the extra features we announced at I/O. System latency is not any different, however, as we try and make the system as fast as possible for all our users.

    So the bad news is that you cannot make the Datastore run faster, even if you pay.
    The good news is that we are all getting the same service in terms of speed, which is a good thing – when everybody is having difficulties, then the community will eventually find a solution.

  2. Memcache fairness – what happens if another website (on the same server) uses the Memcache service extensively, thus making the Memcache entries of my website expire too quickly, due to the memory pressure. Here is what Nick Johnson from Google replied:

    Memcache is segmented by application. Although there is some variation (so that apps that don’t use any memcache don’t take up usable space), every app is guaranteed a fair share of memcache space.

    Excellent system design GAE engineers. Keep up the good work!

Update: Google App Engine engineers continue to do a very good work indeed! You should take a look at the new features announced with the 1.3.6 release of GAE.

Author: Ivan Zahariev

An experienced Linux & IT enthusiast, Engineer by heart, Systems architect & developer.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s