— aleatory

Google App Engine Datastore Gotchas

stormclouds gather
image courtesy johnson7

App Engine is generally a new paradigm for webapp developers; replacing sessions with memcache and a schemaless datastore just two elements requiring new thinking for old problems. Unfortunately there are a few more hidden nuisances which have the potential to waste programming time relatively early on. Here’s four of my personal head-bangers:

1. the datastore doesn’t always store Properties
I’ve had trouble with it refusing to store arbitrary entity props unless I assign them in the entity constructor itself (these fields were optional btw). Just setting prop values after initialisation then put() on the ds didn’t write them.

2. fussy filter parsing

.filter("prop= ",propValue).fetch(1))

returns a NoneType error


silently fails to find expected.

.filter("prop =",propValue).fetch(1))

Only the above will return the expected result.

3. Only possible to execute inequality filters on one property per query
This is a pain in the arse if you want to query whether an input date is between two dates stored in a particular entity – officially there was a workaround whereby the date range is stored in a ListProperty (instead of two fields of type DateProperty) and you do the normal check if input is more than the list (greater than at least one element in the list) and less than the list (less than at least one element in the list).

However the App Engine team has now changed the behaviour in the cloud whereby both the ‘>=’ and ‘<=’ filters are operated on each individual list element and not a lazy test over the whole series i.e. where the existence of two list elements that bounded the input date would have been sufficient the following query now only returns the entity if one of the ListProperty elements is an exact match for it:

WHERE date_range &gt;= :1 AND date_range &lt;= :1

Unfortunately this has not been removed from the dev_server datastore, hence it runs perfectly well locally.

4. And hopefully the 1000 record query limit is well-known by this point.

On a more general note, why is it newfangled tech doesn’t build on top of the old stuff? Re-use would get us to where we want to be a lot sooner. I bitched about this on Twitter at the time and I’ll repeat the message here too because it’s worth doing so frankly, I expect new stuff to do the same things old stuff does as well as any “hey that’s cool” new fandangomatrons it bolts on.

Wave’s another case in point. Not ‘email invented today’, far from it – it’s left so much cutting-edge crowd-sourced participatory stuff out (as well as how to do IM, namely the KISS principle) – that it actually feels like a retrograde step in many ways. Most in fact.

Bottom line – Google needs to make stuff better.

  1. cruzalinhas » blog do chester says: 5 June 20102:39 am

    [...] bata com esse valor calculado. Trocamos quatro comparações de inequalidade (que o App Engine não iria conseguir combinar mesmo, por estarem em dois campos diferentes – latitude e longitude) por uma comparação de igualdade (ok, numa string, mas bem [...]

  2. Some App Engine Hacks at aleatory says: 23 August 20103:14 pm

    [...] get into the fundamentals of cloud computing. But as well as having it’s share of important technical restrictions there are also some structural kinks like no support for naked domains and a limit of 10 apps per [...]

  3. [...] starting out if you’re doing so for the first time it’s worth noting again a few of the old issues I have had with App Engine when I first began working with [...]

Submit comment