Google App Engine Datastore Gotchas
App Engine is generally a new paradigm for webapp developers; replacing sessions with memcache and a schemaless datastore just two elements requiring new thinking for old problems. Unfortunately there are a few more hidden nuisances which have the potential to waste programming time relatively early on. Here’s four of my personal head-bangers:
1. the datastore doesn’t always store Properties
I’ve had trouble with it refusing to store arbitrary entity props unless I assign them in the entity constructor itself (these fields were optional btw). Just setting prop values after initialisation then put() on the ds didn’t write them.
2. fussy filter parsing
returns a NoneType error
silently fails to find expected.
Only the above will return the expected result.
3. Only possible to execute inequality filters on one property per query
This is a pain in the arse if you want to query whether an input date is between two dates stored in a particular entity – officially there was a workaround whereby the date range is stored in a ListProperty (instead of two fields of type DateProperty) and you do the normal check if input is more than the list (greater than at least one element in the list) and less than the list (less than at least one element in the list).
However the App Engine team has now changed the behaviour in the cloud whereby both the ‘>=’ and ‘<=’ filters are operated on each individual list element and not a lazy test over the whole series i.e. where the existence of two list elements that bounded the input date would have been sufficient the following query now only returns the entity if one of the ListProperty elements is an exact match for it:
WHERE date_range >= :1 AND date_range <= :1
Unfortunately this has not been removed from the dev_server datastore, hence it runs perfectly well locally.
4. And hopefully the 1000 record query limit is well-known by this point.
On a more general note, why is it newfangled tech doesn’t build on top of the old stuff? Re-use would get us to where we want to be a lot sooner. I bitched about this on Twitter at the time and I’ll repeat the message here too because it’s worth doing so frankly, I expect new stuff to do the same things old stuff does as well as any “hey that’s cool” new fandangomatrons it bolts on.
Wave’s another case in point. Not ’email invented today’, far from it – it’s left so much cutting-edge crowd-sourced participatory stuff out (as well as how to do IM, namely the KISS principle) – that it actually feels like a retrograde step in many ways. Most in fact.
Bottom line – Google needs to make stuff better.