The theoretical physicist Geoffrey West criticised existing accepted thought in urban theory before coming up with a set of constants that defined the relationship between city size and the output of it’s citizens (Each time a city doubles in size it’s per capita innovation, income, etc increases by 15% – and likewise the negative social actions of crime, pollution…). Previously he found a similar efficiency in biology where the larger an organism was the less energy per unit mass it required to go about it’s life.

It’s this track record in reducing a problem domain to a simple set of rules & constraints that is so impressive. The way in which theoretical physics practitioners go about solving for x – the sense of minimalism that drives the crunching of gigs of data and seemingly chaotic environment into understandable, predictable systems. It’s raw data visualisation in it’s purest form.

And then we have the humble infographic.

A little like urban theory at present far from being a true ‘data’ oriented approach they have morphed from original good intentions into a viral sub-genre seemingly as gratification for the design community while conveying no more than a smattering of anecdotal crumbs as an afterthought. Who cares if it’s insightful so long as it’s typeface looks cool.

Unconvinced?

Look at a recent effort from the popular information is beautiful site. The title leads us to believe there is a message about debt buried in the animated visualisations. Well, that’s a matter for debate. There is a bunch of figures expressed in terms of tetris blocks with area equating to amount. But it’s such a jumble of data without any clever way of giving context or connection beyond one dimensional “Data A vs Data B” scalar quantities.

For instance, the cost of the credit crunch is compared to African debt. One is many times more than the other. Perhaps worth some kind of further analysis, if only to see if there is some kind of basic relationship holding these unrelated numbers together. But it’s actually at the end of the clip, one that started off with everything from Tesco’s revenue to some guy’s net worth to the annual level of corporate tax evasion. The clever bit is apparently getting all these random amounts to slot together nicely as Tetris blocks….

So what? Where is the relationship? What is linking them? Where is the message?

And so here is an animated gimmick that tells us a bunch of unrelated numbers and surprisingly enough doesn’t try to relate them. It’s medium is certainly graphical. Is the data presented really ‘info’? Has the web deluge instead merely managed to dilute ‘info’ to mean any random factoid? I’ll be honest, there are people out there describing themselves as data geeks and I doubt they’ve touched the fundamentals of mathematics since GCSE.

Btw I’m in no way picking on just the above case, it just happened to be the first one I found. The web’s full of similarly vacuous ‘infographics’ that offer little in the way of truly informing people..

Undoubtedly there is an important place for real data graphics in popular science today. Peer through the sea of non-existent ‘insight’ and hubris surrounding the more widely circulated pseudo-data variant. The best graphics can guide the viewer no matter what they’re level of expertise to take in a startling array of data in a matter of seconds and crucially allow expression of context and relationship.

We need to get back to what infographics were developed for: rich visualisations of complex data expressed in a manner that conveys a simple overarching relationship to the observer free from narcissistic clutter and plain randomness.

http://www.networkadvertising.org/managing/opt_out.asp

Incidentally I came by this information by way of Rapleaf, who are one of these ‘database marketing’ companies who engage in datamining browsing habits in a big way. Interestingly I remember them from a TechCrunch article a few years back where they started out innocently enough as an online social networking reputation tool – until eBay didn’t like it encroaching on their space and banned Rapleaf content from sellers auction pages.

If you’re still registered with them and like myself didn’t realise they had morphed into an marketing data company you can delete your account with them here.

Google has done two ‘real-time’ things lately, one good one not so good: Real Time web indexing and real time web search.

With ‘er, hang-on a minute…‘ moments now surfacing in the public domain I find the contrast between the two to be especially important. Google in their traditional engineer style expound the benefits of both in shaving seconds of search: ’11 user hours saved globally each second’; ’50% faster indexing rate of content’; figures that prove the mantra – machines search better than humans.

Machines definitely do the donkey work better than humans. Indexing is a dumb process easily solvable by machine and has been for decades. The migration from batch processing to incremental updating of the search index that Google Caffeine delivers is an essential improvement to real time search.

The Google Instant realtime GUI trick is not such a homerun. Instant brings up a full page of results updated character by character. In cases where the user searches over two or more words – in my experience the vast majority of search – context is vital. Rarely is that context clear until the entire phrase is typed in. This is why google instant, as fast as it undoubtedly is, rarely returns what you’re looking for until you complete your search term.

In any case, the Mind Machine Interface is a delicate thing and only as strong as the weakest link – the human. And it’s the human that has to comprehend this extra flow of data, most of it extraneous.

Google does not yet do the contextual understanding the user must accomplish to use Instant search successfully – and I wouldn’t like them to try, as that would likely involve personalisation based on past searches and as my browsing habits change over time I don’t want past results skewing things.

Incidently an ulterior motive for Google Instant can always be found on the web.

So in conclusion real time is only useful when the data can be transformed into a form easily processed as by the end user. If it cannot it instead serves to exacerbate the problem of information overload rather than lessening it.

The ideal real time UI has yet to be realised.

App Engine is generally a new paradigm for webapp developers; replacing sessions with memcache and a schemaless datastore just two elements requiring new thinking for old problems. Unfortunately there are a few more hidden nuisances which have the potential to waste programming time relatively early on. Here’s four of my personal head-bangers:

1. the datastore doesn’t always store Properties
I’ve had trouble with it refusing to store arbitrary entity props unless I assign them in the entity constructor itself (these fields were optional btw). Just setting prop values after initialisation then put() on the ds didn’t write them.

2. fussy filter parsing

.filter("prop= ",propValue).fetch(1))

returns a NoneType error

.filter("prop=",propValue).fetch(1))

silently fails to find expected.

.filter("prop =",propValue).fetch(1))

Only the above will return the expected result.

3. Only possible to execute inequality filters on one property per query
This is a pain in the arse if you want to query whether an input date is between two dates stored in a particular entity – officially there was a workaround whereby the date range is stored in a ListProperty (instead of two fields of type DateProperty) and you do the normal check if input is more than the list (greater than at least one element in the list) and less than the list (less than at least one element in the list).

However the App Engine team has now changed the behaviour in the cloud whereby both the ‘>=’ and ‘<=’ filters are operated on each individual list element and not a lazy test over the whole series i.e. where the existence of two list elements that bounded the input date would have been sufficient the following query now only returns the entity if one of the ListProperty elements is an exact match for it:

WHERE date_range >= :1 AND date_range <= :1

Unfortunately this has not been removed from the dev_server datastore, hence it runs perfectly well locally.

4. And hopefully the 1000 record query limit is well-known by this point.

On a more general note, why is it newfangled tech doesn’t build on top of the old stuff? Re-use would get us to where we want to be a lot sooner. I bitched about this on Twitter at the time and I’ll repeat the message here too because it’s worth doing so frankly, I expect new stuff to do the same things old stuff does as well as any “hey that’s cool” new fandangomatrons it bolts on.

Wave’s another case in point. Not ‘email invented today’, far from it – it’s left so much cutting-edge crowd-sourced participatory stuff out (as well as how to do IM, namely the KISS principle) – that it actually feels like a retrograde step in many ways. Most in fact.

Bottom line – Google needs to make stuff better.