Comments on Gregable.: Google AppEngine Critical Flaw

Hey Greg. You posted this a while ago. Do you have...

2010-01-07T11:35:36.996-08:00

Hey Greg. You posted this a while ago. Do you have any pointers to more discussions about adding sawmill-ish features?

Being an ex-google engineer, I would love to use GAE for projects; however, without mapreduce/sawmill to create reports it just doesn't make sense. Maybe this is why the dev community is struggling so much with GAE's non-relational database approach.

Postgres, I agree with you completely. However my...

2009-04-12T22:38:00.000-07:00

Postgres, I agree with you completely. However my point is the step of "simply write a routine that populates the counter key based on your existing data set" isn't possible in appengine. At least it isn't possible without a workaround like Evan suggests.

There may have been changes in this arena much more recently than my original post. I see some news that appengine offers additional purchasable quota which might satisfy this requirement. I didn't read enough to be sure. I assume that given time, they will offer some better solutions in this area.

Greg,

You run into this same problem with c...

2009-04-12T16:16:00.000-07:00

Greg,

You run into this same problem with counts even when running on most traditional RDMS systems. Counts simply are not performent. Every time you need to add a new counter, simply write a routine that populates the counter key based on your existing data set, and then update it appropriately in your app going forward.

It's pretty much a hack but I haven't worked with any data storage system that lets you run constant COUNT() operations.

Hi Greg,

I ran into the counting issue you ...

2008-08-01T12:27:00.000-07:00

Hi Greg,

I ran into the counting issue you mention a little while ago. I messed up the counters and needed to rebuild the counts. What I ended up doing is splitting the long-running process into smaller ones and either timing them out after 2 seconds or limiting each request to 5 put operations.

Each request would then forward to a new URL with the current index/row in the path.

(I had to raise Firefox's redirect max to an arbitrarily large number to actually have it run without errors.)

But it did work.

A little The Incredible Machine like but I'm finding that quite a few things are with App Engine (the app also gets hit every minute via a web-based cron service, for example.)

I wrote a bit about how to work around this here:<...

2008-05-16T23:19:00.000-07:00

I wrote a bit about how to work around this here:

http://community.livejournal.com/evan_tech/248465.html