1 of 23

non

deterministic

bug

or

ndb?

2 of 23

Class Zoo(ndb.Model):

my_cool_monkeys = ndb.JsonProperty(repeated=True)

​

3 of 23

1 - Will this work?

monkeys = zoo.my_cool_monkeys

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

4 of 23

1 - Will this work? Yup.

monkeys = zoo.my_cool_monkeys

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

5 of 23

2 - Will this work?

monkeys = zoo.my_cool_monkeys

zoo.put_async()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

6 of 23

2 - Will this work? Yup.

monkeys = zoo.my_cool_monkeys

zoo.put_async()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

7 of 23

3 - Will this work?

monkeys = zoo.my_cool_monkeys

zoo.put_async()

any_ndb_key.get()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

8 of 23

3 - Will this work? NO!

monkeys = zoo.my_cool_monkeys

zoo.put_async()

any_ndb_key.get()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

TypeError:

'_BaseValue' object

is not

subscriptable

9 of 23

Why?

When ndb sees the synchronous

.get()

​

​

10 of 23

Why?

When ndb sees the synchronous

.get()

it remembers to go through with the earlier

.put_async()

​

11 of 23

Why?

When ndb sees the synchronous

.get()

it remembers to go through with the earlier

.put_async()

which mutates the referenced items into _BaseValues :(

​

12 of 23

This will work

zoo.put_async()

any_ndb_key.get()

​

monkeys = zoo.my_cool_monkeys # this moved down

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

Reading from the entity converts from _BaseValue correctly.

13 of 23

This will work

zoo.put_async()

any_ndb_key.get()

​

print zoo.my_cool_monkeys[0][‘name’]

print zoo.my_cool_monkeys[1][‘name’]

Or better yet,

always read from

the entity.

​

14 of 23

non

deterministic

bug

or

ndb?

This _BaseValue TypeError happened

occasionally

on homepage load.

15 of 23

non

deterministic

bug

or

ndb?

So where’s the

non determinism?

​

​

16 of 23

non

deterministic

bug

or

ndb?

So where’s the

non determinism?

​

No where.

​

​

17 of 23

non

deterministic

bug

or

ndb?

So where’s the

non determinism?

​

No where.

​

We thought it was non deterministic because we didn’t

know how to repro this TypeError bug.

18 of 23

Until we realized the TypeError happened every time devappserver restarted.

​

Which clears the instance cache...

​

​

19 of 23

Until we realized the TypeError happened every time devappserver restarted.

​

Which clears the instance cache...

​

...Which means the FMS does a synchronous get()

20 of 23

https://code.google.com/p/appengine-ndb-experiment/issues/detail?id=208

​

http://phabricator.khanacademy.org/D7275

​

And yes, this monkeys in an ndb quiz format is a hat tip to: http://bjk5.com/post/78634742389/three-minute-quiz-app-engine-datastore-performance

Thanks, ndb!

21 of 23

Bonus - Will this work?

monkeys = zoo.my_cool_monkeys

​

zoo.put()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

22 of 23

Bonus - Will this work? NO!

monkeys = zoo.my_cool_monkeys

​

zoo.put()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

TypeError:

'_BaseValue' object

is not

subscriptable

23 of 23

Bonus - Will this work? NO!

monkeys = zoo.my_cool_monkeys

​

zoo.put()

​

print monkeys[0][‘name’]

print monkeys[1][‘name’]

​

# Breaks for the same reason as scenario 3

TypeError:

'_BaseValue' object

is not

subscriptable