Hi! Thanks for dropping by. The talk had a lot more people watching and generated way more interest than I thought it would, so I’ve added a blog post explaining things in more detail, as I covered several libraries without really getting into sufficient detail. I’m a little unkind to some, so I wanted to make sure I explained why. It’s a long, wandering blog post, so good luck ;)

hello.

Kevin Galligan

touchlab

@kpgalligan

ORMs

Why?

Story of discovery

Berlin 2014

Lift with your back

Talk about best practices, and how their origins and propagation are often crap. Think for yourself. Short version, don’t use the helper classes in ORMLite. They’re shit. I know because I wrote them in 2010(ish). At the time there was nothing except Active Android, which didn’t support updating schemas. Took a couple weeks and wrote the ORMLite adapter. If that was published now, you wouldn’t even hear about it, but it was early, so its everywhere.

“have you tried greenDAO? its WAY faster”

I’d been generally aware that other tools existed, but if it ain’t broke and such. However, there was a blog that claimed 1200% performance increase with greenDAO.

in this talk

We’ll cover an overview of the tools out there, how and why performance differs, and give my almost stream of consciousness thoughts on the topic.

not in this talk

I’ll briefly talk about ORM dislike, and why I don’t care and we use them. I will not attempt to convince you of my opinion (much), and I’m not saying this is correct. Its just a type of tool we use, and I think they’ve been pretty harshly maligned.

still hate ORMs? think harm reduction

You shouldn’t smoke, but if you do, try Pall Mall. Its kind of a sensitive topic for some reason, and as we run Droidcon NYC, the NYC Android Meetup, and other events, I find myself often drinking with Android devs. If possible, please refrain from drunkspaining to me why I’m wrong.

“good answer”

sqlite street cred

Adapted ORMLite, have propagated a lot of the early understanding of sqlite and android. ORMLite was written by Gray Watson, whose library is probably one of the most widely used for somebody that doesn’t really code Android. It took a few weeks to write the adapter, and then I kind of moved on. There was nothing else available (sort of), but now we’re mature.

Android is kind of mature

Probably some better options out there

but ORMs?

Generally the arguments are about performance and the mismatch. The other experience I had was working for a Hibernate guy.

know your tools!

Performance issues almost always had to do with doing stupid things and not understanding your tool. If anything, you need to understand your underlying tech better. Not going to try and prove that. It just is.

square peg, round hole

These two things don’t work well together. Expecting an ORM to map whatever your perfect OOP design is to relational data is expecting a lot. If you’re going to use SQL, you have to fit things somehow. You can carve square round adapters, or...

roundish pegs

Make roundish pegs. We use ORM tools to help model data early, then optimize and/or drop down to SQL as needed. Those cat pics aren’t going to post themselves, so you just need to pick a lane and get coding. Not even going to talk about alternate data stores. SQL has survived a lot of alternate options as a solid general purpose tool. Its maybe not ideal, but it works and the vast majority of data you deal with lives in one (ish), so whatever. Defending SQL is like defending water. Imagine if it was just gone. You can’t, really.

features

What features do we generally expect

map tables to objects (basic fields)

map relations between objects

generate DDL

query apis

other (content providers, rx adapters, etc)

query apis are the jazz improv of the apis. All over the place.

comparisons

basic performance

model pollution

primary keys

foreign relations

integration difficulty

stability

threading

anti-patterns

wizard of OZ

kitchen sinks

“API for the rest of us”

WOOZ is hiding the underlying framework and/or changing the way things are done for subjective reasons (ie. no mechanical advantage). kitchen sinks are frameworks that add a tons of unrelated features. Simple apis are confusing useful complexity for “bloat” and generally form and function under the “this is too complicated” falacy.

implementations

which we’ll discuss

ORMLite

GreenDao

DBFlow

Squidb

SugarORM

Cupboard (a bit)

Squeaky

Realm

There are more, and I’ll be posting more detail about these and others over time. Had this idea that hardware and games have the intro experience, and also have unboxing videos. APIs do a better or worse job of that. Since I’ve done a lot of work with sqlite and android, I figure if I can’t get the basic features of your framework within an hour or two, you’ve failed. Will be posting some “unboxing” videos.

what am I looking for?

Just a tool. Should help, but won’t make you a better person and solve all your problems. Should be a lever on top of sql, providing mechanical advantage, reducing boilerplate, and generally make things easier, but you should still be able to directly work with sql (and/or other tools). Quick early dev, optimize later.

complexity

OOP to relational is a compromise, which is worse on android. A lot of the complexity is around foreign relations, both singular and especially collections. Both selection performance, lazy vs eager, but worse when being strict about thread separation. If avoiding background thread data, you can’t do lazy loading, and need to be able to selection a snapshot of your data, which out being anemic, or selecting large amounts of your object graph.

too complex?

multiple primary keys

many to many

value collections

inheritance

version

caching

Many of these frameworks came from people who used server orms. Mobile db’s tend to be way simpler than server db’s, and don’t need the kind of concurrency a server db would. A lot of these features add bloat and confusion (IMHO).

personal preference

Sometimes people just like some api forms over another. I briefly looked at another api that will remain nameless. Most of the readme text was explaining how it was “beautiful”, “simple”, “api for humans”, and I couldn’t actually tell what it did differently than anything else. Rant alert. Show, don’t tell. Show me your api is “beautiful”. Rant over. Sometimes people just won’t like your shit, functionality be damned. Keep this in mind as, since I worked on a few of these things, I’m hugely biased. Difficult to avoid, but I’ll do my best.

performance

“X is faster than Y!” mostly BS

Source-gen 50%-100% > reflection

foreign references are tricky

Speed comparisons have been done on simple table queries. Need more testing, but general results. Refleciton is slower, but its not horrible (on my OP2 Lollipop phone). Foreign references can be useful, but tread lightly.

Realm in particular kind of got under my skin. They claimed 10x-100x performance, and called out ORMLite specifically, so we tried it out. First off, this graph is clearly wrong. greenDAO is faster than ORMLite. Ran some numbers.

Realm didn’t have their benchmark public, so I ran this one. Didn’t go well. Many tweets later, wound up meeting the nice devs at realm, they came to Droidcon NYC, etc. Added a bunch of features, optimized in a later release. Kind of caught them at the wrong time. However, there’s some bullshit out there. I got a copy of their benchmark (which still isn’t on the site), and need to play with that and see how they measure, but summary, performance isn’t that simple. Also, I think their chart numbers are totally not true, and are still on the site, so...

ORMLite - Basics

  • Generic DB ORM, with adapters
  • Mature, lots of apps
  • Reflection based
  • Slow-ish
  • Single primary key
  • Low/medium model intrusion
  • Foreign references OK, but..
  • Doesn’t support views(?) or optimized foreign joins
  • Don’t use the Context base classes

Built for many DB’s, not just sqlite or Android. That leads to lots of extra code and weird features. Model intrusion is low, but annotations are verbose. Foreign refs OK, but basic, and can only control how many levels to go.

ORMLite - Use if

  • You want a lot of SO support
  • You like mature apis
  • You understand foreign relations
  • You like a lot of features you probably don’t need
  • You’re OK with reflection and not being “fastest”

I think it comes in at over 2k on the method count

GreenDAO - Basics

  • Complete model intrusion
  • You write code that generates your models
  • Fast-ish
  • Not apt, though (so custom build step)
  • Setup and config not entirely trivial
  • Single primary key, only long or Long (?)
  • Apparently supports single-query eager fetch

Don’t know greenDAO all that well. Its definitely not my breed of dog, so I haven’t used it much. Only dao that supports single query eager fetch, but not flexible.

GreenDAO - Use if

  • You don’t want to write your shitty model objects, but don’t want to write SQL
  • Think apt is for hipsters

If you don’t like “roundish pegs”, but don’t want to write SQL, you could use this. Definitely just not my breed of dog.

DBFlow - Basics

  • Source gen, fast
  • Medium+ model intrusion
    • Open helper hidden (Wizard of OZ)
    • Weird migrations
  • Poor foreign relations
    • How to control eager fetch? Chasing tail fails.
    • Verbose
  • Multiple primary keys
  • Kitchen Sink/Wizard of OZ
    • Thread manager?
    • Content provider generator
  • Caching?

Thread manager for “transactions”, but hiding the db prevents you from doing your own thing. Unfair, as you can get it, but the docs push you towards using their threading model (which has priorities, etc). Do you eval a framework on what you know or what they say in docs? I do need to dig deeper on this framework, but as this was a search for what we’d use, we wouldn’t use this, so I kind of stopped.

DBFlow - Use if

  • Fast data load (on simple tables)
  • You like the Wizard of OZ thing (personal pref)

SugarORM - Basics

  • Simple
  • Slow
  • Minimal features
  • No config (possible)
  • High model pollution
    • class Message extends SugarRecord<Message>
    • ‘id’ column fixed, why not ‘_id’?
  • Hides most things (Wizard of OZ)
  • High aspirations package com.orm;

Had to hack for tansactions. BTW, always insert/update on transactions. Cupboard thoughts.

SugarORM - Use if

  • You don’t understand SQL
  • Won’t use any foreign relations
  • Don’t

Concerning

SquiDB - Basics

  • High model intrusion
  • Slow-ish*
  • No foreign relations
  • Breaks 4th wall
  • Totally NOT an ORM

* Couldn’t figure out how to add an index, so it would be a little slower than measured if added

SquiDB - Use if

  • Not sure
  • Doesn’t distinguish on features
  • Need to eval more, though

Squeaky - Basics

  • APT port of ORMLite
  • Source-gen, but maintains ORMLite-isms
  • Fast
  • Single field primary key
  • Improvements on ORMLite
  • Immutables *
  • Views/ad-hoc query objects
  • foreign auto-refresh map

Squeaky - Maybe

  • live-mode (maybe)
  • join eager fetches

Might point out that lack of join eager fetch degrades performance significantly, and should be in there.

Squeaky - Use if

  • You like early software and no support
  • You think code is the best documentation
  • You use ORMLite
  • You wrote a different ORM and want to throw shit back at me

We’re using this in production apps. Its in the droidcon app here, although old version. Still deciding on future. See if there’s interest. Supporting a framework like this is lots of work.

Realm - Basics

  • Not SQLite (or row based)
  • Medium-high model intrusion
  • Single field primary key
  • Queries tied to thread (main thread data access?!)
  • Fast (but depends)
  • Some things much faster (counts for example). Column oriented
  • Low level api seems REALLY fast, but proprietary
  • MVCC

Don’t know if low level api will be productive like SQL.

Ream - Use if

  • You can tolerate risk
  • You want a simple coding model
  • You’re OK with strict model design
  • Think sqlite sucks

Main Thread?

Answer is “memory mapped”, but I would assume most devs don't really understand that. Is it time to revisit performance, main thread, and flash? I’m very strict on this, because my first production app was tested on a Galaxy S, which was crazy slow. We’ve had a switch in file system, and much better hardware. Can we do some access in the main thread without risk? A live/lazy-load ORM would be much easier to implement.

threading and foreigns

  • lazy loading is simple
  • thread separation requires a “snapshot” of data
  • simple solutions pull too much (or not enough)
  • complex solutions are complex

29 Million?

https://www.crunchbase.com/organization/realm-2#/entity

What’s the money for? What’s the goal? What happens if things don’t go well? However, if anything should make you feel comfortable about the long term viability of native mobile dev, somebody thought to spend 29 million on building another mobile db.

numbers!

numbers!

let me know if i screwed up

about the benchmark

  • very basic
  • only tests solo table performance (no foreign)
  • two tables, 2k and 20k writes
  • read test just the 20k table
  • not entirely clean between runs because of architecture
  • kind of useful, kind of BS (like all benchmarks)

write

read

Realm is really fast if reading less values.

Kevin Galligan

@kpgalligan

@TouchLabNY

Benchmark: https://github.com/touchlab/android-orm-benchmark-updated

touchlab.co

bye.