Tagless final

Abstraction without guilt

Juan M. Serrano (@juanshac)

Habla Computing

Universidad Rey Juan Carlos

To abstract, or not to abstract

DAO

Quoted DSL

Tagless Final

Repository

MTL

Icons made by Freepik from Flaticon

Efficiency vs. Abstraction

Plain SQL

Abstractions

Less code

Easy to learn

Efficient

Tons of boilerplate

Rocket science

Inefficient

Modularity

Evolvability

Idiomatic code

Tagless-final, by Oleg Kiselyov et al.

Tagless-final, by Oleg Kiselyov et al.

http://okmij.org/ftp/

Tagless-final, or abstraction without guilt

​

Tagless-final

DSLs

Reusability

Evolvability

Idiomatic code

Less code

Easy to learn

Efficient

What we will do?

Variations on a theme of Language-Integrated Query

Which are the country capitals whose population is larger than 8.000.000 million?

Outline

VARIATION 1 - Plain SQL

VARIATION 2 - In-memory HOFs

VARIATION 3 - Repository pattern

VARIATION 4 - MTL-based repositories

VARIATION 5 - Quoted DSLs

VARIATION 6 - Tagless-final DSLs

https://github.com/hablapps/tagless-final-tutorial

VARIATION 1. Plain SQL

VARIATION 1. Plain SQL

Query0

[(String, String)]

ConnectionIO

[List[(String, String)]]

IO

[List[(String, String)]]

List[(String, String)]

JDBC program

IO program

Id program

Efficient

Not modular

Lack of unit testing

Not idiomatic

1

2

JDBC

In-memory

VARIATION 2: In-memory HOFs

Inmutable (flat) data model

(alternatively, hierarchical: optics)

VARIATION 2: In-memory HOFs

Comprehensions as a query language

VARIATION 2: In-memory HOFs

Queries can be modularised

VARIATION 2: In-memory HOFs

Queries can be (unit) tested

1

2

JDBC

In-memory

VARIATION 3: Repository pattern

REPOSITORIES

VARIATION 3: The repository pattern

Scala naïve

REPOSITORIES

VARIATION 3: The repository pattern

​

REPOSITORIES

...

VARIATION 3: The repository pattern

Scala naïve

LOG: execute <unnamed>: select code, name, capital from country

LOG: execute <unnamed>: select id, name, countryCode, population from city where id = $1 DETAIL: parameters: $1 = '1'

LOG: execute <unnamed>: select id, name, countryCode, population from city where id = $1

DETAIL: parameters: $1 = '5'

LOG: execute <unnamed>: select id, name, countryCode, population from city where id = $1

DETAIL: parameters: $1 = '33'

...

LOG: execute <unnamed>: select id, name, countryCode, population from city where id = $1

DETAIL: parameters: $1 = '4074'

QUERY AVALANCHE! or N+1 queries problem

VARIATION 3: The repository pattern

Spring data + Hibernate

REPOSITORIES

VARIATION 3: The repository pattern

Spring data + Hibernate

encode the query

in the method name!

Ad-hoc query

(suffers from avalanche in the query result!)

Ad-hoc query

(no avalanche at all!)

1

2

4

MTL-based

Repositories

JDBC

In-memory

VARIATION 4: MTL-based repositories

Towards MTL

Computation type

(list-like)

VARIATION 4: MTL-based repositories

Towards MTL

Imperative API

Stream-based, JDBC-level, doobie instance

Towards MTL

Previously,

synchronous

Now, declarative

And, we can get rid of mocking! :)

State-based transformations

However … did we solve our performance problem?

vs.

By no means! Same N+1 problem

vs.

LOG: execute <unnamed>:

select C.name, X.name

from city as C, country as X

where C.id = X.capital and C.population > 8000000

​

LOG: execute <unnamed>: BEGIN

LOG: execute <unnamed>/C_1: select code, name, capital from country

execute <unnamed>/C_2: select id, name, countryCode, population from city where id = $1

DETAIL: parameters: $1 = '1'

LOG: execute <unnamed>/C_3: select id, name, countryCode, population from city where id = $1

DETAIL: parameters: $1 = '5'

LOG: execute <unnamed>/C_4: select id, name, countryCode, population from city where id = $1

...

1

2

4

MTL-based

Repositories

JDBC

In-memory

VARIATION 5: Quoted DSLs to the rescue!

Inspired the development of the Scala library ...

https://getquill.io/

Program your query as if you were

using case classes …

Macros

LOG: execute <unnamed>:

select C.name, X.name

from city as C, country as X

where C.id = X.capital and C.population > 8000000

​

1

2

4

MTL-based

Repositories

Quoted DSLs

QUEΛ

Tagles-final

DSLs

5

6

JDBC

In-memory

VARIATION 6: Tagless-final

QUEΛ

Forget about the Scala AST, let’s make our own

Base types (integers) and ops (>, ==)

Optional values and ops (exists)

Product types

World model

Multiset-comprehensions (flatMap, pure, filter)

DSLs embedded as type (constructor) classes

World DSL

Syntax

Type system

(phantom types)

Semantics

Forget about the Scala AST, let’s make our own

Multisets (with comprehensions) DSL

Again, phantom types

Semantics

And we can now write our generic query

… although it’s pretty ugly

With some sugar, we recover the same syntax of List-comprehensions

QUEΛ

Standard semantics: suitable for unit testing

Identity representation,

no phantom types

Standard semantics: suitable for unit testing

Alternatively, we may have used

Repr[T]=StateT[List, World, T]

Standard semantics: suitable for unit testing

Please, show me some semantics that could not be given for MTL-based repos!

type F[T]=String

What about a pretty-printer?

We can’t give a Monad instance for a constant functor :(

How do we obtain an `A`?

How do we convert `A` into `String` (no `Show` instance!)

But we can implement the pretty-printer in tagless-final!

We don’t need an `A`, but a

representation of `A`!

For our purpose,

a variable x_i

Pretty-printing in action!

Scala compiler desugars for-comprehensions into flatMap, filter and map, and the Scala run-time desugars the “macro” flatMap into from, etc.

But … what about generating SQL!?

Query0

[(String, String)]

ConnectionIO

[List[(String, String)]]

IO

[List[(String, String)]]

List[(String, String)]

JDBC program

IO program

Id program

MTL-based

Monad, FunctorFilter, WorldModel

Repo WorldModel

Tagless-final DSLs

QUEΛ, WorldModel

Generating SQL from a “normalized” expression is easy!

SELECT C.name, X.name

FROM city as C, country as X

WHERE C.id = X.capital and C.population > 8000000

FROM

WHERE

SELECT

But the for-comprehension query is rather messy :(

We need first to implement a “normalizer”, then the actual query generator

Query0

[(String, String)]

Tagless-final Repr

QUEΛ, WorldModel

Normalized

[Repr, List[(String, String)]

The normalizer re-associates left-binds, much like in the Free monad

Syntactic

normalization

Semantic normalization

Last, we generate the doobie Query0 from the normalized expression

And we can generate and execute the optimum SQL query!

res26: Fragment = Fragment("select x1.name, x0.name from city x1, country x0 where x0.capital = x1.id and x1.population > 8000000")

1

2

4

MTL-based

Repositories

Quoted DSLs

QUEΛ

Tagles-final

DSLs

5

6

JDBC

In-memory

Conclusion

• Do you want to express queries modularly, and yet generate the most efficient queries? Tagless-final DSLs or Quoted DSLs
• Tagless-final is not MTL: computations vs. representations (including computations)
• Do you like domain-driven development, hexagonal architectures, etc.? Learn tagless-final DSLs!
• No free-lunch: after all, we had to implement a compiler. Be ready to invest time to learn the technique and face unexpected challenges!
• Quoted DSLs reuse the Scala AST, which is great! We had to define from scratch DSLs for base types, options, tuples, comprehensions, etc. On the other hand, we have more control over our syntax
• Future steps: No-SQL interpreters, Staging (generating Scala code), etc.

Thanks for your attention!

juanmanuel.serrano@habla.dev (@juanshac)