1 of 13

Composable, Malleable & Lean Compute Infrastructure for the Future

Mehmet Ozan Kabak, CEO & Founder

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

2 of 13

About Synnada

2

Mehmet Ozan Kabak, Founder & CEO

10+ years experience in machine learning, stream processing, and big data -- academic background in dynamical systems and optimization algorithms

​

Sami Can Tandoğdu, Founder & COO

10+ years experience in enterprise software, data &ai, cyber security, scaled commercial teams for high growth software startups

Infrastructure Team

3 members

Product Team

4 members

ML Team

4 members

Founded

2022

Investors

Collective Spark, DayOne, 500 EE, Expeditions,

Team

Company

HQ

Austin, TX

Open Source

Apache DataFusion

Unified data processing

1 PMC + 3 contributors

Mithril → Coming soon

ML compiler

4 contributors

Raised

$4.3M

Apache DataFusion • Belgrade Meetup • 27 September 2024

3 of 13

We are in the Data Chasm�Building/maintaining data intensive apps is complex

3

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

4 of 13

Example: Creating a traffic forecasting app

4

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

5 of 13

Enter → Apache DataFusion!

5

⚛️

Modular & Extensible

…and Performant!

Rust based

Amazing community

  • Like LLVM for data systems
  • Foundational yet rich in OOTB features
  • Arrow-native, Python friendly
  • Malleable

​

​

​

Nearly 6k!

When we joined…

Thanks to DataFusion we can deconstruct the data chasm!

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

6 of 13

Apache DataFusion: Our work

⚛️

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

7 of 13

Aras: Unified data processing

7

⚛️

Copy and paste, change source

Your PosgreSQL query runs on Kafka!

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

8 of 13

Synnada Composable, Malleable & Lean Infrastructure

8

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

9 of 13

Mithril: Composable ML library

9

⛓️ Build ML models with different backends, infrastructure, optimizers

🐞 Acts as a compiler, helps you debug without dry-runs

🎼 Compose, mix-and-match different models

🧩 Enables UDF generation and model interoperability

🚦Aligns any ML model with DataFusion

📖 Built in-house, soon to be released open source!

​

🤖

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

10 of 13

Workbench: Programmable data blocks

10

🎹 Composable blocks for data applications

😎 Can be used via UI or through APIs

⛑️ Works locally – in the future on public and private clouds

☁️ Prototype data applications locally

➡️ OOTB: Fault tolerance, low latency, (WIP) distribution

🏹 Community edition will be released in Q4

​

🧱

Start prototyping locally

Graduate to cloud

Distribution

Fault tolerance

AutoML

Workspaces

Collaboration

Access control

Observability

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

11 of 13

Road Ahead for DataFusion: Our thoughts

  • We will continue contributing, no letting up!
  • Performance – performance – performance
  • Easier extensibility (e.g. simple UDFs)
  • Increase maintainability
  • Keeping the core simple but powerful
  • Avoid unnecessary code/API churn, be downstream/fork friendly

11

Apache DataFusion • Belgrade Meetup • 27 September 2024

Apache DataFusion • Belgrade Meetup • 27 September 2024

12 of 13

Team

  • We are starting hiring globally
  • Both Data & ML infra teams are expanding
  • Looking for engineers/scientist that are willing to work on hard problems!

12

ML Team

4 members → Growing to 6!

Product Team

4 members

Will be working on:

- Upstream DataFusion

- Spark 2.0

Data Infrastructure Team

3 members → Growing to 5!

Will be working on:

- Model training and inference infrastructure

Apache DataFusion • Belgrade Meetup • 27 September 2024

13 of 13

Thank you

Mehmet Ozan Kabak, CEO & Founder

ozan@synnada.ai

+1 (650) 766-3855

Sami Can Tandogdu, COO & Founder

samican@synnada.ai

+90 532 740 78 62

Apache DataFusion • Belgrade Meetup • 27 September 2024