1 of 38

Agenda

Intro aka WHY are we all doing this again?
Introduction - FVM (compute over state)
Introduction - Bacalhau (compute over data)
Introduction - Lilypad
Let’s make AI in real time!

2 of 38

… what to do with it

Data, Data, Data!

Ally Haire

Developer Relations Engineer | Lilypad PM

DeveloperAlly

🦋

3 of 38

FVM, Bacalhau, Lilypad

Data, Data, Data!

Compute over State and Data

Ally Haire

Developer Relations Engineer | Lilypad PM

DeveloperAlly

🦋

4 of 38

Decentralise the Internet

Filecoin Vision

5 of 38

“The properties of the internet & it’s technologies define our human rights”

Juan Benet 2022

This quote from Juan Benet (who is the founder of Protocol Labs & Filecoin) really sums up one of the reasons I went into tech originally, and that is because of just how fundamentally innovation and technological advancement can change and improve our human lives.

�Tech creates new frontiers for us (like the ability to take photos of black holes or build sustainable energy sources), it can help us solve increasingly complex problems (with faster and more powerful computing crunching over more and more stored and available data) and it also has the capability to enable changes in not just our daily lives but also in society itself (because those daily changes really add up!).

That’s also, though, why it’s so essential that we are creating a future for the underlying technology that we use daily to provide an open and fair web that protects our human rights and helps move humanity forward.

=====

And this a vision that was shared by the early web and the open source movement as well - both of which were founded on being a cooperative, accessible & free sharing of knowledge and information.

========

��And we don’t have to look back too far to see evidence of that.. In the past 100 years advances in healthcare and inventions liike the sewing machine, washing machine, the printing press and then on to the more modern era of computers, the internet and mobile phones have drastically improved the lives of millions – paving the way for major social changes and with it a rethinking of the value and role of people in society as well. ��For example, Household automations gave freedom & empowerment to women across the world. The internet has given mass accessibility to education and ideas as well as enabling collaboration on a global scale - & i certainly would not be able to work collaboratively with people from all across the world from wherever I am without it.

So, to me the real goal is not tech in and of itself. The real goal is people and improving the human condition by leveraging tech and other scientific breakthroughs to do so, in a manner that democratises and provides equality.

This is also the mission of most distributed systems

��=========��And AI and ML is the new frontier here.��That’s also, though, why it’s so essential that we are creating a future for the underlying technology that we use daily to provide an open and fair web that protects our human rights and helps move humanity forward. And that’s never been more important than today where how we are able to use and interact with technology and whether we have access to it at all and whether that access is censored or not, is more important than ever.��So this is the vision we hold here at Protocol Labs and the Filecoin Foundation - a vision for the underlying technology of the future to provide a trustless system that enables the values of fairness, empowers ownership of your own data and promotes an open and accessible web.��And this a vision that was shared by the early web and the open source movement as well - both of which were founded on being a cooperative, accessible & free sharing of knowledge and information.

So this is what drives us to keep building here at the Filecoin Foundation & Protocol Labs and it’s a mission I am genuinely here for and why I’m working in web3!

�And I’m sure many of you maybe have similar stories of why you’re here today as well.��=======

And this a vision that was shared by the early web and the open source movement as well - both of which were founded on being a cooperative, accessible & free sharing of knowledge and information.�=================�

6 of 38

Data, Data, Data!

Data, information and knowledge are some of the most important assets of our connected era and have become critical to human development & cooperation

As you all would know - data is an absolutely essential part of our lives now, so enabling distributed storage and compute mechanisms for data is part of the fundamental mission we’re working on at Filecoin & protocol labs, That of building the tech to enable open, scalable and distributed data systems - which allows for open innovation & provides the base infrastructure needed for a truly open web.

In practice, that means that not only is it essential that we create a distributed storage and retrieval network that enables data sovereignty and provides innate blockchain properties like censorship resistance and trustlessness, - which we’re doing with Filecoin, as well as providing a content verification system with IPFS, but it also means enabling computation of data in a way that aligns with this open web mission - which is where FVM, Bacalhau & Lilypad fit in.

=======

Processing and making sense of that data is essential for not just business profitability and competitive advantage but also to solve some really complex human challenges.

As we all also are aware, data has increasingly become the domain of centralised monopolies who not only use our data for their own profitability and create centralised attack vectors, but also deny open innovation on data sets.

7 of 38

Filecoin is building the foundation for a decentralized internet; with 4 key components

Decentralized Internet

Filecoin Virtual Machine (FVM) Live
Will enable smart contracts capabilities on Filecoin, driving non-linear network demand

In development

Target is Arbitrary compute over data*
Bacalhau is Live
Lilypad Network

Verifiable�Storage

Computation

Filecoin mainet launched in October 2020
16 EiB of available storage capacity, with 63 PiB utilized
Continued growth in Filecoin ecosystem

Currently 1Bn weekly retrieval requests through gateways
Next phase is CDN-quality retrieval (‘000s of retrieval providers, with <1s content delivery)
In development

Retrieval

Programmable�Applications

So this is really the motivation behind what we are working towards at Protocol Labs & Filecoin.

Much like the deFi movement is aiming to democratise the financial industry and inject more trust and confidence into it, Filecoin is aiming to democratise data and the internet by building out a purpose built, internet scale, distributed network for data.

We’ve so far built out the largest decentralised storage network globally in Filecoin - as Jenks was mentioning in his talk earlier & The Filecoin Saturn retrieval network & market is also well underway to allow for faster movement of data across the globe - and even planets in the future.

The third pillar of this mission is programmable applications - and this is where we are turning filecoin into a layer 1 blockchain with smart contracts and programmale applications being enabled by the Filecoin Virtual Machine or FVM.

And the 4th pillar of this mission is computation - & to me this is where data becomes truly useful! Afterall chatgpt would be nothing without the ability to compute over and train on datasets! And we’re enabling this with projects like Bacalhau and Lilypad.

============

��So it’s my strong view that programmable applications & Distributed computing will unlock the next generation of web-scale decentralised applications and bring the power of robust compute to web3.

And this is where FVM, Bacalhau & Lilypad live!

And with programmable applications enabled by FVM and verifiable Bacalhau & Lilypad data computation, we’re also enabling tokenised data marketplaces, open source machine learning algorithms and datasets and many more applications.�

=======

And this is the mission of Filecoin & Protocol Labs.

And this is summed up by this slide on the Filecoin & Protocol labs vision to build a decentralised internet with 4 key components.

We want to build the foundation for a decentralised internet that allows for the democratisation of data and of access to data - non-dependent on restraints of geography, resources or level of access to centralised services. ��And one that not only solves some of the big social and ethical challenges of the web2 cloud data stack, but also some of the engineering challenges, including pricing, resilience and distribution.

We’ve so far built out the largest decentralised storage network globally in Filecoin - as Jenks was mentioning in his talk earlier & The Filecoin Saturn retrieval network & market is also well underway to allow for faster movement of data across the globe - and even planets in the future.

The third pillar of this mission is programmable applications - and

And the 4th pillar of this mission is computation - to me this is where data becomes truly useful! Afterall an LLM is nothing without the ability to compute over and train on datasets!��It’s my strong view that Distributed computing will unlock the next generation of web-scale decentralised applications and bring the power of robust compute to web3.

And this is where Bacalhau & Lilypad live!��

8 of 38

FVM brings on-chain user programmability to Filecoin

Bacalhau brings off-chain verifiable compute - both

unlocking the next layer of innovation and value

Storage

Computation over state

On-chain

Computation over data

…

Off-chain

Storage is only the start…..

Bacalhau & Lilypad

9 of 38

Intro to FVM

Compute over State

10 of 38

The FVM project delivers

on-chain programmability

to the Filecoin network.

11 of 38

Built to support multiple runtimes

Based on WebAssembly�

FEVM - Filecoin EVM!!

12 of 38

NEW!! Smart Contract Deal Making Flow�No Third-Party Required

So what does this mean in context?��Well this slide illustrates some of the powerful use cases that the FVM enables.

And it’s interesting because some of these use cases are actually really hard to build with traditional cloud and data models.��Some of these use cases include

Data DAOs, the dataverse & tokenised data sets -> so basically building up value around important datasets by tokenising them.�You can then capture and represent the value of those datasets to society. And could even exchange those data tokens between peers and request computation services on that data like validation, joins, analysis, feature detection & extraction moving into machine learning. -> and I think you see im veering off into compute with this one and maybe some of the possibilities that live there!
There’s also use cases for NFTs minted, exchanged and stored under a single network.
And for Replication workers which make automatic deals with storage providers with specific levels of replication of a specific data set or
Trustless reputation systems and storage bounties are also possible - so having storage providers compete to host client data - flipping the traditional data storage model.
Futures & derivatives on storage that compose in a defi-like fashion and
Conditional loans for sector pledging on storage or time-locked data sets are also possible on the FVM.

=============

- This includes unlocking the possibility for Data DAOs, the Dataverse, //and tokenized datasets.�(Imagine Kickstarting the dataset economy by tokensiing data sets. You could then capture and represent the value of those datasets to society. And you could even exchange those data tokens between peers and request computation services on that data like validation, joins, analysis, feature detection & extraction moving into machine learning.

�Another exciting possibility unlocked by the FVM is

- Trustless reputation systems�Imagine an overlay network of nodes that could patrol the network performing random deals with storage providers in order to gauge key metrics like Quality of service guarantees, performance, latency and region details. �These nodes can then store these reputation scores on-chain making them traceable and verifiable and enabling users to decide on the terms of service - and hence the storage providers, that they want to use for their data.��Another use case we’re excited about unlocking is

- Replication workers, �(Imagine, anyone is able to write a new smart contract that makes new deals to maintain a specific level of replication of that dataset in the network. You could ensure the resiliency of your data by ensuring it is always stored n times automatically or be able to transfer your data just once into the filecoin network and have a trustless actor replicate and send that to multiple storage locations ‘n’ times.��And you could even create a user defined policy of rules for that data to obey - specifying things like region & location or latency and even price.. and this can all be built right into your smart contract flow. ��======��- Another use case we’re excited about is Smarter Storage markets which will add richer functionality to storage markets by enabling features like auto-renewal of deals or self-repairing deals in the event of sector issues or even time-locked or event-driven data deals where retrieval only occurs under a specified timeframe or event trigger)

And there’s many many more use cases to unlock with FVM including�- NFTs minted, exchanged, and stored under a single roof.

- Storage bounties and auction mechanisms

- enabling L2 bridges

- Futures and derivatives on storage, that compose in a DeFi like fashion

- Conditional loans for sector pledging

====

So really, once you start adding computation onto the filecoin blockchain with FVM, you have both computation and storage under the same roof - and this is really going to be one of the big strengths and differentiators of the Filecoin network.

And, if you think about it, use cases like these are hard, if not impossible, to build on centralised and cloud storage systems. And this is where Filecoin with the FVM is really going to shine with the kind of possibilities it can unlock

=======

Some of the use cases we’ve envisioned are shown in this mind map above. And it’s interesting because some of these use cases are actually really hard to build with traditional cloud and data models, so we’re excited about the possibilities the FVM will unlock.��Some of these use cases include

Use Cases (need use cases slide)

Data DAOs, the dataverse & tokenised data sets.
Or NFTs minted, exchanged and stored under a single network.
Replication workers - imagine - anyone will be able to write a smart contract that makes automatic deals with storage providers with specific levels of replication of a specific data set or
Trustless reputation systems and storage bounties will also be possible - imagine having storage providers compete to host client data - flipping the traditional data storage model. This will be possible with FVM.
Futures & derivatives on storage that compose in a defi-like fashion
Conditional loans for sector pledging on storage or time-locked data sets

There are so many things that can be built on the FVM!�

13 of 38

Checkout some FVM resources

https://linktr.ee/filecoinVM

https://github.com/filecoin-project/fevm-hardhat-kit

https://github.com/filecoin-project/fevm-foundry-kit

https://github.com/filecoin-project/cod-starter-kit

14 of 38

Intro to Bacalhau

Compute over Data

15 of 38

Computation over State vs Computation over data…

Computation over state (on-chain compute):

FVM enables computation over state.

Allows monitoring and automation of chain activities such as monitoring and automating storage deals and markets and enabling tokenisation of data for dataDAOs + more.

Computation over data (off-chain, verifiable compute):

Bacalhau enables complex computation over data - any data

In the context of Filecoin & FVM, Bacalhau will allow processing of activities monitored on chain in a verifiable and deterministic manner.

It enables a rich layer of data processing for Filecoin & FVM

FVM enables computation over state for Filecoin, bringing programmable applications and logic to the Filecoin chain - which in itself is pretty awesome, because it helps enable more user-friendly developer opportunities. And as I mentioned, it also enables all sorts of use cases, both those already available on other L1’s or in the broader web3 community and also some that are really unique to the Filecoin ecosystem such as the opportunity for data markets and DAOs and programmable storage deals and automation.

One thing that blockchain technology really does not do inherently well though, is data processing - and this is just a factor of the cost of computing over a distributed system that needs consensus to provide other powerful properties like trustlessness and censorship resistance.

And data processing and making sense of that data in a way we can verify, is essential for not just business profitability and competitive advantage but also to solve some increasingly complex human challenges as well as just to make our everyday lives a little easier or a little more fact-based (yes I’m looking at you fake-news pandemic and deep fakes!). ��And this is where Bacalhau & Lilypad can help. Both of which enables off-chain computation over data without giving up the decentralisation or verification values inherent to IPFS and Filecoin

16 of 38

Bacal.. - what ??

Bacalhau is a network of open compute resources available to serve any data processing workload

It’s simple to use (you don’t need an AI degree!)
Requires minimal operational overhead or setup
It’s decentralised-first (or edge-first) principled
Aims to provide efficient distributed computation with batched tasks

Learn more about Bacalhau!

@BacalhauProject

https://youtu.be/RZopDyTJ1pk

17 of 38

Lilypad

Off-chain compute with on-chain guarantees

18 of 38

Lilypad is

A fully distributed compute network - oh my! Leverages Bacalhau & key features of blockchain for secure scheduling, verification & payments (jobs + nodes)

19 of 38

The Lilypad Network Aims to

Develop a trustless, distributed compute network that enables internet-scale Data processing, AI, ML and other arbitrary computation from blockchains, while unleashing idle processing power and unlocking a new wave of data economies.

With Lilypad we want to build a trustless, distributed, compute network that enables internet scale Data processing, AI, ML & other arbitrary computation to be callable from smart contracts - with blockchain guarantees, while unleashing idle processing power and unlocking a new wave of data & compute economies in web3.

This is a decentralised compute network where jobs can be called from a smart contract AND where the jobs can consume input data - and we think really unlocking the next generation of web-scale decentralised applications and bringing the power of robust compute to web3.

=====

��Put another way, we are aiming to build a marketplace for compute jobs, and empower web3 applications with internet-scale compute capabilities.��

=====

this is a decentralised compute network where jobs are called from a smart contract AND where the jobs CONSUME INPUT DATA

20 of 38

Introducing…

Lilypad v1 Testnet!

The Lalechuza Testnet is live!

IPC Larana testnet on IPC soon

🎉

21 of 38

Lalechuza Lilypad Testnet Tech

The Lalechuza testnet is built on geth & allows arbitrary untrusted nodes to join the network
The Larana testnet will be released soon & is built on IPC (the scaling network for Filecoin) - and will be the path to mainnet
It works with “modules” which are the (currently deterministic-only) jobs of Lilypad

docs.lilypadnetwork.org/research-and-vision/whitepaper

22 of 38

Lilypad Current Architecture

Bacalhau compute

EVM compatible blockchain

Hardhat/geth -> IPC + FEVM

Bacalhau requester

Bacalhau compute

Bacalhau client/dashboard

BACALHAU CLIENT

See more: docs.lilypadnetwork.org

23 of 38

Show me the code

Running Compute Jobs (& nodes!)

on the Lilypad Network

24 of 38

Cowsay Module demo! [CLI]

Input:

> lilypad run cowsay:v0.0.1 "hello lilypad"

Output:

> ipfs CID

The first thing we enabled on lilypad was obviously cowsay. I mean - seems obvious right? It’s udderly fantastic afterall! LOL jokes aside, this is really just the hello, world of Lilypad.

Here - we’re running Lilypad in a terminal CLI. It’s important to note though that Lilypad is 100% triggerable and useable directly from a smart contract too though.

All we need to do is install the Lilypad cli with one line of code and then we can add our wallet private key to our terminal and run the commands above!

Thats it!

===

�Firstly, we’ll need to install the Lilypad CLI

curl -sSL -O https://bit.ly/get-lilypad && sudo install get-lilypad /usr/local/bin/lilypad

Then we can add our wallet private key to the terminal

export PRIVATE_KEY=<the private key you copied>

Then we run the Cowsay Module….

And it’s well, udderly fantastic I think you’ll agree?

25 of 38

[CLI] Stable Diffusion Module demo! SDXL v0.9

Input:

>lilypad run sdxl:v0.9-lilypad1 '{"prompt": "an astronaut riding on a unicorn", "seed": 1}'

Output:

> ipfs CID

26 of 38

[Smart Contract] Stable Diffusion Module demo!

docs.lilypadnetwork.org

27 of 38

Fine-tuning a Model Module demo! LoRA

Input:

Output:

> ipfs CID

Wait…. A decentralised compute network callable from smart contracts that ALSO consumes INPUT DATA 🤯🤯🤯🤯

We’ll also shortly be releasing a finetuning demo.. so we’re not only going to run an ML model on lilypad, but we’re going to use it to fine-tune a model using user input! This is a pretty impressive feat for a distributed network, because not only are you able to run pretty much any deterministic job on Lilypad, But we can also finetune models on the network - and that includes being able to consume input data.

So this input includes lora_cid which is the output cid of the first step

And images_cid is a zipfile of jpegs

So this is a decentralised compute network where jobs are called from a smart contract AND where the jobs can CONSUME INPUT DATA.�

That is a huge milestone for distributed computing!

This is also truly decentralised AI. Something that we are passionate about on the Lilypad Team…

(and side point - we will be tackling non-determinism in future)

28 of 38

LLM Inference Module - [Community-contributed]

CarpAI was built at AugmentHack

And gives LLM capabilities

to Lilypad!!

https://devpost.com/software/carpai-fmecgh

Thanks Leo!

29 of 38

Run your own node!

You can also add your own node to the Lilypad Network and run your own mediator nodes - if you don’t mind just earning testnet eth :P.��See the docs for more info on this.

docs.lilypadnetwork.org/

lilypad-v1-testnet/verification-mediation-and-job-pricing-on-lilypad-v1

30 of 38

Applications & Use Cases

Unlocking decentralised AI, ML,

Data Preparation & Compute

31 of 38

Unlocking Web3 Applications

Inference AI & ML training jobs
Invoking & supporting generic ZK computations
Cross-chain interoperability complement to bridge protocols
Utilising inbuilt storage on IPFS
Federated Learning consensus (insulated jobs)
IOT & Sensor Data integrations
Providing a platform for Digital twins
Supply chain tracking & analysis
ETL & data preparation jobs

32 of 38

Waterlily.ai - live on FVM mainnet

Full Build Tutorial: https://developerally.com/waterlily-ai

Waterlily.ai is a generative AI art platform that addresses key issues around attribution, consent, and revenue for creators in the digital space by compensating original creators of artwork with micropayments whenever a user selects their style to create a new derivative artwork from a prompt.

See the code: https://github.com/bacalhau-project/waterlily

https://www.youtube.com/watch?v=oXNaCs0eyXI

Taking the concept a little further, we then developed Waterlily.ai - which is live on the FVM mainnet.��Waterlily.ai is is a first of it’s kind user facing app that is aiming to provide a more equitable new paradigm in generative AI Art platforms, which provides attribution and royalties to original artwork creators on every use of the platform.

�The main concept behind Waterlily.ai is giving control back to creators and adding a potential new revenue stream for them to opt into and explore, while still giving a user an easy way to create generative AI art and even NFTs.

�As a user you can navigate to waterlily.ai and simply enter a text prompt of an image you’d like to generate. You can then browse through a selection of both private artist and public domain artists and choose a style you’d like to generate your image in - for example “rainbow unicorn in space” in the style of Claude Monet. You then pay a small fee to the platform via your wallet and 4 images will be generated for you to download or even mint as an NFT.

Under the hood, Waterlily leverages the Filecoin Virtual Machine to enable coordination and automated payments directly to artists whenever their Machine Learning artist model - which is attuned to their particular art style, is used to create new generative images from their art style. All of the user fee outside of the blockchain network running costs are passed on to the artist- or in the case of public domain artworks, are donated to Gray Area - an amazing art community and foundation based in San Francisco.

33 of 38

What’s next for Lilypad?

IPC implementation, robust game theory & more..

34 of 38

Lilypad Network Goals

Address compute needs of current Filecoin users and developers
Make internet-scale compute services available via smart contracts
Power Open new economies and use cases on Filecoin & FVM
Enable additional revenue streams for nodes and SPs

35 of 38

Lilypad Roadmap

August/Sept

Use Case & Collabs

Testnet v2 Testnet + IPC Testnet

Whitepaper

July

Modicum-Based Minimal Testnet

-> Lilypad v1

Litepaper

June

Research, Technical POC, Planning & Organisational

March

Lilypad v0 POC “Bridge”

Waterlily.ai

TBD

Mainnet

2022

Bacalhau

We Are Here

36 of 38

Join the Future

Get In Touch with Us

37 of 38

Get in touch with us!

Get Involved in the future of

data!

Github: github.com/bacalhau-project/lilypad

Twitter: @Lilypad_Tech

YouTube: @LilypadNetwork

Docs: docs.lilypadnetwork.org

Blog: blog.lilypadnetwork.org

Bacalhau Slack: #bacalhau-lilypad

bit.ly/bacalhau-project-slack

DeveloperAlly

38 of 38

Thankyou! Q’s?

DeveloperAlly