1 of 24

I have software I want to build!

2 of 24

I have software I want to build!

use!

───

compile!

okay, patch!

then compile and use!

and share..!

3 of 24

(there are many different ways to go about this)

4 of 24

What’s tricky isn’t bulding.

​

We have build tools.

​

​

What’s tricky is

integrating.

5 of 24

What’s tricky is connecting

different build processes together.

​

What’s tricky is composing systems.

​

​

​

What’s tricky is

integrating.

6 of 24

The best way to figure out how to integrate�something is… to break it apart, first.

7 of 24

What is a “build”?

Files go in…

​

{

"inputs": {

"/": "ipfs:Qm6q7G4hWr26q7G4hWTa5Lf89p"

"/app/gcc": "ipfs:Qmr283FpTaSL883FpL8"

“/task/src”: “git:7c554c3ae7bee8e3b42”

}

}

8 of 24

What is a “build”?

Files go in…

​

Files go out…

​

{

"inputs": {

"/": "ipfs:Qm6q7G4hWr26q7G4hWTa5Lf89p"

"/app/gcc": "ipfs:Qmr283FpTaSL883FpL8"

“/task/src”: “git:7c554c3ae7bee8e3b42”

},

​

​

​

​

​

"outputs": {

"/task/out": {"packtype": "ipfs"}

}

}

​

9 of 24

What is a “build”?

Files go in…

​

Files go out…

​

An action in the middle.

{

"inputs": {

"/": "ipfs:Qm6q7G4hWr26q7G4hWTa5Lf89p"

"/app/gcc": "ipfs:Qmr283FpTaSL883FpL8"

“/task/src”: “git:7c554c3ae7bee8e3b42”

},

"action": {

"exec": [

"/app/gcc/bin/gcc",

"/task/src/cool.c",

"-o", "/task/out/cool.exe" ]

},

"outputs": {

"/task/out": {"packtype": "ipfs"}

}

}

​

10 of 24

What can we do if…

​

we do this with IPLD?

​

{

"inputs": {

"/": "ipfs:Qm6q7G4hWr26q7G4hWTa5Lf89p"

"/app/gcc": "ipfs:Qmr283FpTaSL883FpL8"

“/task/src”: “git:7c554c3ae7bee8e3b42”

},

"action": {

"exec": [

"/app/gcc/bin/gcc",

"/task/src/cool.c",

"-o", "/task/out/cool.exe" ]

},

"outputs": {

"/task/out": {"packtype": "ipfs"}

}

}

​

11 of 24

“Computation-

addressable”

{

"inputs": {

"/": "ipfs:Qm6q7G4hWr26q7G4hWTa5Lf89p"

"/app/gcc": "ipfs:Qmr283FpTaSL883FpL8"

“/task/src”: “git:7c554c3ae7bee8e3b42”

},

"action": {

"exec": [

"/app/gcc/bin/gcc",

"/task/src/cool.c",

"-o", "/task/out/cool.exe" ]

},

"outputs": {

"/task/out": {"packtype": "ipfs"}

}

}

​

{

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

}

}

​

12 of 24

This is a useful building block.

  • Share a complete build instruction easily.
  • Rebuild later, or use in bug reports, etc.
  • Audit logs; explanations of builds that happened previously.
  • Check output hashes to easily check reproducibility.
  • General forcing-function for sanity!

13 of 24

Now let’s try to integrate.

14 of 24

Now let’s try to integrate.

  • Cool, we can address builds we’ve already drafted.

​

...

15 of 24

Now let’s try to integrate.

  • Cool, we can address builds we’ve already drafted.

​

  • But nobody likes copy-pasting hashes;

​

  • We’ll need a way to describe multiple steps and their relationships in advance to be productive.

16 of 24

Now let’s try to integrate.

“describe multiple steps and relate them in advance”.....

​

We have a naming problem.

​

​

17 of 24

So break it down again.

Distributed naming problems are hard.

​

So let’s try to break down the problem into component parts…

​

(until suddenly none of them look like “naming” anymore!)

​

and then integrate again afterwards.

18 of 24

So break it down again.

  • Naming for labeling (still immutable!)
  • Naming for making a graph (can be locally-scoped!)
  • Naming for looking for ‘updates’
  • Naming for grouping and categorization
  • Naming for reputation
  • ...

19 of 24

So break it down again.

  • Naming for labeling (still immutable!)
  • Naming for making a graph (can be locally-scoped!)
  • Naming for looking for ‘updates’
  • Naming for grouping and categorization
  • Naming for reputation
  • …��Some of these use-cases are easier than others.

20 of 24

So break it down again.

  • Naming for labeling (still immutable!)
  • Naming for making a graph (can be locally-scoped!)
  • Naming for looking for ‘updates’
  • Naming for grouping and categorization
  • Naming for reputation
  • …��Some of these use-cases entangle application logic.

21 of 24

locally scoped naming is easy

Lesson for any distributed system design:

​

If you can turn a distributed naming problem into a locally scoped naming problem, do it. It’s easier.�(Then try to make it composable.)

"imports": {

"root-builder": "catalog:early.polydawn.io/monolith/debian-gcc-plus:v1.2017.01.04:linux-amd64",

"app-smsh": "catalog:early.radix.polydawn.io/smsh:candidate:bin-linux-amd64",

"src-gzip": "catalog:early.hyphae.polydawn.io/sources/gzip:v1.9:src",

"autoconfesque": "ingest:pack:tar:../../../shared/autoconfesque"

"steps": {

"build": {

"operation": {

"inputs": {

"/": "root-builder",

"/app/smsh": "app-smsh",

"/task/raw": "src-gzip",

"/radix/autoconfesque": "autoconfesque"

"action": {

"userinfo": {"uid":0},

"exec": [

"/app/smsh/bin/smsh",

"ln -s raw/* src",

"ls -lah",

"/radix/autoconfesque/build.sh",

"mkdir out2"

"mv out/usr/local/bin out2"

"outputs": {

"bin": "/task/out2"

"busybash": {

"operation": {

"inputs": {

"/": "root-busybash",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"rich": {

"operation": {

"inputs": {

"/": "root-debian",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"exports": {

"bin-amd64-linux": "build.bin"

​

​

22 of 24

locally scoped naming is easy

… so what do we do when we want to collaborate and integrate with other people and other authors?

"imports": {

"root-builder": "catalog:early.polydawn.io/monolith/debian-gcc-plus:v1.2017.01.04:linux-amd64",

"app-smsh": "catalog:early.radix.polydawn.io/smsh:candidate:bin-linux-amd64",

"src-gzip": "catalog:early.hyphae.polydawn.io/sources/gzip:v1.9:src",

"autoconfesque": "ingest:pack:tar:../../../shared/autoconfesque"

"steps": {

"build": {

"operation": {

"inputs": {

"/": "root-builder",

"/app/smsh": "app-smsh",

"/task/raw": "src-gzip",

"/radix/autoconfesque": "autoconfesque"

"action": {

"userinfo": {"uid":0},

"exec": [

"/app/smsh/bin/smsh",

"ln -s raw/* src",

"ls -lah",

"/radix/autoconfesque/build.sh",

"mkdir out2"

"mv out/usr/local/bin out2"

"outputs": {

"bin": "/task/out2"

"busybash": {

"operation": {

"inputs": {

"/": "root-busybash",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"rich": {

"operation": {

"inputs": {

"/": "root-debian",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"exports": {

"bin-amd64-linux": "build.bin"

​

​

23 of 24

connect things with content-addressable (IPLD?) messages

… so what do we do when we want to collaborate and integrate with other people and other authors?

​

Can we keep extending the defn of “locally scoped” gradually?

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

​

"imports": {

"root-builder": "catalog:early.polydawn.io/monolith/debian-gcc-plus:v1.2017.01.04:linux-amd64",

"app-smsh": "catalog:early.radix.polydawn.io/smsh:candidate:bin-linux-amd64",

"src-gzip": "catalog:early.hyphae.polydawn.io/sources/gzip:v1.9:src",

"autoconfesque": "ingest:pack:tar:../../../shared/autoconfesque"

"steps": {

"build": {

"operation": {

"inputs": {

"/": "root-builder",

"/app/smsh": "app-smsh",

"/task/raw": "src-gzip",

"/radix/autoconfesque": "autoconfesque"

"action": {

"userinfo": {"uid":0},

"exec": [

"/app/smsh/bin/smsh",

"ln -s raw/* src",

"ls -lah",

"/radix/autoconfesque/build.sh",

"mkdir out2"

"mv out/usr/local/bin out2"

"outputs": {

"bin": "/task/out2"

"busybash": {

"operation": {

"inputs": {

"/": "root-busybash",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"rich": {

"operation": {

"inputs": {

"/": "root-debian",

"/app/smsh": "app-smsh",

"/task": "subject"

"action": {

"exec": [

"/app/smsh/bin/smsh",

"ldd ./bin/gzip",

"./bin/gzip -h"

"exports": {

"bin-amd64-linux": "build.bin"

​

​

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

​

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

​

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

​

"formulaID": "QmVdAD3v91JWPjZXGkTKD6",

"exitCode": 0,

"results": {

"/task/out": "ipfs:QmD9AjmrdCY4QUquhiQ"

​

24 of 24

Docs, more docs, and prototypes

​

repeatr.io�

github.com/polydawn/timeless

​

​