1 of 19

Distributed Prediction Market for Organizations

Thomas Mitchell

Louis Cao

Presenting Polaris:

2 of 19

3 of 19

Polaris Brings to Organizations

4 of 19

5 of 19

6 of 19

7 of 19

Logic Influences

Tickets - Horse Racing

Live Trading - Iowa

Binary Outcomes - Modern

8 of 19

Relational Model

9 of 19

Consistency Model

1 Leader - Sequential Writes, Read-Your-Writes
1 Standby Using MultiAZ
4 Direct Replicants - Read Only, Eventual Consistency

10 of 19

Backend API

Idempotent

SELECT transaction_id

FROM market_transaction

WHERE market_id = %s AND transaction_id = %s

and then:

if cursor.fetchone() is not None:

return transaction_id

Atomic

except Exception as e:

db.rollback()

Safety

WHERE m.id = %s AND (ol.user_id IS NOT NULL OR uor.user_id IS NOT NULL)

11 of 19

Event Bus

Topic-based Servers

Server-side caching

DB Fault Tolerance

Data Replication

System

Design

When the user clicks Buy, the client sends a request to the fastAPI server. That service authenticates the user using a JSON Web Token (stateless), and publishes a single “order” event to a message bus (Kafka). Kafka queues the message securely even if the 4 workers are down. The service immediately returns a receipt / correlation id and a “queued” state.

The servers are partitioned into 4-2-2-2 cluster and are responsible for different types of operations.

Why use event bus: not having the clients to directly connect to individual servers, it divide servers based on function types

This enables us to do server-side caching, because we are partitioning the markets, etc based on their id, so each unique market is hosted on a single server, which allows we can cache the data for that market to that server

Scaling

Separation of powers – add servers that specialize in different tasks so that we can add and remove servers based on what jobs more important

We use a validation cache –

This also ensures sequential execution and data consistency for individual markets, while we don’t really care about …

12 of 19

Topics

Normal Operation	Fallback
market.operations event.lifecycle org.management user.account	market.operations.dlq event.lifecycle.dlq org.management.dlq user.account.dlq

13 of 19

Write

Read

Event-bus v.s. Multi-server

Testing

When the user clicks Buy, the client sends a request to the fastAPI server. That service authenticates the user using a JSON Web Token (stateless), and publishes a single “order” event to a message bus (Kafka). Kafka queues the message securely even if the 4 workers are down. The service immediately returns a receipt / correlation id and a “queued” state.

The servers are partitioned into 4-2-2-2 cluster and are responsible for different types of operations.

Why use event bus: not having the clients to directly connect to individual servers, it divide servers based on function types

This enables us to do server-side caching, because we are partitioning the markets, etc based on their id, so each unique market is hosted on a single server, which allows we can cache the data for that market to that server

Scaling

Separation of powers – add servers that specialize in different tasks so that we can add and remove servers based on what jobs more important

We use a validation cache –

This also ensures sequential execution and data consistency for individual markets, while we don’t really care about …

14 of 19

Bet (3 Brokers)

15 of 19

Throughput v.s. Users

16 of 19

View - Caching

17 of 19

Time (Caching)

18 of 19

Event-bus (3 brokers) v.s. Multi-server - Write

19 of 19

Event-bus (3 brokers) v.s. Multi-server - Write