1 of 10

Improving Pub/Sub performance with Bulk APIs

alpha release v1.10

2 of 10

Publish vs Bulk-Publish – test plan

  • Baseline: 100 individual publish requests with messages of size 1 KiB
  • Actual Test: 1 bulk publish request with 100 messages of size 1 KiB
  • Message types: cloud event and raw payload
  • Components: In-memory pub/sub and Kafka
  • Protocol tested: gRPC

​

​

3 of 10

Publish vs Bulk-Publish – overview

​

​

QPS: 200

Perf connections: 16

Test duration: 1m

4 of 10

Publish vs Bulk-Publish – results

In-memory pub/sub (with cloud event)

​

  • reduced latency for 50th percentile: 47.23ms
  • reduced latency for 75th percentile: 49.78ms
  • reduced latency for 90th percentile: 51.10ms
  • reduced latency for 99th percentile: 54.87ms
  • baseline latency avg: 60.19ms
  • bulk latency avg: 11.76ms
  • reduced latency avg: 48.43ms
  • baseline QPS: 199.79
  • bulk QPS: 199.94187512741487
  • increase in QPS: 0.07417816241837476

Kafka (with cloud event)

​

  • reduced latency for 50th percentile: 102.11ms
  • reduced latency for 75th percentile: 108.35ms
  • reduced latency for 90th percentile: 135.99ms
  • reduced latency for 99th percentile: 109.94ms
  • baseline latency avg: 144.20ms
  • bulk latency avg: 38.12ms
  • reduced latency avg: 106.08ms
  • baseline QPS: 110.89093028006693
  • bulk QPS: 199.7816951947595
  • increase in QPS: 80.16053674560166

5 of 10

Publish vs Bulk-Publish – testimonials

6 of 10

Subscribe vs Bulk-Subscribe – test plan

  • Baseline: Send and receive 100 messages of size 100B using BP+S
  • Actual Test: Send and receive 100 messages of size 100B using BP+BS
  • Components: Kafka
  • Protocol tested: HTTP

​

7 of 10

Subscribe vs Bulk-Subscribe – overview

500 Virtual Users (ramping mode)

8 of 10

Subscribe vs Bulk-Subscribe – results

Subscribe

​

  • "max": 1176.942004,
  • "p(90)": 986.711315,
  • "p(95)": 1066.0373890999997,
  • "avg": 502.74581646805916,
  • "min": 0.309318,
  • "med": 475.461188

Bulk Subscribe

​

  • "max": 470.397741,
  • "p(90)": 248.4924315,
  • "p(95)": 357.75256375000004,
  • "avg": 129.70715183285915,
  • "min": 0.556631,
  • "med": 105.968547

9 of 10

References

  • Add performance test for bulk pubsub publish gRPC #5495
  • Add performance test for bulk subscribe over HTTP #5650
  • https://stackoverflow.com/q/74720305/4654175

​

10 of 10

Thank you!