Is COG Scaleable?

Jeff Albrecht

Tile Server

Blob Store Characteristics

• Minimize Time to First Byte (TTFB) by sending as few requests as possible.
• Connecting to the blob store is expensive, data transfer through an open connection is cheap.
• S3 supports at least 5,500 GETs per second per key prefix.
• Experimentally, its close to ~8,500 GETs to ~15,000 GETs per second

​

​

s3://cog-layers-glad/7/28/49/0231102/tile.tif

Scaling Blob Stores is Easy!

1. Send as few requests as possible.
2. Send those requests as fast and efficiently as possible.

COG Internal Structure

8192

8192

Tile(x=28, y=49, z=7)

Web Optimized COG

• Each internal tile aligned to the web mercator grid, coverings Z7 -> Z12.
• 16,384 images in the world.
• Reading a single XYZ tile requires:
• 1 request to fetch the header.
• 1 request to fetch the block.

​

This is, in theory, the most optimized COG layout for this use case.

tilebench

Caching

Most applications have some sort of local cache.

Requests are load balanced across replicas (round robin).

Requests are load balanced across replicas (round robin).

Requires 3 requests for all replicas to cache the header.

Round Robin

Round Robin

n_replicas = 100

n_images = 100

Round Robin

n_replicas = 100

n_images = 100

Cache (In)efficiency

Distributed Caching Patterns

Cache aside

Peer-to-peer cache

(groupcache)

Forward-proxy

MetaTiling

MetaTiling

• COGs are row major.
• Metatiling logically increases chunk size without changing physical chunk size.

​

(8192, 8192) image with (256x256) chunk size contains 1024 chunks.

(8192, 8192) image with (4096x4096) contains 4 meta-chunks.

Sending Requests Quickly

Send Request Quickly

• Use async/await where possible, it’s more efficient.
• Use lower level languages like rust / go.
• Maintain strong data locality.
• Always request COG headers in one request (know your header size ahead of time).

In Practice

pyasyncio-benchmark

• Built a tool specifically designed to benchmark I/O bound python code across different libraries:
• aioboto3
• aiohttp
• fsspec (s3fs)
• httpx
• obstore
• requests
• rasterio
• All tests run across 20 containers on an m5.8xlarge instance.

https://github.com/geospatial-jeff/pyasyncio-benchmark

​

pyasyncio-benchmark

• Built a tool specifically designed to benchmark I/O bound python code.
• https://github.com/geospatial-jeff/pyasyncio-benchmark

​

Header Requests (m5.8xlarge)

MetaTiling (m5.8xlarge)

Dump all native resolution tiles (1024).

MetaTiling (m5.8xlarge)

ignore httpx :(

Is COG Scaleable?

Yes, it is.

​

​

​

​

​

​

​

​

​

Yes, it is. But our software is not written in a way that encourages or achieves scalable system design in the cloud.

​

​

​

​

​

​

​

Yes, it is. But our software is not written in a way that encourages or achieves scalable system design in the cloud.

​

Software, hardware, and data are in constant co-evolution. Code doesn’t age, but context does change.

​

​

​

Yes, it is. But our software is not written in a way that encourages or achieves scalable system design in the cloud.

​

Software, hardware, and data are in constant co-evolution. Code doesn’t age, but context does change.

​

As the CNG community standardizes on individual data formats (ex. Zarr / COG) it makes sense to invest more in specialized tooling for those formats.

​

Questions