High Throughput JPEG 2000 (HTJ2K)�for Media Applications

Michael Smith, Wavelet Consulting LLC�miksmith@gmail.com

1

Introducing HTJ2K

Joint ISO/ITU standard published in August 2019� ITU-T Rec T.814 | ISO 15444-15

Enhances JPEG 2000 Part 1

• Replaces the slow block coder with a fast block coder (approx. 5% increase in bit rate)
• Keeps everything else from JPEG 2000 Part 1, except quality layers
• Royalty-free intent like JPEG 2000

Lossless transcoding to/from JPEG 2000 Part 1

2

JPEG2000 Part-1 Compression

3

Color Transform, �Level Offset, �Normalization

Discrete �Wavelet�Transform

Quantization

Tier 1 Coding

​

- Context Labeling �- Binary Arithmetic �Coding

Tier 2 Coding

​

• Rate Control
• Bitstream � Organization

Image Samples

Compressed�Data

pixels

pixels

bits

quantized wavelet coefficients

wavelet �coefficients

JPEG2000 Part-15 Compression (HTJ2K)

4

Color Transform, �Level Offset, �Normalization

Discrete �Wavelet�Transform

Quantization

Fast Block Coder

- single CleanUp pass�- optional MagRef�- optional SigProp

Tier 2 Coding

​

• Rate Control
• Bitstream � Organization

Image Samples

Compressed�Data

pixels

pixels

bits

quantized wavelet coefficients

wavelet �coefficients

HTJ2K is fast

5

CPU decoding on 2018 MacBook Pro laptop (6-core)

SDR 3840x2160 24p 4:4:4 10bit (Netflix Meridian content)

CPU encoding has similar speed-ups

* this is decoding at ~3x faster than real-time (24p), so the decoded data throughput is 3x the compressed data rate, i.e., 3 x 4000Mbp s= 12000Mbs = 1.5GBytes/sec which is approaching the limits of the internal SSD read-rate

HTJ2K is flexible

HTJ2K supports any image data structure

• Any resolution – SD, HD, 4K, 8K, 16K, 50K, up to 4 billion × 4 billion
• Any sampling structure - 4:4:4, 4:2:2, 4:2:0, 4:4:4:4, 4:2:2:4, RAW, etc.
• Any number of color components or sample precision (bit depth)
• Floating point pixels values – e.g., ACES EXR files (16-bit half-float)
• Any color space, HDR, SDR, PQ, HLG, BT.709, ACES, XYZ, custom, etc.

HTJ2K supports lossy or lossless compression

HTJ2K supports resolution scalability and spatial random access

HTJ2K supports a range of practical end-to-end latencies

• 5 lines to 3 frames
• 0.1ms to 48ms for 1080/60p line rates

​

6

HTJ2K is now

• Source code available:
• Kakadu – https://KakaduSoftware.com
• OpenJPH – https://github.com/aous72/OpenJPH
• OpenHTJ2K – https://github.com/osamu620/OpenHTJ2K
• OpenHTjs – https://www.npmjs.com/package/openht
• OpenJPEG now includes HTJ2K decoding - https://github.com/uclouvain/openjpeg� OpenJPEG is widely used by open source (Chrome, WebKit, FFMPEG, VLC, Imagemagick, etc.)
• Products:
• BlackMagic DaVinci Resolve 17 (shipping since Nov 2020)
• Rhode & Schwartz Clipster (since Jan 2022)
• ColorFront – in new beta builds – available now
• Standards:
• SMPTE IMF App 4 (Digital Cinema Archiving) - https://github.com/SMPTE/st2067-40-2ED
• SMPTE IMF App 2E (Home Video Masters) – current in Public Committee Draft stage, publication expected in 2022 - https://github.com/SMPTE/st2067-21
• HTJ2K works on the web:

via JavaScript OpenHTjs - https://www.npmjs.com/package/openht

7

HTJ2K enables no-proxy

Try web demo here:

https://demo.noproxy.cloud/

• Playback of HTJ2K MXF file in web browser using Javascript MXF parsing and HTJ2K decoding
• Decode quality dependent on available client download speed
• Click to pause, then full lossless quality is shown
• Forward and Backward playback loop demonstrates intra-only random access.
• Download HTJ2K MXF and play it in full quality using Resolve 17.

8

HTJ2K Resolution Progressive Example

9

LL 2

LH 2

HL 2

HH 2

LH 1

HH 1

HL 1

Wavelet decomposition visualization

J2C code-stream

HTJ2K Resolution Progressive Example

10

LL 2

Not Downloaded or Decoded

Partial Code-stream Download and Decode

11 kilobytes

J2C code-stream

HTJ2K Resolution Progressive Example

11

LL 2

LH 2

HL 2

HH 2

Not Downloaded or Decoded

Partial code-stream download and decode

38 kilobytes

J2C code-stream

HTJ2K Resolution Progressive Example

12

LL 2

LH 2

HL 2

HH 2

LH 1

HH 1

HL 1

Not Downloaded or Decoded

Partial code-stream download and decode

140 kilobytes

J2C code-stream

HTJ2K Resolution Progressive Example

13

LL 2

LH 2

HL 2

HH 2

LH 1

HH 1

HL 1

Full code-stream download and decode

3857 kilobytes

J2C code-stream

HTJ2K is a Prores alternative

14

Bitrate Efficiency is as good or better than Prores.

​

These tests were performed using Resolve 17 on MacOS

XQ

XQ

4444

4444

HQ

HQ

422

422

LT

LT

Proxy

Proxy

HTJ2K Lossless Performance on 16bit half-float imagery

15

ACES OpenEXR 16bit half-float 3840x2160 24p 4:4:4 �Netflix Meridian short – first 10000 frames

Latency vs quality of previous codecs

Previous codecs can be used for either high quality or very low latency, but not both

HTJ2K supports a range of latencies

End-to-End latency is affected by several factors, including:

• number of vertical wavelet transforms
• number of samples coded as an entropy data unit
• rate-control buffering
• computation time

HTJ2K supports a range of latencies

HTJ2K 11-lines

More latency means more quality,

how much latency do you need?

XS – Light Subline

XS – Light

HTJ2K 23-lines

XS - Main

HTJ2K 3-frames

Examples are compressed at 2 bits per pixel with cropping and 4x zoom for display in PowerPoint

More latency means more quality, how much latency do you need?

Trend in this direction

Plot from:�T. Edwards and M. D. Smith, "High Throughput JPEG 2000 for Broadcast and IP-Based Applications," in SMPTE Motion Imaging Journal, vol. 130, no. 4, pp. 22-35, May 2021, doi: 10.5594/JMI.2021.3066183.�https://ieeexplore.ieee.org/document/9424052

HTJ2K has good performance across a range of latencies

HTJ2K supports multi-generation encode/decode

Plot from:�T. Edwards and M. D. Smith, "High Throughput JPEG 2000 for Broadcast and IP-Based Applications," in SMPTE Motion Imaging Journal, vol. 130, no. 4, pp. 22-35, May 2021, doi: 10.5594/JMI.2021.3066183.�https://ieeexplore.ieee.org/document/9424052

HTJ2K is the future

• Perfect for IMF-style and other file-based workflows
• New no-proxy workflows in the cloud on the web
• Works great on commodity CPUs
• Alternative to ProRes for file-based workflow
• Alternative to JPEG-XS for low-latency workflow

21

APPENDIX

22

HTJ2K is not JPEG-XS

23