Ultra HD over ATSC 1.0?!

We didn’t say it’d be easy

Anton Kapela, Channel 3, Eugene - K03IM-D

How’d we get here?

Intro

I’m a comp sci/EE guy; worked in/near A/V/IPTV for much of my career; also data centers, ISPs, microwave, mobility networks, you name it (if it moves bits, I do it)

…But Mostly Ignored Broadcast.

Everything was great, until one day:

Happened to buy a UHD TV in 2020; nothing on the air besides MPEG2…

No ATSC 3 in Madison, and this dumb TV didn’t even support it!

…couldn’t let this stand; had to get something working with UHD resolutions!

​

But …

Question: Can 19.3 megabits do anything visually interesting?

… with HEVC?

And: Can anything DECODE HEVC over a normal Transport Stream (TS) … over the air?

​

Let’s get something working

First step: Buy cheap Dektec DVB-C modulator; test with SPTS, no magic: try to get one audio + video PID + ffmpeg playout working

Takeaway: It works, but dirty - PCR matters, TS shaping matters, GOP params really matter, CODEC options matter most

A Starting Point - Rate Targets

Credit: https://codecalamity.com/encoding-settings-for-hdr-4k-videos-using-10-bit-x265/

Typical UHD rates

Bitrates we want

SSIM we want

Entropy: search and encode!

Setup a “personal cloud” to explore transcoding options in HEVC with x.265

Didn’t anticipate the iterative work & refinement. Almost gave up: commercial encoders make it so easy to get reasonable results

That’s not the “hacker way”

~750 GHz lets you figure it out with brute-force

​

Average Case: “Offline” transcode example - HDR10 to 4.5mbit ABR+VBR:

​

ffmpeg -fflags +discardcorrupt -ec guess_mvs+deblock+favor_inter -i input.mkv -vf'scale=3840:2160:force_original_aspect_ratio=decrease,�pad=3840:2160:-1:-1:color=black'-c:a ac3 -ac 2 -ar 48000 -c:v libx265 -preset medium -x265-params 'hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,50):max-cll=3201,386:keyint=72:ref=5:bframes=3:b-adapt=2:bitrate=4000:vbv-maxrate=4500:vbv-bufsize=8000:merange=256:me=hex:no-open-gop=1:hrd=1:aq-mode=3:pmode=1:rect=1:rc-lookahead=36' -map_metadata -1 -map_chapters -1 output.mkv

​

~3 FPS

This is as good as it gets in 2022/23: ~20-threads per node, latest-greatest x.265 and Intel Xeon E5 v4 cpus, 3840x2160 main10

~70 watt/seconds per frame, per node!

Result: taking non-noisy things close to the visual entropy coding floor: ~4.5 Mbits coded rate @2160

Ok, what about live HEVC UHD transcoding?

• Nothing commercially optimal for me: wanted good VBR at low bitrates, cheap, off the shelf
• First person to mention “but SVT or mainconcept …” gets booted
• Own integration: FOSS code, standard x86 parts, Nvidia GPU, CUDA/NVENC API

Micro ITX 1RU half depth case

Tiny 300W PS

Nvidia GTX-1650 Super GPU (nvenc, npp offload)

i9 10k Extreme CPU

10c/20t

Legit OOB

and IPMI!

Standard copper gige for IP video I/O

Quad channel, lowest-latency supported DDR4

Hot-swap SATA bays

Rude blue LED power light

Custom kapton PCI-e ribbon adapter

​

Magic: Live transcode NASA UHD to 10-bit SDR at ~4 Mbits VBR on GPU:

​

while :; do ffmpeg -fflags +discardcorrupt -i udp://238.1.1.1:2000?fifo_size=400000\?overrun_nonfatal=1\&buffer_size=10000000 -map 0:i:4161 -map 0:i:4164 -vf fps=fps=30000/1001 -pix_fmt p010le -c:v hevc_nvenc -g 90 -preset p7 -b:v 3.2M -maxrate 4.5M -bufsize 8M -bf 2 -refs 5 -rc-lookahead 27 -weighted_pred 0 -b_ref_mode each -nonref_p 0 -spatial_aq 1 -temporal_aq 1 -aq-strength 8 -forced-idr 1 -strict_gop 1 -c:a copy -program title="NASA-4K":st=0:st=1 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -f mpegts -mpegts_start_pid 160 -max_interleave_delta 200000 -muxpreload 2 -flush_packets 0 udp://224.2.2.254:1466?pkt_size=1316\&bitrate=8121600 ; sleep 1 ; done

All of the books in the world contain no more information than is broadcast as video in a single large American city in a single year. Not all bits have equal value.

• Carl Sagan

Critical Enablers for ultra-low rate HD/UHD

Absolutely Essential: to reach the visual-coding-floor in HEVC, we need: VBR, HRD/SEI signaling, long & fully dynamic GoP construction, fully dynamic mode decisions, exhaustive/full-frame motion search area, mixed reference frames, and “smart” bit allocation/rate distortion across the whole GoP and mini-GoPs

• These things have substantial compute/power cost, generally not done by OTT/streamers, and impractical at YouTube/etc scale
• Are eminently practical for broadcast: one screen target, one rate target, one program/stream

Channel 3 Refinement wishlist: efficient hierarchical motion estimation, flexible direct and synthetic prediction modes (ie. “grain simulation” and vector graphics), dynamic NAL SEI support for HDR10/+ and other codec metadata

Other subtle adjustments for “good display compatibility” with VBR-style AVC and HEVC:

• MPTS “transport buffering” burst-spreading constrained by PCR to DTS/PTS delay - often too short/small in multiplexers
• Is the “DVB default” of 750 msec really the limit? No: up to ~2000 msec actually works on every HEVC display/decoder tested! (~1700 msec seems more compatible with some AVC decoders)
• SPTS and small MPTS from a single VBR playout mux (ffmpeg) are fine - have a clear path forward for scaling to multiple programs in external statmux, but must ensure DTS/PTS > PCR
• Limiting HEVC reference frames - seems like 5 is workable everywhere, 8 for a few
• AVC decoders seem “stuck” at 4 refs, even on latest-gen UHD/HEVC-supporting TVs

​

Can’t stop here.

If it works on DVB-C, maybe it’ll also work over ATSC 1.0

​

…we need more features (PSIP, TVCT, EAS switching), aint writing it myself!

8VSB Bcast Lab, and Channel 3

Power Amp

UHF

ant.

Ktech VSB-ENC-150e or ADV-8300

(ASI TS to 8VSB & upconvert)

Ethernet Switch

D2D 5220 and/or TBS260B

(IP to ASI, mux, etc.)

Control systems & OAM terminals

ASI Monitoring

(Ineoquest IQ / ASI Cricket-A6x,

Nucomm ASI-IP)

Mask Filter

Ref In

GPS ant.

Enter: D2D Tech

After semi-exhaustive research, determined D2D Technologies has best price/perf/hackability ratio* (Linux inside!) for muxing & “finishing” (ie. PSIP, EIT, ETT, other ATSC 1 adornments): https://d2dtechnologies.com/d2flex5220/

Got to know Steve Doll and Jessica Colyer, agreed to “try supporting” more codecs in VBR modes

With mux hardware in hand, I:

1. Setup 8VSB/ATSC modulator & lab parts
2. Tested everything
3. Sent bug reports
4. Got new code
5. Goto #2

Herp Derp

• NOTE: The D2D 5220 mux is doing what FFmpeg, today in March 2024 cannot: generate PSIP stuff, robustly combine programs into a multi-program output TS composite, without losing its mind/halting all outputs when any single input halts, complete lack of gapless live input switching/reversion, etc.
• FFmpeg alone cannot generate a ready-to-go ATSC 1.0 MPTS appropriate for OTA broadcast

​

Where we started - Broken VBR

Initially, mux logic would simply drop TS packets that weren’t “within CBR rate estimate window” - oops!

Where we started - Broken VBR handling

Tried to ‘better constrain’ HEVC code rate to be “almost CBR” – still insufficient: I-frame bursts created gaps in TS output cadence. PCR packets spaced too far apart for most receivers to tolerate.

How to support VBR in MPTS?

• New (Thanks Steve!) rate estimation for input -> mux -> output routines
• We now only count bytes between PCRs (ie. out rate = sum input rate)
• Count TS packet bytes between TS with PCR, ignore input null padding, save system tick deltas between PCRs as a 64 bit integer: do math
• robust min/max/mean/sum of squares vs mean
• handily tolerates missing PCRs
• Output round-robin is 1/n portionally fair among buffered TS inputs
• Current algorithm now scales down to ~18 kbits/sec TS rates
• Can now reliably mux ultra-low rate video, audio-only, and ancillary low-speed data programming
• AAC ADTS 8 kb/s mode --> 23.1 kbits/s over TS, now works
• AC3 32 kb/s mode --> 45.1 kbits/s over TS, also now works

Where we are now - Robust VBR

Input Loss

Input Return

With new rate estimation logic, D2D mux properly transmits packets in a weighted-fair round robin scheme; PCR continuity within tolerances; near perfect adaptation to input loss/restoration.

TS/PES pacing is preserved through mux: greater PES/NAL byte counts for I/IDR than B/P-frames reveals corresponding “gaps” of PTS in time, while retaining consistent PCR intervals: receiver remains phase locked + Rx slack buffer managed (ie. PCR < PTS/DTS)

Where we are now - Robust VBR

Input Loss

Input Return

Where we are now - Robust When Maxed Out

Oversubscribed mux rate is fine, within limits

If PTS/DTS greater than PCR timestamp value, we’re fine. Note PCR cleanly “linear” (despite interval variance) while DTS/PTS delta “wanders” due to momentary mux queue depth. Long term VBR encoder control ensures DTS/DTS > PCR (ie. pictures can’t “happen” before current time in PCR).

Where we are now - Maxed Out 4x UHD progs

Hurdle: Cleared, next:

Take this on the road:

• Landed on K03IM-D, Eugene, OR
• Yes, I actually own a real TV station:

https://www.rabbitears.info/market.php?request=print_station&facility_id=185855

https://enterpriseefiling.fcc.gov/dataentry/public/tv/publicFacilityDetails.html?facilityId=185855

• Find content networks (TCN, Funroads, etc.), negotiate

​

Channel 3 Eugene Photo Gallery: https://imgur.com/a/IyNAxlz

Mux packing Feb 2024, 14 programs

In-Studio Demos: What are we seeing?

​

​

​

​

5NINES Madison Datacenter

​

(playout, live and offline transcoding, scheduling, switching

Internet Tubes

K03IM-D Eugene, OR

​

(NASA downlink, NASA transcoding, demo playout, ATSC 1.0 TS mux, SRT monitor output)

Clutch Sound Studios, Madison WI

​

(TS over SRT to ASI, low-power 8VSB modulator and transmitter, ATSC 1.0 demo screens)

UHD Display

Media Things

Various Other Internet Video Sources

Questions?

tkapela@gmail.com

​

Appendix A

Semi-exhaustive FFMPEG notes & etc

RX stream from an SRT source, relay back out to mcast, output connection stats every 4000 msec, using fairly robust options - 2000 msec ARQ time to live, 2100 msec fixed sender/receiver latency, sender buffer allocation 10 megabytes, and overhead bandwidth of 100%

while :; do srt-live-transmit -s 4000 "srt://source.host.net:1234?mode=caller&latency=2100&lossmaxttl=2000&&sndbuf=10000000&oheadbw=100" udp://233.65.202.50:1234 ; sleep 1 ; done

Pulling from YT live source, conforming to 720p at lowest-possible bcastable frame rate of 23.976, low-bitate target

while :; do yt-dlp --socket-timeout 6 https://www.youtube.com/watch?v=xxxx -f best -o - | mbuffer -q -W 32 -m 64k | ffmpeg -async 1 -re -i - -map 0:0 -map 0:1 -c:a ac3 -ab 96k -cutoff 18000 -ac 2 -ar 48000 -vf "minterpolate=fps=24000/1001:mi_mode=blend" -fps_mode cfr -c:v libx264 -g 72 -rc-lookahead 48 -preset veryslow -tune animation -b:v 400K -nal-hrd vbr -maxrate:v 1.5M -bufsize 1.5M -me_method umh -me_range 64 -refs 4 -bf 4 -aq-mode 3 -aq-strength 0.9 -qcomp .1 -flags +loop+qpel+cgop -map_metadata -1 -map_chapters -1 -f mpegts -flush_packets 0 udp://233.65.202.201:1234?pkt_size=1316 ; sleep 1 ; done

Listening to a mcast source, connecting to remote host via TCP, relaying A/V stream to target host

while :; do ffmpeg -i udp://233.65.202.201:1234 -c copy -f mpegts -flush_packets 0 tcp://10.0.3.2:1423?pkt_size=3008 ; sleep 1 ; done

Relaying a unicast TSoTCP stream in a CBR TS, to another TS mux, via unicast UDP

while :; do ffmpeg -i tcp://:1490?listen=1\&listen_timeout=10000\&timeout=20000000 -map 0:v -map 0:a -c:v copy -c:a copy -program title=Some Program Name:st=0:st=1 -f mpegts -mpegts_start_pid 200 -flush_packets 0 -muxdelay 1.7 -muxpreload 4 -muxrate 2000320 -pcr_period 70 udp://10.0.3.19:1402?pkt_size=1316\&bitrate=2000320\&fifo_size=40000\&overrun_nonfatal=1 ; sleep 1 ; done

Rx a slice-encoded (note: single decoder thread to avoid race conditions) 1080i program from mcast source, convert to 10 bit non-subsampled chroma, deinterlace/etc, convert back to yuv420 subsampled, force constant frame rate at 29.97, normalize audio level to ~-6 dBfs peak, various x.264 adjustments tuning encoder for “talking head” news programming; note custom quantizer matrix, output encoded stream towards mcast destination

while :; do ./ffmpeg-6.1-amd64-static/ffmpeg -async 1 -threads:v 1 -i udp://233.65.202.50:1234?fifo_size=200000\&overrun_nonfatal=1\&reuse=1\&timeout=900000000\&buffer_size=10000000 \

-vf format=yuv444p10le,yadif=mode=0:deint=all,format=yuv420p,fps=fps=30000/1001 \

-fps_mode cfr \

-map 0:v:0 -map 0:a:0 \

-af "dynaudnorm=p=.25" \

-c:a ac3 -ac 2 -ab 96k -ar 48000 -cutoff 18000 \

-c:v libx264 -g 90 -preset veryslow -b:v 1M -maxrate:v 2M -bufsize 2M -refs 4 -bf 4 \

-aq-mode 3 -aq-strength 0.90 -b_qfactor 1.0 -b_qoffset 0.0 -qcomp 0.3 \

-dc 9 -subq 10 -weightp 2 -weightb 1 -bidir_refine 4 -mixed-refs 1 \

-8x8dct 1 -partitions all -direct-pred auto -nal-hrd vbr \

-rc-lookahead 30 -me_range 64 -me_method umh -trellis 2 -b_strategy 2 -b-pyramid 2 \

-fast-pskip 0 -flags +qpel+loop+cgop \

-intra_matrix "8,8,9,9,10,10,11,11,8,9,9,10,10,11,11,12,9,9,10,10,11,11,12,12,9,10,10,11,11,12,13,13,10,10,11,11,12,13,13,14,10,11,11,12,13,13,14,15,11,11,12,13,13,14,15,15,11,12,12,13,14,15,15,16" \

-inter_matrix "8,8,9,9,10,10,11,11,8,9,9,10,10,11,11,12,9,9,10,10,11,11,12,12,9,10,10,11,11,12,13,13,10,10,11,11,12,13,13,14,10,11,11,12,13,13,14,15,11,11,12,13,13,14,15,15,11,12,12,13,14,15,15,16" \

-f mpegts -max_interleave_delta 0 -flush_packets 0 udp://233.65.202.15:1234?pkt_size=1316 ; sleep 1 ; done

Pick a random file from a list of files matching a pattern, transcode this file at native playback rate, keeping multichannel audio intact (or conform to 6 chan/5.1 layout); conform all output to 1440x1080 at 4:3 aspect ratio, stop muxing when audio or video inputs have no more data (-shortest), to ensure output a/v pids of matching lengths; restart a/v PTS’s from zero, and signal an explicit TS discontinuity to inform receivers that a new TS is starting

IFS=$'\n' ; while :; do for i in `find "/mnt/space3/blah/" -name "*.mkv" -type f |shuf` ; do \

./ffmpeg-6.1-amd64-static/ffmpeg \

-ec guess_mvs+deblock+favor_inter -err_detect ignore_err \

-readrate_initial_burst 4 -async 1 -re -i "$i" \

-fps_mode cfr -map 0:v -map 0:a \

-ar 48000 -c:a eac3 -b:a 192k -cutoff 18000 -ac 6 -channel_layout "5.1" \

-color_primaries 1 -color_trc 1 -colorspace 1 \

-c:v libx264 -g 72 -preset veryslow -b:v 0.9M -maxrate:v 2M -bufsize 2M -refs 4 -bf 4 \

-psy-rd 1.1:0.5 -deblock -5:-3 -tune grain -b_qfactor 1.0 -b_qoffset 0.0 \

-dc 9 -subq 10 -weightp 2 -weightb 1 -aq-mode 3 -aq-strength 0.80 -qcomp 0.4 \

-mixed-refs 1 -8x8dct 1 -partitions all -direct-pred auto -nal-hrd vbr \

-rc-lookahead 36 -me_range 64 -me_method umh -trellis 2 -b_strategy 2 \

-b-pyramid 2 -bidir_refine 2 -fast-pskip 0 \

-flags +qpel+loop \

-intra_matrix "8,8,8,9,8,9,11,11,11,11,11,11,13,12,13,13,13,13,13,13,13,13,13,13,13,14,14,14,17,17,17,13,13,13,14,13,15,15,16,16,17,17,18,16,18,17,17,17,17,19,19,20,20,20,24,24,23,23,28,28,29,34,34,41" \

-inter_matrix "8,8,9,9,10,10,11,11,8,9,9,10,10,11,11,12,9,9,10,10,11,11,12,12,9,10,10,11,11,12,13,13,10,10,11,11,12,13,13,14,10,11,11,12,13,13,14,15,11,11,12,13,13,14,15,15,11,12,12,13,14,15,15,16" \

-vf "fps=24000/1001,format=yuv444p10le,atadenoise,setpts=PTS-STARTPTS,\

scale=iw*sar:ih:flags=lanczos,setsar=1,\

scale=1440:1080:force_original_aspect_ratio=decrease:flags=lanczos,\

pad=1440:1080:-1:-1:color=black,setsar=1,setdar=4/3,format=yuv420p" \

-af "asetpts=PTS-STARTPTS,volume=+3dB" \

-map_metadata -1 -map_chapters -1 \

-f mpegts -flush_packets 0 -shortest -mpegts_flags initial_discontinuity udp://233.6.2.9:1090?pkt_size=1316 \

; done ; done

Ultra-low-rate mpeg2 video encoding, targeting 1.1 mbits/sec for 16:9 720x480p - note for super-low rates, variable length coding table 0 (vlc) is recommended. We convert to non-subsampled 10 bit 4:4:4 chroma for deinterlacing, noise reduction, and scaling - ie. example assumes input is 1080i, lowrez progressive out. Max motion vector search area at 511 pixels. Set spatial and temporal masking to bias quantizer upwards for higher motion and higher complexity areas, leaving more bits for gradients/smooth areas. Less banding, more reduction on details that won’t be visible to most viewers.

while :; do ./ffmpeg-6.1-amd64-static/ffmpeg -threads 16 \

-ec guess_mvs+deblock+favor_inter -async 1 \

-i http://208.66.132.245:5004/auto/v15.1?overrun_nonfatal=1 \

-map 0:v:0 -map 0:a:0 \

-c:a ac3 -ac 2 -ab 96k -ar 48000 -cutoff 18000 \

-vf "format=yuv444p10le,yadif=mode=0:deint=all,fps=fps=30000/1001,\

hqdn3d=3:3:3:3,scale=720x480:flags=lanczos+accurate_rnd,\

hqdn3d=2:2:2:2,format=yuv420p" \

-fps_mode cfr \

-c:v mpeg2video -threads 16 -me_range 511 -intra_vlc 0 \

-tcplx_mask 0.3 -scplx_mask 0.2 -qcomp 0.06 \

-cmp satd -subcmp satd -mbcmp satd -dct int -idct int \

-qmin 2 -qmax 200 \

-trellis 0 -keyint_min 8 -g 60 -bf 3 -b_strategy 2 -brd_scale 2 -bidir_refine 2 -border_mask 2.0 \

-b:v 1.1M -maxrate 4M -bufsize 4M -dc 9 \

-intra_matrix "8,8,8,9,8,9,11,11,11,11,11,11,13,12,13,13,13,13,13,13,13,13,13,13,13,14,14,14,\

17,17,17,13,13,13,14,13,15,15,16,16,17,17,18,16,18,17,17,17,17,19,19,20,20,20,24,24,23,23,28,28,29,34,34,41" \

-inter_matrix "8,8,8,8,8,8,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,10,10,10,10,10,10,10,\

10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,11,11,11,11,11,11,11,11,11,12" \

-f mpegts -flush_packets 0 udp://233.65.202.151:1234?overrun_nonfatal=1\&pkt_size=1316 ; sleep 1 ; done