Bandwidth is a lie!
WISPAPALOOZA 2025
In memory of Dave Täht (1965-2025)
Dave Täht sadly passed away in April. Dave was:
You’ve probably used his code, without ever realizing it: millions of routers around the world contained Dave’s contributions.
Above all, Dave was a great man - and a great friend.
It boils down to…
Last but not least
I was lying! :)
“In the ‘post-gigabit’ world of bandwidth abundance, it is time to optimize for things other than throughput to keep improving app/user QoE. Bandwidth has been a great proxy for improving QoE for many years but we are now at a point of diminishing marginal utility of more bandwidth. To wit: the only app that needs 1 Gbps is a synthetic speed test. ;-).”
Bufferbloat? Sounds Uncomfortable!
Have your customers ever…
Most routers enqueue packets linearly.
When the queue is full, packets “bounce off the tail” - they are dropped indiscriminately.
No effort is made to balance “flows”.
Many routers have huge buffers to compensate - and “time in buffer” can become a latency problem by itself.
CoDel, FQ_CoDel and CAKE
100% Open Source, included in the Linux kernel.
Bufferbloat is most evident when a connection is heavily loaded with downloads or uploads. It causes increased latency (ping), resulting in poor performance for real-time tasks like VoIP, video chat, and lag in online gaming, and generally makes the internet less responsive.
New and up-and-coming solutions
cake-autorate https://github.com/lynxthecat/cake-autorate/
L4S (RFC 9330) Low Latency, Low Loss, and Scalable Throughput
https://datatracker.ietf.org/doc/rfc9330/
cake_mq (multi queue) https://netdevconf.info/0x19/docs/netdev-0x19-paper16-talk-paper.pdf
FQ-PIE Flow Queue PIE: A Hybrid Packet Scheduler and Active Queue Management Algorithm
https://datatracker.ietf.org/doc/draft-tahiliani-tsvwg-fq-pie/
PURPLE: Dynamic Control of CAKE
https://conference.apnic.net/60/assets/presentation-files/f6110bfa-918f-430a-95d6-60a4524d62cf.pdf
Quality of Outcome - QoO
https://datatracker.ietf.org/doc/draft-ietf-ippm-qoo/
Jonathan Morton (CAKE) is working on the next generation queuing discipline as well!
However…
Shaping inbound on the home router is the wrong place.
It is also CPU intensive there - you need a bigger router.
Requiring router upgrades is expensive.
Doing it right for thousands of customers has multiple bottlenecks.
So ISPs need to add one or more “middlebox(es)” to their network that provides shaping for all of their customers at once.
*Added after WISPAPALOOZA (what to read on the topic) 1/X
What Can I Do About Bufferbloat?
https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/
Bufferbloat and Beyond
https://bufferbloat-and-beyond.net
^^ you can get 2 physical books for FREE and download PDF!
*Added after WISPAPALOOZA (what to read on the topic) 2/X
“It’s the Latency, FCC”
https://www.fcc.gov/ecfs/document/12010651418616/1
Dave Täht, members and supporters of
the Bufferbloat.net community, in response to:
October 25, 2023 - a notice of inquiry (NOI) issued by The FCC into how well we are doing re: Section 706 of the Telecommunications Act of 1996 orders the FCC to “encourage the deployment on a reasonable and timely basis of advanced telecommunications capability to all Americans.”
More can be found here:
*Added after WISPAPALOOZA (what to read on the topic) 3/X
*Added after WISPAPALOOZA (what to read on the topic) 4/X
*Added after WISPAPALOOZA (what to read on the topic) 5/X
"I remember like it was yesterday the night I discovered that Internet capacity was not increasing as fast as UUnet/Worldcom was claiming. This was the key sentence in the paper written by Andrew Odlyzko that I read late one night in August of 2000."
https://25iq.com/2017/11/11/the-1990s-telecom-bubble-what-can-we-learn/amp/
From Andrew Odlyzko himself:
https://lists.bufferbloat.net/nnagain/alpine.DEB.2.21.2311131326070.19466@math-vinh511.math.umn.edu/
*Added after WISPAPALOOZA (what to read on the topic) 6/X
Lessons from History: The Rise and Fall of the Telecom Bubble
- DOUG O'LAUGHLIN
https://www.fabricatedknowledge.com/p/lessons-from-history-the-rise-and
Last but not least, Dave Täht et al at NANOG mailing list:
*Added after WISPAPALOOZA (what to read on the topic) 7/X
Still interested in this topic? Join Bufferbloat mailing lists - from CoDel to FQ-CoDel, CAKE, Starlink, even Net Neutrality - we’ve got you covered!
*Added after WISPAPALOOZA (what to read on the topic) 8/X
Oh and Net Neutrality was also a bufferbloat issue!
I bet you didn’t know. Here is a great explanation from Dave Täht at https://blog.cerowrt.org/tags/network-neutrality/
https://blog.cerowrt.org/post/net_neutrality_isps/
and
https://blog.cerowrt.org/post/net_neutrality_customers/
*Added after WISPAPALOOZA (what to read on the topic) 9/X
Well, and toilets!
Ever heard about that famous, never accepted and published paper from Van Jacobson et al?
RED in a Different Light - Draft of September 30, 1999
https://pollere.net/Pdfdocs/red_light.9.30.pdf
*Added after WISPAPALOOZA (what to read on the topic) 10/X
“Bufferbloat, multi-second queues in multi-gigabit routers, how come?” - Van Jacobson at Netdev0x12 (2018)
Please welcome our panelists!
Alexander Haväng (AppLogic)
Jose Lopez (Bequant)
Dmitry Moiseev (Cambium)
Dan Siemon (Preseem)
Ajith Pasqual (Paraqum)
Herbert Wolverson (LibreQoS)
What Is Your Approach to Queuing And Why?
I Have 10 Gbps, Why Should I Care?
What Are Your QoE Use Cases Aside From Typical ISP Deployments?
What Do You See As The Future Of QoE In A Post-Gigabit Bandwidth World?
Questions From The Audience
Questions From The Panel
Thank You For Joining Us
What Are You Currently Working On With Your Product / What Are You Most Looking Forward To Releasing?
Now The Obvious Question:
How Are You Utilizing AI? Is AI Part Of The Solution For QoE?
How Do You Measure “Quality”, And What Type of Experience?
And What About 9G? How Many Gs Fix QoE?
Thank You For Joining Us