Let's kill all proprietary drivers for good

Please write your questions down and we'll address them at the end

Luis R. Rodriguez - @mcgrof

Adrian Chadd - @erikarn

Latest slides: http://bit.ly/GY54IA

Linux collaboration summit

April, 2012

San Francisco, CA

Disclaimer:

Thoughts here are our own, and

does not necessarily represent Qualcomm or Qualcomm Atheros views.

Linux collaboration summit

April, 2012

San Francisco, CA

Disclaimer:

Thoughts here are our own, and

does not necessarily represent Qualcomm or Qualcomm Atheros views.

- How to kill proprietary drivers

  • Prioritize clean kernel APIs / code
  • Permissively license Linux kernel drivers
  • Localize usage of the GPL
  • BSD or BSD licensed public repo
  • Intellectual property strategies
  • Replace "internal codebases" in favor of working with two upstreams:
    • Linux upstream
    • BSD distribution and/or public BSD licenced repo
  • If driver code unification is desired, make driver unification a community problem
  • We are not attorneys; proactive engineering strategies

- Where did this come from?

  • Atheros / Sam Leffler maintains MadWifi uses net80211
  • Sam Leffler pushes net80211 to BSD
  • Linux rejects net80211 - oh well
  • Sam Leffler, FreeBSD, moves on
  • Linux gets mac80211, cfg80211
  • CRDA / wireless-regdb idea starts
  • Community upstreams ath5k with SFLC's help to share HAL code with OpenBSD
  • MadWifi / NetBSD / FreeBSD picks up Sam Leffler's HAL release
  • No more binary in HALs -- in FOSS Operating System
  • Linux community abandons MadWifi
  • Luis Rodriguez joins Atheros - Linux hacker
  • Upstreaming of the Linux ath9k driver by Atheros under ISC
  • Broadcom follows ISC license strategy
  • Adrian Chadd picks up Sam Leffler's work
  • Adrian Chadd joins Atheros - FreeBSD hacker
  • Review of internal driver infrastructure
  • Proposals for how to help internal driver infrastructure

- A pattern on proprietary drivers

Issue for all non-upstream

Linux & FreeBSD vendor drivers

- Why? How did we get here?

  • BSD, Linux, Microsoft, Apple, Solaris, QNX, etc
  • With the PC, explosion of new hardware components, hundreds of drivers for different OSes
  • With mobile, explosion of other new hardware
  • OS priorities based on type of user
  • Linux only started becoming a priority on the "desktop" recently, and now "mobile"
  • Commercial drivers: good run time tests, validation, nice shiny certification logos
  • Not all FOSS projects have good software.. but
  • Linux, BSDs: very critical software architects, customers may not care about long term benefits

- Talk is cheap

  • The easier things to do:
    • Quit our jobs
    • Not work for any silicon company
    • Run off to the Himalayas, as Sujith does regularly
    • Status quo won't change
  • Addressing the issues are not trivial
    • Politics
    • Archaic ways of doing things
    • Educational challenges
  • Working with the community
    • Perhaps this may work
    • Let's try - need to educate the community too!
  • First review internally, then reach out to the community

- What else is involved in delivering a driver?

It's not just about writing code! That may even work!

  • There may be compliance testing and certifications needed before a product can be sold
  • There may be some silly vendor specific test code (CONFIG_EXPERT)
  • There may also be (a lot, I hope) a lot of internal standards and regression testing
  • There may be some very specific customer extensions that they've paid for, but not allowed for public consumption
  • There may be cross-licensing of patents..
    • This is more prevalent with Video hardware
  • There may be some commercial agreements
  • When hardware comes out, customers expect drivers that day

These may make it difficult to open up a commercially developed codebase

  • All of the above needs to be addressed
  • Break down the problems one by one

- What about multi-OS drivers?

  • Because there's more to writing a driver than just writing the code..
  • .. there may be pressure to share as much code as possible between platforms
  • The Atheros unified drivers targets, amongst others:
    • Windows (XP, Vista, 7, 8)
    • Linux (various 2.6, 3.x kernels)
    • Legacy - vxWorks, *BSD
  • Compare the complexity of a simple 10/100 Ethernet NIC driver to what is required for 802.11 support
    • Do you really think it's feasible to reimplement, from scratch, the entirety of an 802.11 stack for each operating system?
    • Will each OS / stack require its own full tests?

- Multi-OS drivers - the problem

  • But how much can you really share?
    • What is actually, fundamentally different between OSes?
      • device model, threading/locking model, network model, bus model..
  • What do you end up having to do?
    • Find the "middle ground" between all OS platforms
    • This may not be the most { optimal, efficient, concise } way to solve the problem
  • Every company likely does it completely differently
    • Tailored to their specific device needs
    • Can these even co-exist between companies?
    • Companies reinventing the wheel, year after year
    • Why?!

- What is a crap driver?

  • Linux, anything under drivers/staging TAINT_CRAP real technical term
  • diff --stat v3.3..v3.4-rc1 drivers/staging
  • 426 files changed, 50507 insertions(+), 16196 deletions(-)
  • 4032 files changed, 294677 insertions(+), 164920 deletions(-)
  • Linux crap drivers can address run time functionality, really really well
    • Doesn't mean its good
  • Don't use the word "crap" -- banned -- leads to crap
  • Accepting crap means you realize you can improve software
  • Does not prioritize code readability and long term maintenance
  • Does not consider an ecosystem for reinventing the wheel on fixing issues
  • Branching hell, customers fork all your work
  • Even good non-staging drivers / software can become crap, over time you evolve, its quality can be deemed crap
  • Examples even in the Linux kernel:
    • Linux wireless regulatory, evolution
      • Big vendor crap --> upstream --> crap --> re-architect
      • Regulatory simulator in userspace
    • Staging drivers, compat-wireless crap/ for patches - not ready
  • Move on, adapt fast, evolve, accept criticism

- Improve internal driver codebase

  • Linux porting issues:
    • Always have to clean code up
    • Community collaborations not being merged back
    • QCA developer program
  • Code modularization / component owners
  • Atomicity of changes
  • Annotation of fixes (Cc: stable)
  • Distributed development model
  • Clean kernel APIs / code
  • Linux kernel bootcamp for developers
    • Exposure to the process, quality of code
  • Benefits of collaborative development
  • Historically one license for one single project
  • BSD license first, GPL evolution
  • Linux embraces GPLv2
  • FSF engineers GPLv3 - even an edit by IBM, RH, Intel, Novell
  • Linux stays on GPLv2 - why? GPLv2 is ancient. Not easy to move, etc
  • Linux and other projects use "Dual BSD/GPL" for some things, ambiguity created
  • SFLC: GPL-Compatible OK!
    • ath5k: HAL uses simple ISC, 2 Clause BSD license
      • Changes-licensed-under - not needed if you educate properly
      • Signed-off-by - Educate on usage, spread it in other projects!
    • ath9k: first fully permissively licensed driver on Linux
    • Solaris ports ath9k to OpenSolaris, OpenSolaris dies
    • Broadcom follows ISC license approach
  • Call for drivers to be permissively licensed
  • Core kernel functionality: GPL
  • Counter argument: only GPL enforcement will ensure giving back
    • Alternative: proactive engineering to address corporate / FOSS

- Are non-Linux GPL drivers possible?

  • Yes but its a pain
  • Andy Grover:
    • GPLv2 - paravirt Windows driver
  • Writing GPLv2 Windows drivers means you become a minGW developer
  • http://groveronline.com/2008/07/mingw-cross-compilation-adventure/
  • So possible... but painful at present
  • Intel drivers: some permissively licensed files, mostly headers
  • Better use a fully permissive licensed Linux drivers, share with BSDs

- Non-Linux upstream: BSD

  • Motivate BSD contributions by vendors:
    • Guidelines for using permissively licensed drivers on Linux - TBD
    • If you can address permissively licensed Linux drivers you should be able to address BSD as well
  • Why bother?
    • Linux - "forcing" vendors to a software/development standard
    • .. but they still have other platforms to target - Windows, MacOS, etc
    • Can we (partially) reduce their engineering effort to maintain multiple code bases? Especially where Windows (may) ship more units?
    • Not simply about dumping horrible code as BSD! Open all drivers.
  • We want to encourage companies to participate in open development
    • Use BSD or compatible licences
    • Not a replacement for Linux (and *BSD! :) upstream!
    • But you still need to address patent grants
      • Arguably permissive licenses have implicit patent grant
      • Better try to avoid patents in kernel for both BSD and Linux!
  • Open driver participation will help both the company and the community!
    • How many community members see how hardware is developed?

- Intellectual Property

  • Patent trolling:
    • Industry is patent trigger happy
    • Software patents in FOSS: can of worms
    • This issue sucks, let us, engineers simplify the problem!
  • Kernel:
    • Linux kernel GPLv2: deal with it!
    • Help sharing: localize GPL
      • New drivers: permissively licensed
    • Prioritize simple and clean kernel API / drivers over adding IP in kernel. Early architectural review to address IP
    • Punt IP to userspace, otherwise:
      • Explicit patent grant: RCU -- GPLv2 + sort of explicit patent grant Documentation/RCU/RTFP.txt
    • BSD kernel: permissively license patent grant theory
    • ClearBSD license: not accepted, we tried, although GPLv2 compatible
  • Userspace:
    • WebM VP8 lesson: FOSS license + separate patent grant
    • Open solution + explicit patent grant for hardware vendor?
    • Other ideas? Innovate strategies
  • We are not attorneys!
    • Attorneys can address this at the Legal tracks
    • We are just engineers - this is just proactive strategy ideas, brainstorming

- Business justifications

  • Streamline / lead Operating System ecosystems
  • Bring benefit of collaborative development to non-Linux Operating Systems
  • Can the community be relied on?
    • Motivation - understand it
    • Innovation - can leverage off of this work
    • Geographically spread
    • Talent scouting
  • Can the community be accounted for?
    • Hobbyists - top contributor to the Linux kernel
    • GSoC

- Collaborative development

  • Hobbyists, top contributors
    • Linux
    • BSD
    • Wikipedia

  • Ideal situation: a good balance
    • Educate: NDA, public specs, wikis
    • Stimulate motivation
    • Stimulate innovation

  • Can you rely on this? Let's review history

Linux kernel contributions

Linux kernel ath9k contributions

- What are we doing doing?

  • Linux upstream already a priority
    • Target upstream first always - by product of some porting right now though!
    • Automatically backport to older kernel releases: wireless, ethernet, bluetooth
    • http://phb-crystal-ball.org/ - Guestimating when Linux will release the next kernel
    • CONFIG_EXPERT for vendor specific stuff
  • Adrian joins Atheros - FreeBSD developer
  • Lets start off simple, address an Ethernet driver first, Linux alx driver:
    • Rejected on good technical grounds
    • Permissive license considerations - OK for newer chipsets only
    • Example of possible issues in driver unification utopia
  • The simpler the better to start, what's next? Bluetooth: too simple, what's next?
  • We're hiring ;)

- Can we do better?

  • Is driver unification a pipedream?
  • Can we unify drivers on at least
    • Linux
    • BSD / BSD repo
  • Three options:
    • Help with unification - what's in it for us?
      • Better coordination, testing, do we care?
    • Go away: punt problem back to companies
      • Only two upstreams possible: BSD, Linux
    • Open up your crap: open internal driver codebase, used for staging as software architecture / APIs mature?

- If unification is possible...

  • Lesson to be learned:
    • Corporations reinventing the wheels, horribly
    • Can the community help?
    • If we want to help kill proprietary drivers, lets talk about it
  • Live with at the very least two upstreams with feature parity:
    • Linux
    • BSD / BSD repo
  • coccinelle spatch
    • SMPL - language of what patches should look like
    • Originally for collateral evolution of Linux
      • Address kernel API changes / evolution
    • Used in the Linux kernel for code sanity checks
    • Used to help clean drivers out of drivers/staging
  • spdiff
    • Jesper Andersen & Julia L. Lawall - Generic Patch Inference
    • Lets you skip writing SMPL
    • Automatic inference of high level specifications of changes to structured data
    • Give it two patches: it gives you SMPL

- What now?

  • What can we do to do more open driver development (not just upstream Linux)?
  • What verification / validation features are needed for engineering (non wireless)?
  • What experiences do other companies have?
  • SMPL review by a Linguistics major - Adrian
    • Implications on driver design to help SMPL with collateral evolutions

The end

Objectives to consider for killing proprietary drivers in the long run as engineers. Each one may take time to achieve.

  • Architecturally punt IP "problem" to userspace - where appropriate
    • .. or where allowed - eg userland DRM API for video?
  • Dual BSD / Linux strategy
  • Encourage open driver development for all platforms, not just Linux upstream
    • Possble outcome - driver unification

Attorney's homework: FOSS friendly IP strategies in userspace