Unified Network Configuration
Why?
NetworkManager is not suitable for inclusion in the initrd (dracut). For the initrd use case we require a minimal, static network configuration file format that is suitable for network booting and a couple of other things.
On top of that we also believe that NetworkManager is not always necessarily the best choice for the server use case. Server setups require simple, predictable, static, fixed configuration, while NetworkManager provides adaptive, dynamic, interactive configuration.
To deal with the initrd use case we are planning to add a basic network configuration toolset to systemd (which Dracut is now based on), and we hope that this toolset might be useful even outside the immediate use case, i.e. on servers (and embedded systems).
With this specification we hope to turn the configuration files of these systemd tools something we can share among multiple implementations, so that configuration created with NetworkManager can easily be used by the initrd. Independent of the used software implementation, the network configuration format should be sharable between the minimal, static network configuration system we propose for the initrd and the server, and NetworkManager, as well as other networking configuration systems such as Intel’s ConnMan. (Side note: the upstream ConnMan maintainer has already agreed to share a single configuration file format with systemd).
With this specification we want to define a unified format that is acceptable to all these uses, so that the initrd, static server setups, NetworkManager, and ConnMan share the same format, so that network configuration can be moved around between the implementations freely, and in a way that multiple network configuration systems can cooperate nicely. It is key that the various implementations may run side-by-side safely, share configuration, but do not step on each other’s toes.
Compatibility with Fedora/Red Hat-style network configuration as implemented by the initscripts package we intend to provide with runtime “generators”, which pick up and translate these configuration files in the new format at runtime.
General file file formats concepts
We use Windows-style .ini files (i.e. .desktop files) but we define additional semantics, which keep the parser and formatter compatible with GLib’s GKeyFile but allow us to include lists of objects (such as routes, or addresses) while keeping the syntax attractive for administrators who are expected to write this files on their own, with simple tools such as a text editor. More specifically, objects get sections of their own, where the section type prefix is separated from a free form identifier suffix by “:”, example: [IP Route:0], [Address:dmz] and so on.
We isolate link (i.e. low-level interface data) from network (i.e. high-level interface data) information. We place these in separate (but individually “atomic”) files, instead of having one file for all configuration. When configuration is applied to an interface we apply (zero or) one link and (zero or) one network configuration file to it. By isolating link from network we make it easy to move IP configuration between interfaces (and machines), and allow multiple IP configurations for a link that are applied at different times.
The link configuration files are intended to be relatively local to the machine, while the network files are specific to the network, and make sense to be copied between machines (for example, for usage in a Red Hat VPN RPM containing all network definitions for all VPN concentrators).
Link configuration is supposed to be only applied at udev/systemd/CLI level, it is of little relevance to NetworkManager or ConnMan. Network configuration OTOH might be applied by either systemd, NetworkManager or ConnMan.
It is important to underline that we’ll ship a catch-all link file that names devices after their BIOS names, which should be sufficient for most setups. That means link files are not commonly used, and primarily something intended for usage by distributors and hardware vendors rather than system administrators. The split between link and network files is hence nothing system administrators are mandatorily exposed to.
Key here is to make these files attractive for administrators to read and to write at the same time as they are easily generated and parsed from system services such as SCC, NM, systemd. Hence: no XML, no JSON.
Since the matching logic for link and network files is not trivial (though not really complicated either) we will provide a C library interface which implements the lookup, and is available to SCC and NM. It will not implement configuration file parsing, and expects these clients to parse the files on their own, for example via GKeyFile.
Note that at this time we have no intention of defining the individual fields in all detail. Before we write a specification for all these numerous fields we’d like to collect feedback on the general format and structure of the files.
Directory structure
/etc/net/
The main directory for all network settings.
/etc/net/links/*.link
/etc/net/networks/*.network
Directories for interface configuration, contains .link resp. .network files to configure network links and network configuration. .link files are used to configure the actual devices with device properties such as a name or which component (systemd or NM) control the device. .network files actually apply IP configuration (and related) to a device. (.network is hence similar to NetworkManager’s “connections” concept).
On desktop setups we don’t expect that .link files even exist, except for generic ones shipped by the distribution. On server setups they are more likely to exist, but not required either.
The .link files will be applied exclusively by systemd/udev (or by low-level user CLI), while the .network files are to be applied by the respective network management implementation, such as NetworkManager, ConnMan, or systemd.
The name of .link and .network files may be chosen relatively freely. However, if a .link or .network file bears the name of a kernel interface prefixed by “auto-” it will be activated automatically on device hotplug for that interface. (Example: “auto-em1.network”). If prefixed with two-digit numeric prefix the file is included in automatic priority-based matching, and the digits ensure appropriate ordering. (Example: “50-foobar.link”). If it is prefixed with neither it is not included in any automatic applying algorithms.
We’d expect that frequently symlinks are used for “auto-xxx.network” which then points to the actual network file (which is then more descriptively named). But this is not a requirement.
/usr/lib/net/
A similar directory structure as /etc/net/, but for vendor supplied definitions. This is useful to define vendor VPNs, apply default naming rules via .link files, or change names/attributes of certain interfaces by default on specific systems. Files in /etc/run which bear the same name as files in /usr/lib/net/ override the latter.
/run/net/
Also a, similar directory structure as /etc/net/ but suitable for volatile, non-persistent configuration. Files in /run/net/ do override files in /usr/lib/net/, but not those in /etc/net/ if the file names match.
All configuration files in these directories MUST be written atomically (i.e. created under a temporary file name, whose suffix is neither .link nor .network), and then atomically renamed to its names. This is required to avoid races and ensure inotify() on these directories works fine to implement live views of available network configuration.
Both .link and .network files follow the same basic scheme. They begin with a [Match] section that explains what interface the configuration file applies to:
[Match]
VisibleTo=none|system|manager
MACAddress=00:21:86:A2:3C:E5
DevicePath=pci-0000:00:1a.7
Name=lom0
Subsystem=pci
Driver=e1000
FirmwareSlot=2
Type=ethernet|wlan|...
If a kernel network interface is to be configured automatically we try to find a .link and a .network file for it, and apply it. The algorithm for mapping a kernel interface to a pair of .link and .network files is like this:
A [Match] section in a .link or .network file applies when all individual match lines of it apply. The example above lists all possible match expressions, but we’d expect that usually people will only use one match expression, for example MACAddress=. If the [Match] section is empty or does not exist a .link or .network file applies to no interface.
One of the primary reasons for .link files is to assign stable names to interfaces, which are strictly required in some more complex setups or for compatibility reasons.
If a network device is to be configured manually, the user may specify for this operation one .link and one .network file explicitly, or only one of them. The missing file is automatically deduced following the normal scheme for automatic configuration, following the algorithm pointed out above. The tools will refuse applying .network and .link files to interfaces which don’t match their [Match] section (but this could be overridable by a --force command line option or similar).
The VisibleTo= setting in the [Match] specifies which implementation shall see this configuration file. If set to “none” the file will only be considered if the user explicitly asks for it, but never automatically by NetworkManager or by the systemd tools. If set to “system” it is exposed by systemd on device hotplug (that means, considered in the algorithm described above). If set to “manager” it is only visible to NetworkManager’s logic which decides when to apply it (which could be device and/or cable hotplug). If the option is missing the configuration file is visible to all implementations. Using VisibleTo= is useful for hiding large configuration sets from a certain or all UIs.
Note, that by default, all configuration files and all network devices are visible to all implementations at least in a read-only fashion. The VisibleTo= option may be used to exclude certain configuration files from the UI, simply in order to reduce visual clutter and confusion.
.link files contain configuration information about the link in the [Link] section:
[Match]
Type=ethernet
VisibleTo=none|system|manager
[Link]
Description=Yellow Ethernet Connector
Name=yellow0
NamePolicy=none|manual|path|mac|firmware
ManagedBy=none|system|manager
[Ethernet]
SpeedMbit=1000
DuplexMode=full|half
WakeOnLan=phy|magic|off
MACAddress=00:21:86:A2:3C:E5
MTUBytes=1500
The Name= field may be used to rename the kernel interface as it becomes available. NamePolicy= is closely related and allows configuration of automatic naming policies, i.e. where the device name is generated automatically from the device path, from the MAC address or from the firmware assigned slot id. If this option is not set “manual” is implied if Name= is set, otherwise “none”. Description= is a human readable string, that is configured to the kernel interface in the “alias” field.
The ManagedBy= setting may be used to control which implementation should take possession of a network device. It controls for the network device what VisibleTo= controls for the configuration files: whether they shall be visible and exposed.
Note that ManagedBy= and VisibleTo= encode strong hints, but should not totally detach implementations from certain configuration files: “read-only” exposure is OK, and in fact desirable. To be more specific: if ManagedBy=system is set this means NetworkManager should avoid applying configuration or make it editable but it should still show the properties of the device.
.network files contain configuration information about a specific network:
[Match]
Type=ethernet
Mode=static
[Network]
Description=Foocorp LAN
[IP]
Mode=dhcp|static
Address=192.168.0.1/24 ; 2001:6f8:11d6:0:221:86ff:fea2:3ce5/64
Gateway=192.168.1.1
DNSAddress=192.168.0.253
DNSSearchDomain=dns.example.com
IPv4LinkLocal=yes|no
HTTPProxy=http://proxy.redhat.com:3128/
[IP Route:anothernet]
Scope=global
To=192.168.200.0/24
Label=anothernet
Gateway=192.168.1.100
[IP Route:athirdnet]
Scope=global
To=192.168.220.0/24
Label=athirdnet
Gateway=192.168.1.155
Note, that we have multiple routes here for the same interface, in two [IP Route:xxx] sections. Also note that [IP]’s Gateway= is simply short for an [IP Route] section for the default route.
All configuration options which may be specified in .link files may also be specified in .network files, since sometimes the line is blurry where a certain setting belongs. (Example: MTU might need adjusting on a specific network, or due to some specific hardware). However, the reverse is not true. Configuration options such as IP address or routes may not be specified in a link configuration file.
If an option appears in both the .link and the .network file, then the latter will win, thus allowing configuration of a default in .link, which is overridable for a specific network.
Passphrases: for passphrases we’ll have three modes: Specified in the .network files, specified in an external file whose path is declared in the .network files, or if neither is set the system will ask the user.
Locking
Since multiple implementation might want to take possession of the same network interface at the same time we need to make sure that they don’t step on each other’s toes. In order to accomplish this we ask implementations to take a R+W POSIX lock on the file /run/net/lock/<ifindex>.lock as long as they manage that device. The file must have an access mode of 0600, to avoid that unprivileged users can take this lock.
Examples
Example 1: The most basic example is an empty /etc/net/networks/ directory. In that case no link or network configuration is applied.
Example 2: Server with one ethernet interface “em1”, whose name is derived by the logical BIOS-provided slot number:
/etc/net/networks/auto-em1.network
[IP]
Address=192.168.1.1
Example 3: For embedded cases we define a catch-all network :
/etc/net/networks/70-default-ethernet.network
[Match]
Type=ethernet
[IP]
Mode=dhcp
We pick a name that is alphabetically near the end, so that it only applies as fallback if nothing else applies first.
Example 4: Desktop machine with NM in power for all WLAN interfaces
/usr/lib/net/links/95-default-wlan-NM.link
[Match]
Type=wlan
Mode=managed
With this dropped in all WLAN interfaces are controlled by NM. NM should then create .network files as needed for persistent configuration.
We install this file in /usr instead of /etc, since it’s a package default.
Example 5: WLAN router with a wired and a wireless (WLAN) interface, set up for Bob’s home network:
/etc/net/links/50-wired.link
[Match]
Type=ethernet
[Link]
Name=copper
/etc/net/links/50-wireless.link
[Match]
Type=wlan
[Link]
Name=air
/etc/net/networks/50-bobs-lan.network
[Match]
Type=ethernet
[IP]
Mode=static
Address=192.168.1.1/24
/etc/net/networks/50-bobs-wlan.network
[Match]
Type=wlan
[IP]
Mode=static
Address=192.168.2.1/24
We rename the interfaces, to make things more explanatory, and then configure an IP address on each of the interfaces. Since there’s only one .network file around that applies to each interface we do not need to care about the names.
Example 6: A machine that is frequently moved between two different ethernet LANs:
/etc/net/links/50-wired.link
[Match]
Type=ethernet
MACAddress=11:22:33:44:55:66
[Link]
Name=myethernet
/etc/net/networks/50-at-home.network
[Match]
Name=myethernet
[IP]
Mode=static
Address=192.168.1.55/24
Gateway=192.168.1.1
/etc/net/networks/50-coffee-shop.network
[Match]
Name=myethernet
[IP]
Mode=dhcp
This renames the local interface with the MAC address 11:22:33:44:55:66 to “ethernet” and then defines two configurations for it, one for home and one for the coffee shop. By default the home configuration will apply, since it begins with “a” which is alphabetically before “c”. The user needs to use some command line tool to change between the interfaces. If the user wants to make 50-coffee-shop.network the default he could create a symlink:
/etc/net/networks/auto-myethernet.network →
/etc/net/networks/50-coffee-shop.network
Example 7: A VPN client
/etc/net/networks/50-redhat.network
[Link]
Type=cisco-vpn
Name=redhat0
[Network]
Description=Red Hat VPN
[Cisco]
User=bob
Password=xxxxxx
Gateway=47.11.08.15
This creates a VPN interface (i.e. one that is created on user requested, but never found by match), and then configures Cisco VPN parameters for it. In order to keep things simple we encode both Link and Network information in one file here. (As mentioned, we can encode all Link parameters in network files too, but not vice versa)
Example 8: A machine where all Ethernet interfaces showing up shall be assembled as they show up
/etc/net/links/50-bridge.link
[Link]
Type=bridge
Name=bridge0
/etc/net/networks/50-bridge.network
[Match]
Name=bridge0
[IP]
Address=192.168.1.1/24
/etc/net/links/50-default-bridge.link
[Match]
Type=ethernet
[Bridge]
Master=bridge0
The nice thing here is that we have a catch-all file that just adds interfaces as they show up.
Example 9: a machine with one bridge of two interfaces (one wireless, one wired) that shall only be assembled if both are found.
/etc/net/links/50-bridge.link
[Link]
Type=bridge
Name=bridge0
[Bridge]
Slaves=wired0 wireless0
/etc/net/networks/50-bridge.network
[Match]
Name=bridge0
[IP]
Address=192.168.1.1/24
/etc/net/links/50-wired.link
[Match]
Type=ethernet
[Link]
Name=wired0
/etc/net/links/50-wireless.link
[Match]
Type=wlan
[Link]
Name=wireless0
Example 10: a machine with one ethernet and two VLAN interfaces on it:
/etc/net/links/50-wired0.link
[Match]
MACAddress=11:22:33:44:55.66
[Link]
Name=wired0
/etc/net/links/50-company.link
[Link]
Type=vlan
Name=company0
[VLAN]
ID=7
Master=wired0
/etc/net/networks/50-company.network
[Match]
Name=company0
[IP]
Address=192.168.1.1/24
/etc/net/links/50-dmz.link
[Link]
Type=vlan
Name=dmz0
[VLAN]
ID=9
Master=wired0
/etc/net/networks/50-dmz.network
[Match]
Name=dmz0
[IP]
Address=192.168.2.1/24
Example 11: WRTG54g-style setup which is 5 ethernet ports which can be distinguished via VLAN IDs, plus one wlan port. (This is DanW’s first example)
/etc/net/links/50-switch.link
[Match]
Type=ethernet
[Link]
Name=switch
/etc/net/links/50-lan.link
[Link]
Type=vlan
Name=lan
[VLAN]
ID=1
Master=switch
/etc/net/links/50-wan.link
[Link]
Type=vlan
Name=wan
[VLAN]
ID=2
Master=switch
/etc/net/networks/50-wan.network
[Match]
Name=wan
[IP]
Mode=dhcp
/etc/net/links/50-wlan.link
[Match]
Type=wlan
[Link]
Name=wlan
/etc/net/links/50-bridge.link
[Link]
Type=bridge
Name=bridge
[Bridge]
Slaves=wlan lan
/etc/net/network/50-bridge.network
[Match]
Name=bridge
[IP]
Address=192.168.1.1/24
Example 12: Same as 11, but with redundancy (DanW’s second example)
/etc/net/links/50-switch.link
[Match]
Type=ethernet
[Link]
Name=switch
/etc/net/links/50-lan.link
[Link]
Type=vlan
Name=lan
[VLAN]
ID=1
Master=switch
/etc/net/links/50-wan1.link
[Link]
Type=vlan
Name=wan1
[VLAN]
ID=2
Master=switch
/etc/net/links/50-wan2.link
[Link]
Type=vlan
Name=wan2
[VLAN]
ID=3
Master=switch
/etc/net/links/50-wan.link
[Link]
Type=bond
Name=wan
[Bonding]
Slaves=wan1 wan2
/etc/net/networks/50-wan.network
[Match]
Name=wan
[IP]
Mode=dhcp
/etc/net/links/50-wlan.link
[Match]
Type=wlan
[Link]
Name=wlan
/etc/net/links/50-bridge.link
[Link]
Type=bridge
Name=bridge
[Bridge]
Slaves=wlan lan
/etc/net/networks/50-bridge.network
[Match]
Name=bridge
[IP]
Address=192.168.1.1/24
Example 13: Same as 12, but with libvirt in the mix (DanW’s third example)
/etc/net/links/50-switch.link
[Match]
Type=ethernet
[Link]
Name=switch
/etc/net/links/50-lan.link
[Link]
Type=vlan
Name=lan
[VLAN]
ID=1
Master=switch
/etc/net/links/50-wan1.link
[Link]
Type=vlan
Name=wan1
[VLAN]
ID=2
Master=switch
/etc/net/links/50-wan2.link
[Link]
Type=vlan
Name=wan2
[VLAN]
ID=3
Master=switch
/etc/net/links/50-wan.link
[Link]
Type=bond
Name=wan
[Bonding]
Slaves=wan1 wan2
/etc/net/links/50-wan-vnet-bridge.link
[Link]
Type=bridge
Name=wan-vnet-bridge
[Bridge]
Slaves=wan # The wan link we add from “this side”, the vnets from the “other”
/etc/net/links/99-default-vnet-bridge.link
[Match]
Type=virtual
[Bridge]
Master=wan-vnet-bridge
/etc/net/networks/50-wan-vnet-bridge.network
[Match]
Name=wan
[IP]
Mode=dhcp
/etc/net/links/50-wlan.link
[Match]
Type=wlan
[Link]
Name=wlan
/etc/net/links/50-bridge.link
[Link]
Type=bridge
Name=bridge
[Bridge]
Slaves=wlan lan
/etc/net/networks/50-bridge.network
[Match]
Name=bridge
[IP]
Address=192.168.1.1/24
Example 14: Company supplied VPN, WLAN, and LAN definitions (may be installed by a company .rpm)
/usr/lib/net/networks/50-redhat-lan.network
[Match]
Mode=manual
Type=ethernet
[Network]
Description=Red Hat Ethernet
[IP]
Mode=dhcp
HTTPProxy=http://proxy.redhat.com:3128
NTPServer=ntp.redhat.com
/usr/lib/net/networks/50-redhat-wlan.network
[Match]
Mode=manual
Type=wlan
[Network]
Description=Red Hat Wireless
[IP]
Mode=dhcp
HTTPProxy=http://proxy.redhat.com:3128
NTPServer=ntp.redhat.com
[WLAN]
ESSID=Red Hat Wireless
WPAPassphrase=supersecret
/usr/lib/net/networks/50-redhat-vpn-boston.network
[Link]
Type=cisco-vpn
Name=redhat0
[Network]
Description=Red Hat VPN (Boston)
[Cisco]
Gateway=47.11.8.15
/usr/lib/net/networks/50-redhat-vpn-amsterdam.network
[Link]
Type=cisco-vpn
Name=redhat0
[Network]
Description=Red Hat VPN (Amsterdam)
[Cisco]
Gateway=11.22.33.44
These network definitions are shipped in the RPM in /usr/lib/net/networks (instead of /etc/net/networks), since they are supplied by the company. Due to Mode=manual in the [Match] sections we ensure that these configurations are not applied automatically, but only when the user explicitly chooses them. (Note that for VPNs we don’t need to place Mode=manual since these interfaces only show up on demand and manually, never due to hotplug).
A user who wants to make use of these definitions would then simply activate them for his ethernet card “foobar0” by symlinking:
/usr/lib/net/networks/redhat-lan.network →
/etc/net/networks/auto-foobar0.network
Example 15: Distribution default to rename all ethernet interfaces to their BIOS slot (or device path if no BIOS slot is known) that have no other .link file that applies:
/usr/lib/net/links/90-fedora-rename.link
[Match]
Type=ethernet
[Link]
NamePolicy=firmware path
This logic is supposed to provide biosdevname-like functionality with a simpler logic.
Usage by Applications
Let’s have a closer look how SCC, how NM, how udev and how systemd interact with the network configuration files.
Network Manager should not expose a configuration UI (or apply configuration for) .link files, only for .network. When creating new .network files it should probably match devices only by type, not by MAC/Path/BIOS ID. It should apply its magic only for interfaces whose .link file has ManagedBy= not set or where the value is “manager”. It should expose all other links in the UI too, but not allow reconfiguration and upping/downing. When it wants to store information about a new network it should generate a new .network file name for that that includes information about the network to make it sufficiently explanatory (so that admins can recognize the network when they browse through /etc/net/networks/). If ESSIDs and suchlike are not good as identifiers NM should try to make them good for that, for example, by appending -2, -3, -4 or so at the end. (That said, if NM really really wants to use UUIDs for this it can do so, and use them in the file name)
SCC should find the .link and first .network interface that matches, and make it editable if it is in /etc. Otherwise, make it editable but store the data in a new file in /etc. If none is found it should create one, under a user supplied name, and add a match on the BIOS logical ID (if exists), and device path otherwise. (And probably not MAC address, to make hw replaceable, and configuration copyable). It should set ManagedBy=system. It should ignore all interfaces where ManagedBy=manager.
systemd will never edit configuration files. It will (via udev) apply .link configuration for all devices, based on the matching algorithm above. It will also apply .network configuration if Managed= is set to “system” or ManagedBy= is not set at all. It will automatically apply only the first .link and first .network file, but allow users to apply any other file with a command line tool. It will not do link-beat magic.
udev will not deal directly with .link nor .network files but will execute udev rules a “generator” supplied with systemd will create dynamically (at boot and config reload) from the .link and .network files, for a small number of reasons: to rename the interface as it shows up to the right name, and to pull in the systemd network services as necessary.
NOTE: NM only reads, applies and cares for .network files. .link files are applied already at systemd/udev level for it. .link files is nothing NM should ever care about beyond the ManagedBy= field.
Proposed API to Find Configuration Files
As mentioned above we plan to provide a simple API that can be used by the various implementations to find the right configuration file for a network interface. The API will probably look something like this:
typedef struct sd_net sd_net;
/* creates a context object */
sd_net *sd_net_new(const char *whoami);
/* whoami = "system" | "manager" */
void sd_net_free(sd_net *n);
void sd_net_reset();
/* Asks whether a specific interface shall be managed by the app */
bool sd_net_shall_manage(sd_net *n, int ifindex);
/* Gets a list of .link configuration files suitable for the specific interface, or if -1 is passed all configuration files. Fetch the list iteratively with sd_net_next(). */
int sd_net_find_links(sd_net *n, int ifindex);
/* Similar, but for .network files. */
int sd_net_find_networks(sd_net *n, int ifindex);
/* Returns the next .link or .network configuration file, with path, contents and mtime. The caller should pass the returned data pointer into gkeyfile. Each time this function is called will return the next entry.*/
int sd_net_next(sd_net *n, char **ret_path, char **ret_data, struct timespec *ret_mtime);
/* Returns an fd that will get POLLIN whenever the configuration changes. */
int sd_net_notify_fd();
int sd_net_notify_flush();
/* Write a configuration file. Returns the new mtime. */
int sd_net_write(sd_net *n, const char *filename, const char *data, char **ret_path, struct timespec *ret_mtime);
/* Removes a configuration file */
int sd_net_remove(sd_net*n, const char *path);
This is only a suggested API. Take it with a grain of salt.