Network Device Drivers
Dr A Sahu
Dept of Comp Sc & Engg.
IIT Guwahati
Outline
PCI Configuration Header
Status
Register
Command
Register
Device
ID
Vendor
ID
BIST
Cache
Line
Size
Class Code
Class/SubClass/ProgIF
Revision
ID
Base Address 0
Subsystem
Device ID
Subsystem
Vendor ID
CardBus CIS Pointer
reserved
capabilities
pointer
Expansion ROM Base Address
Minimum
Grant
Interrupt
Pin
reserved
Latency
Timer
Header
Type
Base Address 1
Base Address 2
Base Address 3
Base Address 4
Base Address 5
Interrupt
Line
Maximum
Latency
31 0
31 0
16 doublewords
Dwords
1 - 0
3 - 2
5 - 4
7 - 6
9 - 8
11 - 10
13 - 12
15 - 14
Three IA-32 address-spaces
memory
space
(4GB)
i/o space
(64KB)
PCI
configuration
space
(16MB)
accessed using a large variety of processor
instructions (mov, add, or, shr, push, etc.)
and virtual-to-physical address-translation
accessed only by using the processor’s
special ‘in’ and ‘out’ instructions
(without any translation of port-addresses)
i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space
Interface to PCI Configuration Space
reserved
CONFADD
( 0x0CF8)
CONFDAT
( 0x0CFC)
31 23 16 15 11 10 8 7 2 0
E
N
bus
(8-bits)
device
(5-bits)
doubleword
(6-bits)
function
(3-bits)
00
PCI Configuration Space Address Port (32-bits)
PCI Configuration Space Data Port (32-bits)
31 0
Enable Configuration Space Mapping (1=yes, 0=no)
Reading PCI Configuration Data
How ‘transmit’ works
descriptor0
descriptor1
descriptor2
descriptor3
0
0
0
0
Buffer0
Buffer1
Buffer2
Buffer3
List of Buffer-Descriptors
We setup each data-packets that we want to be transmitted in a ‘Buffer’ area in ram
We also create a list of buffer-descriptors and inform the NIC of its location and size
Then, when ready, we tell the NIC to ‘Go!’ (i.e., start transmitting), but let us know when these transmissions are ‘Done’
Random Access Memory
Registers’ Names
Tx-Desc Ring-Buffer
Circular buffer (128-bytes minimum)
TDBA
base-address
TDLEN
(in bytes)
TDH (head)
TDT (tail)
= owned by hardware (nic)
= owned by software (cpu)
0x00
0x10
0x20
0x30
0x40
0x50
0x60
0x70
0x80
Tx-Descriptor Control (0x3828)
0
0
0
0
0
0
0
G
R
A
N
0
0
WTHRESH
(Writeback Threshold)
0
0
0
FRC
DPLX
FRC
SPD
0
HTHRESH
(Host Threshold)
I
L
O
S
0 0
A
S
D
E
0
L
R
S
T
0 0
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
0
0
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
PTHRESH
(Prefetch Threshold)
0
0
Recommended for 82573: 0x01010000 (GRAN=1, WTHRESH=1)
“This register controls the fetching and write back of transmit descriptors.
The three threshhold values are used to determine when descriptors are
read from, and written to, host memory. Their values can be in units of
cache lines or of descriptors (each descriptor is 16 bytes), based on the
value of the GRAN bit (0=cache lines, 1=descriptors). When GRAN = 1,
all descriptors are written back (even if not requested).” --Intel manual
Transmit Control (0x0400)
R
=0
R
=0
R
=0
MULR
TXCSCMT
UNO
RTX
RTLC
R
=0
SW
XOFF
COLD (upper 6-bits)
(COLLISION DISTANCE)
COLD (lower 4-bits)
(COLLISION DISTANCE)
0
ASDV
I
L
O
S
S
L
U
TBI
mode
P
S
P
0 0
R
=0
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
R
=0
E
N
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
SPEED
CT
(COLLISION THRESHOLD)
EN = Transmit Enable SWXOFF = Software XOFF Transmission
PSP = Pad Short Packets RLTC = Retransmit on Late Collision
CT = Collision Threshold (=0xF) UNORTX = Underrun No Re-Transmit
COLD = Collision Distance (=0x3F) TXCSCMT = TxDescriptor Minimum Threshold
MULR = Multiple Request Support
82573L
Tx Configuration Word (0x0178)
82573L
ANE
Tx
Config
ITCE
R
=0
IAME
R
=0
DF
PAR
EN
PB
PAR
EN
Tx
LS
Tx
LS
Flow
=0
R
=0
Phy
Pwr
Down
En
DMA
Dyn
GE
R
=0
RO
DIS
Reserved
(=0)
SPD
BYPS
R
=0
EE
RST
ASD
CHK
R
=0
R
=0
R
=0
R
=0
R
=0
R
=0
R
=0
R
=0
0 0
TxConfigWord
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
ANE = Auto-Negotiation Enable
TxConfig = Transmit Configuration Control bit
TxConfigWord = Transmit Configuration Word
This register has two meanings, depending on the state of the ANE bit
(i.e., setting ANE=1 enables the hardware auto-negotiation machine).
Applicable only in SerDes mode; program as 0 for internal-PHY mode.
TxDesc Command-field
IDE
VLE
DEXT
reserved
=0
RS
IC
IFCS
EOP
7 6 5 4 3 2 1 0
EOP = End Of Packet (1=yes, 0=no)
IFCS = Insert Frame CheckSum (1=yes, 0=no) – provided EOP is set
IC = Insert CheckSum (1=yes, 0=no) as indicated by CSO/CSS fields
RS = Report Status (1=yes, 0=no)
DEXT = Descriptor Extension (1=yes, 0=no) use ‘0’ for Legacy-Mode
VLE = VLAN-Packet Enable (1=yes, 0=no) – provided EOP is set
IDE = Interrupt-Delay Enable (1=yes, 0=no)
TxDesc Status field
reserved
=0
LC
EC
DD
3 2 1 0
DD = Descriptor Done
this bit is written back after the NIC processes the descriptor
provided the descriptor’s RS-bit was set (i.e., Report Status)
EC = Excess Collisions
indicates that the packet has experienced more than the
maximum number of excessive collisions (as defined by
the TCTL.CT field) and therefore was not transmitted.
(This bit is meaningful only in HALF-DUPLEX mode.)
LC = Late Collision
indicates that Late Collision has occurred while operating in
HALF-DUPLEX mode. Note that the collision window size
is dependent on the SPEED: 64-bytes for 10/100-MBps, or
512-bytes for 1000-Mbps.
0
Device Status (0x0008)
?
0
0
0
0
0
0
0
0
0
0
0
GIO
Master
EN
0
0
0
0
0
0
0
PHY
reset
ASDV
I
L
O
S
S
L
U
0
TX
OFF
0 0
F
D
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Function
ID
L
U
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
SPEED
FD = Full-Duplex
LU = Link Up
TXOFF = Transmission Paused
SPEED (00=10Mbps,01=100Mbps, 10=1000Mbps, 11=reserved)
ASDV = Auto-negotiation Speed Detection Value
82573L
some undocumented functionality?
Device Control (0x0000)
PHY
RST
VME
R
=0
TFCE
RFCE
RST
R
=0
R
=0
R
=0
R
=0
R
=0
ADV
D3
WUC
R
=0
D/UD
status
R
=0
R
=0
R
=0
R
=0
R
=0
FRC
DPLX
FRC
SPD
R
=0
SPEED
R
=0
S
L
U
R
=0
R
=0
R
=1
0 0
F
D
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
GIO
M
D
R
=0
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
FD = Full-Duplex SPEED (00=10Mbps, 01=100Mbps, 10=1000Mbps, 11=reserved)
GIOMD = GIO Master Disable ADVD3WUP = Advertise Cold Wake Up Capability
SLU = Set Link Up D/UD = Dock/Undock status RFCE = Rx Flow-Control Enable
FRCSPD = Force Speed RST = Device Reset TFCE = Tx Flow-Control Enable
FRCDPLX = Force Duplex PHYRST = Phy Reset VME = VLAN Mode Enable
82573L
Ethernet packet layout
the packet’s data ‘payload’ goes here
(usually varies from 56 to 1500 bytes)
destination MAC address
(6-bytes)
source MAC address
(6-bytes)
Type/length
(2-bytes)
Cyclic Redundancy
Checksum (4-bytes)
0 6 12 14
How ‘receive’ works
descriptor0
descriptor1
descriptor2
descriptor3
0
0
0
0
Buffer0
Buffer1
Buffer2
Buffer3
List of Buffer-Descriptors
We setup memory-buffers where we want received packets to be placed by the NIC
We also create a list of buffer-descriptors and inform the NIC of its location and size
Then, when ready, we tell the NIC to ‘Go!’ (i.e., start receiving), but to let us know when these receptions have occurred
Random Access Memory
Receive Control (0x0100)
R
=0
0
0
FLXBUF
SE
CRC
BSEX
R
=0
PMCF
DPF
R
=0
CFI
CFI
EN
VFE
BSIZE
B
A
M
R
=0
MO
DTYP
RDMTS
I
L
O
S
S
L
U
LPE
UPE
0 0
R
=0
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
SBP
E
N
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
LBM
MPE
EN = Receive Enable DTYP = Descriptor Type DPF = Discard Pause Frames
SBP = Store Bad Packets MO = Multicast Offset PMCF = Pass MAC Control Frames
UPE = Unicast Promiscuous En BAM = Broadcast Accept Mode BSEX = Buffer Size Extension
MPE = Multicast Promiscuous En BSIZE = Receive Buffer Size SECRC = Strip Ethernet CRC
LPE = Long Packet reception Ena VFE = VLAN Filter Enable FLXBUF = Flexible Buffer size
LBM = Loopback Mode CFIEN = Canonical Form Indicator Enable
RDMTS = Rx-Descriptor Minimum Threshold Size CFI = Cannonical Form Indic
Registers’ Names
Rx-Desc Ring-Buffer
Circular buffer (128-bytes minimum)
RDBA
base-address
RDLEN
(in bytes)
RDH (head)
RDT (tail)
= owned by hardware (nic)
= owned by software (cpu)
0x00
0x10
0x20
0x30
0x40
0x50
0x60
0x70
0x80
Rx-Descriptor Control (0x2828)
R
=0
R
=0
R
=0
R
=0
R
=0
R
=0
R
=0
G
R
A
N
R
=0
R
=0
1
---------
0
ADV
D3
WUC
SDP1
DATA
---------
0
SDP0
DATA
---------
D/UD
status
0
WTHRESH
(Writeback Threshold)
R
=0
R
=0
0
FRC
DPLX
FRC
SPD
0
HTHRESH
(Host Threshold)
R
=0
R
=0
A
S
D
E
0
L
R
S
T
0 0
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
0
0
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
PTHRESH
(Prefetch Threshold)
GRAN (Granularity): 1=descriptor-size, 0=cacheline-size
Prefetch Threshold – A prefetch operation is considered when the number of valid, but unprocessed, receive descriptors that the ethernet controller has in its on-chip buffer drops below this threshold.
Host Threshold - A prefetch occurs if at least this many valid descriptors are available in host memory
Writeback Threshold - This field controls the writing back to host memory of already processed receive descriptors in the ethernet controller’s on-chip buffer which are ready to be written back to host memory
RxDesc Status-field
PIF
IPCS
TCPCS
VP
IXSM
EOP
DD
7 6 5 4 3 2 1 0
DD = Descriptor Done (1=yes, 0=no) shows if nic is finished with descriptor
EOP = End Of Packet (1=yes, 0=no) shows if this packet is logically last
IXSM = Ignore Checksum Indications (1=yes, 0=no)
VP = VLAN Packet match (1=yes, 0=no)
USPCS = UDP Checksum calculated in packet (1=yes, 0=no)
TCPCS = TCP Checksum calculated in packet (1=yes, 0=no)
IPCS = IPv4 Checksum calculated on packet (1=yes, 0=no)
PIF = Passed In-exact Filter (1=yes, 0=no) shows if software must check
UDPCS
RxDesc Error-field
RXE
IPE
TCPE
reserved
=0
SE
CE
7 6 5 4 3 2 1 0
RXE = Received-data Error (1=yes, 0=no)
IPE = IPv4-checksum error
TCPE = TCP/UDP checksum error (1=yes, 0=no)
SEQ = Sequence error (1=yes, 0=no)
SE = Symbol Error (1=yes, 0=no)
CE = CRC Error or alignment error (1=yes, 0=no)
SEQ
reserved
=0
Statistics registers
Initializing the nic’s counters
// Here ‘io’ is the virtual base-address
// of the nic’s i/o-memory region
{
int r;
// clear all of the Pro/1000 controller’s statistical counters
for (r = 0x4000; r < 0x4FFF; r += 4) ioread32( io + r );
}
A few ‘counter’ examples
0x4000 CRCERRS CRC Errors Count
0x400C RXERRC Receive Error Count
0x4014 SCC Single Collision Count
0x4018 ECOL Excessive Collision Count
0x4074 GPRC Good Packets Received
0x4078 BPRC Broadcast Packets Received
0x407C MPRC Multicast Packets Received
0x40D0 TPR Total Packets Received
0x40D4 TPT Total Packets Transmitted
0x40F0 MPTC Multicast Packets Transmitted
0x40F4 BPTC Broadcast Packets Transmitted
A ‘nic.c’ character driver?
open
read
write
my_fops
ioctl
my_ioctl()
my_open()
my_read()
my_write()
my_release()
my_isr()
module_init()
module_exit()
release
Network drivers
Network drivers
Snull: the network interface driver
Assigning IP number
The physical transport of Packets
Connecting to kernel: Device Registration
Connecting to kernel: Device initialization
Private data
Strcu snull_priv {
struct net_devices_stats stats;
int status;
strcut snull_packet *ppool;
struct snul_packet *rx_queue;
int rx_enabled, tc_packele;
u8 *tx_packetdata;
struct sk_bff *skb;
spinlock_t lock;
};
priv=netdriv_priv(dev);
memset(priv,0,sizeof(strcutn null_priv));
spin_lock_init(&priv->lock);
snull_rx_inits(dev,1); //enable revice interrupts
unregister_netdev(snull_dev[i]);
snull_teardown_pool(snull_dev[i]);
free_netdev(snull_dev[i]);
}
Connecting to kernel: Module unloading
Net_device Strcutures
Net_device Strcutures: Hardware info
Net_device: Interface information
Net_device: Device methods
Open() & Close ()
int snull_open(struct net_device *dev) {
/* request_region( ), request_irq( ), Assign the hardware address of the board: use "\0SNULx", where
* x is 0 or 1. The first byte is '\0' to avoid being a multicast
* address (the first byte of multicast addrs is odd). */
memcpy(dev->dev_addr, "\0SNUL0", ETH_ALEN);
if (dev = = snull_devs[1])
dev->dev_addr[ETH_ALEN-1]++; /* \0SNUL1 */
netif_start_queue(dev);
return 0;
}
int snull_release(struct net_device *dev) {
/* release ports, irq and such -- like fops->close */
netif_stop_queue(dev); /* can't transmit any more */
return 0;
}
Tx()
int snull_tx(struct sk_buff *skb, struct net_device *dev){
int len; char *data, shortpkt[ETH_ZLEN];
struct snull_priv *priv = netdev_priv(dev);
data = skb->data; len = skb->len;
if (len < ETH_ZLEN) {
memset(shortpkt, 0, ETH_ZLEN);
memcpy(shortpkt, skb->data, skb->len);
len = ETH_ZLEN;
data = shortpkt;
}
dev->trans_start = jiffies; /* save the timestamp */
/* Remember the skb, so we can free it at interrupt time */
priv->skb = skb;
/* actual deliver of data is device-specific, and not shown here */
snull_hw_tx(data, len, dev); return 0;
}
Rx()
void snull_rx(struct net_device *dev, struct snull_packet *pkt) {
struct sk_buff *skb; struct snull_priv *priv = netdev_priv(dev);
/* * The packet has been retrieved from the transmission * medium. Build an skb around it, so upper layers can handle it */
skb = dev_alloc_skb(pkt->datalen + 2);
if (!skb) {
if (printk_ratelimit( ))
printk(KERN_NOTICE "snull rx: low on mem - packet dropped\n");
priv->stats.rx_dropped++;
goto out;
}
memcpy(skb_put(skb, pkt->datalen), pkt->data, pkt->datalen);
/* Write metadata, and then pass to the receive level */
skb->dev = dev;
skb- >protocol = eth_type_trans(skb, dev);
skb->ip_summed = CHECKSUM_UNNECESSARY; /* don't check it */
priv->stats.rx_packets++;
priv->stats.rx_bytes += pkt->datalen;
netif_rx(skb);
out: return; }
Thanks�Ref: Chap 17, LDD 3e Rubini- Corbet