Network Interface Card �& its Linux programming

Dr A Sahu

Dept of Comp Sc & Engg.

IIT Guwahati

Outline

• PCI Devices
• NIC Cards
• NIC card architecture
• Access to NIC register
• PCI access

Major Player of NIC Cards

• RealTek (Example RTK 8169S)
• Broadcom (Example BCM5751)
• Intel (Example 82573L NIC)

​

• Download manual from vender site
• Or search “Realtek 8169 pdf” in google

​

​

Detecting PCI Devices : Early PCs

• Peripheral devices in the early PCs used fixed i/o-ports and fixed memory-addresses, e.g.:
• Video memory address-range: 0xA0000-0xBFFFF
• Programmable timer i/o-ports: 0x40-0x43
• Keyboard and mouse i/o-ports: 0x60-0x64
• Real-Time Clock’s i/o-ports: 0x70-0x71
• Hard Disk controller’s i/o-ports: 0x01F0-01F7
• Graphics controller’s i/o-ports: 0x03C0-0x3CF
• Serial-port controller’s i/o-ports: 0x03F8-0x03FF
• Parallel-port controller’s i/o-ports: 0x0378-0x037A

The PC’s evolution

• It became clear in the 1990s that there would be contention among equipment vendors for ‘fixed’ resource-addresses, which of course were in limited supply
• Among the goals that motivated the PCI Specification was the creation of a more flexible scheme for allocating addresses that future peripheral devices could use

PCI Configuration Space

PCI Configuration Space Body

(48 doublewords – variable format)

64

doublewords

PCI Configuration Space Header

(16 doublewords – fixed format)

A non-volatile parameter-storage area

for each PCI device-function

PCI Configuration Header

Status

Register

Command

Register

Device

ID

Vendor

ID

BIST

Cache

Line

Size

Class Code

Class/SubClass/ProgIF

Revision

ID

Base Address 0

Subsystem

Device ID

Subsystem

Vendor ID

CardBus CIS Pointer

reserved

capabilities

pointer

Expansion ROM Base Address

Minimum

Grant

Interrupt

Pin

reserved

Latency

Timer

Header

Type

Base Address 1

Base Address 2

Base Address 3

Base Address 4

Base Address 5

Interrupt

Line

Maximum

Latency

31 0

31 0

16 doublewords

Dwords

1 - 0

3 - 2

5 - 4

7 - 6

9 - 8

11 - 10

13 - 12

15 - 14

Three IA-32 address-spaces

​

memory

space

(4GB)

​

i/o space

(64KB)

PCI

configuration

space

(16MB)

accessed using a large variety of processor

instructions (mov, add, or, shr, push, etc.)

and virtual-to-physical address-translation

accessed only by using the processor’s

special ‘in’ and ‘out’ instructions

(without any translation of port-addresses)

i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space

Interface to PCI Configuration Space

reserved

CONFADD

( 0x0CF8)

CONFDAT

( 0x0CFC)

31 23 16 15 11 10 8 7 2 0

E

N

bus

(8-bits)

device

(5-bits)

doubleword

(6-bits)

function

(3-bits)

00

PCI Configuration Space Address Port (32-bits)

PCI Configuration Space Data Port (32-bits)

31 0

Enable Configuration Space Mapping (1=yes, 0=no)

Reading PCI Configuration Data

• Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port
• Step two: Read the designated data from the Configuration-Space Data-Port
• Already discussed PCI-probes pciprobes.c
• Lect 29..Showing vram, pciprobe.cpp

Example: network interface

• We can identify the network interface controller in PC’s by class-code 0x02
• The subclass-code 0x00 is for ‘ethernet’
• We can identify the NIC from its VENDOR and DEVICE identification-numbers:
• VENDOR_ID = 0x14E4
• DEVICE_ID = 0x1677
• You can use the ‘grep’ command to search for these numbers in this header-file:

</usr/src/linux/include/linux/pci_ids.h>

Vendor’s identity example

• The VENDOR-ID 0x14E4 belongs to the Broadcom Corporation
• Information about this firm may be learned from the corporation’s website
• The DEVICE-ID 0x1677 is used to signify Broadcom’s BCM5751 ethernet product

nic

Typical NIC

TX FIFO

RX FIFO

transceiver

LAN

cable

​

​

B

U

S

main

memory

packet

buffer

CPU

Packet filtering capability

• Network Interface’s hardware needs to implement ‘filtering’ of network packets
• Otherwise the PC’s memory-usage and processor-time will be wasted handling packets not meant for this PC to receive

​

network packet’s layout

Destination-address (6-bytes)

Source-address (6-bytes)

Each data-packet begins with the 6-byte device-address

of the network interface which is intended to receive it

Your NIC’s unique address

• You can see the Hardware Address of the ethernet controller on your PC by typing:

$ /sbin/ifconfig

• Look for it in the first line of screen-output that is labeled ‘eth0’, for example:

​

​

eth0 Link encap: Ethernet HWaddr 00:11:43:C9:50:3A

Demo “nic_pci_info.c”

• Lets write a kernel module that lets users see certain register-values which pertain to the network interface in your system :
• (1) the PCI Configuration Space registers
• (2) the Media Access Controller’s address
• It also shows your machine’s node-name (in case you want to save the information)

How we got the MAC-address

• We do not have NIC’s programming datasheet -- but we do have Linux source code for the ‘nic_pci_info.c’ device-driver, which includes a header-file ‘tg3.h’ found here:

</usr/src/linux/drivers/net/>

• If you scroll through the #define directives you will see the offset where the hardware address is stored in the memory-mapped register-space of the ‘nic_pci_info.c’ interface

How we get PCI_NIC registers

Status

Register

Command

Register

DeviceID

0x1677

VendorID

0x14E4

BIST

Cache

Line

Size

Class Code

Class/SubClass/ProgIF

Revision

ID

Base Address 0

Subsystem

Device ID

Subsystem

Vendor ID

CardBus CIS Pointer

reserved

capabilities

pointer

Expansion ROM Base Address

Minimum

Grant

Interrupt

Pin

reserved

Latency

Timer

Header

Type

Base Address 1

Base Address 2

Base Address 3

Base Address 4

Base Address 5

Interrupt

Line

Maximum

Latency

31 0

31 0

16 doublewords

Dwords

1 - 0

3 - 2

5 - 4

7 - 6

9 - 8

11 - 10

13 - 12

15 - 14

Linux helper-functions

#include <linux/pci.h>

struct pci_dev *devp;

unsigned int iomem_base, iomem_size;

void *io;

​

devp = pci_get_device( 0x14E4, 0x1677, NULL );

if ( !devp ) return –ENODEV;

​

iomem_base = pci_resource_start( devp, 0 );

iomem_size = pci_resource_len( devp, 0 );

io = ioremap( iomem_base, iomem_size );

if ( !io ) return -EBUSY;

​

Big-Endian to Little-Endian

mac

1

mac

0

mac

5

mac

4

mac

3

mac

2

0x0410 0x0411 0x0412 0x0413 0x0414 0x0415 0x0416 0x0417

Broadcom network interface storage-addresses

Intel IA-32 character-array storage

mac

0

mac

1

mac

2

mac

3

mac

4

mac

5

Little-Endian to Little-Endian

mac

0

mac

1

mac

2

mac

3

mac

4

mac

5

0x5400 0x5401 0x5402 0x5403 0x5404 0x5405 0x5406 0x5407

Intel network interface storage-addresses

Intel IA-32 character-array storage

mac

0

mac

1

mac

2

mac

3

mac

4

mac

5

For Intel NICs

• For Intel NICs :

#define VENDOR_ID 0x8086 // Intel Corp

#define DEVICE_ID 0x109A // 82573L NIC

• Intel’s filter-register at offset 0x5400 uses the ‘little endian’ storage-convention

Ethernet LAN

host-1

host-2

host-3

host-4

HUB

“Collision Domain”

CSMA/CD = “Carrier Sense Multiple Access/Collision Detection”

Acronyms

• PCI = Peripheral Component Interconnect
• MAC = Media Access Controller
• Phy = Physical-layer functions
• AMT = Active Management Technology
• LOM = LAN On Motherboard

Intel 82573L NIC: Hardware Features

• 32K configurable RX and TX packet FIFO
• IEEE 802.3x Flow Control support
• Host-Memory Receive Buffers 16K/256K
• IEEE 802.3ab Auto-Negotiation
• TCP/UDP checksum off-loading
• Jumbo-frame support (up to 16KB)
• Interrupt-moderation controls

External Architecture

PCI/PCI-e Bus

10/100/1000 PHY

MAC/Controller

MDI interface

SM Bus

interface

EEPROM

Flash

interface

LED

indicators

S/W Defined

pins

GMII/MII

interface

MDIO

interface

Access to PRO/1000 registers

• Device registers are hardware mapped to a range of addresses in physical memory
• You obtain the location (and the length) of this memory-range from a Base Add register in the nic device’s PCI Configuration Space
• Then you request the Linux kernel to setup an I/O ‘remapping’ of this memory-range to ‘virtual’ addresses within kernel-space

i/o-memory remapping

dynamic

ram

nic

registers

vram

IO-APIC

Local-APIC

user

space

APIC registers

kernel code/data

nic registers

vram

‘virtual’ address-space

physical address-space

1-GB

3-GB

portability syntax

• Linux provides device-driver writers with some macros for accessing i/o-memory:

#include <asm/io.h>

unsigned int datum;

iowrite32( datum, address );

datum = ioread32( address );

module_init()

#include <linux/pci.h>

#include <asm/io.h>

#define E1000_STATUS 0x0008

unsigned int iomem_base, iomem_size;

void *io;

​

// remap the device’s i/o-memory into kernel space

devp = pci_get_device( VENDOR_ID, DEVICE_ID, NULL );

if ( !devp ) return –ENODEV;

iomem_base = pci_resource_start( devp, 0 );

iomem_size = pci_resource_len( devp, 0 );

io = ioremap_nocache( iomem_base, iomem_size );

if ( !io ) return –ENOSPC;

​

// read and display the nic’s STATUS register

device_status = ioread32( io + E1000_STATUS );

printk( “ Device Status Register = 0x%08X \n”, status );

0

Device Status (0x0008)

?

0

0

0

0

0

0

0

0

0

0

0

GIO

Master

EN

0

0

0

0

0

0

0

PHY

reset

ASDV

I

L

O

S

S

L

U

0

TX

OFF

0 0

F

D

15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

Function

ID

L

U

31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16

SPEED

FD = Full-Duplex

LU = Link Up

TXOFF = Transmission Paused

SPEED (00=10Mbps,01=100Mbps, 10=1000Mbps, 11=reserved)

ASDV = Auto-negotiation Speed Detection Value

82573L

some undocumented functionality?

Thanks