123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361 |
- ## @file
- #
- # Technical notes for the virtio-net driver.
- #
- # Copyright (C) 2013, Red Hat, Inc.
- #
- # SPDX-License-Identifier: BSD-2-Clause-Patent
- #
- ##
- Disclaimer
- ----------
- All statements concerning standards and specifications are informative and not
- normative. They are made in good faith. Corrections are most welcome on the
- edk2-devel mailing list.
- The following documents have been perused while writing the driver and this
- document:
- - Unified Extensible Firmware Interface Specification, Version 2.3.1, Errata C;
- June 27, 2012
- - Driver Writer's Guide for UEFI 2.3.1, 03/08/2012, Version 1.01;
- - Virtio PCI Card Specification, v0.9.5 DRAFT, 2012 May 7.
- Summary
- -------
- The VirtioNetDxe UEFI_DRIVER implements the Simple Network Protocol for
- virtio-net devices. Higher level protocols are automatically installed on top
- of it by the DXE Core / the ConnectController() boot service, enabling for
- virtio-net devices eg. DHCP configuration, TCP transfers with edk2 StdLib
- applications, and PXE booting in OVMF.
- UEFI driver structure
- ---------------------
- A driver instance, belonging to a given virtio-net device, can be in one of
- four states at any time. The states stack up as follows below. The state
- transitions are labeled with the primary function (and its important callees
- faithfully indented) that implement the transition.
- | ^
- | |
- [DriverBinding.c] | | [DriverBinding.c]
- VirtioNetDriverBindingStart | | VirtioNetDriverBindingStop
- VirtioNetSnpPopulate | | VirtioNetSnpEvacuate
- VirtioNetGetFeatures | |
- v |
- +-------------------------+
- | EfiSimpleNetworkStopped |
- +-------------------------+
- | ^
- [SnpStart.c] | | [SnpStop.c]
- VirtioNetStart | | VirtioNetStop
- | |
- v |
- +-------------------------+
- | EfiSimpleNetworkStarted |
- +-------------------------+
- | ^
- [SnpInitialize.c] | | [SnpShutdown.c]
- VirtioNetInitialize | | VirtioNetShutdown
- VirtioNetInitRing {Rx, Tx} | | VirtioNetShutdownRx [SnpSharedHelpers.c]
- VirtioRingInit | | VirtIo->UnmapSharedBuffer
- VirtioRingMap | | VirtIo->FreeSharedPages
- VirtioNetInitTx | | VirtioNetShutdownTx [SnpSharedHelpers.c]
- VirtIo->AllocateShare... | | VirtIo->UnmapSharedBuffer
- VirtioMapAllBytesInSh... | | VirtIo->FreeSharedPages
- VirtioNetInitRx | | VirtioNetUninitRing [SnpSharedHelpers.c]
- VirtIo->AllocateShare... | | {Tx, Rx}
- VirtioMapAllBytesInSh... | | VirtIo->UnmapSharedBuffer
- | | VirtioRingUninit
- v |
- +-----------------------------+
- | EfiSimpleNetworkInitialized |
- +-----------------------------+
- The state at the top means "nonexistent" and is hence unnamed on the diagram --
- a driver instance actually doesn't exist at that point. The transition
- functions out of and into that state implement the Driver Binding Protocol.
- The lower three states characterize an existent driver instance and are all
- states defined by the Simple Network Protocol. The transition functions between
- them are member functions of the Simple Network Protocol.
- Each transition function validates its expected source state and its
- parameters. For example, VirtioNetDriverBindingStop will refuse to disconnect
- from the controller unless it's in EfiSimpleNetworkStopped.
- Driver instance states (Simple Network Protocol)
- ------------------------------------------------
- In the EfiSimpleNetworkStopped state, the virtio-net device is (has been)
- re-set. No resources are allocated for networking / traffic purposes. The MAC
- address and other device attributes have been retrieved from the device (this
- is necessary for completing the VirtioNetDriverBindingStart transition).
- The EfiSimpleNetworkStarted is completely identical to the
- EfiSimpleNetworkStopped state for virtio-net, in the functional and
- resource-usage sense. This state is mandated / provided by the Simple Network
- Protocol for flexibility that the virtio-net driver doesn't exploit.
- In particular, the EfiSimpleNetworkStarted state is the target of the Shutdown
- SNP member function, and must therefore correspond to a hardware configuration
- where "[it] is safe for another driver to initialize". (Clearly another UEFI
- driver could not do that due to the exclusivity of the driver binding that
- VirtioNetDriverBindingStart() installs, but a later OS driver might qualify.)
- The EfiSimpleNetworkInitialized state is the live state of the virtio NIC / the
- driver instance. Virtio and other resources required for network traffic have
- been allocated, and the following SNP member functions are available (in
- addition to VirtioNetShutdown which leaves the state):
- - VirtioNetReceive [SnpReceive.c]: poll the virtio NIC for an Rx packet that
- may have arrived asynchronously;
- - VirtioNetTransmit [SnpTransmit.c]: queue a Tx packet for asynchronous
- transmission (meant to be used together with VirtioNetGetStatus);
- - VirtioNetGetStatus [SnpGetStatus.c]: query link status and status of pending
- Tx packets;
- - VirtioNetMcastIpToMac [SnpMcastIpToMac.c]: transform a multicast IPv4/IPv6
- address into a multicast MAC address;
- - VirtioNetReceiveFilters [SnpReceiveFilters.c]: emulate unicast / multicast /
- broadcast filter configuration (not their actual effect -- a more liberal
- filter setting than requested is allowed by the UEFI specification).
- The following SNP member functions are not supported [SnpUnsupported.c]:
- - VirtioNetReset: reinitialize the virtio NIC without shutting it down (a loop
- from/to EfiSimpleNetworkInitialized);
- - VirtioNetStationAddress: assign a new MAC address to the virtio NIC,
- - VirtioNetStatistics: collect statistics,
- - VirtioNetNvData: access non-volatile data on the virtio NIC.
- Missing support for these functions is allowed by the UEFI specification and
- doesn't seem to trip up higher level protocols.
- Events and task priority levels
- -------------------------------
- The UEFI specification defines a sophisticated mechanism for asynchronous
- events / callbacks (see "6.1 Event, Timer, and Task Priority Services" for
- details). Such callbacks work like software interrupts, and some notion of
- locking / masking is important to implement critical sections (atomic or
- exclusive access to data or a device). This notion is defined as Task Priority
- Levels.
- The virtio-net driver for OVMF must concern itself with events for two reasons:
- - The Simple Network Protocol provides its clients with a (non-optional) WAIT
- type event called WaitForPacket: it allows them to check or wait for Rx
- packets by polling or blocking on this event. (This functionality overlaps
- with the Receive member function.) The event is available to clients starting
- with EfiSimpleNetworkStopped (inclusive).
- The virtio-net driver is informed about such client polling or blockage by
- receiving an asynchronous callback (a software interrupt). In the callback
- function the driver must interrogate the driver instance state, and if it is
- EfiSimpleNetworkInitialized, access the Rx queue and see if any packets are
- available for consumption. If so, it must signal the WaitForPacket WAIT type
- event, waking the client.
- For simplicity and safety, all parts of the virtio-net driver that access any
- bit of the driver instance (data or device) run at the TPL_CALLBACK level.
- This is the highest level allowed for an SNP implementation, and all code
- protected in this manner satisfies even stricter non-blocking requirements
- than what's documented for TPL_CALLBACK.
- The task priority level for the WaitForPacket callback too is set by the
- driver, the choice is TPL_CALLBACK again. This in effect serializes the
- WaitForPacket callback (VirtioNetIsPacketAvailable [Events.c]) with "normal"
- parts of the driver.
- - According to the Driver Writer's Guide, a network driver should install a
- callback function for the global EXIT_BOOT_SERVICES event (a special NOTIFY
- type event). When the ExitBootServices() boot service has cleaned up internal
- firmware state and is about to pass control to the OS, any network driver has
- to stop any in-flight DMA transfers, lest it corrupts OS memory. For this
- reason EXIT_BOOT_SERVICES is emitted and the network driver must abort
- in-flight DMA transfers.
- This callback (VirtioNetExitBoot) is synchronized with the rest of the driver
- code just the same as explained for WaitForPacket. In
- EfiSimpleNetworkInitialized state it resets the virtio NIC, halting all data
- transfer. After the callback returns, no further driver code is expected to
- be scheduled.
- Virtio internals -- Rx
- ----------------------
- Requests (Rx and Tx alike) are always submitted by the guest and processed by
- the host. For Tx, processing means transmission. For Rx, processing means
- filling in the request with an incoming packet. Submitted requests exist on the
- "Available Ring", and answered (processed) requests show up on the "Used Ring".
- Packet data includes the media (Ethernet) header: destination MAC, source MAC,
- and Ethertype (14 bytes total).
- The following structures implement packet reception. Most of them are defined
- in the Virtio specification, the only driver-specific trait here is the static
- pre-configuration of the two-part descriptor chains, in VirtioNetInitRx. The
- diagram is simplified.
- Available Index Available Index
- last processed incremented
- by the host by the guest
- v -------> v
- Available +-------+-------+-------+-------+-------+
- Ring |DescIdx|DescIdx|DescIdx|DescIdx|DescIdx|
- +-------+-------+-------+-------+-------+
- =D6 =D2
- D2 D3 D4 D5 D6 D7
- Descr. +----------+----------++----------+----------++----------+----------+
- Table |Adr:Len:Nx|Adr:Len:Nx||Adr:Len:Nx|Adr:Len:Nx||Adr:Len:Nx|Adr:Len:Nx|
- +----------+----------++----------+----------++----------+----------+
- =A2 =D3 =A3 =A4 =D5 =A5 =A6 =D7 =A7
- A2 A3 A4 A5 A6 A7
- Receive +---------------+---------------+---------------+
- Destination |vnet hdr:packet|vnet hdr:packet|vnet hdr:packet|
- Area +---------------+---------------+---------------+
- Used Index Used Index incremented
- last processed by the guest by the host
- v -------> v
- Used +-----------+-----------+-----------+-----------+-----------+
- Ring |DescIdx:Len|DescIdx:Len|DescIdx:Len|DescIdx:Len|DescIdx:Len|
- +-----------+-----------+-----------+-----------+-----------+
- =D4
- In VirtioNetInitRx, the guest allocates the fixed size Receive Destination
- Area, which accommodates all packets delivered asynchronously by the host. To
- each packet, a slice of this area is dedicated; each slice is further
- subdivided into virtio-net request header and network packet data. The
- (device-physical) addresses of these sub-slices are denoted with A2, A3, A4 and
- so on. Importantly, an even-subscript "A" always belongs to a virtio-net
- request header, while an odd-subscript "A" always belongs to a packet
- sub-slice.
- Furthermore, the guest lays out a static pattern in the Descriptor Table. For
- each packet that can be in-flight or already arrived from the host,
- VirtioNetInitRx sets up a separate, two-part descriptor chain. For packet N,
- the Nth descriptor chain is set up as follows:
- - the first (=head) descriptor, with even index, points to the fixed-size
- sub-slice receiving the virtio-net request header,
- - the second descriptor (with odd index) points to the fixed (1514 byte) size
- sub-slice receiving the packet data,
- - a link from the first (head) descriptor in the chain is established to the
- second (tail) descriptor in the chain.
- Finally, the guest populates the Available Ring with the indices of the head
- descriptors. All descriptor indices on both the Available Ring and the Used
- Ring are even.
- Packet reception occurs as follows:
- - The host consumes a descriptor index off the Available Ring. This index is
- even (=2*N), and fingers the head descriptor of the chain belonging to packet
- N.
- - The host reads the descriptors D(2*N) and -- following the Next link there
- --- D(2*N+1), and stores the virtio-net request header at A(2*N), and the
- packet data at A(2*N+1).
- - The host places the index of the head descriptor, 2*N, onto the Used Ring,
- and sets the Len field in the same Used Ring Element to the total number of
- bytes transferred for the entire descriptor chain. This enables the guest to
- identify the length of Rx packets.
- - VirtioNetReceive polls the Used Ring. If a new Used Ring Element shows up, it
- copies the data out to the caller, and recycles the index of the head
- descriptor (ie. 2*N) to the Available Ring.
- - Because the host can process (answer) Rx requests in any order theoretically,
- the order of head descriptor indices on each of the Available Ring and the
- Used Ring is virtually random. (Except right after the initial population in
- VirtioNetInitRx, when the Available Ring is full and increasing, and the Used
- Ring is empty.)
- - If the Available Ring is empty, the host is forced to drop packets. If the
- Used Ring is empty, VirtioNetReceive returns EFI_NOT_READY (no packet
- available).
- Virtio internals -- Tx
- ----------------------
- The transmission structure erected by VirtioNetInitTx is similar, it differs
- in the following:
- - There is no Receive Destination Area.
- - Each head descriptor, D(2*N), points to a read-only virtio-net request header
- that is shared by all of the head descriptors. This virtio-net request header
- is never modified by the host.
- - Each tail descriptor is re-pointed to the device-mapped address of the
- caller-supplied packet buffer whenever VirtioNetTransmit places the
- corresponding head descriptor on the Available Ring. A reverse mapping, from
- the device-mapped address to the caller-supplied packet address, is saved in
- an associative data structure that belongs to the driver instance.
- - Per spec, the caller is responsible to hang on to the unmodified packet
- buffer until it is reported transmitted by VirtioNetGetStatus.
- Steps of packet transmission:
- - Client code calls VirtioNetTransmit. VirtioNetTransmit tracks free descriptor
- chains by keeping the indices of their head descriptors in a stack that is
- private to the driver instance. All elements of the stack are even.
- - If the stack is empty (that is, each descriptor chain, in isolation, is
- either pending transmission, or has been processed by the host but not
- yet recycled by a VirtioNetGetStatus call), then VirtioNetTransmit returns
- EFI_NOT_READY.
- - Otherwise the index of a free chain's head descriptor is popped from the
- stack. The linked tail descriptor is re-pointed as discussed above. The head
- descriptor's index is pushed on the Available Ring.
- - The host moves the head descriptor index from the Available Ring to the Used
- Ring when it transmits the packet.
- - Client code calls VirtioNetGetStatus. In case the Used Ring is empty, the
- function reports no Tx completion. Otherwise, a head descriptor's index is
- consumed from the Used Ring and recycled to the private stack. The client
- code's original packet buffer address is calculated by fetching the
- device-mapped address from the tail descriptor (where it has been stored at
- VirtioNetTransmit time), and by looking up the device-mapped address in the
- associative data structure. The reverse-mapped packet buffer address is
- returned to the caller.
- - The Len field of the Used Ring Element is not checked. The host is assumed to
- have transmitted the entire packet -- VirtioNetTransmit had forced it below
- 1514 bytes (inclusive). The Virtio specification suggests this packet size is
- always accepted (and a lower MTU could be encountered on any later hop as
- well). Additionally, there's no good way to report a short transmit via
- VirtioNetGetStatus; EFI_DEVICE_ERROR seems too serious from the specification
- and higher level protocols could interpret it as a fatal condition.
- - The host can theoretically reorder head descriptor indices when moving them
- from the Available Ring to the Used Ring (out of order transmission). Because
- of this (and the choice of a stack over a list for free descriptor chain
- tracking) the order of head descriptor indices on either Ring is
- unpredictable.
|