ixgb.txt 8.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212
  1. Linux* Base Driver for the Intel(R) PRO/10GbE Family of Adapters
  2. ================================================================
  3. November 17, 2004
  4. Contents
  5. ========
  6. - In This Release
  7. - Identifying Your Adapter
  8. - Command Line Parameters
  9. - Improving Performance
  10. - Support
  11. In This Release
  12. ===============
  13. This file describes the Linux* Base Driver for the Intel(R) PRO/10GbE Family
  14. of Adapters, version 1.0.x.
  15. For questions related to hardware requirements, refer to the documentation
  16. supplied with your Intel PRO/10GbE adapter. All hardware requirements listed
  17. apply to use with Linux.
  18. Identifying Your Adapter
  19. ========================
  20. To verify your Intel adapter is supported, find the board ID number on the
  21. adapter. Look for a label that has a barcode and a number in the format
  22. A12345-001.
  23. Use the above information and the Adapter & Driver ID Guide at:
  24. http://support.intel.com/support/network/adapter/pro100/21397.htm
  25. For the latest Intel network drivers for Linux, go to:
  26. http://downloadfinder.intel.com/scripts-df/support_intel.asp
  27. Command Line Parameters
  28. =======================
  29. If the driver is built as a module, the following optional parameters are
  30. used by entering them on the command line with the modprobe or insmod command
  31. using this syntax:
  32. modprobe ixgb [<option>=<VAL1>,<VAL2>,...]
  33. insmod ixgb [<option>=<VAL1>,<VAL2>,...]
  34. For example, with two PRO/10GbE PCI adapters, entering:
  35. insmod ixgb TxDescriptors=80,128
  36. loads the ixgb driver with 80 TX resources for the first adapter and 128 TX
  37. resources for the second adapter.
  38. The default value for each parameter is generally the recommended setting,
  39. unless otherwise noted. Also, if the driver is statically built into the
  40. kernel, the driver is loaded with the default values for all the parameters.
  41. Ethtool can be used to change some of the parameters at runtime.
  42. FlowControl
  43. Valid Range: 0-3 (0=none, 1=Rx only, 2=Tx only, 3=Rx&Tx)
  44. Default: Read from the EEPROM
  45. If EEPROM is not detected, default is 3
  46. This parameter controls the automatic generation(Tx) and response(Rx) to
  47. Ethernet PAUSE frames.
  48. RxDescriptors
  49. Valid Range: 64-512
  50. Default Value: 512
  51. This value is the number of receive descriptors allocated by the driver.
  52. Increasing this value allows the driver to buffer more incoming packets.
  53. Each descriptor is 16 bytes. A receive buffer is also allocated for
  54. each descriptor and can be either 2048, 4056, 8192, or 16384 bytes,
  55. depending on the MTU setting. When the MTU size is 1500 or less, the
  56. receive buffer size is 2048 bytes. When the MTU is greater than 1500 the
  57. receive buffer size will be either 4056, 8192, or 16384 bytes. The
  58. maximum MTU size is 16114.
  59. RxIntDelay
  60. Valid Range: 0-65535 (0=off)
  61. Default Value: 6
  62. This value delays the generation of receive interrupts in units of
  63. 0.8192 microseconds. Receive interrupt reduction can improve CPU
  64. efficiency if properly tuned for specific network traffic. Increasing
  65. this value adds extra latency to frame reception and can end up
  66. decreasing the throughput of TCP traffic. If the system is reporting
  67. dropped receives, this value may be set too high, causing the driver to
  68. run out of available receive descriptors.
  69. TxDescriptors
  70. Valid Range: 64-4096
  71. Default Value: 256
  72. This value is the number of transmit descriptors allocated by the driver.
  73. Increasing this value allows the driver to queue more transmits. Each
  74. descriptor is 16 bytes.
  75. XsumRX
  76. Valid Range: 0-1
  77. Default Value: 1
  78. A value of '1' indicates that the driver should enable IP checksum
  79. offload for received packets (both UDP and TCP) to the adapter hardware.
  80. XsumTX
  81. Valid Range: 0-1
  82. Default Value: 1
  83. A value of '1' indicates that the driver should enable IP checksum
  84. offload for transmitted packets (both UDP and TCP) to the adapter
  85. hardware.
  86. Improving Performance
  87. =====================
  88. With the Intel PRO/10 GbE adapter, the default Linux configuration will very
  89. likely limit the total available throughput artificially. There is a set of
  90. things that when applied together increase the ability of Linux to transmit
  91. and receive data. The following enhancements were originally acquired from
  92. settings published at http://www.spec.org/web99 for various submitted results
  93. using Linux.
  94. NOTE: These changes are only suggestions, and serve as a starting point for
  95. tuning your network performance.
  96. The changes are made in three major ways, listed in order of greatest effect:
  97. - Use ifconfig to modify the mtu (maximum transmission unit) and the txqueuelen
  98. parameter.
  99. - Use sysctl to modify /proc parameters (essentially kernel tuning)
  100. - Use setpci to modify the MMRBC field in PCI-X configuration space to increase
  101. transmit burst lengths on the bus.
  102. NOTE: setpci modifies the adapter's configuration registers to allow it to read
  103. up to 4k bytes at a time (for transmits). However, for some systems the
  104. behavior after modifying this register may be undefined (possibly errors of some
  105. kind). A power-cycle, hard reset or explicitly setting the e6 register back to
  106. 22 (setpci -d 8086:1048 e6.b=22) may be required to get back to a stable
  107. configuration.
  108. - COPY these lines and paste them into ixgb_perf.sh:
  109. #!/bin/bash
  110. echo "configuring network performance , edit this file to change the interface"
  111. # set mmrbc to 4k reads, modify only Intel 10GbE device IDs
  112. setpci -d 8086:1048 e6.b=2e
  113. # set the MTU (max transmission unit) - it requires your switch and clients to change too!
  114. # set the txqueuelen
  115. # your ixgb adapter should be loaded as eth1 for this to work, change if needed
  116. ifconfig eth1 mtu 9000 txqueuelen 1000 up
  117. # call the sysctl utility to modify /proc/sys entries
  118. sysctl -p ./sysctl_ixgb.conf
  119. - END ixgb_perf.sh
  120. - COPY these lines and paste them into sysctl_ixgb.conf:
  121. # some of the defaults may be different for your kernel
  122. # call this file with sysctl -p <this file>
  123. # these are just suggested values that worked well to increase throughput in
  124. # several network benchmark tests, your mileage may vary
  125. ### IPV4 specific settings
  126. net.ipv4.tcp_timestamps = 0 # turns TCP timestamp support off, default 1, reduces CPU use
  127. net.ipv4.tcp_sack = 0 # turn SACK support off, default on
  128. # on systems with a VERY fast bus -> memory interface this is the big gainer
  129. net.ipv4.tcp_rmem = 10000000 10000000 10000000 # sets min/default/max TCP read buffer, default 4096 87380 174760
  130. net.ipv4.tcp_wmem = 10000000 10000000 10000000 # sets min/pressure/max TCP write buffer, default 4096 16384 131072
  131. net.ipv4.tcp_mem = 10000000 10000000 10000000 # sets min/pressure/max TCP buffer space, default 31744 32256 32768
  132. ### CORE settings (mostly for socket and UDP effect)
  133. net.core.rmem_max = 524287 # maximum receive socket buffer size, default 131071
  134. net.core.wmem_max = 524287 # maximum send socket buffer size, default 131071
  135. net.core.rmem_default = 524287 # default receive socket buffer size, default 65535
  136. net.core.wmem_default = 524287 # default send socket buffer size, default 65535
  137. net.core.optmem_max = 524287 # maximum amount of option memory buffers, default 10240
  138. net.core.netdev_max_backlog = 300000 # number of unprocessed input packets before kernel starts dropping them, default 300
  139. - END sysctl_ixgb.conf
  140. Edit the ixgb_perf.sh script if necessary to change eth1 to whatever interface
  141. your ixgb driver is using.
  142. NOTE: Unless these scripts are added to the boot process, these changes will
  143. only last only until the next system reboot.
  144. Resolving Slow UDP Traffic
  145. --------------------------
  146. If your server does not seem to be able to receive UDP traffic as fast as it
  147. can receive TCP traffic, it could be because Linux, by default, does not set
  148. the network stack buffers as large as they need to be to support high UDP
  149. transfer rates. One way to alleviate this problem is to allow more memory to
  150. be used by the IP stack to store incoming data.
  151. For instance, use the commands:
  152. sysctl -w net.core.rmem_max=262143
  153. and
  154. sysctl -w net.core.rmem_default=262143
  155. to increase the read buffer memory max and default to 262143 (256k - 1) from
  156. defaults of max=131071 (128k - 1) and default=65535 (64k - 1). These variables
  157. will increase the amount of memory used by the network stack for receives, and
  158. can be increased significantly more if necessary for your application.
  159. Support
  160. =======
  161. For general information and support, go to the Intel support website at:
  162. http://support.intel.com
  163. If an issue is identified with the released source code on the supported
  164. kernel with a supported adapter, email the specific information related to
  165. the issue to linux.nics@intel.com.