swap_numa.rst 3.0 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980
  1. .. _swap_numa:
  2. ===========================================
  3. Automatically bind swap device to numa node
  4. ===========================================
  5. If the system has more than one swap device and swap device has the node
  6. information, we can make use of this information to decide which swap
  7. device to use in get_swap_pages() to get better performance.
  8. How to use this feature
  9. =======================
  10. Swap device has priority and that decides the order of it to be used. To make
  11. use of automatically binding, there is no need to manipulate priority settings
  12. for swap devices. e.g. on a 2 node machine, assume 2 swap devices swapA and
  13. swapB, with swapA attached to node 0 and swapB attached to node 1, are going
  14. to be swapped on. Simply swapping them on by doing::
  15. # swapon /dev/swapA
  16. # swapon /dev/swapB
  17. Then node 0 will use the two swap devices in the order of swapA then swapB and
  18. node 1 will use the two swap devices in the order of swapB then swapA. Note
  19. that the order of them being swapped on doesn't matter.
  20. A more complex example on a 4 node machine. Assume 6 swap devices are going to
  21. be swapped on: swapA and swapB are attached to node 0, swapC is attached to
  22. node 1, swapD and swapE are attached to node 2 and swapF is attached to node3.
  23. The way to swap them on is the same as above::
  24. # swapon /dev/swapA
  25. # swapon /dev/swapB
  26. # swapon /dev/swapC
  27. # swapon /dev/swapD
  28. # swapon /dev/swapE
  29. # swapon /dev/swapF
  30. Then node 0 will use them in the order of::
  31. swapA/swapB -> swapC -> swapD -> swapE -> swapF
  32. swapA and swapB will be used in a round robin mode before any other swap device.
  33. node 1 will use them in the order of::
  34. swapC -> swapA -> swapB -> swapD -> swapE -> swapF
  35. node 2 will use them in the order of::
  36. swapD/swapE -> swapA -> swapB -> swapC -> swapF
  37. Similaly, swapD and swapE will be used in a round robin mode before any
  38. other swap devices.
  39. node 3 will use them in the order of::
  40. swapF -> swapA -> swapB -> swapC -> swapD -> swapE
  41. Implementation details
  42. ======================
  43. The current code uses a priority based list, swap_avail_list, to decide
  44. which swap device to use and if multiple swap devices share the same
  45. priority, they are used round robin. This change here replaces the single
  46. global swap_avail_list with a per-numa-node list, i.e. for each numa node,
  47. it sees its own priority based list of available swap devices. Swap
  48. device's priority can be promoted on its matching node's swap_avail_list.
  49. The current swap device's priority is set as: user can set a >=0 value,
  50. or the system will pick one starting from -1 then downwards. The priority
  51. value in the swap_avail_list is the negated value of the swap device's
  52. due to plist being sorted from low to high. The new policy doesn't change
  53. the semantics for priority >=0 cases, the previous starting from -1 then
  54. downwards now becomes starting from -2 then downwards and -1 is reserved
  55. as the promoted value. So if multiple swap devices are attached to the same
  56. node, they will all be promoted to priority -1 on that node's plist and will
  57. be used round robin before any other swap devices.