rcuref.rst 5.2 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158
  1. .. SPDX-License-Identifier: GPL-2.0
  2. ====================================================================
  3. Reference-count design for elements of lists/arrays protected by RCU
  4. ====================================================================
  5. Please note that the percpu-ref feature is likely your first
  6. stop if you need to combine reference counts and RCU. Please see
  7. include/linux/percpu-refcount.h for more information. However, in
  8. those unusual cases where percpu-ref would consume too much memory,
  9. please read on.
  10. ------------------------------------------------------------------------
  11. Reference counting on elements of lists which are protected by traditional
  12. reader/writer spinlocks or semaphores are straightforward:
  13. CODE LISTING A::
  14. 1. 2.
  15. add() search_and_reference()
  16. { {
  17. alloc_object read_lock(&list_lock);
  18. ... search_for_element
  19. atomic_set(&el->rc, 1); atomic_inc(&el->rc);
  20. write_lock(&list_lock); ...
  21. add_element read_unlock(&list_lock);
  22. ... ...
  23. write_unlock(&list_lock); }
  24. }
  25. 3. 4.
  26. release_referenced() delete()
  27. { {
  28. ... write_lock(&list_lock);
  29. if(atomic_dec_and_test(&el->rc)) ...
  30. kfree(el);
  31. ... remove_element
  32. } write_unlock(&list_lock);
  33. ...
  34. if (atomic_dec_and_test(&el->rc))
  35. kfree(el);
  36. ...
  37. }
  38. If this list/array is made lock free using RCU as in changing the
  39. write_lock() in add() and delete() to spin_lock() and changing read_lock()
  40. in search_and_reference() to rcu_read_lock(), the atomic_inc() in
  41. search_and_reference() could potentially hold reference to an element which
  42. has already been deleted from the list/array. Use atomic_inc_not_zero()
  43. in this scenario as follows:
  44. CODE LISTING B::
  45. 1. 2.
  46. add() search_and_reference()
  47. { {
  48. alloc_object rcu_read_lock();
  49. ... search_for_element
  50. atomic_set(&el->rc, 1); if (!atomic_inc_not_zero(&el->rc)) {
  51. spin_lock(&list_lock); rcu_read_unlock();
  52. return FAIL;
  53. add_element }
  54. ... ...
  55. spin_unlock(&list_lock); rcu_read_unlock();
  56. } }
  57. 3. 4.
  58. release_referenced() delete()
  59. { {
  60. ... spin_lock(&list_lock);
  61. if (atomic_dec_and_test(&el->rc)) ...
  62. call_rcu(&el->head, el_free); remove_element
  63. ... spin_unlock(&list_lock);
  64. } ...
  65. if (atomic_dec_and_test(&el->rc))
  66. call_rcu(&el->head, el_free);
  67. ...
  68. }
  69. Sometimes, a reference to the element needs to be obtained in the
  70. update (write) stream. In such cases, atomic_inc_not_zero() might be
  71. overkill, since we hold the update-side spinlock. One might instead
  72. use atomic_inc() in such cases.
  73. It is not always convenient to deal with "FAIL" in the
  74. search_and_reference() code path. In such cases, the
  75. atomic_dec_and_test() may be moved from delete() to el_free()
  76. as follows:
  77. CODE LISTING C::
  78. 1. 2.
  79. add() search_and_reference()
  80. { {
  81. alloc_object rcu_read_lock();
  82. ... search_for_element
  83. atomic_set(&el->rc, 1); atomic_inc(&el->rc);
  84. spin_lock(&list_lock); ...
  85. add_element rcu_read_unlock();
  86. ... }
  87. spin_unlock(&list_lock); 4.
  88. } delete()
  89. 3. {
  90. release_referenced() spin_lock(&list_lock);
  91. { ...
  92. ... remove_element
  93. if (atomic_dec_and_test(&el->rc)) spin_unlock(&list_lock);
  94. kfree(el); ...
  95. ... call_rcu(&el->head, el_free);
  96. } ...
  97. 5. }
  98. void el_free(struct rcu_head *rhp)
  99. {
  100. release_referenced();
  101. }
  102. The key point is that the initial reference added by add() is not removed
  103. until after a grace period has elapsed following removal. This means that
  104. search_and_reference() cannot find this element, which means that the value
  105. of el->rc cannot increase. Thus, once it reaches zero, there are no
  106. readers that can or ever will be able to reference the element. The
  107. element can therefore safely be freed. This in turn guarantees that if
  108. any reader finds the element, that reader may safely acquire a reference
  109. without checking the value of the reference counter.
  110. A clear advantage of the RCU-based pattern in listing C over the one
  111. in listing B is that any call to search_and_reference() that locates
  112. a given object will succeed in obtaining a reference to that object,
  113. even given a concurrent invocation of delete() for that same object.
  114. Similarly, a clear advantage of both listings B and C over listing A is
  115. that a call to delete() is not delayed even if there are an arbitrarily
  116. large number of calls to search_and_reference() searching for the same
  117. object that delete() was invoked on. Instead, all that is delayed is
  118. the eventual invocation of kfree(), which is usually not a problem on
  119. modern computer systems, even the small ones.
  120. In cases where delete() can sleep, synchronize_rcu() can be called from
  121. delete(), so that el_free() can be subsumed into delete as follows::
  122. 4.
  123. delete()
  124. {
  125. spin_lock(&list_lock);
  126. ...
  127. remove_element
  128. spin_unlock(&list_lock);
  129. ...
  130. synchronize_rcu();
  131. if (atomic_dec_and_test(&el->rc))
  132. kfree(el);
  133. ...
  134. }
  135. As additional examples in the kernel, the pattern in listing C is used by
  136. reference counting of struct pid, while the pattern in listing B is used by
  137. struct posix_acl.