strparser.rst 8.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240
  1. .. SPDX-License-Identifier: GPL-2.0
  2. =========================
  3. Stream Parser (strparser)
  4. =========================
  5. Introduction
  6. ============
  7. The stream parser (strparser) is a utility that parses messages of an
  8. application layer protocol running over a data stream. The stream
  9. parser works in conjunction with an upper layer in the kernel to provide
  10. kernel support for application layer messages. For instance, Kernel
  11. Connection Multiplexor (KCM) uses the Stream Parser to parse messages
  12. using a BPF program.
  13. The strparser works in one of two modes: receive callback or general
  14. mode.
  15. In receive callback mode, the strparser is called from the data_ready
  16. callback of a TCP socket. Messages are parsed and delivered as they are
  17. received on the socket.
  18. In general mode, a sequence of skbs are fed to strparser from an
  19. outside source. Message are parsed and delivered as the sequence is
  20. processed. This modes allows strparser to be applied to arbitrary
  21. streams of data.
  22. Interface
  23. =========
  24. The API includes a context structure, a set of callbacks, utility
  25. functions, and a data_ready function for receive callback mode. The
  26. callbacks include a parse_msg function that is called to perform
  27. parsing (e.g. BPF parsing in case of KCM), and a rcv_msg function
  28. that is called when a full message has been completed.
  29. Functions
  30. =========
  31. ::
  32. strp_init(struct strparser *strp, struct sock *sk,
  33. const struct strp_callbacks *cb)
  34. Called to initialize a stream parser. strp is a struct of type
  35. strparser that is allocated by the upper layer. sk is the TCP
  36. socket associated with the stream parser for use with receive
  37. callback mode; in general mode this is set to NULL. Callbacks
  38. are called by the stream parser (the callbacks are listed below).
  39. ::
  40. void strp_pause(struct strparser *strp)
  41. Temporarily pause a stream parser. Message parsing is suspended
  42. and no new messages are delivered to the upper layer.
  43. ::
  44. void strp_unpause(struct strparser *strp)
  45. Unpause a paused stream parser.
  46. ::
  47. void strp_stop(struct strparser *strp);
  48. strp_stop is called to completely stop stream parser operations.
  49. This is called internally when the stream parser encounters an
  50. error, and it is called from the upper layer to stop parsing
  51. operations.
  52. ::
  53. void strp_done(struct strparser *strp);
  54. strp_done is called to release any resources held by the stream
  55. parser instance. This must be called after the stream processor
  56. has been stopped.
  57. ::
  58. int strp_process(struct strparser *strp, struct sk_buff *orig_skb,
  59. unsigned int orig_offset, size_t orig_len,
  60. size_t max_msg_size, long timeo)
  61. strp_process is called in general mode for a stream parser to
  62. parse an sk_buff. The number of bytes processed or a negative
  63. error number is returned. Note that strp_process does not
  64. consume the sk_buff. max_msg_size is maximum size the stream
  65. parser will parse. timeo is timeout for completing a message.
  66. ::
  67. void strp_data_ready(struct strparser *strp);
  68. The upper layer calls strp_tcp_data_ready when data is ready on
  69. the lower socket for strparser to process. This should be called
  70. from a data_ready callback that is set on the socket. Note that
  71. maximum messages size is the limit of the receive socket
  72. buffer and message timeout is the receive timeout for the socket.
  73. ::
  74. void strp_check_rcv(struct strparser *strp);
  75. strp_check_rcv is called to check for new messages on the socket.
  76. This is normally called at initialization of a stream parser
  77. instance or after strp_unpause.
  78. Callbacks
  79. =========
  80. There are six callbacks:
  81. ::
  82. int (*parse_msg)(struct strparser *strp, struct sk_buff *skb);
  83. parse_msg is called to determine the length of the next message
  84. in the stream. The upper layer must implement this function. It
  85. should parse the sk_buff as containing the headers for the
  86. next application layer message in the stream.
  87. The skb->cb in the input skb is a struct strp_msg. Only
  88. the offset field is relevant in parse_msg and gives the offset
  89. where the message starts in the skb.
  90. The return values of this function are:
  91. ========= ===========================================================
  92. >0 indicates length of successfully parsed message
  93. 0 indicates more data must be received to parse the message
  94. -ESTRPIPE current message should not be processed by the
  95. kernel, return control of the socket to userspace which
  96. can proceed to read the messages itself
  97. other < 0 Error in parsing, give control back to userspace
  98. assuming that synchronization is lost and the stream
  99. is unrecoverable (application expected to close TCP socket)
  100. ========= ===========================================================
  101. In the case that an error is returned (return value is less than
  102. zero) and the parser is in receive callback mode, then it will set
  103. the error on TCP socket and wake it up. If parse_msg returned
  104. -ESTRPIPE and the stream parser had previously read some bytes for
  105. the current message, then the error set on the attached socket is
  106. ENODATA since the stream is unrecoverable in that case.
  107. ::
  108. void (*lock)(struct strparser *strp)
  109. The lock callback is called to lock the strp structure when
  110. the strparser is performing an asynchronous operation (such as
  111. processing a timeout). In receive callback mode the default
  112. function is to lock_sock for the associated socket. In general
  113. mode the callback must be set appropriately.
  114. ::
  115. void (*unlock)(struct strparser *strp)
  116. The unlock callback is called to release the lock obtained
  117. by the lock callback. In receive callback mode the default
  118. function is release_sock for the associated socket. In general
  119. mode the callback must be set appropriately.
  120. ::
  121. void (*rcv_msg)(struct strparser *strp, struct sk_buff *skb);
  122. rcv_msg is called when a full message has been received and
  123. is queued. The callee must consume the sk_buff; it can
  124. call strp_pause to prevent any further messages from being
  125. received in rcv_msg (see strp_pause above). This callback
  126. must be set.
  127. The skb->cb in the input skb is a struct strp_msg. This
  128. struct contains two fields: offset and full_len. Offset is
  129. where the message starts in the skb, and full_len is the
  130. the length of the message. skb->len - offset may be greater
  131. then full_len since strparser does not trim the skb.
  132. ::
  133. int (*read_sock_done)(struct strparser *strp, int err);
  134. read_sock_done is called when the stream parser is done reading
  135. the TCP socket in receive callback mode. The stream parser may
  136. read multiple messages in a loop and this function allows cleanup
  137. to occur when exiting the loop. If the callback is not set (NULL
  138. in strp_init) a default function is used.
  139. ::
  140. void (*abort_parser)(struct strparser *strp, int err);
  141. This function is called when stream parser encounters an error
  142. in parsing. The default function stops the stream parser and
  143. sets the error in the socket if the parser is in receive callback
  144. mode. The default function can be changed by setting the callback
  145. to non-NULL in strp_init.
  146. Statistics
  147. ==========
  148. Various counters are kept for each stream parser instance. These are in
  149. the strp_stats structure. strp_aggr_stats is a convenience structure for
  150. accumulating statistics for multiple stream parser instances.
  151. save_strp_stats and aggregate_strp_stats are helper functions to save
  152. and aggregate statistics.
  153. Message assembly limits
  154. =======================
  155. The stream parser provide mechanisms to limit the resources consumed by
  156. message assembly.
  157. A timer is set when assembly starts for a new message. In receive
  158. callback mode the message timeout is taken from rcvtime for the
  159. associated TCP socket. In general mode, the timeout is passed as an
  160. argument in strp_process. If the timer fires before assembly completes
  161. the stream parser is aborted and the ETIMEDOUT error is set on the TCP
  162. socket if in receive callback mode.
  163. In receive callback mode, message length is limited to the receive
  164. buffer size of the associated TCP socket. If the length returned by
  165. parse_msg is greater than the socket buffer size then the stream parser
  166. is aborted with EMSGSIZE error set on the TCP socket. Note that this
  167. makes the maximum size of receive skbuffs for a socket with a stream
  168. parser to be 2*sk_rcvbuf of the TCP socket.
  169. In general mode the message length limit is passed in as an argument
  170. to strp_process.
  171. Author
  172. ======
  173. Tom Herbert (tom@quantonium.net)