|
@@ -0,0 +1,491 @@
|
|
|
+.. Copyright 2010 Nicolas Palix <npalix@diku.dk>
|
|
|
+.. Copyright 2010 Julia Lawall <julia@diku.dk>
|
|
|
+.. Copyright 2010 Gilles Muller <Gilles.Muller@lip6.fr>
|
|
|
+
|
|
|
+.. highlight:: none
|
|
|
+
|
|
|
+Coccinelle
|
|
|
+==========
|
|
|
+
|
|
|
+Coccinelle is a tool for pattern matching and text transformation that has
|
|
|
+many uses in kernel development, including the application of complex,
|
|
|
+tree-wide patches and detection of problematic programming patterns.
|
|
|
+
|
|
|
+Getting Coccinelle
|
|
|
+-------------------
|
|
|
+
|
|
|
+The semantic patches included in the kernel use features and options
|
|
|
+which are provided by Coccinelle version 1.0.0-rc11 and above.
|
|
|
+Using earlier versions will fail as the option names used by
|
|
|
+the Coccinelle files and coccicheck have been updated.
|
|
|
+
|
|
|
+Coccinelle is available through the package manager
|
|
|
+of many distributions, e.g. :
|
|
|
+
|
|
|
+ - Debian
|
|
|
+ - Fedora
|
|
|
+ - Ubuntu
|
|
|
+ - OpenSUSE
|
|
|
+ - Arch Linux
|
|
|
+ - NetBSD
|
|
|
+ - FreeBSD
|
|
|
+
|
|
|
+You can get the latest version released from the Coccinelle homepage at
|
|
|
+http://coccinelle.lip6.fr/
|
|
|
+
|
|
|
+Information and tips about Coccinelle are also provided on the wiki
|
|
|
+pages at http://cocci.ekstranet.diku.dk/wiki/doku.php
|
|
|
+
|
|
|
+Once you have it, run the following command::
|
|
|
+
|
|
|
+ ./configure
|
|
|
+ make
|
|
|
+
|
|
|
+as a regular user, and install it with::
|
|
|
+
|
|
|
+ sudo make install
|
|
|
+
|
|
|
+Supplemental documentation
|
|
|
+---------------------------
|
|
|
+
|
|
|
+For supplemental documentation refer to the wiki:
|
|
|
+
|
|
|
+https://bottest.wiki.kernel.org/coccicheck
|
|
|
+
|
|
|
+The wiki documentation always refers to the linux-next version of the script.
|
|
|
+
|
|
|
+Using Coccinelle on the Linux kernel
|
|
|
+------------------------------------
|
|
|
+
|
|
|
+A Coccinelle-specific target is defined in the top level
|
|
|
+Makefile. This target is named ``coccicheck`` and calls the ``coccicheck``
|
|
|
+front-end in the ``scripts`` directory.
|
|
|
+
|
|
|
+Four basic modes are defined: ``patch``, ``report``, ``context``, and
|
|
|
+``org``. The mode to use is specified by setting the MODE variable with
|
|
|
+``MODE=<mode>``.
|
|
|
+
|
|
|
+- ``patch`` proposes a fix, when possible.
|
|
|
+
|
|
|
+- ``report`` generates a list in the following format:
|
|
|
+ file:line:column-column: message
|
|
|
+
|
|
|
+- ``context`` highlights lines of interest and their context in a
|
|
|
+ diff-like style.Lines of interest are indicated with ``-``.
|
|
|
+
|
|
|
+- ``org`` generates a report in the Org mode format of Emacs.
|
|
|
+
|
|
|
+Note that not all semantic patches implement all modes. For easy use
|
|
|
+of Coccinelle, the default mode is "report".
|
|
|
+
|
|
|
+Two other modes provide some common combinations of these modes.
|
|
|
+
|
|
|
+- ``chain`` tries the previous modes in the order above until one succeeds.
|
|
|
+
|
|
|
+- ``rep+ctxt`` runs successively the report mode and the context mode.
|
|
|
+ It should be used with the C option (described later)
|
|
|
+ which checks the code on a file basis.
|
|
|
+
|
|
|
+Examples
|
|
|
+~~~~~~~~
|
|
|
+
|
|
|
+To make a report for every semantic patch, run the following command::
|
|
|
+
|
|
|
+ make coccicheck MODE=report
|
|
|
+
|
|
|
+To produce patches, run::
|
|
|
+
|
|
|
+ make coccicheck MODE=patch
|
|
|
+
|
|
|
+
|
|
|
+The coccicheck target applies every semantic patch available in the
|
|
|
+sub-directories of ``scripts/coccinelle`` to the entire Linux kernel.
|
|
|
+
|
|
|
+For each semantic patch, a commit message is proposed. It gives a
|
|
|
+description of the problem being checked by the semantic patch, and
|
|
|
+includes a reference to Coccinelle.
|
|
|
+
|
|
|
+As any static code analyzer, Coccinelle produces false
|
|
|
+positives. Thus, reports must be carefully checked, and patches
|
|
|
+reviewed.
|
|
|
+
|
|
|
+To enable verbose messages set the V= variable, for example::
|
|
|
+
|
|
|
+ make coccicheck MODE=report V=1
|
|
|
+
|
|
|
+Coccinelle parallelization
|
|
|
+---------------------------
|
|
|
+
|
|
|
+By default, coccicheck tries to run as parallel as possible. To change
|
|
|
+the parallelism, set the J= variable. For example, to run across 4 CPUs::
|
|
|
+
|
|
|
+ make coccicheck MODE=report J=4
|
|
|
+
|
|
|
+As of Coccinelle 1.0.2 Coccinelle uses Ocaml parmap for parallelization,
|
|
|
+if support for this is detected you will benefit from parmap parallelization.
|
|
|
+
|
|
|
+When parmap is enabled coccicheck will enable dynamic load balancing by using
|
|
|
+``--chunksize 1`` argument, this ensures we keep feeding threads with work
|
|
|
+one by one, so that we avoid the situation where most work gets done by only
|
|
|
+a few threads. With dynamic load balancing, if a thread finishes early we keep
|
|
|
+feeding it more work.
|
|
|
+
|
|
|
+When parmap is enabled, if an error occurs in Coccinelle, this error
|
|
|
+value is propagated back, the return value of the ``make coccicheck``
|
|
|
+captures this return value.
|
|
|
+
|
|
|
+Using Coccinelle with a single semantic patch
|
|
|
+---------------------------------------------
|
|
|
+
|
|
|
+The optional make variable COCCI can be used to check a single
|
|
|
+semantic patch. In that case, the variable must be initialized with
|
|
|
+the name of the semantic patch to apply.
|
|
|
+
|
|
|
+For instance::
|
|
|
+
|
|
|
+ make coccicheck COCCI=<my_SP.cocci> MODE=patch
|
|
|
+
|
|
|
+or::
|
|
|
+
|
|
|
+ make coccicheck COCCI=<my_SP.cocci> MODE=report
|
|
|
+
|
|
|
+
|
|
|
+Controlling Which Files are Processed by Coccinelle
|
|
|
+---------------------------------------------------
|
|
|
+
|
|
|
+By default the entire kernel source tree is checked.
|
|
|
+
|
|
|
+To apply Coccinelle to a specific directory, ``M=`` can be used.
|
|
|
+For example, to check drivers/net/wireless/ one may write::
|
|
|
+
|
|
|
+ make coccicheck M=drivers/net/wireless/
|
|
|
+
|
|
|
+To apply Coccinelle on a file basis, instead of a directory basis, the
|
|
|
+following command may be used::
|
|
|
+
|
|
|
+ make C=1 CHECK="scripts/coccicheck"
|
|
|
+
|
|
|
+To check only newly edited code, use the value 2 for the C flag, i.e.::
|
|
|
+
|
|
|
+ make C=2 CHECK="scripts/coccicheck"
|
|
|
+
|
|
|
+In these modes, which works on a file basis, there is no information
|
|
|
+about semantic patches displayed, and no commit message proposed.
|
|
|
+
|
|
|
+This runs every semantic patch in scripts/coccinelle by default. The
|
|
|
+COCCI variable may additionally be used to only apply a single
|
|
|
+semantic patch as shown in the previous section.
|
|
|
+
|
|
|
+The "report" mode is the default. You can select another one with the
|
|
|
+MODE variable explained above.
|
|
|
+
|
|
|
+Debugging Coccinelle SmPL patches
|
|
|
+---------------------------------
|
|
|
+
|
|
|
+Using coccicheck is best as it provides in the spatch command line
|
|
|
+include options matching the options used when we compile the kernel.
|
|
|
+You can learn what these options are by using V=1, you could then
|
|
|
+manually run Coccinelle with debug options added.
|
|
|
+
|
|
|
+Alternatively you can debug running Coccinelle against SmPL patches
|
|
|
+by asking for stderr to be redirected to stderr, by default stderr
|
|
|
+is redirected to /dev/null, if you'd like to capture stderr you
|
|
|
+can specify the ``DEBUG_FILE="file.txt"`` option to coccicheck. For
|
|
|
+instance::
|
|
|
+
|
|
|
+ rm -f cocci.err
|
|
|
+ make coccicheck COCCI=scripts/coccinelle/free/kfree.cocci MODE=report DEBUG_FILE=cocci.err
|
|
|
+ cat cocci.err
|
|
|
+
|
|
|
+You can use SPFLAGS to add debugging flags, for instance you may want to
|
|
|
+add both --profile --show-trying to SPFLAGS when debugging. For instance
|
|
|
+you may want to use::
|
|
|
+
|
|
|
+ rm -f err.log
|
|
|
+ export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci
|
|
|
+ make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c
|
|
|
+
|
|
|
+err.log will now have the profiling information, while stdout will
|
|
|
+provide some progress information as Coccinelle moves forward with
|
|
|
+work.
|
|
|
+
|
|
|
+DEBUG_FILE support is only supported when using coccinelle >= 1.2.
|
|
|
+
|
|
|
+.cocciconfig support
|
|
|
+--------------------
|
|
|
+
|
|
|
+Coccinelle supports reading .cocciconfig for default Coccinelle options that
|
|
|
+should be used every time spatch is spawned, the order of precedence for
|
|
|
+variables for .cocciconfig is as follows:
|
|
|
+
|
|
|
+- Your current user's home directory is processed first
|
|
|
+- Your directory from which spatch is called is processed next
|
|
|
+- The directory provided with the --dir option is processed last, if used
|
|
|
+
|
|
|
+Since coccicheck runs through make, it naturally runs from the kernel
|
|
|
+proper dir, as such the second rule above would be implied for picking up a
|
|
|
+.cocciconfig when using ``make coccicheck``.
|
|
|
+
|
|
|
+``make coccicheck`` also supports using M= targets.If you do not supply
|
|
|
+any M= target, it is assumed you want to target the entire kernel.
|
|
|
+The kernel coccicheck script has::
|
|
|
+
|
|
|
+ if [ "$KBUILD_EXTMOD" = "" ] ; then
|
|
|
+ OPTIONS="--dir $srctree $COCCIINCLUDE"
|
|
|
+ else
|
|
|
+ OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
|
|
|
+ fi
|
|
|
+
|
|
|
+KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases
|
|
|
+the spatch --dir argument is used, as such third rule applies when whether M=
|
|
|
+is used or not, and when M= is used the target directory can have its own
|
|
|
+.cocciconfig file. When M= is not passed as an argument to coccicheck the
|
|
|
+target directory is the same as the directory from where spatch was called.
|
|
|
+
|
|
|
+If not using the kernel's coccicheck target, keep the above precedence
|
|
|
+order logic of .cocciconfig reading. If using the kernel's coccicheck target,
|
|
|
+override any of the kernel's .coccicheck's settings using SPFLAGS.
|
|
|
+
|
|
|
+We help Coccinelle when used against Linux with a set of sensible defaults
|
|
|
+options for Linux with our own Linux .cocciconfig. This hints to coccinelle
|
|
|
+git can be used for ``git grep`` queries over coccigrep. A timeout of 200
|
|
|
+seconds should suffice for now.
|
|
|
+
|
|
|
+The options picked up by coccinelle when reading a .cocciconfig do not appear
|
|
|
+as arguments to spatch processes running on your system, to confirm what
|
|
|
+options will be used by Coccinelle run::
|
|
|
+
|
|
|
+ spatch --print-options-only
|
|
|
+
|
|
|
+You can override with your own preferred index option by using SPFLAGS. Take
|
|
|
+note that when there are conflicting options Coccinelle takes precedence for
|
|
|
+the last options passed. Using .cocciconfig is possible to use idutils, however
|
|
|
+given the order of precedence followed by Coccinelle, since the kernel now
|
|
|
+carries its own .cocciconfig, you will need to use SPFLAGS to use idutils if
|
|
|
+desired. See below section "Additional flags" for more details on how to use
|
|
|
+idutils.
|
|
|
+
|
|
|
+Additional flags
|
|
|
+----------------
|
|
|
+
|
|
|
+Additional flags can be passed to spatch through the SPFLAGS
|
|
|
+variable. This works as Coccinelle respects the last flags
|
|
|
+given to it when options are in conflict. ::
|
|
|
+
|
|
|
+ make SPFLAGS=--use-glimpse coccicheck
|
|
|
+
|
|
|
+Coccinelle supports idutils as well but requires coccinelle >= 1.0.6.
|
|
|
+When no ID file is specified coccinelle assumes your ID database file
|
|
|
+is in the file .id-utils.index on the top level of the kernel, coccinelle
|
|
|
+carries a script scripts/idutils_index.sh which creates the database with::
|
|
|
+
|
|
|
+ mkid -i C --output .id-utils.index
|
|
|
+
|
|
|
+If you have another database filename you can also just symlink with this
|
|
|
+name. ::
|
|
|
+
|
|
|
+ make SPFLAGS=--use-idutils coccicheck
|
|
|
+
|
|
|
+Alternatively you can specify the database filename explicitly, for
|
|
|
+instance::
|
|
|
+
|
|
|
+ make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck
|
|
|
+
|
|
|
+See ``spatch --help`` to learn more about spatch options.
|
|
|
+
|
|
|
+Note that the ``--use-glimpse`` and ``--use-idutils`` options
|
|
|
+require external tools for indexing the code. None of them is
|
|
|
+thus active by default. However, by indexing the code with
|
|
|
+one of these tools, and according to the cocci file used,
|
|
|
+spatch could proceed the entire code base more quickly.
|
|
|
+
|
|
|
+SmPL patch specific options
|
|
|
+---------------------------
|
|
|
+
|
|
|
+SmPL patches can have their own requirements for options passed
|
|
|
+to Coccinelle. SmPL patch specific options can be provided by
|
|
|
+providing them at the top of the SmPL patch, for instance::
|
|
|
+
|
|
|
+ // Options: --no-includes --include-headers
|
|
|
+
|
|
|
+SmPL patch Coccinelle requirements
|
|
|
+----------------------------------
|
|
|
+
|
|
|
+As Coccinelle features get added some more advanced SmPL patches
|
|
|
+may require newer versions of Coccinelle. If an SmPL patch requires
|
|
|
+at least a version of Coccinelle, this can be specified as follows,
|
|
|
+as an example if requiring at least Coccinelle >= 1.0.5::
|
|
|
+
|
|
|
+ // Requires: 1.0.5
|
|
|
+
|
|
|
+Proposing new semantic patches
|
|
|
+-------------------------------
|
|
|
+
|
|
|
+New semantic patches can be proposed and submitted by kernel
|
|
|
+developers. For sake of clarity, they should be organized in the
|
|
|
+sub-directories of ``scripts/coccinelle/``.
|
|
|
+
|
|
|
+
|
|
|
+Detailed description of the ``report`` mode
|
|
|
+-------------------------------------------
|
|
|
+
|
|
|
+``report`` generates a list in the following format::
|
|
|
+
|
|
|
+ file:line:column-column: message
|
|
|
+
|
|
|
+Example
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+Running::
|
|
|
+
|
|
|
+ make coccicheck MODE=report COCCI=scripts/coccinelle/api/err_cast.cocci
|
|
|
+
|
|
|
+will execute the following part of the SmPL script::
|
|
|
+
|
|
|
+ <smpl>
|
|
|
+ @r depends on !context && !patch && (org || report)@
|
|
|
+ expression x;
|
|
|
+ position p;
|
|
|
+ @@
|
|
|
+
|
|
|
+ ERR_PTR@p(PTR_ERR(x))
|
|
|
+
|
|
|
+ @script:python depends on report@
|
|
|
+ p << r.p;
|
|
|
+ x << r.x;
|
|
|
+ @@
|
|
|
+
|
|
|
+ msg="ERR_CAST can be used with %s" % (x)
|
|
|
+ coccilib.report.print_report(p[0], msg)
|
|
|
+ </smpl>
|
|
|
+
|
|
|
+This SmPL excerpt generates entries on the standard output, as
|
|
|
+illustrated below::
|
|
|
+
|
|
|
+ /home/user/linux/crypto/ctr.c:188:9-16: ERR_CAST can be used with alg
|
|
|
+ /home/user/linux/crypto/authenc.c:619:9-16: ERR_CAST can be used with auth
|
|
|
+ /home/user/linux/crypto/xts.c:227:9-16: ERR_CAST can be used with alg
|
|
|
+
|
|
|
+
|
|
|
+Detailed description of the ``patch`` mode
|
|
|
+------------------------------------------
|
|
|
+
|
|
|
+When the ``patch`` mode is available, it proposes a fix for each problem
|
|
|
+identified.
|
|
|
+
|
|
|
+Example
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+Running::
|
|
|
+
|
|
|
+ make coccicheck MODE=patch COCCI=scripts/coccinelle/api/err_cast.cocci
|
|
|
+
|
|
|
+will execute the following part of the SmPL script::
|
|
|
+
|
|
|
+ <smpl>
|
|
|
+ @ depends on !context && patch && !org && !report @
|
|
|
+ expression x;
|
|
|
+ @@
|
|
|
+
|
|
|
+ - ERR_PTR(PTR_ERR(x))
|
|
|
+ + ERR_CAST(x)
|
|
|
+ </smpl>
|
|
|
+
|
|
|
+This SmPL excerpt generates patch hunks on the standard output, as
|
|
|
+illustrated below::
|
|
|
+
|
|
|
+ diff -u -p a/crypto/ctr.c b/crypto/ctr.c
|
|
|
+ --- a/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200
|
|
|
+ +++ b/crypto/ctr.c 2010-06-03 23:44:49.000000000 +0200
|
|
|
+ @@ -185,7 +185,7 @@ static struct crypto_instance *crypto_ct
|
|
|
+ alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER,
|
|
|
+ CRYPTO_ALG_TYPE_MASK);
|
|
|
+ if (IS_ERR(alg))
|
|
|
+ - return ERR_PTR(PTR_ERR(alg));
|
|
|
+ + return ERR_CAST(alg);
|
|
|
+
|
|
|
+ /* Block size must be >= 4 bytes. */
|
|
|
+ err = -EINVAL;
|
|
|
+
|
|
|
+Detailed description of the ``context`` mode
|
|
|
+--------------------------------------------
|
|
|
+
|
|
|
+``context`` highlights lines of interest and their context
|
|
|
+in a diff-like style.
|
|
|
+
|
|
|
+ **NOTE**: The diff-like output generated is NOT an applicable patch. The
|
|
|
+ intent of the ``context`` mode is to highlight the important lines
|
|
|
+ (annotated with minus, ``-``) and gives some surrounding context
|
|
|
+ lines around. This output can be used with the diff mode of
|
|
|
+ Emacs to review the code.
|
|
|
+
|
|
|
+Example
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+Running::
|
|
|
+
|
|
|
+ make coccicheck MODE=context COCCI=scripts/coccinelle/api/err_cast.cocci
|
|
|
+
|
|
|
+will execute the following part of the SmPL script::
|
|
|
+
|
|
|
+ <smpl>
|
|
|
+ @ depends on context && !patch && !org && !report@
|
|
|
+ expression x;
|
|
|
+ @@
|
|
|
+
|
|
|
+ * ERR_PTR(PTR_ERR(x))
|
|
|
+ </smpl>
|
|
|
+
|
|
|
+This SmPL excerpt generates diff hunks on the standard output, as
|
|
|
+illustrated below::
|
|
|
+
|
|
|
+ diff -u -p /home/user/linux/crypto/ctr.c /tmp/nothing
|
|
|
+ --- /home/user/linux/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200
|
|
|
+ +++ /tmp/nothing
|
|
|
+ @@ -185,7 +185,6 @@ static struct crypto_instance *crypto_ct
|
|
|
+ alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER,
|
|
|
+ CRYPTO_ALG_TYPE_MASK);
|
|
|
+ if (IS_ERR(alg))
|
|
|
+ - return ERR_PTR(PTR_ERR(alg));
|
|
|
+
|
|
|
+ /* Block size must be >= 4 bytes. */
|
|
|
+ err = -EINVAL;
|
|
|
+
|
|
|
+Detailed description of the ``org`` mode
|
|
|
+----------------------------------------
|
|
|
+
|
|
|
+``org`` generates a report in the Org mode format of Emacs.
|
|
|
+
|
|
|
+Example
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+Running::
|
|
|
+
|
|
|
+ make coccicheck MODE=org COCCI=scripts/coccinelle/api/err_cast.cocci
|
|
|
+
|
|
|
+will execute the following part of the SmPL script::
|
|
|
+
|
|
|
+ <smpl>
|
|
|
+ @r depends on !context && !patch && (org || report)@
|
|
|
+ expression x;
|
|
|
+ position p;
|
|
|
+ @@
|
|
|
+
|
|
|
+ ERR_PTR@p(PTR_ERR(x))
|
|
|
+
|
|
|
+ @script:python depends on org@
|
|
|
+ p << r.p;
|
|
|
+ x << r.x;
|
|
|
+ @@
|
|
|
+
|
|
|
+ msg="ERR_CAST can be used with %s" % (x)
|
|
|
+ msg_safe=msg.replace("[","@(").replace("]",")")
|
|
|
+ coccilib.org.print_todo(p[0], msg_safe)
|
|
|
+ </smpl>
|
|
|
+
|
|
|
+This SmPL excerpt generates Org entries on the standard output, as
|
|
|
+illustrated below::
|
|
|
+
|
|
|
+ * TODO [[view:/home/user/linux/crypto/ctr.c::face=ovl-face1::linb=188::colb=9::cole=16][ERR_CAST can be used with alg]]
|
|
|
+ * TODO [[view:/home/user/linux/crypto/authenc.c::face=ovl-face1::linb=619::colb=9::cole=16][ERR_CAST can be used with auth]]
|
|
|
+ * TODO [[view:/home/user/linux/crypto/xts.c::face=ovl-face1::linb=227::colb=9::cole=16][ERR_CAST can be used with alg]]
|