camera-sensor.rst 5.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134
  1. .. SPDX-License-Identifier: GPL-2.0
  2. Writing camera sensor drivers
  3. =============================
  4. CSI-2
  5. -----
  6. Please see what is written on :ref:`MIPI_CSI_2`.
  7. Handling clocks
  8. ---------------
  9. Camera sensors have an internal clock tree including a PLL and a number of
  10. divisors. The clock tree is generally configured by the driver based on a few
  11. input parameters that are specific to the hardware:: the external clock frequency
  12. and the link frequency. The two parameters generally are obtained from system
  13. firmware. No other frequencies should be used in any circumstances.
  14. The reason why the clock frequencies are so important is that the clock signals
  15. come out of the SoC, and in many cases a specific frequency is designed to be
  16. used in the system. Using another frequency may cause harmful effects
  17. elsewhere. Therefore only the pre-determined frequencies are configurable by the
  18. user.
  19. Frame size
  20. ----------
  21. There are two distinct ways to configure the frame size produced by camera
  22. sensors.
  23. Freely configurable camera sensor drivers
  24. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  25. Freely configurable camera sensor drivers expose the device's internal
  26. processing pipeline as one or more sub-devices with different cropping and
  27. scaling configurations. The output size of the device is the result of a series
  28. of cropping and scaling operations from the device's pixel array's size.
  29. An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp).
  30. Register list based drivers
  31. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  32. Register list based drivers generally, instead of able to configure the device
  33. they control based on user requests, are limited to a number of preset
  34. configurations that combine a number of different parameters that on hardware
  35. level are independent. How a driver picks such configuration is based on the
  36. format set on a source pad at the end of the device's internal pipeline.
  37. Most sensor drivers are implemented this way, see e.g.
  38. drivers/media/i2c/imx319.c for an example.
  39. Frame interval configuration
  40. ----------------------------
  41. There are two different methods for obtaining possibilities for different frame
  42. intervals as well as configuring the frame interval. Which one to implement
  43. depends on the type of the device.
  44. Raw camera sensors
  45. ~~~~~~~~~~~~~~~~~~
  46. Instead of a high level parameter such as frame interval, the frame interval is
  47. a result of the configuration of a number of camera sensor implementation
  48. specific parameters. Luckily, these parameters tend to be the same for more or
  49. less all modern raw camera sensors.
  50. The frame interval is calculated using the following equation::
  51. frame interval = (analogue crop width + horizontal blanking) *
  52. (analogue crop height + vertical blanking) / pixel rate
  53. The formula is bus independent and is applicable for raw timing parameters on
  54. large variety of devices beyond camera sensors. Devices that have no analogue
  55. crop, use the full source image size, i.e. pixel array size.
  56. Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
  57. ``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The
  58. pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The
  59. unit of that control is Hz.
  60. Register list based drivers need to implement read-only sub-device nodes for the
  61. purpose. Devices that are not register list based need these to configure the
  62. device's internal processing pipeline.
  63. The first entity in the linear pipeline is the pixel array. The pixel array may
  64. be followed by other entities that are there to allow configuring binning,
  65. skipping, scaling or digital crop :ref:`v4l2-subdev-selections`.
  66. USB cameras etc. devices
  67. ~~~~~~~~~~~~~~~~~~~~~~~~
  68. USB video class hardware, as well as many cameras offering a similar higher
  69. level interface natively, generally use the concept of frame interval (or frame
  70. rate) on device level in firmware or hardware. This means lower level controls
  71. implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
  72. frame interval on these devices.
  73. Power management
  74. ----------------
  75. Always use runtime PM to manage the power states of your device. Camera sensor
  76. drivers are in no way special in this respect: they are responsible for
  77. controlling the power state of the device they otherwise control as well. In
  78. general, the device must be powered on at least when its registers are being
  79. accessed and when it is streaming.
  80. Existing camera sensor drivers may rely on the old
  81. :c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to
  82. manage their power state. This is however **deprecated**. If you feel you need
  83. to begin calling an s_power from an ISP or a bridge driver, instead please add
  84. runtime PM support to the sensor driver you are using. Likewise, new drivers
  85. should not use s_power.
  86. Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and
  87. ``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI
  88. and DT based systems.
  89. Control framework
  90. ~~~~~~~~~~~~~~~~~
  91. ``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime
  92. PM ``runtime_resume`` callback, as it has no way to figure out the power state
  93. of the device. This is because the power state of the device is only changed
  94. after the power state transition has taken place. The ``s_ctrl`` callback can be
  95. used to obtain device's power state after the power state transition:
  96. .. c:function::
  97. int pm_runtime_get_if_in_use(struct device *dev);
  98. The function returns a non-zero value if it succeeded getting the power count or
  99. runtime PM was disabled, in either of which cases the driver may proceed to
  100. access the device.