Camera

The legacy camera stack can be re-enabled in Bullseye using the following steps.

These steps are shown in the following video.

The flex cable inserts into the connector labelled CAMERA on the Raspberry Pi, which is located between the Ethernet and HDMI ports. The cable must be inserted with the silver contacts facing the HDMI port. To open the connector, pull the tabs on the top of the connector upwards, then towards the Ethernet port. The flex cable should be inserted firmly into the connector, with care taken not to bend the flex at too acute an angle. To close the connector, push the top part of the connector towards the HDMI port and down, while holding the flex cable in place.

We have created a video to illustrate the process of connecting the camera. Although the video shows the original camera on the original Raspberry Pi 1, the principle is the same for all camera boards:

Depending on the model, the camera may come with a small piece of translucent blue plastic film covering the lens. This is only present to protect the lens while it is being mailed to you, and needs to be removed by gently peeling it off.

libcamera is an open source Linux community project. More information is available at the libcamera website.

The libcamera source code can be found and checked out from the official libcamera repository.

Underneath the libcamera core, Raspberry Pi provides a custom pipeline handler, which is the layer that libcamera uses to drive the sensor and ISP (Image Signal Processor) on the Raspberry Pi itself. Also part of this is a collection of well-known control algorithms, or IPAs (Image Processing Algorithms) in libcamera parlance, such as AEC/AGC (Auto Exposure/Gain Control), AWB (Auto White Balance), ALSC (Auto Lens Shading Correction) and so on.

All this code is open source and now runs on the Raspberry Pi’s ARM cores. There is only a very thin layer of code on the GPU which translates Raspberry Pi’s own control parameters into register writes for the Broadcom ISP.

Raspberry Pi’s implementation of libcamera supports not only the three standard Raspberry Pi cameras (the OV5647 or V1 camera, the IMX219 or V2 camera and the IMX477 or HQ camera) but also third party senors such as the IMX290, IMX327, OV9281, IMX378. Raspberry Pi is keen to work with vendors who would like to see their sensors supported directly by libcamera.

Moreover, Raspberry Pi supplies a tuning file for each of these sensors which can be edited to change the processing performed by the Raspberry Pi hardware on the raw images received from the image sensor, including aspects like the colour processing, the amount of noise suppression or the behaviour of the control algorithms.

For further information on libcamera for the Raspberry Pi, please consult the Tuning Guide for the Raspberry Pi cameras and libcamera.

When running a Raspberry Pi OS based on Bullseye, the 5 basic libcamera-apps are already installed. In this case, official Raspberry Pi cameras will also be detected and enabled automatically.

You can check that everything is working by entering:

You should see a camera preview window for about 5 seconds.

Users running Buster will need to install one of the libcamera-apps packages first and then configure their /boot/config.txt file with the appropriate overlay for the connected camera. Be aware that libcamera and the legacy raspicam stack cannot operate at the same time - to return to the legacy stack after using libcamera you will need to comment out the dtoverlay change you made and reboot the system.

You may need to alter the camera configuration in your /boot/config.txt file if:

If you do need to add your own dtoverlay, the following are currently recognised.

To override the automatic camera detection, Bullseye users will also need to delete the entry camera_auto_detect=1 if present in the config.txt file. Your Raspberry Pi will need to be rebooted after editing this file.

libcamera-apps uses a 3rd party library to interpret command line options. This includes long form options where the option name consists of more than one character preceded by --, and short form options which can only be a single character preceded by a single -. For the most part option names are chosen to match those used by the legacy raspicam applications with the exception that we can no longer handle multi-character option names with a single -. Any such legacy options have been dropped and the long form with -- must be used instead.

The options are classified broadly into 3 groups, namely those that are common, those that are specific to still images, and those that are for video encoding. They are supported in an identical manner across all the applications where they apply.

Please refer to the command line options documentation for a complete list.

Raspberry Pi’s libcamera implementation includes a tuning file for each different type of camera module. This is a file that describes or "tunes" the parameters that will be passed to the algorithms and hardware to produce the best image quality. libcamera is only able to determine automatically the image sensor being used, not the module as a whole - even though the whole module affects the "tuning".

For this reason it is sometimes necessary to override the default tuning file for a particular sensor.

For example, the NOIR (no IR-filter) versions of sensors require different AWB settings to the standard versions, so the IMX219 NOIR should be run using

If you are using a Soho Enterprises SE327M12 module you should use

Notice how this also means that users can copy an existing tuning file and alter it according to their own preferences, so long as the --tuning-file parameter is pointed to the new version.

Finally, the --tuning-file parameter, in common with other libcamera-hello command line options, applies identically across all the libcamera-apps.

Most of the libcamera-apps display a preview image in a window. When X Windows is not running it will draw directly to the display using Linux DRM (Direct Rendering Manager), otherwise it will attempt to use X Windows. Both paths use zero-copy buffer sharing with the GPU, and a consequence of this is that X forwarding is not supported.

For this reason there is a third kind of preview window which does support X forwarding, and can be requested with the --qt-preview option. This implementation does not benefit from zero-copy buffer sharing nor from 3D acceleration which makes it computationally expensive (especially for large previews), and so is not normally recommended.

The preview window can be suppressed entirely with the -n (--nopreview) option.

The --info-text option allows the user to request that certain helpful image information is displayed on the window title bar using "% directives". For example

will display the current red and blue gain values.

For the HQ camera, use --info-text "%focus" to display the focus measure, which will be helpful for focusing the lens.

A full description of the --info-text parameter is given in the command line options documentation.

All the libcamera-apps allow the user to run the camera with fixed shutter speed and gain. For example

would capture an image with an exposure of 20ms and a gain of 1.5x. Note that the gain will be applied as analogue gain within the sensor up until it reaches the maximum analogue gain permitted by the kernel sensor driver, after which the remainder will be applied as digital gain.

Raspberry Pi’s AEC/AGC algorithm allows applications to specify exposure compensation, that is, the ability to make images darker or brighter by a given number of stops, as follows

libcamera-still allows files to be saved in a number of different formats. It supports both png and bmp encoding. It also allows files to be saved as a binary dump of RGB or YUV pixels with no encoding or file format at all. In these latter cases the application reading the files will have to understand the pixel arrangement for itself.

Note that the format in which the image is saved depends on the -e (equivalently --encoding) option and is not selected automatically based on the output file name.

Raw images are the images produced directly by the image sensor, before any processing is applied to them either by the ISP (Image Signal Processor) or any of the CPU cores. For colour image sensors these are usually Bayer format images. Note that raw images are quite different from the processed but unencoded RGB or YUV images that we saw earlier.

To capture a raw image use

Here, the -r option (also --raw) indicates to capture the raw image as well as the JPEG. In fact, the raw image is the exact image from which the JPEG was produced. Raw images are saved in DNG (Adobe Digital Negative) format and are compatible with many standard applications, such as dcraw or RawTherapee. The raw image is saved to a file with the same name but the extension .dng, thus test.dng in this case.

These DNG files contain metadata pertaining to the image capture, including black levels, white balance information and the colour matrix used by the ISP to produce the JPEG. This makes these DNG files much more convenient for later "by hand" raw conversion with some of the aforementioned tools. Using exiftool shows all the metadata encoded into the DNG file:

We note that there is only a single calibrated illuminant (the one determined by the AWB algorithm even though it gets labelled always as "D65"), and that dividing the ISO number by 100 gives the analogue gain that was being used.

To capture very long exposure images, we need to be careful to disable the AEC/AGC and AWB because these algorithms will otherwise force the user to wait for a number of frames while they converge. The way to disable them is to supply explicit values. Additionally, the entire preview phase of the capture can be skipped with the --immediate option.

So to perform a 100 second exposure capture, use

libcamera-still -o long_exposure.jpg --shutter 100000000 --gain 1 --awbgains 1,1 --immediate

For reference, the maximum exposure times of the three official Raspberry Pi cameras can be found in this table.

There is support for motion JPEG, and also for uncompressed and unformatted YUV420, for example

In both cases the --codec parameter determines the output format, not the extension of the output file.

The --segment parameter breaks output files up into chunks of the segment size (given in milliseconds). This is quite handy for breaking a motion JPEG stream up into individual JPEG files by specifying very short (1 millisecond) segments.

Observe that the output file name is normally only sensible if we avoid over-writing the previous file every time, such as by using a file name that includes a counter (as above). More information on output file names is available below.

It is possible to use the libav backend as a network streaming source for audio/video. To do this, the output filename specified by the -o argument must be given as a protocol url, see ffmpeg protocols for more details on protocol usage. Some examples:

To stream audio/video using TCP

To stream audio/video using UDP

Sets the size and location of the preview window (both X Windows and DRM versions). It does not affect the resolution or aspect ratio of images being requested from the camera. The camera images will be scaled to the size of the preview window for display, and will be pillar/letter-boxed to fit.

Example: libcamera-hello -p 100,100,500,500

Forces the preview window to use the whole screen, and the window will have no border or title bar. Again the image may be pillar/letter-boxed.

Example libcamera-still -f -o test.jpg

The preview window is switched to use the Qt-based implementation. This option is not normally recommended because it no longer uses zero-copy buffer sharing nor GPU acceleration and is therefore very expensive, however, it does support X forwarding (which the other preview implementations do not).

The Qt preview window does not support the --fullscreen option. Generally it is advised to try and keep the preview window small.

Example libcamera-hello --qt-preview

The preview window is suppressed entirely.

Example libcamera-still -n -o test.jpg

The supplied string is set as the title of the preview window (when running under X Windows). Additionally the string may contain a number of % directives which are substituted with information from the image metadata. The permitted directives are

When not provided, the --info-text string defaults to "#%frame (%fps fps) exp %exp ag %ag dg %dg".

Example: libcamera-hello --info-test "Focus measure: %focus

These numbers specify the output resolution of the camera images captured by libcamera-still, libcamera-jpeg and libcamera-vid.

For libcamera-raw, it affects the size of the raw frames captured. Where a camera has a 2x2 binned readout mode, specifying a resolution not larger than this binned mode will result in the capture of 2x2 binned raw frames.

For libcamera-hello these parameters have no effect.

Examples:

libcamera-vid -o test.h264 --width 1920 --height 1080 will capture 1080p video.

libcamera-still -r -o test.jpg --width 2028 --height 1520 will capture a 2028x1520 resolution JPEG. When using the HQ camera the sensor will be driven in its 2x2 binned mode so the raw file - captured in test.dng - will contain a 2028x1520 raw Bayer image.

These options affect only the preview (meaning both libcamera-hello and the preview phase of libcamera-jpeg and libcamera-still), and specify the image size that will be requested from the camera for the preview window. They have no effect on captured still images or videos. Nor do they affect the preview window as the images are resized to fit.

Example: libcamera-hello --viewfinder-width 640 --viewfinder-height 480

This option forces the sensor to be driven in its full resolution readout mode for still and video capture, irrespective of the requested output resolution (given by --width and --height). It has no effect for libcamera-hello.

Using this option often incurs a frame rate penalty, as larger resolution frames are slower to read out.

Example: libcamera-raw -t 2000 --segment 1 --rawfull -o test%03d.raw will cause multiple full resolution raw frames to be captured. On the HQ camera each frame will be about 18MB in size. Without the --rawfull option the default video output resolution would have caused the 2x2 binned mode to be selected, resulting in 4.5MB raw frames.

This option is more general than --rawfull and allows the precise selection of one of the camera modes. The mode should be specified by giving its width, height, bit-depth and packing, separated by colons. These numbers do not have to be exact as the system will select the closest it can find. Moreover, the bit-depth and packing are optional (defaulting to 12 and P for "packed" respectively). For example:

The --mode option affects the mode choice for video recording and stills capture. To control the mode choice during the preview phase prior to stills capture, please use the --viewfinder-mode option.

This option is identical to the --mode option except that it applies only during the preview phase of stills capture (also used by the libcamera-hello application).

libcamera allows the possibility of delivering a second lower resolution image stream from the camera system to the application. This stream is available in both the preview and the video modes (i.e. libcamera-hello and the preview phase of libcamera-still, and libcamera-vid), and can be used, among other things, for image analysis. For stills captures, the low resolution image stream is not available.

The low resolution stream has the same field of view as the other image streams. If a different aspect ratio is specified for the low resolution stream, then those images will be squashed so that the pixels are no longer square.

During video recording (libcamera-vid), specifying a low resolution stream will disable some extra colour denoise processing that would normally occur.

Example: libcamera-hello --lores-width 224 --lores-height 224

Note that the low resolution stream is not particularly useful unless used in conjunction with image post-processing.

These options affect the order of read-out from the sensor, and can be used to mirror the image horizontally, and/or flip it vertically. The --rotation option permits only the value 0 or 180, so note that 90 or 270 degree rotations are not supported. Moreover, --rotation 180 is identical to --hflip --vflip.

Example: libcamera-hello --vflip --hflip

The --roi (region of interest) option allows the user to select a particular crop from the full field of view provided by the sensor. The coordinates are specified as a proportion of the available field of view, so that --roi 0,0,1,1 would have no effect at all.

The --roi parameter implements what is commonly referred to as "digital zoom".

Example libcamera-hello --roi 0.25,0.25,0.5,0.5 will select exactly a quarter of the total number of pixels cropped from the centre of the image.

The following options affect the image processing and control algorithms that affect the camera image quality.

The given <number> adjusts the image sharpness. The value zero means that no sharpening is applied, the value 1.0 uses the default amount of sharpening, and values greater than 1.0 use extra sharpening.

Example: libcamera-still -o test.jpg --sharpness 2.0

The given <number> adjusts the image contrast. The value zero produces minimum contrast, the value 1.0 uses the default amount of contrast, and values greater than 1.0 apply extra contrast.

Example: libcamera-still -o test.jpg --contrast 1.5

The given <number> adjusts the image brightness. The value -1.0 produces an (almost) black image, the value 1.0 produces an almost entirely white image and the value 0.0 produces standard image brightness.

Note that the brightness parameter adds (or subtracts) an offset from all pixels in the output image. The --ev option is often more appropriate.

Example: libcamera-still -o test.jpg --brightness 0.2

The given <number> adjusts the colour saturation. The value zero produces a greyscale image, the value 1.0 uses the default amount of sautration, and values greater than 1.0 apply extra colour saturation.

Example: libcamera-still -o test.jpg --saturation 0.8

Sets the EV compensation of the image in units of stops, in the range -10 to 10. Default is 0. It works by raising or lowering the target values the AEC/AGC algorithm is attempting to match.

Example: libcamera-still -o test.jpg --ev 0.3

The shutter time is fixed to the given value. The gain will still be allowed to vary (unless that is also fixed).

Note that this shutter time may not be achieved if the camera is running at a frame rate that is too fast to allow it. In this case the --framerate option may be used to lower the frame rate. The maximum possible shutter times for the official Raspberry Pi supported can be found in this table.

Using values above these maximums will result in undefined behaviour. Cameras will also have different minimum shutter times, though in practice this is not important as they are all low enough to expose bright scenes appropriately.

Example: libcamera-hello --shutter 30000

These two options are actually identical, and set the combined analogue and digital gains that will be used. The --analoggain form is permitted so as to be more compatible with the legacy raspicam applications. Where the requested gain can be supplied by the sensor driver, then only analogue gain will be used. Once the analogue gain reaches the maximum permitted value, then extra gain beyond this will be supplied as digital gain.

Note that there are circumstances where the digital gain can go above 1 even when the analogue gain limit is not exceeded. This can occur when

Sets the metering mode of the AEC/AGC algorithm. This may one of the following values

For more information on defining a custom metering mode, and also on how to adjust the region weights in the existing metering modes, please refer to the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-still -o test.jpg --metering spot

The exposure profile may be either normal, sport or long. Changing the exposure profile should not affect the overall exposure of an image, but the sport mode will tend to prefer shorter exposure times and larger gains to achieve the same net result.

Exposure profiles can be edited in the camera tuning file. Please refer to the Tuning guide for the Raspberry Pi cameras and libcamera for more information.

Example: libcamera-still -o test.jpg --exposure sport

This option sets the AWB algorithm into the named AWB mode. Valid modes are:

There is no mode that turns the AWB off, instead fixed colour gains should be specified with the --awbgains option.

Note that these values are only approximate, the values could vary according to the camera tuning.

For more information on AWB modes and how to define a custom one, please refer to the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-still -o test.jpg --awb tungsten

This option accepts a red and a blue gain value and uses them directly in place of running the AWB algorithm. Setting non-zero values here has the effect of disabling the AWB calculation.

Example: libcamera-still -o test.jpg --awbgains 1.5,2.0

The following denoise modes are supported:

Note that even the use of fast colour denoise can result in lower framerates. The high quality colour denoise will normally result in much lower framerates.

Example: libcamera-vid -o test.h264 --denoise cdn_off

This identifies the name of the JSON format tuning file that should be used. The tuning file covers many aspects of the image processing, including the AEC/AGC, AWB, colour shading correction, colour processing, denoising and so forth.

For more information on the camera tuning file, please consult the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-hello --tuning-file ~/my-camera-tuning.json

--output sets the name of the output file to which the output image or video is written. Besides regular file names, this may take the following special values:

Examples:

libcamera-vid -t 100000 --segment 10000 -o chunk%04d.h264 records a 100 second file in 10 second segments, where each file is named chunk.h264 but with the inclusion of an incrementing counter. Note that %04d writes the count to a string, but padded up to a total width of at least 4 characters by adding leading zeroes.

libcamera-vid -t 0 --inline -o udp://192.168.1.13:5000 stream H.264 video to network address 192.168.1.13 on port 5000.

When outputting to files with an incrementing counter (e.g. %d in the output file name), wrap the counter back to zero when it reaches this value.

Example: libcamera-vid -t 0 --codec mjpeg --segment 1 --wrap 100 -o image%d.jpg

--flush causes output files to be flushed to disk as soon as every frame is written, rather than waiting for the system to do it.

Example: libcamera-vid -t 10000 --flush -o test.h264

The --post-process-file option specifies a JSON file that configures the post-processing that the imaging pipeline applies to camera images before they reach the application. It can be thought of as a replacement for the legacy raspicam "image effects".

Post-processing is a large topic and admits the use of 3rd party software like OpenCV and TensorFlowLite to analyse and manipulate images. For more information, please refer to the section on post-processing.

Example: libcamera-hello --post-process-file negate.json

This might apply a "negate" effect to an image, if the file negate.json is appropriately configured.

The negate stage requires no 3rd party libraries.

On a Raspberry Pi 3 device or a Raspberry Pi 4 running a 32-bit OS, it may execute more quickly if recompiled using -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon. (Please see the build instructions.)

The negate stage has no user-configurable parameters.

Default negate.json file:

Example:

The hdr stage implements both HDR (high dynamic range) imaging and DRC (dynamic range compression). The terminology that we use here regards DRC as operating on single images, and HDR works by accumulating multiple under-exposed images and then performing the same algorithm as DRC.

The hdr stage has no dependencies on 3rd party libraries, but (like some other stages) may execute more quickly on Raspberry Pi 3 or Raspberry Pi 4 devices running a 32-bit OS if recompiled using -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon (please see the build instructions). Specifically, the image accumulation stage will run quicker and result in fewer frame drops, though the tonemapping part of the process is unchanged.

The basic procedure is that we take the image (which in the case of HDR may be multiple images accumulated together) and apply an edge-preserving smoothing filter to generate a low pass (LP) image. We define the high pass (HP) image to be the difference between the LP image and the original. Next we apply a global tonemap to the LP image and add back the HP image. This procedure, in contrast to applying the tonemap directly to the original image, prevents us from squashing and losing all the local contrast in the resulting image.

It is worth noting that this all happens using fully-processed images, once the ISP has finished with them. HDR normally works better when carried out in the raw (Bayer) domain, as signals are still linear and have greater bit-depth. We expect to implement such functionality once libcamera exports an API for "re-processing" Bayer images that do not come from the sensor, but which application code can pass in.

In summary, the user-configurable parameters fall broadly into three groups: those that define the LP filter, those responsible for the global tonemapping, and those responsible for re-applying the local contrast.

We note that the overall strength of the processing is best controlled by changing the global_tonemap_strength and local_tonemap_strength parameters.

The full processing takes between 2 and 3 seconds for a 12MP image on a Raspberry Pi 4. The stage runs only on the still image capture, it ignores preview and video images. In particular, when accumulating multiple frames, the stage "swallows" the output images so that the application does not receive them, and finally sends through only the combined and processed image.

Default drc.json file for DRC:

Example:

Without DRC:

With full-strength DRC: (use libcamera-still -o test.jpg --post-process-file drc.json)

Default hdr.json file for HDR:

Example:

Without HDR:

With HDR: (use libcamera-still -o test.jpg --ev -2 --denoise cdn_off --post-process-file hdr.json)

The motion_detect stage works by analysing frames from the low resolution image stream, which must be configured for it to work. It compares a region of interest ("roi") in the frame to the corresponding part of a previous one and if enough pixels are sufficiently different, that will be taken to indicate motion. The result is added to the metadata under "motion_detect.result".

This stage has no dependencies on any 3rd party libraries.

It has the following tunable parameters. The dimensions are always given as a proportion of the low resolution image size.

Default motion_detect.json configuration file:

Note that the field difference_m and difference_c, and the value of region_threshold, can be adjusted to make the algorithm more or less sensitive to motion.

If the amount of computation needs to be reduced (perhaps you have other stages that need a larger low resolution image), the amount of computation can be reduced using the hskip and vskip parameters.

To use the motion_detect stage you might enter the following example command:

libcamera-hello --lores-width 128 --lores-height 96 --post-process-file motion_detect.json

The sobel_cv stage has the following user-configurable parameters:

Default sobel_cv.json file:

Example:

This stage uses the OpenCV Haar classifier to detect faces in an image. It returns the face locations in the metadata (under the key "face_detect.results"), and optionally draws them on the image.

The face_detect_cv stage has the following user-configurable parameters:

The `face_detect_cv" stage runs only during preview and video capture; it ignores still image capture. It runs on the low resolution stream which would normally be configured to a resolution from about 320x240 to 640x480 pixels.

Default face_detect_cv.json file:

Example:

This stage allows text to be written into the top corner of images. It allows the same % substitutions as the --info-text parameter.

The stage does not output any metadata, but if it finds metadata under the key "annotate.text" it will write this text in place of anything in the JSON configuration file. This allows other post-processing stages to pass it text strings to be written onto the top of the images.

The annotate_cv stage has the following user-configurable parameters:

Default annotate_cv.json file:

Example:

object_classify_tf uses a Google MobileNet v1 model to classify objects in the camera image. It can be obtained from storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz, which will need to be uncompressed. You will also need the labels.txt file which can be found in storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz.

This stage has the following configuratble parameters.

Example object_classify_tf.json file:

The stage operates on a low resolution stream image of size 224x224, so it could be used as follows:

libcamera-hello --post-process-file object_classify_tf.json --lores-width 224 --lores-height 224

pose_estimation_tf uses a Google MobileNet v1 model posenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflite that can be found at github.com/Qengineering/TensorFlow_Lite_Pose_RPi_32-bits.

This stage has the following configurable parameters.

Also provided is a separate plot_pose_cv stage which can be included in the JSON configuration file and which will draw the detected pose onto the main image. This stage has the following configuration parameters.

Example pose_estimation_tf.json file:

The stage operates on a low resolution stream image of size 257x257 (but which must be rounded up to 258x258 for YUV420 images), so it could be used as follows:

libcamera-hello --post-process-file pose_estimation_tf.json --lores-width 258 --lores-height 258

object_detect_tf uses a Google MobileNet v1 SSD (Single Shot Detector) model. The model and labels files can be downloaded from storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip.

This stage has the following configurable parameters.

Also provided is a separate object_detect_draw_cv stage which can be included in the JSON configuration file and which will draw the detected objects onto the main image. This stage has the following configuration parameters.

Example object_detect_tf.json file:

The stage operates on a low resolution stream image of size 300x300. The following example would pass a 300x300 crop to the detector from the centre of the 400x300 low resolution image.

libcamera-hello --post-process-file object_detect_tf.json --lores-width 400 --lores-height 300

segmentation_tf uses a Google MobileNet v1 model. The model file can be downloaded from tfhub.dev/tensorflow/lite-model/deeplabv3/1/metadata/2?lite-format=tflite, whilst the labels file can be found in the assets folder, named segmentation_labels.txt.

This stage runs on an image of size 257x257. Because YUV420 images must have even dimensions, the low resolution image should be at least 258 pixels in both width and height. The stage adds a vector of 257x257 values to the image metadata where each value indicates which of the categories (listed in the labels file) that the pixel belongs to. Optionally, a representation of the segmentation can be drawn into the bottom right corner of the image.

This stage has the following configurable parameters.

Example segmentation_tf.json file:

This example takes a square camera image and reduces it to 258x258 pixels in size. In fact the stage also works well when non-square images are squashed unequally down to 258x258 pixels without cropping. The image below shows the segmentation map in the bottom right hand corner.

libcamera-hello --post-process-file segmentation_tf.json --lores-width 258 --lores-height 258 --viewfinder-width 1024 --viewfinder-height 1024

Post-processing stages have a simple API, and users can create their own by deriving from the PostProcessingStage class. The member functions that must be implemented are listed below, though note that some may be unnecessary for simple stages.

Some helpful hints on writing your own stages:

The easiest example to start with is negate_stage.cpp, which "negates" an image (turning black white, and vice versa). Aside from a small amount of derived class boiler-plate, it contains barely half a dozen lines of code.

Next up in complexity is sobel_cv_stage.cpp. This implements a Sobel filter using just a few lines of OpenCV functions.

For stages wanting to analyse images using TensorFlowLite we provide the TfStage base class. This provides a certain amount of boilerplate code and makes it much easier to implement new TFLite-based stages by deriving from this class. In particular, it delegates the execution of the model to another thread, so that the full camera framerate is still maintained - it is just the model that will run at a lower framerate.

The TfStage class implements all the public PostProcessingStage methods that normally have to be redefined, with the exception of the Name method which must still be supplied. It then presents the following virtual methods which derived classes should implement instead.

For further information, readers are referred to the supplied example code implementing the ObjectClassifyTfStage and PoseEstimationTfStage classes.

There are two libcamera-apps packages available, that contain the necessary executables:

For Bullseye users, official Raspberry Pi cameras should be detected automatically. Other users will need to edit their /boot/config.txt file if they have not done so previously.

These applications depend on a number of library packages which are named library-name<n> where <n> is a version number (actually the ABI, or Application Binary Interface, version), and which stands at zero at the time of writing. Thus we have the following:

These will be installed automatically when needed.

libcamera-apps can be rebuilt on their own without installing and building libcamera and libepoxy from scratch. To enable this, the following packages should be installed:

Subsequently libcamera-apps can be checked out from Github and rebuilt.

You can rebuild libcamera-apps without first rebuilding the whole of libcamera and libepoxy. If you do not need support for the X11/GLES preview window then libepoxy can be omitted entirely. Mostly this will include Raspberry Pi OS Lite users, and they must be sure to use -DENABLE_X11=0 when running cmake later. These users should run:

All other users should execute:

If you want to use the Qt preview window, please also execute

Now proceed directly to the instructions for building libcamera-apps. Raspberry Pi OS Lite users should check that git is installed first (sudo apt install -y git).

Rebuilding libcamera from scratch should be necessary only if you need the latest features that may not yet have reached the apt repositories, or if you need to customise its behaviour in some way.

First install all the necessary dependencies for libcamera.

All users should then install the following:

In the meson commands below we have enabled the gstreamer plugin. If you do not need this you can set -Dgstreamer=disabled instead and the next pair of dependencies will not be required. But if you do leave gstreamer enabled, then you will need the following:

Now we can check out and build libcamera itself.

Next we recommend that Raspberry Pi OS Lite users run

Users of Raspberry Pi OS can instead use

The only difference is that the latter also builds the qcam test application, which has dependencies on Qt and X Windows (after completing the libcamera build users can run build/src/qcam/qcam to verify that libcamera is functioning correctly).

To complete the libcamera build, please run

Rebuilding libepoxy should not normally be necessary as this library changes only very rarely. If you do want to build it from scratch, however, please follow the instructions below.

Start by installing the necessary dependencies.

Next, check out and build libepoxy.

First fetch the necessary dependencies for libcamera-apps.

The libcamera-apps build process begins with the following:

At this point you will need to run cmake after deciding what extra flags to pass it. The valid flags are:

For Raspberry Pi OS users we recommend the following cmake command:

and for Raspberry Pi OS Lite users:

In both cases, consider -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon if you are using a 32-bit OS on a Raspberry Pi 3 or Raspberry Pi 4. Consider -DENABLE_OPENCV=1 if you have installed OpenCV and wish to use OpenCV-based post-processing stages. Finally also consider -DENABLE_TFLITE=1 if you have installed TensorFlow Lite and wish to use it in post-processing stages.

After executing the cmake command of your choice, the whole process concludes with the following:

Finally, if you have not already done so, please be sure to follow the dtoverlay and display driver instructions in the Getting Started section (and rebooting if you changed anything there).

libcamera-hello is much the easiest application to understand. The only thing it does with the camera images is extract the CompletedRequestPtr (a shared pointer to the CompletedRequest) from the message:

and forward it to the preview window:

One important thing to note is that every CompletedRequest must be recycled back to the camera system so that the buffers can be reused, otherwise it will simply run out of buffers in which to receive new camera frames. This recycling process happens automatically when all references to the CompletedRequest are dropped, using C++'s shared pointer and custom deleter mechanisms.

In libcamera-hello therefore, two things must happen for the CompletedRequest to be returned to the camera.

libcamera-vid is not unlike libcamera-hello, but it adds a codec to the event loop and the preview. Before the event loop starts, we must configure that encoder with a callback which says what happens to the buffer containing the encoded image data.

Here we send the buffer to the Output object which may write it to a file, or send it over the network, according to our choice when we started the application.

The encoder also takes a new reference to the CompletedRequest, so once the event loop, the preview window and the encoder all drop their references, the CompletedRequest will be recycled automatically back to the camera system.

libcamera-raw is not so very different from libcamera-vid. It too uses an encoder, although this time it is a "dummy" encoder called the NullEncoder. This just treats the input image directly as the output buffer and is careful not to drop its reference to the input until the output callback has dealt with it first.

This time, however, we do not forward anything to the preview window, though we could have displayed the (processed) video stream if we had wanted.

The use of the NullEncoder is possibly overkill in this application, as we could probably just send the image straight to the Output object. However, it serves to underline the general principle that it is normally a bad idea to do too much work directly in the event loop, and time-consuming processes are often better left to other threads.

We discuss libcamera-jpeg rather than libcamera-still as the basic idea (that of switching the camera from preview into capture mode) is the same, and libcamera-jpeg has far fewer additional options (such as timelapse capture) that serve to distract from the basic function.

libcamera-jpeg starts the camera in preview mode in the usual way, but at the appropriate moment stops it and switches to still capture:

Then the event loop will grab the first frame that emerges once it’s no longer in preview mode, and saves this as a JPEG.

Most of the image processing applied to frames from the sensor is done by the hardware ISP (Image Signal Processor). This processing is governed by a set of control algorithms and these in turn must have a wide range of parameters supplied to them. These parameters are tuned specifically for each sensor and are collected together in a JSON file known as the camera tuning file.

This tuning file can be inspected and edited by users. Using the --tuning-file command line option, users can point the system at completely custom camera tuning files.

libcamera makes it possible to support 3rd party sensors (that is, sensors other than Raspberry Pi’s officially supported sensors) on the Raspberry Pi platform. To accomplish this, a working open source sensor driver must be provided, which the authors are happy to submit to the Linux kernel. There are a couple of extra files need to be added to libcamera which supply device-specific information that is available from the kernel drivers, including the previously discussed camera tuning file.

Raspberry Pi also supplies a tuning tool which automates the generation of the tuning file from a few simple calibration images.

Both these topics are rather beyond the scope of the documentation here, however, full information is available in the Tuning Guide for the Raspberry Pi cameras and libcamera.

Select Preferences and Raspberry Pi Configuration from the desktop menu: a window will appear. Select the Interfaces tab, then click on the enable camera option. Click OK. You will need to reboot for the changes to take effect.

Open the raspi-config tool from the terminal:

Select Interfacing Options then Camera and press Enter. Choose Yes then Ok. Go to Finish and you’ll be prompted to reboot.

To test that the system is installed and working, try the following command:

The display should show a five-second preview from the camera and then take a picture, saved to the file test.jpg, whilst displaying various informational messages.

With a camera module connected and enabled, enter the following command in the terminal to take a picture:

In this example the camera has been positioned upside-down. If the camera is placed in this position, the image must be flipped to appear the right way up.

With the camera placed upside-down, the image must be rotated 180° to be displayed correctly. The way to correct for this is to apply both a vertical and a horizontal flip by passing in the -vf and -hf flags:

Now the photo has been captured correctly.

The camera module takes pictures at a resolution of 2592 x 1944 which is 5,038,848 pixels or 5 megapixels.

A photo taken with the camera module will be around 2.4MB. This is about 425 photos per GB.

Taking 1 photo per minute would take up 1GB in about 7 hours. This is a rate of about 144MB per hour or 3.3GB per day.

You can create a Bash script which takes a picture with the camera. To create a script, open up your editor of choice and write the following example code:

This script will take a picture and name the file with a timestamp.

You’ll also need to make sure the path exists by creating the camera folder:

Say we saved it as camera.sh, we would first make the file executable:

Then run with:

For a full list of possible options, run raspistill with no arguments. To scroll, redirect stderr to stdout and pipe the output to less:

Use the arrow keys to scroll and type q to exit.

With a camera module connected and enabled, record a video using the following command:

Remember to use -hf and -vf to flip the image if required, like with raspistill

This will save a 5 second video file to the path given here as vid.h264 (default length of time).

To specify the length of the video taken, pass in the -t flag with a number of milliseconds. For example:

This will record 10 seconds of video.

For a full list of possible options, run raspivid with no arguments, or pipe this command through less and scroll through:

Use the arrow keys to scroll and type q to exit.

The Raspberry Pi captures video as a raw H264 video stream. Many media players will refuse to play it, or play it at an incorrect speed, unless it is "wrapped" in a suitable container format like MP4. The easiest way to obtain an MP4 file from the raspivid command is using MP4Box.

Install MP4Box with this command:

Capture your raw video with raspivid and wrap it in an MP4 container like this:

Alternatively, wrap MP4 around your existing raspivid output, like this:

Allows the user to define the size of the preview window and its location on the screen. Note this will be superimposed over the top of any other windows/graphics.

Forces the preview window to use the whole screen. Note that the aspect ratio of the incoming image will be retained, so there may be bars on some edges.

Disables the preview window completely. Note that even though the preview is disabled, the camera will still be producing frames, so will be using power.

Sets the opacity of the preview windows. 0 = invisible, 255 = fully opaque.

Sets the sharpness of the image. 0 is the default.

Sets the contrast of the image. 0 is the default.

Sets the brightness of the image. 50 is the default. 0 is black, 100 is white.

Sets the colour saturation of the image. 0 is the default.

Sets the ISO to be used for captures.

In video mode only, turns on video stabilisation.

Sets the EV compensation of the image. Default is 0.

Possible options are:

Note that not all of these settings may be implemented, depending on camera tuning.

Set a mode to compensate for lights flickering at the mains frequency, which can be seen as a dark horizontal band across an image. Flicker avoidance locks the exposure time to a multiple of the mains flicker frequency (8.33ms for 60Hz, or 10ms for 50Hz). This means that images can be noisier as the control algorithm has to increase the gain instead of exposure time should it wish for an intermediate exposure value. auto can be confused by external factors, therefore it is preferable to leave this setting off unless actually required.

Possible options are:

Modes for which colour temperature ranges (K) are available have these settings in brackets.

Note that not all of these settings may be implemented, depending on camera type.

Set an effect to be applied to the image:

Note that not all of these settings may be available in all circumstances.

The supplied U and V parameters (range 0 - 255) are applied to the U and Y channels of the image. For example, --colfx 128:128 should result in a monochrome image.

Specify the metering mode used for the preview and capture:

Sets the rotation of the image in the viewfinder and resulting image. This can take any value from 0 upwards, but due to hardware constraints only 0, 90, 180, and 270 degree rotations are supported.

Flips the preview and saved image horizontally.

Flips the preview and saved image vertically.

Allows the specification of the area of the sensor to be used as the source for the preview and capture. This is defined as x,y for the top-left corner, and a width and height, with all values in normalised coordinates (0.0 - 1.0). So, to set a ROI at halfway across and down the sensor, and a width and height of a quarter of the sensor, use:

Sets the shutter open time to the specified value (in microseconds). Shutter speed limits are as follows:

Using values above these maximums will result in undefined behaviour.

DRC changes the images by increasing the range of dark areas, and decreasing the brighter areas. This can improve the image in low light areas.

By default, DRC is off.

Force recomputation of statistics on stills capture pass. Digital gain and AWB are recomputed based on the actual capture frame statistics, rather than the preceding preview frame.

Sets blue and red gains (as floating point numbers) to be applied when -awb off is set e.g. -awbg 1.5,1.2

Sets the analog gain value directly on the sensor (floating point value from 1.0 to 8.0 for the OV5647 sensor on Camera Module V1, and 1.0 to 12.0 for the IMX219 sensor on Camera Module V2 and the IMX447 on the HQ Camera).

Sets the digital gain value applied by the ISP (floating point value from 1.0 to 64.0, but values over about 4.0 will produce overexposed images)

Sets a specified sensor mode, disabling the automatic selection. Possible values depend on the version of the Camera Module being used:

Version 1.x (OV5647)

Version 2.x (IMX219)

¹For frame rates over 120fps, it is necessary to turn off automatic exposure and gain control using -ex off. Doing so should achieve the higher frame rates, but exposure time and gains will need to be set to fixed values supplied by the user.

HQ Camera

Selects which camera to use on a multi-camera system. Use 0 or 1.

Adds some text and/or metadata to the picture.

Metadata is indicated using a bitmask notation, so add them together to show multiple parameters. For example, 12 will show time(4) and date(8), since 4+8=12.

Text may include date/time placeholders by using the '%' character, as used by strftime.

Specifies annotation size, text colour, and background colour. Colours are in hex YUV format.

Size ranges from 6 - 160; default is 32. Asking for an invalid size should give you the default.

Select the specified stereo imaging mode; sbs selects side-by-side mode, tb selects top/bottom mode; off turns off stereo mode (the default).

Halves the width and height of the stereo image.

Swaps the camera order used in stereoscopic imaging; NOTE: currently not working.

Retrieves the current camera settings and writes them to stdout.

Quality 100 is almost completely uncompressed. 75 is a good all-round value.

This option inserts the raw Bayer data from the camera into the JPEG metadata.

Specifies the output filename. If not specified, no file is saved. If the filename is '-', then all output is sent to stdout.

Makes a file system link under this name to the latest frame.

Outputs debugging/information messages during the program run.

The program will run for the specified length of time, entered in milliseconds. It then takes the capture and saves it if an output is specified. If a timeout value is not specified, then it is set to 5 seconds (-t 5000). Note that low values (less than 500ms, although it can depend on other settings) may not give enough time for the camera to start up and provide enough frames for the automatic algorithms like AWB and AGC to provide accurate results.

If set to 0, the preview will run indefinitely, until stopped with CTRL-C. In this case no capture is made.

The specific value is the time between shots in milliseconds. Note that you should specify %04d at the point in the filename where you want a frame count number to appear. So, for example, the code below will produce a capture every 2 seconds, over a total period of 30s, named image0001.jpg, image0002.jpg and so on, through to image0015.jpg.

Note that the %04d indicates a 4-digit number, with leading zeroes added to make the required number of digits. So, for example, %08d would result in an 8-digit number.

If a time-lapse value of 0 is entered, the application will take pictures as fast as possible. Note that there’s an minimum enforced pause of 30ms between captures to ensure that exposure calculations can be made.

Specifies the first frame number in the timelapse. Useful if you have already saved a number of frames, and want to start again at the next frame.

Instead of a simple frame number, the timelapse file names will use a date/time value of the format aabbccddee, where aa is the month, bb is the day of the month, cc is the hour, dd is the minute, and ee is the second.

Instead of a simple frame number, the timelapse file names will use a single number which is the Unix timestamp, i.e. the seconds since 1970.

Allows specification of the thumbnail image inserted into the JPEG file. If not specified, defaults are a size of 64x48 at quality 35.

if --thumb none is specified, no thumbnail information will be placed in the file. This reduces the file size slightly.

This options cycles through the range of camera options. No capture is taken, and the demo will end at the end of the timeout period, irrespective of whether all the options have been cycled. The time between cycles should be specified as a millisecond value.

Valid options are jpg, bmp, gif, and png. Note that unaccelerated image types (GIF, PNG, BMP) will take much longer to save than jpg, which is hardware accelerated. Also note that the filename suffix is completely ignored when deciding the encoding of a file.

Sets the JPEG restart marker interval to a specific value. Can be useful for lossy transport streams because it allows a broken JPEG file to still be partially displayed.

Allows the insertion of specific EXIF tags into the JPEG image. You can have up to 32 EXIF tag entries. This is useful for tasks like adding GPS metadata. For example, to set the longitude:

would set the longitude to 5 degs, 10 minutes, 15 seconds. See EXIF documentation for more details on the range of tags available; the supported tags are as follows:

Note that a small subset of these tags will be set automatically by the camera system, but will be overridden by any EXIF options on the command line.

Setting --exif none will prevent any EXIF information being stored in the file. This reduces the file size slightly.

Applies real-time EXIF information from any attached GPS dongle (using GSPD) to the image; requires libgps.so to be installed.

This runs the preview window using the full resolution capture mode. Maximum frames per second in this mode is 15fps, and the preview will have the same field of view as the capture. Captures should happen more quickly, as no mode change should be required. This feature is currently under development.

The camera is run for the requested time (-t), and a capture can be initiated throughout that time by pressing the Enter key. Pressing X then Enter will exit the application before the timeout is reached. If the timeout is set to 0, the camera will run indefinitely until the user presses X then Enter. Using the verbose option (-v) will display a prompt asking for user input, otherwise no prompt is displayed.

The camera is run for the requested time (-t), and a capture can be initiated throughout that time by sending a USR1 signal to the camera process. This can be done using the kill command. You can find the camera process ID using the pgrep raspistill command.

kill -USR1 <process id of raspistill>

Sets burst capture mode. This prevents the camera from returning to preview mode in between captures, meaning that captures can be taken closer together.

Width of resulting video. This should be between 64 and 1920.

Height of resulting video. This should be between 64 and 1080.

Use bits per second, so 10Mbps would be -b 10000000. For H264, 1080p30 a high quality bitrate would be 15Mbps or more. Maximum bitrate is 25Mbps (-b 25000000), but much over 17Mbps won’t show noticeable improvement at 1080p30.

Specify the output filename. If not specified, no file is saved. If the filename is '-', then all output is sent to stdout.

To connect to a remote IPv4 host, use tcp or udp followed by the required IP Address. e.g. tcp://192.168.1.2:1234 or udp://192.168.1.2:1234.

To listen on a TCP port (IPv4) and wait for an incoming connection use --listen (-l) option, e.g. raspivid -l -o tcp://0.0.0.0:3333 will bind to all network interfaces, raspivid -l -o tcp://192.168.1.1:3333 will bind to a local IPv4.

When using a network connection as the data sink, this option will make the system wait for a connection from the remote system before sending data.

Outputs debugging/information messages during the program run.

The total length of time that the program will run for. If not specified, the default is 5000ms (5 seconds). If set to 0, the application will run indefinitely until stopped with Ctrl-C.

This options cycles through the range of camera options. No recording is done, and the demo will end at the end of the timeout period, irrespective of whether all the options have been cycled. The time between cycles should be specified as a millisecond value.

At present, the minimum frame rate allowed is 2fps, and the maximum is 30fps. This is likely to change in the future.

Switch on an option to display the preview after compression. This will show any compression artefacts in the preview window. In normal operation, the preview will show the camera output prior to being compressed. This option is not guaranteed to work in future releases.

Sets the intra refresh period (GoP) rate for the recorded video. H264 video uses a complete frame (I-frame) every intra refresh period, from which subsequent frames are based. This option specifies the number of frames between each I-frame. Larger numbers here will reduce the size of the resulting video, and smaller numbers make the stream less error-prone.

Sets the initial quantisation parameter for the stream. Varies from approximately 10 to 40, and will greatly affect the quality of the recording. Higher values reduce quality and decrease file size. Combine this setting with a bitrate of 0 to set a completely variable bitrate.

Sets the H264 profile to be used for the encoding. Options are:

Specifies the H264 encoder level to use for encoding. Options are 4, 4.1, and 4.2.

Sets the H264 intra-refresh type. Possible options are cyclic, adaptive, both, and cyclicrows.

Forces the stream to include PPS and SPS headers on every I-frame. Needed for certain streaming cases e.g. Apple HLS. These headers are small, so don’t greatly increase the file size.

Insert timing information into the SPS block.

This options allows the video capture to be paused and restarted at particular time intervals. Two values are required: the on time and the off time. On time is the amount of time the video is captured, and off time is the amount it is paused. The total time of the recording is defined by the timeout option. Note that the recording may take slightly over the timeout setting depending on the on and off times.

For example:

will record for a period of 25 seconds. The recording will be over a timeframe consisting of 2500ms (2.5s) segments with 5000ms (5s) gaps, repeating over the 20s. So the entire recording will actually be only 10s long, since 4 segments of 2.5s = 10s separated by 5s gaps. So:

2.5 record — 5 pause - 2.5 record — 5 pause - 2.5 record — 5 pause — 2.5 record

gives a total recording period of 25s, but only 10s of actual recorded footage.

On each press of the Enter key, the recording will be paused or restarted. Pressing X then Enter will stop recording and close the application. Note that the timeout value will be used to signal the end of recording, but is only checked after each Enter keypress; so if the system is waiting for a keypress, even if the timeout has expired, it will still wait for the keypress before exiting.

Sending a USR1 signal to the raspivid process will toggle between recording and paused. This can be done using the kill command, as below. You can find the raspivid process ID using pgrep raspivid.

kill -USR1 <process id of raspivid>

Note that the timeout value will be used to indicate the end of recording, but is only checked after each receipt of the SIGUSR1 signal; so if the system is waiting for a signal, even if the timeout has expired, it will still wait for the signal before exiting.

When in a signal or keypress mode, each time recording is restarted, a new file is created.

Select circular buffer mode. All encoded data is stored in a circular buffer until a trigger is activated, then the buffer is saved.

Turns on output of motion vectors from the H264 encoder to the specified file name.

Forces a flush of output data buffers as soon as video data is written. This bypasses any OS caching of written data, and can decrease latency.

Saves timestamp information to the specified file. Useful as an input file to mkvmerge.

Specifies the encoder codec to use. Options are H264 and MJPEG. H264 can encode up to 1080p, whereas MJPEG can encode up to the sensor size, but at decreased framerates due to the higher processing and storage requirements.

Define whether the camera will start paused or will immediately start recording. Options are record or pause. Note that if you are using a simple timeout, and initial is set to pause, no output will be recorded.

Rather than creating a single file, the file is split into segments of approximately the number of milliseconds specified. In order to provide different filenames, you should add %04d or similar at the point in the filename where you want a segment count number to appear e.g:

will produce video clips of approximately 3000ms (3s) long, named video0001.h264, video0002.h264 etc. The clips should be seamless (no frame drops between clips), but the accuracy of each clip length will depend on the intraframe period, as the segments will always start on an I-frame. They will therefore always be equal or longer to the specified period.

The most recent version of Raspivid will also allow the file name to be time-based, rather than using a segment number. For example:

will produce file names formatted like so: video_Fri Jul 20 16:23:48 2018.h264

There are many different formatting options available. Note than the %d and %u options are not available, as they are used for the segment number formatting, and that some combinations may produce invalid file names.

When outputting segments, this is the maximum the segment number can reach before it’s reset to 1, giving the ability to keep recording segments, but overwriting the oldest one. So if set to 4, in the segment example above, the files produced will be video0001.h264, video0002.h264, video0003.h264, and video0004.h264. Once video0004.h264 is recorded, the count will reset to 1, and video0001.h264 will be overwritten.

When outputting segments, this is the initial segment number, giving the ability to resume a previous recording from a given segment. The default value is 1.

Specify the output file name for any raw data files requested.

Specify the raw format to be used if raw output requested. Options as yuv, rgb, and grey. grey simply saves the Y channel of the YUV image.

Many of the options for raspiyuv are the same as those for raspistill. This section shows the differences.

Unsupported options:

Extra options :

This option forces the image to be saved as RGB data with 8 bits per channel, rather than YUV420.

Note that the image buffers saved in raspiyuv are padded to a horizontal size divisible by 32, so there may be unused bytes at the end of each line. Buffers are also padded vertically to be divisible by 16, and in the YUV mode, each plane of Y,U,V is padded in this way.

Only outputs the luma (Y) channel of the YUV image. This is effectively the black and white, or intensity, part of the image.

Saves the image data as BGR data rather than YUV.

By default, captures are done at the highest resolution supported by the sensor. This can be changed using the -w and -h command line options.

Take a default capture after 2s (times are specified in milliseconds) on the viewfinder, saving in image.jpg:

Take a capture at a different resolution:

Reduce the quality considerably to reduce file size:

Force the preview to appear at coordinate 100,100, with width 300 pixels and height 200 pixels:

Disable preview entirely:

Save the image as a PNG file (lossless compression, but slower than JPEG). Note that the filename suffix is ignored when choosing the image encoding:

Add some EXIF information to the JPEG. This sets the Artist tag name to Boris, and the GPS altitude to 123.5m. Note that if setting GPS tags you should set as a minimum GPSLatitude, GPSLatitudeRef, GPSLongitude, GPSLongitudeRef, GPSAltitude, and GPSAltitudeRef:

Set an emboss image effect:

Set the U and V channels of the YUV image to specific values (128:128 produces a greyscale image):

Run preview for 2s, with no saved image:

Take a time-lapse picture, every 10 seconds for 10 minutes (10 minutes = 600000ms), naming the files image_num_001_today.jpg, image_num_002_today.jpg and so on, with the latest picture also available under the name latest.jpg:

Take a picture and send the image data to stdout:

Take a picture and send the image data to a file:

Run the camera forever, taking a picture when Enter is pressed:

Image size and preview settings are the same as for stills capture. Default size for video recording is 1080p (1920x1080).

Record a 5s clip with default settings (1080p30):

Record a 5s clip at a specified bitrate (3.5Mbps):

Record a 5s clip at a specified framerate (5fps):

Encode a 5s camera stream and send the image data to stdout:

Encode a 5s camera stream and send the image data to a file:

The usual output from raspistill is a compressed JPEG file that has passed through all the stages of image processing to produce a high-quality image. However, JPEG, being a lossy format does throw away some information that the user may want.

raspistill has an encoding option that allows you to specify the output format: either jpg, bmp, png or gif. All but jpg are lossless, so no data is thrown away in an effort to improve compression, but do require conversion from the original YUV, and because these formats do not have hardware support they produce images slightly more slowly than JPEG.

e.g.

raspistill --encoding png -o fred.png

Another option is to output completely formatted YUV420 or RGB data using the raspiyuv application.

For some applications, such as astrophotography, having the raw Bayer data direct from the sensor can be useful. This data will need to be post-processed to produce a useful image.

raspistill has a raw option that will cause the Bayer data to be output.

raspistill --raw -o fred.jpg

The raw data is appended to the end of the JPEG file and will need to be extracted.

Both libcamera-still and raspistill have a built in time-lapse mode, using the --timelapse command line switch. The value that follows the switch is the time between shots in milliseconds:

or

A good way to automate taking a picture at a regular interval is using cron. Open the cron table for editing:

This will either ask which editor you would like to use, or open in your default editor. Once you have the file open in an editor, add the following line to schedule taking a picture every minute (referring to the Bash script from the raspistill page, though you can use libcamera-still in exactly the same way):

Save and exit and you should see the message:

Make sure that you use e.g. %04d to ensure that each image is written to a new file: if you don’t, then each new image will overwrite the previous file.

Now you’ll need to stitch the photos together into a video. You can do this on the Raspberry Pi using ffmpeg but the processing will be slow. You may prefer to transfer the image files to your desktop computer or laptop and produce the video there.

First you will need to install ffmpeg if it’s not already installed.

Now you can use the ffmpeg tool to convert your JPEG files into an mp4 video:

On a Raspberry Pi 3, this can encode a little more than two frames per second. The performance of other Raspberry Pi models will vary. The parameters used are:

ffmpeg has a comprehensive parameter set for varying encoding options and other settings. These can be listed using ffmpeg --help.

To stream using the RTP protocol, on the server you could use:

And in the client window:

We conclude with an example that streams from one machine to another. Let us assume that the client machine has the IP address 192.168.0.3. On the server (a Raspberry Pi) the pipeline is identical, but for the destination address:

If the client is not a Raspberry Pi it may have different gstreamer elements available. For a Linux PC we might use:

libcamera provides a libcamerasrc gstreamer element which can be used directly instead of libcamera-vid. On the server you could use:

and on the client we use the same playback pipeline as previously.

The closed source GPU firmware has drivers for Unicam and three camera sensors plus a bridge chip. They are the Raspberry Pi Camera v1.3 (Omnivision OV5647), Raspberry Pi Camera v2.1 (Sony IMX219), Raspberry Pi HQ camera (Sony IMX477), and an unsupported driver for the Toshiba TC358743 HDMI->CSI2 bridge chip.

This driver integrates the source driver, Unicam, ISP, and tuner control into a full camera stack delivering processed output images. It can be used via MMAL, OpenMAX IL and V4L2 using the bcm2835-v4l2 kernel module. Only Raspberry Pi cameras are supported via this interface.

This was an interim option before the V4L2 driver was available. The MMAL component vc.ril.rawcam allows receiving of the raw CSI2 data in the same way as the V4L2 driver, but all source configuration has to be done by userland over whatever interface the source requires. The raspiraw application is available on github. It uses this component and the standard I2C register sets for OV5647, IMX219, and ADV7282M to support streaming.

There is a fully open source kernel driver available for the Unicam block; this is a kernel module called bcm2835-unicam. This interfaces to V4L2 subdevice drivers for the source to deliver the raw frames. This bcm2835-unicam driver controls the sensor, and configures the CSI-2 receiver so that the peripheral will write the raw frames (after Debayer) to SDRAM for V4L2 to deliver to applications. Except for this ability to unpack the CSI-2 Bayer formats to 16bits/pixel, there is no image processing between the image source (e.g. camera sensor) and bcm2835-unicam placing the image data in SDRAM.

Mainline Linux has a range of existing drivers. The Raspberry Pi kernel tree has some additional drivers and device tree overlays to configure them that have all been tested and confirmed to work. They include:

As the subdevice driver is also a kernel driver, with a standardised API, 3rd parties are free to write their own for any source of their choosing.

The sensor driver for a camera sensor is responsible for all configuration of the device, usually via I2C or SPI. Rather than writing a driver from scratch, it is often easier to take an existing driver as a basis and modify it as appropriate.

The IMX219 driver is a good starting point. This driver supports both 8bit and 10bit Bayer readout, so enumerating frame formats and frame sizes is slightly more involved.

Sensors generally support V4L2 user controls. Not all these controls need to be implemented in a driver. The IMX219 driver only implements a small subset, listed below, the implementation of which is handled by the imx219_set_ctrl function.

In the case of the IMX219, many of these controls map directly onto register writes to the sensor itself.

Further guidance can be found in libcamera’s sensor driver requirements, and also in chapter 3 of the Raspberry Pi Camera Tuning Guide.

These are devices that convert an incoming video stream, for example HDMI or composite, into a CSI-2 stream that can be accepted by the Raspberry Pi CSI-2 receiver.

Handling bridge chips is more complicated, as unlike camera sensors they have to respond to the incoming signal and report that to the application.

The mechanisms for handling bridge chips can be broadly split into either analogue or digital.

When using ioctls in the sections below, an S in the ioctl name means it is a set function, whilst G is a get function and _ENUM enumerates a set of permitted values.