PIXCI® Imaging Drivers: Video for Linux Driver (V4L2)

1. PIXCI® Imaging Drivers: Video for Linux Driver (V4L2)

The Video for Linux Driver (V4L2) for PIXCI® frame grabbers provides a Linux standard V4L2 API to control PIXCI® frame grabbers and retrieve video and image data.

The PIXCI® V4L2 driver allows simple integration of PIXCI® frame grabbers into GStreamer, Matlab, OpenCV, and other programming environments supporting V4L2; allows use of ‘vlc’ and other third party image/video display applications; and provides an alternative API for creating custom applications incorporating PIXCI® frame grabbers.

Installation

The PIXCI® V4L2 driver is distributed with the EPIX® XCAP imaging application. The XCAP application does not require use of the V4L2 driver, and does not automatically install it.

The PIXCI® V4L2 works in conjunction with the PIXCI® device driver; the PIXCI® device driver must be installed prior to the PIXCI® V4L2 driver. Both can be installed and/or compiled, as necessary, by using XCAP’s Driver Assistant; click XCAP’s PIXCI®, PIXCI® Open/Close, Close (if open), Driver Assistant.

Alternatively, the PIXCI® device driver and/or the PIXCI® V4L2 driver can be installed and/or compiled without running the XCAP application. Browse XCAP’s drivers subdirectory (i.e. /usr/local/xcap/drivers, assuming the default installation directory) for the necessary shell scripts, make files, object files, source files, etc.

Driver Configuration

The PIXCI® V4L2 driver’s parameters allows choosing V4L2’s memory mapped streaming mode versus V4L2’s non-stream read modes, allows selecting the number of V4L2 streaming buffers, allows choosing ‘snap’ versus ‘live’ non-stream read modes, allows selecting the V4L2 /dev/video? index or accept default, allows retaining the field counter prepended to image data, allows forcing image data to be labeled as 8 bit monochrome, and allows selecting verbose messages via the kernel log.

The XCAP application provides a dialog to view and change the commonly used PIXCI® V4L2 driver configuration parameters; see XCAP’s Driver Assistant.

The PIXCI® V4L2 driver’s configuration parameters can be changed without running the XCAP application; the parameters are described and specified in file /etc/default/pixciv4l.

The default driver parameters suffice for most applications. The popular exception is disabling V4L2 streaming mode for use with a digital camera in async reset (i.e. triggered) mode, thereby reducing latency between ‘read’() and the actual arming (i.e. ‘Snap’) of the frame grabber, and also providing optional triggering of the camera via the ‘read’() (as selected by the Video Configuration, below).

Video Configuration

The PIXCI® V4L2 driver’s video setup must be initialized before using the V4L2 API; the video setup is lost after system reboot or re-installation of the PIXCI® device driver.

The video setup of the PIXCI® V4L2 driver can be initialized via the pixcinit utility program. The video setup can be specified by video format name, or via a video setup file previously exported by the XCAP application, such as:

    pixcinit -formatfile videosetup.fmt
    pixcinit -formatname NTSC

or abbreviated as:

    pixcinit -ff videosetup.fmt
    pixcinit -fn NTSC

Or further abbreviated as:

    pixcinit videosetup.fmt
    pixcinit NTSC

assuming that the video setup file’s name doesn’t matches a video format name (or, avoid confusion by entering ‘./NTSC’, ‘/usr/home/wombat/NTSC’, etc.). Quotes are required for format names with embedded spaces:

    pixcinit -fn "Video 720x480i 60Hz"
    pixcinit "Video 720x480i 60Hz"

The pixcinit program can also be used to:

a. List the allowable video format names for each model of frame grabber:

    pixcinit -formatnames

b. Provide text for error codes that might be reported in the kernel log (i.e. via ‘dmesg’):

    pixcinit -error ##

c. Provide text for fault codes that might be reported in the kernel log:

    pixcinit -fault ##

Use

    pixcinit --help

for additional usage instructions.

The pixcinit utility program is a non-GUI, non-interactive, application, and can be run on ‘headless’ systems. The pixcinit is distributed with the PIXCI® V4L2 driver.

The PIXCI® V4L2 driver’s video setup can also be initialized by running the XCAP application or an application built around the XCLIB SDK/API.

Controls

For analog cameras, PIXCI® frame grabbers provide video processing adjustments within the frame grabber. The PIXCI® V4L2 driver provides V4L2 controls for:

A. PIXCI® SV4, SV5: Brightness, Hue, Luma Gain, Chroma U Gain, and Chroma V Gain.

B. PIXCI® SV7: Contrast, Brightness, Hue, Saturation and AGC.

C. PIXCI® SV8: Contrast, Brightness, Hue, Luma Gain, Chroma Gain, Luma AGC, and Chroma AGC.

D. PIXCI® A110: Digital Gain, Analog Gain, Digital Offset, and Analog Offset.

E. PIXCI® A310: Red, Green, and Blue variations of Digital Gain, Analog Gain, Digital Offset, and Analog Offset.

For digital cameras, video processing is performed within the camera, not within the PIXCI® frame grabber. Typical cameras may provide adjustments via RS−232 serial, or via Camera Link serial (i.e. similar to RS−232, but integrated within the Camera Link cable). Serial commands are not standardized; they differ from one camera manufacturer to another, or between cameras of the same manufacturer. Typically, the camera manufacturer provides a configuration application, or instructions on using a ‘terminal’ program (i.e. ‘cutecom’, ‘minicom’, XCAP’s Serial Terminal, etc.). In the case of Camera Link serial, the PIXCI® ‘tty’ driver provides access via /dev/ttySPIXCI?.

Additional V4L2 controls are provided for:

a. The PIXCI® frame grabber’s General Purpose Input and General Purpose Output signals (i.e. ‘Parallel I/O’) providing sensing and control without any direct effect upon video.

b. Selection of the PIXCI® V4L2 driver’s ‘Non-Streaming Read Mode’; modification of this mode takes effect after closing and re-opening the PIXCI® V4L2 driver.

For use in custom applications, numeric ID’s for controls can be discovered by probing user-class control ID’s via the V4L2 API.

Image and Pixel Format

The PIXCI® V4L2 driver complies with the Linux V4L2 design guideline: ‘performing format conversions within the kernel is explicitly frowned upon’; the pixel data received from the frame grabber is the data provided to the application.

Many image and pixel formats produced by PIXCI® frame grabbers match V4L2 standard formats, such as Monochrome (i.e. Grey), RGB, RGB+pad, BGR, BGR+pad, CbYCrY (or variations), or Bayer format. These are described via a standard V4L2 ‘pixelformat’ FourCC code.

Typical exceptions are: (a) A digital camera that doesn’t output pixels in the normal left-to-right and top-to-bottom order,[1] even after allowances for top-bottom and left-right flip, (b) A high bandwidth camera requiring bit packing of pixel data[2], (c) A ‘Field Counter’ has been prepended to the image data (but, see below for option to remove the field counter). These image and pixel formats are described via a custom FourCC code; see pixciv4l.c and its ‘pixelhintToFourCC()’ function for a description of the custom FourCC codes.

In practice, custom applications using one of the unusual cameras with out-of-order pixels or requiring bit-packing will typically ignore the FourCC code and instead ‘hard-code’ knowledge of the camera’s requirements and data format. Further, the PIXCI® V4L2 driver provides an option to force labeling of the image data bytes as 8 bit monochrome so that third party applications and programming environments won’t object to the nonstandard FourCC code (the option does not modify the image data).

Selected PIXCI® frame grabbers count incoming video fields, whether intended to be captured or not, and prepend the count to the image data; this is intended to assist checking whether any video fields have been skipped.[3] However, the presence of the field counter prevents an otherwise standard pixel and image format from complying with the V4L2 standard format. The PIXCI® V4L2 driver provides an option to remove the field counter from the pixel data. Use of this option incurs no additional overhead on newer Linux kernels, and minimal overhead on older kernels.

Configuration via XCAP

The initial configuration of PIXCI® frame grabbers for analog cameras, including contrast, brightness, gain, pixel format, etc.; and the initial configuration of many digital cameras, can be done by using the XCAP application.

Using XCAP, configure the frame grabber and/or camera as desired. Use PIXCI®, PIXCI® Export Video to export the video setup. Use pixcinit, as described above, to load the video setup (i.e. video format) file.

Streaming Capture Mode

In V4L2 streaming mode, the PIXCI® V4L2 driver uses frame buffers provided by the Linux V4L2 core rather than frame buffers allocated by the PIXCI® driver. The number of frame buffers, and thus the ‘depth’ of the capture queue is determined by a PIXCI® V4L2 driver parameter (a default value), and/or by the application (negotiated via the V4L2 API).

To conserve memory, such as when intending to use a large number of frame buffers and/or using a very high resolution camera: Reduce the PIXCI® device driver’s frame buffer memory to be sufficient for one frame buffer; that frame buffer isn’t used during V4L2 capture, but is required for correct configuration and operation of the frame grabber.

In V4L2 streaming modes, the Linux V4L2 core implements read() as a minimal, one frame, stream requiring overhead to set up each stream. The Non-Stream Snap Capture mode (below) provides less overhead between an application’s read() and the actual arming of frame grabber, and is recommended for applications requiring precise timing of frame capture.

Non-Stream Capture Modes

In V4L2 non-streaming mode, the PIXCI® V4L2 driver offers three submodes.

Read Snap Mode: In PIXCI® frame grabber terminology, each read() corresponds to a video ‘snap’; the next full frame following the snap is captured. In V4L2 terminology, all frames preceding the read() are discarded. In read()’s non-blocking mode, at least two read()’s are required, one read() to issue the snap (or, temporarily cease discarding in V4L2 terminology), a later read() to return pixel data.

Assuming the camera is in free-run (i.e. not async trigger) mode: the latency from read() to receipt of image data is one to two frame periods; zero to one frame period to wait for the camera’s next top-of-frame, one frame period to capture the frame.[4]

When using a camera in async trigger mode, with the video configuration specifying triggering via button or function call, the ‘snap’ also triggers the camera.

The Read Snap Mode requires one frame buffer, located in the PIXCI® device driver’s frame buffer memory.

Read Live Newest Mode: In PIXCI® frame grabber terminology, the PIXCI® continuously captures frames into the PIXCI® device driver’s frame buffer memory. A read() returns the newest unread frame; if the newest frame has been previously read, the next captured frame is read. In V4L2 terminology, a two buffer capture queue is used, with old unread frames discarded.

The latency from read() to receipt of image data is zero if the newest captured frame has not been read; otherwise the latency is zero to one frame period.

The Read Live Newest Mode requires three frame buffers, located in the PIXCI® device driver’s frame buffer memory.

Read Live Oldest Mode: In PIXCI® frame grabber terminology, the PIXCI® continuously captures frames into a circular queue in the PIXCI® device driver’s frame buffer memory. A read() returns the oldest unread frame; if all frames have been read, the next captured frame is read. In V4L2 terminology, a N buffer capture queue is used with with old unread frames discarded.

The latency from read() to receipt of image data is zero if the queue has an unread frame; otherwise the latency is zero to one frame period.

As for other software using the PIXCI® device driver’s frame buffer memory, the queue size depends on the size of the frame buffer memory and the memory required for each image; a minimum of three frame buffers is required.

For most cameras, the Read Live Oldest Mode allows applications to implement video to disk, and similar tasks that don’t allow dropping frames, while using the simpler V4L2 read() instead of the more complicated V4L2 streaming modes. By specifying sufficient frame buffer memory and frame buffers, the PIXCI® device driver can queue images - independently of V4L2 - until the images are read, thereby allowing for short delays due to HDD/SSD I/O overhead in the application’s process, or delays due to the application’s pausing while other applications execute.

Dma-Buf Streaming Capture Mode

As a special case of V4L2 streaming mode, the PIXCI® V4L2 driver supports V4L2’s ‘Dma-Buf(fer)’ streaming mode. The frame buffer(s) provided by the Linux V4L2 core can be exported to a Dma-Buf aware application or to a different driver. Or, frame buffers exported by a Dma-Buf aware application or by a different driver can be used by the PIXCI® V4L2 driver in lieu of frame buffers provided by the Linux V4L2 core.

A common use of Dma-Buf mode is to share DMA buffers with a GPU, and thus avoid copying of pixel data from the PIXCI® V4L2 driver’s DMA buffers to GPU memory.

Application Software

The PIXCI® V4L2 driver can be used with most common, third party, ‘video camera’ applications supporting V4L2, such as ‘cheese’ or ‘vlc’.

Most third party applications support the image and pixel format of PIXCI® frame grabbers for analog cameras in 8 bit depth mode. Third party applications might object to the atypical, larger than usual, resolutions offered by the PIXCI® A110 and A310. Third party applications might not support 10 bit depth modes.

Most third party applications should support the image and pixel format of PIXCI® frame grabbers for digital cameras in 8 bit monochrome mode; but may not support 10/12/14/16 bit depths, or Bayer format, or atypical resolutions. Third party applications generally don’t support image and pixel formats involving ‘multi-tap correction’ or ‘bit-packing’.

Many third party applications only support V4L2 streaming mode; the ‘vlc’ application supports both V4L2 streaming and non-streaming modes.

The open-source ‘v4l2exam.c’, provided at http://linuxtv.org/docs.php, is an excellent starting point for writing custom applications intending to incorporate PIXCI® frame grabbers via the PIXCI® V4L2 driver.

Compatibility

The PIXCI® V4L2 driver supports current PIXCI® frame grabbers.

The PIXCI® V4L2 driver does not support frame grabbers which were no longer in production as of the PIXCI® V4L2 driver’s release, i.e. PIXCI® A, CL3SD, D, D24, D32, SV2, and SV3 frame grabbers. Frame grabbers using on-board frame buffer memory are only supported in V4L2 non-streaming mode, i.e. PIXCI® SI2.

On 64 bit machines, older versions of the Linux V4L2 ‘core’, may not support use of 32 bit frame grabbers in V4L2 streaming mode, i.e. PIXCI® CL1, CL2, D2X, D3X, SI, SV4, and SV5.

Linux kernels earlier than 3.8.0 are not supported.

Tech Support

For general questions regarding the Linux Video for Linux (V4L2) API and how to create applications using the API, consult one of the Linux programming forums or the ‘LinuxTVWiki’, search for relevant topics via Google or other search engines, and/or consult the Linux source code.

For questions specifically related to PIXCI® frame grabbers and the PIXCI® V4L2 driver, email v4l2@epixinc.com .

Updated: 27 March 2023

Footnotes __________

1. Referred to as ‘Multi-Tap’ (corrections) in the XCAP application.

2. For example, packing four 10 bit values in five instead of eight
bytes, or packing two 12 bit values in three instead of four bytes.

3. Typically, frame grabbers supporting analog video or interlaced video
do not provide the field counter.

4. We describe the latency due to video timing of a camera in free-run
mode, with capture always synchronized to the start of a video frame.
It is not intended to include the common, much smaller, overhead due
to process and thread scheduling (of the application program), memory
swapping or paging (of the application program), and the overhead of
copying image data from kernel space to the application program’s mem-
ory space.