PIXCI® Imaging Drivers: Media Foundation Frame Server (Windows 'Camera' Device & DirectShow Source)

1. PIXCI® Imaging Drivers: Media Foundation Frame Server (Windows ’Camera’ Device & DirectShow Source)

The Media Foundation Frame Server driver for PIXCI® frame grabbers provides a Windows Media Foundation, Media Source driver to control PIXCI® frame grabbers and retrieve video and image data; i.e. image ‘samples’ in Media Foundation (MF) terminology. It is identified in the Windows’ Device Manager and other applications as ‘PIXCI® Frame Source’.

The PIXCI® Frame Source driver allows simple integration of PIXCI® frame grabbers into GStreamer, LabView, Matlab, Python, OpenCV, and other application environments supporting Media Foundation[1] or DirectShow API’s; allows use of the Windows ‘Camera’ and other third party image/video display applications; and provides an alternative to the XCLIB API for creating custom applications incorporating PIXCI® frame grabbers.

Installation

The PIXCI® Frame Source driver is distributed with the EPIX® XCAP imaging application. The XCAP application does not require use of the Frame Source driver, and does not automatically install it.

The PIXCI® Frame Source driver works in conjunction with the PIXCI® device driver. The PIXCI® device driver is the same as is used, and required, by all applications using the PIXCI® frame grabber. Both can be installed by using XCAP’s Driver Assistant; click XCAP’s PIXCI®, PIXCI® Open/Close, Close (if open), Driver Assistant.

Multiple Frame Grabbers

The

PIXCI® Frame Source

identified in the Windows’ Device Manager and other applications, refers to the default PIXCI® frame grabber or unit[2]. Or, if the default consists of multiple frame grabbers, then to the first frame grabber of the default set. The default is the frame grabber selected by the PIXCI® frame grabber driver’s configuration parameters, as might be set via XCAP’s Driver Assistant.

The

PIXCI® Frame Source 0
PIXCI® Frame Source 1
    ...

identified in the Windows’ Device Manager and other applications, refers to the indicated PIXCI® frame grabber or unit.

Driver Configuration

Configuration options for the PIXCI® Frame Source are in the Windows registry, under HKEY_LOCAL_MACHINE and SOFTWARE\EPIX\PIXCI\FrameServer. These can be set using XCAP’s Driver Assistant, or by directly editing the registry. In (rough) order of importance:

• VideoFormatFile. Path of a video format (.fmt) file previously exported by XCAP, used to configure the PIXCI® frame grabber and/or camera. String value. Default "" (ignored).

• VideoFormatName. Name of a video format, used to configure the PIXCI® frame grabber and camera. See XCAP’s Driver Assistant for legitimate, frame grabber specific, names. String value. Default "" (ignored).

• ForceBitsPerPixie. Override, and force conversion of, the number of bits per pixel component value in image ‘samples’ delivered to the Windows Frame Server. DWORD value. Values 0 (ignored), 8, 10, 12, or 16. Default 8.

• ForceRGB. Override, and force conversion of, the pixel color space in image ‘samples’ delivered to the Windows Frame Server. DWORD value. Values 0 (ignored), 3 for RGB, or 4 for RGB+Pad. Default 4.

• ForceSampleRate. Override the rate at which the Windows Frame Server is to ‘pull’ images from the PIXCI® Frame Source. A string representing an integer (e.g. ‘30’), real (e.g. ‘29.97’), or floating-point (e.g. ‘2997E-2’) value (as the Windows registry lacks native support for floating point values). String value, frames per second. Default "" (ignored).

• ForceMaxWidth. Override the image’s horizontal dimension. The ‘sample’ width, in pixel columns, is the smaller of the captured image width and ForceMaxWidth. The center of the captured image is used. See discussion below. DWORD value. Default 0 (ignored).

• ForceMaxHeight. Override the image’s vertical dimension. The ‘sample’ height, in pixel lines, is the smaller of the captured image height and ForceMaxHeight. The center of the captured image is used. See discussion below. DWORD value. Default 0 (ignored).

• ForceMinWidth. Override the image’s horizontal dimension. The ‘sample’ width, in pixel columns, is the larger of the captured image width and ForceMinWidth. The captured image is padded with black pixels on left and right. See discussion below. DWORD value. Default 0 (ignored).

• ForceMinHeight. Override the image’s vertical dimension. The ‘sample’ height, in pixel lines, is the larger of the captured image height and ForceMinHeight. The captured image is padded with black pixels on top and bottom. See discussion below. DWORD value. Default 0 (ignored).

• RightJustifyConversion. Left justify conversion of a pixel value from N to M bits, with N less than M, is done by right-shifting the value by (M-N) bits. Right justify conversion does not shift bits. DWORD value. Default 0 (ignored), or 1 for right justify conversion.

• SampleMode. One of:

1: Sample Snap Mode.
2: Sample Live Newest Mode.
3: Sample Live Oldest Mode.

See discussion below. DWORD value. Default 2.

• PropertyFlags. Options related to handling of controls; i.e. ‘properties’. See discussion below. DWORD value. Default 0.

• DriverConfigurationParms. Driver configuration parameters, described in the XCAP Reference Manual. Rarely used; setting the driver configuration parameters in the PIXCI® device driver is more common; also, some driver configuration parameters can only be set via the PIXCI® device driver. String value. Default "" (ignored).

• LogFile. Path of the PIXCI® Frame Source log file; the directories must pre-exist, the file is created and/or appended. Logs configuration parameters, video parameters, and errors. The log file is NOT cleared with each use of the PIXCI® Frame Source, and will continue to increase in size. String value. Default "" (ignored).

For drivers identified as

PIXCI® Frame Source 0
PIXCI® Frame Source 1
    ...

these configuration options, except LogFile, can be overridden by similar named values under

SOFTWARE\EPIX\PIXCI\FrameServer\00
SOFTWARE\EPIX\PIXCI\FrameServer\01
    ...

respectively. Sharing a LogFile among multiple frame grabbers or units is not recommended; the LogFile option, if to be used, is not inherited, but specified individually for:

PIXCI® Frame Source 0
PIXCI® Frame Source 1
    ...

under

SOFTWARE\EPIX\PIXCI\FrameServer\00
SOFTWARE\EPIX\PIXCI\FrameServer\01
    ...

respectively.

Controls (Properties)

For analog cameras, PIXCI® frame grabbers provide video processing adjustments within the frame grabber. For PIXCI® SV4, SV5, SV7, and SV8, the PIXCI® Frame Source supports Media Foundation properties:

KSPROPERTY_VIDEOPROCAMP_BRIGHTNESS
KSPROPERTY_VIDEOPROCAMP_CONTRAST
KSPROPERTY_VIDEOPROCAMP_HUE
KSPROPERTY_VIDEOPROCAMP_SATURATION

See Microsoft documentation for more information on these properties and property names.

If bit 0 of PropertyFlags is set, handling of properties is disabled. This can be used to prevent adjustments by the MF application; such as if the settings of the VideoFormatFile are to be frozen, or if a separate application is to handle the adjustments.

Time Stamps

The Windows Media Foundation Frame Server requires that each ‘sample’ (i.e. image) be time stamped. This is typically done by stamping each image as it is supplied to the MF frame server, using the system time supplied by the MF frame server, and is sufficient for displaying live video.

Custom applications using the Media Foundation API’s might prefer more accurate image time stamps. If the PIXCI® device driver is configured to use:

Time Stamping w. Performance Counter

and the:

Stamp per-frame Buffer Status

isn’t disabled (see XCAP’s Driver Assistant, Advanced Options), the time stamps supplied with each image are instead derived from the PIXCI® device driver’s capture time stamps. The capture time is stamped during the end-of-capture hardware interrupt processing and thus has lower latency and better accuracy than time stamping from within the MF frame server.

Sample Rate

The Windows Media Foundation Frame Server uses a ‘pull model’; it must be told the rate at which to pull new images from the ‘PIXCI® Frame Source’. Too low of a rate and the video display will be slower than expected; too fast of a rate results in unnecessary overhead.

The sample rate supplied to the Media Foundation Frame Server is derived from the PIXCI® video setup, specified either via the VideoFormatFile or VideoFormatName registry values. For analog frame grabbers and cameras, the rate is fixed, known, and the values correct. For many digital cameras the correct value may not be known; as the camera might be re-configured via switches or serial commands, or periodically triggered by an external signal.

If not zero, the ForceSampleRate registry value overrides any information in the PIXCI® video setup, and specifies the frame rate at which the Media Foundation Frame Server is asked to ‘pull’ images. It does not affect the configuration of the PIXCI® frame grabber or camera.

Sample Pixel Format

If ForceRGB and ForceBitsPerPixie are 0, the bit depth and color space of pixel data is that of the frame grabber’s and/or camera’s ‘native’ format - modified, as necessary, if MF doesn’t support that bit depth and color space.

For PIXCI® SV4 and SV5 frame grabbers, the native pixel formats are monochrome, YUY2 (i.e. YUYV), UYVY, RGB, or RGB+Pad, as selected in the video setup. For PIXCI® SV7 and SV8 frame grabbers, the native pixel formats are monochrome, YUY2 (i.e. YUYV), or UYVY as selected in the video setup. For the PIXCI® A110 frame grabber, the native pixel format is monochrome. For the PIXCI® A310 frame grabber, the native pixel format is RGB. For PIXCI® D2X, D3X, D3XE, CL1, E1 and other Camera Link frame grabbers, the native pixel format is typically monochrome, RGB, or RGB+Pad, as selected in the video setup; but might be YUY2 (i.e. YUYV), UYVY or other variant as determined by a specific camera.

Applications using Media Foundation may not support all pixel formats; for example, the ‘Camera’ application included with Windows 10 (1909) does not appear to support monochrome data, nor more than 8 bits per pixel component.

In the interest of maximum compatibility with various MF applications, the ForceRGB and ForceBitsPerPixie options default to RGB+Pad, 8 bits per component.

Use of ForceRGB and/or ForceBitsPerPixie does not affect the configuration of the PIXCI® frame grabber or camera.

Sample Modes

Sample Snap Mode: In PIXCI® frame grabber terminology, each request by MF for a ‘sample’ (i.e. image) corresponds to a video ‘snap’; the next full frame following the snap is captured.

Assuming the camera is in free-run (i.e. not async trigger) mode: the latency from ‘sample’ request to receipt of image data is one to two frame periods; zero to one frame period to wait for the camera’s next top-of-frame, one frame period to capture the frame.[3]

When using a camera in async trigger mode, with the video configuration specifying triggering via button or function call, the ‘snap’ also triggers the camera.

The Sample Snap Mode requires one frame buffer, located in the PIXCI® device driver’s frame buffer memory.

Sample Live Newest Mode: In PIXCI® frame grabber terminology, the PIXCI® continuously captures frames into the PIXCI® device driver’s frame buffer memory. Each request by MF for a ‘sample’ (i.e. image) returns the newest unread frame; if the newest frame has been previously returned as a sample, the next captured frame is returned.

The latency from ‘sample’ request to receipt of image data is zero if the newest captured frame has not been read; otherwise the latency is zero to one frame period.

The Sample Live Newest Mode requires three frame buffers, located in the PIXCI® device driver’s frame buffer memory.

Sample Live Oldest Mode: In PIXCI® frame grabber terminology, the PIXCI® continuously captures frames into a circular queue in the PIXCI® device driver’s frame buffer memory. Each request by MF for a ‘sample’ (i.e. image) returns the oldest unread frame; if all frames have been returned as a sample, the next captured frame is returned.

The latency from ‘sample’ request to receipt of image data is zero if the queue has an unread frame; otherwise the latency is zero to one frame period.

As for other software using the PIXCI® device driver’s frame buffer memory, the queue size depends on the size of the frame buffer memory and the memory required for each image; a minimum of three frame buffers is required.

The Sample Live Oldest Mode allows applications to implement video to disk, and similar tasks that don’t allow dropping frames. By specifying sufficient frame buffer memory and frame buffers, the PIXCI® device driver can queue images - independently of the PIXCI® Frame Server - until the images are read, thereby allowing for short delays in HDD/SSD I/O or pauses in the application’s process or thread execution.

Sample Dimensions

The MF ‘sample’ dimensions, in pixel lines and columns, are normally the same as the PIXCI® frame grabber’s image dimensions.

The ForceMaxWidth and ForceMaxHeight allow using smaller dimensions for the MF sample; the center portion of the image is used. The ForceMinWidth and ForceMinHeight allow using larger dimensions for the the MF sample; the image is centered and edges padded with black pixels. Using the same value for ForceMaxWidth and ForceMinWidth, or using the same value for ForceMaxHeight and ForceMinHeight, forces the specified width or height, respectively,

Use of these options do not cause resizing (resampling) of the image data. Use of these options does not affect the configuration of the PIXCI® frame grabber or camera.

Diagnosing Problems

The LogFile enables echoing of parameters found in the registry, and echoing of applicable parameters from the video format.

Unlike the XCAP application and most XCLIB applications, faults preventing opening of the PIXCI® frame grabber and run-time faults can’t be displayed by a Media Foundation Frame Server via a pop-up dialog; faults are instead noted in the log file.

Common problems, reported via the log (specific text may vary):

• Video format name invalid. Self explanatory.

• Video format file not found or invalid. Self explanatory.

• No response to camera’s serial command. Camera not powered on or connected, mismatch of camera and/or wrong video format file.

Other common issues:

• Displayed image is not recognizable. Force RGB+Pad 8/32 bits with ForceRGB=4, and ForceBitsPerPixie=8 for better compatibility with many Media Foundation applications. Or, an incorrect video format name or video format file was specified for current camera.

• Displayed image is wrong aspect ratio. Some cameras and frame grabbers create non-square pixels. The MF Frame Server is notified of the aspect ratio, but the application may not support non-square pixels. Use a different application for displaying images and video.

• Display rate too slow. Set ForceSampleRate to the desired display rate.

• Displayed image is diagonally ‘torn’. The application does not support the video’s horizontal pixel dimension. For example, the ‘Camera’ application included with Windows 10 (1909) works correctly with ‘common’ values of 128, 512, 640, 720, 1024, 1536, or 1792 pixels per line but not with 1020 or 2040 pixels per line.[4] Use a different application for displaying images and video. Or, configure the frame grabber and/or camera to different image dimensions. Or, use ForceMinWidth, ForceMinHeight, ForceMaxWidth, and/or ForceMaxHeight, to force the ‘sample’ dimensions to acceptable values.

• Superfluous messages in log. The PIXCI® Frame Source driver is automatically started as Windows boots and/or as the driver is installed post-boot. For cameras using serial commands and a VideoFormatFile containing serial commands to be sent to the camera, an error will be logged if the camera is not connected or not powered on during boot. Such a message need not be treated as an error; but is deserving of this explanation.

Compatibility

The PIXCI® Frame Source driver supports current PIXCI® frame grabbers.

The PIXCI® Frame Source driver does not support frame grabbers which were no longer in production as of the PIXCI® Frame Source driver’s release, i.e. PIXCI® A, CL3SD, D, D24, D32, SV2, and SV3 frame grabbers.

The Windows Media Foundation Frame Server is supported in Windows 10 (1809) or later.

Tech Support

For general questions regarding the Windows Media Foundation and DirectShow API’s, and how to create applications using these API’s, consult one of the Windows programming forums, or search for relevant books and articles via Google or other search engines. Also see Microsoft’s ‘MFCaptureD3D’ and ‘MFCaptureToFile’ SDK examples.

For questions specifically related to PIXCI® frame grabbers and the PIXCI® Frame Source driver, email directshow@epixinc.com .

Updated: 27 March 2023

Footnotes __________

1. The Windows Media Foundation (MF) is the successor to Windows Direct-
Show.

2. Selected PIXCI® frame grabber boards have multiple, independent func-
tional ‘units’; each is handled as a unique PIXCI® frame grabber.

3. We describe the latency due to video timing of a camera in free-run
mode, with capture always synchronized to the start of a video frame.
It is not intended to include the common, much smaller, overhead due
to process and thread scheduling (of the application program), delays
internal to the Media Foundation frame server, and the overhead of
copying image data.

4. This isn’t intended to be a complete list of resolutions which are, or
aren’t, supported by the Windows’ Camera application.