TM-4 Firewire Video Interface


Table of Contents

    Introduction
    Files
    Ports Package Setup
    Quartus Project Settings
    PLL MegaFunction Setup
    Interface Details
    Hardware Interface
    Software API Reference

Introduction

    The Firewire video interface is a collection of VHDL modules and C source files that allows a user circuit on the TM-4 to capture streaming video from one or two Firewire video cameras.

    This initial release of the interface assumes the following:

    It's possible to reconfigure the source files to change these, but resolutions above 640x480 have not been yet tested.

    Two cameras do not have to be synchronized in order to capture video from both cameras, but if they were, the interface should spit out matched pixels on the same clock cycle. This functionality has not yet been tested.

    Finally, the support for both the Firewire protocol and the IIDC Digital Camera protocol are rudimentary and are implemented just enough to capture video. Many features deemed mandatory by both protocols are absent since they are not relied upon by this very specific task of capturing video.

Files

    Download this .zip file and extract it into your project directory. It should contain these files:

    dm_fifo.vhd
    firecam.c
    firecam.h
    firevid.vhd
    firevid_decoder_yuv411.vhd
    firewire.h
    llc_iface.c
    llc_iface.h
    llc_iface.vhd
    llc_pll.vhd
    yuv_to_rgb.vhd
    camctrl.c
    elemctrl.c
    makefile
    firevid.ports

Ports Package Setup

    This interface uses the TM4 Ports Package to allow the software API to communicate with the hardware connected to the Firewire chipset. There are four signals involved: signals_in, signals_out, sigin_wanted, and sigin_ready, which must be connected to the 'firevid' module.

    You must either create an instance of tm4_portmux, or modify an existing one to add these signals. The firevid.ports file contains the definitions for these signals (copy/paste the entries into an existing .ports file for FPGA1 if you are already using a tm4_portmux on that chip).

Quartus Project Settings

    There is one setting that must be added to the QSF file for the project for FPGA1:

    set_instance_assignment -name INCREASE_DELAY_TO_OUTPUT_PIN ON -to tm4_fir_bclkA

    Replace bclkA with bclkB (or include both) depending on which of the two Firewire interfaces on FPGA1 are being used.

PLL MegaFunction Setup

    This circuit uses a PLL to generate a clock for the Firewire chip outside the FPGA, and it must be configured to know the TM4 global clock frequency that will be fed to it.

    Use the Quartus II MegaWizard Plugin Manager (from the Tools menu) to modify the "llc_pll.vhd" file and change the input clock frequency to the correct value.

Interface Details

    There is a hardware and a software component to the interface. The hardware side consists of a single entity, "firevid", that sits between the Firewire signals external to the TM4 and the user circuit. The signals exposed to the user circuit provide row/column/RGB data along with a few control signals.

    The software component interfaces with the hardware using the TM-4 Ports Package, and is used to control the Firewire camera(s) over the Firewire bus. All this functionality has been abstracted away into a small API written in C that allows the user to change the camera's video mode, start/stop streaming, and adjust camera settings like zoom and focus.

Hardware Interface

    The user circuit on fpga1 must instantiate the entity "firevid" included with this package. Here is the VHDL component declaration along with a description of all the signals:

    entity firevid is
    port
    (
      tm4_glbclk0 : in std_logic;
      resetn : in std_logic;

      -- Connect to one of the TM4's Firewire chipsets
      tm4_fir_ma : out std_logic_vector(0 to 6);
      tm4_fir_md : inout std_logic_vector(0 to 15);
      tm4_fir_mwrn : out std_logic;
      tm4_fir_mcsn : out std_logic;
      tm4_fir_mcan : in std_logic;
      tm4_fir_tean : in std_logic;
      tm4_fir_coldfire : out std_logic;
      tm4_fir_lendian : out std_logic;
      tm4_fir_m8bit : out std_logic;
      tm4_fir_mcmode : out std_logic;
      tm4_fir_bclk : out std_logic;
      tm4_fir_resetn : out std_logic;
      tm4_fir_dmclk : in std_logic;
      tm4_fir_dmd : inout std_logic_vector(0 to 15);
      tm4_fir_dmrw : in std_logic;
      tm4_fir_cystart : in std_logic;
      tm4_fir_pktflag : in std_logic;
      tm4_fir_dmpre : in std_logic;
      tm4_fir_dmdone : in std_logic;
      tm4_fir_dmerror : in std_logic;
      tm4_fir_dmready : out std_logic;

      -- Connect to instance of tm4_portmux (with proper entries added
      -- in the .ports file for FPGA1
      signals_in : in std_logic_vector(31 downto 0);
      signals_out : out std_logic_vector(31 downto 0);
      sigin_wanted : out std_logic;
      sigin_ready : in std_logic;

      -- Camera A signals
      columnA : buffer std_logic_vector(9 downto 0);
      rowA : buffer std_logic_vector(9 downto 0);
      redA : out std_logic_vector(7 downto 0);
      greenA : out std_logic_vector(7 downto 0);
      blueA : out std_logic_vector(7 downto 0);
      datavalidA : buffer std_logic;
      newlineA : out std_logic;
      newframeA : buffer std_logic;

      -- Camera B signals
      columnB : buffer std_logic_vector(9 downto 0);
      rowB : buffer std_logic_vector(9 downto 0);
      redB : out std_logic_vector(7 downto 0);
      greenB : out std_logic_vector(7 downto 0);
      blueB : out std_logic_vector(7 downto 0);
      datavalidB : buffer std_logic;
      newlineB : out std_logic;
      newframeB : buffer std_logic
    ); end firevid;
    Signal Name Mode Width Description
    tm4_glbclk0in1 Connect this to one of the two TM-4 global clocks tm4_glbclk0 or tm4_glbclk1. The clock must be configured to be at least 50MHz for the interface to work correctly.
    resetnin1 Active low reset.
    tm4_fir_*   Connect these signals to one of the two Firewire chipsets on FPGA1, which use the same signal names as these but have an 'A' or a 'B' appended to them (exception: tm4_fir_resetn connects to the pins tm4_fir_resetB or tm4_fir_resetn).
    signals_inin32 Connect these signals to an instance of tm4_portmux on FPGA1
    signals_outout32
    sigin_wantedout1
    sigin_readyin1
    There are two copies of the following signals, one for each camera. All of them are synchronized to the rising edge of tm4_glbclk0.
    columnout10X coordinate of pixel
    rowout10Y coordinate of pixel
    redout8Color components of pixel
    green
    blue
    datavalidout1Is high when column/row/rgb are valid, low otherwise
    newlineout1High if this pixel is the first pixel of a scanline (column==0)
    newframeout1High if this pixel is the first pixel of a frame (column==0 && row==0)

Software API reference

    The provided API exposes functions that:

    To use the API, #include "firecam.h" in your program, and link with firecam.c, llc_iface.c, and ~tm4/lib/linux_tmports.o. As an example, two programs that use the API, camctrl and elemctrl, have been provided, along with a makefile.

    Many of the functions require a cam_id parameter to specify which of the (up to two) cameras to target. This parameter should be 0 when there is one camera connected, or 0/1 when there are two. There is no way to assign which physical camera is #0 or #1 when both are connected, but it usually corresponds to which Firewire port on the TM4 the camera is connected to. With one camera, its cam_id is always 0.

    int firecam_open()

    Description Call this once at the start of your program, before using any of the other functions.
    Returns0 on success, -1 on error


    int firecam_init()

    DescriptionSets up the Firewire chipset on the TM4. Call this once after programming the TM4's FPGAs. Do not call if one or both cameras are currently streaming video.
    Returns 0 on success, -1 on error


    int firecam_reset(int cam_id)

    DescriptionResets a camera to its factory settings.
    Parameters
    cam_id : Which camera to reset (0 or 1)
    Returns 0 on success, -1 on error


    int firecam_format_available(int cam_id, int format)

    Description

    Queries a camera to see whether a particular video format is available or not. A video format defines a group of video modes (see below). The video formats defined in firecam.h are:

    FORMAT_VGA : VGA non-compressed format (maximum 640x480)
    FORMAT_SVGA1 : Super VGA non-compressed format 1
    FORMAT_SVGA2 : Super VGA non-compressed format 2

    Only FORMAT_VGA is currently supported in this API.

    Parameters
    cam_id : Which camera to query (0 or 1)
    format : Which format to query (only FORMAT_VGA is supported)
    Returns 0 if format unavailable, 1 if available, and -1 on error


    int firecam_mode_available(int cam_id, int format, int mode)

    Description

    Queries a camera to see whether a particular video mode is supported for a given video format. A video mode defines the resolution and pixel format of the video stream. The video modes defined in firecam.h are:

    MODE_160_120_YUV444 : 160x120 YUV 4:4:4
    MODE_320_240_YUV422 : 320x240 YUV 4:2:2
    *MODE_640_480_YUV411 : 640x480 YUV 4:1:1
    MODE_640_480_YUV422 : 640x480 YUV 4:2:2
    MODE_640_480_RGB : 640x480 24-bit RGB
    MODE_640_480_MONO8 : 640x480 8-bit grayscale
    MODE_640_480_MONO16 : 640x480 16-bit grayscale

    These video modes are to be used with the FORMAT_VGA format.

    * The hardware is set up by default to decode this mode. This is the only valid option unless the hardware decoder is changed.

    Parameters
    cam_id : Which camera (0 or 1) to query
    format : Set to FORMAT_VGA
    mode : Which of the above modes to query
    Returns 0 if mode unavailable, 1 if available, and -1 on error


    int firecam_rate_available(int cam_id, int format, int mode, int rate)

    Description

    Queries a camera to see whether a particular frame rate is possible for a given format/mode combination. The frame rates defined in firecam.h are:

    RATE_7_5 : 7.5fps
    RATE_15 : 15fps
    RATE_30 : 30fps
    Parameters
    cam_id : Which camera (0 or 1) to query
    format : Set to FORMAT_VGA
    mode : Which mode to query the framerate for (see above)
    rate : Which framerate to query
    Returns 0 if rate unavailable, 1 if available, and -1 on error


    int firecam_set_format(int cam_id, int format)

    Description Configures a camera's video format. See firecam_format_available for available formats. Currently, the only valid format is FORMAT_VGA. Do not call while camera is streaming video.
    Parameters
    cam_id : Which camera (0 or 1) to configure
    format : The format to set (only FORMAT_VGA is valid right now)
    Returns 0 on success, -1 on failure


    int firecam_set_mode(int cam_id, int mode)

    Description Configures a camera's video mode. See firecam_mode_available for a list of valid modes. Do not call while camera is streaming video.
    Parameters
    cam_id : Which camera (0 or 1) to configure.
    mode : The mode to set
    Returns 0 on success, -1 on failure


    int firecam_set_rate(itn cam_id, int rate)

    Description Configures a camera's framerate. See firecam_rate_available for a list of valid framerates. Do not call while the camera is streaming video.
    Parameters
    cam_id : Which camera (0 or 1) to configure.
    rate : The framerate to set
    Returns 0 on success, -1 on failure


    int firecam_start_streaming(int cam_id, int speed)

    Description

    Tells a camera to start streaming video using its configured format/mode/rate settings, at a particular speed:

    SPD_100 : 100Mbps
    SPD_200 : 200Mbps
    *SPD_400 : 400Mbps

    *Certain mode/framerate combinations require a minimum speed to operate, so use SPD_400 to be safe. If too low a speed is chosen, this function will give no indication in its return value that it failed. Use firecam_is_streaming to confirm that the call was successful instead.

    Parameters
    cam_id : Which camera (0 or 1)
    speed : One of the above bus speeds to stream video at.
    Returns 0 on success, -1 on failure


    int firecam_stop_streaming(int cam_id)

    Description Tells a camera to stop streaming video.
    Parameters cam_id : Which camera (0 or 1)
    Returns 0 on success, -1 on failure


    int firecam_is_streaming(int cam_id)

    Description Queries a camera whether it is streaming video or not.
    Parameters cam_id : Which camera to query (0 or 1)
    Returns 0 if not streaming, 1 if streaming, and -1 on error


    int firecam_query_element(int cam_id, int element_id, elem_query_t* out)

    Description

    Queries the camera for support for a particular feature element. A feature element is a setting on the camera that controls the picture. Examples of feature elements include brightness, contrast, focus, and zoom.

    The feature elements defined in firecam.h are:

    ELEM_BRIGHTNESS
    ELEM_AUTO_EXPOSURE
    ELEM_SHARPNESS
    ELEM_WHITE_BALANCE
    ELEM_HUE
    ELEM_SATURATION
    ELEM_GAMMA
    ELEM_SHUTTER
    ELEM_GAIN
    ELEM_IRIS
    ELEM_FOCUS
    ELEM_TEMPERATURE
    ELEM_TRIGGER
    ELEM_ZOOM
    ELEM_PAN
    ELEM_TILT
    ELEM_FILTER

    Refer to the IIDC Digital Camera Specification for more details about the feature elements.

    The result of the query will be stored in a struct of type elem_query_t:

    typedef struct
    {
      int available;
      int auto_control;
      int man_control;
      int on_off;
      int min_value;
      int max_value;
    } elem_query_t;

    available : Is 1 if the feature element is controllable
    auto_control : Is 1 if the camera can be set to automatically control this element.
    man_control : Is 1 if the user can manually control this element.
    on_off : Is 1 if this feature can be toggled on/off
    min_value :

    Minimum value for manual control. For ELEM_TRIGGER, the lowest 4 bits of this field instead indicate which trigger modes are available:

    Bit 0: Trigger mode 3
    Bit 1: Trigger mode 2
    Bit 2: Trigger mode 1
    Bit 3: Trigger mode 0
    max_value : Maximum value for manual control. For ELEM_TRIGGER, this field is unused.
    Parameters
    cam_id : Which camera to control (0 or 1)
    element_id : Which of the above defined elements to query
    out : Pointer to an already allocated instance of elem_query_t in which to store the result.
    Returns 0 if successful, -1 on error


    int firecam_get_elem_status(int cam_id, int element_id, elem_control_t* out)

    Description

    Fetches the current status of a particular feature element.

    Results are returned in a struct of type elem_control_t that is defined in firecam.h:

    typedef struct
    {
      int on_off;
      int auto_control;
      int value;
      int value2;
    } elem_control_t;

    on_off : 1 if the element is turned on, 0 if turned off
    auto_control : 1 if the element is under automatic control by the camera, 0 if under manual control.
    value : For most elements, this is simply its value, which has an element-dependent meaning. For ELEM_TRIGGER, this specifies the Trigger Parameter.
    value2 : This field only used for 3 elements:
    ELEM_WHITE_BALANCE : Specifies the U/B color phase shift.
    ELEM_TRIGGER :

    Specifies the trigger mode, which is one of 4 constants defined in firecam.h:

    TRIGGER_MODE_0
    TRIGGER_MODE_1
    TRIGGER_MODE_2
    TRIGGER_MODE_3
    ELEM_TEMPERATURE : The target temperature.
    Parameters
    cam_id : Which camera (0 or 1)
    element_id : Which element to query
    out : Pointer to an already allocated instance of elem_control_t to store the result in.
    Returns 0 if successful, -1 on error


    int firecam_set_elem_status(int cam_id, int elem_id, elem_controL_t* in)

    Description Sets the status/value of a particular feature element.
    Parameters
    cam_id : Which camera (0 or 1)
    element_id : Which element to control
    in : Pointer to an instance of elem_control_t that has the settings you wish to apply to this feature element.
    Returns 0 if successful, -1 on error