Triggerring CSI Kameramodul 5MP OV5640 at particular frequency

Technical Support
, ,
1

Hi Tx Team,

I have a CSI OV5640 camera, which I am using to stream video using imx6 and GStreamer plugins. Now I have a certain custom device, that outputs a set of data(D1,D2) at 60Hz. I want to synchronize this set of data (D1 & D2) to the frames being captured by the camera. Both the custom device and OV5640 are connected to the imx6 processor.

For eg. the camera operates at 30 fps, which gives me each frame at ~33ms. This means two sets of data (D1 & D2) from the custom device, for each frame at a given point of time.

so at time t1 ->  for frame F1,  I have D1 & D2 captured at t0 and D1 & D2 captured at t1. 
F1 D10 D20 D11 D12 -> complete data at time t1.

  1. The custom device provides a trigger signal, which can trigger the camera at 30Hz. I was wondering if its possible to trigger the camera OV5640 every 30Hz, so that the data from custom device and camera is in sync. I had a look into datasheet of OV5640, which says that it supports snapshot operation, but somehow there is no description available on how to do this. Could you let me know if this is possible to do?

  2. It would also be fine, if the triggering can be done vice-versa, i.e. the camera triggering the custom device.

  3. Could you suggest whats the best way to synchronize the custom device and camera? Gstreamer provides, timestamp features, which I am considering to use if none of the above solutions work. Would that be a good option?

  4. Consider this simple eg:

    gst-launch-1.0 -v imxv4l2videosrc device=/dev/video1 !
    video/x-raw,width=100,height=50,framerate=30/1 ! filesink location=xyz.avi

How could I modify this to make it trigger from an external device at 30Hz?

2

Hi @soc_learner

Unfortunately the VSYNC signal isn’t available on the camera module, if it would you could use this to synchronize the sensor. Maybe you can try to experiment with the STROBE signal? Please check page 4-16 in this document (multiple pulse output):

If you manage to set skip frames close to zero you should be able to sync your sensor with the camera. The Strobe signal is available on the camera connector (Pin 1). I don’t know if it is possible to set skip frames close to zero but maybe you can give it a try?

Probably it is easier to synchronize the sensor with the camera than vice versa. The exact time when a frame is taken will also involve the exposure time, etc. So synchronizing the camera with a sensor could be hard.

Using the timestamp could be a solution but probably it’s not accurate enough. What are you planning to synchronize, maybe an IMU?

Regards,
Stefan

3
  1. Using the strobe seems too difficult. But would be an ideal option. In case, the skipped frames cannot be set to zero, I can still run the camera at 60 fps and set the skipped frames to 1, which will give me approx. 30 fps.
  2. the custom device does have a signal ‘send_latest_sample’, which when triggered gives me the latest sample. But this sample could have unknown delay of 16ms to 1ms (running freq. = 60 hz). It does not guarantee the latest sample. However, this seems to be a viable option now for the initial testing. Also I would need the 2 sets of data, so I must have to consider upsampling. Any idea on how to do this?
  3. Yes!! you got it right, I am trying to sync the IMU data with video frame.

Any other suggestions from your side?

4

However, I still would be needing a VSync signal to know when the complete frame is captured, to trigger the sensor. And as you said, this is missing in OV5640, how could I know when the frame capture is complete and trigger the sensor, keeping in mind that I will be using GStreamer framework for video streaming?

5

Hi @soc_learner

I would recommend you to use OpenCV for what you want to do. You have to check if the iMX6 is fast enough for doing upsampling in software. If not you may have to check if the IPU can accelerate this task.

For testing you could e.g. do a read on the VideoCapture device in OpenCV if read returns you can synchronize the IMU. Of course this isn’t really accurate but is probably the easiest way to do some tests.

Regards,
Stefan

6

Hi Stefan,

Are any other cameras with CSI2-MIPI interface and with VSync trigger available that I can use? Does Toradex provide any such camera module?

7

Hi @soc_learner,
@stefan_e.tx will be back in office tomorrow. Then he will answer your questions.

8

Hi @soc_learner

Unfortunately Toradex doesn’t provide a CSI2-MIPI camera module that provides the VSYNC signal. I’m afraid you would have to design your own board that sends the VSYNC to your sensor. Maybe a board that adapts something like this to our camera connector?

Regards,
Stefan