Verdin iMX8M Plus: Using 2x MIPI CSI cameras at the same time

Technical Support
, ,
1

Hi team, a customer is using our Verdin iMX8M Plus with ADV7280 and cannot manage to make both work at the same time.

They managed to make them work separately, but whenever they launch the second, the first one would close with the following error:

isi-m2m 32e00000.isi:m2m_device: ISI channel[0] is busy

Device tree information:
verdin-imx8mp_adv7280_20_overlay.dts (1.6 KB)
verdin-imx8mp_adv7280_21_overlay.dts (1.7 KB)

[    1.896254] adv7180 2-0021: chip found @ 0x21 (30a40000.i2c)
[    1.925694] adv7180 2-0020: chip found @ 0x20 (30a40000.i2c)
[    2.312755] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0
[    2.325555] mx8-img-md: Registered mxc_isi.1.capture as /dev/video1
[    2.331954] mx8-img-md: Registered sensor subdevice: adv7180 2-0020 (1)
[    2.338604] mx8-img-md: Registered sensor subdevice: adv7180 2-0021 (2)
[    2.345274] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[    2.351922] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[    2.358431] mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]
[    2.365105] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.1]
[    2.371602] mx8-img-md: created link [adv7180 2-0020] => [mxc-mipi-csi2.0]
[    2.378545] mx8-img-md: created link [adv7180 2-0021] => [mxc-mipi-csi2.1]
[    2.385497] mxc-md 32c00000.bus:camera: mxc_md_create_links
[    3.290442] isi-m2m 32e00000.isi:m2m_device: Register m2m success for ISI.0

Gstreamer:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=720,height=480,framerate=30/1 ! queue ! filesink location=test0 &
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=720,height=480,framerate=30/1 ! queue ! filesink location=test1 &

System:

Verdin iMX8M Plus V1.0D
Verdin Development Board V1.1C
Verdin iMX8M Plus mezzanine 1.0
Linux version 5.4.161-5.7.0-devel+git.0f0011824921

Any idea? Thanks!
Alvaro.

2

Hello @alvaro.tx,
It seems that the same device is being opened twice. The error that the customer sees is coming from here:
http://git.toradex.cn/cgit/linux-toradex.git/tree/drivers/staging/media/imx/imx8-isi-m2m.c?h=toradex_5.4-2.3.x-imx&id=7fa0b7d9767732b73f85b53952913256922c817b#n451

 442 static int mxc_isi_m2m_open(struct file *file)                                                                                                            
 443 {                                                                                                                                                         
 444         struct video_device *vdev = video_devdata(file);                                                                                                  
 445         struct mxc_isi_m2m_dev *isi_m2m = video_drvdata(file);                                                                                            
 446         struct mxc_isi_dev *mxc_isi = mxc_isi_get_hostdata(isi_m2m->pdev);                                                                                
 447         struct device *dev = &isi_m2m->pdev->dev;                                                                                                         
 448         struct mxc_isi_ctx *mxc_ctx = NULL;                                                                                                               
 449         int ret = 0;                                                                                                                                      
 450                                                                                                                                                           
 451         if (atomic_read(&mxc_isi->usage_count) > 0) {                                                                                                     
 452                 dev_err(dev, "ISI channel[%d] is busy\n", isi_m2m->id);                                                                                   
 453                 return -EBUSY;                                                                                                                            
 454         }

As we can see, the file argument in this function is the file descriptor that should be associated with the device that was opened. However, if this error happens when we call this line:

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=720,height=480,framerate=30/1 ! queue ! filesink location=test1 &

What could be happening is that the /dev/video1 device is not mapped correctly to the minor device as it should. We could check that with:

ls -l /dev|grep video
crw-rw---- 1 root video   199,   0 Apr 18 14:15 galcore

And look for the major, minor pair, which in this example is 199, 0. The minor should be different for both video0 and video1.

You also mention that they managed to get both devices working separately. In the case that they are capturing video1, are they correctly getting the image associated with that channel? If for some reason both devices are opening the same interface, I would expect them both to have the same video input as video0.

Regards,
Rafael

1 Like
3

Thanks @rafael.tx ! We have reached with the customer with your points. We will get back with any update.

Alvaro.

4

Hi @rafael.tx , we got an update from the customer.

root@verdin-imx8mp-07154581:~# ls -l /dev | grep video
crw-rw---- 1 root video   250,   0 Jul  6 04:56 cec0
crw-rw---- 1 root video    29,   0 Jul  6 04:56 fb0
crw-rw---- 1 root video   199,   0 Jul  6 04:56 galcore
crw-rw---- 1 root video   251,   0 Jul  6 04:56 media0
crw-rw---- 1 root video    81,   2 Jul  6 04:56 v4l-subdev0
crw-rw---- 1 root video    81,   3 Jul  6 04:56 v4l-subdev1
crw-rw---- 1 root video    81,   4 Jul  6 04:56 v4l-subdev2
crw-rw---- 1 root video    81,   5 Jul  6 04:56 v4l-subdev3
crw-rw---- 1 root video    81,   0 Jul  6 04:56 video0
crw-rw---- 1 root video    81,   1 Jul  6 04:56 video1
crw-rw---- 1 root video    81,   6 Jul  6 04:56 video2

This is the output from ls -l /dev | grep video

We also confirmed they are different cameras by the image that is coming from them.

Our understanding is that the device support descriptors are as follows:

v4l-subdev0:mxc-mipi-csi2.0
v4l-subdev1:mxc-mipi-csi2.1
v4l-subdev2:adv7180 2-0020
v4l-subdev3:adv7180 2-0021
video0:mxc_isi.0.capture
video1:mxc_isi.1.capture
video2:mxc_isi.0.m2m

In imx8mp.dtsi, the setting of isi-m2m exists only in isi_0, and since isi-m2m is only one device, we think that it is shared by two cameras…

If you add the isi-m2m setting to isi_1 of the device tree
crw-rw ---- 1 root video 81, 7 Jul 6 05:16 video3
video3: mxc_isi.1.m2m can be created.

However, even in this case, it seems that video2:mxc_isi.0.m2m is used for both video0 and video1, and if two units are used at the same time, we will get the error:
isi-m2m 32e00000.isi: m2m_device: ISI channel [0] is busy

When opening the device by calling
static int mxc_isi_m2m_open (struct file * file) to
video2: mxc_isi.0.m2m for video0
video3: mxc_isi.1.m2m for video1
we think that the two units will work at the same time, but we do not know where they are opened.

5

Hi @rafael.tx , the customer managed to solve the issue by renaming the devices.

When further investigating, when gst-launch-1.0 was started, for some reason all /dev/video* open, so the customer renamed these to /dev/_video*

Then used the previous Gstreamer pipelines but with the updated devices:

gst-launch-1.0 v4l2src device=/dev/_video0
gst-launch-1.0 v4l2src device=/dev/_video1

BTW, it seems that mxc_isi_m2m was not used in the above.

Any idea on why this happens? I don’t think it should have to be used this way :sweat_smile:

6

Let me see if I understood:
By the time the first gst-launch-1.0 was running, all of the video devices would have been opened?

8

Yes, exactly

9

Could we try to run the pipelines again (with the standard device names) through strace? I’m wondering if it’s gstreamer that’s opening the devices multiple times, or if it’s something else.
If it’s gstreamer, running gst-launc-1.0 through strace would show us open calls not only to the requested device but also to the other devices.

1 Like
10

Hi @rafael.tx , the customer made also some interesting findings: They renamed /dev/video1 to /dev/video99. The strace corresponds to Test1, find them attached:
strace.zip (35.6 KB)

strace -r -o strace1.txt gst-launch-1.0 v4l2src device=/dev/video0
strace -r -o strace2.txt gst-launch-1.0 v4l2src device=/dev/video99

Test1:

gst-launch-1.0 v4l2src device=/dev/video0
gst-launch-1.0 v4l2src device=/dev/video99

doesn’t work: When opening video99, it closes video0 like always.

Test2:

gst-launch-1.0 v4l2src device=/dev/video99
gst-launch-1.0 v4l2src device=/dev/video0

works OK. When opening video0, video99 continues working fine.

Kind regards,
Alvaro.

11

The customer found the reason:

It seems that the mxc-isi-capture for video0 and video1, basic only knows ON/OFF. So when gstreamer goes through the / dev / video device at startup, the camera that is already running is powered off and stops working.

Basically renaming the video devices to somewhere else, solves the issue.