MIPI CSI-2 Camera Gstreamer error

sjjin · September 7, 2023, 1:53am

ear all,

i’m working with the following configuration:
Apalis iMX8QM 4GB WB IT V1.1D
Ixora Carrier Board V 1.2A
Linux BSP 6.3.0

I would like to improve the ability to load video via MIPI CSI-2.
When you run the MIPI CSI-2 driver, a capture sub device is created, but /dev/videoX is not created.

I got the following results:

# v4l2-ctl --list-devices
amphion vpu decoder (platform: amphion-vpu):
        /dev/video0
        /dev/video1
        /dev/media0

mxc-jpeg codec (platform:58400000.jpegdec):
        /dev/video2

mxc-jpeg codec (platform:58450000.jpegenc):
        /dev/video3

FSL Capture Media Device (platform:mxc-md):
        /dev/media1

# ls -al /dev/v4l-subdev*
crw-rw---- 1 root video 81, 2 Feb 23 06:33 /dev/v4l-subdev0
crw-rw---- 1 root video 81, 3 Feb 23 06:33 /dev/v4l-subdev1

# media-ctl -d /dev/media1 -p
Media controller API version 5.15.77

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial
bus info
hw revision     0x0
driver version  5.15.77

Device topology
- entity 1: mxc-mipi-csi2.0 (8 pads, 4 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                <- "tp2855_mipi 5-0044":0 [ENABLED,IMMUTABLE]
        pad1: Sink
                <- "tp2855_mipi 5-0044":1 [ENABLED,IMMUTABLE]
        pad2: Sink
                <- "tp2855_mipi 5-0044":2 [ENABLED,IMMUTABLE]
        pad3: Sink
                <- "tp2855_mipi 5-0044":3 [ENABLED,IMMUTABLE]
        pad4: Source
        pad5: Source
        pad6: Source
        pad7: Source

- entity 10: tp2855_mipi 5-0044 (4 pads, 4 links)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:YUYV8_1X16/1920x1080 field:none colorspace:jpeg]
                -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]
        pad1: Source
                -> "mxc-mipi-csi2.0":1 [ENABLED,IMMUTABLE]
        pad2: Source
                -> "mxc-mipi-csi2.0":2 [ENABLED,IMMUTABLE]
        pad3: Source
                -> "mxc-mipi-csi2.0":3 [ENABLED,IMMUTABLE]

Can someone help me with this, pls?
Best regards,

alvaro.tx · September 12, 2023, 8:05am

What camera are you using? Do you know if the driver is being recognized?

sjjin · September 12, 2023, 10:25am

Hi @alvaro.tx,

We use AHD camera.

AHD camera input was converted to mipi-csi2 through TP2855 (AHD Decoder).

I was able to create /dev/video2 node in mxc_isi.4.capture.

- entity 18: mxc_isi.4.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: Sink
                <- "mxc_isi.4":12 [ENABLED]

However, when I tried to play it with gstreamer, it stopped in the following state:

# gst-launch-1.0 v4l2src device=/dev/video2 ! autovideosink
[ 1337.132101] usb 3-1.2: reset high-speed USB device number 4 using ci_hdrc
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Can you help me solve this problem?

Best regards,
sjjin

alvaro.tx · September 14, 2023, 11:34am

Hi @sjjin , I’m still not convinced that the driver is properly recognized. How did you add the TP2855 driver? Can you share the device tree and kernel config?

Thanks,
Alvaro.

sjjin · September 14, 2023, 10:49pm

Hi @alvaro.tx,

Thank you for answer.
I will provide the tp2855 device tree:

/* Apalis I2C3 (CAM) */                                                    
&i2c3 {                                                                    
    #address-cells = <1>;                                                  
    #size-cells = <0>;                                                     
    status = "okay";                                                       
                                                                           
    tp2855_mipi@44  {                                                      
        compatible = "techpoint,tp2855_mipi";                              
        reg = <0x44>;                                                      
        pinctrl-names = "default";                                         
        pinctrl-0 = <&pinctrl_gpio1>;                                      
        clocks = <&clk_dummy>;                                             
        clock-names = "capture_mclk";                                      
        mclk = <27000000>;                                                 
        mclk_source = <0>;                                                 
        mipi_csi;                                                          
        rst-gpios = <&lsio_gpio0 8 GPIO_ACTIVE_HIGH>;   /* Apalis GPIO1 */ 
        virtual-channel;                                                   
        status = "okay";                                                   
        port {                                                             
            tp2855_1_ep: endpoint {                                        
                remote-endpoint = <&mipi_csi1_ep>;                         
                data-lanes = <1 2 3 4>;                                    
            };                                                             
        };                                                                 
    };                                                                     
};                                                                         
                                                                           
&isi_4 {                                                                   
    status = "okay";                                                       
                                                                           
    cap_device {                                                           
        status = "okay";                                                   
    };                                                                     
};                                                                         
                                                                           
                                                                           
&mipi_csi_1 {                                                              
    #address-cells = <1>;                                                  
    #size-cells = <0>;                                                     
    status = "okay";                                                       
    virtual-channel;                                                       
                                                                           
    /* Camera 1  MIPI CSI-2 (CSIS1) */                                     
    port@1 {                                                               
        reg = <1>;                                                         
        mipi_csi1_ep: endpoint {                                           
            remote-endpoint = <&tp2855_1_ep>;                              
            data-lanes = <1 2 3 4>;                                        
        };                                                                 
    };                                                                     
};

The tp2855 provides OSC 27MHz clk.

Can you help me solve this problem?

Kind regards,
sjjin

alvaro.tx · September 15, 2023, 7:14am

Dear @sjjin , please send the following:

Output of dmesg
Output of lsmod
Output of zcat /proc/config.gz
Output of ls -l /dev/

Thanks,
Alvaro.

sjjin · September 15, 2023, 8:48am

Hi @alvaro.tx,

Thank you for answer.
We will respond to your requests(Attached as a txt file):

Output of dmesg
→ dmeg.txt (39.9 KB)
Output of lsmod
→ lsmod.txt (1.4 KB)
Output of zcat /proc/config.gz
→ config.gz.txt (198.8 KB)
Output of ls -l /dev/
→ ls.txt (15.7 KB)

Do you have any additional requests for debugging?

Best regards,
sjjin

alvaro.tx · September 20, 2023, 10:54am

Hi @sjjin, sorry for the delay.

Well, the TP2855 is there according to the Kernel configuration. Do you have any information or reference on how this was added? This is not in our default BSP and I couldn’t find any information in Google about a Linux driver.

When you run the MIPI CSI-2 driver, a capture sub device is created, but /dev/videoX is not created.

I could see 6 video devices (video0-5) so I understand that this is not a problem anymore. Do you get the same response.

gst-launch-1.0 v4l2src device=/dev/video2 ! autovideosink

Checking the NXP Linux User Guide for 5.15 (Used in BSP 6.3.0), this is the correct Gstreamer pipeline for a TV decoder however, I don’t think this is your usecase. Have you tried any other pipelines for Webcam preview like the following?

gst-launch-1.0 v4l2src device=/dev/video2 ! 'video/x-raw, format=(string)UYVY,width=640,height=480,framerate=(fraction)30/1' ! autovideosink

Also, pls check the output for gst-inspect-1.0 v4l2src

Apalis iMX8QM 4GB WB IT V1.1D
Ixora Carrier Board V 1.2A
Linux BSP 6.3.0

Are you using our Multimedia Image 6.3.0? Do you have a HDMI display connected? What do you see in the screen?

Kind regards,
Alvaro.

sjjin · September 21, 2023, 12:54am

Hi @alvaro.tx, Thank you for answer.

Well, the TP2855 is there according to the Kernel configuration. Do you have any information or reference on how this was added? This is not in our default BSP and I couldn’t find any information in Google about a Linux driver.

Permissions are limited, but you can find it on the NXP Community:

i.mx8x ADAS support - NXP Community

We can find information the kernel driver for the TP2855 at rockchip. share Git hub:

I could see 6 video devices (video0-5) so I understand that this is not a problem anymore. Do you get the same response.

You are right.

Checking the NXP Linux User Guide for 5.15 (Used in BSP 6.3.0), this is the correct Gstreamer pipeline for a TV decoder however, I don’t think this is your usecase. Have you tried any other pipelines for Webcam preview like the following?

I also tried other pipelines.
However, when I tried to play it with gstreamer, it stopped in the following state:

# gst-launch-1.0 v4l2src device=/dev/video2 ! 'video/x-raw, format=(string)UYVY,width=1920,height=1080,framerate=(fraction)30/1' ! autovideosink
[ 1835.096744] usb 3-1.2: reset high-speed USB device number 3 using ci_hdrc
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Also, pls check the output for gst-inspect-1.0 v4l2src

results of gst-inspect-1.0 v4l2src:
gst-inspect-1.0 v4l2src.txt (10.8 KB)

Are you using our Multimedia Image 6.3.0? Do you have a HDMI display connected? What do you see in the screen?

I am using Multimedia Image 6.3.0.

TDX Wayland with XWayland 6.3.0-devel-20230920070229+build.0 (kirkstone) apalis-imx8-14759928 ttyLP1
Apalis-iMX8_Reference-Multimedia-Image

apalis-imx8-14759928 login:

HDMI is connected to the monitor.
The Weston screen is displayed. Share as a photo file:

hdmi_output1920×1706 162 KB

Additionally, when I check v4l2-ctl --list-formats-ext -d /dev/video2, Interval: Discrete nans (nan fps) is repeated:

# v4l2-ctl --list-formats-ext -d /dev/video2
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'RGBP' (16-bit RGB 5-6-5)
                Size: Discrete 0x0
                        Interval: Discrete nans (nan fps)
                        Interval: Discrete nans (nan fps)
                        Interval: Discrete nans (nan fps)
                        Interval: Discrete nans (nan fps)
                        Interval: Discrete nans (nan fps)

Can you help me solve this problem?

Best regards,
sjjin

alvaro.tx · September 21, 2023, 9:10am

Hi @sjjin , thanks for all the information. Based on the logs, I think you have a USB/UVC camera. Does Gstreamer work for this USB camera? That should be very straightforward.

Kindly confirm.

The TP2855 seems to be correctly identified so my suspicion is with the Gstreamer pipeline. Let’s confirm that the USB camera is working.

sjjin · September 21, 2023, 10:50pm

Hi @alvaro.tx, Thank you for answer.

I got USB/UVC information using v4l2-ctl --list-devices:

UVC Camera (046d:0825) (usb-ci_hdrc.1-1.2):
        /dev/video5
        /dev/video6
        /dev/media2

The video was confirmed normally when played with gstreamer.

# gst-launch-1.0 v4l2src device=/dev/video5 ! autovideosink
[  211.928876] usb 3-1.2: reset high-speed USB device number 3 using ci_hdrc
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:04.4 / 99:99:99.

Do you have any additional requests for debugging?

Best regards,
sjjin