[Apalis iMX6] Scaling video makes screen blank repeatedly

Technical Support
, , , , ,
1

Hello,

I’m using Linux with upstream kernel 5.10.19 on Apalis iMX6Q 2GB IT and Ixora v1.1A carrier board.
The board connects to a fullHD monitor (HP 24fw) through HDMI cable.

I can scale a 720p@30fps video to 1920x1080 just fine with gstreamer:

gst-launch-1.0 filesrc location=/root/720.mkv ! matroskademux ! h264parse \
             ! v4l2h264dec capture-io-mode=dmabuf \
              ! v4l2convert output-io-mode=dmabuf-import \
              ! video/x-raw,width=1920,height=1080 \
              ! kmssink

However, with a little stress on system, scaling video makes screen blank repeatedly.
You can see that in this clip: https://i.imgur.com/RwoICC3.mp4

To be easier to test, I made an example Qt widget which just calls update() every 66ms and draw nothing: main.cpp · main · Trung Do / qt-timer-test · GitLab

# Run qt example
./qt-timer-test -platform offscreen

# In another terminal, play video with gstreamer command above
gst-launch-1.0 ...

I also do some other tests and see:

  • Issue still happens when replace kmssink by fakesink (not display video, just scale). Console screen is blank repeatedly even harder.
  • Without the Qt example above, issue also can happen if enable kernel debug log and if there is a lot of logs provided by dmesg.
    Example: Enable DYNAMIC_DEBUG in kernel config and enable log for ipu-image-convert.c by echo 'file drivers/gpu/ipu-v3/ipu-image-convert.c +p' > /sys/kernel/debug/dynamic_debug/control

Do you have any ideas where the issue may come from? (limit of CPU, memory, etc.)

Update: Packages in system are from Debian 11

  • Gstreamer: 1.18.4
  • Qt: 5.15.2

Thank you very much,
Trung

2

Hi @dothanhtrung , could it be some thermal throttling? Do you have any heatsink attached? Did you check the temperature of the system? Temperature Sensor (Linux) | Toradex Developer Center

1 Like
3

could it be some thermal throttling?

Thank you. This should be one of things I need to check.

Do you have any heatsink attached?

No, the board is exposed to the air.

Did you check the temperature of the system?

The temperature is 50~52°C while idling, 53~54°C when play gstreamer alone, and 53~55°C when play gstreamer with Qt timer example.

Thanks,
Trung

4

Doesn’t look like a thermal issue then… Did you check the performance with top/htop? Did you manage to get the same issue with another module or does it happen only with a single module?

Just to check if it is a performance related issue with the VPU vs GPU, did you check in the Qt demo from our Multimedia image if you get a similar issue?

5

Did you check the performance with top/htop?

Gstreamer pipeline takes 13~15% CPU and ~16MB RAM, while Qt timer example takes ~8% CPU and 14MB RAM. They are run at the same time. There is no suddenly change in CPU usage at the moment blanking happens.

Did you manage to get the same issue with another module or does it happen only with a single module?

Sorry, I don’t understand what ‘module’ here stands for.

Just to check if it is a performance related issue with the VPU vs GPU, did you check in the Qt demo from our Multimedia image if you get a similar issue?

Do you mean the app Qt5_CinematicExperience, I run it with command ./Qt5_CinematicExperience --fullscreen -platform linuxfb, and the results are:

  • Qt5_CinematicExperience + Qt timer test: Issue doesn’t happen. I guess Qt5_CinematicExperience doesn’t do any scaling.
  • Qt5_CinematicExperience + Gstreamer with v4l2convert: Issue still happens.

Additional information, v4l2convert is using /dev/video8 (imx-csc-scaler) for scaling video frame.

# v4l2-ctl -D -d /dev/video8 
Driver Info:
        Driver name      : imx-csc-scaler
        Card type        : imx-csc-scaler
        Bus info         : platform:imx-csc-scaler
        Driver version   : 5.10.19
        Capabilities     : 0x84208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : imx-media
        Model            : imx-media
        Serial           : 
        Bus info         : 
        Media version    : 5.10.19
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.10.19
FAIL: could not find device 81:22 in topology

Thanks,
Trung

6

Thanks for the quick tests. Indeed, it seems the issue seems related to the VPU/Scaling. I will ping the Linux BSP team and see if we can further check.

Another Apalis iMX6. Just to confirm this is not an isolated HW issue but if really this is something that happens in all Apalis iMX6.

7

@dothanhtrung Just to clarify:

Issue still happens when replace kmssink by fakesink (not display video, just scale). Console screen is blank repeatedly even harder.

In this case, no video is shown but the Weston desktop and the screen also blinks, correct? Just to confirm this is unrelated to the HDMI signal.

My guess is that the VPU/GPU is really on the limit and can’t keep up… Adding that additional stress from the Qt application or the extra debug logging doesn’t help. Upscaling is quite demanding.

8

I was also thinking about this: Upstream kernel doesn’t have very good support for GPU/VPU and downstream is normally advised. Any reason why you are choosing upstream for your solution? Have you tried this on a downstream kernel?

9

Indeed, it seems the issue seems related to the VPU/Scaling. I will ping the Linux BSP team and see if we can further check.

That’s nice. Thank you very much. Please ask me if you want additional information.

Another Apalis iMX6. Just to confirm this is not an isolated HW issue but if really this is something that happens in all Apalis iMX6.

Yes, I got this issue on other Apalis iMX6 too.

In this case, no video is shown but the Weston desktop and the screen also blinks, correct? Just to confirm this is unrelated to the HDMI signal.

Sorry for not make this clear. I do the check on the console text (weston is not running), and yes, screen also blinks although no video is shown.
In more serious case (when Qt app make more stress by drawing thing), when screen is blank for too long (more than 5 seconds I think), monitor lost the signal and shows message “Input Signal Not Found”

I was also thinking about this: Upstream kernel doesn’t have very good support for GPU/VPU and downstream is normally advised. Any reason why you are choosing upstream for your solution? Have you tried this on a downstream kernel?

I am doing investigation to see if upstream kernel (and also packages) supports the board enough. It will help to keep system more up to date. But thank you for your suggestion. I will take a try on a downstream kernel.

Thanks,
Trung

10

In addition, the same issue happens with downscaling from 1080p video to 720p too.

Thanks,
Trung

11

Thanks for the info @dothanhtrung. Please try on the downstream kernel (ideally on our latest Multimedia image) and see if you can reproduce the issue :pray: I’m concerned that the mainline kernel for the VPU/GPU is not quite there yet…

Thanks a lot. Alvaro.

12

Hi Trung,

Do you see the problem if scaling is not done, for example, by playing a 1080p file?

It would be nice if you could report this problem to the kernel mailing list.

To get the people and lists to report this, you could run:

./scripts/get_maintainer.pl drivers/gpu/drm/imx/

Regards,

Fabio Estevam