GPU Monitoring on TorizonCore

Hello,

I was wondering how to monitor GPU usage on TorizonCore. I am using TorizonCore 6.1.0 with a Verdin iMX8M Plus.
I tried modifying the image using Yocto and following this: Install gputop on imx8mp verdin - Technical Support - Toradex Community
However, this did not work as intended. No gpuProfiler seems to be there and gpuinfo.sh script returns that the platform is not supported.

What is a simple way to monitor GPU usage on TorizonCore ?

Thank you for your help.

Greetings @mnano,

However, this did not work as intended. No gpuProfiler seems to be there and gpuinfo.sh script returns that the platform is not supported.

Looking at the other thread you referenced, this seems like the outcome the other customer got as well.

As an alternative GPU monitoring tool we have tried gputop. We have a proof of concept container with gputop you can try out. To do so here are the following steps:

  • First of all on TorizonCore you need to set some kernel arguments like so: sudo fw_setenv tdxargs "galcore.powerManagement=0 galcore.showArgs=1 galcore.gpuProfiler=1"
    • You’ll need to reboot after doing this to apply the new arguments. After the reboot check /proc/cmdlin to make sure these were applied.
  • Then you can run a container that has the gputop tool built in like so: docker run -it --privileged yagomellotoradex/imxgputop:0.1.0
    • This should bring up some text output with information on the GPU, you can navigate the information with the left and right arrow keys. The text output flickers whenever it refreshes but apparently this is normal.

On a side-note this gputop tool is provided and maintained by NXP. For more information on the tool and how to interpret the output you can find the repository for this tool here: https://source.codeaurora.org/external/imx/imx-gputop/

The tool is just provided as is, we can’t help much with any issues or questions with the tool since as I said it comes all from NXP.

Best Regards,
Jeremias

2 Likes

I’ve just tried the proof of concept container with gputop, but it doesn’t work.
I mean, it says that no client are attached to GPU, and all the usage info are not filled.
The only fields that are filled are related to DDR0 and DDR1.

@vix Assuming you set the required kernel arguments and launched the container correctly, I’m not sure what could be the issue. That said this container really was just a one-time proof of concept and we don’t actively maintain it. There’s also the tool itself which I’m not too versed on it’s mechanics since it’s created and maintained by NXP: GitHub - nxp-imx/imx-gputop: i.MX GPU Performance Tool

Best Regards,
Jeremias

@jeremias.tx can you share the Dockerfile for the imxgputop container so that I can try to rebuild it against the lastest Debian-base?

can you share the Dockerfile for the imxgputop container so that I can try to rebuild it against the lastest Debian-base?

Sure here’s a zip file of the code-repo for this container image:
imx-gputop-container-main.zip (2.3 KB)

It’s pretty straightforward it’s mostly just cloning and building the tool from NXP directly. If there’s any issues with the tool itself you’ll need to raise this with NXP who maintains the tool.

Best Regards,
Jeremias

install_imxgputop.h included in the zip doesn’t work anymore, sice it tries to clone imx-gputop from source.codeaurora.org which is now archived

# cloning the imx gputop
git clone git://source.codeaurora.org/external/imx/imx-gputop.git -b release

shouldbe changed to

# cloning the imx gputop
git clone https://github.com/nxp-imx/imx-gputop.git -b release

After a deeper investigation, imx-gputop gives some information, but it’s not clear if they’re right or not.

It says that iMX8M-Plus has:

  • one GC7000 unit (3D)
  • one GC8000 unit (3D)
  • one GC520 unit (2D)

but the datasheet says that only GC7000UL and GC520 are present.
I’ve already asked this to NXP tech support.

Using Torizon Chromium container I tested the WebGL FishTank example and I get the following results
immagine
It seems that the GPU node 0 is 100% loaded (I think that the node 1 is the “fake” CG8000).
Based on my expectations, the performances are quite poor (14 fps with 1 fish)

The question to Toradex engineers is: is this the hardware limit of the GPU for Verdin iMX8M-Plus?
Or do you expect someting better?

As I told you in the other thread: TorizonCore 6.1 cannot render anymore 3D content on Verdin iMX8M-Mini - #38 by jeremias.tx

There were possible issues found with our chromium container and hardware acceleration on the Verdin i.MX8M Plus. Nothing was done here yet, so these issues are probably still present. Therefore it doesn’t really make sense to examine the performance of our chromium container here given the known issues.

Best Regards,
Jeremias

I know,
but for our project, this issue is a showstopper.
And so we’ve bee spending time in investigating by ourselves, because at the moment there isn’t a date for a fix from Toradex (I understand this, it’s up to Toradex fix the roadmap).
I add to the community every new info I get, in case it is useful to other users.

Let me know if I can do something to help Toradex engineers debugging or testing the issue.

1 Like

I’ve just get the answer fron NXP tech support:

  • the GC8000 unit (3D) shown in gputop is the NPU

The issue with Chromium acceleration is being raised with our development team. Though nothing can be shared at the moment.

Best Regards,
Jeremias

Hello @jeremias.tx
can you share some kind of update on the investigation on this issue?

Hi @vix !

Which issue? Could you be more specific?

The topic of this thread (GPU Monitoring on Torizon OS) is solved.

If you are referring to the other thread (TorizonCore 6.1 cannot render anymore 3D content on Verdin iMX8M-Mini - #38 by jeremias.tx), could you please ask in there? We prefer to keep the topics organized :slight_smile:

Best regards,

Hi @henrique.tx

You’re right: the GPU monitoring works with imx-gputop container.
And it shows that GPU acceleration is not used by Chromium container.

The issue with Chromium acceleration is being raised with our development team. Though nothing can be shared at the moment.

I refer to this issue

Thanks for the answer, @vix !

So, seems like the topic at TorizonCore 6.1 cannot render anymore 3D content on Verdin iMX8M-Mini - #38 by jeremias.tx is better aligned to your question, isn’t?

Since the topic here is solved, could you please ask there?

Not sure about it.
But if you prefer, I can go on on the other topic…