RPMSG Character Device Driver in Torizon Core 6.2

Technical Support
,
1

Following on from this thread, I would like to implement an RPMSG communication from the M4 to the Linux userspace and back.

/dev/rpmsg_cntrl0 … 3 are present

On Linux side:
I understand the approach in such a way that via the /dev/rpmsg_ctrlX
and ioctl(fd, RPMSG_CREATE_EPT_IOCTL, &ept) a new endpoint is created.

this then appears e.g. as /dev/rpmsg0 if I use /dev/rpmsg_ctrl0
I should then be able to read and write to /dev/rpmsg0.
When I call:
fd = open("/dev/rpmsg_ctrl0", O_RDWR);
I get a:
‘-1’ for all 4 control devices (access denied … ?)

I think there is something fundamentally wrong.
The TTY example works on the M4 side, which creates / wants to create an endpoint itself.
The /dev/ttyRPMSG31 is not created because the rpmsg_tty kernel module is not present.
(See LOCAL_EPT_ADDR is 30 or 31 … M40 or M41)

At least the M4 program overcomes the code:

while (0 == rpmsg_lite_is_link_up(my_rpmsg))
         ;

because the message:

PRINTF("\r\nNameservice sent, ready for incoming messages...\r\n");

appears on the serial debug output.

First I need to know why I can’t access /dev/rpmsg_ctrlX from a docker container as described here:
application-development

Maybe the solution is very simple?

Best regards

additionally
on the apalis module in the torizon home directory: /var/rootdirs/home/torizon
created a folder named ‘rpmsg_char_dev’. included is a file called ‘rpmsg_char_dev’… what for?

Here are some printouts of my surroundings

root@apalis-imx8-07201831:/var/rootdirs/home/torizon# ls -la /dev/rpm*
crw------- 1 root root 511, 0 Apr 28  2022 /dev/rpmsg_ctrl0
crw------- 1 root root 511, 1 Apr 28  2022 /dev/rpmsg_ctrl1
crw------- 1 root root 511, 2 Apr 28  2022 /dev/rpmsg_ctrl2
crw------- 1 root root 511, 3 Apr 28  2022 /dev/rpmsg_ctrl3

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           5.15.77-6.2.0+git.aa0ff7e3554e #1-TorizonCore SMP PREEMPT Wed Mar 29 15:33:40 UTC 2023
Kernel command line:      pci=nomsi root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/fde04e5dc537e7d0fb52b2b3582fcf35a6a4c9a662edf9914167a2bada0feb30/0
Distro name:              NAME="TorizonCore"
Distro version:           VERSION_ID=6.2.0-build.2
Hostname:                 apalis-imx8-07201831
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Apalis iMX8QM V1.1 on Apalis Ixora V1.2 Carrier Board
Toradex version:          0037 V1.1E
Serial number:            07201831
Processor arch:           aarch64
------------------------------------------------------------


root@apalis-imx8-07201831:/var/rootdirs/home/torizon# dmesg | grep rpmsg
[    0.083998] imx rpmsg driver is registered.
[    1.430106] imx-rpmsg 90000000.rpmsg0: assigned reserved memory node vdevbuffer@90400000
[    1.466745] virtio_rpmsg_bus virtio0: rpmsg host is online
[    1.496753] virtio_rpmsg_bus virtio1: rpmsg host is online
[    1.497274] imx-rpmsg 90100000.rpmsg1: assigned reserved memory node vdevbuffer@90400000
[    1.498083] virtio_rpmsg_bus virtio2: rpmsg host is online
[    1.498911] virtio_rpmsg_bus virtio3: rpmsg host is online
[    1.498940] virtio_rpmsg_bus virtio3: creating channel rpmsg-virtual-tty-channel-1 addr 0x1f
[    1.500965] virtio_rpmsg_bus virtio2: creating channel rpmsg-i2c-channel addr 0x1
[    1.501139] i2c-rpmsg virtio2.rpmsg-i2c-channel.-1.1: new channel: 0x400 -> 0x1!
2

Hi @gerko,

Before going deep into this problem, I would like to ask if you can share your overlay source code that enables the RPMSG and Remote Proc features with us. So I can run the same code on my side and check how the char device can be handled with TorizonCore.

Thanks!

Best Regards,
Hiago.

3

here it is:
branch: toradex_5.15-2.1.x-imx

added to linux/arch/arm64/boot/dts/freescale/imx8qm.dtsi at line 473 after thermal_zones:

	
    rpmsg0: rpmsg0{
		compatible = "fsl,imx8qm-rpmsg";
		/* up to now, the following channels are used in imx rpmsg
		 * - tx1/rx1: messages channel.
		 * - general interrupt1: remote proc finish re-init rpmsg stack
		 *   when A core is partition reset.
		 */
		mbox-names = "tx", "rx", "rxdb";
		mboxes = <&lsio_mu5 0 1
			  &lsio_mu5 1 1
			  &lsio_mu5 3 1>;
		mub-partition = <3>;
		status = "disabled";
	};

	rpmsg1: rpmsg1{
		compatible = "fsl,imx8qm-rpmsg";
		/* up to now, the following channels are used in imx rpmsg
		 * - tx1/rx1: messages channel.
		 * - general interrupt1: remote proc finish re-init rpmsg stack
		 *   when A core is partition reset.
		 */
		mbox-names = "tx", "rx", "rxdb";
		mboxes = <&lsio_mu6 0 1
			  &lsio_mu6 1 1
			  &lsio_mu6 3 1>;
		mub-partition = <4>;
		status = "disabled";
	};

added to linux/arch/arm64/boot/dts/freescale/imx8-apalis-v1.1.dtsi ca. at line 890


&rpmsg0{
	/*
	 * 64K for one rpmsg instance:
	 */
	vdev-nums = <2>;
	reg = <0x0 0x90000000 0x0 0x20000>;
	memory-region = <&vdevbuffer>;
};

&rpmsg1{
	/*
	 * 64K for one rpmsg instance:
	 */
	vdev-nums = <2>;
	reg = <0x0 0x90100000 0x0 0x20000>;
	memory-region = <&vdevbuffer>;
};

ant the 2 dto’s from Toradex Support (hfranco.tx) : Downdload DTO

I hope it’s OK so.
Greetings
Gerald

4

After some research, the question obviously narrows down to:
Are the kernel modules rpmsg_char and rpmsg_cntrl with the NXP
RPMsg-Lite component (out of the box) compatible / usable?

Best regards
Gerald

5

Hi @gerko,

Sorry for the delay, it took me some time to check how NXP does the device tree on their modules and test these configurations on Apalis iMX8.

I was able to create an overlay and test RPMSG on both Cortex-M cores at the same time and it works.

Here is the overlay blob for you to download: https://share.toradex.com/4ljtmw74cn8k3sr (it will be available for 3 months).

You can enable it in your overlays.txt file, here is what mine looks like (I just added the HMP overlay on top of the default ones for TorizonCore 6):

torizon@apalis-imx8-06980201:~$ cat /boot/ostree/torizon-fde04e5dc537e7d0fb52b2b3582fcf35a6a4c9a662edf9914167a2bada0feb30/dtb/overlays.txt 
fdt_overlays=apalis-imx8_hdmi_overlay.dtbo apalis-imx8_spi1_spidev_overlay.dtbo apalis-imx8_spi2_spidev_overlay.dtbo apalis-imx8_hmp_overlay.dtbo

Here is the info about my setup:

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           5.15.77-6.2.0+git.aa0ff7e3554e #1-TorizonCore SMP PREEMPT Wed Mar 29 15:33:40 UTC 2023
Kernel command line:      pci=nomsi root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/fde04e5dc537e7d0fb52b2b3582fcf35a6a4c9a662edf9914167a2bada0feb30/0
Distro name:              NAME="TorizonCore"
Distro version:           VERSION_ID=6.2.0-build.2
Hostname:                 apalis-imx8-06980201
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Apalis iMX8QM V1.1 on Apalis Evaluation Board
Toradex version:          0037 V1.1C
Serial number:            06980201
Processor arch:           aarch64
------------------------------------------------------------

And finally, here is the overlay source code:

// Enable RPMSG support for Apalis iMX8

/dts-v1/;
/plugin/;

/ {
	compatible = "toradex,apalis-imx8-v1.1",
		     "toradex,apalis-imx8",
		     "fsl,imx8qm";
};

&{/} {
	reserved-memory {

		#address-cells = <2>;
		#size-cells = <2>;
		ranges;

		vdev0vring0: vdev0vring0@90000000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90000000 0 0x8000>;
			no-map;
		};

		vdev0vring1: vdev0vring1@90008000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90008000 0 0x8000>;
			no-map;
		};

		vdev1vring0: vdev1vring0@90010000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90010000 0 0x8000>;
			no-map;
		};

		vdev1vring1: vdev1vring1@90018000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90018000 0 0x8000>;
			no-map;
		};

		vdev2vring0: vdev0vring0@90100000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90100000 0 0x8000>;
			no-map;
		};

		vdev2vring1: vdev0vring1@90108000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90108000 0 0x8000>;
			no-map;
		};

		vdev3vring0: vdev1vring0@90110000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90110000 0 0x8000>;
			no-map;
		};

		vdev3vring1: vdev1vring1@90118000 {
			compatible = "shared-dma-pool";
			reg = <0 0x90118000 0 0x8000>;
			no-map;
		};

		vdevbuffer: vdevbuffer {
                        compatible = "shared-dma-pool";
			reg = <0 0x90400000 0 0x100000>;
			no-map;
		};
	};

	rpmsg0: rpmsg@0 {
		compatible = "fsl,imx8qm-rpmsg";
		mbox-names = "tx", "rx", "rxdb";
		mboxes = <&lsio_mu5 0 1
			  &lsio_mu5 1 1
			  &lsio_mu5 3 1>;
		mub-partition = <3>;
		vdev-nums = <2>;
		reg = <0 0x90000000 0 0x20000>;
		memory-region = <&vdevbuffer>, <&vdev0vring0>, <&vdev0vring1>,
				<&vdev1vring0>, <&vdev1vring1>;
		status = "okay";
	};

	rpmsg1: rpmsg1 {
		compatible = "fsl,imx8qm-rpmsg";
		mbox-names = "tx", "rx", "rxdb";
		mboxes = <&lsio_mu6 0 1
			  &lsio_mu6 1 1
			  &lsio_mu6 3 1>;
		mub-partition = <4>;
		vdev-nums = <2>;
		reg = <0 0x90100000 0 0x20000>;
		memory-region = <&vdevbuffer>, <&vdev2vring0>, <&vdev2vring1>,
				<&vdev3vring0>, <&vdev3vring1>;
		status = "okay";
	};
};

&iomuxc {
	pinctrl-0 = <&pinctrl_cam1_gpios>, <&pinctrl_dap1_gpios>,
		    <&pinctrl_esai0_gpios>, <&pinctrl_fec2_gpios>,
		    <&pinctrl_gpio3>, <&pinctrl_gpio4>, <&pinctrl_gpio_keys>,
		    <&pinctrl_gpio_usbh_oc_n>,
		    <&pinctrl_lvds0_i2c0_gpio>, <&pinctrl_lvds1_i2c0_gpios>,
		    <&pinctrl_mipi_dsi_0_1_en>, <&pinctrl_mipi_dsi1_gpios>,
		    <&pinctrl_mlb_gpios>, <&pinctrl_qspi1a_gpios>,
		    <&pinctrl_sata1_act>, <&pinctrl_sim0_gpios>,
		    <&pinctrl_usdhc1_gpios>;
};

&lpuart2 {
	status = "disabled";
};

&lsio_pwm0 {
	status = "disabled";
};

&lsio_pwm1 {
	status = "disabled";
};

The iomuxc, lpuart, and PWM parts are disabled to make the UART that comes from the Cortex-M’s enabled all the time, so you can still see the debug messages from Cortex-M 0 and 1 all the time, while Linux is still running.

After loading the overlay and the multicore demo from NXP, you can simply run:

$ sudo modprobe imx_rpmsg_tty

To load the TTY RPMSG demo inside Linux. You will see a “hello-world” message on the cortex M side and you will also notice that now you have the rpmsg devices enabled now:

torizon@apalis-imx8-06980201:~$ ls /dev/ | grep -i rpmsg
rpmsg_ctrl0
rpmsg_ctrl1
rpmsg_ctrl2
rpmsg_ctrl3
ttyRPMSG30
ttyRPMSG31

About this question, they’re enabled. You can check inside your module by running

$ zcat /proc/config.gz | grep -i rpmsg

This will show the configs related to rpmsg that are enabled by default on TorizonCore.

Let me know if you have any questions.

Best Regards,
Hiago.

6

Hi @hfranco.tx,

Thank you for the effort.
That strikes me as odd.

`Kernel version:

5.15.77

-6.2.0+git.aa0ff7e3554e #1-TorizonCore SMP PREEMPT Wed Mar 29 15:33:40 UTC 2023`

Do I understand correctly that the kernel version used here is ‘5.15.77’, which corresponds to this one devicetree soure file:

https://git.toradex.com/cgit/linux-toradex.git/tree/arch/arm64/boot/dts/freescale/imx8qm.dtsi?h=toradex_5.15-2.1.x-imx

?
Which DeviceTree sources are used here?
Then thre is no rpmsg node to set status okay.

Best
Gerald

7

Hi @gerko,

For TorizonCore 6, the kernel version is 5.15.77. There is no rpmsg node on this device tree, that’s why I had to create it from scratch on my overlay.

Please check Embedded Linux Release Matrix | Toradex Developer Center for more information about the versions being used.

Best Regards,
Hiago.

8

OK, I could have figured that out myself.
If only because the ‘&’ in front of rpmsg0|1 is missing in the overlay.
I will test it and hope to now have access to the cntrl devices

…
Your overlay works fine for me.

To get /dev/ttyRPMSG30 and/or /dev/ttyRPMSG31,
have you loaded the corresponding sample binaries onto the M4 cores.

That’s good so far.

We now also have the rpmsg_ctrl0 … 3 control files.
Unfortunately, NXP does not have any examples that I know of that use the character device versions.
I’m already failing, like this:

   struct rpmsg_endpoint_info info = { "DIAG_CNTL", 0, 0 };
   int fd = open("/dev/rpmsg_ctrl0", O_RDWR);
   ioctl(fd, RPMSG_CREATE_EPT_IOCTL, &info);

create an endpoint
I hope I made myself clear.

Best
Gerald

9

Hi @gerko,

Can you please check the ping pong example from NXP? Probably they use the character device on the ping pong example.

There is the ping pong demo inside the cortex-m code and also on the Linux kernel, here: imx_rpmsg_pingpong.c « rpmsg « drivers - linux-toradex.git - Linux kernel for Apalis, Colibri and Verdin modules

Best Regards,
Hiago.

10

OK.
Here some screenshots:
The ping_pong example is loaded at boot time.
Then modprobed the ping_pong kernel module …

ss_01
ss_011378×448 34.2 KB

At M4 serial console:

RPMSG Ping-Pong FreeRTOS RTOS API Demo Task started...
RPMSG Share Base Addr is 0x90110000
Link is up!
app_srtm: I2C service registered
Nameservice announce sent.
Waiting for ping...
Sending pong...
Waiting for ping...
...
Ping pong done, deinitializing...
Looping forever...

The ping pong example works.
After the 100 ping-pongs, the kernel module says goodbye.
I want to have a simple device like writing to or reading from a file, that appears e.g. under /dev/rpmsg_chrx.

There are OpenAMP Examples for STM e.g.:
Linux_RPMsg_framework_overview#RPMsg_char or TI. But we (NXP) uses RPMSG_Lite.

My question again: Do you have any idea why I can’t access the rpmsg_ctrl0…3?

Best
Gerald

11

Hi @gerko,

Good to know this is working now.

Yes, unfortunately, NXP chose to use its own implementation of rpmsg, which it’s different from the upstream kernel approach (OpenAMP). You can find the implementation of rpmsg-lite here: GitHub - nxp-mcuxpresso/rpmsg-lite: RPMsg implementation for small MCUs

I don’t know exactly how the NXP implementation works, in this case, I can share the source codes with you so you can have a look at them and understand why and how these devices are being used:

Unfortunately, I don’t have specific information about them, you should be able to use the char device in the same way the ping pong example is using.

Best Regards,
Hiago.

12

I came up with a few approaches to solve this issue:

one:
I created a kernel module that is a mixture of pingpong and a simple characterdevice driver.
It even worked both ways. However, very simple and with weak points.

other one:
Well, after some trial and error, I was able to create one or more endpoint devices from a docker application via /dev/rpmsg_ctrl0 if I manually run chmod a+rw /dev/rpmsg_ctrl0 beforehand.
I think chown torizon:torizon works too.

However, I had no access to the devices (/dev/rpmsg0 …) that were then created.
There is therefore a massive rights problem. I suspect that there is a ‘correct’ and simple solution for this.

Can someone help?

13

Hi @gerko,

Unfortunately, we don’t have any examples than drivers and MCUXpresso code that comes from NXP. We simply enable them in our downstream kernel and do a test with the demos, like the ping pong or the echo demo. I’d say these are the best resources (check the drivers and the MCU code) to see how these are implemented and how they access RPMSG.

Best Regards,
Hiago.

14

Hi @gerko,

Hope you are doing well! May I know if you have any updates on this topic?

15

The character device drivers are available under 6.2. However, I cannot use them. I need a working example on MCU side, match NXP RPMSG_lite.

Greetings
Gerald