Memory layout on an iMX8M Mini

[ksalamun]

Hello,

I have a question regarding the memory layout of the Cortex-A53 and Cortex-M4 CPU platforms on an iMX8M Mini.

We are trying to create a shared memory segment in the TCM between the A-53 and M-4 cores. We followed the documentation provided on the wiki page. The memory areas TCML, TCMU, and DDR for M4 described in the table correspond to the description given in the reference manual. However, we are confused by the description of the DDR memory segment for the A-53 core. According to the reference manual, the start address of the DDR is 0x40000000 for the A-53 core and the table on the wiki page indicates that the start address is 0x00000000. Could you please clarify this and explain to which addresses is the DDR mapped in the A-53 core? Is the TCM included in this mapping?

[hfranco.tx]

Hi @ksalamun,

Checking the Reference Manual, here is the memory map for the A53 cortex:

It starts at 0x000_0000 and goes up to 0xFFFF_FFFF. The difference is that M4 can only access the region 0x4000_0000 to 0xFFFF_FFFF, while the cortex A53 can access the whole memory.

The number “1” and “2” we can see in 0x1_0000_0000 and 0x2_3FFF_FFF respectively is just a correction to enable the cortex A53 to access more than 4GB of RAM, given that cortex A53 is 64 bits and cortex M4 is 32 bits.

Therefore, you can only access regions from 0x4000_0000 to 0xFFFF_FFFF with the cortex M4.

Checking the linker script that comes with the MCUXpresso (MIMX8MM3xxxxx_cm4_ddr_ram.ld), we can see these areas being used:

/* Specify the memory areas */
MEMORY
{
  m_interrupts          (RX)  : ORIGIN = 0x80000000, LENGTH = 0x00000240
  m_text                (RX)  : ORIGIN = 0x80000240, LENGTH = 0x001FFDC0
  m_data                (RW)  : ORIGIN = 0x80200000, LENGTH = 0x00200000
  m_data2               (RW)  : ORIGIN = 0x80400000, LENGTH = 0x00C00000
}

So, if your code is going to DDR instead of TCM memory, it will be allocated starting at 0x8000_0000 it has roughly 16MB of memory allocated. Keep that in mind when you are using the DDR memory with Linux, those areas should be reserved to avoid any conflicts.

Best Regards,
Hiago.

[ksalamun]

Hello,

Thank you for your response. We are aware that we can access only the addresses from 0x4000_0000 to 0xFFFF_FFFF with the Cortex-M4 since it is a 32-bit platform. We also use a modified linker script that allocates the M4 code in the DDR, and we reserved this DDR segment in our device tree overlay.

Still, we don’t understand the following.
The DDR address space on the wiki page:

The DDR address space in the reference manual for Cortex-A53:

The DDR address space in the reference manual for Cortex-M4:

The DDR address space for M4 on the wiki page and on the reference manual match. However, the DDR address space for A53 on the wiki page does not match the address space listed in the reference manual.

For example, according to the reference manual, address space from 0x00000000 to 0x0003FFFF corresponds to Boot ROM:

The address space from the wiki page, according to the reference manual, covers system peripherals and reserved address spaces.

We also examined the memory layout described in the device tree (imx8mm.dtsi) and concluded that it matches the DDR address space description given in the reference manual.

memory@40000000 {
	device_type = "memory";
	reg = <0x0 0x40000000 0 0x80000000>;
};

According to the device tree configuration, the 2GB of RAM starting at the address from the reference manual is reserved.

Are we misinterpreting the table on the wiki page?

[hfranco.tx]

Hi @ksalamun,

I understand your point.
I think there is a problem with our developer article: it’s unclear which address spaces are being used. I will modify this article to improve it with your input.

Here is my understanding of the NXP manuals:

@ksalamun

The DDR address space for M4 on the wiki page and on the reference manual match. However, the DDR address space for A53 on the wiki page does not match the address space listed in the reference manual.

For example, according to the reference manual, address space from 0x00000000 to 0x0003FFFF corresponds to Boot ROM:

You are correct about the BOOT ROM, it starts at address 0x0000_0000 and DDR starts at 0x0000_0000. However, they’re not the same address: according to the NXP manual, the DDR starts at 0x1_0000_0000, which in this case is different from 0x0000_0000. I think there is a misunderstanding with those two address spaces.

I believe that is what the article is trying to show in this part:

Although it’s saying it starts at 0x0000_0000, it’s not the same as the 0x0000_0000 from the Boot ROM.
It says it starts from 0x8_0000_0000 and goes to 0xB_FFFF_FFFF (which is different from 0x0000_0000 to 0xFFFF_FFFF). Here I believe there is a typo, the NXP says it starts from 0x1_0000_0000 instead of 0x8_0000_0000. I’ll check that with the team and correct it.

That is my understanding from the NXP manual: it’s correct to say DDR starts at 0x0000_0000 (0x1_0000_0000) and BOOT ROM starts at 0x0000_0000 at the same time because, physically, they are different. Let me know if I made myself clear.

Best Regards,
Hiago.

[ksalamun]

Hi @hfranco.tx,
Thank you for your response.

We still have some doubts regarding this topic.

Firstly, your answer regarding the typo on the wiki page is confusing us:

Although it’s saying it starts at 0x0000_0000, it’s not the same as the 0x0000_0000 from the Boot ROM.
It says it starts from 0x8_0000_0000 and goes to 0xB_FFFF_FFFF (which is different from 0x0000_0000 to 0xFFFF_FFFF). Here I believe there is a typo, the NXP says it starts from 0x1_0000_0000 instead of 0x8_0000_0000. I’ll check that with the team and correct it.

If the address 0x8_0000_0000 is a typo and it should be 0x1_0000_0000, how to explain the end address of DDR range?
On the wiki page, it is 0xB_FFFF_FFFF, while in the reference manual, the end address of DDR range is 0x2_3FFF_FFFF. Moreover, DDR range from 0x1_0000_0000 to 0xB_FFFF_FFFF would mean that the total size of DDR is ~43 GB, which doesn’t make much sense.

Secondly, according to the reference manual, the DDR consists of two address spaces: 0x4000_0000 - 0xFFFF_FFFF and 0x1_0000_0000 - 0x2_3FFF_FFFF. This configuration implies that DDR can be accessed using 32-bit and 64-bit addressing.
How are DDR memory locations actually addressed by each core?

We presume that addressing is done by one of the two possibilities:

1. A53: 0x4000_0000 - 0xBFFF_FFFF (max. 0xFFFF_FFFF)
   M4: 0x4000_0000 - 0xBFFF_FFFF
   → Physical memory locations are accessed by writing the same 32-bit memory address into the memory controller.

2. A53: 0x1_0000_0000 - 0x1_7FFF_FFFF (max. 0x2_3FFF_FFFF)
   M4: 0x4000_0000 - 0x0xBFFF_FFFF
   → Physical memory locations are accessed by writing different memory addresses to the memory controller. More precisely, M4 and A53 write different 32-bit and 64-bit addresses, respectively, to the memory controller. In this case, if DDR physical address requested by the application from A53 and M4 are the same (e.g. DDR memory page at 0x1_0000), how are these addresses translated to respective 32-bit and 64-bit addresses (0x4001_0000 and 0x1_0001_0000)?
   If the application is accessing the peripheral memory address 0x0000_0000 (boot ROM) and the DDR address “0x0000_0000” (we are not sure if the enquoted address is accessed at 0x4000_0000 or 0x1_0000_0000), how and when are these two addresses differentiated? Is the DDR address “0x0000_0000” somehow translated to 0x4000_0000 or 0x1_0000_0000?

Could you please clarify which of these approaches applies on iMX8M Mini platform? If none of the described approaches applies, could you please explain how is DDR memory addressing from different cores handled?

Thirdly, how to explain the configuration in the device tree, where it says that the RAM address space is from 0x4000_0000 to 0x8000_0000?
This configuration implies that the first approach described above is relevant.
In this case, how is the access to DDR through the memory controller handled - is the DDR addressed by 32-bit addresses or are they translated to 64-bit addresses at some point?

[hfranco.tx]

Hi @ksalamun,

Let me go through your questions. Let me know if something isn’t clear.

@ksalamun

On the wiki page, it is 0xB_FFFF_FFFF, while in the reference manual, the end address of the DDR range is 0x2_3FFF_FFFF. Moreover, DDR range from 0x1_0000_0000 to 0xB_FFFF_FFFF would mean that the total size of DDR is ~43 GB, which doesn’t make much sense.

There was indeed a typo in our article. I already submitted it to our team so they can correct it. The full address for Verdin is 0x1_0000_0000 to 0x2_3FFF_FFFF for cortex A53 and 0x4000_0000 to 0xFFFF_FFFF, which is accessible by A54 and M4.
0xFFFF_FFFF would mean approximately 4GB of addressable memory. Which makes sense now. Not this is only the addressable memory, not the real physical memory that is being used by the system.

@ksalamun

Secondly, according to the reference manual, the DDR consists of two address spaces: 0x4000_0000 - 0xFFFF_FFFF and 0x1_0000_0000 - 0x2_3FFF_FFFF. This configuration implies that DDR can be accessed using 32-bit and 64-bit addressing.
How are DDR memory locations addressed by each core?

A53 can access the regions from 0x1_0000_0000 to 0x2_3FFF_FFFF. Which is 0x1_0000_0000 to 0x1_FFFF_FFFF and 0x2_0000_0000 to 0x2_3FFFF_FFFF. If we add those numbers, we get ~4GB (0x1_0000_0000 → 0x1_FFFF_FFFF) + ~1GB (0x2_0000_0000 → 0x2_3FFF_FFFF) = ~5GB. Here is the 64-bit part, which is only accessible by A53. This makes sense if we take a look at the memory map:

5GB is the maximum addressable by core A53. The M4 core, on the other hand, can address memory from 0x4000_0000 to 0xFFFF_FFFF, which is the 32-bit part.

Now regarding how it’s being translated inside the processor, I’m not sure which device translates it and how it’s done. What we do know is that the DDR controller makes sure to differentiate the 0x1_0000_000 and 0x0000_0000 because they are different addresses (DDR and BOOTROM).

@ksalamun

Thirdly, how to explain the configuration in the device tree, where it says that the RAM address space is from 0x4000_0000 to 0x8000_0000?
This configuration implies that the first approach described above is relevant.
In this case, how is the access to DDR through the memory controller handled - is the DDR addressed by 32-bit addresses or are they translated to 64-bit addresses at some point?

The device tree specifies only the 32-bit part because the other 64-bit DDR is handled directly with the DDR controller. Probably if we take a look into the kernel drivers, the processor will communicate with the DDR controller to check how much memory is available and this information is going to be passed to the kernel dynamically.
Again, unfortunately, it’s hard to say how this is being translated/handled by the processor internally. We don’t have this type of information from NXP. You can ask about it directly at commnunity.nxp.com, so they can provide you with more information related to this part.

Best Regards,
Hiago.

[ksalamun]

Hi @hfranco.tx,
Thank you for your response.

The full address for Verdin is 0x1_0000_0000 to 0x2_3FFF_FFFF for cortex A53 and 0x4000_0000 to 0xFFFF_FFFF, which is accessible by A54 and M4.

We agree with this statement regarding the A-53 core, but the addresses accessible by M4 should be 0x4000_0000 to 0xBFFF_FFFF, according to the reference manual. The information about the memory address range accessible by M4 is essential for the correct configuration of the linker script and the execution of M4 applications.

The device tree specifies only the 32-bit part because the other 64-bit DDR is handled directly with the DDR controller.

The DDR address space in the dts from 0x4000_0000 to 0xC000_0000 (size 0x8000_0000) is consistent with the information from the reference manual. Moreover, according to the Verdin iMX8M Mini datasheet, the DDR memory is 32-bit, and the DDR size is 2 GB, which directly corresponds to the memory specification in the dts. Therefore, we can conclude that only 32-bit addressing is supported for DDR access by A-53 core on the Verdin iMX8M Mini.

[hfranco.tx]

Hi @ksalamun,

@ksalamun

We agree with this statement regarding the A-53 core, but the addresses accessible by M4 should be 0x4000_0000 to 0xBFFF_FFFF, according to the reference manual. The information about the memory address range accessible by M4 is essential for the correct configuration of the linker script and the execution of M4 applications.

I’m sorry, you are correct. It’s from 4000_0000 to BFFF_FFFF:

@ksalamun

The DDR address space in the dts from 0x4000_0000 to 0xC000_0000 (size 0x8000_0000) is consistent with the information from the reference manual. Moreover, according to the Verdin iMX8M Mini datasheet, the DDR memory is 32-bit, and the DDR size is 2 GB, which directly corresponds to the memory specification in the dts. Therefore, we can conclude that only 32-bit addressing is supported for DDR access by A-53 core on the Verdin iMX8M Mini.

You are also correct. One can check section 1.2.2 Memory inside the Mini datasheet.

Best Regards,
Hiago.