Kernel in rootfs /boot

I would like to put the kernel (and the kernel device tree) into the rootfs partition image (rootfs/boot) instead of into the VFAT partition where it is put at in the related section of update.sh. How do I have to patch update.sh to get this done?

Perhaps the simplest approach would not involve modifying update.sh, but rather copying the kernel & device tree into the unpacked rootfs provided in the image tarball (into ‘rootfs/boot’) and modifying the U-boot environment to load these files from the rootfs.

If you put the kernel & dtb files into the image’s unpacked rootfs, update.sh will include them into the root.ext3 rootfs file that it generates.

Then you must modify the U-boot boot variables to load these files from the rootfs rather than the FAT32 boot partition, primarily the ‘emmcboot’ & ‘emmcdtbload’ variables.

@brandon.tx This is what I have discovered recently. When booting u-boot states something about the kernel which is displayed as “Angstrom” (references the kernel packed into the boot partition as I use Ubuntu in /rootfs/boot). However after booting I end up in Ubuntu. I use u-boot from the Apalis TK1 Linux Image. I cannot remember that I already changed the u-boot environment variables there. The variables cannot be set accordingly in there right now. This confuses me. It seems like I have some kind of “multi” boot now. In conclusion I should be fine if I would delete the kernel and dtb integration into the boot partition in the relevant section of update.sh and modify the kernel and dtb loading in u-boot like suggested, right?

Our upcoming V2.7 beta 1 release will integrate regular distro boot capability which I advice you to make use of.

Thanks for the info!

During debugging I adjust the u-boot environment variables in there instead of in the mate-toradex recipe for /include/configs/apalis-tk1.h. In emmcdtbload I need to adjust ${fdt_addr_r} and of load mmc 0:1 (because of another partition layout), ${soc}-apalis-${fdt_board}.dtb (with that one in rootfs/boot/). How get I the ${fdt_addr_r} for my partitioning?

I use the following partitioning:

EMMC_SIZE=$(expr 1024 \* 15642 \* 2)

# Boot partition [in sectors of 512]
BOOT_START=$(expr 1024 \* 4 \* 2)
# Rootfs A partition [in sectors of 512]
ROOTFS_A_START=$(expr 1024 \* 20 \* 2)
# Rootfs B partition [in sectors of 512]
ROOTFS_B_START=$(expr 1024 \* 6020 \* 2)
# Data partition [in sectors of 512]
DATA_START=$(expr 1024 \* 12020 \* 2)

${PARTED} -a none -s ${BINARIES}/mbr.bin unit s mkpart primary fat32 ${BOOT_START} $(expr ${ROOTFS_A_START} - 1 )
# the partition of rootfs+kernel+dtb A and B are greater than the fs size
# on the target the file systems are then grown to the full size of the partitions
${PARTED} -a none -s ${BINARIES}/mbr.bin unit s mkpart primary ext3 ${ROOTFS_A_START} $(expr ${ROOTFS_B_START} \- 1)
${PARTED} -a none -s ${BINARIES}/mbr.bin unit s mkpart primary ext3 ${ROOTFS_B_START} $(expr ${DATA_START} \- 1)
${PARTED} -a none -s ${BINARIES}/mbr.bin unit s mkpart primary ext3 ${DATA_START} $(expr ${EMMC_SIZE} \- 1)

That ${fdt_addr_r} address is an address in relocated RAM memory and has nothing to do with any storage partitioning at all.

You are right. I mixed things up. I should be fine if I just leave ${fdt_addr_r} as is…

I have the kernel (zImage) and the dtb (tegra124-adberg.dtb, etc.) in the boot directory of the rootfs (/rootfs/boot) which is located in the third partition (in the second partition resides another rootfs). I modified the u-boot variables as follows:

Apalis TK1 # printenv
boot_file=**zImage**
emmcargs=ip=off root=/dev/mmcblk0p**3** rw rootfstype=ext3 rootwait
emmcboot=run setup; setenv bootargs ${defargs} ${emmcargs} ${setupargs} ${vidargs}; echo Booting from internal eMMC chip...; run emmcdtbload; load mmc 0:**3** ${kernel_addr_r} **/boot/**${boot_file} && run fdt_fixup && bootm ${kernel_addr_r} - ${dtbparam}
emmcdtbload=setenv dtbparam; load mmc 0:**3** ${fdt_addr_r} **/boot/tegra124-ardbeg.dtb** && setenv dtbparam ${fdt_addr_r}

But when I try to boot I get the following error:

Apalis TK1 # run emmcboot
Booting from internal eMMC chip...
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **

The file system type is ext3 for sure (I checked it after booting into the default uImage with sudo blkid -o list).

How about the following for you?

Apalis TK1 # mmc dev 0 0
switch to partitions #0, OK
mmc0(part 0) is current device
Apalis TK1 # mmc part

Partition Map for MMC device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     8192            32768           bf99e9b6-01     0c
  2     40960           30679040        bf99e9b6-02     83
Apalis TK1 # ls mmc 0:1 
  5471848   uimage 
    49687   tegra124-apalis-eval.dtb 

2 file(s), 0 dir(s)

Apalis TK1 # ls mmc 0:2
<DIR>       1024 .
<DIR>       1024 ..
<DIR>      12288 lost+found
<DIR>       3072 bin
<DIR>       1024 boot
<DIR>       1024 dev
<DIR>       4096 etc
<DIR>       1024 home
<DIR>       5120 lib
<DIR>       1024 media
<DIR>       1024 mnt
<DIR>       1024 proc
<DIR>       1024 run
<DIR>       3072 sbin
<DIR>       1024 sys
<DIR>       1024 tmp
<DIR>       1024 usr
<DIR>       1024 var
<DIR>       1024 srv

Please note that the above is for a regular installation with just two partitions.

If you properly flashed a third partition you should be able to list contents of it from within U-Boot as well.

The partition table is created:

Apalis TK1 # mmc part

Partition Map for MMC device 0  --   Partition Type: DOS                        

Part    Start Sector    Num Sectors     UUID            Type                    
  1     8192            32768           0006423e-01     0c
  2     40960           12288000        0006423e-02     83
  3     12328960        12288000        0006423e-03     83
  4     24616960        7417856         0006423e-04     83

The original boot partition (VFAT partition of the Apalis Image) is empty like I want it to be:

Apalis TK1 # ls mmc 0:1                                                         

0 file(s), 0 dir(s)

The partition 2 does contain the rootfs A (with kernel and dtb in boot/) like expected:

Apalis TK1 # ls mmc 0:2
<DIR>       4096 .
<DIR>       4096 ..
<DIR>      16384 lost+found
               9 README.txt
<DIR>       4096 bin
<DIR>       4096 boot
...

Apalis TK1 # ls mmc 0:2 boot                                                    
<DIR>       4096 .                                                              
<DIR>       4096 ..                                                             
           40698 tegra124-loki-02-cell-battery.dtb
           82446 tegra124-ardbeg.dtb
           27478 tegra124-bonaire.dtb
           82624 tegra124-tn8-e1780-1100-a02-01.dtb
         6227528 zImage
           98447 tegra124-tn8-e1780-1100-a03-01.dtb
           81560 tegra124-ardbeg-e1780-1000-a00-01.dtb
           83296 tegra124-ardbeg-e1780-1000-a03-00-battery.dtb
           83400 tegra124-ardbeg-e1780-1000-a03-00-sata.dtb
           56294 tegra124-laguna-pm358.dtb
           37620 tegra124-vcm30_t124.dtb
           77006 tegra124-tn8-e1784-1101-a00.dtb
           80133 tegra124-tn8-e1784-1100-a00.dtb
           71705 tegra124-tn8-p1761-1470-a00.dtb
           82446 tegra124-ardbeg-e1780-1000-a00-00.dtb
           52378 tegra124-loki-thor195-e2549-a00.dtb
           37624 tegra124-ardbeg-vcm30-t124.dtb
           98667 tegra124-tn8-e1780-1100-a02-battery.dtb
           53469 tegra124-norrin.dtb
           59711 tegra124-e1782_sku1100.dtb
           14001 tegra124-bonaire_sim.dtb
           59661 tegra124-jetson_tk1-pm375-000-c00-00.dtb
<DIR>       4096 extlinux
           66565 tegra124-ardbeg-e1792-1100-a00-01.dtb
           54456 tegra124-loki-e2548-a00.dtb
           99057 tegra124-tn8-e1780-1100-a03-battery.dtb
           62515 tegra124-laguna.dtb
           97334 tegra124-tn8-e1922-1100-a00.dtb
           67451 tegra124-ardbeg-e1792-1100-a00-00.dtb
           73534 tegra124-tn8-p1761-1270-a00-battery.dtb
           74591 tegra124-tn8-p1761-1270-a03-battery.dtb
           73804 tegra124-tn8-p1761-1270-a02.dtb
           83374 tegra124-ardbeg-e1780-1000-a03-00-powerconfig.dtb
           50954 tegra124-foster.dtb
           74060 tegra124-tn8-p1761-1270-a02-battery.dtb
           66549 tegra124-ardbeg-e1780-1000-a03-01.dtb
           98238 tegra124-tn8-e1780-1100-a02.dtb
           90107 tegra124-e1791.dtb
           83304 tegra124-ardbeg-a03-00.dtb
           53469 tegra124-norrin-pm374-0001-a00-00.dtb
           39280 tegra124-thor195.dtb
           13801 tegra124-p2360.dtb
           63701 tegra124-e1923.dtb
           40698 tegra124-loki-03-cell-battery.dtb
           77435 tegra124-tn8-e1784-1101-a00-battery.dtb
           53469 tegra124-norrin-pm374-0001-a01-00.dtb
           98628 tegra124-tn8-e1780-1100-a03.dtb
           54266 tegra124-loki-e2530-a03.dtb
           72134 tegra124-tn8-p1761-1470-a00-battery.dtb
           98876 tegra124-tn8-e1780-1100-a03-01-battery.dtb
           83304 tegra124-ardbeg-e1780-1000-a03-00.dtb
           54254 tegra124-loki.dtb
           54266 tegra124-loki-e2530-a01.dtb
           73105 tegra124-tn8-p1761-1270-a00.dtb
           40727 tegra124-loki-mods.dtb
           82442 tegra124-ardbeg-e1780-1000-a00-00-battery.dtb
           74435 tegra124-tn8-p1761-1270-a03.dtb

I am also able to load the kernel A and dtb A (configured in /boot/extlinux.conf) of rootfs A in partition 2:

Apalis TK1 # load mmc 0:2 ${kernel_addr_r} /boot/zImage
6227528 bytes read in 233 ms (25.5 MiB/s)
Apalis TK1 # load mmc 0:2 ${fdt_addr_r} /boot/tegra124-jetson_tk1-pm375-000-c00-00.dtb
59661 bytes read in 146 ms (398.4 KiB/s)

However the partition 3 seems not to contain the rootfs B like I expected it to be:

Apalis TK1 # ls mmc 0:3
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **

@Florian_K Can you share the commands you used for flashing the rootfs to each partition? The regular ‘update’ scripts we provide for U-boot will only flash the rootfs to the 2nd partition, so they will require some modification.

I adjusted the u-boot environment variables to boot the kernel in the rootfs A in partition 2 (and corrected some errors in the previous rootfs B ~ partition 3):

Apalis TK1 # setenv emmcargs 'ip=off root=/dev/mmcblk0p2 rw rootfstype=ext3 rootwait'
Apalis TK1 # printenv
...
emmcargs=ip=off root=/dev/mmcblk0p2 rw rootfstype=ext3 rootwait
emmcboot=run setup; setenv bootargs ${defargs} ${emmcargs} ${setupargs} ${vidargs}; echo Booting from internal eMMC chip...; run emmcdtbload; load mmc 0:2 ${kernel_addr_r} /boot/zImage && run fdt_fixup && bootz ${kernel_addr_r} - ${dtbparam}
emmcdtbload=setenv dtbparam; load mmc 0:2 ${fdt_addr_r} /boot/tegra124-jetson_tk1-pm375-000-c00-00.dtb && setenv dtbparam ${fdt_addr_r}
Apalis TK1 # savenv

Then I tried to boot into rootfs A, but it does not start the kernel (the process seems to just freezed):

Apalis TK1 # run emmcboot
Booting from internal eMMC chip...
59661 bytes read in 147 ms (395.5 KiB/s)
6227528 bytes read in 233 ms (25.5 MiB/s)
Kernel image @ 0x81000000 [ 0x000000 - 0x5f0648 ]
## Flattened Device Tree blob at 82000000
   Booting using the fdt blob at 0x82000000
   Using Device Tree in place at 82000000, end 8201190c

Starting kernel ...

… with the same result when poweroff/poweron:

U-Boot 2015.04 (Nov 21 2016 - 20:39:53)

TEGRA124
DRAM:  2 GiB
MMC:   Tegra SD/MMC: 0, Tegra SD/MMC: 1, Tegra SD/MMC: 2
tegra-pcie: PCI regions:
tegra-pcie:   I/O: 0x12000000-0x12010000
tegra-pcie:   non-prefetchable memory: 0x13000000-0x20000000
tegra-pcie:   prefetchable memory: 0x20000000-0x40000000
tegra-pcie: 2x1, 1x1 configuration
tegra-pcie: probing port 1, using 1 lanes
In:    serial
Out:   serial
Err:   serial
Missing Toradex config block
Model: Toradex Apalis TK1 2GB
Net:   e1000: no NVM
e1000#0
Error: e1000#0 address not set.

Hit any key to stop autoboot:  0 
Booting from internal eMMC chip...
59661 bytes read in 146 ms (398.4 KiB/s)
6227528 bytes read in 233 ms (25.5 MiB/s)
Kernel image @ 0x81000000 [ 0x000000 - 0x5f0648 ]
## Flattened Device Tree blob at 82000000
   Booting using the fdt blob at 0x82000000
   Using Device Tree in place at 82000000, end 8201190c

Starting kernel ...

Missing Toradex config block should be “restored” I guess? How can I get this done?

@brandon.tx I created rootfs_bext3 by modifying update.sh:

echo ""echo "Creating rootfs partition image A"
#make the partition size size(rootfs used + MIN_PARTITION_FREE_SIZE)
#add about 4% to the rootfs to account for fs overhead. (/1024/985 instead of /1024/1024).
#add 512 bytes per file to account for small files
#(resize it later on target to fill the size of the partition it lives in)
NUMBER_OF_FILES=`sudo find ${ROOTFSPATH} | wc -l`
EXT_SIZE=`sudo du -DsB1 ${ROOTFSPATH} | awk -v min=$MIN_PARTITION_FREE_SIZE -v f=${NUMBER_OF_FILES} \
    	'{rootfs_size=$1+f*512;rootfs_size=int(rootfs_size/1024/985); print (rootfs_size+min) }'`

rm -f ${BINARIES}/${IMAGEFILE}
sudo $LOCPATH/genext3fs.sh -d ${ROOTFSPATH} -b ${EXT_SIZE} ${BINARIES}/${IMAGEFILE} || exit 1

echo ""
echo "Creating rootfs partition image B"
#make the partition size size(rootfs used + MIN_PARTITION_FREE_SIZE)
#add about 4% to the rootfs to account for fs overhead. (/1024/985 instead of /1024/1024).
#add 512 bytes per file to account for small files
#(resize it later on target to fill the size of the partition it lives in)
NUMBER_OF_FILES_B=`sudo find ${ROOTFSPATH_B} | wc -l`
EXT_SIZE_B=`sudo du -DsB1 ${ROOTFSPATH_B} | awk -v min=$MIN_PARTITION_FREE_SIZE -v f=${NUMBER_OF_FILES_B} \
    	'{rootfs_size=$1+f*512;rootfs_size=int(rootfs_size/1024/985); print (rootfs_size+min) }'`

rm -f ${BINARIES}/${IMAGEFILE_B}
# single mount point /mnt/trdx-rootfs set for partition A before will be overriden for partition B
sudo $LOCPATH/genext3fs.sh -d ${ROOTFSPATH_B} -b ${EXT_SIZE_B} ${BINARIES}/${IMAGEFILE_B} || exit 1

And I adjusted flash_eth.scr and updated the u-boot scripts (running mk-u-boot-scripts.sh):

setenv cp_file_chunk 'tftpboot ${loadaddr} ${board_name}/root.ext3-${filenum}; run set_blkcnt; mmc dev 0 0 && mmc write ${loadaddr} ${blkstart} ${blkcnt}'
setenv cp_file_chunk_b 'tftpboot ${loadaddr} ${board_name}/root_b.ext3-${filenum}; run set_blkcnt; mmc dev 0 0 && mmc write ${loadaddr} ${blkstart} ${blkcnt}'
...
setenv update_rootfs 'setenv filesize 4000000; setenv filenum 10; setenv blkstart a000; while test ${filesize} -eq "4000000"; do run cp_file_chunk; run set_nextnum; done'
setenv update_rootfs_b 'setenv filesize 4000000; setenv filenum 10; setenv blkstart a000; while test ${filesize} -eq "4000000"; do run cp_file_chunk_b; run set_nextnum; done'
...
setenv update 'run update_uboot; run update_1; run update_2; run update_rootfs; reset'
setenv update_b 'run update_uboot; run update_1; run update_2; run update_rootfs_b; reset'

The Toradex Apalis TK1 hardware is not the NVIDIA Jetson TK1 so you won’t succeed in booting a kernel and/or device tree from that one.

Concerning the config block have a look at the cfgblock create U-Boot command.

@Florian_K You can use cfgblock create in U-boot to recreate the config block. Perhaps also ensure that the ‘setup’ variable is still defined in U-boot. Is there a reason you decided to use a zImage? Perhaps try loading the uImage instead.

@Florian_K The problem here is that you set blkstart to the same value for both rootfs inside update_rootfs & update_rootfs_b.

I did not decided yet. Right now I did just want to try to boot different image types (namely uImage and zImage). But It should not make any difference, right?

Oh, you are right! Would it work if I would drop the kernel and dtb from the original VFAT partition of the Apalis Image into the rootfs/boot/ directory?

Sure, that should work.