NFS boot problem on Colibri T20 (LinuxImageV2.6) with bootargs

After having u-boot and the linux kernel built from the toradex sources (git), deployed to flash and booting them successfully on my Colibri T20 (V1.1 C) on Iris (V1.1 A), I’m now trying to setup NFS/TFTP boot. But I ran into some problems, and it does not work yet. – My goal is to run a (Qt) hello world program directly deployed to the NFS rootfs; it would be great if that would finally work.

In short the problem is this: the bootargs are not passed correctly over the the kernel when running nfsboot in u-boot. It still mounts the rootfs from flash as it seems.

What I’ve done so far:

Besides some minor things that confuse me a bit, normal update and boot from flash worked with the self built images. - So, the next step on my todo list was to get NFS/TFTP boot working…

NFS rootfs problem:
Loading the device tree blob / kernel from TFTP worked with a slight modification of the u-boot environment variables (“setenv bootfile zImage” fixed my issue with nfsboot, that calls the dhcp command that – for some reason – automatically downloaded the device tree blob again from TFTP instead of dtb/zImage, I don’t now if this makes sense):

Colibri T20 # setenv serverip 10.0.0.7
Colibri T20 # setenv ipaddr 10.0.0.8
Colibri T20 # setenv ethaddr 00:14:2d:48:8a:58
Colibri T20 # setenv defargs $defargs nfsrootdebug      
Colibri T20 # printenv defargs                    
defargs=vmalloc=128M usb_high_speed=1 nfsrootdebug
Colibri T20 # saveenv
Saving Environment to NAND...
Erasing NAND...
Erasing at 0x280000 -- 100% complete.
Writing to NAND... OK

Colibri T20 # setenv nfsboot_new 'usb start; run setup; setenv bootargs ${defargs} ${mtdparts} ${nfsargs_mod} ${setupargs} ${vidargs}; echo Booting via DHCP/TFTP/NFS (NEW)...; run nfsdtbload; setenv bootfile zImage; dhcp ${kernel_addr_r} && bootz ${kernel_addr_r} - ${dtbparam}' 
                                         
Colibri T20 # setenv nfsargs_mod 'ip=:::::eth0:on root=/colibri_t20_rootfs rw netdevwait'

This successfully downloads the device tree blob and kernel from TFTP, and starts the boot process:

Colibri T20 # run nfsboot_new 
Booting via DHCP/TFTP/NFS (NEW)...
Using asx0 device
TFTP from server 10.0.0.7; our IP address is 10.0.0.28
Filename 'tegra20-colibri-eval-v3.dtb'.
Load address: 0x2000000
Loading: EHCI timed out on TD - token=0x88008d80
T ###  0 Bytes
	 2 KiB/s
done
Bytes transferred = 13292 (33ec hex)
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address 10.0.0.28 (1565 ms)
Using asx0 device
TFTP from server 10.0.0.7; our IP address is 10.0.0.28
Filename 'zImage'.
Load address: 0x1000000
Loading: #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ##############################  0 Bytes
	 1.9 MiB/s
done
Bytes transferred = 4477160 (4450e8 hex)
Kernel image @ 0x1000000 [ 0x000000 - 0x4450e8 ]
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
modrel: bootargs: vmalloc=128M usb_high_speed=1 nfsrootdebug mtdparts=tegra_nand:2m(u-boot)ro,1m(u-boot-env),1m(cfgblock)ro,-(ubi) ip=:::::eth0:on root=/colibri_t20_rootfs rw netdevwait asix_mac=00:14:2d:48:8a:58 consoleblank=0 no_console_suspend=1 console=tty1 console=ttyS0,115200n8 debug_uartport=lsport,0 mem=372M@0M fbmem=12M@372M nvmem=128M@384M video=tegrafb0:640x480-16@60
   Using Device Tree in place at 02000000, end 020063eb

Starting kernel ...

[    0.000000] Linux version 3.1.10-g34c5824 (rel@elo) (gcc version 5.2.1 20151005 (Linaro GCC 5.2-2015.11-2) ) #1 SMP PREEMPT Thu Jun 2 22:26:37 CEST 2016
[    0.000000] CPU: ARMv7 Processor [411fc090] revision 0 (ARMv7), cr=10c5387d
[    0.000000] CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine: Toradex Colibri T20
[    0.000000] Found fbmem: 00c00000@17400000
[    0.000000] Found nvmem: 08000000@18000000
[    0.000000] Tegra reserved memory:
[    0.000000] LP0:                     00000000 - 00000000
[    0.000000] Bootloader framebuffer:  17400000 - 17ffffff
[    0.000000] Bootloader framebuffer2: 00000000 - 00000000
[    0.000000] Framebuffer:             16700000 - 16ffffff
[    0.000000] 2nd Framebuffer:         17000000 - 17ffffff
[    0.000000] Carveout:                18000000 - 1fffffff
[    0.000000] Vpr:                     00000000 - 00000000
[    0.000000] Memory policy: ECC disabled, Data cache writealloc
[    0.000000] Tegra SKU: 8 Rev: A03 CPU Process: 1 Core Process: 2 Speedo ID: 1
[    0.000000] Tegra Revision: A03 prime SKU: 0x8 CPU Process: 1 Core Process: 2
[    0.000000] L310 cache controller enabled
[    0.000000] l2x0: 8 ways, CACHE_ID 0x410000c4, AUX_CTRL 0x7e080001, Cache size: 1048576 B
[    0.000000] PERCPU: Embedded 8 pages/cpu @c0cf6000 s10336 r8192 d14240 u32768
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 90904
[    0.000000] Kernel command line: vmalloc=128M usb_high_speed=1 nfsrootdebug ubi.mtd=ubi root=ubi0:rootfs rootfstype=ubifs ubi.fm_autoconvert=1 mtdparts=tegra_nand:2m(u-boot)ro,1m(u-boot-env),1m(cfgblock)ro,-(ubi) asix_mac=00:14:2d:48:8a:58 consoleblank=0 no_console_suspend=1 console=tty1 console=ttyS0,115200n8 debug_uartport=lsport,0 mem=372M@0M fbmem=12M@372M nvmem=128M@384M video=tegrafb0:640x480-16@60
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 358MB = 358MB total
[    0.000000] Memory: 352840k/352840k available, 28088k reserved, 0K highmem
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
[    0.000000]     DMA     : 0xff000000 - 0xffe00000   (  14 MB)
[    0.000000]     vmalloc : 0xd7800000 - 0xf8000000   ( 520 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xd7400000   ( 372 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0893a68   (8751 kB)
[    0.000000]       .init : 0xc0894000 - 0xc08e5860   ( 327 kB)
[    0.000000]       .data : 0xc08e6000 - 0xc095fcf4   ( 488 kB)
[    0.000000]        .bss : 0xc095fd18 - 0xc09d3668   ( 463 kB)

But the rootfs is not mounted via NFS yet, probably because the bootargs are somehow not passed correctly to the kernel by u-boot?

I’ve added a printf to u-boot-toradex/arch/arm/lib/bootm.c:207 to print out the commandline (bootargs) visible above in the boot log (see modrel: … above). They don’t match the command Kernel command line printed in the boot log afterwards.

Modifying the nfsboot_new to stop before invoking bootz, and printing out the bootargs shows this, which is closer to what’s intended (root set “correctly” to colibri_t20_rootfs):

Colibri T20 # printenv bootargs 
bootargs=vmalloc=128M usb_high_speed=1 nfsrootdebug mtdparts=tegra_nand:2m(u-boot)ro,1m(u-boot-env),1m(cfgblock)ro,-(ubi) ip=:::::eth0:on root=/colibri_t20_rootfs rw netdevwait asix_mac=00:14:2d:48:8a:58 consoleblank=0 no_console_suspend=1 console=tty1 console=ttyS0,115200n8 debug_uartport=lsport,0 mem=372M@0M fbmem=12M@372M nvmem=128M@384M video=tegrafb0:640x480-16@60

My rootfs is hosted on /srv/nfs4/colibri_t20_rootfs, and I’d like to pass that to the root=xx bootarg… Any idea how to accomplish this would be greatly appreciated! I’m new to embedded/linux development!

If you adhere to the following article about your network infrastructure aka DHCP/TFTP/NFS server et. al. setup it should really be as easy as entering ‘run nfsboot’ from within the U-Boot serial debug console.

Here you go with a boot log of such a nfsboot session.

More information about various boot scenarios may be found in the following article.

Thanks for the hint! I’ve seen the first article before but didn’t realize that you use the DHCP server to configure bootfile filename and NFS rootfs path, never saw that before actually! I’ll give that a try, and install a second ethernet card just for development, just to do this properly…

Hi. I’ve installed a second ethernet card, and got a step further: the dtb and zImage are now loaded when calling ‘run nfsboot’ in u-boot. However, I can’t mount the rootfs yet via NFS (still flash ubi fs).

So I’ve collected some info on my current setup…

That’s the slightly modified /etc/dhcpd.conf from the documentation, used with a dhcpd server on the arch linux host (I don’t know if all those default settings are required/correct):

# option definitions common to all served networks...
option domain-name "colibri.net";
option domain-name-servers ns1.example.org;

default-lease-time 600;
max-lease-time 7200;

# Use this to enable / disable dynamic dns updates globally.
ddns-update-style none;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

subnet 192.168.10.0 netmask 255.255.255.0 {
        default-lease-time              86400;
        max-lease-time                  86400;
        option broadcast-address        192.168.10.255;
        option domain-name              "colibri.net";
        option domain-name-servers      ns1.example.org;
        option ip-forwarding            off;
        option routers                  192.168.10.1;
        option subnet-mask              255.255.255.0;
        interface                       enp5s1;
#        interface                       eth1;
        range                           192.168.10.32 192.168.10.254;
}

#MAC address dependent IP assignment, used for the toradex target device
host eval {
        filename                        "zImage";
        fixed-address                   192.168.10.2;
        hardware ethernet               00:14:2d:48:8a:58;
#        hardware ethernet               00:14:2d:49:79:2c;
        next-server                     192.168.10.7;
        option host-name                "colibri";
        option root-path                "colibri_t20_rootfs,v4,tcp,clientaddr=0.0.0.0";
#        option root-path                "192.168.10.1:/srv/nfs/rootfs,wsize=1024,rsize=1024,v3";
}

Those are the network devices on the host:

$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp5s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 80:1f:02:4c:15:50 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.7/24 brd 192.168.10.255 scope global enp5s1
       valid_lft forever preferred_lft forever
    inet6 fe80::c33b:dc89:e226:f25e/64 scope link 
       valid_lft forever preferred_lft forever
3: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:26:18:a9:0e:d7 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.7/24 brd 10.0.0.255 scope global enp2s0
       valid_lft forever preferred_lft forever
    inet6 fe80::226:18ff:fea9:ed7/64 scope link 
       valid_lft forever preferred_lft forever

And the shortened bootlog, after running ‘run nfsboot’ directly:

U-Boot SPL 2015.04-dirty (Jun 03 2016 - 12:33:23)


U-Boot 2015.04-dirty (Jun 03 2016 - 12:33:23)

TEGRA20
DRAM:  512 MiB
NAND:  1024 MiB
MMC:   Tegra SD/MMC: 0
In:    serial
Out:   lcd
Err:   lcd
Model: Toradex Colibri T20 256MB V1.1C, Serial# 04747858
Net:   Net Initialization Skipped
No ethernet found.
Hit any key to stop autoboot:  0 
Colibri T20 # 
Colibri T20 # run nfsboot
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
USB1:   USB EHCI 1.00
scanning bus 1 for devices... 1 USB Device(s) found
USB2:   USB EHCI 1.00
scanning bus 2 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
       scanning usb for ethernet devices... 
Warning: asx0 using MAC address from net device
1 Ethernet Device(s) found
Booting via DHCP/TFTP/NFS...
Waiting for Ethernet connection... done.
Using asx0 device
TFTP from server 192.168.10.7; our IP address is 192.168.10.2
Filename 'tegra20-colibri-eval-v3.dtb'.
Load address: 0x2000000
Loading: EHCI timed out on TD - token=0x8008d80
T ###  0 Bytes
	 1000 Bytes/s
done
Bytes transferred = 13292 (33ec hex)
BOOTP broadcast 1
DHCP client bound to address 192.168.10.2 (953 ms)
Using asx0 device
TFTP from server 192.168.10.7; our IP address is 192.168.10.2
Filename 'zImage'.
Load address: 0x1000000
Loading: #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ##############################  0 Bytes
	 1.9 MiB/s
done
Bytes transferred = 4477160 (4450e8 hex)
Kernel image @ 0x1000000 [ 0x000000 - 0x4450e8 ]
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
modrel: bootargs: vmalloc=128M usb_high_speed=1 nfsrootdebug mtdparts=tegra_nand:2m(u-boot)ro,1m(u-boot-env),1m(cfgblock)ro,-(ubi) ip=:::::eth0:on root=/dev/nfs rw netdevwait asix_mac=00:14:2d:48:8a:58 consoleblank=0 no_console_suspend=1 console=tty1 console=ttyS0,115200n8 debug_uartport=lsport,0 mem=372M@0M fbmem=12M@372M nvmem=128M@384M video=tegrafb0:640x480-16@60
   Using Device Tree in place at 02000000, end 020063eb

Starting kernel ...

[    0.000000] Linux version 3.1.10-g34c5824 (rel@elo) (gcc version 5.2.1 20151005 (Linaro GCC 5.2-2015.11-2) ) #1 SMP PREEMPT Thu Jun 2 22:26:37 CEST 2016
[    0.000000] CPU: ARMv7 Processor [411fc090] revision 0 (ARMv7), cr=10c5387d
[    0.000000] CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine: Toradex Colibri T20
[    0.000000] Found fbmem: 00c00000@17400000
[    0.000000] Found nvmem: 08000000@18000000
[    0.000000] Tegra reserved memory:
[    0.000000] LP0:                     00000000 - 00000000
[    0.000000] Bootloader framebuffer:  17400000 - 17ffffff
[    0.000000] Bootloader framebuffer2: 00000000 - 00000000
[    0.000000] Framebuffer:             16700000 - 16ffffff
[    0.000000] 2nd Framebuffer:         17000000 - 17ffffff
[    0.000000] Carveout:                18000000 - 1fffffff
[    0.000000] Vpr:                     00000000 - 00000000
[    0.000000] Memory policy: ECC disabled, Data cache writealloc
[    0.000000] Tegra SKU: 8 Rev: A03 CPU Process: 1 Core Process: 2 Speedo ID: 1
[    0.000000] Tegra Revision: A03 prime SKU: 0x8 CPU Process: 1 Core Process: 2
[    0.000000] L310 cache controller enabled
[    0.000000] l2x0: 8 ways, CACHE_ID 0x410000c4, AUX_CTRL 0x7e080001, Cache size: 1048576 B
[    0.000000] PERCPU: Embedded 8 pages/cpu @c0cf6000 s10336 r8192 d14240 u32768
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 90904
[    0.000000] Kernel command line: vmalloc=128M usb_high_speed=1 nfsrootdebug ubi.mtd=ubi root=ubi0:rootfs rootfstype=ubifs ubi.fm_autoconvert=1 mtdparts=tegra_nand:2m(u-boot)ro,1m(u-boot-env),1m(cfgblock)ro,-(ubi) asix_mac=00:14:2d:48:8a:58 consoleblank=0 no_console_suspend=1 console=tty1 console=ttyS0,115200n8 debug_uartport=lsport,0 mem=372M@0M fbmem=12M@372M nvmem=128M@384M video=tegrafb0:640x480-16@60
(...)

Full bootlog.

And the /etc/exports for NFS v4 server:

/srv/nfs4	10.0.0.0/24(rw,fsid=root,no_subtree_check)
/srv/nfs4	192.168.10.1/24(no_root_squash,no_subtree_check,rw,fsid=root)

On the booted Colibri I can actually mount the root file system like this:

# mount -o rw,vers=4,nolock,hard -t nfs 192.168.10.7:/colibri_t20_rootfs /mnt
[  500.214842] NFS: nfs mount opts='vers=4,nolock,hard,addr=192.168.10.7,clientaddr=192.168.10.2'
[  500.223601] NFS:   parsing nfs mount option 'vers=4'
[  500.228633] NFS:   parsing nfs mount option 'nolock'
[  500.233685] NFS:   parsing nfs mount option 'hard'
[  500.238509] NFS:   parsing nfs mount option 'addr=192.168.10.7'
[  500.244532] NFS:   parsing nfs mount option 'clientaddr=192.168.10.2'
[  500.251034] NFS: MNTPATH: '/colibri_t20_rootfs'
[  500.255637] --> nfs4_try_mount()
[  500.495022] <-- nfs4_try_mount() = 0
root@colibri:~# ls /mnt/
bin    dev    home   media  proc   sbin   tmp    var
boot   etc    lib    mnt    run    sys    usr

While trying to solve the problem, I got the bootargs over to the linux init process at some point by modifying u-boot variables temporarily again. But I cannot reproduce this right now, and it didn’t fully boot because of some NFS v3 <-> v4 conflict, that finally caused a “kernel panic” halt while booting.

It might have something to do with the DHCP server configuration. I was able to modify the the bootargs by changing the “option root-path” in the dhcpd.conf at some point. - Would there be an easy workaround to let u-boot take the root-path/NFS config from a user defined variable instead of DHCP?

Any suggestions what I could try next?

Although I do not really know why, it finally works now. After trying to fix it by changing u-boot stuff, debugging the DHCP config with wireshark and trying out different configs without success, I recompiled u-boot and updated the whole system from SD card again. And now ‘run nfsboot’ (with prior setup of the ipaddr, serverip, ethaddr, defargs variables) works out of the box.

Just for the logs, here the modified dhcpd.conf that does the job now. It might work with another root-path option string, but I leave that for now:

# option definitions common to all served networks...
option domain-name "colibri.net";
option domain-name-servers ns1.example.org;

default-lease-time 600;
max-lease-time 7200;

# Use this to enable / disable dynamic dns updates globally.
ddns-update-style none;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

subnet 192.168.10.0 netmask 255.255.255.0 {
        default-lease-time              86400;
        max-lease-time                  86400;
        option broadcast-address        192.168.10.255;
        option domain-name              "colibri.net";
        option domain-name-servers      ns1.example.org;
        option ip-forwarding            off;
        option routers                  192.168.10.1;
        option subnet-mask              255.255.255.0;
        interface                       enp5s1;
#        interface                       eth1;
        range                           192.168.10.1 192.168.10.254;
}

#MAC address dependent IP assignment, used for the toradex target device
host eval {
        filename                        "zImage";
        fixed-address                   192.168.10.2;
        hardware ethernet               00:14:2d:48:8a:58;
#        hardware ethernet               00:14:2d:49:79:2c;
        next-server                     192.168.10.7;
        option host-name                "colibri-t20";
#       option root-path                "192.168.10.7:/colibri_t20_rootfs";
        option root-path                "192.168.10.7:/rootfs,v4,tcp,clientaddr=0.0.0.0";
#        option root-path                "rootfs,v4,tcp,clientaddr=0.0.0.0";
#        option root-path                "192.168.10.7:/colibri_t20_rootfs,v4,tcp,clientaddr=0.0.0.0";
#        option root-path                "192.168.10.1:/srv/nfs/rootfs,wsize=1024,rsize=1024,v3";
}

The ‘option root-path “rootfs,v4,tcp,clientaddr=0.0.0.0”;’ above works fine as well.

But something is still faulty about the NFS mount. It seems to have both – version 2 and 4 – args present in the nfs mount options visible in the bootlog. It seems to override the version 2 with version 4, by appending conflicting args to that nfs mount opts string:

[    8.474668] Root-NFS: DHCPv4 option 17: rootfs,v4,tcp,clientaddr=0.0.0.0
[    8.489308] NFS: nfs mount opts='vers=2,udp,rsize=4096,wsize=4096,v4,tcp,clientaddr=0.0.0.0,nolock,addr=192.168.10.7'
[    8.514991] NFS:   parsing nfs mount option 'vers=2'
[    8.515065] NFS:   parsing nfs mount option 'udp'
[    8.515104] NFS:   parsing nfs mount option 'rsize=4096'
[    8.515162] NFS:   parsing nfs mount option 'wsize=4096'
[    8.515215] NFS:   parsing nfs mount option 'v4'
[    8.515250] NFS:   parsing nfs mount option 'tcp'
[    8.515288] NFS:   parsing nfs mount option 'clientaddr=0.0.0.0'
[    8.515347] NFS:   parsing nfs mount option 'nolock'
[    8.515385] NFS:   parsing nfs mount option 'addr=192.168.10.7'
[    8.515455] NFS: MNTPATH: 'rootfs'
[    8.515479] --> nfs4_try_mount()
[    8.675782] <-- nfs4_try_mount() = 0
[    8.688205] VFS: Mounted root (nfs4 filesystem) on device 0:14.

But it still successfully mounts it with vers=4:

root@colibri-t20:~# mount
192.168.10.7:/rootfs on / type nfs4 (rw,relatime,vers=4,rsize=4096,wsize=4096,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=0.0.0.0,minorversion=0,local_lock=none,addr=192.168.10.7)

(full bootlog here)

Any help on how to setup the module properly would be great!

Congratulations!

I have to admit I have been there before and while it all seems utterly broken it does actually work at the end (;-p). After all please remember that this is a quite ancient kernel we are talking about.

Thanks :wink: but I’m sorry to disappoint you, I’m somehow facing the same problem as in the beginning again. - I tried to replicate the situation above (by resetting/updating the whole module), but without much luck yet. The system always defaults back to ubifs on flash - again…

(full bootlog here)

After all please remember that this is a quite ancient kernel we are talking about.

That’s good to know. I need to find out where the source of that problem lies. Is it the kernel, bootargs, nfs server, dhcp server…? I still don’t really have a clue. What’s really puzzling me is that the kernel command line somehow says root=ubi0:rootsfs, but u-boot seams to pass root=/dev/nfs… I need to find out how to turn on u-boot debugging output tomorrow… Do you have an idea on how to determine where this strange behavior comes from? Should switch over to another linux image instead of 2.6 (which is beta IIRC)?

Ok, I think it becomes clearer where the problem with NFS boot stems from. It has to do with U-boot not passing the right kernel parameters from NFS boot.

In u-boot-toradex/arch/arm/lib/bootm.c in boot_prep_linux(), the commandline variable points to the u-boot bootargs string (printed out above in the bootlogs), which then gets added to a list of “boot tags” (ATAGS) in setup_commandline_tag(). But when (IMAGE_ENABLE_OF_LIBFDT && images->ft_len) is true, only the flattened device tree gets used and setup_commandline_tag() will never be called!

Booting from flash / ubifs calls setup_commandline_tag() as expected, and passes over the kernel boot parameters (I’ve added some printfs to verify that). - But when I ‘run nfsboot’ on the freshly updated Colirbi, it downloads the DTB via TFTP (tegra20-colibri-eval-v3.dtb (wrong filename for my Colibri T20/Iris combo, I’ve just renamed the self built DTB to that)), and then never passes the right Kernel command line args, so it will never mount the rootfs via NFS.

A workaround that seems to help, is to modify the nfsboot u-boot command and remove ‘run nfsdtbload’, so that the DTB is not loaded at all. - It still boots fine (probably by taking it from flash?).

Colibri T20 # printenv nfsboot 
nfsboot=usb start; run setup; setenv bootargs ${defargs} ${mtdparts} ${nfsargs} ${setupargs} ${vidargs}; echo Booting via DHCP/TFTP/NFS...; run nfsdtbload; dhcp ${kernel_addr_r} && bootz ${kernel_addr_r} - ${dtbparam}

Colibri T20 # printenv nfsdtbload
nfsdtbload=setenv dtbparam; tftp ${fdt_addr_r} ${soc}-colibri-${fdt_board}.dtb && setenv dtbparam ${fdt_addr_r}

Colibri T20 # setenv nfsboot_new 'usb start; run setup; setenv bootargs ${defargs} ${mtdparts} ${nfsargs} ${setupargs} ${vidargs}; echo Booting via DHCP/TFTP/NFS... (mod); dhcp ${kernel_addr_r} && bootz ${kernel_addr_r} - ${dtbparam}'

Colibri T20 # run nfsboot_new

(full bootlog here)

I hope this makes sense, I don’t have much experience with all that and try to learn more about it right now…

Since that bootm.c u-boot source file is probably totally generic and has not been adapted for the Toradex hardware, it still confuses me quite a bit. Why should the command line args not be passed when a device tree blob is loaded… I don’t really get it.

Please have a look into this… Any advice to fix this, also for others, in the Toradx git sources would be great!!

@u-boot debug output:
It was not possible to enable u-boot debug printouts. I added a #define DEBUG at top of /u-boot-toradex/include/configs/tegra-common.h, to see all the debug("…") messages in u-boot, and tried to rebuild it, but I get this:

u-boot-toradex $ make -j3 2>&1 | tee build.log
scripts/kconfig/conf --silentoldconfig Kconfig
  CHK     include/config.h
(...)
arm-linux-gnueabihf-ld.bfd: SPL image plus BSS too big
scripts/Makefile.spl:207: recipe for target 'spl/u-boot-spl' failed
make[1]: *** [spl/u-boot-spl] Error 1
Makefile:1252: recipe for target 'spl/u-boot-spl' failed
make: *** [spl/u-boot-spl] Error 2

Ok, well yes, our downstream kernel does not use a device tree at all by default so just leaving it away will also help.

Sorry, that I did not notice you passing one before.

That’s good to know, thanks for the quick reply!

It would be great to leave a note in the Toradex documentation to save others all the trouble. E.g. here: http://developer.toradex.com/knowledge-base/linux-booting#prepare-kernel-and-rootfs:

Copy the kernel and if applicable the device-tree to the tftp directory.Make sure that the kernel file name matches with the filename field you specified in the DHCP configuration and the device tree name with the name configured in U-Boot.

That was not all too clear to me…

Also, it would great to fix the default u-boot environment in the u-boot-toradex sources, so that ‘run nfsboot’ works out of the box…

BTW, I’ve found some infos that cleared my confusion between ATAGS vs device tree up a bit in the official docs:
https://www.kernel.org/doc/Documentation/arm/Booting