Connman not starting with root on nfs

This is the error:

root@colibri-t30:~# systemctl status connman
● connman.service - Connection service
   Loaded: loaded (/lib/systemd/system/connman.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
Condition: start condition failed at Mon 2017-08-14 16:15:43 UTC; 36s ago
           ConditionKernelCommandLine=!root=/dev/nfs was not met

I have a custom kernel, but I think this not the issue here.

Hi

This is done on purpose, have a look here. With connman enabled the interface over which you serve the rootfs is disconnected and then reconnected when connman starts. The boot process does not survive this.

If you really need connman for other interfaces than the one serving the rootfs you can blacklist the one interface which connman is not to touch and remove the conditional start in the systemd service file.

Max

Actually I don’t really need connman, however /etc/resolv.conf is linked to /var/run/connman/resolv.conf

ls -al /etc/resolv.conf
lrwxrwxrwx    1 root     root            28 Aug 28 13:32 /etc/resolv.conf -> /var/run/connman/resolv.conf

…which does not exist:

ls -al /var/run/connman/resolv.conf
ls: /var/run/connman/resolv.conf: No such file or directory

If I remove /etc/resolv.conf and follow Manual Configuration for Wired Connections to enable DHCP the system boot hangs:

         Starting Network Service...
[  OK  ] Started Network Service.
[  OK  ] Reached target Network.
         Starting Network Name Resolution...
         Starting Permit User Sessions...
[  *** ] (2 of 2) A start job is running for...Name Resolution (15s / 1min 34s)[   30.123093] INFO: task systemd-journa.
[   30.134159]       Not tainted 4.1.41-00005-g3f68dc7c600c-dirty #2
[   30.144258] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[   30.160024] systemd-journal D 806775e4     0   180      1 0x00000000
[   30.170622] Backtrace:
[   30.177080] [<80677418>] (__schedule) from [<806779b8>] (schedule+0x44/0x9c)
[   30.188262]  r10:841d0484 r9:00000000 r8:806782f8 r7:00000002 r6:7fffffff r5:00000000
[   30.204534]  r4:86aba000
[   30.211201] [<80677974>] (schedule) from [<8067a514>] (schedule_timeout+0x150/0x190)
[   30.227409]  r5:00000000 r4:7fffffff
[   30.235402] [<8067a3c4>] (schedule_timeout) from [<806773e0>] (io_schedule_timeout+0x80/0xb8)
[   30.252829]  r9:00000000 r8:806782f8 r7:00000002 r6:7fffffff r5:00000000 r4:9fb21c28
[   30.270149] [<80677360>] (io_schedule_timeout) from [<80678338>] (bit_wait_io+0x40/0x64)
[   30.287592]  r7:00000002 r6:9fb559bc r5:86abbd7c r4:86abbd70
[   30.298142] [<806782f8>] (bit_wait_io) from [<80677e6c>] (__wait_on_bit+0x84/0xb8)
[   30.314939] [<80677de8>] (__wait_on_bit) from [<800b2374>] (wait_on_page_bit+0xd0/0xd8)
[   30.332156]  r9:00000000 r8:0000000e r7:000001ff r6:86abbdc4 r5:00000025 r4:9fb55800
[   30.349398] [<800b22a4>] (wait_on_page_bit) from [<800b24a8>] (filemap_fdatawait_range+0xd0/0x12c)
[   30.367718]  r5:00000001 r4:9ff4a500
[   30.376030] [<800b23d8>] (filemap_fdatawait_range) from [<800b2560>] (filemap_fdatawait+0x5c/0x68)
[   30.394404]  r10:00000000 r9:000081a0 r8:84167440 r7:7fffffff r6:ffffffff r5:00000000
[   30.411852]  r4:001fffff
[   30.419158] [<800b2504>] (filemap_fdatawait) from [<800b3fb4>] (filemap_write_and_wait+0x54/0x7c)
[   30.437487]  r5:841d0484 r4:00000000
[   30.445903] [<800b3f60>] (filemap_write_and_wait) from [<801c9024>] (nfs_wb_all+0x18/0x6c)
[   30.463622]  r7:86abbec0 r6:86abbef8 r5:84167440 r4:841d03b0
[   30.474213] [<801c900c>] (nfs_wb_all) from [<801bda58>] (nfs_setattr+0x19c/0x1b8)
[   30.490992]  r5:84167440 r4:841d03b0
[   30.499263] [<801bd8bc>] (nfs_setattr) from [<8010df1c>] (notify_change+0x16c/0x334)
[   30.516170]  r7:86abbec0 r6:86abbef8 r5:841d03b0 r4:00002068
[   30.518696] [<8010ddb0>] (notify_change) from [<800f1464>] (do_truncate+0x84/0xa8)
[   30.518723]  r10:84167440 r9:841d03b0 r8:00000000 r7:00000000 r6:00200000 r5:86a12180
[   30.518732]  r4:84167440
[   30.518751] [<800f13e0>] (do_truncate) from [<800f1848>] (do_sys_ftruncate+0x120/0x198)
[   30.518771]  r7:00000000 r6:00200000 r5:86a12180 r4:86a12180
[   30.518791] [<800f1728>] (do_sys_ftruncate) from [<800f1938>] (SyS_ftruncate64+0x1c/0x24)
[   30.518816]  r10:00000000 r9:86aba000 r8:8000fa04 r7:000000c2 r6:7ebb66f8 r5:7ebb66fc
[   30.518823]  r4:559c4788
[   30.518853] [<800f191c>] (SyS_ftruncate64) from [<8000f860>] (ret_fast_syscall+0x0/0x3c)
[*     ] (1 of 2) A start job is running for...t User Sessions (18s / no limit)

The system boots again if I remove /etc/systemd/network/wired.network, but then DNS configuration is still missing.

As a Gentoo OpenRC user I am not really familiar with systemd network configuration, so maybe I am just missing something?

Well, if you boot from NFS then the Linux kernel already took care of the IP configuration and any attempt to interfere with that from user space (e.g. doing DHCP) will potentially disrupt the NFS connection to the root file system therefore leading to what you are seeing.

Ok, I see. The kernel is successfully configuring IP:

[    5.713028] Sending DHCP requests ..., OK
[   13.503009] IP-Config: Got DHCP answer from 172.16.0.1, my address is 172.16.4.233
[   13.518884] IP-Config: Complete:
[   13.526149]      device=eth0, hwaddr=00:14:2d:4d:3c:5e, ipaddr=172.16.4.233, mask=255.255.0.0, gw=172.16.0.254
[   13.544404]      host=172.16.4.233, domain=pe.loc, nis-domain=(none)
[   13.555050]      bootserver=0.0.0.0, rootserver=172.16.4.186, rootpath=
[   13.561732]      nameserver0=172.16.0.1

But in the end there is no /etc/resolv.conf and therefore no name resolution:

nslookup google.de
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

nslookup: can't resolve 'google.de'

After copying my hosts /etc/resolv.conf and logging in it works:

nslookup google.de
Server:    172.16.0.1
Address 1: 172.16.0.1 server1.pe.loc

Name:      google.de
Address 1: 216.58.214.99 fra16s05-in-f3.1e100.net
Address 2: 2a00:1450:4001:812::2003 fra16s05-in-x03.1e100.net

Is it possible to boot from NFS AND have name resolution working automatically?

Normally connman takes care of resolv.conf, but since we have to disable connman that does not really work.

I guess connman (or an alternative user space DHCP client) would have to have a special NFS mode in which it does not reconfigure the interface but only takes care of DNS. At least for connman I am not aware of such a mode.

However, the kernels built-in DHCP server seems to also get the DNS server and export it via /proc/net/pnp. You can symlink /etc/resolv.conf

rm /etc/resolv.conf
ln -s /proc/net/pnp /etc/resolv.conf

See also: https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

However, on reboot systemd-tmpfiles links the file back to the connman resolv.conf file, you have to remove/disable the entry in /etc/tmpfiles.d/connman_resolvconf.conf.