PiCore, RasPi3B and boot code nfsmount

[CentralWare]

I have a rack built with 23 RasPi cards, 8 of which are Pi3.
For the Pi2 cards, the following works flawlessly: nfsmount=10.0.2.1:/nfs/[Pi ID] opt=nfs tce=nfs
The exact same command on the Pi3 hangs the mount on boot.  (These are headless, so until I open the case and connect a cable, I cannot see what it's complaining about, if anything.)

Now, on a Pi3 which I haven't done the above to yet, manually I get:
sudo mount -t nfs 10.0.2.1:/nfs/[Pi ID] /mnt/nfs       hangs the session
sudo mount -o nolock -t nfs 10.0.2.1:/nfs /mnt/nfs  works
* I was under the impression (old memory) nolock was in the tc-config start-up script, so this doesn't make sense to me, but I haven't yet compared internals.

Any ideas what the differences are in the start-up when using nfsmount on a Pi2 versus a Pi3?

uname -r: 4.4.39-piCore_v7+

Thanks guys!

[patrikg]

I think you need to read this thread. As tips and trix for nfs.

http://forum.tinycorelinux.net/index.php/topic,19913.msg123639.html#msg123639

You can as I, boot the pi3 without any sd card.
And pi1-pi2 with regular raspberry sd card boot and chain boot u-boot and chain boot to nfs.
Here some old thread with some suggestions how to do it with scripts.

http://forum.tinycorelinux.net/index.php/topic,21356.msg133578.html#msg133578

Happy coding.

[CentralWare]

I ran a search on the forum for nfsmount before posting, just in case someone had already come across this issue.  The posts you listed were in the results, but thank you.

[CentralWare]

@bela/curaga:

Please see tc-config after

Code: [Select]

[ -z "$DHCP_RAN" ] && wait4Server $SERVER $NOPING

Within the section after nfs-client start both mount commands should have -o nolock after $MOUNT/mount.

PiCore 8.1.5/8.1.5v7 on RasPi2B/RasPi3B

[curaga]

That shouldn't be necessary. It's used in the busybox mount case, since without the NFS utils there is no locking handler, but if you have nfs-utils installed, locking should work properly.

[CentralWare]

@curaga

7.x/armv6/tcz - no nfs-utils.tcz
7.x/armv7/tcz - no nfs-utils.tcz
8.x/armv6/tcz - no nfs-utils.tcz
8.x/armv7/tcz - no nfs-utils.tcz
9.x/armv6/tcz - no nfs-utils.tcz
9.x/armv7/tcz - no nfs-utils.tcz
* I didn't go any further back

Otherwise I wouldn't have even suggested the post (and/or remastered an image specific to Pi2/Pi3 on our end)

[curaga]

If you don't have nfs-utils installed, it's not possible for the script to go to the nfs-utils case, and mount will be called with -o nolock?

https://github.com/tinycorelinux/Core-scripts/blob/master/etc/init.d/tc-config#L359

[CentralWare]

@curaga,

I'm starting with a fresh image (just to ensure there are no remnants from previous alterations), but here are my results thus far.
Bare in mind I just connected a Pi2 and Pi3 to monitors thus I do not have visual results from the previous testing.

RasPi2B using nfsmount=server:path seems to work perfectly fine (with nolock) with options of tce=nfs opt=nfs
RasPi3B using the exact same image hangs; it LOOKS as though it may be DHCP related (no network available at that point) but does not give up an error.

I'm currently embedding nfs-utils from 5.x/armv6 onto an 8.x/armv7 image as I do not have the means to compile until after this Pi rack is completed; I'll also be adding more visual notes in tc-config to see if I can track down where it's hanging

[CentralWare]

I think I might have (in theory) figured this out.  Chicken and The Egg applies.

Command Line: nfsmount=SERVER:PATH tce=nfs opt=nfs
This line in itself is self-defeating as there's no way to load nfs-utils.tcz if my theory is correct.

• TC-CONFIG: assume nfs-utils is located in LOCAL:/tce/optional which isn't loaded yet.
• AoE, NFS, etc. are called (nfs-utils is still not loaded at this point?)
• TCEDIR is set to SERVER:PATH thus nfs-utils is then loaded AFTER nfsmount takes place IF it's found in SERVER:/path/tce/optional

...so says my educated guess at least 

Cure: Remaster with nfs-utils and its dependencies embedded if locking is needed; otherwise use nolock. Possibly create a backup attempt to connect manually (nolock instead of nfs-client) should the first attempt fail.

Suggestion: For all network based file systems, I'd recommend launching their connection as a background task (ie: /etc/init.d/net_nfs.sh $CMDLINE) and then just monitor for success from within tc-config with a timeout implemented allowing the system to move forward in case AoE, NFS, etc. were to fail connectivity.  Add timeout=## as a boot code to override what ever the default would be.

[patrikg]

Is the option "nolock" the correct one?
Isn't the addr= the correct option to get it working ? To the server ip.

And when i have the problem with nfs, i read lot's of error messages in the syslog.

[CentralWare]

@patrikg

No, it hasn't been necessary to use -o addr=[ServerIP] since as far back as I've been using NFS, but odds are it's available as an option for a specific reason, just not a reason I've come across the need for personally.  -o nolock IF memory serves is an addition to NFS (somewhere in NFS2 days I think) where file locking was added to help prevent Computer "A" from handling a given file which Computer "B" is already using) thus the nolock option disables file locking if the NFS Client being used doesn't support it.

[curaga]

Gerald wrote most of the networking support. IIRC the intention was to either remaster NFS/AoE/etc utils in, or to load them via httplist/similar. With those use cases, everything necessary is on the share, and so booting without them would not be useful.

Anyway, looking forward to results on where it's hanging.

[CentralWare]

@curaga
SO FAR, there are a few issues.

RasPi2B WITHOUT nfs-utils seems to connect to a remote NFS3/NFS4 just fine using nolock

RasPi3B WITHOUT nfs-utils hangs somewhere between wait4server and the mount command with no output

RasPi3B WITH nfs-utils (from repo 5.x/armv6)

• Items in /usr/local/tce.installed aren't launched until AFTER nfsmount causing missing library issues -- corrected manually in tc-config just by launching any files found there; immediately after LO is given its IP address.
• /usr/.../init.d/nfs-client launches sm-notify -q and SM complains that it doesn't know what -q is all about.  I swapped with -f for the time being just to shut it up.
• In 5 trial runs, if the NFS server's hard drives are sleeping, nfs-client timed out on about 20% of the hardware and seemed to hang at $MOUNT; it's approximately a 4-5 minute (or so) silent wait before it's allowed to continue which I had not given it when this thread began.  The nfs share is mounted once the network connection is eventually established. (It's not waiting for input; pressing <ENTER> has no effect.)
• In the same 5 trials, between 4 and 8 RasPi3 devices were launched. This rack of 23 Pi's has a dedicated 24-port GBe managed switch with a CAT-6 GBe upstream. The NFS server is a QNAP NAS with NFS3/NFS4 both enabled and the most basic share settings open to the entire network and dual GBe interfaces. It was assumed this layout should have been more than sufficient for this test, but on average, when launching more than 6 units, at least one of the units cannot hand-shake with the NFS server "fast enough" during each test, which seems to act like a no-answer to the nfs client and it just sits there for a few minutes before moving forward and failing the connection.

* SYSLOG is being added to the mix for further details now that I know I can eventually get into the device after it "hangs"

My next set of tests are Static IP based (preventing the wait time I'm hoping) followed by rearranging tc-config and placing all of the AoE/NFS/etc mounts further down the stream giving the hardware a little more time to settle before relying on networking

[Juanito]

does the same thing happen with RPi3 using the 9.x/armv7 repo?

[CentralWare]

@juanito
I'm about a day away from having 8.x tested and remastered to accommodate some of the issues at hand; then I'll test 9.x (though using nfs-utils from way-back since I won't have the option to compile.)