Questions re: PXE Booting

[althalus]

Right now I'm toying with the prospect of netbooting TC. Right now I'm imagining diskless virtual machines that load extensions into RAM from a TFTP / NFS server, with a persistent /opt file (and maybe backup file?) also on said TFTP / NFS server.

I can see here some pretty solid looking instructions on getting TC to netboot.

With my limited understanding of how TFTP and NFS work, I imagine that loading extensions to RAM would be done by just populating the tftp list file, and then setting up NFS for the /opt directory. (Is it possible to load to directly to RAM from NFS?)

And this is where my current level of knowledge fails me - I'll be honest, I've done no research as yet into how to handle this with NFS, right now I'm just braindumping. How can you have each machine have it's own /opt and it's own backup on an NFS server?

Ideally, it would go something like this:
Machine A boots, gets ip <ip1>.
Machine A connects to nfsserv:/opt/
nfsserv recognises <ip1> and server /nfs/<ip1>/opt/
Machine B boots, gets ip <ip2>.
Machine B connects to nfsserv:/opt/
nfsserv recognises <ip2> and server /nfs/<ip2>/opt/

What I'm imagining here is a system where the OS consists of (lighttpd|cherokee|nginx|other esoteric lightweight webserver capable of wsgi/fcgi), python, django. Web directory stored in /opt, tce-optional dir stored in /opt (ie - extra tcz's for say, django-tagging or markdown libraries) with entries in bootlocal.sh to load these optional tcz's that may differ between servers.'

But in all honesty, have no idea how to make the magic happen on the NFS server.

Maybe if NFS can't do that sort of magic, then have a seperate entry for each netbooting machine in the PXE config would work. But that wouldn't be quite as friendly, particularly if there were enough virtual machines in the system that they didn't all fit on the one screen (Which wouldn't take many machines at all).

I dunno. Just some thoughts. Off to research NFS and see if I can answer my own questions.

[gerald_clark]

You can have a separate PXE config file per MAC address.
You specify in the PXE config file what NFS directory is mounted as /mnt/nfs.
This can ( should ) be a different in each config file.
/opt is in RAM, but is restored from mydata.tcz in the backup on that NFS share.
All extensions in the tce directory in that share are automatically loaded.
The bootlocal.sh can mount any additional directories as desired.

TCL acts just like a local drive is installed.  The difference is that the drive (NFS)
is mounted at /mnt/nfs instead of /mnt/hdaX or /mnt/sdaX.

[althalus]

Oh? PXE can do that? If it can do so based on IP address , that would be perfect, though mac addresses are workable. Would just need to add another IP to the interface as part of the restore process to prevent DNS headaches. I guess I'll look into that then. I kind of expected to find out that NFS doesn't do what I want it to do, but a man is allowed to indulge in baseless hopes and wishes every now and then.

You can set the path to the restore file with restore= option, yeah? Something like restore=nfs/vm1? That combined with seperate PXE's for each MAC would be perfect solution to my problem.

[gerald_clark]

The config file is looked for in this order:

1. GUID
2. 01-AA-BB-CC-DD-EE-FF ( MAC address )
3. 89ABCDEF ( IP address in HEX )
4. 89ABCDE   ( Start dropping Nybbles )
5. 89ABCD
6. 89ABC
7. 89AB
8. 89A
9. 89
10. 8
11. default

[althalus]

Thanks man, saved me the time trying to find the info. =)

[Kingdomcome]

I am trying/testing PXE + NFS for the very first time in VMs.  Using TC2.5 for server and client side. PXE boot is successful and NFS is being mounted correctly, but the tce/ dir I set up in the nfs mount is not being autodetected during boot.  I can add the "tce=nfs" bootcode to the default pxe config and extensions get loaded as expected.  I am just wondering if this is the expected or desired behavior.

[gerald_clark]

Perhaps you have another empty tce directory that is being found before the nfs one.

[Kingdomcome]

I checked that, did not find any other tce dirs in either VM.  the output of "cat /opt/.tce_dir" showed "/tmp/tce" so auto-detection did not find it. It was only a quick test using a couple of temporary VM.  I plan to put this into production in my network this weekend and will report back with what I find in my permanent setup.

[althalus]

Ok, using the builtin TC_terminal_server according to the wiki, DHCP wasn't working. However, DNSMasq seems to solve both the TFTP and DHCP problem in one hit and works great.

Next is getting DNS working. Full list of installed extensions:
dnsmasq.tcz
filesystems-2.6.29.1-tinycore.tczm
nfs-utils.tcz
portmap.tcz
tcp_wrappers.tczl

Added an entry to /usr/local/etc/exports

However, I can't actually mount anything via NFS.

I can see something is listening on the NFS port via netstat -atp, however the process using port 2049 isn't being displayed, only a  - (dash)

I also can't telnet to the NFS port.

Very bizarre.


EDIT: Also, /proc/fs/nfs/exports is empty, as is everything in /var/lib/nfs/

[gerald_clark]

OK, I just setup a TC to run nfs server.  It is slightly broken, but this will get it working.
In /opt/bootlocal.sh add the following 1 lines.

/usr/local/etc/init.d/nfs-server start
/usr/local/etc/init.d/nfs-server restart

[althalus]

hm, adding those to /opt/bootlocal.sh didn't help. No change. Still time out on connecting to NFS, still empty /proc/fs/nfs/exports, still empty /var/lib/nfs/etab...

EDIT: Found my problem!!
I was playing with /usr/local/etc/exports

This file doesn't get read by anything (and therefore probably shouldn't be created when the extension is installed, and maybe a note that the exports file has to be in /etc is in order - it's incredibly confusing, when almost everything else ends up in /usr/local)

So, once I started editing the correct exports file, things started working. =)

EDIT2: It worked right up until a reboot, and now I can not seem to reproduce what I did to make it work.

EDIT3: Well, with exports set up correctly in /usr/local/etc and /etc it seems to work as expected. It seems a little odd, but hmm...
You can connect locally, but connecting from a seperate virtual machine doesn't seem to work. THAT still times out. Still got a way to go yet. NFS is not apparently as straight forward as they want you to think.

[Kingdomcome]

I had no problems setting up and starting the server after manually loading filesystems-2.6.29.1-tinycore.tczm and a little googling to learn about the exports file syntax. My steps were:
1) edit /etc/exports - added one share /home/tc *(rw,sync,insecure,no_all_squash)
2) export /etc/exports - exportfs -ra
3) start nfs server - /usr/local/etc/init.d/nfs-server start

Mounting the nfs share in another VM was slightly more difficult.  I created the mountpoint /mnt/nfs and attempted to use the mount -t nfs command to manually mount the share.  The mount command appeared to hang, so I stopped it with a Ctrl-C and tried 2 or 3 more times with the same command (and correct syntax). I then edited /etc/fstab to to add the nfs mount and tired to run "mount /mnt/nfs" after which I got the message that the device or resource was busy. Surprisingly, running "ls /mnt/nfs" showed that the share had been mounted successfully.
Im not sure what this exactly means but perhaps the nfs-utils extension needs looking at, I read Jason's comments that it is a carry-over from the 1.x repo, and I see in the info that it still list filesystems-2.6.26.tczm and the dep file does not include the filesystems extensions, I must manually load it before starting the server or any action on the client box.
One question I just thought of while writing this is: Do you need to start the nfs-client daemon before attempting to mount on a client box or should having the modules that are contained in the filesystems extension be enough?  Thanks to all who have worked on nfs and pxe and the integration into the base.

[danielibarnes]

I have NFS mounted right now with 2.5 (in a VMware VM), and I only have the nfs-utils, portmap, and tcp_wrappers extensions loaded. I do not load the filesystems-2.6.29.1-tinycore extension at all. I execute "pidof portmap || portmap" in my scripts to start portmap if it is not already running before I mount any NFS shares.

As an aside, my NFS server is Solaris using ZFS. A simple "zfs set sharenfs=on" enables NFS. I love it. My network setup is:
Solaris - NIS/NFS/TFTP
Tiny Core - DHCP/DNS (via dnsmasq)
This combination makes a solid base for a simple network and works amazingly well.

[gerald_clark]

What I see here is that rpc.mountd does not start.
an nfs-server restart or start-stop-daemon --start --exec /usr/local/sbin/rpc.mountd
and mounts from another machine are immediate.

[althalus]

Ok, and now this morning, everything is working exactly as expected. Perhaps I didn't have the client utils started properly last night on the client...?

EDIT: Seems the client tools need to be started on the server too, as well as the server tools, else things don't work.

I know have a successful PXE virtual server setup.

The host is a linux mint based server running virtualbox (And booted with nohz=off - This was NEEDED to prevent the NFS server freezing up, which subsequently prevented backups or booting diskless vms.).
using uml-utilities, i set up a tunnel, tap0.
Guests are set up with bridged networking, connecting to tap0. Internet access is done by turning on packet forwarding on the host and setting up routes properly for the virtual network.

Guest 1 is a TC server. Dnsmasq for TFTP, DHCP, and DNS for the clients, Extensions are all loaded via tftp.

Each IP address has it's own pxe file specifying it's own NFS directory.

Backups get stored in the NFS directory.

A server boots, gets an IP address, restores the /home and /usr/local/etc/cherokee settings, and then starts cherokee.

Everything works fairly seemlessly now, but NFS was painful to set up.