Automount USB with UDEV rules, Problem [SOLVED WITH RESTRICTIONS]

[mindmachine]

Yep, man you did it!!! Here is my mounttime.txt:

tc@box:~$ cat mounttime.txt
01:02:03
sdb
01:02:05
sdb1
Cat returned sdb1
nodev sysfs nodev rootfs nodev bdev nodev proc nodev cgroup nodev cpuset nodev tmpfs nodev sockfs nodev usbfs nodev pipefs nodev anon_inodefs nodev rpc_pipefs nodev devpts nodev ramfs ext3 ext2 ext4 cramfs vfat iso9660 nodev nfs nodev nfs4 nodev fuse fuseblk nodev fusectl nodev pstore nodev mqueue squashfs ntfs

It's mounted r-xr-xr-x atm, though. mount -w doesn't help. Probably no other parameter as well, as I assume, that the driver doesn't support this?

[Rich]

Hi mindmachine
I've been fighting with this for hours and I can't get it to auto mount in RW mode. It's probably something simple
but I'm not seeing it. I'm afraid I'm at my wits end on this one. Maybe someone else has some ideas.

[tinypoodle]

Have you excluded that the read-only kernel driver would be in use?

[Rich]

Hi tinypoodle
Now I have, lsmod shows it is being used. What I don't understand is why:

Code: [Select]

sudo ntfsmount /dev/$i /mnt/$imounts the drive RW from the command line but not when run from the script (with or without sudo).
Running the script:

Code: [Select]

#!/bin/sh
echo `date +"%T"` >> /home/tc/mounttime
echo $1 >> /home/tc/mounttime
rebuildfstab
for i in $(cat /etc/fstab | grep $1 | cut -c 6-9) # we check fstab to avoid getting e.g. sdb1/ returned
do
echo "Cat returned $i" >> /home/tc/mounttime
mntcommand="mount /dev/$i /mnt/$i"
grep $i /etc/fstab | grep -q ntfs
[ 0 -eq $? ] && mntcommand="sudo /usr/local/bin/ntfsmount /dev/$i /mnt/$i"
grep -q $i /etc/mtab
[ 0 -ne $? ] && $mntcommand >>/home/tc/mount.txt 2>&1
donefrom the command line using sudo and passing in sdc1 mounts it RW.

[Rich]

Hi tinypoodle
Just a clarification, when udev calls the script, it mounts RO. Udev rule:

Code: [Select]

ACTION=="add",          SUBSYSTEM=="block",     RUN+="/bin/sh -c '/home/tc/mount.sh %k'"I'm guessing maybe it's how the run command is calling the script.

[mindmachine]

Just a short comment, as I'm away for a 60th birthday the rest of the day....
This is what I experienced before. So using ntfs-3g doesn't mount at all, when using the script, but mounts rw, when used by cmdline. Using the ntfsprogs mounts (at least) ro by script, and rw, when called from cmdline. So this wopuld actually point to the udev-rule. That's where we should focus on. Maybe, we could write some local udev rule that takes care of this problem?

EDIT: Another thing I found out about UDEV is, that it is effectively paused while any commands, started by the "RUN+=" list assignment, are executed.
Source:

http://www.reactivated.net/writing_udev_rules.html#external-run

So it might be, that some udev command missing at this time or being executed after completion of our mount-script will have undesired effect? I#m not at home, but perhaps

sudo udevadm --debug monitor

might spit out something helpful? (Information mostly for me to try this evening :-)!)

[Rich]

Hi mindmachine
I think I have some answers, execute:

Code: [Select]

tc@box:~/tcprovides$ lsmod
Module                  Size  Used by    Tainted: P 
ntfs                   69632  0
cifs                  180224  2
speedstep_lib          12288  0If the ntfs module shows up in the list, make sure you don't have any NTFS drives mounted, and then execute:

Code: [Select]

sudo modprobe -r ntfsThat will remove the RO driver. This version of the script auto mounted RW when I plugged in the drive:

Code: [Select]

#!/bin/sh
echo `date +"%T"` >> /home/tc/mounttime
echo $1 >> /home/tc/mounttime
rebuildfstab
for i in $(cat /etc/fstab | grep $1 | cut -c 6-9) # we check fstab to avoid getting e.g. sdb1/ returned
do
echo "Cat returned $i" >> /home/tc/mounttime
mntcommand="mount /dev/$i /mnt/$i"
grep $i /etc/fstab | grep -q ntfs
[ 0 -eq $? ] && mntcommand="/usr/local/bin/ntfsmount /dev/$i /mnt/$i"
grep -q $i /etc/mtab
[ 0 -ne $? ] && $mntcommand >>/home/tc/mount.txt 2>&1
doneTurns out the mount applet loads the RO driver if you use it to mount an NTFS drive. Why it defaults to the
RO driver when ntfsmount is run from a script called by udev, I don't know.

[mindmachine]

Hi Rich!
Yes, I can confirm, that this solution works. lsmod did not give any results to ntfs, but blackmarking the driver made the device mount as expected. Cool :-)! THANK YOU! I really feel sorry, that my problems, whenever posted in a forum, turn out to be a really pestering annoyance :-). I appreciate your help a lot and thanks for your time (hours, I suppose, if not days??) and effort!!!!!

Remaining problem is, when you plug in an external USB-drive with more than one partition, only the last one will use the ntfsmount command, the rest is mounted the "regular" way, that is "being not mounted at all" :-) . But I will think about this myself, where to include the correct loop until all partitions are mounted.

Next thing is, how do we deal with data integrity problems?  In another post, this problem is mentioned, as far as I do remember this correctly. We could take advantage of ntfscheck in those cases and mount with the -o force parameter afterwards.

[tinypoodle]

Next thing is, how do we deal with data integrity problems?

forcemount and data integrity appear to be mutually exclusive.

We could take advantage of ntfscheck

Where have you seen that? Any reference?

[Rich]

Hi mindmachine

I really feel sorry, that my problems, whenever posted in a forum, turn out to be a really pestering annoyance :-).

One mans pestering annoyance is another mans challenge.

Remaining problem is, when you plug in an external USB-drive with more than one partition, only the last one will use the ntfsmount command, ...

Confirmed, that would be a race condition. Here is a version that fixes it:

Code: [Select]

#!/bin/sh
mypid=$$
echo `date +"%T"`" New PID=$mypid device=$1" >> /home/tc/mounttime
while true
do
sleep 1
otherpid=999999
for i in `pidof mount.sh`
do
[ $i -lt $otherpid ] && otherpid=$i
done
[ $mypid -le $otherpid ] && break
done
echo `date +"%T"`" Current PID=$mypid device=$1" >> /home/tc/mounttime
rebuildfstab
for i in $(cat /etc/fstab | grep $1 | cut -c 6-9) # we check fstab to avoid getting e.g. sdb1/ returned
do
echo `date +"%T"`" Cat returned $i" >> /home/tc/mounttime
mntcommand="mount /dev/$i /mnt/$i"
grep $i /etc/fstab | grep -q ntfs
[ 0 -eq $? ] && mntcommand="/usr/local/bin/ntfsmount /dev/$i /mnt/$i"
grep -q $i /etc/mtab
[ 0 -ne $? ] && $mntcommand >>/home/tc/mount.txt 2>&1
doneWhen the script starts, it retrieves the value of its PID. It then drops into an endless loop that sleeps for one second and
then gets the PIDs of any other copies of mount.sh that are running, and saves the pid with the lowest number. It then
compares its own PID with the lowest found. If its own PID is the lowest, it breaks out of the loop, otherwise it loops
again. I reformatted a thumb drive with four partitions, ntfs, ntfs, vfat32, and ext2. Here are the results when plugged in:

Code: [Select]

tc@box:~$ cat ./mounttime
00:16:09 New PID=31830 device=sdc
00:16:10 Current PID=31830 device=sdc
00:16:12 New PID=32132 device=sdc3
00:16:12 New PID=32133 device=sdc1
00:16:12 New PID=32131 device=sdc4
00:16:12 New PID=32134 device=sdc2
00:16:14 Current PID=32131 device=sdc4
00:16:18 Cat returned sdc4
00:16:19 Current PID=32132 device=sdc3
00:16:21 Cat returned sdc3
00:16:22 Current PID=32133 device=sdc1
00:16:24 Cat returned sdc1
00:16:25 Current PID=32134 device=sdc2
00:16:27 Cat returned sdc2You can see the script being called four times at 00:16:12, followed by each instance of the script executing one at a
time in ascending PID order.

    [EDIT]: Changed script, result of cat command, and updated description.

[Rich]

Hi mindmachine
Before I edited my previous post, the script could still have failed had someone plugged a second device in prior to
all the partitions getting mounted on the first device. The script still has two possible failure modes that I can see.

1. If you look at the results of the cat command, you'll notice the new PIDs don't have to arrive in numerical order. If
it takes more than one second for all the instances of the script to start, a lower numbered PID could sneak in late
and start running before the current instance finishes. I think the chances of that happening are remote, but it is
possible. You could up the sleep command to two seconds if you want a greater safety margin.

2. Though I can't cite a reference, I seem to remember reading that the PID numbers eventually reach a maximum
value and wrap around and recycle unused numbers from the bottom of the range. If you happened to plug in a second
drive just after the PIDs wrap, and the first drive hasn't finished mounting, the script for the second drive will try to
start mounting the second drive. While possible, I think it's highly unlikely.

In both cases, the only result would be unmounted partitions.

[mindmachine]

@ tinypoodle

forcemount and data integrity appear to be mutually exclusive.

That is true, of course 
I'm a bit confused right now, as I vaguely remember me trying to mount a broken ntfs-stick, getting some kind of "boot into Windows TWICE" error and beeing unable to mount. I then ran some command, I remember it to be ntfscheck to fix this problem. Now, doing some research, I can't find any reference to this. Strange, probably me getting old... ... Sorry!

So what about ntfsfix (what most likely has been the command I used??!!) and the -o force command afterwards? Would ntfsfix be mighty enough to clear the worst problems, keep data integrity on the stick and make it mountable. We could then grep the output of a mount command (like mounttime.txt) for "TWICE" (or whatever) and run ntfsfix and try to mount the drive with -o force afterwards.
Would that be a practicable solution?  How do other distributions deal with these problems? In my strictly personal opinion, a working procedure for USB-automount would be an improvement, wouldn't it? How is this handled, when using the GUI-mount/unmount feature? This problem would occur there as well, if you do have a non integer device, right?

@Rich
Cool! 
I agree that a situation as described is quite unlikely. For my purposes, this solution would be fully sufficient and worst case, a device not mounting, would not be a show stopper at all.
For a reason of safety, we could build in a delay, before we start to run the script. If we add "sleep 2" before we start, we would in any case avoid mixing up the mount procedures. As I can see from your output, creation of the process PIDs for your sdc-device is handled within a second. Would a simple delay be an option? We could also check for existing mount processes at start and only delay and wait for the first (second, third..) process to finish, if we get a positive response.

[tinypoodle]

@ tinypoodle

forcemount and data integrity appear to be mutually exclusive.

That is true, of course 
I'm a bit confused right now, as I vaguely remember me trying to mount a broken ntfs-stick, getting some kind of "boot into Windows TWICE" error and beeing unable to mount. I then ran some command, I remember it to be ntfscheck to fix this problem. Now, doing some research, I can't find any reference to this. Strange, probably me getting old... ... Sorry!

I can remember having read about plans of a ntfsfsck utility long before ntfs-3g forked from ntfsprogs, but nothing has ever been realized. Simple fact there is no such a thing for Linux.

So what about ntfsfix (what most likely has been the command I used??!!) and the -o force command afterwards? Would ntfsfix be mighty enough to clear the worst problems, keep data integrity on the stick and make it mountable.

Not safely, IMHO

We could then grep the output of a mount command (like mounttime.txt) for "TWICE" (or whatever) and run ntfsfix and try to mount the drive with -o force afterwards.
Would that be a practicable solution?

Highly risky.
Have a look at the man page
http://linuxmanpages.net/manpages/fedora17/man8/ntfsfix.8.html
and if you are still not convinced then have a look at the source code of ntfsfix.
It's main achievement is to mark the volume for windows to run chkdisk at next boot.

How do other distributions deal with these problems?

Not sure why you talk about a problem.
Good practice under UNIX-like operating systems says to mount filesystems only when and as long as needed.
That is exactly where automount could play in (not sure if it would be compatible with core) - which has nothing to do with forcemounting.
http://linux.die.net/man/8/automount
http://tldp.org/HOWTO/Automount-1.html

Another good practice says to mount file systems where writing is not a need read-only. A common example would be to have a dedicated partition for /usr (required to be mounted at boot!) mounted read-only.

In my strictly personal opinion, a working procedure for USB-automount would be an improvement, wouldn't it?

With traditional scatter install distros, often the root partition gets first mounted read-only and remounted read-write after a fsck at boot time.
I guess there could be a way to implement a similar routine for removable storage which gets attached at runtime, but I can't see a big need for it.

With the emerge of udev there was a project "usbmount".
It is unmaintained and last development took place in 2005...
http://usbmount.alioth.debian.org/#history

How is this handled, when using the GUI-mount/unmount feature? This problem would occur there as well, if you do have a non integer device, right?

Only if a user actively mounts a non integer device.
What makes all the difference is that with any mount facilitation, be it by GUI, automount, or simply setting "mount" SUID, it is in the full capacity and responsibility of the user if they want a filesystem to be checked first or to mount it read-only or read-write.

[mindmachine]

Thanks for clarification!!

Good practice under UNIX-like operating systems says to mount filesystems only when and as long as needed.

That is true and I fully agree. In my case, I do use TC as a plattform to run rdesktop to access WIN7 running in virtualbox on an linux server (my wife is using it). So to be able to also use USB-drives, I've been looking for a possibility to handle USB-sticks plugged into the Thinclient. These are handed over by network and should be rw.
Maybe I'm completely on a wrong trail to achieve this goal?? There must be some way to make these locally attached drives accessible, but how?

[Rich]

Hi mindmachine
After some thought I decided to approach the problem from a different angle. Rather than trying to fight the
asynchronous nature of the events calling the script multiple times, I changed the script so that it is always running
and handles events synchronously:

Code: [Select]

#!/bin/sh
QUEUE=/tmp/automountqueue
[ ! -e "$QUEUE" ] && mkfifo $QUEUE

while true
do
for i in `cat $QUEUE`
do
echo `date +"%T"`" Request to mount $i" >> /home/tc/mounttime
rebuildfstab
i=$(cat /etc/fstab | grep -w $i | cut -c 6-9) # we check fstab to avoid getting e.g. sdb1/ returned
echo `date +"%T"`" Cat returned $i" >> /home/tc/mounttime
[ -z $i ] && continue
echo `date +"%T"`" Mounting $i" >> /home/tc/mounttime
mntcommand="mount /dev/$i /mnt/$i"
grep $i /etc/fstab | grep -q ntfs
[ 0 -eq $? ] && mntcommand="/usr/local/bin/ntfsmount /dev/$i /mnt/$i"
grep -q $i /etc/mtab
[ 0 -ne $? ] && $mntcommand >>/home/tc/mount.txt 2>&1
done
echo `/usr/local/bin/date +"%T"`" Done" >> /home/tc/mounttime
doneThe result after plugging in a thumb drive are:

Code: [Select]

tc@box:~$ cat mounttime
16:00:21 Request to mount sdc
16:00:25 Cat returned
16:00:25 Done
16:00:25 Request to mount sdc2
16:00:27 Cat returned sdc2
16:00:27 Mounting sdc2
16:00:28 Request to mount sdc1
16:00:30 Cat returned sdc1
16:00:30 Mounting sdc1
16:00:30 Request to mount sdc3
16:00:32 Cat returned sdc3
16:00:32 Mounting sdc3
16:00:33 Request to mount sdc4
16:00:34 Cat returned sdc4
16:00:34 Mounting sdc4
16:00:35 DoneSDC is skipped, and SDC2, SDC1, SDC3, and SDC4 are processed one at a time, no race conditions.
The add rule in  /etc/udev/rules.d/98-tc.rules  looks like this:

Code: [Select]

ACTION=="add", SUBSYSTEM=="block", RUN+="/bin/sh -c '/bin/echo %k >> /tmp/automountqueue'"The script must be run as root, and can be started automatically by your bootlocal.sh script.