Getting double folders - /usr/ and /usr_1/ on custom tcz

[onelife]

Hi there Forum,

I seem to be going in circles with making a custom TCZ that will place a boot script in the /usr/local/tce.installed/ directory that runs md5sum check on the mydata.tgz backup file.

The issue is not that the tcz isn't loading, but for some reason I am getting a 2nd copy in a duplicate foler called /usr_1/ - seems very odd.

Example :

snb@snb:~$ md5sum /usr_1/local/tce.installed/md5mydata
baffab92405c6f437de2e7653dc523b4  /usr_1/local/tce.installed/md5mydata

snb@snb:~$ md5sum /usr/local/tce.installed/md5mydata
baffab92405c6f437de2e7653dc523b4  /usr/local/tce.installed/md5mydata

Both folders have my exact script and md5 shows the files are identical - I don't know how maybe I have told it to load twice?

By squashfs folder path is all in order from what I know as :

/squash-root/usr/local/tce.installed/md5mydata

Any suggestions on how I fix this issue?

BIG Thanks.

[Juanito]

This issue happens when you create a squashfs package on top of itself.

If you create a squashfs package and wish to change something, you need to delete it before re-creating a squashfs package with the same name.

[Rich]

Hi Juanito

... If you create a squashfs package and wish to change something, you need to delete it before re-creating a squashfs package with the same name.

Or you can tell  mksquashfs  to handle it for you:

Code: [Select]

mksquashfs SOURCE DESTINATION -noappend

[onelife]

THANK YOU THANK YOU!!!! I would never have thought this was based on the mksquashfs command. All sorted now. Apprecaite your help.

[onelife]

I thought perhaps another user might be trying to also make a md5sum check of their mydata.tgz - I don't know if this will help anyone but FYI, after getting the squashfs correctly working it seems to help 100%. FYI, my md5data bash script is similar to this :

Code: [Select]

#!/bin/sh

if [ -f /mnt/mmcblk0p2/tce/md5mydata ] ; then
sudo cat /mnt/mmcblk0p2/tce/md5mydata | md5sum -c > /tmp/md5sum
        if grep "FAILED" /tmp/md5sum ; then
        echo "MD5SUM FAILED"
                if [ -f /mnt/mmcblk0p2/tce/mydata.corrupt ] ; then
                echo "NO BACKUP AVAILABLE - GOING INTO RESCUE"
sudo /usr/local/etc/init.d/openssh start
sleep 20
sudo sed -i -e "s|#PermitEmptyPasswords no|PermitEmptyPasswords yes|" /usr/local/etc/ssh/sshd_config
sudo /usr/local/etc/init.d/openssh restart
sleep 1
                exit
                else
                sudo mv -f /mnt/mmcblk0p2/tce/mydata.tgz /mnt/mmcblk0p2/tce/mydata.corrupt
                sleep 1
                sudo mv -f /mnt/mmcblk0p2/tce/mydatabk.tgz /mnt/mmcblk0p2/tce/mydata.tgz
                sleep 1
                sync
                sleep 2
                sudo reboot
                fi
        else
        echo "MD5SUM PASSED"
        fi
else
echo "NO MD5SUM - CHECKING SYSTEM"
fi
exit
And essentially the file /tce/md5mydata file is created whenever I call filetool.sh by my own script :

Code: [Select]

#!/bin/sh

TCRASPBERRY=/home/snb/update_skynetwork/log/tcraspberry
DAILYFILETOOL=/home/snb/update_skynetwork/log/dailyfiletool

if [ -f $DAILYFILETOOL ] ;then
echo "Waiting Until 01h00 to run filetool.sh"
exit
fi

if [ -f $TCRASPBERRY ] ;then
filetool.sh -b
sleep 2
sync
sleep 2
sudo chmod 777 /mnt/mmcblk0p2/tce/md5mydata
sudo md5sum /mnt/mmcblk0p2/tce/mydata.tgz > /mnt/mmcblk0p2/tce/md5mydata
echo "Backup Complete"
else
echo "System NOT Tiny Core"
fi

exit
So on boot the md5mydata script checks the md5sum of the backup and if it doesn't match, then move bk backup into position and reboot.

There are a few pieces I've removed from my script as once in "rescue mode" I have a few things run such as ssh start etc.

But as I say, I hope maybe this can help someone elese if they too experience corrupt mydata.tgz files and are looking for a way to auto recover to the mydatabk.tgz file.

Thanks

[Rich]

Hi onelife
Just a few comments. You need to learn proper indentation. If I had to apply a name to your formatting, it would probably be called
SRI (Semi Random Indentation) . What you have makes it difficult to follow which lines of code belong to which  if  clause. Here
is my suggestion of how to make it more readable:

Code: [Select]

#!/bin/sh

if [ -f /mnt/mmcblk0p2/tce/md5mydata ] ; then
sudo cat /mnt/mmcblk0p2/tce/md5mydata | md5sum -c > /tmp/md5sum
if grep "FAILED" /tmp/md5sum ; then
echo "MD5SUM FAILED"
if [ -f /mnt/mmcblk0p2/tce/mydata.corrupt ] ; then
echo "NO BACKUP AVAILABLE - GOING INTO RESCUE"
sudo /usr/local/etc/init.d/openssh start
sleep 20
sudo sed -i -e "s|#PermitEmptyPasswords no|PermitEmptyPasswords yes|" /usr/local/etc/ssh/sshd_config
sudo /usr/local/etc/init.d/openssh restart
sleep 1
exit
else
sudo mv -f /mnt/mmcblk0p2/tce/mydata.tgz /mnt/mmcblk0p2/tce/mydata.corrupt
sleep 1
sudo mv -f /mnt/mmcblk0p2/tce/mydatabk.tgz /mnt/mmcblk0p2/tce/mydata.tgz
sleep 1
sync
sleep 2
sudo reboot
fi
else
echo "MD5SUM PASSED"
fi
else
echo "NO MD5SUM - CHECKING SYSTEM"
fi
exitNow you can see where each  if  clause ends and which  else  belongs to which  if. And don't use the spacebar for indentation, that's
what the  Tab  key is for.

What's with all the sleep commands here:

Code: [Select]

else
sudo mv -f /mnt/mmcblk0p2/tce/mydata.tgz /mnt/mmcblk0p2/tce/mydata.corrupt
sleep 1
sudo mv -f /mnt/mmcblk0p2/tce/mydatabk.tgz /mnt/mmcblk0p2/tce/mydata.tgz
sleep 1
sync
sleep 2
sudo reboot
fiThe kernel should handle completing the first move before the second one starts.

Does it not work correctly like this:

Code: [Select]

else
sudo mv -f /mnt/mmcblk0p2/tce/mydata.tgz /mnt/mmcblk0p2/tce/mydata.corrupt
sudo mv -f /mnt/mmcblk0p2/tce/mydatabk.tgz /mnt/mmcblk0p2/tce/mydata.tgz
sync
sudo reboot
fi
The same basic comments apply to your other script.

Will it not work if you change this:

Code: [Select]

filetool.sh -b
sleep 2
sync
sleep 2
sudo chmod 777 /mnt/mmcblk0p2/tce/md5mydata
sudo md5sum /mnt/mmcblk0p2/tce/mydata.tgz > /mnt/mmcblk0p2/tce/md5mydata
to this:

Code: [Select]

filetool.sh -b
sudo chmod 777 /mnt/mmcblk0p2/tce/md5mydata
sync
sudo md5sum /mnt/mmcblk0p2/tce/mydata.tgz > /mnt/mmcblk0p2/tce/md5mydata

[jazzbiker]

Hi, Rich!

Maybe my 2 cents are misplaced, but for slow flash drives sync doesn't guarantee that data have found their place, and sometimes those sleeps are the only way to success.

For example here are citations from tc-istall.sh:

Code: [Select]

partition_setup_common(){                                         
  echo "Writing zero's to beginning of /dev/$DEVICE"             
  dd if=/dev/zero of=/dev/$DEVICE bs=1k count=1 > /dev/null 2>&1 &
  sync; sleep 1                                                   
  echo "Partitioning /dev/$DEVICE"                               
  if [ "$FORMAT" == "vfat" ]; then                                   
    echo -e "n\np\n1\n\n\na\n1\nt\nc\nw" | fdisk /dev/$DEVICE > /dev/null 2>&1
  else                                                                       
    echo -e "n\np\n1\n\n\na\n1\nt\n83\nw" | fdisk /dev/$DEVICE > /dev/null 2>&1
  fi                                                                           
  sync; sleep 1                                                               
}                                                                             

and then after calling partition_setup_common:

Code: [Select]

hdd_setup(){                                                     
  partition_setup_common                                         
  sync; sleep 1                                                   
  rebuildfstab                                                   
...

My own TC lives on the flash drives, local hard disks are used as additional storage and I can confirm, that for scripts those sleeps, seeming extraneous, sometimes are unavoidable.

[Greg Erskine]

Hey onelife,

I was just wondering what problem you are having that requires you to md5sum mydata.tgz

I been using piCore since it was invented, do many backups and reboots a day and I have lots of RPi's. I never had a problem with mydata.tgz getting corrupted, except when it is very big and you run out of space on your SD card.

So I check the size of mydata.tgz and give a message if it is getting large.

regards
Greg

[onelife]

Hi there Greg,

Thank you for checking in on this - Essentially I have caught on more than one occasion the backup fails as sadly here in SA we have regular power cuts. If the power goes off while the system happens to be running a filetool.sh, the created backup is then corrupt.

I actually made a video of this in 2014 and posted it here on the forum. I suspect though this is unique to our situation here in SA. Yes, the backup process is super quick and the chance is very very slim BUT it has happened on several occasions.

Out of interest so we can compare a "ratio" as such - How many pi's are you running TC on?

Chat soon - Thank you.

[Paul_123]

lol, I have about 10 running at any given time.   And another 5 or so used during development cycles.   The ones used during development get many many backups and power cycles.

[Greg Erskine]

I usually have about 7 running but at the moment only 3. I had too do a "cleanup" a couple of weeks ago. lol.

Like Paul, the RPi used for development and testing get a lot of backups and reboots.

Can't say I had a power failure at the exact time I was doing a backup.

[onelife]

Aha, okay - I think the power issue is 100% the cause. As I say, I've tested this condition and it for sure causes the backup to be corrupt.

I then guess it's simply a ratio factor - Currently I have just under 2000 pi's running 24/7.

So hopefully my md5sum check can help stop the few failures I do get.

Big Thanks again for your thoughts on this - Here's to what is a superb OS - Hopefully soon I will attempt upgrading to 9.X or more - But sadly the modules / tcz's I require are not yet on 9.x - So for now 90% of my pi's are on 8.1.5 and lower.

If anyone is ever interested in getting involved commercially with my project I would only be to happy to welcome some needed guru advice.

Keep safe - Kind Regards,

[Juanito]

We've moved onto piCore-11.x now - which extensions you believe are missing?

[onelife]

Aha, I last tried beta 9.X and at that time many were missing. My onboot.lst as example is :

Code: [Select]

mc.tcz
mpg123.tcz
rsync.tcz
bash.tcz
coreutils.tcz
vtun.tcz
lzo.tcz
visudo.tcz
cpulimit.tcz
dermixd.tcz
java.tcz
libmagic.tcz
libsndfile.tcz
studio.tcz
ntp.tcz
libvorbis.tcz
flac.tcz
firmware-ralinkwifi.tcz
libusb.tcz
usbutils.tcz
wifi.tcz
usb-modeswitch-data.tcz
usb-modswitch.tcz
firmware-brcmfmac43430.tcz
wireless-4.4.13-piCore+.tcz
rtc-4.4.13-piCore+.tcz
usb-serial-4.4.13-piCore+.tcz
alsa.tcz
alsa-plugins.tcz
alsa-utils.tcz
wireless-4.4.13-piCore_v7+.tcz
rtc-4.4.13-piCore_v7+.tcz
usb-serial-4.4.13-piCore_v7+.tcz
libacl.tcz
openssh.tcz
alsa-modules-4.4.19-piCore_v7+.tcz
alsa-modules-4.4.19-piCore+.tcz
alsa-modules-4.4.39-piCore_v7+.tcz
usb-serial-4.4.39-piCore_v7+.tcz
wireless-4.4.39-piCore_v7+.tcz
firmware.tcz
alsa-modules-4.4.39-piCore+.tcz
libevdev.tcz
libevent.tcz
libopus.tcz
beamer.tcz
netcat.tcz
md5mydata.tcz
ffmpeg.tcz
There are a few custom tcz's such as :

Code: [Select]

dermixd
java
studio
beamer
md5mydata
But the critical missing parts were example, alsa modules, usb serial and ffmpeg.

I'm not sure if the repo for 11.X is now "open" that we can see the available tcz's now?

Maybe just maybe it's now further along and I can attempt an 11.x build

Thank you!

    [EDIT]: Added code tags.  Rich

[Juanito]

alsa-modules, usb-serial and ffmpeg are present in both the 9.x and 11.x repos - you can see what's in the 11.x repos at:

http://tinycorelinux.net/11.x/armv6/tcz/
http://tinycorelinux.net/11.x/armv7/tcz/

Note that deps are recursive, so there are a lot of lines that can be eliminated from your onboot.list