Tiny Core Linux
Tiny Core Base => Raspberry Pi => Topic started by: ashfame on July 06, 2020, 08:12:39 PM
-
Hi,
As I understand the bootable media is essentially unmounted after boot up. Is it safe to pull out the sd card at this time?
I am aware that no writes are supposed to happen to the bootable media, so there is nothing in the disk cache waiting to be written at all.
Do I still risk corrupting my SD card when pulling it out after boot up? If yes, what's the recommended way to do so?
-
Hi ashfame
... As I understand the bootable media is essentially unmounted after boot up. ...
Partition #1 which contains the boot files gets unmounted. Partition #2 contains your tce directory which contains your extensions. It
does not get unmounted. For that, you need to:
1. You need to have a copy2fs.flg file in your tce directory.
2. You need to unmount it yourself.
-
Gotcha! Do you know where the command for unmounting first partition is?
And should I unmount second partition from /opt/bootlocal.sh script or somewhere else?
Also, as long as all partitions are unmounted, it's completely safe to pull out the memory card? I wonder why full-fledged linux OS always have two options - unmount & eject. I understand eject can additionally unmount other partitions as well and force the disk cache to dump its content on to the physical disk. but how does that translate into TC context?
-
http://wiki.tinycorelinux.net/doku.php?id=wiki:remastering_with_ezremaster#options_for_adding_extensions_to_your_remaster
-
http://wiki.tinycorelinux.net/doku.php?id=wiki:remastering_with_ezremaster#options_for_adding_extensions_to_your_remaster
Hi xor, I have read that section before.
Are you stating I should use "Extract TCZ to in to initrd" option in remaster so that my extensions are autoloaded? I think I will still have to unmount the second partition unless I put everything on a single partition in my remaster.
Or are you stating using the copy2fs.flg file, it will unmount the second partition as well?
Please clarify.
-
Unless you write the script you think
I cannot fully understand you and make a correct comment
http://wiki.tinycorelinux.net/doku.php?id=wiki:remastering_with_ezremaster#options_for_adding_extensions_to_your_remaster
Hi xor, I have read that section before.
Are you stating I should use "Extract TCZ to in to initrd" option in remaster so that my extensions are autoloaded? I think I will still have to unmount the second partition unless I put everything on a single partition in my remaster.
Or are you stating using the copy2fs.flg file, it will unmount the second partition as well?
Please clarify.
-
Also, as long as all partitions are unmounted, it's completely safe to pull out the memory card? I wonder why full-fledged linux OS always have two options - unmount & eject. I understand eject can additionally unmount other partitions as well and force the disk cache to dump its content on to the physical disk. but how does that translate into TC context?
Yes, it would be safe to remove the sd card - do this: $ sync
$ sudo umount /mnt/mmcblk0p2
-
Yes, it would be safe to remove the sd card - do this: $ sync
$ sudo umount /mnt/mmcblk0p2
Makes sense, but I thought TinyCore will only read from both the partitions and never write unless we explicitly tell it to save the state persistently. So, we won't have any remaining writes in the disk cache ever as its only being read from. Correct? That would mean, we don't need sync, just unmount it.
Just trying to understand the internals clearly :)
-
You could almost certainly do without sync.
-
You could almost certainly do without sync.
ok great, thank you!
-
Unless you write the script you think
I cannot fully understand you and make a correct comment
http://wiki.tinycorelinux.net/doku.php?id=wiki:remastering_with_ezremaster#options_for_adding_extensions_to_your_remaster
Hi xor, I have read that section before.
Are you stating I should use "Extract TCZ to in to initrd" option in remaster so that my extensions are autoloaded? I think I will still have to unmount the second partition unless I put everything on a single partition in my remaster.
Or are you stating using the copy2fs.flg file, it will unmount the second partition as well?
Please clarify.
Sorry, I was just trying to understand why you pointed me towards that link in the context of the question asked.
-
Hi ashfame
The umount command will refuse to unmount a "device" if its filesystem is in use:
1. If you open a file with an editor or the less command, that filesystem is in use.
2. If you execute cd /mnt/mmcblk0p2 in a terminal, that filesystem is in use.
3. If you are using a file manager to look at a directory, that filesystem is in use.
If you try to unmount a filesystem that is in use, you will receive a target is busy error message.
That brings us to umount and sync. You don't need sync. The umount command waits for writes to complete. I put together a little
slideshow to demonstrate. Each image contains 2 terminals. The top terminal monitors whether the device is busy (1) or not (0).
The bottom terminal shows the commands as I executed them.
Start: The device is not busy (0). No commands have been given yet.
(http://forum.tinycorelinux.net/index.php?action=dlattach;topic=24015.0;attach=5434)
Copy: We copy a large file (~100 Mbytes) to a USB thumb drive. The device is busy (1). Notice the command prompt has returned
even though the copy command has not completed.
(http://forum.tinycorelinux.net/index.php?action=dlattach;topic=24015.0;attach=5435)
Unmount: We issue a umount command. The device is still busy (1) because it is still being written to. Notice the command prompt
has NOT returned because the umount command has not completed. It will be blocked until all pending writes are
completed.
(http://forum.tinycorelinux.net/index.php?action=dlattach;topic=24015.0;attach=5436)
Done: The device is no longer busy (0) and the umount command completes.
(http://forum.tinycorelinux.net/index.php?action=dlattach;topic=24015.0;attach=5437)
-
I am writing on google translate,
Unless you write the script you think
I cannot fully understand you and make a correct comment
http://wiki.tinycorelinux.net/doku.php?id=wiki:remastering_with_ezremaster#options_for_adding_extensions_to_your_remaster
Hi xor, I have read that section before.
Are you stating I should use "Extract TCZ to in to initrd" option in remaster so that my extensions are autoloaded? I think I will still have to unmount the second partition unless I put everything on a single partition in my remaster.
Or are you stating using the copy2fs.flg file, it will unmount the second partition as well?
Please clarify.
Sorry, I was just trying to understand why you pointed me towards that link in the context of the question asked.
-
I put together a little slideshow to demonstrate. Each image contains 2 terminals. The top terminal monitors whether the device is busy (1) or not (0). The bottom terminal shows the commands as I executed them.
Thanks a lot for going to that length of explaining it. This really explains it :)
Unmount all partitions and they will finish the writes to physical disk and only then return. Brilliant! So one only needs to care about all partitions being unmounted & it can be ejected. Thanks again!
-
I am writing on google translate
Thanks anyway! :)
-
Hi ashfame
... Unmount all partitions and they will finish the writes to physical disk and only then return. Brilliant! So one only needs to care about all partitions being unmounted & it can be ejected. Thanks again!
I highlighted that in bold because that is only true if that filesystem is not in use. For instance, if you don't have a copy2fs.flg file, then
your extensions will be loop mounted in /tmp/tcloop/. That means the filesystem on /mnt/mmcblk0p2 is in use because that's where
your tce/optional directory is located and the .tcz files are mounted in /tmp/tcloop/. Under those circumstances the umount
command will return right away with an error message regardless of whether any writes are pending or not.
The point I'm trying to make is be certain that the umount command succeeded when it returns prior to pulling the card. Commands
return a status code to indicate whether they succeed or fail. Zero indicates success.
You can test and act on the status like this:
tc@E310:~$ ls abc && echo passed || echo failed
ls: cannot access 'abc': No such file or directory
failed
tc@E310:~$ ls X && echo passed || echo failed
X
passed
tc@E310:~$
Replace the ls command with your umount command and replace the echo commands with the action you want to take for each case.
Or you can test the status separately instead of a one liner:
#!/bin/sh
# Script to unmount a partition. $1 is the partition name passed to this script.
sudo umount /mnt/$1
# The status of the most recent command is contained in $? which automatically gets cleared once read.
case "$?" in
0) echo "umount succeeded."
echo "Congratulations."
echo "$1 unmounted." ;;
*) echo "umount failed."
echo "Better luck next time."
echo "$1 is still mounted." ;;Just replace the echo commands with the actions you want to take. Although this script does not test if you passed it a value, it
will still indicate that umount failed.
-
Hi ashfame
I highlighted that in bold because that is only true if that filesystem is not in use. For instance, if you don't have a copy2fs.flg file, then
your extensions will be loop mounted in /tmp/tcloop/. That means the filesystem on /mnt/mmcblk0p2 is in use because that's where
your tce/optional directory is located and the .tcz files are mounted in /tmp/tcloop/. Under those circumstances the umount
command will return right away with an error message regardless of whether any writes are pending or not.
The point I'm trying to make is be certain that the umount command succeeded when it returns prior to pulling the card. Commands
return a status code to indicate whether they succeed or fail. Zero indicates success.
Oh understood, rely on the exit code of the command, gotcha, Thanks again for clearing my misunderstanding! :)
-
Hi ashfame
... As I understand the bootable media is essentially unmounted after boot up. ...
Partition #1 which contains the boot files gets unmounted. Partition #2 contains your tce directory which contains your extensions. It
does not get unmounted. For that, you need to:
1. You need to have a copy2fs.flg file in your tce directory.
2. You need to unmount it yourself.
Hi Rich, wanted to confirm the exact location of copy2fs.flg Is it right under tce directory or tce/optional directory? And is there a way to verify the flag file is indeed forcing the copy to memory behaviour as intended?
-
Hi ashfame
The copy2fs.flg file goes right under the tce directory.
I don't recall off the top of my head how to verify it. Files under /usr probably won't be links back to /tmp/tcloop.
-
Hi ashfame
The copy2fs.flg file goes right under the tce directory.
I don't recall off the top of my head how to verify it. Files under /usr probably won't be links back to /tmp/tcloop.
I tried copy2fs.flg under both directories - directly under tce and tce/optional, but in both cases, if I unmount the partition2 after booting up before trying to load Xorg by (tce-load -i Xorg), it doesn't work. So, its clearly not copying over the extensions to file system in memory.
-
IIRC, the copy2fs.flg means instead of loop mounting extensions it loads the files into RAM.
This happens when the extension is installed, so if you unmount partition before you load the extension it won't find the extension.
I think you are thinking that ALL the extensions are copied into RAM during the boot process regardless of them being installed.
-
IIRC, the copy2fs.flg means instead of loop mounting extensions it loads the files into RAM.
This happens when the extension is installed, so if you unmount partition before you load the extension it won't find the extension.
I think you are thinking that ALL the extensions are copied into RAM during the boot process regardless of them being installed.
Hi Greg,
That's exactly what I was thinking. So how do I install an extension while modifying the image file, prior to writing files to the SD card? And when an extension is installed, does it still reside on the second partition (tce/optional)?
I need an automated way of preparing the disk image that's ready to boot for my use-case. Thank you!
-
I don't recall off the top of my head how to verify it. Files under /usr probably won't be links back to /tmp/tcloop.
If you use copy2fs, the extension symlinks will not appear in /tmp/tcloop, but loaded extensions will be shown by "tce-status -i".
-
Hi ashfame
... I need an automated way of preparing the disk image that's ready to boot for my use-case. Thank you!
The first thing you need to do is expand the second partition in the image file to make room for more extensions. I'm using a copy of
piCore-9.0.3.img to demonstrate the steps required. Adjust sizes for your own needs. You will need the following extensions for this:
parted.tcz
util-linux.tcz
These are the characteristics of the original image:
tc@E310:~/PiCore/TC9$ fdisk -l piCore-9.0.3.img
Disk piCore-9.0.3.img: 49 MiB, 51380224 bytes, 100352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0009bf4f
Device Boot Start End Sectors Size Id Type
piCore-9.0.3.img1 8192 77823 69632 34M c W95 FAT32 (LBA)
piCore-9.0.3.img2 77824 100351 22528 11M 83 Linux
tc@E310:~/PiCore/TC9$It is 49M in size. 4M + 34M + 11M = 49M. The 4M comes from partition 1 being offset 4M from the beginning of the image.
First we create a destination for a new image about 60M in size:
tc@E310:~/PiCore/TC9$ dd if=/dev/zero of=1.img bs=1K count=60176
60176+0 records in
60176+0 records out
61620224 bytes (62 MB, 59 MiB) copied, 0.690125 s, 89.3 MB/s
tc@E310:~/PiCore/TC9$
Next we loop mount the file as a device:
tc@E310:~/PiCore/TC9$ sudo losetup --show --find --partscan 1.img
/dev/loop332
tc@E310:~/PiCore/TC9$
Then copy the original image to the larger file:
tc@E310:~/PiCore/TC9$ dd if=piCore-9.0.3.img of=/dev/loop332
100352+0 records in
100352+0 records out
51380224 bytes (51 MB, 49 MiB) copied, 1.58843 s, 32.3 MB/s
tc@E310:~/PiCore/TC9$
This is the result. The new image has the same characteristics as the original except it's in a larger file.
tc@E310:~/PiCore/TC9$ fdisk -l /dev/loop332
Disk /dev/loop332: 58.8 MiB, 61620224 bytes, 120352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0009bf4f
Device Boot Start End Sectors Size Id Type
/dev/loop332p1 8192 77823 69632 34M c W95 FAT32 (LBA)
/dev/loop332p2 77824 100351 22528 11M 83 Linux
tc@E310:~/PiCore/TC9$
Expand partition 2 to the end of the file:
tc@E310:~/PiCore/TC9$ sudo parted -s /dev/loop332 resizepart 2 100%
This needs to be done before expanding the filesystem to inform the kernel the MBR changed:
tc@E310:~/PiCore/TC9$ e2fsck -f /dev/loop332p2
e2fsck 1.44.4 (18-Aug-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop332p2: 55/2816 files (1.8% non-contiguous), 9660/11264 blocks
tc@E310:~/PiCore/TC9$
Finally, we expand the filesystem:
tc@E310:~/PiCore/TC9$ resize2fs -p /dev/loop332p2
resize2fs 1.44.4 (18-Aug-2018)
Resizing the filesystem on /dev/loop332p2 to 21264 (1k) blocks.
The filesystem on /dev/loop332p2 is now 21264 (1k) blocks long.
tc@E310:~/PiCore/TC9$
Checking the new image you'll see partition 2 has been expanded from 11M to 20.8M:
tc@E310:~/PiCore/TC9$ fdisk -l 1.img
Disk 1.img: 58.8 MiB, 61620224 bytes, 120352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0009bf4f
Device Boot Start End Sectors Size Id Type
1.img1 8192 77823 69632 34M c W95 FAT32 (LBA)
1.img2 77824 120351 42528 20.8M 83 Linux
tc@E310:~/PiCore/TC9$
... So how do I install an extension while modifying the image file, prior to writing files to the SD card? And when an extension is installed, does it still reside on the second partition (tce/optional)?
Mount the second partition:
tc@E310:~/PiCore/TC9$ mkdir p2
tc@E310:~/PiCore/TC9$ sudo mount /dev/loop332p2 p2
tc@E310:~/PiCore/TC9$
Now you can add extensions to p2/tce/optional and update p2/tce/onboot.lst.
When you are done, unmount partition 2 and release the loop device:
tc@E310:~/PiCore/TC9$ sudo umount p2
tc@E310:~/PiCore/TC9$ sudo losetup -d /dev/loop332
[EDIT]: Added required extensions list to instructions. Rich
[EDIT]: Added missing sudo prefix to the losetup commands. Rich
-
Hi ashfame
I should have mentioned you need the following extensions for the above operations:
parted.tcz
util-linux.tcz
I will add that information to my previous post. Hope that didn't cause you any problems.
You do not have to perform these operations on a Pi. You can do this using an x86 machine running Tinycore while using the Fetch.sh
script to retrieve PiCore extensions.
-
Hi ashfame
I should have mentioned you need the following extensions for the above operations:
parted.tcz
util-linux.tcz
I will add that information to my previous post. Hope that didn't cause you any problems.
You do not have to perform these operations on a Pi. You can do this using an x86 machine running Tinycore while using the Fetch.sh
script to retrieve PiCore extensions.
Hi Rich, thanks for explaining that, you are the best! :)
I have just sat down to work on the OS build, after working on the actual project that will run on it, during the day.
I understand your well written instructions very clearly and they will be helpful when I start running out of space, but since I am using piCore11, they seem to have larger partitions with much more empty space than piCore9.
For now, I am able to put up extensions as-is, but when the time comes to including my custom extension for the primary application that's supposed to run on the air-gapped OS build, I will definitely have to do it.
This cleared my another misunderstanding regarding "installation" of extensions. Its more like loading the extensions into memory for use. So, it simply becomes a matter of providing the tcz files and then specifying them in boot.lst so that they are loaded during the boot. And copy2fs.flg dictates they are copied to memory based file system, rather than just being mounted, hence the possibility of un-mounting 2nd partition after boot.
The 2 packages you mentioned - parted.tcz & util-linux.tcz, its my understanding that I will need them if I want to do this expansion while running piCore only. Otherwise, I can just get by doing this on a x86 box, right?
-
Hi ashfame
... The 2 packages you mentioned - parted.tcz & util-linux.tcz, its my understanding that I will need them if I want to do this expansion while running piCore only. Otherwise, I can just get by doing this on a x86 box, right?
Yes, you can do this all on an x86 box. You only need those extensions on the machine you use to expand and alter the image file on.
parted.tcz is needed for expanding the partition.
util-linux.tcz is needed for the losetup program it provides. The losetup program makes it possible to treat the .img file like a
real device.
-
Hi ashfame
... The 2 packages you mentioned - parted.tcz & util-linux.tcz, its my understanding that I will need them if I want to do this expansion while running piCore only. Otherwise, I can just get by doing this on a x86 box, right?
Yes, you can do this all on an x86 box. You only need those extensions on the machine you use to expand and alter the image file on.
parted.tcz is needed for expanding the partition.
util-linux.tcz is needed for the losetup program it provides. The losetup program makes it possible to treat the .img file like a
real device.
Sorry, I am confused. If I am doing this on x86, why do I need tcz extensions for that? which essentially only runs in TinyCore?
-
Hi ashfame
Those are x86 extensions that get installed on the x86 box because they are needed to make alterations to the image file. They are not
added to the image file or needed on the Pi.
-
Hi ashfame
Those are x86 extensions that get installed on the x86 box because they are needed to make alterations to the image file. They are not
added to the image file or needed on the Pi.
Hi Rich, I think its the tcz extension that confuses me. TCZ files are TinyCore extensions, right?
I understand I will need the required packages to run on x86, which in this case is parted and utils for ubuntu, if I am right. but you said parted.tcz & util-linux.tcz so I got confused.
-
Hi ashfame
... TCZ files are TinyCore extensions, right? ...
Yes.
... which in this case is parted and utils for ubuntu, ...
I didn't realize you were planning on doing this on a Ubuntu box. In that case, watch out for file ownership issues. In Tinycore
user:group are typically tc:staff when they aren't root:root or root:staff. Any files you create on the Ubuntu box will probably
have the wrong user:group attributes. Where that might cause problems, I don't know.
-
... which in this case is parted and utils for ubuntu, ...
I didn't realize you were planning on doing this on a Ubuntu box. In that case, watch out for file ownership issues. In Tinycore
user:group are typically tc:staff when they aren't root:root or root:staff. Any files you create on the Ubuntu box will probably
have the wrong user:group attributes. Where that might cause problems, I don't know.
Thanks for the heads up! For now, I am using root user on a fresh Ubuntu VPS to generate the disk image before I download the image and write it to SD Card using my Macbook Pro, which is where I use the in-built card reader.
My primary machine is a gaming Windows PC though. As I recall reading, using a root user would not mess up the file permissions but you are right, it might cause an issue without me realizing it, when editing files on partition 1. Perhaps it would be better if I do this on a TinyCore x86 in a VM on my windows machine. That should totally cover the base for file permissions concern, right?
[EDIT]: Fixed quote tags. Rich
-
Hi ashfame
... Perhaps it would be better if I do this on a TinyCore x86 in a VM on my windows machine. That should totally cover the base for file permissions concern, right?
That would be fine.