Tiny Core Linux
Tiny Core Extensions => TCE Q&A Forum => Topic started by: gmc on May 17, 2013, 10:59:37 PM
-
Greetings,
I think this is probably an ezremaster problem so this looks like the right place to post.
I am trying to remaster TCL 4.7.7 after installing some apps and making some configuration changes. When I run ezmaster and only include .tcz extensions the process works and the completed .iso image burns and boots properly.
When I add .scm's to the process the finished .iso image will burn fine, but the boot process fails to complete.
The message(S) follow:
Booting the kernel.
Kernel panic - not syncing: VFS: Not able to mount root fs on unknown block(0,1) Pid:1 comm: swapper Not tainted 3.0.21-tinycore #3021...
I have run the ezremaster process twice just to make sure something didn't just "glitch". Both times gave the same results.
I have also tried using three different computers and two different optical drives, just in case...
I am at a total loss as to how to proceed from this point. Any ideas would be appreciated.
Thnaks.
-
The message says it couldn't load the initrd, so seems the remastering process failed. Before Brian has a chance to check it, make sure you have enough RAM on the comp where you remaster.
Testing in a VM before burning is OK too, saves some coasters.
-
The message says it couldn't load the initrd, so seems the remastering process failed. Before Brian has a chance to check it, make sure you have enough RAM on the comp where you remaster.
I think you might be on to something. How can I know if I am running low on memory? I have a system with 256MB RAM. 8MB is shared video, the rest is available to TC. Would there be any error messages in the log file? It makes sense that it would work with less files in the remaster and that adding more files to the mix could result in an image that exceeds available RAM. I think I'll go see if the log file has anything interesting to report that I missed the first time through it...
-
I'd run an app like watcher or htop to monitor free mem.
Also, in case you don't have a generous amount of swapspace yet, it couln't harm to add.
-
I'd run an app like watcher or htop to monitor free mem.
Also, in case you don't have a generous amount of swapspace yet, it couln't harm to add.
Perfect! I'll try it right away.
As for swap space, I had purposely avoided it because this particular machine only has a DOM (Disk On Module) with a limited number of write cycles before death.
If I remember correctly, swap space can be set at boot time and can be on any drive, yes? I have an external USB/HD formatted NTFS. I also have the NTFS extension installed. Do you think that the system could handle using the NTFS drive for swap space? If not, maybe I could implement a temporary swap space on the DOM just long enough to remaster. I hate to shorten the DOM's life that way though. Maybe I will move what's on the external USB/HD and repartition it with an ext2 partition.
Thanks again for the help.
-
I'd run an app like watcher or htop to monitor free mem.
Also, in case you don't have a generous amount of swapspace yet, it couln't harm to add.
Perfect! I'll try it right away.
As for swap space, I had purposely avoided it because this particular machine only has a DOM (Disk On Module) with a limited number of write cycles before death.
If I remember correctly, swap space can be set at boot time and can be on any drive, yes? I have an external USB/HD formatted NTFS. I also have the NTFS extension installed. Do you think that the system could handle using the NTFS drive for swap space? If not, maybe I could implement a temporary swap space on the DOM just long enough to remaster. I hate to shorten the DOM's life that way though. Maybe I will move what's on the external USB/HD and repartition it with an ext2 partition.
Thanks again for the help.
Update:
I installed watcher and htop. Cool!
But something interesting showed up. htop shows that I do have swap space! Does TC automatically add swapspace? I know I did not set it up.
So, how do I tell which drive is being used for swapspace?
Thanks.
-
cat /proc/swaps
-
cat /proc/swaps
Here is what I get when I try that command.
cat /proc/swaps
Filename Type Size Used Priority
/dev/zram0 partition 240264 0 -1
What is zram0? I have this on two different machines, one with an external USB/HD (plus the internal DOM) and the other with only the DOM. Both show the same zram0 swap drive. I hope this isn't connected with the DOM.
-
Hi gmc
Tinycore sets aside part of your RAM to use as swap space. If you have an empty USB thumb drive, use that for swap.
I don't know for sure, but I don't think Linux can use NTFS for swap.
-
OK, I ran watcher and htop while running ezremaster again. Memory useage never went over 87.8%, swap never went over 21%. The new .iso image boots to the same place and stops with the same message.
For whatever it is worth, I checked the ezremaster log file for errors. The only thing I could find was where the .scm's were being set up. There were messages saying that certain directories could not be found. These seem to all be related to dependencies that don't exist. Not sure if that helps or not.
-
Hi gmc
Tinycore sets aside part of your RAM to use as swap space. If you have an empty USB thumb drive, use that for swap.
I don't know for sure, but I don't think Linux can use NTFS for swap.
Thanks Rich,
I was wondering if that were the case. It seems kind of strange that TC would use RAM for swap space. Afterall, isn't swap space used to swap out what's in RAM when there isn't enough RAM to held everything?
As for the USB thumb drive - I understood that they also have a limited number of write cycles before failure, like other flash memory devices. Just curios...
Thanks for the feedback.
-
Hi gmc
It's actually compressed swap in RAM. Thumb drives also have a limited number of write cycles, somewhere in the
1 to 10 million range these days I believe. Wearing out a $5 thumb drive is more palatable than wearing out your DOM.
-
cat /proc/swaps
Here is what I get when I try that command.
cat /proc/swaps
Filename Type Size Used Priority
/dev/zram0 partition 240264 0 -1
What is zram0? I have this on two different machines, one with an external USB/HD (plus the internal DOM) and the other with only the DOM. Both show the same zram0 swap drive. I hope this isn't connected with the DOM.
This makes the impression that you would more likely have 1GB of RAM.
-
Hi gmc
Tinycore sets aside part of your RAM to use as swap space. If you have an empty USB thumb drive, use that for swap.
I don't know for sure, but I don't think Linux can use NTFS for swap.
I couldn't imagine any particular reason where NTFS would differ from any other rw mounted fs regarding swapspace, besides from the known issues regarding mounting of NTFS in rw mode in general.
IIRC, swapfiles on NTFS had been used by some even back in the days when changing filesizes wasn't (safely) supported by any driver.
-
OK, I ran watcher and htop while running ezremaster again. Memory useage never went over 87.8%, swap never went over 21%. The new .iso image boots to the same place and stops with the same message.
Note that besides from restrictions of mem usage there is also a restriction of rootfs size being set to 90% of available RAM by default - though I'd suspect you would get some error msg when running out of space.
-
Hi gmc
It's actually compressed swap in RAM. Thumb drives also have a limited number of write cycles, somewhere in the
1 to 10 million range these days I believe. Wearing out a $5 thumb drive is more palatable than wearing out your DOM.
Oh, compressed! That's very interesting.
I like your logic on cheap thumb drives. I haven't bought any in so long that it never occurred to me.
Thanks!
-
cat /proc/swaps
Here is what I get when I try that command.
cat /proc/swaps
Filename Type Size Used Priority
/dev/zram0 partition 240264 0 -1
This makes the impression that you would more likely have 1GB of RAM.
You are so right! On the machine where I copied the above output I do indeed have 1GB. The other machine only has 256MB total, with 8MB of that going to shared video. And that is the one with the working PXE server setup that I so much want to remaster.
-
One of the characteristics of Core is that it is highly nomadic.
Unless you wanted to use something like local_initramfs.sh which depends on currently loaded modules, I see no absolute reason to remaster on the target machine.
I have created remasters for a diskless machine to be booted via PXE on a different machine and it worked out very well.
-
One of the characteristics of Core is that it is highly nomadic.
Unless you wanted to use something like local_initramfs.sh which depends on currently loaded modules, I see no absolute reason to remaster on the target machine.
I have created remasters for a diskless machine to be booted via PXE on a different machine and it worked out very well.
Actually, that is one of the main goals I had for the remaster. Another is for a complete working backup, ready to run out of the box, so to speak.
-
Hi gmc
It's actually compressed swap in RAM. Thumb drives also have a limited number of write cycles, somewhere in the
1 to 10 million range these days I believe. Wearing out a $5 thumb drive is more palatable than wearing out your DOM.
Oh, compressed! That's very interesting.
I like your logic on cheap thumb drives. I haven't bought any in so long that it never occurred to me.
Thanks!
Update: We had to go to town today so while we were out and about I checked the prices on new thumb drives. I was pleased to find that I could buy a "small" (8GB) PNY drive for $6. So I did. At that price you are right - it won't matter if it goes bad, it becomes a "wear" item, like tires on the car. Thanks for the tip.
-
OK - Time for an update.
I removed the DOM (128MB Disk On MOdule) and installed TC 4.7.7 to an 8GB USB thumb drive. I added a second, 256MB USB thumb drive as a swap drive. Then I ran ezremaster again. This time maximum memory useage went to 97% and swap useage maxed out at 90%. After a very long wait ezremaster failed with this mesage at the end of the log file:
+ sudo advdef -z4 /tmp/ezremaster/image/boot/core.gz
Error writing on /tmp/ezremaster/image/boot/core.gz [at void convert_gz(adv_fz*, adv_fz*):redef.cc:447]
+ exit 23
The new swap drive is being recognized:
oot@box:/# cat /proc/swaps
Filename Type Size Used Priority
/dev/zram0 partition 61084 51896 -1
/dev/sdb1 partition 251900 150452 -2
Does this appear to be related to the kernel panic problem or is it something new?
Thanks...
-
Hi gmc
I think the system ran out of memory. See:
http://forum.tinycorelinux.net/index.php/topic,15299.msg88554.html#msg88554
-
After a process exits, memory gets freed up, so not of much relevance.
You can check in real time with
watch cat /proc/swaps
-
After a process exits, memory gets freed up, so not of much relevance.
You can check in real time with
watch cat /proc/swaps
I was using conky and watching almost constantly - I never did see it go higher than I mentioned, but since conky only samples it could have missed that critical moment when memory was filled.
Do you think the "watch cat..." method would be more responsive than conky? I'm going to try Rich's tip on temporary space.
-
Hi gmc
I think the system ran out of memory. See:
http://forum.tinycorelinux.net/index.php/topic,15299.msg88554.html#msg88554
Thanks for the links - very useful. I tried using sda1 for my temp storage but ezremaster fails immediately and there is no log to be found anywhere. I tried letting ezremaster create the directory itself and then tried again after creating it myself. Neither method worked. I'm going to try some more tests while I wait for additional ideas from TC Headquarters.
-
Hi gmc
Did you also set the ownership and permissions?
sudo mkdir /mnt/sda1/ezremastertemp
sudo chown tc:staff /mnt/sda1/ezremastertemp
chmod 775 /mnt/sda1/ezremastertempAnd then point the first page ezremaster to that temp directory rather than using the default RAM based /tmp/ezremaster.
-
After a process exits, memory gets freed up, so not of much relevance.
You can check in real time with
watch cat /proc/swaps
I was using conky and watching almost constantly - I never did see it go higher than I mentioned, but since conky only samples it could have missed that critical moment when memory was filled.
Do you think the "watch cat..." method would be more responsive than conky? I'm going to try Rich's tip on temporary space.
watch -h
[...]
-n Loop period in seconds (default 2)
-
Hi gmc
Did you also set the ownership and permissions?
sudo mkdir /mnt/sda1/ezremastertemp
sudo chown tc:staff /mnt/sda1/ezremastertemp
chmod 775 /mnt/sda1/ezremastertempAnd then point the first page ezremaster to that temp directory rather than using the default RAM based /tmp/ezremaster.
I thought I had it set right but did not. I compared the permissions and ownership with the one I created on the external HDD (NTFS) with the one I made on the USB thumb drive (ext2) - they were identical but only the HDD temp storage would work with ezremaster. I wanted to avoid that one because it required the ntfs utility to run, and that used up precious RAM.
Before I got your response I tried starting ezremater from a root console and that part worked. I was then able to use the USB thumb drive for temp storage.
At that point I ran ezremaster again. This time I had a 256MB USB thumb drive formatted as linux swap, a second USB thumb drive (8GB) for temp storage, running as root, monitoring with htop and watch -n 1 cat /proc/swaps.
Using this setup I saw a significant reduction in memory useage - so low that the 256MB swap drive was never touched. RAM swap was only minimally used, generally around 2 or 3 percent.
So, did it work?
No. I still got the same kernel panic when booting the new CD.
But wait! There is good news! I went back tot he wiki and read through the entire article on ezremaster again. (It makes more sense after using it for a while and then reading the guide again.) This time I ran ezremaster again but chose differnet options. Instead of choosing 'Extract TCZ in to initrd' I selectively chose 'Inside initrd' and Inside initrd on boot'.
Success!
Ironically, after re-reading the wiki I realized that extracting TCZ's into the initrd was the worst choice I could have made for booting on these thin clients - they don't have enough memory as it is, they sure don't need more things loaded into RAM at boot time!
Maybe at a later time I will try the 'extract TCZ' method again with the HP T5740 since it has lots of memory.
Anyway - I want to tell you how much I appreciate the time and effort you have put into helping me with this. I have learned so much through the process, thank you. I do have another question or two for follow-up it you don't mind. I'll use another post to keep things separate.
-
OK, now for my next question. When I booted the remastered CD I noticed that some things were included and some were not. On the source machine I was using home=sda1 and opt=sda1. I removed home and opt from .filetool.lst and added root. I did not include home=sda1 or opt=sda1 in the remaster.
On booting the remastered CD I found that the files in root were restored but that home and opt were not. Iam surmizing that ezremaster does not include /home or /opt uless they are included in .filetool.lst?
It would be good to know exactly what ezremaster does and does not include in a remaster.
Thanks.
-
One more question if I may.
When I made the working remaster I chose 'Inside initrd apps on boot' for some programs and 'inside initrd apps' for others. (On the source system they were all installed as 'onboot'.
On booting from the CD I found that all the programs were 'onboot', even those that were selected as 'Inside initrd apps'. Can someone explain to me what the options mean? I have read the wiki artilcle again, several times, but must still be missing something.
Thanks.
-
I hope you try "app_inside_initrd_onboot" I use this method to add extensions and make a ezremaster ISO each day without issue.
Also don't forget to add your backup "mydata.mtz" file so as to add your home directory files. (if I understand your problem)
-
OK, now for my next question. When I booted the remastered CD I noticed that some things were included and some were not. On the source machine I was using home=sda1 and opt=sda1. I removed home and opt from .filetool.lst and added root. I did not include home=sda1 or opt=sda1 in the remaster.
On booting the remastered CD I found that the files in root were restored but that home and opt were not. Iam surmizing that ezremaster does not include /home or /opt uless they are included in .filetool.lst?
I'd regard /home & /opt being scatter mounted vs. being included in a remaster as mutually exclusive, regardless of use of ezremaster.
-
I hope you try "app_inside_initrd_onboot" I use this method to add extensions and make a ezremaster ISO each day without issue.
Also don't forget to add your backup "mydata.mtz" file so as to add your home directory files. (if I understand your problem)
Thanks - I will continue to use the 'app inside' method. As for the home and opt directories, it is almost certainly related to them being , as tinypoodle noted, "scatter mounted". I suppose that I'll have to set up the machine in the more traditional backup scheme if I want them included in the remaster.
Thanks for taking the time to help.
-
I'd regard /home & /opt being scatter mounted vs. being included in a remaster as mutually exclusive, regardless of use of ezremaster.
I probably would too if I knew as much as you. "Scatter mounted". The phrase made me smile.
In any case, I will revert back to a more vanilla setup and have /home and /opt backed up the traditional way. Then they will be included in the remaster.
Thanks for the thoughts...
-
If you select a mydata.tgz during the ezremaster wizard the contents of your mydata.tgz backup will be extracted into the remaster. The only things included in your mydata.tgz are what is in the .filetool.lst. If you are using persistent home/opt these will not be included. However - you can manually copy them in yourself in the last step of ezremaster before the ISO is created. You can really customize or do anything you want with the remaster at that point.
Regarding your question about "onboot" extensions in ezremaster: If you add them "onboot" ezremaster just adds them to the onboot.lst file so that they are loaded automatically. Please check the extensions that you didn't add onboot - you should find that they are not listed in the onboot.lst file.
Regarding the confusion about the different options to add extensions: ezremaster supports lots of different options - some of them work better than others depending on your machine and what you are trying to do. The "extract to initrd" is unique to ezremaster and doesn't work with all extensions.
As always, if anyone has any suggestions on how to improve ezremaster or make it easier to understand please let me know.
thanks,
Brian
-
...If you are using persistent home/opt these will not be included. However - you can manually copy them in yourself in the last step of ezremaster before the ISO is created. You can really customize or do anything you want with the remaster at that point...
Thanks for the explanations, they help. I would like to know how to add /opt and /home from persistant storage. Is there a link to those instructions or can you offer some guidance?
Thanks much.
Glen
-
On the "Remaster Step #2" screen, click the blue link for the "initrd filesystem Directory". This will open a terminal. Make a note of the directory it put you in (will be something like /tmp/ezremaster/extract). Type "sudo su", and then "cd" back to the temp initrd directory (i.e. cd /tmp/ezremaster/extract). From here you can do something like "cp -rp /home ." to copy your /home directory in to your temp initrd directory. This directory will end up being the root filesystem for your remaster, so you can make any changes you want or copy any other files in to it.
Once you have customized this kind of stuff, go back to the ezremaster window and click "Create ISO"
Let me know if you have any questions about this.
take care,
Brian
-
On the "Remaster Step #2" screen, click the blue link for the "initrd filesystem Directory". This will open a terminal. Make a note of the directory it put you in (will be something like /tmp/ezremaster/extract). Type "sudo su", and then "cd" back to the temp initrd directory (i.e. cd /tmp/ezremaster/extract). From here you can do something like "cp -rp /home ." to copy your /home directory in to your temp initrd directory. This directory will end up being the root filesystem for your remaster, so you can make any changes you want or copy any other files in to it.
Once you have customized this kind of stuff, go back to the ezremaster window and click "Create ISO"
Let me know if you have any questions about this.
take care,
Brian
Thanks Brian - this is exactly what I needed to get started. I am so looking forward to this...
Have a geat day!
Glen