Remaster Script

[danielibarnes]

I use Tiny Core Linux to create virtual appliances, and since I like to keep things current I frequently remaster it. I created a script to make this easier, and I am sharing it so that others may benefit (and hopefully improve upon).

Features
The script will optionally do any or all of the following:
1) Rebuild the kernel according to a specific .config
2) Rebuild the tinycore.gz root filesystem, including:

• kernel modules from step #1, and
• custom files

3) Create a new ISO image which includes:

• kernel from step #1
• tinycore.gz from step #2
• custom isolinux files (isolinux.cfg and others)
• any desired extensions

Instructions
1) Load the following extensions into Tiny Core:

• advcomp
• bash
• compiletc
• coreutils
• mkisofs-tools
• perl_xml

Not all of these are necessary (especially if the kernel isn't compiled).

2) Copy the following files into a src/ directory in the same directory as the remaster.sh script:

• 2.6.26-tinycore.patch
• config-2.6.26-tinycore
• linux-2.6.26.tar.bz2
• tinycore_1.1.iso (or any other versions)

Here are some helpful links for these files:
ftp://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/release/src/
ftp://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/release

3) Run the remaster script. It takes two parameters:

• the "project" directory in which all the custom files reside
• the version of Tiny Core to remaster (1.1, 1.2rc1, etc.)

Operation
Here is what the script does, step by step:
1) Loop mount the image (tinycore_VERSION.iso) and copy the files into a temporary directory.
2) Look for a tce/ directory in the project directory and copy files from it into the temporary directory.
3) If a rootfs/ directory or "modlist" file exists, then:

• extract the original tinycore.gz into a temporary directory, and
• copy files from rootfs/ into it (if it exists).

4) If a "config" file exists and a kernel/ directory does not exist, then:

• extract the kernel source,
• "make mrproper",
• patch the kernel,
• copy config from the project directory to .config,
• "make oldconfig", and
• build the kernel.

5) If the kernel/linux-2.6.26/arch/x86/boot/bzImage file exists, copy it.
6) If a "modlist" file exists and the kernel/linux-2.6.26 directory exists, copy the specified kernel modules
7) Rebuild the tinycore.gz root filesystem if a new one was created in steps 4 or 6.
8) If an isolinux/ directory exists, then copy the custom isolinux files.
9) Create a new customized ISO image.

The output is placed in the out/ directory within the project directory. If the kernel is built, it will be placed in the project directory unless a kernel/ directory already exists. This will avoid building the kernel next time.

Summary
1) To build a new kernel, create a "config" file and place it in the project directory.
2) To use files (bzImage and/or modules) from a previously built kernel, place the kernel tree in a kernel/ directory.
3) To copy kernel modules into tinycore.gz, create a "modlist" file containing pathnames of the kernel modules.
4) To copy custom files into tinycore.gz, create a rootfs/ directory and place files there.
5) To create a custom isolinux boot configuration, create a isolinux/ directory and place files there.

Attached is a file containing a script and an example remaster project based on my VMware customizations. It does not contain a tce/ directory to save space. Extract the file and run the script with "sh remaster.sh example 1.1" to remaster against version 1.1.

The root filesystem customization in the example simply adds a modified /etc/inittab. The kernel is rebuilt to include support for LSI Logic and BusLogic SCSI adapters (emulated by VMware), remove support for the pcnet32 network adapter (so the VMware virtual adapter driver can load), and add support for VMI paravirtualization. The custom isolinux.cfg is almost exactly like the original; I just added a few boot parameters to the default and changed the timeout.

I tested the script quite a bit, but let me know if any improvements or fixes are necessary.

I compiled VMware Tools and created an extension for my personal use, but I am uncertain if distributution is allowed by VMware.

[^thehatsrule^]

Took a quick glance... looks pretty good.

Just wondering why the `trap` is there, and why there is the check for some of the programs when they were already found by `which`

[mikshaw]

Just wondering why the `trap` is there

It looks like it's to clean up the temp files in the event that the script is cancelled before it completes.

why there is the check for some of the programs when they were already found by `which`

We see similar things in other scripts, where a variable is set and then the value of the variable is checked.  It seems redundant to me too, but it's not uncommon.   On another note, I'd probably also be more specific about a fail, passing an error message to it:

Code: [Select]

fail() {
echo $*  #not sure what would be most useful here: $1, $@, $*
exit 1
}

for i in advdef make mkisofs ; do
echo -n "Finding $i: "
which $i || fail "Cannot find $i"
done


[danielibarnes]

Thanks for the comments.

The trap is for cleanup. It's just personal script style, as is the "which". I have a template I use when writing a new script, and it contains ideas and other stuff I pick up from other scripts. It looks like overkill for a small script like this, but it also serves as a base for others I have which are more complicated.

[helander]

Hi,

I am new to TinyCore and I believe that this script is precisely what I want/need.
Being new I have not found my ways around yet, so I have no clue on from where to download this script

Any help on how I can get this script is highly appreciated.

Kind Regards

/Lars

[mikshaw]

It is attached to the first post....look for the paperclip at the end of the post.

I'm not convinced that remastering is what you need as a new user.  Considering TC was built from scratch it is a bit different than your typical distro.  If a problem occurs during a remaster, it may not be as easy to debug.  I'm not saying don't do it, just suggesting that you check out TC's abilities as-is for a start.

[helander]

It is attached to the first post....look for the paperclip at the end of the post.

I'm not convinced that remastering is what you need as a new user.  Considering TC was built from scratch it is a bit different than your typical distro.  If a problem occurs during a remaster, it may not be as easy to debug.  I'm not saying don't do it, just suggesting that you check out TC's abilities as-is for a start.

Thanks. Found it and downloaded it

One thing I will need to do is to build my own version of the kernel, since the current kernel does not match all my needs. I have built kernels for a multitude of distros and having a "framework" to that in helps a lot. Since building the kernel is done within the remaster script I saw that as useful, either to use fully or just to look at a process that works

Primarily I will build a kernel configured with CONFIGURE_PREEMPT and secondly one with CONFIGURE_PREEMPT_RT. The latter requires the RT patches to be applied so I will start out with the former, which only requires a small modification of the current TC kernel config.

Further along I might want to build an ISO with a specific set of packages installed - and for creating that the remaster script seemed useful.

Kind Regards

/Lars

[mikshaw]

ahh...so as a "new user" you meant specifically to TC, and definitely not to linux =o)

I have very little experience or success with compiling the kernel myself, so on that subject you are already way ahead of me, so I won't do anything more to dissuade you from your quest.

On the subject of packages, though, i will suggest you consider sticking with extensions rather than adding them to a remaster as installed applications/  Tis will allow you to more easily upgrade specific apps, and might save you some ram by using tcz if that is a concern for you.

[florian]

I'm in similar position that mikshaw in that I don't  have much experience with kernel tuning and even recompiling. I admire people who attempt such things and often promise myself that someday I will 

Meanwhile I've also found that on the forums, some are are quick to think they need to remaster. Of course there are cases where you really need a remaster (for modified kernel for instance), but in most cases this is not necessary thanks to the mechanisms offered by tc with the boot codes and its fantastic extension system

[rhombic]

   This has been a most informative conversation and I am certiain it will help my efforts.

I do have a special need.  I need to install an Kontron JIDA I/O package library and it would be handy if someone could show me how to "sneek" this in without a total rebuild.

Otherwise I am about to  boldly go where .... and follow the script thanks

bobo

[danielibarnes]

To add to florian's comment, I, too, notice there is often confusion about remastering vs. extensions. There are very few cases where you need to remaster, namely to:

• Rebuild the kernel to enable/disable options
• Include modules for hardware required to boot (HDD controllers, for example)
• Build a self-contained TC CD for a specific purpose (appliance).
• Include or modify files which are used before extensions are loaded (like /etc/inittab)

In short, you only need to remaster for something that is needed before extensions are loaded. Everything else can go into an extension. Keep in mind that an extension in its simplest case (.tce) is just a tar.gz which is extracted to the root filesystem at bootup. If you want to add programs or modify the filesystem, follow the wiki guidelines for creating an extension. That is the real strength of TC, and it will serve you well.

Daniel

[mwhit95]

I just run into a problem with extensions so I was looking at a remaster.  I am using about 20 or so tcz type extensions and I noticed some of them didn't load because it was too many loops.  I was thinking that it might be good to take all of the smaller extensions and combine into one but I am not sure how to do that.

[curaga]

You can increase the default up to 255 (max_loop=xx as a boot argument). The default has been upped to 80 in tc2.

[JoXo009]

I created a script to make this easier

Sonds great.

But I fear the remaster script file disappeared in the meantime.

Unfortunately the so-called helpful links don't link to specific files but to web folders only.

(The content of these foldes becomes visible in your browser it you copy the link into the address bar and exchange 'http' for 'ftp'.)

And in fact the four files listed under 'Instructions top2' do exist in that folder. See my posting
http://forum.tinycorelinux.net/index.php?topic=2794.msg14191#msg14191

But just the main file, the remaster script which (see: 'Instructions top2') should bear the name 'remaster.sh' doesn't exist within the linked web-folders.

As this thread got over 2,500 hits and earlier postings discuss the content of the remaster file I suppose that the linked web-folder contained that file at least till May.

But in the meantime that file seems to have disappeared.

Any help appreciated.

[JoXo009]

By the way.

Using my FileTransferClient and the original link:
ftp://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/release/src/

the result is the same.

There is no remaster script with the name 'remaster.sh' anymore.