Tiny Core Linux

General TC => Programming & Scripting - Unofficial => Topic started by: dentonlt on November 29, 2009, 02:21:13 AM

Title: direct me to TCZ moving tools?
Post by: dentonlt on November 29, 2009, 02:21:13 AM
I find myself often enabling/disabling extensions before reboot, manually moving extensions and dependencies around. I've made myself a short script to move an extension and its dependencies from /tce/optional into /tce. Next step is the reverse - moving an extension from /tce to /tce/optional. Have to check dependencies and such, yes, but I don't think this will be a problem.

Do these tools already exist? Maybe I'm reinventing the wheel for no reason ...  :-X
Title: Re: direct me to TCZ moving tools?
Post by: 4-stroke on November 29, 2009, 02:31:22 AM
If you select extensions in appbrowser and choose "download only" it will be installed in tce/optional and won't be loaded at boot. When you want to load it you choose "install local" in appbrowser and select the extension and it will be loaded until you reboot again.

Hope that makes some sense...
Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on November 29, 2009, 03:52:57 AM
Thanks for the quick reply, 4-stroke. Your answer is clear, but I already use that part  :)

I am looking for tools that let me relocate extensions & their dependencies - a sort of "local" set of extension manager tools. Sometimes I have an extension in /tce/optional that I'd like to move to /tce for the next boot. Sometimes I have things that I don't want to pop up on the next boot (move /tce -> /tce/optional). I don't think the app manager is designed to do these ... is it?

I have a dirty but effective pair of working scripts (below), but I imagine I've missed something stock.

I do figure it's bad form to recurse my scripts, so I don't think these are the smartest way of going about this problem. I'd rather make a list of all dependencies, then edit the list, then only move the files that need to be moved. Right now, disable-tce.sh moves some files twice ...

It works, but ... I'm looking for somebody else's solution.

enable-tce.sh: a dirty "move that extension from /tce/optional to /tce, or check that all its pieces made it from /tce/optional to /tce"
Code: [Select]
#!/bin/sh

#dentonlt, Nov 2009 - for TC 2.6
#Enable an extension, or check that it is properly stored in the main tce directory. So either:
#(1) move it from /tce/optional to /tce, OR
#(2) check an extension in /tce to be sure it has all of its dependencies from /tce/optional

if [ -z "$1" ]; then
echo "enable-tce.sh: move an extension and all of its dependencies to the main extension directory"
echo -e "USAGE: enable-tce.sh [-c] [FILENAME] \n"
echo -e "Include the correct extension (tcz, tczl, or otherwise) on FILENAME.\n"
echo -e "To check an extension's dependencies, call enable-tce.sh from the main extension directory."
exit 1
fi

#the file we want to enable
EXTENSION=$1

#get the current tce directory
TCEDIR=`cat /opt/.tce_dir`

#change to the tce directory:
CALLDIR=$PWD

#When called from TCEDIR, check the dependencies of an extension currently in the TCE directory
if [ $CALLDIR = $TCEDIR ] && [ -f $EXTENSION ]; then
echo "Checking $EXTENSION dependencies are in $TCEDIR: "
if [ -f "./$EXTENSION.dep" ]; then
cat "$EXTENSION.dep" |
while read FILE
do
if [ -f "./$FILE" ]; then #if the file is in $TCEDIR, move on.
echo "$FILE was okay."
continue
elif [ -f "./optional/$FILE" ]; then
mv -f ./optional/$FILE* ./
echo "Moved $FILE* to $TCEDIR."
fi
done
fi
#and get any other parts of the extension out of optional, too
cd optional
if [ -f $EXTENSION* ]; then
echo "Moving $TCEDIR/optional/$EXTENSION* to $TCEDIR"
mv -f $EXTENSION* ../
fi
echo "... done!"
cd $CALLDIR
exit 1
else #not called from $TCEDIR ... so go to the optional directory and enable the extension there.
cd $TCEDIR/optional
fi

#check to see that the extension exists
if [ ! -f "$EXTENSION" ]; then
if [ -f "../$EXTENSION" ]; then
cd ..
enable-tce.sh $EXTENSION >/dev/null #check dependencies by recursing.
echo "I found $EXTENSION in $TCEDIR - it was already enabled. I've checked its dependencies, too."
exit 1
fi
echo "I couldn't find $EXTENSION in $TCEDIR or $TCEDIR/optional. Check the file name to be sure you're correct."
exit 1
fi

#look for a dependency file
if [ ! -f "$TCEDIR/optional/$1.dep" ]; then
echo "$EXTENSION has no dependencies or the dep file is missing. Moving $EXTENSION*."
mv -f $EXTENSION* ../
echo "done!"
exit 1
fi

echo "Moving files:"

cat $EXTENSION.dep |
while read FILE
do
if [ -f "$FILE" ]; then

if [ -f "$FILE.dep" ]; then
echo -n "$FILE* and dependencies, "
enable-tce.sh $FILE >/dev/null
else
echo -n "$FILE*, "
mv -f $FILE* ../
fi
fi
done


echo -n "$EXTENSION* "
mv -f $EXTENSION* ../

echo -e "... done!\n"
cd $CALLDIR

disable-tce.sh: a dirty "go the other way."
Code: [Select]
#!/bin/sh

#dentonlt, Nov 2009. For TC 2.6
#move an extension and all of its dependencies to the optional extension directory

if [ -z "$1" ]; then
echo "disable-tce: move an extension and all of its dependencies to the main extension directory"
echo -e "USAGE: disable-tce [FILENAME] \n"
echo -e "Include the correct extension (tcz, tczl, or otherwise) on FILENAME.\n"
exit 1
fi

#get the current tce directory
TCEDIR=`cat /opt/.tce_dir`

#check to see that the extension exists
if [ ! -e "$TCEDIR/$1" ]; then
echo -e "I could not find $TCEDIR/$1. Check the file name. Is $TCEDIR mounted?\n"
exit 1
fi

#look for a dependency file
if [ ! -e "$TCEDIR/$1.dep" ]; then
echo "$1 has no dependencies or the dep file is missing. Moving $1*."
mv -f $1* ./optional/
echo "done!"
exit 1
fi

#change to the tce directory:
CALLDIR=$PWD
cd $TCEDIR

echo "Moving files:"

cat $1.dep |
while read FILE
do
if [ -f "$FILE" ]; then

if [ -f "$FILE.dep" ]; then
echo -n "$FILE* and dependencies, "
disable-tce.sh $FILE >/dev/null
else
echo -n "$FILE*, "
mv -f $FILE* ./optional/
fi
fi
done


echo -n "$1* "
mv -f $1* ./optional/

#now that all the files have been moved to the optional directory, we have to be sure that we didn't remove any dependencies.
#go through all the files in $TCEDIR and check that they are enabled.

echo -e "Rechecking dependencies ... "
cd $TCEDIR
ls -1 $TCEDIR/*.dep |
while read FILE
do
if [ -f $FILE ]; then
cat $FILE |
while read DEPFILE
do
enable-tce.sh $DEPFILE >/dev/null
done # reading dependency file
fi # if it was actually a file
done # searching directory for dep files

echo -e " ... done!\n"
cd $CALLDIR
Title: Re: direct me to TCZ moving tools?
Post by: 4-stroke on November 29, 2009, 06:01:36 AM
Oh. Well, since I don't understand why you have to move the files around like that, someone else is going to have to help you.  ;D
Title: Re: direct me to TCZ moving tools?
Post by: Jason W on November 29, 2009, 04:53:28 PM
Here is a shorter version of what I think you are wanting to do.  That is, move an extension along with it's dependencies and their related files to another directory without overwriting them if they are already are there.  Of course, the cp command can be substituted for simply copying and then the check for being in the tce directory can be removed.  A check is issued in the last function to make sure all dependencies are in place in the destination directory.

Of course, you don't want to do this on tce directory in use.

Move extension and deps into /tmp/tce from /mnt/hda2/tce:

cd /mnt/hda2/tce
movetc.sh gtk2.tczl /tmp/tce

Move them back:

cd /tmp/tce
movetc.sh gtk2.tczl /mnt/hda2/tce


Code: [Select]
#!/bin/sh
# Issue in the source directory and specify the destination dir.
# Like  movetc.sh gtk2.tczl /mnt/hda2/tce2


if [ "`pwd`" == "`cat /opt/.tce_dir`" ]; then
     echo "Do not move extensions from a running tce directory"
     exit 1
fi



if [ ! -d "$2" ]; then
     echo ""$2" does not exist, exiting."
    exit 1
fi

if [ ! -f "$1" ]; then
    echo ""$1" does not exist, exiting."
   exit 1
fi

if cat *.dep | grep "$1" > /dev/null 2>&1; then
    echo ""$1" is needed by other extensions.  exiting"
    exit 1
fi

if [ -f "$1".dep ]; then
for I in "$1".dep; do
    for E in `cat "$I"`; do
        yes n | mv "$E"* "$2"/
    done
done
fi

yes n | mv "$1"* "$2"/

cd "$2"

for I in `ls *.dep`; do
    for E in `cat "$I"`; do
    [ -f "$E" ] || echo ""$E" missing in "$2" as a dependency of "$1"."
    done
done
Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on November 30, 2009, 05:05:51 AM
Thanks for that, Jason. If only I (1) wrote more and (2) spent more time reading the extension loading code. I neglected to remember an extension is probably mounted! big oops.

I guess I am still looking for a way to 'move' an extension ... but maybe it's clearly to say I need a way to change 'extension profiles', or something like that.  I just want to mark or move a bunch of extensions to be loaded on next boot ... and then reboot.

When I am toying around, there are enough moves/changes & reboots that I'd rather not have to step through appbrowser to install things manually each time I reboot. I also don't want to repeatedly delete and re-download things that I will use again soon.

I guess the only 'safe' way to move an active extension out of the tce directory is to reboot into base, move the files, then reboot?
Title: Re: direct me to TCZ moving tools?
Post by: Juanito on November 30, 2009, 05:19:46 AM
You could try saving related extensions in a number of folders under tce and then modifying one of the dep files (eg the first one alphabetically) in each folder to contain the names of all the extensions in that folder - that way you only have to click on one extension name per folder to load them all.
Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on November 30, 2009, 05:43:52 AM
Juanito - yes, that's an idea. Only concern is that I'll probably have to duplicate some extension files into many folders ... ? :/

New idea - what if I do the deed and try to hack the tce loading scripts? Process:


Then, I don't duplicate any files, I don't have to worry about moving/removing mounted extensions, I can just reboot one time to both remove/add extensions, and I don't have to re-download things ...

and I can stick to editing text files rather than writing ugly scripts.  :P

I guess a perk would be if I could get appbrowser to download a whole .profile ... which wouldn't be hard, I guess ... ? Then extension collections become more easily transferable.
Title: Re: direct me to TCZ moving tools?
Post by: Jason W on November 30, 2009, 06:07:21 AM
Of course, using "ln -s" instead of cp or mv would save space.  You could create a new directory, then create a list of extensions called list, and then do something like"

for I in `cat list`; do movetcsh "$I" /mnt/newdirectory; done


After replacing the mv command in the script with "ln -s".  Then use that new directory as your tce directory.  Then the "yes n |" part could be done away with, as well as the check for being the current tce directory.  Each new directory will be your profile, and will not take up more space.
 
May or may not help with what you want, am at work and don't have time to ponder it further.  But it would work.

You would have each profile be a different tce directory with extensions that are symlinks to files in the main tce directory. 
Title: Re: direct me to TCZ moving tools?
Post by: Jason W on November 30, 2009, 06:22:52 AM
Of course, you could even add a --ram flag  that would echo each extension name to a tcz2ram.lst in the new directory if you wanted the extension and it's deps to be loaded to ram. 
Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on November 30, 2009, 11:38:58 AM
Ah ... using links in subdirectories sounds like it could work.

I am not sure if that would leave the same problem with removing an extension from the current profile/setup? I guess I could just copy the directory (all the links) into a new directory, and then reboot into the new directory ...

That's a lot closer, though!

Now it's my turn to go off to work, and I'll have to come back to this.
Title: Re: direct me to TCZ moving tools?
Post by: Jason W on November 30, 2009, 11:50:07 AM
In my thinking, moving extensions around is not needed.  But linking them to new tce directories (profiles) would create the different profiles.  Then to update the all extensions, update the main tce directory and all the profiles would be current.  That is actually better than having multiple copies in different directories.
Title: Re: direct me to TCZ moving tools?
Post by: jpeters on November 30, 2009, 03:46:54 PM
In my thinking, moving extensions around is not needed.  But linking them to new tce directories (profiles) would create the different profiles.  Then to update the all extensions, update the main tce directory and all the profiles would be current.  That is actually better than having multiple copies in different directories.

That was also my approach with the load-grp script.
Title: Re: direct me to TCZ moving tools?
Post by: Lee on November 30, 2009, 08:39:31 PM
How about a different grub entry for each profile:

Code: [Select]
title TC2.6 Profile A
root (hd0,0)
kernel /boot/tc2.6/bzImage quiet maxloop=255 tce=sda1/tce2.6a
initrd /boot/tc2.6/tinycore.gz

title TC2.6 Profile B
root (hd0,0)
kernel /boot/tc2.6/bzImage quiet maxloop=255 tce=sda1/tce2.6b
initrd /boot/tc2.6/tinycore.gz

etc

where sda1/tce2.6a and sda1/tce2.6b just contain symlinks to extensions stored in some local directory - perhaps in sd1/tce2.6a/optional.

Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on December 01, 2009, 05:53:51 AM
hats: thanks for moving the thread. Has definitely become a scripts topic.

jpeters: ooh, I haven't seen that. I'll read up here on the forums ... maybe you've had a conversation about it?

Lee: yeah, I considered doing that. Frankly, I'm too irregular with hitting esc (or selecting another profile) at bootup. I'd like it if I can just set a profile and reboot. Keep it simple.

Today I tried putting symbolic links into my extension directory ... the application browser doesn't like that - it won't load a link. Same on reboot - links don't get loaded.

I'll try hard links, and read up on the tce-load and tce-setup scripts. Happy for advice!

Down side with relying on links is that it's not possible to use profiles on a FAT32 drive ... (or at least I couldn't massage that into happening).
Title: Re: direct me to TCZ moving tools?
Post by: gerald_clark on December 01, 2009, 11:33:42 AM
Use relative symlinks.
cd tce.set1
ln -s ../tce.all/mc.tcz mc.tcz
Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on December 01, 2009, 12:42:49 PM
Ah - the links indeed work fine. I had moved the target files ... [sigh]

Now adjusting my little script to set up those links.

Haven't yet finished how to set them up for next boot, but it will be possible.

Title: Re: direct me to TCZ moving tools?
Post by: dentonlt on December 02, 2009, 05:26:35 AM
Finally up and running! So, I have some tools to make and enable "profiles", a group of extensions that will load (or not load) at next boot. Components:

* a few lines in tce-setup to search for a profile; bail to normal if there is no profile
* setprofile - a script to set what extension set/profile will boot next
* saveprofile, restoreprofile, linktc.sh - scripts for making an extension profile
* data in /profiles, a subdirectory of the main tce directory - stores profile lists (about 40k for my current lists and their symlinks, future state, current state, and error messages)

So ... I can pick a profile that will be enabled/disabled at next boot. I am using it to select from a few different setups:

* base
* build environment
* netbook
* default (load everything as normal by disabling profile)

I do love GRUB, but it's nice not to have to set up different GRUB entries and select them at boot time.

I guess this setup isn't philosophically tight - It relies on having data outside of mydata.tgz, stored alongside the extensions. But it works for me =)

Thank you all for the help. Special kudos to Jason W - the linking script is a very basic change to the movetc script you sent earlier. I really appreciate it.

EDIT:
now that I've gone through all that ... it remains quite complex! I'm thinking of other ways to skin this.

EDIT:
going through and writing scripts to do this via .dep files ... (I'll dig up who told me to do that in the first place :o ). It may be possible to create an extension that pulls this off.