WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Script to extract and re-package core.gz  (Read 14471 times)

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Script to extract and re-package core.gz
« on: June 05, 2013, 12:04:05 AM »
Script to extract and re-package core.gz, core64.gz and corepure64.gz

With core updates coming so frequently I decided it was time to automate to some degree the process of extracting core, add my personal touches like updated driver modules etc etc then finally re-packaging.  I created this script to answer that call and figured maybe it will be of use to others in the same predicament.

This script has made easy work of re-packaging core.gz and use it surprisingly often, every new core update actually.  sharing it seemed the most logical course, hope someone finds it as useful.


Simply place the script together with core.gz in /tmp directory then open a terminal and run once to extract.  Make changes as necessary (I mostly update kernel modules rather than use extensions), then run the script again to re-package.  Use with core 4.x


this is updated version as of 05/26/14
Code: [Select]
#!/bin/sh
# Author coreplayer2
# date 05/26/14

. /etc/init.d/tc-functions


################################################################

##draw at full screen width a line in blue ( use xy=(line#) for row position )
f_line() {
wide=`stty size | cut -d" " -f2`
printf "\033["$xy";0H${BLUE}"
printf '%*s\n' "$(( wide-1 ))" '' | tr ' ' =
printf "${NORMAL}"
}

f_cleanup() {
sudo rm -rf /tmp/core_extract
sudo rm -rf /tmp/core_package
}

f_extract() {
# prepare directories
mkdir -p /tmp/core_extract
mkdir -p /tmp/core_package

#extract core*.gz
cd /tmp/core_extract && zcat /tmp/$mycore | $user cpio -i -H newc -d
}

f_repack() {
cd /tmp/core_extract

#update modules list
$user depmod -a -b /tmp/core_extract `uname -r`

#re-package and compress new core*gz
$user find | $user cpio -o -H newc | $user gzip -2 > /tmp/core_package/$mycore
$user advdef -z4 /tmp/core_package/$mycore
}

f_chkconn(){
echo "${MAGENTA}Testing connection to server, please wait...${NORMAL}"
busybox wget -s -T 20 http://tinycorelinux.net/index.html
case $? in
0) echo "${GREEN}connection ok${NORMAL}"; cx=0;;
1) echo "${YELLOW}check internet connection, then try again${NORMAL}"; sleep 5; cx=1;;
esac
}

################################################################
##main script

trap 'f_cleanup ; trap 2 ; kill -2 $$' 1 2 3 13 15
##install advcomp dep
if [ ! -e /usr/local/tce.installed/advcomp ]; then
      tce-load -i advcomp
fi
if [ ! -e /usr/local/tce.installed/advcomp ]; then
  f_chkconn
    if [ "x$cx" == x0 ]; then
      echo -e " ${BLUE}fetching missing dependancy${NORMAL} "
      tce-load -wil advcomp
    fi
fi
if [ ! -e /usr/local/tce.installed/advcomp ]; then
 echo -e " ${YELLOW}Dependancy not found, exiting...${NORMAL} "
 sleep 5
 exit
fi

clear
printf "\033[2;2H${MAGENTA}Info${NORMAL} "
xy=3
f_line

cat <<-info
 Use this script to extract and/or re-package core.gz,
 core64.gz, module.gz or corepure64.gz
 
 Notes
 1. When repackaging a x86_64 core/module, use this script
    on same architecture (booted to core64 or corepure64)

 2. Extracting will remove any previously found extracted
    /tmp/core_extract working directories

 Copy your core*.gz or module.gz to /tmp directory then

 Step 1 (extract)
 Select "E" to extract when prompted

 Then after completing modifications

 Step 2 (repackage)
 run this script again to repackage
 Select "P" to re-package when prompted

info
read -p "${CYAN} Enter ${NORMAL}when ready to continue, or ${YELLOW}Ctrl+C ${NORMAL}to quit "


[ `/usr/bin/id -u` -ne 0 ] && user=sudo


if [ -e /tmp/core_package/*.gz ]; then
echo -e "\n ${YELLOW}re-packaged core/module.gz exists already, "
read -n1 -p " Delete and startover..? ${WHITE}(${MAGENTA}y${NORMAL}/${MAGENTA}n${WHITE}) " yn
case $yn in
[yY] ) f_cleanup;;
[nN] ) echo -e "\n exiting.."; sleep 2; exit;;
esac
fi

##test for 1 working archive in /tmp dir
archive=$(find /tmp -maxdepth 1 -name "*.gz" | wc -l)
if [ x"$archive" == x0 ]; then
 clear
 printf "\033[2;4H${MAGENTA}Archives found = ${YELLOW}$archive${NORMAL} "
 xy=3
 f_line
 printf "\033[4;4H${YELLOW}Archive not found, \n   ${NORMAL}Please copy core/module*.gz to /tmp directory and try again\n\n "
 read -p "  ${CYAN}Enter ${NORMAL}when ready to exit "
 exit
elif [ x"$archive" != x0 -a x"$archive" != x1 ]; then
 clear
 printf "\033[2;4H${MAGENTA}Archives found = ${YELLOW}$archive${NORMAL}\n "
 xy=3
 f_line
 printf "\033[4;4H${YELLOW}Only 1 expected\n   ${NORMAL}Please remove unneeded core-module*.gz copies from /tmp directory and try again\n\n "
 read -p "  ${CYAN}Enter ${NORMAL}when ready to exit "
 exit
fi


mycore=$($user find /tmp -maxdepth 1 -name "*.gz")
mycore=${mycore##*/}

##Extract or RePackage
if [ -d /tmp/core_extract ]; then
clear
printf "\033[2;4H${MAGENTA}Processing ${WHITE}$mycore${NORMAL} "
xy=3
f_line

## Extract or package choice
printf "\033[5;4H${BLUE}Choose ${WHITE}(${MAGENTA}P${NORMAL}/${MAGENTA}p${WHITE})${BLUE} to re-Package, or"
printf "\033[6;4HChoose ${WHITE}(${MAGENTA}E${NORMAL}/${MAGENTA}e${WHITE})${BLUE} to startover and Extract, or${NORMAL} "
printf "\033[7;4H${YELLOW}Ctrl+C ${NORMAL}to quit \n\n"

while [ "$ep" != [ep] ]; do
read -s -n1 -p "" ep

case $ep in
[Ee] )
echo -e "${BLUE} Extracting ${WHITE}$mycore${NORMAL} \n"
f_cleanup
set -x
f_extract
set +x
echo -e "\n${WHITE} $mycore ${BLUE}extract completed${NORMAL} \n\n when ready, run this script again with ${MAGENTA}P${NORMAL} option to re-Package \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
;;

[Pp] )
    if [ -d /tmp/core_extract ]; then

echo -e "${BLUE}RePacking ${WHITE}$mycore ${BLUE}please wait...${NORMAL} \n"
set -x
f_repack
set +x
echo -e "\n${WHITE} $mycore ${BLUE}repackage complete${NORMAL} "
sudo rm -rf /tmp/core_extract
echo -e "\n re-packaged $mycore can be found in /tmp/core_package/$mycore \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
    else
echo -e "\n${RED} Invalid option selected \n No working \"Extracted directory\" found, exiting... ${NORMAL} "
sleep 3

    fi

;;

* )
echo -e "Key not supported, please try again "
;;

esac
done
else
clear
printf "\033[2;4H${MAGENTA}Processing ${WHITE}$mycore${NORMAL} "
xy=3
f_line

## Extract or package choice
printf "\033[5;4H${BLUE}Choose ${WHITE}(${MAGENTA}E${NORMAL}/${MAGENTA}e${WHITE})${BLUE} to Extract archive, or${NORMAL} "
printf "\033[6;4H${YELLOW}Ctrl+C ${NORMAL}to quit \n\n"

while [ "$e" != e ]; do
read -s -n1 -p "" e

case $e in
[Ee] )
echo -e "${BLUE} Extracting ${WHITE}$mycore${NORMAL} \n"
f_cleanup
set -x
f_extract
set +x
echo -e "\n${WHITE} $mycore ${BLUE}extract completed${NORMAL} \n\n when ready, run this script again with ${MAGENTA}P${NORMAL} option to re-Package \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
;;


* )
echo -e "Key not supported, please try again "
;;

esac
done
fi

exit


« Last Edit: May 26, 2014, 10:42:27 PM by coreplayer2 »

Offline beroje

  • Full Member
  • ***
  • Posts: 130
Re: Script to extract and re-package core.gz
« Reply #1 on: June 05, 2013, 08:04:00 PM »

Gracias

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Script to extract and re-package core.gz
« Reply #2 on: May 26, 2014, 10:39:27 PM »
Ok, I've been working on this script lately and now have a revised version which runs on corepure64 and core 32 bit tc-5.x systems. 

I have intention to submit the newly created extension, however am still testing on various systems.

I've attached the current script as a file here if anyone is interested.


if adding or removing driver modules from an x86_64 version of core, ie; core64.gz, or corepure64.gz, or modules64.gz use this extension / script from a booted corepure64 system.  Likewise if adding or removing driver modules from an x86 (32bit) version of tinycore, ie; core.gz, or corepure.gz, or modules.gz, use this extension / script from a booted tinycore x86 system.

copy modules64.gz, or core.gz etc etc to /tmp dir and simply run the script (or load the extension and run from the desktop menu item), then follow the prompts


After new modules are added, run the script again this time use the P option to repackage


new modules64.gz or core.gz archives may be found in /tmp/core_package/

Am grateful for feedback


« Last Edit: May 26, 2014, 10:47:35 PM by coreplayer2 »

Offline mainmachine

  • Newbie
  • *
  • Posts: 1
Re: Script to extract and re-package core.gz
« Reply #3 on: May 27, 2014, 06:59:51 PM »
I have found this script to be incredibly useful in a remastering project I've worked on the past three months.

It definitely deserves to be it's own package.

Great time saver... keep up the good work!

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Script to extract and re-package core.gz
« Reply #4 on: July 22, 2014, 09:55:06 AM »
thanks mainmachine, am glad someone found it useful. 

Finally got around to making an extension based on this script, am submitting " core-remaster.tcz "  asap to both x86 & x86_64 repo's
(remember it's intended for minor modifications of core.gz, corepure64.gz, modules*.gz, or rootfs*.gz only)


here is the final script
Code: [Select]
#!/bin/sh
# Author coreplayer2
# date 07/12/14

. /etc/init.d/tc-functions

# requires a connection to download advcomp.tcz if not found locally


################################################################

##draw at full screen width a line in blue ( use xy=(line#) for row position )
f_line() {
wide=`stty size | cut -d" " -f2`
printf "\033["$xy";0H${BLUE}"
printf '%*s\n' "$(( wide-1 ))" '' | tr ' ' =
printf "${NORMAL}"
}

f_cleanup() {
sudo rm -rf /tmp/core_extract
sudo rm -rf /tmp/core_package
}

f_extract() {
# prepare directories
mkdir -p /tmp/core_extract
mkdir -p /tmp/core_package

#extract core*.gz
cd /tmp/core_extract && zcat /tmp/$mycore | $user cpio -i -H newc -d
}

f_repack() {
cd /tmp/core_extract

#update modules list
$user depmod -a -b /tmp/core_extract `uname -r`

#re-package and compress new core*gz
$user find | $user cpio -o -H newc | $user gzip -2 > /tmp/core_package/$mycore
$user advdef -z4 /tmp/core_package/$mycore
}

f_chkconn(){
echo "${MAGENTA}Testing connection to server, please wait...${NORMAL}"
busybox wget -s -T 20 http://tinycorelinux.net/index.html
case $? in
0) echo "${GREEN}connection ok${NORMAL}"; cx=0;;
1) echo "${YELLOW}check internet connection, then try again${NORMAL}"; sleep 5; cx=1;;
esac
}

################################################################
##main script

trap 'f_cleanup ; trap 2 ; kill -2 $$' 1 2 3 13 15
##install advcomp dep
if [ ! -e /usr/local/tce.installed/advcomp ]; then
      tce-load -i advcomp
fi
if [ ! -e /usr/local/tce.installed/advcomp ]; then
  f_chkconn
    if [ "x$cx" == x0 ]; then
      echo -e " ${BLUE}fetching missing dependancy${NORMAL} "
      tce-load -wil advcomp
    fi
fi
if [ ! -e /usr/local/tce.installed/advcomp ]; then
 echo -e " ${YELLOW}Dependancy not found, exiting...${NORMAL} "
 sleep 5
 exit
fi

clear
printf "\033[1;2H${MAGENTA}core.gz remaster${NORMAL} "
xy=2
f_line

cat <<-info
 To extract then re-package core.gz,
 core64.gz, module.gz or corepure64.gz

 First copy core*.gz to /tmp directory then

 To extract
 Select "E" when prompted

 To repackage
 run this script again
 Select "P" when prompted

 Notes
 1. When repackaging a x86_64 core/module.gz, use this script
    on same architecture (booted to core64 or corepure64)

 2. Extracting will remove any previously extracted
    /tmp/core_extract working directories

info
read -p "${CYAN} Enter ${NORMAL}when ready to continue, or ${YELLOW}Ctrl+C ${NORMAL}to quit "


[ `/usr/bin/id -u` -ne 0 ] && user=sudo


if [ -e /tmp/core_package/*.gz ]; then
echo -e "\n ${YELLOW}re-packaged core/module.gz exists already, "
read -n1 -p " Delete and startover..? ${WHITE}(${MAGENTA}y${NORMAL}/${MAGENTA}n${WHITE}) " yn
case $yn in
[yY] ) f_cleanup;;
[nN] ) echo -e "\n exiting.."; sleep 2; exit;;
esac
fi

##test for 1 working archive in /tmp dir
archive=$(find /tmp -maxdepth 1 -name "*.gz" | wc -l)
if [ x"$archive" == x0 ]; then
 clear
 printf "\033[2;4H${MAGENTA}Archives found = ${YELLOW}$archive${NORMAL} "
 xy=3
 f_line
 printf "\033[4;4H${YELLOW}Archive not found, \n   ${NORMAL}Please copy core/module*.gz to /tmp directory and try again\n\n "
 read -p "  ${CYAN}Enter ${NORMAL}when ready to exit "
 exit
elif [ x"$archive" != x0 -a x"$archive" != x1 ]; then
 clear
 printf "\033[2;4H${MAGENTA}Archives found = ${YELLOW}$archive${NORMAL}\n "
 xy=3
 f_line
 printf "\033[4;4H${YELLOW}Only 1 expected\n   ${NORMAL}Please remove unneeded core-module*.gz copies from /tmp directory and try again\n\n "
 read -p "  ${CYAN}Enter ${NORMAL}when ready to exit "
 exit
fi


mycore=$($user find /tmp -maxdepth 1 -name "*.gz")
mycore=${mycore##*/}

##Extract or RePackage
if [ -d /tmp/core_extract ]; then
clear
printf "\033[2;4H${MAGENTA}Processing ${WHITE}$mycore${NORMAL} "
xy=3
f_line

## Extract or package choice
printf "\033[5;4H${BLUE}Choose ${WHITE}(${MAGENTA}P${NORMAL}/${MAGENTA}p${WHITE})${BLUE} to re-Package, or"
printf "\033[6;4HChoose ${WHITE}(${MAGENTA}E${NORMAL}/${MAGENTA}e${WHITE})${BLUE} to startover and Extract, or${NORMAL} "
printf "\033[7;4H${YELLOW}Ctrl+C ${NORMAL}to quit \n\n"

while [ "$ep" != [ep] ]; do
read -s -n1 -p "" ep

case $ep in
[Ee] )
echo -e "${BLUE} Extracting ${WHITE}$mycore${NORMAL} \n"
f_cleanup
set -x
f_extract
set +x
echo -e "\n${WHITE} $mycore ${BLUE}extract completed${NORMAL} \n\n when ready, run this script again with ${MAGENTA}P${NORMAL} option to re-Package \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
;;

[Pp] )
    if [ -d /tmp/core_extract ]; then

echo -e "${BLUE}RePacking ${WHITE}$mycore ${BLUE}please wait...${NORMAL} \n"
set -x
f_repack
set +x
echo -e "\n${WHITE} $mycore ${BLUE}repackage complete${NORMAL} "
sudo rm -rf /tmp/core_extract
echo -e "\n re-packaged $mycore can be found in /tmp/core_package/$mycore \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
    else
echo -e "\n${RED} Invalid option selected \n No working \"Extracted directory\" found, exiting... ${NORMAL} "
sleep 3

    fi

;;

* )
echo -e "Key not supported, please try again "
;;

esac
done
else
clear
printf "\033[2;4H${MAGENTA}Processing ${WHITE}$mycore${NORMAL} "
xy=3
f_line

## Extract or package choice
printf "\033[5;4H${BLUE}Choose ${WHITE}(${MAGENTA}E${NORMAL}/${MAGENTA}e${WHITE})${BLUE} to Extract archive, or${NORMAL} "
printf "\033[6;4H${YELLOW}Ctrl+C ${NORMAL}to quit \n\n"

while [ "$e" != e ]; do
read -s -n1 -p "" e

case $e in
[Ee] )
echo -e "${BLUE} Extracting ${WHITE}$mycore${NORMAL} \n"
f_cleanup
set -x
f_extract
set +x
echo -e "\n${WHITE} $mycore ${BLUE}extract completed${NORMAL} \n\n when ready, run this script again with ${MAGENTA}P${NORMAL} option to re-Package \n"
read -p "${CYAN} Enter ${NORMAL}when ready to exit "
exit
;;


* )
echo -e "Key not supported, please try again "
;;

esac
done
fi

exit
« Last Edit: July 22, 2014, 10:14:01 AM by coreplayer2 »

Offline nick65go

  • Hero Member
  • *****
  • Posts: 799
Re: Script to extract and re-package core.gz
« Reply #5 on: September 17, 2014, 09:57:09 AM »
Hi coreplayer2,

Thank you for your very good script. I use it for its main purpose, but also to learn ash scripting, so I analyzed each line of it. I have a suggestion for your line:
$user depmod -a -b /tmp/core_extract `uname -r`

depmod -h shows that busybox version does not use the "-a" parameter.
also depmod create file "modules.symbols" which is not in original corepure64.gz, so it can be deleted
moreover, the file module.dep has strings like "kernel.tcelocal/drivers" instead of simple "kernel/drivers/" as in original gz (because the simlink under /lib/module/); so a "sed" command should be good;

busybox stty allow me a huge (multiple of 4800)  serial speed of 921600 bit/s for drawing that blue line on screen..
wide=`stty 921600 size | cut -d" " -f2`


summary: delete extra file, process with sed, then gzip the image;
Oh, one more time: Many thanks for the script!
« Last Edit: September 27, 2014, 10:54:35 AM by nick65go »

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Script to extract and re-package core.gz
« Reply #6 on: November 02, 2014, 09:59:52 PM »
Thanks for the improvements, I'll update this ASAP


 

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Script to extract and re-package core.gz
« Reply #7 on: December 20, 2014, 10:18:51 PM »
OMG!!!   I lost this thread..   ok I've got some work to do here  brb with an update

Offline ipmeel

  • Jr. Member
  • **
  • Posts: 59
Re: Script to extract and re-package core.gz
« Reply #8 on: October 08, 2016, 11:49:48 PM »
User extension 'core-remaster' and this script.
System hangs while boot at.. 'Setting hostname to box Done.'

Offline ipmeel

  • Jr. Member
  • **
  • Posts: 59
Re: Script to extract and re-package core.gz
« Reply #9 on: October 09, 2016, 01:27:19 AM »
User extension 'core-remaster' and this script.
System hangs while boot at.. 'Setting hostname to box Done.'
Please ignore.