Tiny Core Linux
General TC => Programming & Scripting - Unofficial => Topic started by: Jason W on July 25, 2009, 07:33:55 AM
-
I am going to be auditing extensions more closely, and basically using a script such as this one. It would behoove extension submitters to run this script in the directory where their extension files are before submitting to help ensure the integrity of the repo. This basically checks the md5s, checks to see if the tce and tcz extensions are valid tar.gz or mountable files, and checks the info files for basic info. Suggestions for features or improvement are welcome. I will be adding improvements in time but this I think is a good starting point. Proper list files are created by this script.
Errors that are caught before submission save a lot of time for everyone compared to those found later, so thanks in advance for checking extensions before submission. And thanks for the continued hard work that is being put into contributions.
TC version 3 Script attached.
#!/bin/sh
# Extension audit script
. /etc/init.d/tc-functions
MIRROR=`cat /opt/tcemirror`
VERSION="3.x"
THISDIR=`pwd`
if [ ! -f /usr/local/tce.installed/file ]; then
echo "${RED}Install the file.tcz extension.${NORMAL}"
exit 1
fi
if [ ! -f /usr/local/tce.installed/diffutils ]; then
echo "${RED}Install the diffutils.tcz extension.${NORMAL}"
exit 1
fi
if [ ! -f /usr/local/tce.installed/squashfs-tools-4.x ]; then
echo "${RED}Install the squashfs-tools extension.${NORMAL}"
exit 1
fi
if [ ! -f /usr/local/tce.installed/wget ]; then
echo "${RED}Install wget extension.${NORMAL}"
exit 1
fi
checkstartup() {
if [ -d "$TMP2"/usr/local/tce.installed/ ]; then
if [ `ls "$TMP2"/usr/local/tce.installed/ | wc -l` == "1" ]; then
OUT=$(ls -d -l "$TMP2"/usr/local/tce.installed)
PERM=$(echo $OUT | cut -d" " -f1)
OWNER=$(echo $OUT | cut -d" " -f3-4)
[ "$PERM" == "drwxrwxr-x" ] || [ "$PERM" == "drwxrwsr-x" ] || export FILE=1
[ "$OWNER" == "root staff" ] || export FILE=1
if [ -f "$TMP2"/usr/local/tce.installed/"$BASENAME" ]; then
if [ ! -x "$TMP2"/usr/local/tce.installed/"$BASENAME" ]; then
export FILE=1
echo "${YELLOW}Will rebuild ${BLUE}"$F"${YELLOW} for wrong /usr/local/tce.installed perms.${NORMAL}"
echo "$F" >> /tmp/submitqc/wrongstartscriptperms
fi
fi
if [ ! -z `ls "$TMP2"/usr/local/tce.installed/` ] && [ ! -f "$TMP2"/usr/local/tce.installed/"$BASENAME" ]; then
echo "${YELLOW}Will rebuild ${BLUE}"$F"${YELLOW} for wrong /usr/local/tce.installed script name.${NORMAL}"
echo "$F" >> /tmp/submitqc/wrongstartscriptname
export FILE=1
fi
if [ "$FILE" == "1" ]; then
rebuild
fi
unset FILE
else
echo "${BLUE}"$F"${RED} has more than one /usr/local/tce.installed script. Please remake.${NORMAL}"
unset FILE
fi
fi
}
checkmaintainer() {
unset NAMES
if wget -O /tmp/submitqc/."$F".info "$MIRROR""$VERSION"/tcz/"$F".info > /dev/null 2>&1; then
MAINTAINER=`grep "Extension_by" /tmp/submitqc/."$F".info | cut -d: -f2`
SUBMITTER=`grep "Extension_by" "$F".info | cut -d: -f2`
echo "Extension: "$F"" >> /tmp/submitqc/tcemaintainer
echo "Maintainer is "$MAINTAINER"" >> /tmp/submitqc/tcemaintainer
echo "Submitter is "$SUBMITTER"" >> /tmp/submitqc/tcemaintainer
echo " " >> /tmp/submitqc/tcemaintainer
echo " "
echo "${BLUE}"$F"${YELLOW} is maintained by ${GREEN}"$MAINTAINER"${YELLOW}. You are ${GREEN}"$SUBMITTER"${YELLOW}. Make sure the maintainer is aware you are updating his extension.${NORMAL}"
fi
echo " "
echo "$F" > /tmp/submitqc/.extname
echo "EXT: "$F" - SIMILARLY NAMED EXTENISIONS:" >> /tmp/submitqc/similarextensions
> /tmp/submitqc/.temp
for I in `grep $(basename "$F" .tcz) /tmp/submitqc/.info.lst`; do echo "$I".tcz >> /tmp/submitqc/.temp; done
for I in `cat /tmp/submitqc/.info.lst `; do grep "$I" /tmp/submitqc/.extname >/dev/null && echo "$I".tcz >> /tmp/submitqc/.temp; done
cat /tmp/submitqc/.temp | uniq >> /tmp/submitqc/similarextensions
NAMES=`cat /tmp/submitqc/.temp | uniq`
if [ ! -z "$NAMES" ]; then
echo "${BLUE}"$F"${YELLOW} has some similarly named extensions in the repo. Make sure yours does not \
collide with these:${NORMAL}"
echo " "
echo "${GREEN}"$NAMES"${NORMAL}"
echo " " >> /tmp/submitqc/similarextensions
echo " "
fi
}
checkdiff() {
[ -f /tmp/submitqc/listfilediffs/"$F".difflist ] && rm /tmp/submitqc/listfilediffs/"$F".difflist
[ -d /tmp/submitqc/listfilediffs/ ] || mkdir -p /tmp/submitqc/listfilediffs/
#> /tmp/submitqc/listfilediffs/"$F".difflist
if wget -O /tmp/submitqc/."$F".repolist "$MIRROR""$VERSION"/tcz/"$F".list > /dev/null 2>&1; then
diff -NBa -U 0 /tmp/submitqc/."$F".repolist "$F".list > /tmp/submitqc/listfilediffs/"$F".difflist
if [ -s /tmp/submitqc/listfilediffs/"$F".difflist ]; then
echo " "
sed -i '1,3d' /tmp/submitqc/listfilediffs/"$F".difflist
sed -i '1i ---' /tmp/submitqc/listfilediffs/"$F".difflist
sed -i '1i Minus means file is in the existing repo ext. and not in yours' /tmp/submitqc/listfilediffs/"$F".difflist
sed -i '1i Plus means file is in your ext. and not in repos' /tmp/submitqc/listfilediffs/"$F".difflist
echo " "
echo ${BLUE}"$F": "${YELLOW}There are filename differences in your extension and the one\
existing in the repo,they are noted above and stored in ${BLUE}/tmp/submitqc/listfilediffs/"$F".difflist.${NORMAL}"
fi
sleep 5
fi
[ -f /tmp/submitqc/."$F".repolist ] && rm /tmp/submitqc/."$F".repolist
}
checkappend() {
if find "$TMP2" -maxdepth 1 | grep "_1"; then
export APPEND=1
fi
if [ "$APPEND" == "1" ]; then
echo "${BLUE}"$F"${RED} has unwanted data appended to it. Please remake.${NORMAL}"
echo "$F" >> /tmp/submitqc/appendeddata
unset APPEND
sleep 3
fi
}
checkblock() {
if ! unsquashfs -s "$F" | grep "Block size" | grep "4096" > /dev/null; then
echo "${BLUE}"$F"${YELLOW} is not 4096 block size. Rebuilding with -b 4096.${NORMAL}"
rebuild
fi
}
rebuild() {
mkdir "$F".123456789
cp -a "$TMP2"/* "$F".123456789/
chmod 775 "$F".123456789/
#if [ "$BADPERM" == "1" ]; then
#for I in `find "$F".123456789/ -type d`; do stat -c '%a' "$I" | grep 755 >/dev/null || \
#stat -c '%a' "$I" | grep 775 >/dev/null || stat -c '%a' "$I" | grep 2755 >/dev/null || stat -c '%a' "$I" | grep 2775 >/dev/null || chmod 755 "$I" ; done
#fi
if [ ! -z `ls "$F".123456789/usr/local/tce.installed/` ] && [ ! -f "$F".123456789/usr/local/tce.installed/"$BASENAME" ]; then
mv `find "$F".123456789/usr/local/tce.installed/ -not -type d | tail -n 1` "$F".123456789/usr/local/tce.installed/"$BASENAME"
fi
if [ -d "$F".123456789/usr/local/tce.installed ]; then
chmod -R 775 "$F".123456789/usr/local/tce.installed
chown -R root:staff "$F".123456789/usr/local/tce.installed
fi
sudo busybox umount -d "$TMP2"
mksquashfs "$F".123456789 "$F".new -b 4096
mv "$F".new "$F"
rm -r "$F".123456789
sudo busybox mount -o loop "$F" "$TMP2" > /dev/null 2>&1
[ -f "$F".md5.txt ] && rm "$F".md5.txt
}
checkbasedirperm() {
if ! su tc -c "ls "$THISDIR"/"$TMP2"" > /dev/null 2>&1; then
echo "${YELLOW}Rebuilding "$F" for unreadable base squashfs directory.${NORMAL}"
rebuild
fi
}
checkdirperms() {
unset BADPERM
for I in `find "$TMP2" -type d`; do
if ! stat -c '%a' "$I" | grep 755 > /dev/null && \
! stat -c '%a' "$I" | grep 775 >/dev/null && ! stat -c '%a' "$I" | grep 2755 >/dev/null && ! stat -c '%a' "$I" | grep 2775 >/dev/null; then
#export BADPERM=1
DIR=`echo "$I" | sed "s:"$TMP2"::"`
echo ""$F": `stat -c '%a' "$I"` "$DIR"" >> /tmp/submitqc/baddirperms
echo "${BLUE}"$F": ${GREEN}"$DIR"${YELLOW} directory has suspicious permissionss: ${GREEN}`stat -c '%a' "$I"`${NORMAL}"
fi
done
sleep 3
}
echo_green() {
if [ -f "$F".md5.txt ]; then
echo "${BLUE}"$F"${GREEN} is a valid tcz file. Checking MD5 for ${BLUE}"$F":${NORMAL}"
md5sum -c "$F".md5.txt
if [ ! "$?" == "0" ]; then
echo "${RED}Md5sum failed for "$F"${NORMAL}"
echo "$F" >> /tmp/wrongmd5
fi
else
echo "${BLUE}"$F"${GREEN} is a valid tcz file. Creating MD5 for ${BLUE}"$F":${NORMAL}"
md5sum "$F" > "$F".md5.txt
fi
cd "$TMP2"
find `ls` -not -type d > ../"$F".list
cd ..
zsyncmake "$F" > /dev/null 2>&1
}
checkdep() {
if `ls *.dep > /dev/null 2>&1`; then
[ "$(ls -A /tmp/submitqc/missingdeps/)" ] && rm /tmp/submitqc/missingdeps/*
for I in `ls *.dep`; do
for F in `cat "$I"`; do
if echo "$F" | grep "2.6.33.3" > /dev/null 2>&1; then
echo "${RED}The ${BLUE}"$F"${RED} entry in ${BLUE}"$I" ${RED}needs to use the KERNEL variable instead of kernel version number.${NORMAL}"
echo "The "$F" entry in "$I" needs to use the KERNEL variable instead of kernel version number." >> /tmp/submitqc/missingdeps/"$I"
echo " " >> /tmp/submitqc/missingdeps/"$I"
elif echo "$F" | grep "KERNEL" > /dev/null 2>&1; then
if B=`echo "$F" | sed 's:KERNEL:2.6.33.3-tinycore:g'` && wget --spider "$MIRROR""$VERSION"/tcz/"$B" \
> /dev/null 2>&1; then
sleep .001
elif B=`echo "$F" | sed 's:KERNEL:2.6.33.3-tinycore64:g'` && wget --spider "$MIRROR""$VERSION"/tcz/"$B" \
> /dev/null 2>&1; then
sleep .001
else
echo ""$F" is missing in the repo as a dependency of "$I"" >> /tmp/submitqc/missingdeps/"$I"
echo "${BLUE}"$F"${RED} is missing in the repo as a dependency of ${BLUE}"$I"${NORMAL}"
echo " " >> /tmp/submitqc/missingdeps/"$I"
fi
else
if ! wget --spider "$MIRROR""$VERSION"/tcz/"$F" \
> /dev/null 2>&1; then
echo ""$F" is missing in the repo as a dependency of "$I"" >> /tmp/submitqc/missingdeps/"$I"
echo "${BLUE}"$F"${RED} is missing in the repo as a dependency of ${BLUE}"$I"${NORMAL}"
echo " " >> /tmp/submitqc/missingdeps/"$I"
if [ -f "$F" ]; then
echo "${BLUE}"$F"${GREEN}, however is present in this batch of extensions being checked for upload.${NORMAL}"
echo ""$F", however, is present in this batch of extensions being checked for upload." >> /tmp/submitqc/missingdeps/"$I"
fi
fi
fi
done
done
[ "$(ls -A /tmp/submitqc/missingdeps/)" ] || echo "${GREEN}Dep files seem to have no detectable errors.${NORMAL}"
fi
}
checkinfo() {
for I in `ls *.info`; do
NAME=`basename "$I" .info`
INFONAME=`cat "$I" | grep "Title" | awk '{print $2}'`
if [ ! "$NAME" == "$INFONAME" ]; then
echo "${RED}Name field in ${BLUE}"$I"${RED} is incorrect.${NORMAL}"
echo "Name: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
fi
echo "The following errors are found in "$I". No news is good news:"
[ -z `awk '/Change-log:/ { print $2 }' "$I"` ] && echo "Change-log: field is not valid in "$I"." && echo "Change-log: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Comments:/ { print $2 }' "$I"` ] && echo "Comments: field is not valid in "$I"." && echo "Comments: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Original-site:/ { print $2 }' "$I"` ] && echo "Original-site: field is not valid in "$I"." && echo "Original-site: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Author:/ { print $2 }' "$I"` ] && echo "Author: field is not valid in "$I"." && echo "Author: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Title:/ { print $2 }' "$I"` ] && echo "Title: field is not valid in "$I"." && echo "Title: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Description:/ { print $2 }' "$I"` ] && echo "Description: field is not valid in "$I"." && echo "Description: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Version:/ { print $2 }' "$I"` ] && echo "Version: field is not valid in "$I"." && echo "Version: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Copying-policy:/ { print $2 }' "$I"` ] && echo "Copying-policy: field is not valid in "$I"." && echo "Copying-policy: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Extension_by:/ { print $2 }' "$I"` ] && echo "Extension_by: field is not valid in "$I"." && echo "Extension_by: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Size:/ { print $2 }' "$I"` ] && echo "Size: field is not valid in "$I"." && echo "Size: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
[ -z `awk '/Current:/ { print $2 }' "$I"` ] && echo "Current: field is not valid in "$I"." && echo "Current: field is not valid in "$I"." >> /tmp/submitqc/wronginfofile
file "$I" | grep "CRLF" && dos2unix -u "$I" && echo "${YELLOW}Fixing CRLF line terminators in ${BLUE}"$I"${YELLOW}. Please don't use Windows apps to edit Linux files.${NORMAL}" \
&& echo ""$I" has CRLF line terminators in it. Please don't use Windows apps to edit Linux files." >> /tmp/submitqc/wronginfofile
sleep 1
done
}
if [ ! -z "$1" ]; then
[ -f /tmp/submitqc/.info.lst ] && rm /tmp/submitqc/.info.lst
wget -O /tmp/submitqc/.info.lst.gz "$MIRROR""$VERSION"/tcz/info.lst.gz > /dev/null 2>&1
gunzip /tmp/submitqc/.info.lst.gz
sed -i 's:.tcz::g' /tmp/submitqc/.info.lst
if [ "$1" == "checkdep" ]; then
[ -d /tmp/submitqc/missingdeps ] || mkdir -p /tmp/submitqc/missingdeps
checkdep
exit 0
fi
if [ "$1" == "checkinfo" ]; then
[ -d /tmp/submitqc ] || mkdir /tmp/submitqc
checkinfo
exit 0
fi
if [ "$1" == "checkmaintainer" ]; then
[ -d /tmp/submitqc ] || mkdir /tmp/submitqc
> /tmp/submitqc/tcemaintainer
for F in `ls *.tcz`; do
checkmaintainer
done
exit 0
fi
fi
[ -d /tmp/submitqc ] && rm -r /tmp/submitqc
mkdir -p /tmp/submitqc/missingdeps
[ -f /tmp/submitqc/.info.lst ] && rm /tmp/submitqc/.info.lst
wget -O /tmp/submitqc/.info.lst.gz "$MIRROR""$VERSION"/tcz/info.lst.gz > /dev/null 2>&1
gunzip /tmp/submitqc/.info.lst.gz
sed -i 's:.tcz::g' /tmp/submitqc/.info.lst
> /tmp/submitqc/corrupttcz
> /tmp/submitqc/wrongblocksize
> /tmp/submitqc/wrongmd5
> /tmp/submitqc/tcemaintainer
> /tmp/submitqc/similarextensions
> /tmp/submitqc/appendeddata
> /tmp/submitqc/wrongstartscriptperms
> /tmp/submitqc/wrongstartscriptname
> /tmp/submitqc/wronginfofile
> /tmp/submitqc/baddirperms
for F in `ls | xargs file | grep ".tcz" | grep "Squashfs" | cut -f 1 -d :`; do
TMP2=""$F".tmpdir12345678"
if df | grep "$TMP2" > /dev/null 2>&1; then
umount "$TMP2"
fi
[ -d "$TMP2" ] && rm -rf "$TMP2"
mkdir "$TMP2"
BASENAME=`basename "$F" .tcz`
if `sudo busybox mount -o loop "$F" "$TMP2" > /dev/null 2>&1`; then
checkappend
checkstartup
checkbasedirperm
checkdirperms
if [ -f "$F".list ]; then
checkdiff
fi
if [ -f "$F".info ]; then
checkmaintainer
fi
checkblock
echo_green
busybox umount -d "$TMP2"
sleep 2
rm -rf "$TMP2"
echo " "
echo ${GREEN}##############################################${NORMAL}
echo ${GREEN}##############################################${NORMAL}
else
echo "${BLUE}"$F"${RED} is a corrupt tcz file, please remake.${NORMAL}"
echo "$F" >> /tmp/submitqc/corrupttcz
rm -rf "$TMP2"
fi
done
echo " "
checkdep
echo " "
checkinfo
sudo chown tc:staff *.tcz*
sudo chmod 664 *.tcz*
if [ ! -z `cat /tmp/submitqc/corrupttcz` ]; then
echo "${RED}YOU HAVE CORRUPT TCZ FILES. PLEASE REVIEW \
${GREEN}/tmp/submitqc/corrupttcz${NORMAL}"
fi
-
It could also warn if there are any funny permissions, or files installed outside of the usual areas (just warn, since either may be intended)
-
I've been meaning to do something like this myself, but haven't had time yet. I'm glad you got it started. A few thoughts:
- Separate each check into a function, then call all the functions at the end. This will make it modular and easier to read as well as allow for options which perform a subset of checks, like only checking the md5sum of all the files.
- Your script requires the "file" extension to be installed, so it should probably check that $(which file) returns a value.
- Check for clobbering directories in the TCE extensions.
- Audit the .dep file for correctness. When I create an extension, I run a script that auto-generates the .dep by checking each executable in the extension for library dependencies. It then finds the name of the extension by looking for the shared library file in /tmp/tcloop (I use .tcz). It's a bit involved, but given a tce directory it could check the .dep against extensions in that directory.
- Check the size field of .info by using du -h
- Check for PPI Compatible
-
File permissions, correct md5.txt, validity of tce/tcz files is the most important. But I would like to automate whatever can be done if it can be done safely and reliably. I included some more checks, and will make it more elegent and modular in time, like checking md5 only, creating list files or whatever. Clobbering directories will be addressed. Personally, I think dep files are best made by hand, but if there is a quick and simple way to check for misspelling of entries that would be considered. Calculating sizes will be considered.
It would be good to make the boring, tedious part of extension preparing as easy as possible. But I am also careful not to go down a slippery slope of complexity as remembered with tce-uninstall.
-
Maybe it would be wise to put the script in the "compiletc" extension or a stand-alone one?
-
Perhaps, but it is easier to update being posted here.
I attached the script in an attachment to the original post in this thread for easy download. Simply adding executable permissions is required.
-
The script has been updated for tcz only. Please use it before submitting.
-
the stat applet is no longer available in the base....
http://forum.tinycorelinux.net/index.php?topic=3365.0
-
A check for coreutils has been added.
-
Checking for the case of tc:staff ownership rather than the case against root ownership of tce.installed and such has been added.
-
Updated to check for user root, group staff, and 775 perms of /usr/local/tce.* directories.
-
any consideration given to adding this script to the compiletc extension? Retrieving the script for the forums is quite difficult in TC/MC booted base norestore and only loading compile deps.
-
This script has seen about 4 or 5 updates today, so it would be a burden to include it in an extension at this point. Perhaps when it stabilizes it can find a home.
Oh, and added one more thing to check for - CRLF line terminators in the info file that cause it to appear funny in the appbrowser.
-
Since this is used also for checking tce dir; needed extensions to run script?
-
For running against a tce directory, "base norestore" would be the way to do it and then install coreutils into /tmp/tce. I can drop the perl dependency as dos2unix is in base which I will use instead. Stat was moved into coreutils so coreutils has to remain a dep.
-
For running against a tce directory, "base norestore" would be the way to do it and then install coreutils into /tmp/tce. I can drop the perl dependency as dos2unix is in base which I will use instead. Stat was moved into coreutils so coreutils has to remain a dep.
might add that it creates .list files in the tce dir (or wherever it's placed).
-
"fix" script
Here's a quick hack that fixed extension permissions in my tce dir. eg, "fix gnumeric"
Finds extension in tce_dir and sets tce.menu, tce.installed. tce.icons to root:staff/775.
Rebuilds and copies to tce_dir. Deletes old md5 file.
Rerunning audit creates new md5
#!/bin/ash
TCEDIR="$(cat /opt/.tce_dir)"
[ -d /tmp/extract ] || sudo mkdir /tmp/extract
[ -d /tmp/work ] || sudo mkdir /tmp/work
cd ${TCEDIR}
APP="$(busybox find ${1}* | busybox grep -v .dep | busybox grep -v .txt | busybox grep -v .list )"
if [ -z ${APP} ]; then
echo "no match in tce dir"
exit 1
fi
sudo mount ${APP} /tmp/extract -o loop,ro
sudo cp -Rp /tmp/extract/* /tmp/work
cd /tmp/work/usr/local
if [ -d tce.installed ]; then
sudo chown -R root:staff tce.installed
sudo chmod 775 tce.installed
sudo chmod 644 tce.installed/*
fi
if [ -d tce.menu ]; then
sudo chown -R root:staff tce.menu
sudo chmod 775 tce.menu
sudo chmod 644 tce.menu/*
fi
if [ -d tce.icons ]; then
sudo chown -R root:staff tce.icons
sudo chmod 775 tce.icons
sudo chmod 644 tce.icons/*
fi
cd /tmp/work
sudo mksquashfs ./ /tmp/${APP}
sudo cp -p /tmp/${APP} ${TCEDIR}
sudo rm ${TCEDIR}/${APP}.md5.txt
###### cleanup
sudo umount /tmp/extract
sudo rmdir /tmp/extract
sudo rm -R /tmp/work
sudo rm /tmp/${APP}
exit 0
-
I will add a switch to prevent creating of list files as they would interfere with tce2tcz when run on the tce directory. I didn't initially intend for this to be an audit tool for the tce directory but rather for extensions ready to be submitted. But it does serve as a quick check for existing extensions in the tce dir especially in regards to the tce.installed thing.
I did use a "fix it" script on my mirror, using a check like in the audit script.
-
I will add a switch to prevent creating of list files as they would interfere with tce2tcz when run on the tce directory. I didn't initially intend for this to be an audit tool for the tce directory but rather for extensions ready to be submitted. But it does serve as a quick check for existing extensions in the tce dir especially in regards to the tce.installed thing.
I did use a "fix it" script on my mirror, using a check like in the audit script.
I already altered update and load scripts in case people want them (only 4K for a record of extension files without an internet connection).
EDIT: added 644 perm to tce/*
-
Is this script still working with updated squashfs? I tried it with a newly compiled dialog.tcz, and it complained that the file was corrupted, although it worked fine.
edit: seems to work after I rebooted and tried again.
-
Updated to check for 4096 block size and echo a red error message on failing md5sum.
-
Great tool, thanks!
-
Updated to check for unwanted appended data and recreate list files even if they are present.
-
How about auto editing the info file? this can be done for the size and the extension name...
Change Title field:
sed -i 's!^Title:.*$!Title:\t\t$F!' $F.info
Change Size field:
sed -i 's!^Size:.*$!Size:\t\t`du -h $F | cut -f 1`!' $F.info
-
Sure, I will add it in.
-
The squashfs block 4096 check needs to have unsquashfs, this check needs to be added
if [ ! -f /usr/local/tce.installed/squashfs-tools-4.0 ]; then
echo "Install squashfs-tools-4.0.tcz before proceeding."
exit 1
fi
The chown line for the tce* directories needs to have a sudo.
The -noappend flag can be added to the mksquashfs command and the rm line can be removed.
-
Thanks, I will work that in too. Will get to both soon.
-
Done
-
Added a check for missing entries in dep files.
-
line 82: syntax error: unexpected "then" (expecting "done")
-
I copied and pasted it and it works fine here.
-
recopied and repasted but:
/home/tc/.local/bin/tce-check: line 82: syntax error: unexpected "then" (expecting "done"can u publish this script also as an attachment of the post?
Thanx
-
wheneve u modify this script and also the others, plz add another post saying that u modified it
-
indentation is not perfect: look at line 80
-
Attachment made. I have made some portability changes. At one time I was bugfixing quite a bit and didn't want to announce every little change. But since the script is pretty much completed, I will announce any new features or feature removal.
-
tc-ita.tcz is a corrupt tcz file, please remake.
The following errors are found in tc-ita.tcz.info. No news is good news:
changing the lineTMP2=`mktemp -d tmp.XXXXX`like this:
TMP2=`mktemp -d`it works
-
There should probably be six X's instead of five.
-
`mktemp -d` makes 10 characters, changed to mktemp -d.
-
`mktemp -d` was causing problems with me, so using manual directory creation, along with other bug fixes.
-
u could post your script also in the wiki
-
Scripts should not be posted in the wiki - a link back to the thread should be the way to go.
-
If Bash isn't installed then there is an error so the Bash check is useless. Try this:
#!/bin/sh
if [ ! `which bash` ]; then
echo "Install bash."
exit 1
fi
bash
RED=`echo -e '\e[1;31m'`
YELLOW=`echo -e '\e[1;33m'`
WHITE=`echo -e '\e[00m'`
GREEN=`echo -e '\e[0;32m'`
if [ ! `which file` ]; then
echo "Install the file extension."
exit 1
fi
if [ ! `which mksquashfs` ]; then
echo "Install the squashfs-tools extension."
exit 1
fi
And the info file check can be changed to:for I in `ls *.info`; do
INFOERR=""
echo "Checking for errors in "$I"..."
[ -z `awk '/Title:/ { print $2 }' "$I"` ] && INFOERR=`echo -e "\tTitle: field is not valid in "$I"."\n`
[ -z `awk '/Description:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tDescription: field is not valid in "$I"."\n`
[ -z `awk '/Version:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tVersion: field is not valid in "$I"."\n`
[ -z `awk '/Author:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tAuthor: field is not valid in "$I"."\n`
[ -z `awk '/Original-site:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tOriginal-site: field is not valid in "$I"."\n`
[ -z `awk '/Copying-policy:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tCopying-policy: field is not valid in "$I"."\n`
[ -z `awk '/Size:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tSize: field is not valid in "$I"."\n`
[ -z `awk '/Extension_by:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tExtension_by: field is not valid in "$I"."\n`
[ -z `awk '/Comments:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tComments: field is not valid in "$I"."\n`
[ -z `awk '/Change-log:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tChange-log: field is not valid in "$I"."\n`
[ -z `awk '/Current:/ { print $2 }' "$I"` ] && INFOERR=$INFOERR`echo -e "\tCurrent: field is not valid in "$I"."\n`
file "$I" | grep "CRLF" && dos2unix -u "$I" && INFOERR=$INFOERR`echo -e "\t${YELLOW}Fixed CRLF line terminators in "$I". Please don't use Windows apps to edit Linux files.${WHITE}"\n`
if [ -n $INFOERR ]; then
echo "${RED}$INFOERR"
else
echo -e "\bNo Errors found."
fi
sleep 1
done
-
Thanks, I will add that in soon.
-
Dropped bash requirement, added robc's suggestion and also added the listing of .list file differences from the extension being audited and the one in the repo to draw attention to list file content differences when updating existing extensions. Particular attention in list file diffs is those of /usr/local/lib/lib.so.X number differences that often result in breakage of dependent apps when the extension being updated is a library one. This just raises a yellow flag on file content differences for the submitter to determine the relevance.
A version for 2.x and 3.x are attached to original post.
-
robc-
I undid your info changes as I saw errors and don't have time to troubleshoot. If you would confirm it working or bugfix it I would be glad to include it in.
-
In addition to some fixes, I added a function that echoes both your name from your info file and the extension maintainers name if the extension you are checking already exists in the repo. If the names don't match, you need to contact the extension maker if you desire to update the existing extension. Please use it and make sure that you are not sending an intended new submission or an update for an extension already existing by someone else. There are 2509 tcz extensions in the repo, and it is easy to overlook one, and this tool will help in that regard.
I use this script on every extension that comes through, but always more efficient to catch one's own errors before they are sent in.
-
Added some small features and improvements, also using /opt/tcemirror for wget command to allow more universal access. I don't plan on announcing every small change, but I think for now it is where I want it.
Some files that are created to tell you of issues:
/tmp/submitqc/corrupttcz - lists corrupt tcz extensions
/tmp/submitqc/wrongblocksize - lists extensions that are not 4096 block size.
/tmp/submitqc/wrongmd5 - lists extensions that fail md5sum
/tmp/submitqc/tcemaintainer - lists extensions being audited that have existing maintainers and maintainer names.
-
Added an additional check for extensions in the repo with similar names, so one can make sure that their extension does not already exist in a similarly named format. For example, running the script on emelfm2-locale.tcz will list these extensions as existing in similar named format:
emelfm.tcz emelfm2.tcz emelfm2-locale.tcz
Then the submitter can look to see if any of those existing extensions are the same as he is about to upload, or if his extension will in any way conflict with those that are already there.
This information is stored in this file on each run of the audit script:
/tmp/submitqc/similarextensions
Running the script on emelfm.tcz, emelfm2.tcz, and emelfm2-locale.tcz produced this output in the similarextensions file that can be studied after the script is run, handy in the case of auditing a long list of extensions at one time:
emelfm.tcz: similar extensions
emelfm2-locale.tcz
emelfm2.tcz
emelfm.tcz
emelfm2-locale.tcz: similar extensions
emelfm2-locale.tcz
emelfm2.tcz
emelfm.tcz
emelfm2.tcz: similar extensions
emelfm2-locale.tcz
emelfm2.tcz
emelfm.tcz
Also, here is the output of the tcemaintainer file in that same directory:
Extension: emelfm.tcz
Maintainer is juanito
Submitter is juanito
Extension: emelfm2-locale.tcz
Maintainer is Jason W
Submitter is Jason W
Extension: emelfm2.tcz
Maintainer is Jason W
Submitter is Jason W
-
I just came across this page today, after having submitted several extensions over the past three months. I can't help but wonder who else has missed this. I have therefore added a note to the wiki (http://wiki.tinycorelinux.com/Creating+Extensions#Testing (http://wiki.tinycorelinux.com/Creating+Extensions#Testing)), since that is where people will be looking for information on creating an extension, not the forums.
-
Thanks!
I also made this a sticky so it doesn't get buried.
-
I added the check for the KERNEL variable to see if either a 32 bit or 64 bit version exists of entries in dep files, all entries in dep files that do not exist in the repo are stored in /tmp/submitqc/missingdeps.
-
Added a check for the readability as user TC of the base squashfs directory, to prevent non-readable extensions from getting through.
-
Added checking for suspicious directory permissions, echoed in the terminal and
stored in /tmp/submitqc/badperm.
-
Added check for wrong startup script name, results echoed in
> /tmp/submitqc/wrongstartscriptname
The extension is automatically rebuilt with proper script name if there is only one file in tce.installed yet it is
not the same name as the extension. If there are two or more files in tce.installed, then no
automatic rebuilding and renaming takes place since this condition can't be reliably corrected.
-
tc@box1:/mnt/hda3/jls/estensioni/pygobject-2.7$ submit3
/home/tc/.local/bin/submit3: line 407: stat: not found
-
Stat is part of coreutils, but I have also seen that error message as when I boot I see /usr/local/tce.installed/coreutils but the extension itself is not loaded. It is likely a bug in an extension somewhere and I will look more into it later today, have not had the chance to yet.
-
I think this script should be put in the repo as an extension. That way the deps will be installed with it. Also it will allow for further automation of build scripts.
-
Though I keep my remaster, fetch-and-package, and other scripts in this section and out of the repo, the audit script is different as it is used as part of the submission process and recommended for extension makers to either use or at least be aware of what is needed to be correct and factor that into their own routines.
With it in the repo, it will be updated each time tce-update is run if I have made any changes to it. And deps will be automatically installed and ready to go.
I have a few minor adjustments to make to it and I will ponder placing it in the repo.
-
Now in the repo as submitqc.tcz, and the script name is submitqc.
-
Hi Jason W
The graphics-2.6.33.3-tinycore.tcz entry in nvidia-96.43.20-module-2.6.33.3-tinycore.tcz.dep needs to use the KERNEL variable instead of kernel version number.
This message could be a little clearer, since you have to replace more than just the number:
The graphics-2.6.33.3-tinycore.tcz entry in nvidia-96.43.20-module-2.6.33.3-tinycore.tcz.dep needs to have "version-tinycore" replaced with the word "KERNEL" .
The script also ran into trouble trying to fix one of my errors:
chown: changing ownership of `nvidia-96.43.20-glx.tcz.123456789/usr/local/tce.installed/nvidia-96.43.20-glx': Operation not permitted
chown: changing ownership of `nvidia-96.43.20-glx.tcz.123456789/usr/local/tce.installed': Operation not permitted
I'm guessing there should be a sudo in front of the chmod and chown commands.
Due to cutting and pasting I introduced a typo in my dependency file:
nvidia-96.43.20-module-KERNEL.tcz.dep instead of nvidia-96.43.20-module-KERNEL.tcz
The dependency check did not flag it as a missing dependency.
Overall, a very helpful utility, thank you.
-
Thanks, I have not tended to the script in quite a while, but I will give it some attention soon.
-
Thanks for the extension, I use it as a extension check whether submitted or not. Are you making one for scm's?
-
Actually, with scms the checks would be fewer and simpler. I will surely make a script for it soon.
-
I have not actually announced it, but I have let Robert Schumann, aka gutmensch, handle the main part of the tcz repo since I am occupied with the scms. And the best help I can give overall is not to meddle and let him tend to it unless my help is solicited.
But I will update the audit script, I plan to simplify it a little, since it really does not need to try to verify .desktop files for one, as it is always giving false negatives on that. I am considering adding in the 'check for library breakage' routine, but as it is now the data for that is made on my own mirror at home, and is not real time with ibiblio. Which means it can do as much harm as good if someone has a good extension that will not break the repo but the data is old and falsely states that breakage would occur. It is good for my personal use, and perhaps to run against existing stuff knowing that the data used may not be current. But if not in real time with the repo, it doesn't need to be in the audit routine.
-
Updated, simplified dep check, removed suspicious directory perms check as there are many different dir perms for different apps expected - cannot check for them all, removed desktop file check as again unexpected entries are admissible.
-
The script doesn't recognizes me as package maintainer, maybe because there are spaces in my name?
-
It seems to work fine here, with the spaces
root@box:/home/tc# submitqc4
There are extensions without md5.txt files : mldonkey.tcz. Do you want to proceed and create md5.txt files for these and check the existing ones? y/n y
mldonkey.tcz is maintained by jls_legalize <unsenepopiu at tin dot it>. You are jls_legalize <unsenepopiu at tin dot it>. Make sure the maintainer is aware you are updating his extension.
mldonkey.tcz has some similarly named extensions in the repo. Make sure yours does not collide with these:
mldonkey.tcz
The following errors are found in mldonkey.tcz.info. No news is good news:
mldonkey.tcz is a valid tcz file. Creating MD5 for mldonkey.tcz:
#######
-
There is a link in the wiki to this thread, howto to submit extensions (http://wiki.tinycorelinux.net/wiki:creating_extensions).
It's a bit cumbersome to read thru this thread, to filter out the relevant information.
Suggestion: put the most recent info in the first posting or put the most recent info into the wiki.
When using the submitqc6 script as user "tc" it complains about missing permissions to remove files in the temporary mounted directory and finally fails to umount the extension.
When running this script as root (sudo) it seems ok. Is this by design? If so, it should be documented.
-
Hi Alatun
from the wiki
If you prefer the build script way, it is easy to build root:root by running the script as root.
and lower down...
sudo chown -R root:staff /tmp/package/usr/local/tce.installed
sudo chmod -R 775 /tmp/package/usr/local/tce.installed
This shows you that you have 2 choices in building and they are
build as root or
build as local user but fix permission, ownership using sudo
so choose your build method and then either submitqc6 will run as root
or run it as sudo.
good luck
-
Hi Alatun
from the wiki
If you prefer the build script way, it is easy to build root:root by running the script as root.
and lower down...
sudo chown -R root:staff /tmp/package/usr/local/tce.installed
sudo chmod -R 775 /tmp/package/usr/local/tce.installed
This shows you that you have 2 choices in building and they are
build as root or
build as local user but fix permission, ownership using sudo
so choose your build method and then either submitqc6 will run as root
or run it as sudo.
good luck
Hm, the stuff you are quoting deals with "Custom Startup Scripts".
I was refering to the submitqc6 script (a script to check the quality of extensions),
which is no custom startup script, so I'm confused what you like to tell me???
-
When using the submitqc6 script as user "tc" it complains about missing permissions to remove files in the temporary mounted directory and finally fails to umount the extension... When running this script as root (sudo) it seems ok. Is this by design? If so, it should be documented.
I experience the same behaviour/error and end up running submitqc6 with sudo, then all is good. If this is correct, it would be useful if the submitqc6 info file was updated, which presently reads: "submitqc6" is the command to run.
-
Seeing as there is still interest (after 6 years!), I took on a rewrite of this script starting around June/July. The first version hit the repo about a week or so ago. New version, with many bug fixes, is now in the submission queue as of this morning (2014/10/04).
submitqc is now cross-repo, tested on 5.x/6.x and x86/x86_64. Run without sudo. [fingers crossed]
Use 'submitqc --help' to see the options.
Checks include ... well, lots of things. Open the script and read the Change Log if you're interested.
Kudos to gordonselfish/gordon64 and coreplayer2 for their bug checking assistance & advice thus far.
Lots left to do/fix, but I hope this update is a good representative of JasonW's original intent.
-
Thanks for your hard work dentonlt.
Lots left to do/fix, but I hope this update is a good representative of JasonW's original intent.
Is the script still in testing or can it now be used to formally submit an extension?
-
Is the script still in testing or can it now be used to formally submit an extension?
To submit? no. To check your extensions, yes. There are still bugs in it, though, so it would be smart to check over your extension yourself, too.
At the least, the current script can help identify some common errors in submissions. Hope it's good for that.
-
Update 20151004 hit the repo today. Next update (to submit in the next week?) adds library checking to help identify missing files in dep list. Knock on wood.
Sent from my HTC_0P6B6 using Tapatalk
-
20151017 update is now in the submission queue.
Major change is that I've added dynamic library checking. This scans the extension binaries for dynamic lib requirements, then compares this against the libraries included in extension & the extension dep list. As possible, missing libs are resolved against the local repo and then suggested additions to dep list are made.
Also added --core, --arch, --license, and --changes cli options. Cross-repo maintainers may find the former useful.
I've run this script against much of my local repo (500 extensions in each of x86 and x86_64) and it seems stable. Bug reports and patches welcome, of course.
-
Is the simple way to disable "Copyright notice" warning in submitqc script - it can not found license file in package.tcz, because it as doc files are located in package-doc.tcz.
-
Note:
Please check the license condition. Although we can strip a number of documents from the main tcz and have a separate doc.tcz, this is not always possible. [...]
http://wiki.tinycorelinux.net/wiki:creating_extensions#license
-
Is the simple way to disable "Copyright notice" warning in submitqc script - it can not found license file in package.tcz, because it as doc files are located in package-doc.tcz.
_madRat, Misalf: thanks for mentioning this, and the warning re license checking. I don't currently have flags/args for disabling specific tests, but that's probably a good idea for the future. I'll add it to the queue.
Overall, just use good judgement when you're submitting. Some errors/warnings will be acceptable depending upon the package you're submitting. The script currently warns you to contact the maintainer even if you're the current maintainer, for example ...
Other news: thanks to andyj for his notes on armv7 - this lets submitqc run on arm now. I have no arm hardware for testing, however - the script may eat your data! Bug reports and test results much needed and welcome.
submitqc submitted today for 8 repos: 5.x, 6.x, 7.x on x86, x86_64, and armv7.
-
dentonlt, first of, thanks for contributing and maintaining submitqc.
I find text coloring in the output of scripts is very useful, however, I noticed there is none in the in the current submitqc, contrary to former versions. While there are lots of warnings which might be considered redundant, depending an what extensions are checked (not of your concern), it can be quite cumbersome to check the hole output by reading every line, compared to 'viewing' (f.e. red=error, yellow=warning, green=success etc.).
Any plans for fancy colors for those read-impaired people like me?
-
dentonlt, first of, thanks for contributing and maintaining submitqc.
I find text coloring in the output of scripts is very useful, however, I noticed there is none in the in the current submitqc, contrary to former versions. While there are lots of warnings which might be considered redundant, depending an what extensions are checked (not of your concern), it can be quite cumbersome to check the hole output by reading every line, compared to 'viewing' (f.e. red=error, yellow=warning, green=success etc.).
Any plans for fancy colors for those read-impaired people like me?
Hey, misalf:
Damn, so sorry that I missed your post - thought I had the tapatalk notifications on!
In short, use --color or -c. Hoping you already discovered that. Cheers ~
-
Ah, K then. Will make an alias or something.
Thanks!
-
Happy New Year people ! :D
submitqc is a great script, a very useful tool that saves us lots of trouble.
I've been making extensions by following the instructions on wiki.
http://wiki.tinycorelinux.net/wiki:creating_extensions (http://wiki.tinycorelinux.net/wiki:creating_extensions)
sudo chown -R root:staff /tmp/package/usr/local/tce.installed
sudo chmod -R 775 /tmp/package/usr/local/tce.installed
However, every time I make an extension, I get the "wrongstartscriptperms" message.
I never take much heed since the script "fixes" the permission automatically.
Today, however, I got curious and checked the script.
tc@box:~$ head -n 619 /usr/local/bin/submitqc | tail -n 71
# if ext has a startup file, check startup name, permissions, location, etc.
checkstartup() {
echo -n "${BLUE}${SCRIPT}: ${F} startup files ok? ${NORMAL}"
if [ ! -d usr/local/tce.installed/ ]; then
echo "${GREEN}None present. Ok, I think.${NORMAL}"
return # no startup script
fi
# count files in startup script dir; get name of startup script
SSDIR="usr/local/tce.installed/"
SS="$(ls -A $SSDIR)"
COUNT="$(echo "$SS" | wc -l)"
# is there only 1 startup file, named same as extension?
case $COUNT in
0) echo -e "${RED}Has startup folder but no script. Repair manually.${NORMAL}"
echo -e "${F}: startup folder without script. Repair manually." \
>> /tmp/submitqc/startupscript
;;
1) if [ "${SS}" != "${BASENAME}" ]; then
echo -ne "\n${YELLOW}script name ($SS) is incorrect. Renaming ${BASENAME}. ${NORMAL}"
echo "$F startup named $SS renamed $BASENAME" >> /tmp/submitqc/wrongstartscriptname
SS="$(find $SSDIR -not -type d | tail -n 1)"
mv "${SS}" "${SSDIR}${BASENAME}"
echo "${YELLOW}Done.${NORMAL}"
CHANGES=1
fi
;;
*) echo -e "${RED}Multiple startup files, none named $F. Repair manually.${NORMAL}"
echo -e "${F}: multiple startup files, none named $F. Repair manually." \
>> /tmp/submitqc/startupscript
;;
esac
DIRERROR=
if [ ! "$(stat -c%a ${SSDIR})" = 775 ]; then
sudo chmod 775 "usr/local/tce.installed"
echo -ne "\n\t${YELLOW}Startup directory permissions were corrected.${NORMAL}"
DIRERROR=1
fi
if [ "$(stat -c'%U %G' ${SSDIR})" != "root staff" ]; then
sudo chown root:staff "${SSDIR}"
echo -ne "\n\t${YELLOW}Startup directory ownership was corrected.${NORMAL}"
DIRERROR=1
fi
if [ ! "$(stat -c%a ${SSDIR}/$BASENAME)" = 755 ]; then
sudo chmod 755 "${SSDIR}/$BASENAME"
echo -ne "\n\t${YELLOW}Startup script permissions were corrected.${NORMAL}"
DIRERROR=1
fi
if [ ! "$(stat -c'%U %G' ${SSDIR}/$BASENAME)" = "tc staff" ]; then
sudo chown tc:staff "${SSDIR}/$BASENAME"
echo -ne "\n\t${YELLOW}Startup script ownership was corrected.${NORMAL}"
DIRERROR=1
fi
if [ $DIRERROR ]; then
echo "$F" >> /tmp/submitqc/wrongstartscriptperms
CHANGES=1
echo
else
echo "${GREEN}Ok.${NORMAL}"
fi
} # end checkstartup()
"tce.installed" --> "drwxrwxr-x root:staff"
"start-up-script" --> "-rwxr-xr-x tc:staff"
:o
I thought their owner should both be "root:staff"
with permission "775"
while every thing else as "root:root"
I'm confused. :-\
P.S.
I'm on x86_64 and the submitqc.tcz.md5.txt look like this:
7b157080f71142c84a5faf2cc66b3308 submitqc.tcz
Most extensions on repo have this problem
-
Only the /usr/local/tce.installed folder needs to be root:staff - the files in the folder can be tc:staff
-
The tce.installed directory is used for two things: To indicate which extensions are loaded and to run an extension's startup script if present.
staff needs write permission for the tce.installed directory so that there can be created (empty) files for extensions that don't come with a startup script.
The startup scripts are ran as root though. So I guess the startup scripts could be root:root but it's not that important?
Is this actually a problem or just a discontinuity?
-
Only the /usr/local/tce.installed folder needs to be root:staff - the files in the folder can be tc:staff
The tce.installed directory is used for two things: To indicate which extensions are loaded and to run an extension's startup script if present.
staff needs write permission for the tce.installed directory so that there can be created (empty) files for extensions that don't come with a startup script.
The startup scripts are ran as root though. So I guess the startup scripts could be root:root but it's not that important?
Is this actually a problem or just a discontinuity?
IMHO, since the wiki says that you should set it as "root:staff"
We should keep it consistent :)
BTW, some users don't use "tc" as their username ;)
(submitqc might not work if a user set his username differently, not a big deal thought)
I think it would be better to modify the script to fit the wiki's instructions
-
The user "tc" should be always present though. Similar to "Default" on Windows.
Anyway, I agree that "tc" probably shouldn't be used for permissions if not necessary.
-
Hi !
I have a tortoise like internet speed and I am tired of submitqc fetching the info.lst.gz.
I highly suggest to update the script so that it uses recently fetched info.lst.gz and also check the remote copy's zsync file so that it only downloads the blocks that are different. I also suggest md5 checking for both local and remote info.lst.gz.
Also when checking the extension, it shows :
Copyright notice not found at usr/local/share/doc/ClipIt
Confirm that this is not needed.
Checking repo for copyright statement in ClipIt.tcz ...
ClipIt.tcz not found on server.
Hence I request to also add --no-copyright or --copyright= options so that the script identifies the copyright file.
-
Hi !
I think that it is better for the script to update the size field in .info file after stripping of binaries, if the binaries are unstripped.
-
Hi Sashank999
if I run submitqc --libsand have no size in the info files....submitqc will add them.
I prefer to have my build script "strip" where ever possible rather than use the switch supplied by submitqc.
eg as per wiki we have....multiple mentions of strip
http://wiki.tinycorelinux.net/doku.php?id=wiki:creating_extensions.html
but I prefer ...I build to /tmp
P=TCE-name
SRC=some source name
cd /tmp
# wget command to get source
# unpack
mkdir $P
cd $SRC
# compile commands
make install-strip DESTDIR=/tmp/$P
cd /tmp
I know you have already read Rich script
http://forum.tinycorelinux.net/index.php?action=dlattach;topic=23844.0;attach=5365
-
Hi aus9
I think it is better to add it to submitqc.
-
Hello all !
I have created a patch for submitqc that totally integrates infocreator script inside submitqc. No need of infocreator script. Just apply the patch and run submitqc. I have only changed the "checkinfo()" function inside the original submitqc. I encourage everyone to try it :D
-
Hey, all ~ happy isolation-days greetings from Australia =) The pandemic seems to have created some at-home time to revisit old projects!
Sashank999 - thanks for the infocreator work. I'll have to read all the code to see how you've done it - thanks for posting the patch. Obviously, it's a space that deserved some attention! Unless somebody thinks that we shouldn't include some info file options/cleanup inside of submitqc, I think it makes sense to merge/include this.
Thanks also for suggesting not downloading the info.lst.gz if not required. I don't see any arguments against that one.
@aus9, thanks for your build script framework, too. I'll have to go back to submitqc and check how/when I'm running strip.
Re the stripping,
Seeing as it's been ages since I submitted a submitqc update ... it may take me a bit to remind myself how it all fits together! I'll grab the current version from the repo, read the current docs, and make a formal update/submission in the days ahead.
Hope you're all well out there. Stay safe =)
~ Denton
-
Hi welcome back.
My only issue with submitqc --libs is I am unable to work out the correct dir own and perms
and then owner and perms for /usr/local/tce.installed/filename
when you look at your system be it TC64 or TC32 there appears to be some randomness.
-
Oops, look like I submitted the wrong patch before. I actually stopped working on it as the last one gone wrong.
I think I should start working on it. Since I know how (some) python modules look, I have also added python things into my info script. I need to work on integrating them into submitqc.
-
@dentonlt
Due to a recent stuff up by me, where I failed to notice I was a missing
:
in my info file. Here is a test to prove current submitqc does not check all of the info file structure,
for you consideration.
tce-load -i squashfs-tools submitqc
cd /tmp
rm -rf submitqc zzz
P=zzz
mkdir -p $P/usr/local/$P
mksquashfs $P $P.tcz
md5sum $P.tcz > $P.tcz.md5.txt
cd $P
find usr -not -type d > /tmp/$P.tcz.list
sed 's|usr|/usr|g' -i /tmp/$Z.tcz.list
cd /tmp
echo 'Title: zzz.tcz
Description: z
Version: z
Author: z
Original-site: z
Copying-policy: z
Size: 4.0K
Extension_by: z
Tags z
Comments: z
Change-log: 2020/08/06
Current: 2020/08/06 ' > $P.tcz.info
submitqc --libs
# output SNIP
# submitqc: There are warnings or errors for your extensions. See: /tmp/submitqc/copyright
I was missing : after tags
I am aware that some people prefer the EOF method of generating an info file where top 2 lines read
cat >> $P.tcz.info <<'EOF'
Title: something.tcz
SNIP bottom 2 lines read
Current: 2020/08/06 something
EOF
thanks for reading.
-
I had a look at coreplayers's firefox get latest script. My coding skills need improving so I may be wrong
It appears there is agreement with the create TCE wiki that dir is root:staff for /usr/local/tce.installed
then the script under that dir is tc:staff
which means this part of the wiki may be wrong
sudo chown -R root:staff /tmp/package/usr/local/tce.installed
I can not remember if I originally wrote that....but feel free to blame me, as I would have based it on something someone said to me. If its wrong, lets fix. If its right then maybe submitqc is wrong?
-
What is important is that /usr/local/tce.installed is root:staff - the file in it can be $USER:staff or root:staff
-
Damn, I really need to schedule time that's not on school holidays ...
Currently trying to get a gtkpod extension done for TC11 so that I can get my stuff off an old iPod. I'll keep reading through all this to integrate changes.
For all the Christmas-y types out there, Merry Christmas =D
-
What is important is that /usr/local/tce.installed is root:staff - the file in it can be $USER:staff or root:staff
Umm ... since $USER is pretty much anybody, I guess I have two options for the startup script:
- check for & allow only root:staff & tc:staff
- just check the group is staff, accept any user.
I'm thinking the first one is more conservative, so I'm going to do that for now. Happy to take advice/direction otherwise.
I'm also thinking there isn't ever a need for a maintainer to provide an extension startup script that must be run by any user other than tc or root. I am admittedly really rusty, though. Maybe I'm missing a whole slew of cases where a non-root non-tc username is advised?
-
@dentonlt
Due to a recent stuff up by me, where I failed to notice I was a missing
:
in my info file. Here is a test to prove current submitqc does not check all of the info file structure,
for you consideration.
Ahhh, you're very detailed, @aus9! It does mean it probably misses other things, too.
I'll browse both Sashank999's code and the current submitqc. I think I also have some sample/test info files that I used in 2016 ... somewhere ...
-
HEllo forum,
THat feature which consolidates any extensions dep's (in tcl control panel) seems good.
(check deps)
thx
v :) :) :)