Tiny Core Linux
Tiny Core Base => TCB Talk => Topic started by: jazzbiker on March 22, 2023, 12:55:04 PM
-
Hi Core,
The changes to .tcz.dep files are not so rare. The dark side of changing of the .tcz.dep file without changing of the corresponding .tcz is that tce-update will not take care of such change. I want to propose the (simple?) workaround for this issue/feature.
The price to be paid is adding 2 lines to tce-update in upgrade() function:
extend
REPO=`grep -w " $TARGET" md5.db`
to
REPO=`grep -w " $TARGET" md5.db`
REPO_DEP=`grep -w " $TARGET".dep md5.db`
and extend
echo "$REPO" > "$TARGET".md5.txt
to
echo "$REPO" > "$TARGET".md5.txt
echo "$REPO_DEP" >> "$TARGET".md5.txt
Such change seems to work with all the extensions done in the usual way - REPO_DEP variable will be empty. But in case .tcz.md5.txt file will be created by the extension's maintainer with
md5sum some.tcz some.tcz.dep > some.tcz.md5.txt
then it seem to contain information about md5sums of extension itself and its tcz.dep file too and will trigger extension upgrade in case .tcz.dep will be changed even if the extension itself stay unchanged.
If maintainer use traditional
md5sum some.tcz > some.tcz.md5.txt
everything works as usual.
This idea is not tested, but if it will be encountered worth spending some time - it would be nice.
What's Your opinion?
Have a nice Core!
-
Hi jazzbiker
In the Apps GUI:
Click Apps->Maintenance->Dependencies And Deletions
The Apps button will now say Dependencies.
Now try:
Dependencies->Update .dep files
and:
Dependencies->Fetch Missing Dependencies
-
Thanks, Rich!
I didn't know this tip because I rarely use GUI Apps, almost never, if to be correct. Of course it is nice application and as we can see has some functionality absent in the CLI tce-* suite.
Let me summarize the steps to obtain guaranteed update:
1. GUI Apps -> Maintenance -> Dependencies and Deletions
2. GUI Dependencies -> Update .dep files
3. GUI Dependencies -> Fetch Missing Dependencies
4. GUI Dependencies -> Exit Dependency Check
5a. GUI Apps -> Maintenance -> Check for Updates
or
5b. tce-update
Right?
Want to notice that steps 2 and 3 are not very fast, maybe they may be described as the slow ones.
Is all the .dep database scanned during these steps execution?
-
Hi, jazzbiker. I also don't use the Apps GUI. Many moons ago Rich gave me the CLI commands necessary, in correct order, to guarantee a full update. I put the commands into a script which I called update-everything
The script looks like this:
#!/bin/sh
echo "Building package database..."
tce-audit builddb
echo "Downloading fresh .dep files..."
tce-audit updatedeps
echo "Looking for missing dependencies..."
tce-audit fetchmissing
echo "Updating extensions..."
tce-update
I wish you a happy Core!
-
Hi GNUser,
TinyCore makes me happier, it's true :-) Thank You for this reciept. Hope Rich will not put some awfull curses on You for propagating such esoteric knowledge among the ignoramuses ;-)
-
Hi jazzbiker
... Hope Rich will not put some awfull curses on You for propagating such esoteric knowledge among the ignoramuses ;-)
Are you kidding? I'm glad GNUser remembered it when I didn't. ;D
-
thankyou all for this post.
at the risk of pretending I know something about Apps,
remove any md5.txt for private extensions like
mylocale or repack of large TCEs like some firmware
or private updates that you plan to submit etc ( or not in case the license is too restricted etc )
-
Hi aus9. Yes, removing the md5.txt file is easiest way to keep custom extensions untouched during extension updates. It's also an easy way of marking which extensions are custom/private.
I even have this trivial script called no-md5, which shows me which of the extensions in tcedir/optional directories are custom :)
#!/bin/sh
cd /etc/sysconfig/tcedir/optional
for file in *.tcz; do [ ! -e ${file}.md5.txt ] && echo $file; done
-
Continuing the topic. Initial post was just an idea - make tce-update handle .tcz.md5.txt files containing both .tcz and .tcz.dep sums. Proposed in the initial post workaround was wrong. Here is the next attempt. Attached tce-update supposed to provide the handling of extended .tcz.md5.txt files. It is tested updating the working TC13 x86 system - successfully.
Here is the diff:
--- tce-update.orig 2022-05-08 15:39:05.000000000 +0300
+++ tce-update 2023-03-23 17:24:42.618946654 +0200
@@ -16,6 +16,7 @@
fetch(){
[ -f "$1" ] && rm -f "$1"
wget -cq "$MIRROR"/"$1"
+ wget -cq "$MIRROR"/"$1".dep
}
fetchzsync(){
[ -f "$1" ] && rm -f "$1"
@@ -23,13 +24,14 @@
rm -f "$1".zs-old
else
wget -cq "$MIRROR"/"$1" # Either there was no zsync file on the server, or it failed
+ wget -cq "$MIRROR"/"$1".dep
fi
}
chkMd5Only(){
MYMD5=`cat "$F"`
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
[ "$MYMD5" == "$REPO" ] || echo $(basename "${F%.md5.txt}")
else
@@ -37,9 +39,18 @@
fi
}
+slice(){
+ while [ "$*" ]
+ do
+ echo $1" "$2
+ shift
+ shift
+ done
+}
+
upgrade(){
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
MYMD5=`cat "$F"`
if [ "$MYMD5" != "$REPO" ]; then
@@ -52,11 +63,10 @@
else
fetch "$TARGET" 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
- fetch "$TARGET".dep 2>/dev/null
- if [ "$?" == 0 ]; then
+ if [ -f /tmp/"$TARGET".dep ]; then
dep_vars /tmp/"$TARGET".dep
mv /tmp/"$TARGET".dep "$TARGET_DIR"/.
fi
@@ -86,7 +96,7 @@
for F in $DEPS; do
for TARGET in `cat "$F"`; do
if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
echo -n "Fetching required dependency: $TARGET "
if [ -n "$USEZSYNC" ]; then
@@ -94,7 +104,7 @@
else
fetch "$TARGET" 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
Some comments.
1. fetch() and fetchzsync() now load both .tcz and .tcz.dep
2. REPO variable may contain two records, that's why function slice() is used to separate them
3. "-w" option for grep is not used. Leading space in " $TARGET" allows to select both .tcz and .tcz.dep (if present)
-
The next version, fixing errors and using tce-load to load missing dependencies.
Interventions into fetch() and fetchzsync() were wrong, rolled back.
The main changes are made in check_dependencies() function. Original version is able to fetch only the single level of missing dependencies. That's why I used tce-load to make this job. If dependency is missing then we can load it immediately into the tce directory, no need to use upgrade subdir.
tce-load message is redirected into separate "checkdep_errors.lst" file and is appended to the "upgrade_errors.lst" file in case tce-load fails.
Here is the diff:
--- tce-update.orig 2022-05-08 15:39:05.000000000 +0300
+++ tce-update 2023-03-24 11:28:10.086770967 +0200
@@ -29,7 +29,7 @@
chkMd5Only(){
MYMD5=`cat "$F"`
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
[ "$MYMD5" == "$REPO" ] || echo $(basename "${F%.md5.txt}")
else
@@ -37,9 +37,18 @@
fi
}
+slice(){
+ while [ "$*" ]
+ do
+ echo $1" "$2
+ shift
+ shift
+ done
+}
+
upgrade(){
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
MYMD5=`cat "$F"`
if [ "$MYMD5" != "$REPO" ]; then
@@ -49,14 +58,15 @@
echo -n "Updating $TARGET "
if [ -n "$USEZSYNC" ]; then
fetchzsync "$TARGET" 2>/dev/null
+ fetchzsync "$TARGET".dep 2>/dev/null
else
fetch "$TARGET" 2>/dev/null
+ fetch "$TARGET".dep 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
- fetch "$TARGET".dep 2>/dev/null
- if [ "$?" == 0 ]; then
+ if [ -f /tmp/"$TARGET".dep ]; then
dep_vars /tmp/"$TARGET".dep
mv /tmp/"$TARGET".dep "$TARGET_DIR"/.
fi
@@ -80,37 +90,32 @@
check_dependencies(){
-# echo "${BLUE}Checking dependencies:${NORMAL}"
+ echo "${BLUE}Checking dependencies:${NORMAL}"
DEPS="$(/bin/busybox find "$UPGRADE_DIR" -regex '.*\.tcz\.dep$' | sort)"
[ -f $TCEDIR/tcz-black.lst ] && DEPS=$(echo $DEPS | grep -v -f $TCEDIR/tcz-black.lst)
+
+ DEPS_TO_LOAD=""
+
for F in $DEPS; do
for TARGET in `cat "$F"`; do
- if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
- if [ -n "$REPO" ]; then
- echo -n "Fetching required dependency: $TARGET "
- if [ -n "$USEZSYNC" ]; then
- fetchzsync "$TARGET" 2>/dev/null
- else
- fetch "$TARGET" 2>/dev/null
- fi
- echo "$REPO" > "$TARGET".md5.txt
- md5sum -cs "$TARGET".md5.txt
- if [ "$?" == 0 ]; then
- mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
- mv /tmp/"$TARGET" "$TARGET_DIR"/. 2>/dev/null
- UPDATED_APPS=1
- echo "OK"
- else
- echo "Failed"
- echo "Error occurred while fetching dependency: $TARGET" >> "$ERRLIST"
- rm /tmp/"$TARGET".md5.txt
- rm /tmp/"$TARGET"
- fi
- fi
+ if [ ! -f "$UPGRADE_DIR"/"$TARGET" ]; then
+ DEPS_TO_LOAD="$DEPS_TO_LOAD $TARGET"
fi
done
done
+
+ if [ "$DEPS_TO_LOAD" ]
+ then
+ tce-load -w $DEPS_TO_LOAD 2>&1 > "$CHECKDEP_LIST"
+ if [ "$?" == 0 ]
+ then
+ UPDATED_APPS=1
+ echo "OK"
+ else
+ echo "Failed"
+ cat "$CHECKDEP_LIST" >> "$ERRLIST"
+ fi
+ fi
}
check_bootoptions(){
@@ -204,6 +209,8 @@
cd /tmp
ERRLIST="upgrade_errors.lst"
> "$ERRLIST"
+CHECKDEP_LIST="checkdep_errors.lst"
+> "$CHECKDEP_LIST"
UPDATED_APPS=0
[ -e md5.db ] && rm -f md5.db
tce-fetch.sh md5.db.gz
The side effect of using tce-load is that it uses wget to fetch extensions, and wget injects some messages directly into the console bypassing the stderr file.
-
Once again, correcting stderr redirection in check_dependencies(), renaming of CHECKDEP_LIST (it is not errors file) and making it "local" for check_dependencies().
diff:
--- tce-update.orig 2022-05-08 15:39:05.000000000 +0300
+++ tce-update 2023-03-24 12:37:26.814252041 +0200
@@ -17,6 +17,7 @@
[ -f "$1" ] && rm -f "$1"
wget -cq "$MIRROR"/"$1"
}
+
fetchzsync(){
[ -f "$1" ] && rm -f "$1"
if zsync -i "$UPGRADE_DIR"/"$1" -q "$MIRROR"/"$1".zsync; then
@@ -29,7 +30,7 @@
chkMd5Only(){
MYMD5=`cat "$F"`
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
[ "$MYMD5" == "$REPO" ] || echo $(basename "${F%.md5.txt}")
else
@@ -37,9 +38,18 @@
fi
}
+slice(){
+ while [ "$*" ]
+ do
+ echo $1" "$2
+ shift
+ shift
+ done
+}
+
upgrade(){
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
MYMD5=`cat "$F"`
if [ "$MYMD5" != "$REPO" ]; then
@@ -49,14 +59,15 @@
echo -n "Updating $TARGET "
if [ -n "$USEZSYNC" ]; then
fetchzsync "$TARGET" 2>/dev/null
+ fetchzsync "$TARGET".dep 2>/dev/null
else
fetch "$TARGET" 2>/dev/null
+ fetch "$TARGET".dep 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
- fetch "$TARGET".dep 2>/dev/null
- if [ "$?" == 0 ]; then
+ if [ -f /tmp/"$TARGET".dep ]; then
dep_vars /tmp/"$TARGET".dep
mv /tmp/"$TARGET".dep "$TARGET_DIR"/.
fi
@@ -80,37 +91,34 @@
check_dependencies(){
-# echo "${BLUE}Checking dependencies:${NORMAL}"
+ CHECKDEP_LIST="checkdep.lst"
+ > "$CHECKDEP_LIST"
+ echo "${BLUE}Checking dependencies:${NORMAL}"
DEPS="$(/bin/busybox find "$UPGRADE_DIR" -regex '.*\.tcz\.dep$' | sort)"
[ -f $TCEDIR/tcz-black.lst ] && DEPS=$(echo $DEPS | grep -v -f $TCEDIR/tcz-black.lst)
+
+ DEPS_TO_LOAD=""
+
for F in $DEPS; do
for TARGET in `cat "$F"`; do
- if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
- if [ -n "$REPO" ]; then
- echo -n "Fetching required dependency: $TARGET "
- if [ -n "$USEZSYNC" ]; then
- fetchzsync "$TARGET" 2>/dev/null
- else
- fetch "$TARGET" 2>/dev/null
- fi
- echo "$REPO" > "$TARGET".md5.txt
- md5sum -cs "$TARGET".md5.txt
- if [ "$?" == 0 ]; then
- mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
- mv /tmp/"$TARGET" "$TARGET_DIR"/. 2>/dev/null
- UPDATED_APPS=1
- echo "OK"
- else
- echo "Failed"
- echo "Error occurred while fetching dependency: $TARGET" >> "$ERRLIST"
- rm /tmp/"$TARGET".md5.txt
- rm /tmp/"$TARGET"
- fi
- fi
+ if [ ! -f "$UPGRADE_DIR"/"$TARGET" ]; then
+ DEPS_TO_LOAD="$DEPS_TO_LOAD $TARGET"
fi
done
done
+
+ if [ "$DEPS_TO_LOAD" ]
+ then
+ tce-load -w $DEPS_TO_LOAD > "$CHECKDEP_LIST"
+ if [ "$?" == 0 ]
+ then
+ UPDATED_APPS=1
+ echo "OK"
+ else
+ echo "Failed"
+ cat "$CHECKDEP_LIST" >> "$ERRLIST"
+ fi
+ fi
}
check_bootoptions(){
-
Correction in check_dependencies(): only upgrade/*.dep files are processed.
diff:
--- tce-update.orig 2022-05-08 15:39:05.000000000 +0300
+++ tce-update 2023-03-24 13:24:18.015689122 +0200
@@ -17,6 +17,7 @@
[ -f "$1" ] && rm -f "$1"
wget -cq "$MIRROR"/"$1"
}
+
fetchzsync(){
[ -f "$1" ] && rm -f "$1"
if zsync -i "$UPGRADE_DIR"/"$1" -q "$MIRROR"/"$1".zsync; then
@@ -29,7 +30,7 @@
chkMd5Only(){
MYMD5=`cat "$F"`
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
[ "$MYMD5" == "$REPO" ] || echo $(basename "${F%.md5.txt}")
else
@@ -37,9 +38,18 @@
fi
}
+slice(){
+ while [ "$*" ]
+ do
+ echo $1" "$2
+ shift
+ shift
+ done
+}
+
upgrade(){
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
MYMD5=`cat "$F"`
if [ "$MYMD5" != "$REPO" ]; then
@@ -49,14 +59,15 @@
echo -n "Updating $TARGET "
if [ -n "$USEZSYNC" ]; then
fetchzsync "$TARGET" 2>/dev/null
+ fetchzsync "$TARGET".dep 2>/dev/null
else
fetch "$TARGET" 2>/dev/null
+ fetch "$TARGET".dep 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
- fetch "$TARGET".dep 2>/dev/null
- if [ "$?" == 0 ]; then
+ if [ -f /tmp/"$TARGET".dep ]; then
dep_vars /tmp/"$TARGET".dep
mv /tmp/"$TARGET".dep "$TARGET_DIR"/.
fi
@@ -80,37 +91,34 @@
check_dependencies(){
-# echo "${BLUE}Checking dependencies:${NORMAL}"
- DEPS="$(/bin/busybox find "$UPGRADE_DIR" -regex '.*\.tcz\.dep$' | sort)"
+ echo "${BLUE}Checking dependencies:${NORMAL}"
+ DEPS="$(/bin/busybox find "$TARGET_DIR" -regex '.*\.tcz\.dep$' | sort)"
[ -f $TCEDIR/tcz-black.lst ] && DEPS=$(echo $DEPS | grep -v -f $TCEDIR/tcz-black.lst)
+
+ DEPS_TO_LOAD=""
+
for F in $DEPS; do
for TARGET in `cat "$F"`; do
- if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
- if [ -n "$REPO" ]; then
- echo -n "Fetching required dependency: $TARGET "
- if [ -n "$USEZSYNC" ]; then
- fetchzsync "$TARGET" 2>/dev/null
- else
- fetch "$TARGET" 2>/dev/null
- fi
- echo "$REPO" > "$TARGET".md5.txt
- md5sum -cs "$TARGET".md5.txt
- if [ "$?" == 0 ]; then
- mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
- mv /tmp/"$TARGET" "$TARGET_DIR"/. 2>/dev/null
- UPDATED_APPS=1
- echo "OK"
- else
- echo "Failed"
- echo "Error occurred while fetching dependency: $TARGET" >> "$ERRLIST"
- rm /tmp/"$TARGET".md5.txt
- rm /tmp/"$TARGET"
- fi
- fi
+ if [ ! -f "$UPGRADE_DIR"/"$TARGET" ]; then
+ DEPS_TO_LOAD="$DEPS_TO_LOAD $TARGET"
fi
done
done
+
+ CHECKDEP_LIST="checkdep.lst"
+
+ if [ "$DEPS_TO_LOAD" ]
+ then
+ tce-load -w $DEPS_TO_LOAD > "$CHECKDEP_LIST"
+ if [ "$?" == 0 ]
+ then
+ UPDATED_APPS=1
+ echo "OK"
+ else
+ echo "Failed"
+ cat "$CHECKDEP_LIST" >> "$ERRLIST"
+ fi
+ fi
}
check_bootoptions(){
-
CHECKDEP_LIST is "global" again and is cleared per every tce-update run.
Some cleaning: no need to set UPDATED_APPS in check_dependencies()
diff:
--- tce-update.orig
+++ tce-update
@@ -17,6 +17,7 @@
[ -f "$1" ] && rm -f "$1"
wget -cq "$MIRROR"/"$1"
}
+
fetchzsync(){
[ -f "$1" ] && rm -f "$1"
if zsync -i "$UPGRADE_DIR"/"$1" -q "$MIRROR"/"$1".zsync; then
@@ -29,7 +30,7 @@
chkMd5Only(){
MYMD5=`cat "$F"`
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
[ "$MYMD5" == "$REPO" ] || echo $(basename "${F%.md5.txt}")
else
@@ -37,9 +38,18 @@
fi
}
+slice(){
+ while [ "$*" ]
+ do
+ echo $1" "$2
+ shift
+ shift
+ done
+}
+
upgrade(){
TARGET="$(getbasefile "$F" 2)"
- REPO=`grep -w " $TARGET" md5.db`
+ REPO=`grep " $TARGET" md5.db`
if [ -n "$REPO" ]; then
MYMD5=`cat "$F"`
if [ "$MYMD5" != "$REPO" ]; then
@@ -49,14 +59,15 @@
echo -n "Updating $TARGET "
if [ -n "$USEZSYNC" ]; then
fetchzsync "$TARGET" 2>/dev/null
+ fetchzsync "$TARGET".dep 2>/dev/null
else
fetch "$TARGET" 2>/dev/null
+ fetch "$TARGET".dep 2>/dev/null
fi
- echo "$REPO" > "$TARGET".md5.txt
+ slice $REPO > "$TARGET".md5.txt
md5sum -cs "$TARGET".md5.txt
if [ "$?" == 0 ]; then
- fetch "$TARGET".dep 2>/dev/null
- if [ "$?" == 0 ]; then
+ if [ -f /tmp/"$TARGET".dep ]; then
dep_vars /tmp/"$TARGET".dep
mv /tmp/"$TARGET".dep "$TARGET_DIR"/.
fi
@@ -80,37 +91,27 @@
check_dependencies(){
-# echo "${BLUE}Checking dependencies:${NORMAL}"
- DEPS="$(/bin/busybox find "$UPGRADE_DIR" -regex '.*\.tcz\.dep$' | sort)"
+ echo "${BLUE}Checking dependencies:${NORMAL}"
+ DEPS="$(/bin/busybox find "$TARGET_DIR" -regex '.*\.tcz\.dep$' | sort)"
[ -f $TCEDIR/tcz-black.lst ] && DEPS=$(echo $DEPS | grep -v -f $TCEDIR/tcz-black.lst)
+
+ DEPS_TO_LOAD=""
+
for F in $DEPS; do
for TARGET in `cat "$F"`; do
- if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
- if [ -n "$REPO" ]; then
- echo -n "Fetching required dependency: $TARGET "
- if [ -n "$USEZSYNC" ]; then
- fetchzsync "$TARGET" 2>/dev/null
- else
- fetch "$TARGET" 2>/dev/null
- fi
- echo "$REPO" > "$TARGET".md5.txt
- md5sum -cs "$TARGET".md5.txt
- if [ "$?" == 0 ]; then
- mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
- mv /tmp/"$TARGET" "$TARGET_DIR"/. 2>/dev/null
- UPDATED_APPS=1
- echo "OK"
- else
- echo "Failed"
- echo "Error occurred while fetching dependency: $TARGET" >> "$ERRLIST"
- rm /tmp/"$TARGET".md5.txt
- rm /tmp/"$TARGET"
- fi
- fi
+ if [ ! -f "$UPGRADE_DIR"/"$TARGET" ]; then
+ DEPS_TO_LOAD="$DEPS_TO_LOAD $TARGET"
fi
done
done
+
+ if [ "$DEPS_TO_LOAD" ] && ! tce-load -w $DEPS_TO_LOAD > "$CHECKDEP_LIST"
+ then
+ echo "Failed"
+ cat "$CHECKDEP_LIST" >> "$ERRLIST"
+ else
+ echo "OK"
+ fi
}
check_bootoptions(){
@@ -204,6 +205,8 @@
cd /tmp
ERRLIST="upgrade_errors.lst"
> "$ERRLIST"
+CHECKDEP_LIST="checkdep.lst"
+> "$CHECKDEP_LIST"
UPDATED_APPS=0
[ -e md5.db ] && rm -f md5.db
tce-fetch.sh md5.db.gz
-
... I put the commands into a script which I called update-everything ...
I've made in the same way, and everything is nice but execution time. Something like 1m50s against 15s for tce-update. Of course it is not the critical time, but difference is quite significant, while my tce13/optional is not the biggest one I suppose. Still it is correct way to update the system.
Thanks You and Rich!
-
In the http://forum.tinycorelinux.net/index.php/topic,26164.0.html thread GNUser arose the question about the behaviour of squashfs tools, concerning resquashing causing the md5 sum change. Later in the same thread Rich explains why it is natural and correct. In appliance to my thread subject, this squashfs tools fature can be used for signaling to tce tools the necessity of refreshing some extension even without changing its content.
In case the .dep file of some extension has changed the maintainer of the extension resquash an extension, this action leads to md5 sum change and triggers extension upgade (both .tcz and .tcz.dep) by tce-update without the need to use tce-audit functions.
I think the thread can be marked as solved, thanks Rich, GNUser and aus9!
Have a nice Core!
-
Still if the upgraded .dep file refers at the missing extensions the
tce-audit builddb
tce-audit fetchmissing
may be necessary. But builddb and fetchmissing seems to take less time than updatedeps:
tc@box:~$ time tce-audit builddb
Building Extension Database from /etc/sysconfig/tcedir/optional
.........................................................................................
real 0m 3.73s
user 0m 1.52s
sys 0m 1.92s
tc@box:~$ time tce-audit updatedeps
Updating .dep files from /etc/sysconfig/tcedir/optional
...
Building Extension Database from /etc/sysconfig/tcedir/optional
.........................................................................................
real 0m 53.06s
user 0m 3.20s
sys 0m 6.83s
tc@box:~$ time tce-audit fetchmissing
real 0m 30.21s
user 0m 13.68s
sys 0m 16.80s
-
tce-update making the tce-audit builddb and fetchmissing job:
--- tce-update.orig
+++ tce-update
@@ -17,6 +17,7 @@
[ -f "$1" ] && rm -f "$1"
wget -cq "$MIRROR"/"$1"
}
+
fetchzsync(){
[ -f "$1" ] && rm -f "$1"
if zsync -i "$UPGRADE_DIR"/"$1" -q "$MIRROR"/"$1".zsync; then
@@ -78,39 +79,27 @@
fi
}
-
check_dependencies(){
-# echo "${BLUE}Checking dependencies:${NORMAL}"
- DEPS="$(/bin/busybox find "$UPGRADE_DIR" -regex '.*\.tcz\.dep$' | sort)"
- [ -f $TCEDIR/tcz-black.lst ] && DEPS=$(echo $DEPS | grep -v -f $TCEDIR/tcz-black.lst)
+ echo "${BLUE}Checking dependencies:${NORMAL}"
+ DEPS="$(/bin/busybox find "$TARGET_DIR" -regex '.*\.tcz\.dep$' | sort)"
+
+ DEPS_TO_LOAD=""
+
for F in $DEPS; do
for TARGET in `cat "$F"`; do
- if [ ! -f "$UPGRADE_DIR"/"$TARGET" ] && [ ! -f "$UPGRADE_DIR"/upgrade/"$TARGET" ]; then
- REPO=`grep -w "$TARGET" md5.db`
- if [ -n "$REPO" ]; then
- echo -n "Fetching required dependency: $TARGET "
- if [ -n "$USEZSYNC" ]; then
- fetchzsync "$TARGET" 2>/dev/null
- else
- fetch "$TARGET" 2>/dev/null
- fi
- echo "$REPO" > "$TARGET".md5.txt
- md5sum -cs "$TARGET".md5.txt
- if [ "$?" == 0 ]; then
- mv /tmp/"$TARGET".md5.txt "$TARGET_DIR"/. 2>/dev/null
- mv /tmp/"$TARGET" "$TARGET_DIR"/. 2>/dev/null
- UPDATED_APPS=1
- echo "OK"
- else
- echo "Failed"
- echo "Error occurred while fetching dependency: $TARGET" >> "$ERRLIST"
- rm /tmp/"$TARGET".md5.txt
- rm /tmp/"$TARGET"
- fi
- fi
+ if [ ! -f "$UPGRADE_DIR"/"$TARGET" ]; then
+ DEPS_TO_LOAD="$DEPS_TO_LOAD $TARGET"
fi
done
done
+
+ if [ "$DEPS_TO_LOAD" ] && ! tce-load -w $DEPS_TO_LOAD > "$CHECKDEP_LIST"
+ then
+ echo "Failed"
+ cat "$CHECKDEP_LIST" >> "$ERRLIST"
+ else
+ echo "OK"
+ fi
}
check_bootoptions(){
@@ -204,6 +193,8 @@
cd /tmp
ERRLIST="upgrade_errors.lst"
> "$ERRLIST"
+CHECKDEP_LIST="checkdep.lst"
+> "$CHECKDEP_LIST"
UPDATED_APPS=0
[ -e md5.db ] && rm -f md5.db
tce-fetch.sh md5.db.gz
-
everything is nice but execution time. Something like 1m50s against 15s for tce-update.
The long execution time is because the naive script I shared in Reply #3 redownloads every .dep file that exists on your system and always runs fetchmissing.
I improved update-everything so that:
1. The only dep file that's downloaded is dep.db.gz*
2. Only changed .dep files are copied to your tcedir/optional directory
3. fetchmissing only runs if at least one of your .dep files has changed
jazzbiker, the new (smart and fast) version of update-everything is attached. I'd be happy if you'd test it and let me know your thoughts.
* This is thanks to the incredible power of awk, which can recreate every .dep file that exists on the official mirror from a single file (deb.db.gz), in a fraction of a second.
-
Hi GNUser,
I think Your hack is nice. Though dep.db.gz is accessible only for tc14.
Here are the benchmarks on my tce14.
Plain update.sh by Rich:
real 0m 45.78s
user 0m 8.47s
sys 0m 14.76s
Your update-everything:
real 0m 13.72s
user 0m 4.69s
sys 0m 5.46s
And my tce-update from post #16
real 0m 9.67s
user 0m 3.62s
sys 0m 2.62s
Aren't You using tce-fetch.sh?
-
By the way, Your update-everything and update.sh (as I named Rich's version) are not equivalent!
First I run update.sh and it found nothing to upgrade.
Then I run update-everything, and it reported:
local shared-mime-info.tcz.dep is different from one in mirror, fixing it now...
local dbus-glib.tcz.dep is different from one in mirror, fixing it now...
local atk.tcz.dep is different from one in mirror, fixing it now...
local dbus.tcz.dep is different from one in mirror, fixing it now...
If I run update-everything once again, it tells nothing.
But! If I run update.sh, and then again update-everything, it reports about the same differing .dep files once again! I think dep.db.dz in TC14 x86 repo is not updated.
What are Your thoughts?
-
Wow, jazzbiker, nice catch. My assumption is that dep.db.tz on the official repo is updated anytime a .dep file is added or modified. If this is not the case, then the efficient way my new update-everything script handles .dep files is, unfortunately, unreliable.
Question for the developers (curaga, Juanito, Rich):
Does dep.db.tz always reflect the latest version of repo's .dep files, or does dep.db.tz only get updated sporadically? If keeping it updated is not the current practice but possible to implement, pretty please implement it.
P.S. jazzbiker, regarding tce-fetch.sh, I've never had to call it directly. I think tce-update and tce call tce-fetch.sh where needed.
-
Hi GNUser,
I've made a little investigation and found that for all 4 files which were told by update-everything as rotten the difference consists of empty lines. It means that dep.db.gz is up-to-date and You need to make the processing of the .dep files a little bit smarter. You are triggering on the empty lines, that's not good.
-
Hi jazzbiker. Thank you for your investigation. I'm on it!
-
jazzbiker, where were the blank lines?
did awk generate them in /tmp/depfiles/foo.tcz or were they present in your original /etc/sysconfig/tcedir/optional/foo.tcz?
-
Hi jazzbiker. Never mind where the blank lines are. It's more efficient to ignore them in the comparison than to remove them before comparing.
I changed the comparison method. Fixed script is attached :)
-
Hi GNUser,
Life is tough, forget relaxing:
tc@box:~$ hexdump -C /mnt/sdb3/tce14/optional/dbus.tcz.dep
00000000 65 78 70 61 74 32 2e 74 63 7a 0a 20 0a |expat2.tcz. .|
0000000d
tc@box:~$ hexdump -C /tmp/depfiles/dbus.tcz.dep
00000000 65 78 70 61 74 32 2e 74 63 7a 0a |expat2.tcz.|
0000000b
Extra line consisting of the single space ...
-
Hi jazzbiker. Fortunately, the creators of diff thought of everything.
-Bbw flags should ignore absolutely all whitespace.
Unfortunately, in Busybox diff, -Bbw is broken (try diff -Bbwsq on two files that differ only by whitespace and you'll see that files are reported as being different). In GNU diff, -Bbw works exactly as expected.
diffutils.tcz is now a dependency of the smart and fast update-everything script, unfortunately.
Here is the (hopefully) final version and time to relax :)
-
dep.db.gz is updated whenever the repo files are updated. This does mean if someone edits a .dep file manually, it may be out of sync until a new extension gets added etc.
-
Thanks for the information, curaga.
I think the efficiency gained by downloading dep.db.gz (rather than redownloading every .dep present on one's system) is worth the minimal risk of dep.db.gz being temporarily out of sync due to a manual .dep edit on the repo.
For full system update, using dep.db.gz translates into a win for the user (update process takes seconds rather than, potentially, minutes) and a win for your server (one wget request per user rather than, potentially, hundreds).
-
Hi curaga
If I place an updated .dep file in staging and then run update repo, does
dep.db.gz get updated then?
-
Here is the (hopefully) final version and time to relax :)
Something of that kind, I guess ;-)
By the way, isn't using GNU diff a shooting crows with the nuke? Haven't You tried
[ "$(cat file1)" == "${cat file2}" ]
?
-
Hi jazzbiker. I agree that GNU diff is shooting a crow with a nuke here. cat is a good idea, but doesn't work by itself doesn't work because it preserves whitespace. echo eats whitespace, so if we combine cat and echo then it works :)
Also, I did some experiments and noticed that fetchmissing is not needed. If a new .dep file creates the need to fetch an additional dependency, the tce-update at the end takes care of it (tce-update has a "Fetching required dependency..." step).
So here is latest version:
1. Uses cat+echo instead of GNU diff for the comparison
2. fetchmissing is eliminated because tce-update already takes care of that
Your thoughts?
EDIT: If local .dep file differs from repo's .dep file only by order in which dependencies are listed, the update-everything script will still see the difference and will update the .dep file. This is desirable because sometimes the order in which the dependencies are loaded is important.
-
cat is a good idea, but doesn't work by itself doesn't work because it preserves whitespace. echo eats whitespace, so if we combine cat and echo then it works :)
Damn! Shell is the best example how the road to hell is paved with the good intentions :-( You with Rich can ban me, but in my opinion loving shell is the kind of perversion ... Sorry for dumb complaining.
-
Hi GNUser,
I can confirm that upon my personal tce14/optional in its actual state Rich's version and Your current version of update-everything work in the same way. But I gave up predicting in what kind shell will push us into the back in another circumstances/repo state. Maybe even GNU nuke is the final solution.
I can not agree about excluding fetchmissing. tce-audit fetchmissing uses recursive tce-load, while check_dependencies() in original tce-update provides fetching of the only one level of dependencies.
-
I can not agree about excluding fetchmissing. tce-audit fetchmissing uses recursive tce-load, while check_dependencies() in original tce-update provides fetching of the only one level of dependencies.
Nice catch, jazzbiker. I put tce-audit fetchmissing back in.
I did some tests and, if a .dep file has been updated and now includes a new dependency, tce-audit fetchmissing does nothing whatsoever unless you re-run tce-audit builddb after the .dep file has been fixed. Tricky, tricky. Script now takes care of this.
Latest version of script (5.0) is attached.
-
in my opinion loving shell is the kind of perversion ... Sorry for dumb complaining.
The shell sure does have a lot of warts and whiskers, can't argue with you there. But I love it, anyway. I'm okay with being strange ;D
-
The shell sure does have a lot of warts and whiskers, can't argue with you there. But I love it, anyway.
I know some people love gambling ;-) Well, not a single word more about my inability to understand everything in this world.
update.sh:
real 0m 45.41s
user 0m 8.70s
sys 0m 14.94s
update-everything:
real 0m 13.56s
user 0m 5.40s
sys 0m 6.05s
The gain is significant. 5.0 is nice number for the final version!
-
jazzbiker, thank you for your collaboration. The script is that much better for it.
But I gave up predicting in what kind shell will push us into the back in another circumstances/repo state. Maybe even GNU nuke is the final solution.
No need for nukes. As long as the shell is Busybox ash and we are using Busybox utilities, script's behavior will be predictable. This part:
. /etc/init.d/tc-functions
useBusyboxtakes care of the latter (ensuring Busybox utilities). Since, technically, /bin/sh can be a link pointing to any shell whatsoever, we should probably change the shebang from /bin/sh to /bin/ash (to ensure the script runs in Busybox ash, which is the shell you and I have used to test the script).
I attach the script (again) with more specific shebang. I made no other changes.
Happy hacking!
-
Hi GNUser
The rebuildfstab script uses:
#!/bin/busybox ashThis way it doesn't matter where the busybox link is located or if it even exists.
It also includes this:
useBusybox
PATH="/bin:/sbin:/usr/bin:/usr/sbin"
export PATHuseBusybox only includes some of the busybox commands, cut and ps are 2 of the commands that are missing.
The PATH setting pretty much forces all busybox commands to be used.
-
Hi Rich. Thanks for the better shebang. Restricting the environment further by specifying PATH is a great idea. Version bump to 5.1.
jazzbiker will appreciate the added safeguards. Not all surprises are good surprises.
-
One last thought: No need to build package database twice. It can be done only once, after syncing .dep files and before tce-update fetchmissing.
I'm going to stop attaching latest version of the script. This link will always point to latest version:
http://gnuser.ddns.net/public/update-everything
P.S. Rich, please feel free to delete all my attachments from this thread if you think it would reduce clutter and save space on the forum server.
-
Hi GNUser
... P.S. Rich, please feel free to delete all my attachments from this thread if you think it would reduce clutter and save space on the forum server.
Thanks, the number of attachments does need to be trimmed down again.
-
Hi Rich. I have a question about this:
It also includes this:
useBusybox
PATH="/bin:/sbin:/usr/bin:/usr/sbin"
export PATHuseBusybox only includes some of the busybox commands, cut and ps are 2 of the commands that are missing.
The PATH setting pretty much forces all busybox commands to be used.
It seems redundant to call useBusybox and also set this limited PATH that basically only allows busybox and base system commands. On a TCL system, the latter (limited PATH) accomplishes everything the former (useBusybox) does, and then some.
Can you think of any reason why a script such as rebuildfstab or update-everything should call useBusybox and set the PATH?
-
Hi Rich,
Posts #8 - #12 may be easily removed along with the attachments.
-
Hi GNUser
... Can you think of any reason why a script such as rebuildfstab or update-everything should call useBusybox and set the PATH?
No, I can't. All I can tell you is rebuildfstab had both at least as far back as TC4.
It's possible PATH was added out of frustration to quickly force all busybox functions as defaults.
-
Hi Rich. Thank you for confirming my suspicion. And for being such a good archeologist.
I don't mind using a bigger gun to cover against a wider range of possible interference. But I terribly dislike redundancy--not only because of bloat, but also because it makes it difficult to guess what I was thinking when I look at a script again much later. I updated the script (including the link in Reply #40) accordingly.
-
Hi curaga
If I place an updated .dep file in staging and then run update repo, does
dep.db.gz get updated then?
I'm not sure putting just a dep file in staging would work, but the same step that generates provides.db also generates dep.db.
-
Here is the (hopefully) final version and time to relax :)
Something of that kind, I guess ;-)
By the way, isn't using GNU diff a shooting crows with the nuke? Haven't You tried
[ "$(cat file1)" == "${cat file2}" ]
?
I think I have seen a better way of this and faster.
You not need to load cat for doing this.
$(cat file1)
You can do this.
$(< file1)
Very clever, but I don't know if this works with every shell ?
-
Hi patrickg.
You not need to load cat for doing this.
$(cat file1)
You can do this.
$(< file1)
Very clever, but I don't know if this works with every shell ?
It does not seem to work, at least not in Busybox ash. Try this with files foo.dep and bar.dep:
[ "$(< foo.dep)" = "$(< bar.dep)" ] && echo "files are identical" || echo "files are different"The goal is to see "files are identical" if the two files contain the same words (i.e., extensions) and differ only by whitespace (i.e., leading whitespace, trailing whitespace, extra lines, extra lines with spaces). The code snippet above results in "files are identical" all the time, even if the two files contain totally different words.
I'm okay with using cat and echo for this. It gets the job done, requires nothing outside base system, and does not cross over beyond clever into cryptic.
-
Hi curaga
I'm not sure putting just a dep file in staging would work, but the same step that generates provides.db also generates dep.db.
OK, I just tried it and dep.db.gz does get updated when only a .dep file is present.
-
Sorry for confusing, it does work in bash.
-
Hi patrikg. No it doesn't, unfortunately. I just tested it in bash: If foo.dep and bar.dep differ only by whitespace (empty lines, for example), then I get "files are different" when I use the code snipped I posted in Reply #48. This is not what we need.
It was a good thought, though. Thanks for pitching in.
-
Hi jazzbiker. Believe it or not, I found yet another opportunity for increased efficiency in the update-everything script, which will further decrease your execution time.
TL;DR explanation:
In version 6.1 of the update-everything script, tce-audit builddb is always one of the steps. However, it only needs to run if .dep files have changed, so that tce-audit fetchmissing works properly. Version 7.0 of the script (available at the link in Reply #40) takes this into account.
Detailed explanation:
tce-audit builddb produces three files:
/tmp/audit_results.txt
/etc/sysconfig/tcedir/optional/tce.db
/etc/sysconfig/tcedir/optional/tce.lst.
It turns out that tce-update does not utilize any of these files. I confirmed this not only by examining the tce-update script, but also by running numerous tests. As far as tce-update is concerned, it's all about the .md5.txt files :)
Based on earlier tests, I know that if .dep files have changed and your system is now missing some dependencies, then tce-audit builddb needs to run before tce-audit fetchmissing or else the missing dependencies are not fetched.
So here is the current (version 7.0) logic of the script:
Are system's .dep files current?
If yes: Run tce-update
If no: Update the .dep files that are outdated, run tce-audit builddb, run tce-audit fetchmissing, run tce-update
-
Strange it works for me in bash, and for more confusing things in bash scripts, i think you missing some chars in the if/test statement. You have to evaluate twice i think in the if/test statement. And i think you missing one equal sign.
So i should try something like this.
[[ "$(< foo.dep)" == "$(< bar.dep)" ]]
And for more confusing things in the shell, i think the char [ is a command, alias to test.
So maybe it's fine with only one equal sign, but with the if statement i think you need to use dubble equal signs
to not assign the variable with some value.(if using only one equal sign).
And all this is in bash shell. Not ash.
Like i said before I am not any programmer.
Happy hacking.
-
Hi patrikg. You are right that [ is an alias for the test command.
[[ is only available in bash. It does have more features than test.
In all fairness I did not test your suggestion in bash with [[, only in bash with [
I'll take your word for it that your suggestion works with bash's [[, but I don't want to drag in bash as a dependency for the script.
[ foo == bar ] is a bashism, see https://mywiki.wooledge.org/Bashism. Some POSIX shells can work with that, but the correct POSIX syntax for comparison is [ foo = bar ]
I know that in many contexts == is for comparison and = is for assignment, but not in this context.
-
Hi jazzbiker. Believe it or not, I found yet another opportunity for increased efficiency in the update-everything script, which will further decrease your execution time.
Hi GNUser,
Believe it or not, but the new version is really faster :-) I've run it in the same conditions with the same tce14 directory content:
real 0m 11.31s
user 0m 4.65s
sys 0m 4.67s
to be compared with the measurements in the post #36.
Well done!
-
Now i have tested i little bit with busybox ash.
I thinks it not working.
See this.
~ $ [ "$(</etc/passwd)" = "$(</etc/passwd)" ] && echo Working
Working
~ $ [ "$(</etc/passwd)" = "qwertyu" ] && echo Working
~ $
And for more test.
[patrik@hp-arch-new ~]$ ./busybox ash
~ $ echo "test" > test1
~ $ echo "test" > test2
~ $ [ "$(<test1)" = "$(<test2)" ] && echo Working
Working
~ $ echo "test1" > test3
~ $ [ "$(<test1)" = "$(<test3)" ] && echo Working
Working
~ $ cat test3
test1
~ $ [ "$(cat test1)" = "$(cat test3)" ] && echo Working
~ $ [ "$(cat test1)" = "$(cat test2)" ] && echo Working
Working
~ $
I am not any programmer and i don't understand whats happening with this!! You are correct, this is not working.
[patrik@hp-arch-new ~]$ ./busybox
BusyBox v1.35.0 (2022-01-17 19:57:02 CET) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2015.
Licensed under GPLv2. See source distribution for detailed
-
Hi GNUser,
Your current 7.0 version calls tce-update. tce-update includes check_dependencies() function, which does noop after fetchmissing. I've commented calls of check_dependencies() in original tce-update and here are the measurements:
real 0m 10.44s
user 0m 4.29s
sys 0m 4.23s
Edit: I may be wrong, but I have the feeling that check_dependencies() in tce-update is the reminder from the ancient times when tce-load was not performing recursive loading.
-
Hi patrikg. I think you are missing a nuance here. The .dep files being compared need to be considered identical if they differ only by whitespace. Please take a look earlier in this thread where jazzbiker points out the need for this.
Hi jazzbiker. It's true that tce-update does not need to look for missing dependencies if fetchmissing has already taken care of that. Unfortunately tce-update does not currently give us the option of skipping check_dependencies (short of commenting-out the call to the function as you did).
-
Hi jazzbiker. Thank you for finding the possibility of yet another efficiency. I'm going to submit a pull request to add --skip-dependency-check option to tce-update. Then, logic of update-everything can be upgraded to this:
Are system's .dep files current?
If yes: Run "tce-update --skip-dependency-check"
If no: Update the .dep files that are outdated, run tce-audit builddb, run tce-audit fetchmissing, run "tce-update --skip-dependency-check"
This should achieve a further decrease in your execution time :)
-
Hi GNUser,
In my opinion in the current state of TC there is no need in calling check_dependencies() in tce-update. I propose You to write the pull-request for commenting out of the 2 calls of check_dependencies().
-
... i don't understand whats happening with this!! ... this is not working.
Hi patrikg,
Thank You, very nice description of shell scripting! By the way I think the leading "s" in the word "shell" may be omitted, for best descriptivity and saving some internet traffic ;-)
-
Hi jazzbiker. I disagree with commenting-out the calls to the function. If a naive user upgrades system using only tce-update, .dep file is re-downloaded for every extension being upgraded. The check_dependencies function provides the valuable service of downloading any missing dependencies in this scenario.
The value that update-everything adds is that it checks .dep files of all local extensions--those that are about to be upgraded as well as those that are not. It is only in this special scenario (where all .dep files have been checked and any missing dependencies taken care of) that check_dependencies() does nothing except waste CPU cycles and your time.
I created a pull request in GitHub to add --skip-dependency-check to tce-update. If/when the pull request is merged and the new tce-update becomes part of the base system, I will change line 29 in update-everything from tce-update to tce-update --skip-dependency-check and the script will become a little bit faster.
-
Hi jazzbiker, thx for your thoughts.
And omit the "s" is what it is.*lol*
-
Hi jazzbiker. I disagree with commenting-out the calls to the function. If a naive user upgrades system using only tce-update, .dep file is re-downloaded for every extension being upgraded. The check_dependencies function provides the valuable service of downloading any missing dependencies in this scenario.
Hi GNUser, check_dependencies() provides only single-level dependency fetching, opposite to tce-load, utilized by "tce-audit fetchmissing" (by the way we mentioned it earlier in this thread), so life of naive user is sad and unpredictable like trying to write scripts in one widely used language, until he'd got the good news about Your excellent update-everything!
-
i don't understand whats happening with this!!
Haha, I've been there before. In situations like this, I think it's best to just try a different approach. Fortunately, in this case the cat+echo dynamic duo has already won the day.
Sometimes, if you are convinced you understand all the nuances and the outcome is still not what you'd expect, it may be that you found a bug in the software.
-
Hi GNUser, check_dependencies() provides only single-level dependency fetching, opposite to tce-load, utilized by "tce-audit fetchmissing" (by the way we mentioned it earlier in this thread)
Good point, but right now I'm just interested in winning the small battle of optimizing update-everything. For this, all I need is a way to opt-out of check_dependencies(). What you are suggesting would involve a bigger war that I don't want to fight today.
so life of naive user is sad and unpredictable
Yes, it is. It's not easy being a naive user.
P.S. Would TCL developers like me to create a pull request to add update-everything to base system? With the insights provided by jazzbiker and Rich--not to mention the freakish power of awk--the script has morphed into something quite fast and reliable.
-
I am now in school for learning more about IT Security.
How we now can defeat the security problems like not using username and passwords only for logging in.
And more using 2FA or using some kind of hardware token like Yubikey.
Now we have some lab's to do, and using tools like john.
And just run that command spits out some usage: like this.
John the Ripper password cracker, version 1.8.0
Copyright (c) 1996-2013 by Solar Designer
Homepage: http://www.openwall.com/john/
Usage: john [OPTIONS] [PASSWORD-FILES]
--single "single crack" mode
--wordlist=FILE --stdin wordlist mode, read words from FILE or stdin
--rules enable word mangling rules for wordlist mode
--incremental[=MODE] "incremental" mode [using section MODE]
And then just try to load the man page with same program spits out.
OPTIONS
All the options recognized by john start with a single dash (`-'). A summary of options is included below.
-external:MODE
Enables an external mode, using external functions defined in ~/john.ini's [List.External:MODE] section.
-format:NAME
Allows you to override the ciphertext format detection. Currently, valid format names are DES, BSDI, MD5, BF, AFS, LM.
And now to the first line in the options section, says only single dash.
And now see how many you need to provide when just running the command...., two dash or single dash (like hamlet to be or not to be)
-
Hi GNUser
If an extension does not have a .md5.txt file included, do you still update the .dep file?
For example:
Someone wants to use xyzzy.tcz from TC10 which depends on libffi.tcz (libffi.so.6)
They are running TC13 and that newer xyzzy.tcz depends on libffi.tcz (libffi.so.7)
So they modified their .dep file to include libffi6.tcz instead of libffi.tcz for backward compatibility.
-
Hi GNUser
If an extension does not have a .md5.txt file included, do you still update the .dep file?
Hi Rich. No. update-everything does not touch foo.tcz.dep if local foo.tcz.md5.txt is missing:
elif [ ! -f $OPTIONALDIR/$extension.md5.txt ]; then
continue # if user deleted local md5.txt, it's likely a custom extension--leave it alone
-
Hi GNUser,
You cheating? How can You preliminarily insert into Your script the workaround on the bug which Rich will find few hours later?
-
Haha. Yes. I broke into Rich's wizard tower and stole his crystal ball, which has been working quite well. Three days ago I already knew he would ask this question.