Possible bug for 64 bit submitqc for tce.installed issue

[aus9]

Possible bug for checking of tce.installed using 64 bit submitqc

First my submitqc has md5sum
7b157080f71142c84a5faf2cc66b3308  submitqc.tcz

I do not care about all tests, as submitqc seems to cause no problems for them
so if you have time, can someone on 64 bit either confirm or deny the following test.

I create a tcz using root powers
get submitqc to check it
post a snippet of correction it does
unpack the TCZ and show what has occurred ....YMMV

copy and paste from quote box into your favourite terminal please after becoming root by

Code: [Select]

sudo su

# test we are root
if [ "$(id -u)" != "0" ]; then
   echo "run as root now exitting"
   exit 1
fi

USER=`cat /etc/sysconfig/tcuser`
su -c "tce-load -i submitqc" $USER
cd /tmp
mkdir -p test/usr/local/tce.installed
touch test/usr/local/tce.installed/test
# ^^ normal executable was also checked but this is shorter
chmod 775 -R test/usr/local/tce.installed
chown root:staff -R test/usr/local/tce.installed
mksquashfs test test.tcz
submitqc --fix

Please ignore the warning I have not made an info file or md5 or dep

What you might see if bug is there?

Code: [Select]

tc@box:~$ sudo su
root@box:/home/tc# if [ "$(id -u)" != "0" ]; then
>    echo "run as root now exitting"
>    exit 1
> fi
root@box:/home/tc#
root@box:/home/tc# USER=`cat /etc/sysconfig/tcuser`
root@box:/home/tc# su -c "tce-load -i submitqc" $USER
zsync.tcz: OK
libzstd.tcz: OK
lzo.tcz: OK
squashfs-tools.tcz: OK
binutils.tcz: OK
grep.tcz: OK
libcap.tcz: OK
coreutils.tcz: OK
nettle.tcz: OK
gnutls.tcz: OK
wget.tcz: OK
diffutils.tcz: OK
file.tcz: OK
submitqc.tcz: OK
root@box:/home/tc# cd /tmp
root@box:/tmp# mkdir -p test/usr/local/tce.installed
root@box:/tmp# touch test/usr/local/tce.installed/test
root@box:/tmp# # ^^ normal executable was also checked but this is shorter
root@box:/tmp# chmod 775 -R test/usr/local/tce.installed
root@box:/tmp# chown root:staff test/usr/local/tce.installed
root@box:/tmp# mksquashfs test test.tcz
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on test.tcz, block size 4096.


Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 4096
compressed data, compressed metadata, compressed fragments, no xattrs
duplicates are removed
Filesystem size 0.29 Kbytes (0.00 Mbytes)
69.93% of uncompressed filesystem size (0.42 Kbytes)
Inode table size 80 bytes (0.08 Kbytes)
49.38% of uncompressed inode table size (162 bytes)
Directory table size 76 bytes (0.07 Kbytes)
71.03% of uncompressed directory table size (107 bytes)
Number of duplicate files found 1
Number of inodes 5
Number of files 1
Number of fragments 0
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 4
Number of ids (unique uids + gids) 2
Number of uids 1
root (0)
Number of gids 2
root (0)
staff (50)
root@box:/tmp# submitqc --fix

==============================================================
This is the Tiny Core extension submission quality testing tool.
This pre-release is UNDER TESTING for TC5/6 and x86/x86_64.

see --help for usage instructions.

Release: SOURCE RUN
==============================================================

Checking against Core release 9.x on x86_64 / 4.14.10-tinycore64

submitqc: loading required extensions ... Done.
submitqc: removing old log files. Ok.
submitqc: making space for new log files. Done.
submitqc: checking http://repo.tinycorelinux.net/ (timeout 3 sec) ... Connected!
submitqc: fetching info.lst.gz ... Ok.

submitqc: test.tcz checks begin =====
submitqc: test.tcz squashfs tree has files in it? Ok.
submitqc: test.tcz info file structure ok?
test.tcz.info file not found. Do NOT submit without it.
submitqc: test.tcz info similar to others in repository?
There are similar named extensions in repo:
firefox_getLatest
googletest-dev
googletest
mariadb-10.1-test
Ensure there are no conflicts between test.tcz and other extensions.
submitqc: test.tcz dep file compared to repository copy? No dep file in repo - done.
submitqc: test.tcz dependencies exist? No dep file. Ok.
submitqc: test.tcz mountable? Passed mount check.
submitqc: test.tcz correct block size? Yes.
submitqc: test.tcz unsquashing for further tests ... Done.
submitqc: test.tcz squashfs tree ok? Ok.
submitqc: test.tcz startup files ok?
Startup script permissions were corrected.
Startup script ownership was corrected.
submitqc: test.tcz includes copyright notice?
Copyright notice not found at usr/local/share/doc/test
Confirm that this is not needed.
Checking repo for copyright statement in test.tcz ...
test.tcz not found on server.
submitqc: test.tcz file/directory ownerships & permissions ok? Ok.
submitqc: test.tcz binaries have been stripped? Ok.
submitqc: test.tcz list file looks ok?
test.tcz.list missing. Creating ... Done.
submitqc: test.tcz comparing list to mirror copy. Not on mirror.
submitqc: test.tcz to enable library inclusion tests, use --libs.
Without lib tests, check is considered incomplete.
submitqc: make squashfs with changes ... Done. test.tcz + test.tcz.md5.txt updated.
submitqc: test.tcz checking md5.txt. Ok.
submitqc: test.tcz squash tree checks complete. Cleaning up ... Done.
submitqc: test.tcz creating new test.tcz.zsync Done.
submitqc: test.tcz.build-dep or build-dep included?
Not found, but it is suggested/requested.
submitqc: test.tcz.tgz included?
Not found, but it is suggested/requested.
submitqc: test.tcz: setting ownership & permissions of test.tcz* ... Ok!
submitqc: test.tcz checks end =====


submitqc: There are warnings or errors for your extensions. See:
/tmp/submitqc/copyright
/tmp/submitqc/missingdeps/test.tcz.dep-libs
/tmp/submitqc/noinfofile
/tmp/submitqc/similarextensions
/tmp/submitqc/wrongstartscriptperms
Some errors may have been corrected - see logs and above.
Fix errors and run submitqc again before submitting.
Log files will be destroyed the next time submitqc is run.
Without warnings or errors, this message will go away.
Let me repeat the bits that concerns me please

Code: [Select]

Startup script permissions were corrected.
Startup script ownership was corrected.
/tmp/submitqc/wrongstartscriptperms
Now lets check what it did to me
copy and paste from quote box please

unsquashfs test.tcz
ls -al squashfs-root/usr/local/tce.installed/

What I get

Code: [Select]

unsquashfs test.tcz
Parallel unsquashfs: Using 4 processors
1 inodes (0 blocks) to write



created 1 files
created 4 directories
created 0 symlinks
created 0 devices
created 0 fifos
root@box:/tmp# ls -al squashfs-root/usr/local/tce.installed/
total 0
drwxrwxr-x 2 root staff 60 Aug  8 14:06 .
drwxr-xr-x 3 root root  60 Aug  8 14:06 ..
-rwxr-xr-x 1 tc   staff  0 Aug  8 14:06 test
according to the wiki and the forum post for submissions.....that dir and script  is supposed to be
root:staff
775

thanks for reading

I sent a pm to the current maintainer  dentonlt but he has not been active for a long time
and its been more than one week since pm sent

IMHO the dir is fine but the script is wrong?

[aus9]

I forgot to paste the -R results above but they are the same as without the -R.

quote box is still good.

[coreplayer2]

Possible bug for checking of tce.installed using 64 bit submitqc
...

unsquashfs test.tcz
ls -al squashfs-root/usr/local/tce.installed/

What I get

Code: [Select]

unsquashfs test.tcz
Parallel unsquashfs: Using 4 processors
1 inodes (0 blocks) to write



created 1 files
created 4 directories
created 0 symlinks
created 0 devices
created 0 fifos
root@box:/tmp# ls -al squashfs-root/usr/local/tce.installed/
total 0
drwxrwxr-x 2 root staff 60 Aug  8 14:06 .
drwxr-xr-x 3 root root  60 Aug  8 14:06 ..
-rwxr-xr-x 1 tc   staff  0 Aug  8 14:06 test
according to the wiki and the forum post for submissions.....that dir and script  is supposed to be
root:staff
775

thanks for reading

I sent a pm to the current maintainer  dentonlt but he has not been active for a long time
and its been more than one week since pm sent

IMHO the dir is fine but the script is wrong?

I haven't spoken to dentonit lately but I guess I'll take a look at this in the meantime..

To my knowledge directory permissions should be 775 root:staff
iirc the only requirement for the install script is that it's executable.

does your install script not run?

[coreplayer2]

As pointed out by gutmensch in this post http://forum.tinycorelinux.net/index.php/topic,14988.msg85727.html#msg85727, tce.installed directory has a special tc use which requires 775 & root:staff. 

[coreplayer2]

aus9  rather than unpacking the extension, why not load the extension and then verify ownership of the startup file?

I'll check the corepure64 permissions later but in tinycore ownership of the startup file appears inherited or set during the boot process regardless of assigned ownership during creation.

[aus9]

good suggestion.

I had to change a few things as name=test conflicted with an exe from coreutils as I wanted to make an exe
so test name changed to z.....with an u loc bin called z which shows the following

Code: [Select]

tce-load -i z
this is a test
z.tcz: OK
tc@box:~$ z
aberr mc-root   tce       wm_errors  z.tcz.list
appserr  mc-tc    tcloop       z z.tcz.md5.txt
k5_skip  submitqc  Temp-1f254851-6e52-45f9-a51e-8e5c07694220  z.tcz z.tcz.zsync
tc@box:~$ ls -al /usr/local/tce.installed/z
lrwxrwxrwx 1 root root 39 Aug  8 19:25 /usr/local/tce.installed/z -> /tmp/tcloop/z/usr/local/tce.installed/z
as you can tell...the install script is just an echo command to show "this is a test" and the exe lists /tmp
but I loaded z.tcz and its md5 into my tcedir ....which is what most people would do rather than load from /tmp.

To my eyesight....even tho submitqc --fix does this to z

Code: [Select]

submitqc: There are warnings or errors for your extensions. See:
/tmp/submitqc/copyright
/tmp/submitqc/missingdeps/z.tcz.dep-libs
/tmp/submitqc/noinfofile
/tmp/submitqc/similarextensions
/tmp/submitqc/wrongstartscriptperms
Some errors may have been corrected - see logs and above
I think I can infer that it is unnecessary to have a build script that says

(sudo) chmod -R 775 <package-name>/usr/local/tce.installed
(sudo) chown -R root:staff <package-name>/usr/local/tce.installed

as it appears to be ignored?

instead maybe (sudo if run as non-root)
(sudo) chmod 775 <package-name>/usr/local/tce.installed
maybe 755 <package-name>/usr/local/tce.installed/<package-name>
(sudo) chown  root:staff <package-name>/usr/local/tce.installed
(sudo) chown  root:root <package-name>/usr/local/tce.installed//<package-name>

or am I over complicating it?

[aus9]

If I may add an unrelated issue I see sometimes in running submitqc....FYI

Code: [Select]

WARNING: dependency wireless-KERNEL.tcz missing - tests will be inconclusive.

WARNING: dependency input-joystick-KERNEL.tcz missing - tests will be inconclusive.
Ok!

WARNING: error during tests - inconclusive

snip

Without lib tests, check is considered incomplete
the bottom line appears if you run only submitqc --fix
which is why in the past I have run submitqc --libs

I am guessing submitqc does not interpret our current Kernel release and is searching literally?

[coreplayer2]

Keep it simple....   
No need to waste effort adjusting permissions either manually or in build scripts,  submitqc --fix will attempt to maintain extension consistency.
Just run submitqc --fix in a temporary directory where the newly created extension and dep file is located,  submitqc will fix any required issues while remainder of errors are for your info only.  eg license files in wrong location etc etc..   manually fix as necessary and run submitqc again to create a new md5 file,  then copy to your tce/optional directory for testing. 
Make any adjustments to the info file and tar up the whole temp directory and submit
Easy huh?  Don't get all wound up about informational notices submitqc spits out..
Submitqc --(options) are for informational purposes,  submitqc --fix will correct most errors, note the summary messages for manual adjustments.
Not all errors need fixing, they're mostly advisory to inform of misplaced license files etc etc.  run submitqc a second time to see if any issues remain after the last --fix


 

[polikuo]

I've pointed out this issue years ago. link
I prefer ownership "root:staff" with permission "775", but it stays that way... 

[coreplayer2]

I've pointed out this issue years ago. link
I prefer ownership "root:staff" with permission "775", but it stays that way... 

I believe TC's special use rules applies only to the installed directory, not the files within it.

[polikuo]

I believe TC's special use rules applies only to the installed directory, not the files within it.

Since the wiki specifically states that all files should be owned by "root"

All files root:root, 644 for files, 755 for executables, 755 for directories

It just feels weird to have the script owned by "tc"

[aus9]

wiki...I am to blame for poor writing. It was me who made up those rules after a discussion with gutmensch. In context you may see that I wrote (or I think I wrote)
Tiny Core special settings, e.g. root:staff for /usr/local/tce.installed and 775
snip
All files root:root, 644 for files, 755 for executables, 755 for directories

Meaning I need to obey the top rule before I can apply the rules below. Maybe I should have written that as ...........All "other" files root:root etc

2)

I believe TC's special use rules applies only to the installed directory, not the files within it.

and this is what I was seeking clarity on. thanks now that I read the link
I see Juanito has said

Only the /usr/local/tce.installed folder needs to be root:staff - the files in the folder can be tc:staff

but I was not aware of his post when I did my stuff to that wiki years ago.

3) But being anal....you may see lots of people including a moderator in a recent post with me
showed a build -compile script where he used the recursive switch

chown -R root:staff  package-name/usr/local/tce.installed  (from memory he wrote a variable for package-name etc)

and believe it or not....I think build scripts that have a single line using -R are easy and manageable.  It was probably me who wrote that chown in the wiki too....as I use /tmp to build packages

IMHO for tce.installed, we should keep it simple...so checking is easier and quicker?
IMHO it seems non-intuitive to have the dir with one owner group and file below a different owner

4) and lets pretend I don't know what I am talking about

just run

Code: [Select]

ls -al /usr/local/tce.installed when you have a number of packages loaded more than xbase.lst

to my eyesight it looks messy or un-professional to see both root:staff and tc:staff

thats my 2 cents worth

[coreplayer2]

...
just run

Code: [Select]

ls -al /usr/local/tce.installed when you have a number of packages loaded more than xbase.lst

to my eyesight it looks messy or un-professional to see both root:staff and tc:staff

thats my 2 cents worth

So the issue is cosmetic? 

When you see files within tce.installed you'll notice many zero byte files which are created by TC during the extension install process.  Note the ownership and permissions on these files..

[aus9]

I would not tag it as cosmetic. But to summarize our efforts so far.

We agree that tce.installed is to be owned at dir level by root:staff

What I am now focus on is the issue with the wiki under the section called

Adding Custom Startup Scripts
snip

Code: [Select]

sudo chown -R root:staff /tmp/package/usr/local/tce.installed
sudo chmod -R 775 /tmp/package/usr/local/tce.installed
I know I am not the only person to have this in their builds.  I am pretty sure those lines existed well before I came along. What do I now put in my build script if the actual script is NOT to be root:staff

Depending on whether Juanito or other Moderators have time to reply....I am happy to amend the wiki
now that we have full transparency on this issue.

Sorry in advance but I do regard this as a trivial issue. I want my build scripts and other members who submit to pass first time if possible

and members may not all use "submitqc --fix" in their build scripts so am hoping to get a clear answer on this.....and yes its still bugging me

cheers

gordon

[coreplayer2]

...
to my eyesight it looks messy or un-professional to see both root:staff and tc:staff
...

I'm fairly sure that whether files within tce.installed directory are owned by root:root, root;staff, or $USER:staff is dependent on who invoked the install process using tce-load 

So i'm sorry to say you're never going to be happy about this...