WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: extension audit script  (Read 138727 times)

Offline dentonlt

  • Sr. Member
  • ****
  • Posts: 318
    • the trombone analog
Re: extension audit script
« Reply #75 on: October 16, 2015, 10:07:24 PM »
20151017 update is now in the submission queue.

Major change is that I've added dynamic library checking. This scans the extension binaries for dynamic lib requirements, then compares this against the libraries included in extension & the extension dep list. As possible, missing libs are resolved against the local repo and then suggested additions to dep list are made.

Also added --core, --arch, --license, and --changes cli options. Cross-repo maintainers may find the former useful.

I've run this script against much of my local repo (500 extensions in each of x86 and x86_64) and it seems stable. Bug reports and patches welcome, of course.

_madRat

  • Guest
Re: extension audit script
« Reply #76 on: February 29, 2016, 06:49:25 AM »
Is the simple way to disable "Copyright notice" warning in submitqc script - it can not found license file in package.tcz, because it as doc files are located in package-doc.tcz.
« Last Edit: February 29, 2016, 06:51:08 AM by _madRat »

Offline Misalf

  • Hero Member
  • *****
  • Posts: 1702
Re: extension audit script
« Reply #77 on: February 29, 2016, 09:13:01 AM »
Note:
Quote
Please check the license condition. Although we can strip a number of documents from the main tcz and have a separate doc.tcz, this is not always possible. [...]
http://wiki.tinycorelinux.net/wiki:creating_extensions#license
Download a copy and keep it handy: Core book ;)

Offline dentonlt

  • Sr. Member
  • ****
  • Posts: 318
    • the trombone analog
Re: extension audit script
« Reply #78 on: March 16, 2016, 03:41:39 PM »
Is the simple way to disable "Copyright notice" warning in submitqc script - it can not found license file in package.tcz, because it as doc files are located in package-doc.tcz.

_madRat, Misalf: thanks for mentioning this, and the warning re license checking. I don't currently have flags/args for disabling specific tests, but that's probably a good idea for the future. I'll add it to the queue.

Overall, just use good judgement when you're submitting. Some errors/warnings will be acceptable depending upon the package you're submitting. The script currently warns you to contact the maintainer even if you're the current maintainer, for example ...

Other news: thanks to andyj for his notes on armv7 - this lets submitqc run on arm now. I have no arm hardware for testing, however - the script may eat your data! Bug reports and test results much needed and welcome.

submitqc submitted today for 8 repos: 5.x, 6.x, 7.x on x86, x86_64, and armv7.

Offline Misalf

  • Hero Member
  • *****
  • Posts: 1702
Re: extension audit script
« Reply #79 on: March 16, 2016, 05:03:04 PM »
dentonlt, first of, thanks for contributing and maintaining submitqc.
I find text coloring in the output of scripts is very useful, however, I noticed there is none in the in the current submitqc, contrary to former versions. While there are lots of warnings which might be considered redundant, depending an what extensions are checked (not of your concern), it can be quite cumbersome to check the hole output by reading every line, compared to 'viewing' (f.e. red=error, yellow=warning, green=success etc.).

Any plans for fancy colors for those read-impaired people like me?
« Last Edit: March 16, 2016, 05:05:57 PM by Misalf »
Download a copy and keep it handy: Core book ;)

Offline dentonlt

  • Sr. Member
  • ****
  • Posts: 318
    • the trombone analog
Re: extension audit script
« Reply #80 on: May 17, 2016, 03:38:24 PM »
dentonlt, first of, thanks for contributing and maintaining submitqc.
I find text coloring in the output of scripts is very useful, however, I noticed there is none in the in the current submitqc, contrary to former versions. While there are lots of warnings which might be considered redundant, depending an what extensions are checked (not of your concern), it can be quite cumbersome to check the hole output by reading every line, compared to 'viewing' (f.e. red=error, yellow=warning, green=success etc.).

Any plans for fancy colors for those read-impaired people like me?
Hey, misalf:

Damn, so sorry that I missed your post - thought I had the tapatalk notifications on!

In short, use --color or -c. Hoping you already discovered that. Cheers ~

Offline Misalf

  • Hero Member
  • *****
  • Posts: 1702
Re: extension audit script
« Reply #81 on: May 17, 2016, 07:48:57 PM »
Ah, K then. Will make an alias or something.
Thanks!
Download a copy and keep it handy: Core book ;)

Offline polikuo

  • Hero Member
  • *****
  • Posts: 714
Re: extension audit script
« Reply #82 on: December 31, 2016, 05:31:18 AM »
Happy New Year people !  :D

submitqc is a great script, a very useful tool that saves us lots of trouble.

I've been making extensions by following the instructions on wiki.
http://wiki.tinycorelinux.net/wiki:creating_extensions

Quote
sudo chown -R root:staff /tmp/package/usr/local/tce.installed
sudo chmod -R 775 /tmp/package/usr/local/tce.installed

However, every time I make an extension, I get the "wrongstartscriptperms" message.

I never take much heed since the script "fixes" the permission automatically.

Today, however, I got curious and checked the script.

Code: [Select]
tc@box:~$ head -n 619 /usr/local/bin/submitqc | tail -n 71
# if ext has a startup file, check startup name, permissions, location, etc.
checkstartup() {
 
  echo -n "${BLUE}${SCRIPT}: ${F} startup files ok? ${NORMAL}"

  if [ ! -d usr/local/tce.installed/ ]; then
    echo "${GREEN}None present. Ok, I think.${NORMAL}"
    return # no startup script
  fi

  # count files in startup script dir; get name of startup script
  SSDIR="usr/local/tce.installed/"
  SS="$(ls -A $SSDIR)"
  COUNT="$(echo "$SS" | wc -l)"

  # is there only 1 startup file, named same as extension?
  case $COUNT in
    0)  echo -e "${RED}Has startup folder but no script. Repair manually.${NORMAL}"
  echo -e "${F}: startup folder without script. Repair manually." \
>> /tmp/submitqc/startupscript
        ;;
    1)  if [ "${SS}" != "${BASENAME}" ]; then
          echo -ne "\n${YELLOW}script name ($SS) is incorrect. Renaming ${BASENAME}. ${NORMAL}"
          echo "$F startup named $SS renamed $BASENAME" >> /tmp/submitqc/wrongstartscriptname
          SS="$(find $SSDIR -not -type d | tail -n 1)"
          mv "${SS}" "${SSDIR}${BASENAME}"
          echo "${YELLOW}Done.${NORMAL}"
          CHANGES=1
        fi
        ;;
    *)  echo -e "${RED}Multiple startup files, none named $F. Repair manually.${NORMAL}"
  echo -e "${F}: multiple startup files, none named $F. Repair manually." \
>> /tmp/submitqc/startupscript
        ;;
  esac

  DIRERROR=
 
  if [ ! "$(stat -c%a ${SSDIR})" = 775 ]; then
    sudo chmod 775 "usr/local/tce.installed"
    echo -ne "\n\t${YELLOW}Startup directory permissions were corrected.${NORMAL}"
    DIRERROR=1
  fi

  if [ "$(stat -c'%U %G' ${SSDIR})" != "root staff" ]; then
    sudo chown root:staff "${SSDIR}"
    echo -ne "\n\t${YELLOW}Startup directory ownership was corrected.${NORMAL}"
    DIRERROR=1
  fi
 
  if [ ! "$(stat -c%a ${SSDIR}/$BASENAME)" = 755 ]; then
    sudo chmod 755 "${SSDIR}/$BASENAME"
    echo -ne "\n\t${YELLOW}Startup script permissions were corrected.${NORMAL}"
    DIRERROR=1
  fi

  if [ ! "$(stat -c'%U %G' ${SSDIR}/$BASENAME)" = "tc staff" ]; then
    sudo chown tc:staff "${SSDIR}/$BASENAME"
    echo -ne "\n\t${YELLOW}Startup script ownership was corrected.${NORMAL}"
    DIRERROR=1
  fi

  if [ $DIRERROR ]; then
    echo "$F" >> /tmp/submitqc/wrongstartscriptperms
    CHANGES=1
    echo
  else
    echo "${GREEN}Ok.${NORMAL}"
  fi

} # end checkstartup()

"tce.installed" --> "drwxrwxr-x root:staff"
"start-up-script" --> "-rwxr-xr-x tc:staff"

 :o

I thought their owner should both be "root:staff"
with permission "775"
while every thing else as "root:root"

I'm confused.  :-\

P.S.
I'm on x86_64 and the submitqc.tcz.md5.txt look like this:
7b157080f71142c84a5faf2cc66b3308  submitqc.tcz

Most extensions on repo have this problem
« Last Edit: December 31, 2016, 05:34:27 AM by polikuo »

Offline Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14516
Re: extension audit script
« Reply #83 on: December 31, 2016, 05:37:08 AM »
Only the /usr/local/tce.installed folder needs to be root:staff - the files in the folder can be tc:staff

Offline Misalf

  • Hero Member
  • *****
  • Posts: 1702
Re: extension audit script
« Reply #84 on: December 31, 2016, 05:58:40 AM »
The  tce.installed  directory is used for two things: To indicate which extensions are loaded and to run an extension's startup script if present.

staff  needs write permission for the  tce.installed  directory so that there can be created (empty) files for extensions that don't come with a startup script.
The startup scripts are ran as root though. So I guess the startup scripts could be  root:root  but it's not that important?

Is this actually a problem or just a discontinuity?
Download a copy and keep it handy: Core book ;)

Offline polikuo

  • Hero Member
  • *****
  • Posts: 714
Re: extension audit script
« Reply #85 on: December 31, 2016, 07:12:25 AM »
Only the /usr/local/tce.installed folder needs to be root:staff - the files in the folder can be tc:staff

The  tce.installed  directory is used for two things: To indicate which extensions are loaded and to run an extension's startup script if present.

staff  needs write permission for the  tce.installed  directory so that there can be created (empty) files for extensions that don't come with a startup script.
The startup scripts are ran as root though. So I guess the startup scripts could be  root:root  but it's not that important?

Is this actually a problem or just a discontinuity?

IMHO, since the wiki says that you should set it as "root:staff"
We should keep it consistent  :)
BTW, some users don't use "tc" as their username  ;)
(submitqc might not work if a user set his username differently, not a big deal thought)
I think it would be better to modify the script to fit the wiki's instructions

Offline Misalf

  • Hero Member
  • *****
  • Posts: 1702
Re: extension audit script
« Reply #86 on: December 31, 2016, 07:25:55 AM »
The user "tc" should be always present though. Similar to "Default" on Windows.
Anyway, I agree that "tc" probably shouldn't be used for permissions if not necessary.
Download a copy and keep it handy: Core book ;)

Offline Sashank999

  • Sr. Member
  • ****
  • Posts: 388
Re: extension audit script
« Reply #87 on: July 02, 2020, 07:34:19 AM »
Hi !

I have a tortoise like internet speed and I am tired of submitqc fetching the info.lst.gz.

I highly suggest to update the script so that it uses recently fetched info.lst.gz and also check the remote copy's zsync file so that it only downloads the blocks that are different. I also suggest md5 checking for both local and remote info.lst.gz.

Also when checking the extension, it shows :
        Copyright notice not found at usr/local/share/doc/ClipIt
        Confirm that this is not needed.
        Checking repo for copyright statement in ClipIt.tcz ...
        ClipIt.tcz not found on server.

Hence I request to also add --no-copyright or --copyright= options so that the script identifies the copyright file.
« Last Edit: July 02, 2020, 07:38:15 AM by Sashank999 »

Offline Sashank999

  • Sr. Member
  • ****
  • Posts: 388
Re: extension audit script
« Reply #88 on: July 04, 2020, 09:56:30 PM »
Hi !

I think that it is better for the script to update the size field in .info file after stripping of binaries, if the binaries are unstripped.

aus9

  • Guest
Re: extension audit script
« Reply #89 on: July 04, 2020, 10:15:28 PM »
Hi Sashank999
if I run
Code: [Select]
submitqc --libsand have no size in the info files....submitqc will add them.

I prefer to have my build script "strip" where ever  possible rather than use the switch supplied by submitqc.
eg as per wiki we have....multiple mentions of strip
http://wiki.tinycorelinux.net/doku.php?id=wiki:creating_extensions.html

but I prefer ...I build to /tmp
Code: [Select]
P=TCE-name
SRC=some source name
cd /tmp
# wget  command to get source
# unpack
mkdir $P
cd $SRC
# compile commands
make install-strip DESTDIR=/tmp/$P
cd /tmp

I know you have already read Rich script
http://forum.tinycorelinux.net/index.php?action=dlattach;topic=23844.0;attach=5365
« Last Edit: October 27, 2022, 08:10:02 PM by Rich »