Proposal: new file dep.db.gz on tc server, having all *.tcz.dep inside

[Rich]

Hi GNUser

...
3. Running any depends-on.sh foo or depends-on.sh foo.tcz
- Expected matches: Should not include foo.tcz because it seems silly to say that an extension depends on itself
- Actual matches always include foo.tcz ...

The  .db  files contain  Records  like this:

Code: [Select]

aalib-dev.tcz
aalib.tcz
libX11-dev.tcz

A  Record  consists of  Fields , 4 in this case. Field1 is the extension , Field2  and
Field3  are the dependencies, and  Field4  is a blank line which marks the end of
a  Record.

In awk, $0 is the entire  Record.  $1 is Field1, $2 is Field2, etc. The search is
performed on $0. For extensions which have no dependencies, $0 and $1 are
the same.

... I attached a little patch that fixes these issues.

Code: [Select]

awk 'BEGIN {FS="\n";RS=""} /\n'${TARGET_WITH_EXTENSION}'/{print $1}' "$TCEDIR"/"$DB" | grep -v "^${TARGET_WITH_EXTENSION}"This will filter out  avahi-ui-gtk3, gtk3-dev, gtk3-gir, vte-gtk3-dev, wxwidgetsgtk3-dev, etc
from the results when you do a fuzzy search for  gtk3.

This should handle that correctly:

Code: [Select]

awk 'BEGIN {FS="\n";RS=""} /\n'${TARGET_WITH_EXTENSION}'/{if($1 != $0) print $1}' "$TCEDIR"/"$DB"
Nice trick placing  \n  in front of  TARGET.  I'll have to make a note of that. If I recall
correctly, the internet is littered with statements that line breaks are ignored when
doing a search of $0.

[GNUser]

Code: [Select]

awk 'BEGIN {FS="\n";RS=""} /\n'${TARGET_WITH_EXTENSION}'/{print $1}' "$TCEDIR"/"$DB" | grep -v "^${TARGET_WITH_EXTENSION}"This will filter out  avahi-ui-gtk3, gtk3-dev, gtk3-gir, vte-gtk3-dev, wxwidgetsgtk3-dev, etc
from the results when you do a fuzzy search for  gtk3.

Hi Rich. No worries, that line of code is only for exact searches.

I tested the current GitHub version of the script quite thoroughly. I think you'll find that it handles fuzzy and exact searches exactly as expected. Give it a try:
https://github.com/tinycorelinux/Core-scripts/blob/master/usr/bin/depends-on.sh

If you get any unexpected results with the current version at the link, I will buy you a beer.

[Rich]

Hi GNUser
Do a fuzzy search for  acpid  or  acpid.tcz  and one of the responses is  acpid.tcz.
acpid.tcz  has no dependencies, case 3 in your reply #7, so it should be filtered out.

Fixing it in the awk command lets it skip the print command and avoids calling grep.

Since you seem to have better access, could you fix my comment (line #35) in the script:

# This downloads a fresh copy of dep.dbgz if any of the following are true:

There's a period missing ( dep.db.gz ).

[GNUser]

I thought case 3 in my reply #7 was permissible for fuzzy searches.
But I do find it annoying even for fuzzy searches.
I will try to fix case 3 for both exact and fuzzy searches.

I will fix line #35 for you at the same time.

[nick65go]

Hi GNUser and Rich. May I have an additional proposal? This is to have a parameter of how far UP in hierarchy we want to go/see.
ex: who ask for  libxcb.tcz? Few answers, one is  libX11.tcz. OK, but what about  libXrender.tcz (up one level) ?

Code: [Select]

      libXrender.tcz
         libX11.tcz
            libxcb.tcz
               libXau.tcz
               libXdmcp.tcz
The main idea was to see if I replace/ destroy a tcz (because I miss its deps, and it can not load, etc), who is affected.
Maybe these observations will crystalize the awk logic.

[GNUser]

Hi nick65go. The script is not that fancy. It only searches the .dep files, not the dependency tree.

[GNUser]

In awk, $0 is the entire  Record.  $1 is Field1, $2 is Field2, etc. The search is
performed on $0. For extensions which have no dependencies, $0 and $1 are
the same.

I see. In that case, I think what we really want is to limit the search to fields $2 through last. I need to chew on this a bit.

[nick65go]

One scenario: for me, it all started with the choose (for example) between gtk2 versus gtk3.
I wanted to know which apps need gtk2. Also I wanted to know which apps need gtk3.
Then I may choose the same apps which uses gtk2 (from a previous TC version) if it is the same version in gtk3.
So I need to evaluate what drags more tcz deps in RAM, just because gtk3, without any improvement /correction.

ex: If you look with apps-GUI it lists a lot of dep for a tiny tcz, and you think the RAM need is big, but is not because other tcz (Xorg, gtk3) were already in RAM.
So I compare wrong that  xpopple-xpdf.tcz [164K] needs

Code: [Select]

  Total size (bytes)                         17895424,  17.07 MBversus flaxpdf.tcz [216 KB] which needs

Code: [Select]

Total size (bytes)                         17158144,  16.36 MBis not 164KB vs. 216 KB [aka 52 KB gzip-ed] is more like 17.07 MB vs. 16.36 MB [aka 0.71 MB x 1024 = 724 KB];Above listed values are just for demo.

And here comes the option for a "level -deep", when further UP, all deps are common.

[Rich]

Hi GNUser
I've been down this road. $0 is the fastest way to search for something. To the
best of my knowledge, once you try breaking out individual fields, you need loop
counters and things slow way down.

If you want to do a search for extensions with no dependencies:

Code: [Select]

awk 'BEGIN {FS="\n";RS=""} NF==1 {print $1}' "$TCEDIR"/"$DB"

[GNUser]

Do a fuzzy search for  acpid  or  acpid.tcz  and one of the responses is  acpid.tcz.
acpid.tcz  has no dependencies.

I think what you dislike about this response is not that acpid.tcz has no dependencies. It's that acpid.tcz is not one of acpid.tcz's dependencies, right?

Put it another way, if acpid.tcz did have some other random dependency, you would still not want it to be a response.

I'm not sure how to solve this other than limiting the search to the fields that actually represent dependencies (i.e., fields $2 through last). I think it may be best to just leave it alone--i.e., live with the fact that fuzzy search looks at entire record and may match on $1.

[Rich]

Hi GNUser
Changing the  print  command to this will filter out matches on extensions
with no dependencies:

Code: [Select]

{if($1 != $0) print $1}

[Rich]

Hi GNUser
This should work too and might be faster:

Code: [Select]

{if(NF > 1) print $1}

[GNUser]

Hi Rich. Yes, either one of those would filter out extensions without dependencies. But it would still not solve many cases of the third bug I mentioned in Reply #7.

With either change you're proposing, if acpid.tcz depended on any random extension unrelated to your search (e.g., yad.tcz) and you did a fuzzy search for  acpid  or  acpid.tcz  , one of the responses would still be acpid.tcz.

IMHO either change would add overhead and only seem to work sometimes. I don't think there is an easy fix for that little bug in fuzzy searches (short of ensuring that the matched field is >= $2, which would probably involve looping through fields).

Edit: My vote is to leave the fuzzy search alone. But it's your child, so feel free to change it if you like.

[Rich]

Hi GNUser

... if acpid.tcz depended on any random extension unrelated to your search (e.g., yad.tcz) and you did a fuzzy search for  acpid  or  acpid.tcz  , one of the responses would still be acpid.tcz. ...

How could  "acpid.tcz depended on any random extension"  if it has no dependencies? 

[GNUser]

I made your change to the attached script. Try it.

Code: [Select]

$ ./depends-on.sh -f acpid
No responses, so fuzzy search seems to be matching only on dependencies.

Code: [Select]

$ ./depends-on.sh -f yad
wifi-manager.tcz
yad.tczOops. yad.tcz does not actually depend on any extension containing yad in the name.

This is what I mean. Your change only fixes the unexpected results sometimes, so it's not really getting to the actual problem.