few service questions

[dicorer]

I would like to fully understand the whole sce operational logic and structure.

Please correct me if any assertion is wrong.
So, we have lots of deb packages, let's take for example one, "fluxbox".
I understood so far that into sce file is created \usr\local\sce\fluxbox\ which contains 2 files:
fluxbox-data.md5sum and fluxbox.md5sum
The file fluxbox.md5sum is clear, contains all deb wget from the ubuntu repo.
The file fluxbox-data.md5sum contains: fluxbox-data and fluxbox-deb2sce
Questions:

1) The fluxbox-data, I guess, is that file I see into import/debs called fluxbox-data.tar.gz, correct?
What is special about that file?
what is the exact url from where it was dl?

2) The fluxbox-deb2sce, what is it? where is?
Was it dl too? if yes, what is the exact dl url?

3) is it possible that some package could not have been wget at all from the repo, but fully dl ONLY from TinyCore repo (in deb or other format)?
if yes, where can I find it?
is it listed somehow into \usr\local\sce\<package>?

4) Could you please point me exactly where, into sce-import tools, is done any dl from TinyCore repo?

Thanks

[dicorer]

problems?
I need to investigate few misbehaviors (bugs?) I experienced, but with my very limited free time, I'm unable to perform any comprehensive reverse engineering issues tracing.
For that reason I'm asking those questions, so that I can place my debugging traps in the right places, and so on.

[jls]

Hi dCorer
sorry for the short answer due to limited time
http://tinycorelinux.net/dCore/import/
http://tinycorelinux.net/dCore/x86/import/

[dicorer]

OMG!
there are tar.gz in both /x86/import/ and /import/ !

Thanks for your answer. Ok, modify questions:

1) where is done inside sce-import&tools the dl from /import/ ?
2) where is done inside sce-import&tools the dl from /x86/import/ ?
3) in the case that, one package is available in Ubuntu repo: is it possible (allowed by sce-import specifications) that such package was dl from those 2 Tinycore repos ONLY (nothing from Ubuntu repo)?

[Jason W]

The dCore/import directory contains files that are not arch specific, that work for both x86 and arm.  Data files, startup scripts, etc.

The dCore/x86/import is x86 specific files, namely prebuilt packages.

Things in dCore/x86/import are used in priority over those in the Debian repo, allows us to host a package with the same name as a Debian one in the case the Debian one will not work with dCore.

[dicorer]

Hi, just now I was looking at those scripts, thanks for the clarification.
btw, noticing few glitches, eg. PREBUILTMIRROR and/or IMPORTMIRROR result unused in some scripts, or script searchprebuilt result orphan, or TCEDIR/SCE/SCEDIR and those lines at beginning of scripts are duplicated, they could be regrouped in tc-functions... if feeling like cleaning a bit

GULP!
meaning all those in dCore/x86/import are reported verified not working (for dCore) ubuntu debs?

[Jason W]

Hi,

The packages in dCore/x86/import are normally things that don't exist in Debian/Ubuntu.  The goal is to always use the Debian package, but if  needed we just build our own.

I will look at the scripts as far as tidying up.  Now that dCore is pretty mature and not rapidly changing like it was for a long time.

[dicorer]

I just went by alphabetical order, tried "aterm".
Package aterm exists both in ubuntu repo and in dCore/x86/import/ repo.
First, I dl the dCore aterm, the one which comes by sce-import, runs ok.
Then I dl the ubuntu aterm manually, placed the exec in dCore, runs ok too.

So, why dl a dCore aterm, while also the ubuntu aterm works the same?

Another question:
Since PREBUILTMIRROR/IMPORTMIRROR exist in many scripts, could you please point me where prioritization is done?
Something like, where is such clause, similar as "if [ $PAK exists dCore/x86/import/  ]  wget dCore/x86/import/$PAK.tar.gz else wget ubuntu/$PAK.deb " ?

[Jason W]

Aterm was one of the first packages, done when dCore was new and only for Wheezy.

And it is if the package name exists in PKGPREBUILTDEP, then the prebuilt package is used.

[dicorer]

Hi, thanks for the hint.
This we I'll check more custom packages, to match them with the "pure" ubuntu ones.
So I guess I'll try add an new option to sce-import (smthg like ubuntu-only) to ease this task.

Fyi, I already can confirm, the dCore packages containing dCore-tinyCore kernel modules do not work in my setup (obviously)
Please standby for more questions...

[dicorer]

ub-dCore-utopic.gz
/bin/lsmmod -> ../bin/busybox

[dicorer]

debGetDeps:

Code: [Select]

elif grep "^$PKG:" /tmp/PKGPREBUILTDEP > /dev/null 2>&1; then
DEPLIST="$DEPLIST `grep "^$PKG:" /tmp/PKGPREBUILTDEP | cut -f2 -d:`"
elseDEPLIST is appended with its previous value?
if previous DEPLIST value is dirty, dont we risk to load wrong PKGs?

also

2 same greps are done here, since getDeps is recursive, might be better optimized for speed?
e.g. load the first grep result in a variable, and use it down in replace of the second grep?

=================
sce-import , the -l option.
if given a wrong list file name, it drops down the cp usage... might appropriate to trap this exception?

[Jason W]

To ensure that DEPLIST is not dirty at the invokation of debGetDeps, I reckon I could unset it before the function, but it is working now and I have not seen any cases of corruption.

Thanks for your insight.  But I spent many, many afternoons in trial, error, and testing polishing and tweaking the dependency gathering routine that was the fastest yet has to be accurate.  It is documented in the threads here. 

To get the dependency list of vlc takes 5.01 seconds to gather 135 package names.   That is with parsing PKGPREBUILTDEP, the extra debinx files, and the main one in that order until found.  I am satisfied with the current speed and do not wish to break things in an effort to further streamline.

But if you have a code suggestion, I will listen since we are not messing with the dep gathering from the debinx files but just from the prebuilt list.

[dicorer]

I remember this issue I reported earlier here:
http://forum.tinycorelinux.net/index.php/topic,18263.msg111030.html#msg111030
"coreutils" and "crda" were imported erroneously, it might be that?
I also agree unsetting the value before could solve the case, anyway I didn't try.

About speed, I do all under virtual machine, that's why I can feel the weight of every instruction there.
I didn't perform any non-functional test of course, because in fact I thought you must have done that already, so just pointed out some options I got in mind.
Clearly, if you already did all tests cases possible, small word, no point in checking that again.

As I said, my efforts were in fact to be focused in disabling (not improving) the prebuilt packs and bed2sce custom scripts access.
I successfully accomplish that, which gave me much higher granularity of options when selecting packages.

I'm happy that my test results are confirming my expectations.
Afaik, both rebuilt packs and deb2sce scripts can be disabled safety, in the case the application works well already without.
In case it doesn't work, it seemed to me, its always matter of a small missing detail, and a look to the deb2sce script helped in finding that detail.
I had the feeling that some scripts and pack actually in use, contain outdated procedures when applied to ubuntu pack, but I might be wrong.

Anyway, something to really keep in mind, came out of my tests: dCore busybox is bugged
And unfortunately the few versions at ubuntu repo, even more bugged! 
Apart depmod problem, the cp likes to messup, when dealing with symlink, meaning links get overwritten and targets get lost.
That happen mostly when loading in different sce, packages belonging to the same functionality, e.g. libs in one sce, the dev in another sce... but not only.
Also that, I didn't investigate further, clearly I'm not going to fix busybox bugs, simply I went by workarounds: pay attention to where put similar debs and don't duplicate them in other sce.

Then, I might have other technical matters to report, but I don't know if you are interested.

[Jason W]

coreutils does not get imported erroneously, it replaces the busybox commands of the same name, it is set up like that by design and has been by Tinycore since the beginning.

I have not seen an issue with DEPLIST being corrupt, if you find an instance please report the specifics.

To grep PKGPREBUILTDEP for the phrase "file", it takes about 2 seconds to grep it 1000 times.  About 8 seconds when double grepping, and about 5 seconds when exporting the first grep to a variable and then testing that variable to be not null.  So a little time can be saved, but not much.  Grepping that file once results in a 0.00 time usage.

There could be some use for disabling the prebuilt section, but I have never heard that requested.  And there is no reason or use for disabling the startup scripts as they exist for one thing - to make the package work as expected.  And the startup scripts are designed to work across Debian and Ubuntu, and across package versions.  Portability is a main goal of dCore.

As for buggy busybox or Debian/Ubuntu builds, the place to report them would be those upstream projects.  But for Debian or Ubuntu, first install that system, test the package, and then report.  If a package works on Debian but not dCore, then it is a dCore issue and we can deal with it here.  Specific commands, details, and results are most helpful.

I am interested in issues or bug reports, but much of what you are reporting is dealing with your own custom use of dCore.