Tiny Core Linux
dCore Import Debian Packages to Mountable SCE extensions => dCore X86 => Topic started by: dicorer on April 21, 2015, 07:01:55 AM
-
I would like to fully understand the whole sce operational logic and structure.
Please correct me if any assertion is wrong.
So, we have lots of deb packages, let's take for example one, "fluxbox".
I understood so far that into sce file is created \usr\local\sce\fluxbox\ which contains 2 files:
fluxbox-data.md5sum and fluxbox.md5sum
The file fluxbox.md5sum is clear, contains all deb wget from the ubuntu repo.
The file fluxbox-data.md5sum contains: fluxbox-data and fluxbox-deb2sce
Questions:
1) The fluxbox-data, I guess, is that file I see into import/debs called fluxbox-data.tar.gz, correct?
What is special about that file?
what is the exact url from where it was dl?
2) The fluxbox-deb2sce, what is it? where is?
Was it dl too? if yes, what is the exact dl url?
3) is it possible that some package could not have been wget at all from the repo, but fully dl ONLY from TinyCore repo (in deb or other format)?
if yes, where can I find it?
is it listed somehow into \usr\local\sce\<package>?
4) Could you please point me exactly where, into sce-import tools, is done any dl from TinyCore repo?
Thanks
-
problems? :)
I need to investigate few misbehaviors (bugs?) I experienced, but with my very limited free time, I'm unable to perform any comprehensive reverse engineering issues tracing.
For that reason I'm asking those questions, so that I can place my debugging traps in the right places, and so on.
-
Hi dCorer
sorry for the short answer due to limited time
http://tinycorelinux.net/dCore/import/ (http://tinycorelinux.net/dCore/import/)
http://tinycorelinux.net/dCore/x86/import/ (http://tinycorelinux.net/dCore/x86/import/)
-
OMG!
there are tar.gz in both /x86/import/ and /import/ !
Thanks for your answer. Ok, modify questions:
1) where is done inside sce-import&tools the dl from /import/ ?
2) where is done inside sce-import&tools the dl from /x86/import/ ?
3) in the case that, one package is available in Ubuntu repo: is it possible (allowed by sce-import specifications) that such package was dl from those 2 Tinycore repos ONLY (nothing from Ubuntu repo)?
-
The dCore/import directory contains files that are not arch specific, that work for both x86 and arm. Data files, startup scripts, etc.
The dCore/x86/import is x86 specific files, namely prebuilt packages.
Things in dCore/x86/import are used in priority over those in the Debian repo, allows us to host a package with the same name as a Debian one in the case the Debian one will not work with dCore.
-
Hi, just now I was looking at those scripts, thanks for the clarification.
btw, noticing few glitches, eg. PREBUILTMIRROR and/or IMPORTMIRROR result unused in some scripts, or script searchprebuilt result orphan, or TCEDIR/SCE/SCEDIR and those lines at beginning of scripts are duplicated, they could be regrouped in tc-functions... if feeling like cleaning a bit ;)
GULP!
meaning all those in dCore/x86/import are reported verified not working (for dCore) ubuntu debs?
-
Hi,
The packages in dCore/x86/import are normally things that don't exist in Debian/Ubuntu. The goal is to always use the Debian package, but if needed we just build our own.
I will look at the scripts as far as tidying up. Now that dCore is pretty mature and not rapidly changing like it was for a long time.
-
I just went by alphabetical order, tried "aterm".
Package aterm exists both in ubuntu repo and in dCore/x86/import/ repo.
First, I dl the dCore aterm, the one which comes by sce-import, runs ok.
Then I dl the ubuntu aterm manually, placed the exec in dCore, runs ok too.
So, why dl a dCore aterm, while also the ubuntu aterm works the same?
Another question:
Since PREBUILTMIRROR/IMPORTMIRROR exist in many scripts, could you please point me where prioritization is done?
Something like, where is such clause, similar as "if [ $PAK exists dCore/x86/import/ ] wget dCore/x86/import/$PAK.tar.gz else wget ubuntu/$PAK.deb " ?
-
Aterm was one of the first packages, done when dCore was new and only for Wheezy.
And it is if the package name exists in PKGPREBUILTDEP, then the prebuilt package is used.
-
Hi, thanks for the hint.
This we I'll check more custom packages, to match them with the "pure" ubuntu ones.
So I guess I'll try add an new option to sce-import (smthg like ubuntu-only) to ease this task.
Fyi, I already can confirm, the dCore packages containing dCore-tinyCore kernel modules do not work in my setup (obviously)
Please standby for more questions... ;)
-
ub-dCore-utopic.gz
/bin/lsmmod -> ../bin/busybox
-
debGetDeps:
elif grep "^$PKG:" /tmp/PKGPREBUILTDEP > /dev/null 2>&1; then
DEPLIST="$DEPLIST `grep "^$PKG:" /tmp/PKGPREBUILTDEP | cut -f2 -d:`"
elseDEPLIST is appended with its previous value?
if previous DEPLIST value is dirty, dont we risk to load wrong PKGs?
also
2 same greps are done here, since getDeps is recursive, might be better optimized for speed?
e.g. load the first grep result in a variable, and use it down in replace of the second grep?
=================
sce-import , the -l option.
if given a wrong list file name, it drops down the cp usage... might appropriate to trap this exception?
-
To ensure that DEPLIST is not dirty at the invokation of debGetDeps, I reckon I could unset it before the function, but it is working now and I have not seen any cases of corruption.
Thanks for your insight. But I spent many, many afternoons in trial, error, and testing polishing and tweaking the dependency gathering routine that was the fastest yet has to be accurate. It is documented in the threads here.
To get the dependency list of vlc takes 5.01 seconds to gather 135 package names. That is with parsing PKGPREBUILTDEP, the extra debinx files, and the main one in that order until found. I am satisfied with the current speed and do not wish to break things in an effort to further streamline.
But if you have a code suggestion, I will listen since we are not messing with the dep gathering from the debinx files but just from the prebuilt list.
-
I remember this issue I reported earlier here:
http://forum.tinycorelinux.net/index.php/topic,18263.msg111030.html#msg111030
"coreutils" and "crda" were imported erroneously, it might be that?
I also agree unsetting the value before could solve the case, anyway I didn't try.
About speed, I do all under virtual machine, that's why I can feel the weight of every instruction there.
I didn't perform any non-functional test of course, because in fact I thought you must have done that already, so just pointed out some options I got in mind.
Clearly, if you already did all tests cases possible, small word, no point in checking that again.
As I said, my efforts were in fact to be focused in disabling (not improving) the prebuilt packs and bed2sce custom scripts access.
I successfully accomplish that, which gave me much higher granularity of options when selecting packages.
I'm happy that my test results are confirming my expectations.
Afaik, both rebuilt packs and deb2sce scripts can be disabled safety, in the case the application works well already without.
In case it doesn't work, it seemed to me, its always matter of a small missing detail, and a look to the deb2sce script helped in finding that detail.
I had the feeling that some scripts and pack actually in use, contain outdated procedures when applied to ubuntu pack, but I might be wrong.
Anyway, something to really keep in mind, came out of my tests: dCore busybox is bugged ;)
And unfortunately the few versions at ubuntu repo, even more bugged! ;D
Apart depmod problem, the cp likes to messup, when dealing with symlink, meaning links get overwritten and targets get lost.
That happen mostly when loading in different sce, packages belonging to the same functionality, e.g. libs in one sce, the dev in another sce... but not only.
Also that, I didn't investigate further, clearly I'm not going to fix busybox bugs, simply I went by workarounds: pay attention to where put similar debs and don't duplicate them in other sce.
Then, I might have other technical matters to report, but I don't know if you are interested.
-
coreutils does not get imported erroneously, it replaces the busybox commands of the same name, it is set up like that by design and has been by Tinycore since the beginning.
I have not seen an issue with DEPLIST being corrupt, if you find an instance please report the specifics.
To grep PKGPREBUILTDEP for the phrase "file", it takes about 2 seconds to grep it 1000 times. About 8 seconds when double grepping, and about 5 seconds when exporting the first grep to a variable and then testing that variable to be not null. So a little time can be saved, but not much. Grepping that file once results in a 0.00 time usage.
There could be some use for disabling the prebuilt section, but I have never heard that requested. And there is no reason or use for disabling the startup scripts as they exist for one thing - to make the package work as expected. And the startup scripts are designed to work across Debian and Ubuntu, and across package versions. Portability is a main goal of dCore.
As for buggy busybox or Debian/Ubuntu builds, the place to report them would be those upstream projects. But for Debian or Ubuntu, first install that system, test the package, and then report. If a package works on Debian but not dCore, then it is a dCore issue and we can deal with it here. Specific commands, details, and results are most helpful.
I am interested in issues or bug reports, but much of what you are reporting is dealing with your own custom use of dCore.
-
Hi,
And the startup scripts are designed to work across Debian and Ubuntu, and across package versions. Portability is a main goal of dCore.
okkey, that was a new dCore feature to me unknown, till now :)
Can you please explain, how to setup sce-import, to use DEBIAN Index file and to dl packs from DEBIAN repo (not the usual Ubuntu)?
Than, again from getDeps:
if grep "^Package: $PKG$" "$TCEDIR"/debinx.* > /dev/null 2>&1 ; then
DEPLIST=`debGetPkgList $PKG 2>/dev/null`
[ -s /tmp/PKGEXTRAREPODEP ] && DEPLIST="$DEPLIST `grep "^$PKG:" /tmp/PKGEXTRAREPODEP | cut -f2 -d:`"
Can you please explain, what are debinx.* files, and what is PKGEXTRAREPODEP?
Thanks
-
For the Debian debinx file, you mean to use a Debian Packages file on ub-dCore-trusty or ub-dCore-utopic? I would not recommend it, but the way would be to change the files in /usr/share/doc/tc, that is where the repo and distro info is kept.
The debinx.* files are the Packages files from extra repos specified in /opt/debextra. PKGEXTRAREPODEP is just that, added dependencies for those packages in extra repos that need deps specified to work.
-
you mean to use a Debian Packages file on ub-dCore-trusty or ub-dCore-utopic?
I guess, I mean yes, setup sce-import to load the default index DEBIAN file containing all packs dependencies, and dl a DEBIAN pack... is that possible?
I would not recommend it
Why not?
Smtng nasty with DEBIAN architecture?
but the way would be to change the files in /usr/share/doc/tc, that is where the repo and distro info is kept.
In my utopic I have only repo and distro, where is the repo url?
The debinx.* files are the Packages files from extra repos specified in /opt/debextra. PKGEXTRAREPODEP is just that, added dependencies for those packages in extra repos that need deps specified to work.
I don't have that debextra, I guess I have to create it optionally.
So debinx.* are packages files, deb files?
What is an "extra repo"?
Thanks
-
Hi dicorer,
A lot of this concepts are in different post that you probably have to read. I recommend you to use the Search function. ;)
I'm not an expert programmer (better to say I'm an amateur :( ) but I'm using ub-dCore and dCore from the beginning with a lot of satisfaction, using "extra repos" and so on, and of course have read all the concerning about this two distributions. :)
luck with it!
-
Hi,
please could you be so nice to share your dCore debinx.* files, /opt/debextra and PKGEXTRAREPODEP contents, so I can understand what is that feature about?
Thanks
-
debinx.* are autogenerated
jls@box:~$ ls /opt/debextra
01-libreoffice multiverse security-restricted updates-multiverse
05-niko2040 pidgin-developers security-universe updates-restricted
bodhi restictred ubuntuzilla updates-universe
canonical-partner security-main universe whatsapp-purple
googletalk security-multiverse updates-main
jls@box:~$ cat /opt/debextra/01-libreoffice
http://ppa.launchpad.net/libreoffice/ppa/ubuntu trusty main
-
right! thank you :)
so, lets say, just for example, I want to sce-import Opera browser, the deb file is exactly here:
http://get.geo.opera.com/pub/opera/linux/1216/opera_12.16.1860_i386.deb
How do I set up /opt/debextra/<what?>, debinx.*<what?>, and PKGEXTRAREPODEP ?
-
U need a repo for debian
-
forget debinx.* and PKGEXTRAREPODEP
debinx.* are autogenerated
in my case debextra:
tc@ub-dC-tr-aao:~$ ls /opt/debextra
repo1 repo2 repo3 repo7
tc@ub-dC-tr-aao:~$ cat /opt/debextra/repo1
http://us.archive.ubuntu.com/ubuntu trusty multiverse
tc@ub-dC-tr-aao:~$ cat /opt/debextra/repo7
http://download.virtualbox.org/virtualbox/debian trusty contrib
are simple files with a link to ppa or repository
-
sorry I dont understand.
Where can I find, or how I create a repo for debian, so I can sce-import http://get.geo.opera.com/pub/opera/linux/1216/opera_12.16.1860_i386.deb?
If I have to create a new file , can you create it for me, just for example, pls?
-
just create a directory in /opt with the name debextra ---> /opt/debextra
then create a file with the name you want in /opt/debextra, with your favorite editor or with vi:
vi repoopera
paste in this empty file the link to the repo:
http://get.geo.opera.com/pub/opera/linux/1216/
save it
done!
When you execute sce-import it will look to this repo for the program you are searching for, in this case for "Opera"
(better when the repo is specifically for the distribution version (trusty, utopic, etc) because the program will run better. See for example:
http://www.ubuntuupdates.org/ppa/opera
(http://www.ubuntuupdates.org/ppa/opera)
for ubuntu versions.
In this case paste in /opt/debextra/repoopera:
http://deb.opera.com/opera/ stable non-free
-
paste in this empty file the link to the repo:
http://get.geo.opera.com/pub/opera/linux/1216/
save it
done!
thanks but, I don't know at you, here it doesn't work ;)
even hacking /usr/share/doc/tc/repo.txt with a null line, it aborts warning repo unavailable.
I suspect this feature has not been implemented in dCore still, right?
It's implemented only dl from strictly debian-structured repos, not from "wild" ones, correct?
for ubuntu versions.
In this case paste in /opt/debextra/repoopera:
http://deb.opera.com/opera/ stable non-free
sorry, bad luck also with that...
libgstreamer-plugins 100% |*******************************| 438k 0:00:00 ETA
Merging libgstreamer-plugins-base0.10-0
cat: can't open '/opt/debextra/Package': No such file or directory
/usr/bin/deb2sce: line 432: can't open Package: no such file
wget: not an http or ftp url: /pool/non-free/o/opera/opera_12.16.1860_i386.deb
failed on download of pool/non-free/o/opera/opera_12.16.1860_i386.deb
wget: not an http or ftp url: /pool/non-free/o/opera
ar: can't open '/etc/sysconfig/tcedir/import/debs/opera_12.16.1860_i386.deb': No such file or directory
ar: can't open '/etc/sysconfig/tcedir/import/debs/opera_12.16.1860_i386.deb': No such file or directory
/usr/bin/debExtract: line 42: can't create /tmp/: Is a directory
Failure to extract opera_12.16.1860_i386.deb, exiting..
-
even hacking /usr/share/doc/tc/repo.txt with a null line, it aborts warning repo unavailable.
? ? ? ? ? ? ? ?
another thing:
cat: can't open '/opt/debextra/Package': No such file or directory
/usr/bin/deb2sce: line 432: can't open Package: no such file
I can't understand this. You have not created any new file Package with the link on it?
Well, from the beginning:
I have created a file /opt/debextra/repo9 with:
http://deb.opera.com/opera/ stable non-free
then in an xterm window execute sce-import opera:
tc@ub-dC-tr-aao:~$ sce-import opera
Using the -u option.
ubuntu
#################### 100.0% 0.0 kBps DONE
verifying download...checksum matches OK
used 11134976 local, fetched 0
Using the repo http://us.archive.ubuntu.com/ubuntu trusty multiverse
Using the repo http://archive.ubuntu.com/ubuntu trusty universe
Using the repo http://archive.canonical.com/ precise partner
Using the repo http://download.virtualbox.org/virtualbox/debian trusty contrib
Using the repo http://deb.opera.com/opera/ stable non-free
Using the repo http://us.archive.ubuntu.com/ubuntu ubuntu trusty main
Do you want to import opera? (y/N):
No errors and the repo is there, you can see it in the line "Using the repo http://deb.opera.com/opera/ stable non-free", so I can import now "opera" using this repo. The other repos are my other files in /opt/debextra (repo1, repo2, etc.).
Where is the problem? No problem here. Easy!
-
another thing:
cat: can't open '/opt/debextra/Package': No such file or directory
/usr/bin/deb2sce: line 432: can't open Package: no such file
I can't understand this. You have not created any new file Package with the link on it?
you tell me ;)
Well, from the beginning:
I have created a file /opt/debextra/repo9 with:
http://deb.opera.com/opera/ stable non-free
then in an xterm window execute sce-import opera:
tc@ub-dC-tr-aao:~$ sce-import opera
Using the -u option.
ubuntu
#################### 100.0% 0.0 kBps DONE
verifying download...checksum matches OK
used 11134976 local, fetched 0
Using the repo http://us.archive.ubuntu.com/ubuntu trusty multiverse
Using the repo http://archive.ubuntu.com/ubuntu trusty universe
Using the repo http://archive.canonical.com/ precise partner
Using the repo http://download.virtualbox.org/virtualbox/debian trusty contrib
Using the repo http://deb.opera.com/opera/ stable non-free
Using the repo http://us.archive.ubuntu.com/ubuntu ubuntu trusty main
Do you want to import opera? (y/N):
No errors and the repo is there, you can see it in the line "Using the repo http://deb.opera.com/opera/ stable non-free", so I can import now "opera" using this repo. The other repos are my other files in /opt/debextra (repo1, repo2, etc.).
Where is the problem? No problem here. Easy!
mh-mh, and what happen if you type "y" to that prompt?
-
Yes, you are right with the error, but I think it's because Opera is not from a standard ubuntu repo and for this reason perhaps need a script to adapt to dCore o precompiled version for dCore
I have made the prove to download it and then execute deb2sce with this results:
tc@ub-dC-tr-ordjd:/mnt/sda4/ubtdC$ sudo deb2sce opera_12.16.1860_i386.deb
Using Package Index: /etc/sysconfig/tcedir/ubuntu_trusty_main_i386_Packages
Using ubuntu Mirror: http://us.archive.ubuntu.com/ubuntu
opera_12.16.1860_i386.deb is not a standard ubuntu, dCore precompiled, or meta package, exiting..
I have googled a little and found that to install opera in ubuntu you have to download the standard version, install it manually and then the installation point to a repo for updates
Read this links:
http://tutorialforlinux.com/2014/03/30/how-to-install-opera-for-ubuntu-14-04-trusty-lts-3264bit-linux-easy-visual-guide/
(http://tutorialforlinux.com/2014/03/30/how-to-install-opera-for-ubuntu-14-04-trusty-lts-3264bit-linux-easy-visual-guide/)
http://www.linuxeasyway.com/2014/08/ubuntu-1404-lts-trusty-tahr-opera-web.html
(http://www.linuxeasyway.com/2014/08/ubuntu-1404-lts-trusty-tahr-opera-web.html)
After the Manual Download and Installation of the Opera Ubuntu .deb Package the Opera Ubuntu Repository is Automatically Added to Facilitate the Browser Updating Task!
So I think the best way to import opera for dCore is to demand a prebuilt package for dCore and test it.
-
well, thanks for the "Yes, you are right" ... most of the people I work with, in the best case they would turn around and say "sorry Im busy now" ;)
-
Thanks JLS and Onyarian for your help.
dicorer - as has been said, forum searches would reveal much of this stuff. And my advice would be to then use dCore in the standard recommended manner, or expect breakage.
As for opera, there is a repo for it, I just imported it:
http://forum.tinycorelinux.net/index.php/topic,17807.0.html
-
I might wrong, but most of the questions Im asking, are not available by forum search, and I see even the senior members dont have very clear control of the tool.
But thats not a problem.
Latest issue, it seems me and Onyarian just used dCore in the standard & recommended manner, nevertheless it broke.
But also thats not a problem of course.
Instead, I simply would like to kindly ask you, if you could please answer my previous questions, which got lost:
you mean to use a Debian Packages file on ub-dCore-trusty or ub-dCore-utopic?
I guess, I mean yes, setup sce-import to load the default index DEBIAN file containing all packs dependencies, and dl a DEBIAN pack... is that possible (implemented feature)?
I would not recommend it
Why not?
Smtng nasty with DEBIAN architecture? or what is it?
Thanks
-
Debian Wheezy packages are made to work with Wheezy, in terms of library versions and other compatibility. Same with each Debian and Ubuntu release.
Using packages from Jessie on Wheezy are going to cause problems if they even work at all. Nothing wrong with the Debian package pool, just needs to stay with the right release. If you want Debian Jessie packages, simply use dCore-jessie. You can change the values in /usr/share/doc/tc, but do so at your own risk as things will break.
I imported Opera, and the import session and package worked. Should not matter if it is dCore or ub-dCore, Opera is binary and works with all.
Below works for me:
Create file /opt/debextra/opera
Put the below entry in that file:
http://deb.opera.com/opera/ stable non-free
sce-import -n opera
It imports, and loads, and works.
I wish I could reproduce the errors. I will reboot with ub-dCore-utopic to make sure opera works.
-
Ok, I see the bug in the ub-dCore-utopic import of opera. It has nothing to do with Debian versus Ubuntu, but a script bug. I will check into it.
-
Simple fix, uploaded. See Release thread for details.
-
any reason why exists ub-dCore-utopic.gz/bin/lsmmod ?
-
Busybox symiink.
-
lsMMod?
-
lsmod typo. Fixed.
-
are you interested in my reports? yes or not?
-
Yes, I just fixed the last one which was a misnamed file.
-
Hi,
allright then.
Coming back, please see that, even if it looks a trivial matter, I could not be.
Afaik, as for dCore init logic, lsmod is already in /bb.
Duplicating it in /bin, could (randomly) be messed up by busybox cp (as I warned before).
Thats because standard sce-import always imports kmod pkg.
So it could end up with a random setup, where lsmod could be resolved either as /bb, as kmod, as busybox sym or broken link.
Of course, it depends on the application which calls it, on PATH, and so on.
Asof, btw, kmod is one of the many pkgs I exclude from import, with my mod.
ok?
-
I believe /bin/lsmod was in base as some apps expect it there, like /bin/bash for bash. But /bin/lsmod already existing as a symlink will prevent the one from kmod being installed there.
Here is where the startup scripts come into play. I will make kmod remove that busybox symlink, and replace it with the /bin/lsmod from kmod. That way busybox lsmod until kmod is installed.
Kmod is not always imported as a dep for me. The dependency list for the file package:
file
gcc-4.9-base
libc6
libgcc1
libmagic1
zlib1g
/tmp/.pkglist (END)
-
Hi,
Here is where the startup scripts come into play. I will make kmod remove that busybox symlink, and replace it with the /bin/lsmod from kmod. That way busybox lsmod until kmod is installed.
I was talking in general, lsmod was just the one example passing by...
Anyway, if I'm understanding well, you are going to "invert" the SCEs import functional logic... isn't it a bit dangerous? ;)
Afaik, now it works such as, links and files already in place are NOT substituted with new ones coming from subsequent SCEs - first which comes takes the place (excluding random cp glitches)
What you want to do is inverting that behavior - the last coming always kicks out the others, isn't it?
Well, I modded for a slightly more conservative strategy, you interested?
Kmod is not always imported as a dep for me. The dependency list for the file package:
when I say always, I meant kmod and any other pkgs are always dragged in, if ubuntu dependency index says so.
Only "advanced" users can mod that out.
-
And in general, busybox and the imported coreutils and other packages do not collide, busybox is used unless the larger package is imported.
lsmod is not 'inverting' import, just preserving it's desired effect regardless of where the busybox link is located.
-
Hi,
And in general, busybox and the imported coreutils and other packages do not collide, busybox is used unless the larger package is imported.
Thats correct in most of cases, but I found some exceptions, and I had to manage.
lsmod is not 'inverting' import, just preserving it's desired effect regardless of where the busybox link is located.
So, Im afraid I didn't understand what modification you are going to do.
lsmod link is still in /bin?
-
I have previously explained the /bin/lsmod existence.