Additional repos

[Jason W]

I have put in the ability to add extra repos to choose packages from.  As it involves some major changes I have put the changes in a testing dCore.gz in the release candidates section.

http://tinycorelinux.net/5.x/x86/release_candidates/dCore.gz

To use this option, create a directory in /opt called debextra.   Then add the mirror and repo in a file in that directory, one entry per file.  /opt/debextra/repo1 might contain:

http://www.deb-multimedia.org main


Another file may contain the non-free backports, I think the main is also needed for the backports of that repo:

http://www.deb-multimedia.org wheezy-backports non-free


Or the wheezy-updates of one of the main debian repos:

http://ftp-mirror.internap.com/pub/debian wheezy-updates main


And so on.  The package list files from the mirrors are parsed in the alphabetic or numeric order their files appear in that directory and the first match is chosen.  And all extra repos have priority over the main one or the custom builds in the case of the same package name. The first entry is the mirror, the second is the distribution, and the third is the repo (main, non-free, contrib).  In the absence of a distribution listed, wheezy is used.  The ones I recommend outside of wheezy are the wheezy-backports and wheezy-proposed-updates, and wheezy-updates.   Testing and unstable has a newer glibc so is not recommended.

Please test and report feedback.

[netnomad]

hi jason w,

great enhancement!!!

i guess there is a little typo in the sources-line
http://www.deb-multimedia.org main
perhaps it should be
http://www.deb-multimedia.org wheezy main non-free

additionally we probably need the deb-multimedia-keyring.

nevertheless i have a problem to connect despite i use the deb-multimedia-keyring:
tc@box:~$ importsce -s minitube
#################### 100.0% 0.0 kBps DONE   

verifying download...checksum matches OK
used 8796160 local, fetched 0
Using the wheezy main repo of mirror http://www.deb-multimedia.org
wget: server returned error: HTTP/1.1 404 Not Found
bunzip2: invalid magic
Mirror http://www.deb-multimedia.org is unavailable
Using the wheezy-updates main repo of mirror http://ftp-mirror.internap.com/pub/debian
Using the wheezy main repo of mirror: http://ftp.us.debian.org/debian
minitube is not available as a package.  Exiting...

the other two repos work fine, so far as i see.

thank you for that new great tool.

[Jason W]

Though I am trying to follow the familiar Debian format of the sources.lst, it is not exactly the same.  The entry

http://www.deb-multimedia.org main

is the same as using

http://www.deb-multimedia.org wheezy main

as wheezy is used in the absence of that field.  I can remove that safeguard, though.  However, main and non-free are two separate repos on a mirror.  To include them both on one line looks simple, but would increase the complexity of the code to support this considerably.  Main and non-free need to be listed on two separate mirror files.

Right now we require a repo that follows the Debian standard, as those that are not will have the same package listed in the same Packages.bz2 file, one for i386 and one for x86_64.  And our package routine assumes one entry per Packages file, but there may be a way to work that in.

You are using the deb-multimedia keyring?  I have been using the multimedia repo with no keyring.   What is your exact entry in your deb-multimedia mirror file?

[netnomad]

hi jascon w,

yes, i use the deb-multimedia-keyring.
tc@box:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          900M   84M  816M  10% /
tmpfs           500M     0  500M   0% /dev/shm
/dev/sdb2       3.5G  2.1G  1.4G  61% /mnt/sdb2
/dev/loop0       14M   14M     0 100% /tmp/tcloop/original-modules-3.8.13-tinycore
/dev/loop1      325M  325M     0 100% /tmp/tcloop/mydesktop
/dev/loop2      9.7M  9.7M     0 100% /tmp/tcloop/deb-multimedia-keyring

tc@box:/opt/debextra$ cat repo1
http://www.deb-multimedia.org wheezy main
tc@box:/opt/debextra$ cat repo2
http://www.deb-multimedia.org wheezy non-free
tc@box:/opt/debextra$ cat repo3
http://www.deb-multimedia.org wheezy-backports non-free
tc@box:/opt/debextra$ cat repo4
http://ftp-mirror.internap.com/pub/debian wheezy-updates main

tc@box:~$ importsce -s
Enter starting characters of package sought: minitube
#################### 100.0% 0.0 kBps DONE   
verifying download...checksum matches OK
used 8796160 local, fetched 0
Using the wheezy main repo of mirror http://www.deb-multimedia.org
Using the wheezy non-free repo of mirror http://www.deb-multimedia.org
wget: server returned error: HTTP/1.1 404 Not Found
bunzip2: invalid magic
Mirror http://www.deb-multimedia.org is unavailable
Using the wheezy-updates main repo of mirror http://ftp-mirror.internap.com/pub/debian
Using the wheezy main repo of mirror: http://ftp.us.debian.org/debian
minitube is not available as a package.  Exiting...

thank you for your help.

[netnomad]

hi jason w,

http://www.deb-multimedia.org wheezy-backports non-free
doesn't exit,
i guess you mean:
http://ftp.debian.org/debian/ wheezy-backports main ?

tc@box:~$ importsce -s minitube
#################### 100.0% 0.0 kBps DONE   
verifying download...checksum matches OK
used 8796160 local, fetched 0
Using the wheezy main repo of mirror http://www.deb-multimedia.org
Using the wheezy non-free repo of mirror http://www.deb-multimedia.org
Using the wheezy-backports main repo of mirror http://ftp.debian.org/debian/
Using the wheezy-updates main repo of mirror http://ftp-mirror.internap.com/pub/debian
Using the wheezy main repo of mirror: http://ftp.us.debian.org/debian
minitube is not available as a package.  Exiting...

tc@box:/opt/debextra$ cat repo_wheezy_backports
http://ftp.debian.org/debian/ wheezy-backports main
tc@box:/opt/debextra$ cat repo_multimedia_main 
http://www.deb-multimedia.org wheezy main
tc@box:/opt/debextra$ cat repo_multimedia_nonfree
http://www.deb-multimedia.org wheezy non-free
tc@box:/opt/debextra$ cat repo_wheezy_backports   
http://ftp.debian.org/debian/ wheezy-backports main
tc@box:/opt/debextra$ cat repo_wheezy_updates   
http://ftp-mirror.internap.com/pub/debian wheezy-updates main

[Jason W]

The backports non-free doesn't exist after all, though it is just an example I will change it to something that does.

[netnomad]

hi jason w,

now repos with names like
tc@box:/opt/debextra/repo1 numbered everything work fine for me.

repos with an own name-system like
tc@box:/opt/debextra/repo_wheezy_backports
work not so fine for the downloading-process of the packages,
because importsce is looking for the name convention as described above:
when importsce wants to fetch the packages, it is looking for repo1 or comparable.

[Jason W]

There is one repo that does that, and says to put  ./   ./   where the wheezy and main would go.  But then there is the issue mentioned of having multiple arch's in the same Packages file, which at the moment is not supported.  It should only be a matter of honing the awk script that searches the file, and if it can be done in a way that does not too much negatively affect performance then maybe we can go that route.

[Jason W]

Oh, I see what you are saying.  I did not know there is a limitation on the naming of the files in /opt aside from the usual unix standard - no spaces, no slashes, etc.  I will look into it.

EDIT:  I have no issue with a filename like below:

/opt/debextra/debian_multimedia_main.

[netnomad]

hi jason w,

perhaps importupdatecheck needs also to be aware about this enhancement of the additional repos?

thank you for your help and commitment.

[Jason W]

Ni netnomad,  One day I may can work it in.  It would require that the mirror files in /opt/debextra be the same as when the extension was imported though. 

The bad thing about creating tools is that either development has to be held back to not break existing tools, or the tools get broken.  For now I will focus on getting the ability to use other repos in place and polished, then perhaps importupdatecheck can be updated.

I am finishing being able to use Package files that have multiple arch's in them, which will support the use of other Debian mirrors that use one Package file for more than one arch.  A bug or two to fix then I will upload that to be tested.

[netnomad]

hi jason w,

that's fine, you make a great job!

thank you for every step in further development.

[Jason W]

Ok, support for Packages.bz2 files that have multiple arch's in the same file has been added, along with some bugfixes.  Please test, updated dCore.gz and iso are in the release candidates section:

http://repo.tinycorelinux.net/5.x/x86/release_candidates/

[Jason W]

Reposted with a bugfix.

[netnomad]

hi jason w,

tc@box:/opt/debextra$ cat repo1
http://www.deb-multimedia.org main
tc@box:/opt/debextra$ cat repo2
http://ftp.debian.org/debian/ wheezy-backports main
tc@box:/opt/debextra$ cat repo3
http://ftp-mirror.internap.com/pub/debian wheezy-updates main non-free

tc@box:/opt/debextra$ importsce -s minitube
#################### 100.0% 0.0 kBps DONE   
verifying download...checksum matches OK
used 8798208 local, fetched 0
Using the wheezy main repo of mirror http://www.deb-multimedia.org
Using the wheezy-backports main repo of mirror http://ftp.debian.org/debian/
Using the wheezy-updates main repo of mirror http://ftp-mirror.internap.com/pub/debian
Using the wheezy main repo of mirror: http://ftp.us.debian.org/debian
minitube is not available as a package.  Exiting...

is the non-free ignored in repo3?
the rest seems to work fine for me

even importupdatecheck seems to work with the additional repos:

tc@box:~$ cd .TCE/sce && importupdatecheck mydesktop
#################### 100.0% 0.0 kBps DONE   
verifying download...checksum matches OK
used 8798208 local, fetched 0
Using the wheezy main repo of mirror http://www.deb-multimedia.org
Using the wheezy-backports main repo of mirror http://ftp.debian.org/debian/
Using the wheezy-updates main repo of mirror http://ftp-mirror.internap.com/pub/debian
Using the wheezy main repo of mirror: http://ftp.us.debian.org/debian
Checking for updated deb packages, please be patient..
Checking for updated dCore startup scripts, please be patient..
libvlc5
libvlccore5
vlc-data
libtag1-vanilla
libswscale2
libtag1c2a
libwbclient0
libsmbclient
libpostproc52
libdca0
vlc
vlc-nox
libjson0
libpulse0
libavformat53
libxvidcore4
libmp3lame0
libavcodec53
libavutil51
tzdata
libusb-1.0-0
fluxbox
openssh-server
openssh-client
ssh
tzdata
The above have package updates available in Debian, import mydesktop again to get them.
No dCore custom startup scripts inside mydesktop.sce have updates available.

the update-requests result from the updates in the additional repos.