Conflicting package sources not properly handled

[sm8ps]

I want to import the package cherrytree which is available in version 0.32.0-1 from the main ubuntu trusty repository. I added ppa:vincent-c/cherrytree to debextra/ which provides version 0.35.9-1~ppa1~trusty1. It generates a file with the following content:
http://ppa.launchpad.net/vincent-c/cherrytree/ubuntu trusty main

Nevertheless, the package is taken from the official repository by sce-import and also sce-update does not recognize the more recent version from the ppa. -- Does anybody else experience similar behavior or is it just me again?   

[Jason W]

Most surely because of the trusty-security entry that is found in /opt/debextra before the vincent-c-cherokee.

Given this, I will call the security entris /opt/debextra/zzz-trusty-security to avoid this in the future as that needs to be the last file in all cases to be read by sce-import in /opt/debextra.

[sm8ps]

I am not sure it is a matter of the security repos. The package is available from a regular repo; the ppa simply adds more recent versions.

Code: [Select]

/mnt/sda8/tce$ grep cherrytree *
debinx.ubuntu-trusty-universe:Package: cherrytree
debinx.ubuntu-trusty-universe:Filename: pool/universe/c/cherrytree/cherrytree_0.32.0-1_all.deb
debinx.ubuntu-trusty-universe:Homepage: http://www.giuspen.com/cherrytree/
debinx.vincent-c-cherrytree-main:Package: cherrytree
debinx.vincent-c-cherrytree-main:Filename: pool/main/c/cherrytree/cherrytree_0.35.9-1~ppa1~trusty1_all.deb
When I rename the debinx-file to zzz-debinx.ubuntu-trusty-universe sce-import fetches the right package. Though, such a work-around can only be applied if one knows about the different available versions.

sce-import seems to take the first available package. Is that correct? It would be desirable instead if it compared versions of all suitable packages. Is there a way to automate this other than manually renaming sources files?

For instance, I currently have the following debinx-files:

Code: [Select]

debinx.enlightenment-git-ppa-ubuntu-trusty  debinx.ubuntu-trusty-multiverse           debinx.ubuntu-trusty-security-universe
debinx.midori-ppa-main                      debinx.ubuntu-trusty-restricted           debinx.ubuntu-trusty-universe
debinx.owncloud_client-opensuse             debinx.ubuntu-trusty-security-main        debinx.vincent-c-cherrytree-main
debinx.trusty-security                      debinx.ubuntu-trusty-security-multiverse
debinx.ubuntu-trusty-canonical_partner      debinx.ubuntu-trusty-security-restricted

[Jason W]

So zzz-trusty-security as a name works as expected.  I don't imagine any extra repos are going to have the hame zzzz- as a prefix, so that solution is the best given that the first repo is picked based on numeric first and then alphabetical order.  This gives you total control which repo you want to pull from in what order.

Comparing package versions would add much complexity on my end, especially considering there may be more than 2 to compare, and also take away your control of what repo you want to pull what package from.   And almost always the extra repos contain the newer package.

You won't have to manually rename the security files in /opt/debextra, I will in the base dCore-*.gz images.

[sm8ps]

Comparing package versions would add much complexity on my end, especially considering there may be more than 2 to compare, and also take away your control of what repo you want to pull what package from.   And almost always the extra repos contain the newer package.

That is what I expected. I will add this explanation to the wiki because it is important to know.

You won't have to manually rename the security files in /opt/debextra, I will in the base dCore-*.gz images.

I still do not think that this solution is enough. In my case the "conflicting" repo, containing the obsolete package, is ubuntu-trusty-universe as opposed to a security repo. So there does not seem to be a way around manually "grading" repos alphabetically.

In this case I suggest to use a two digit numbered system instead of letters. So the security repos would become 90-trusty-security and the regular repos would become 80-trusty. That would give the flexibility to sort the repos in order of importance and also to easily juggle them around if need be. In my case, I would put all the ubuntu repos (universe, multiverse, restricted, canonical_partner) also into 80 and their corresponding security repos in 90 as there is no over-lap), the PPAs would go into the 50, say.

[Jason W]

The main repo is always last behind the /opt/debinx files.  And the security repo would need to be second to last, hence the zzz- prefix.  Not everyone is going to want to have to number their debinx's although they can if they want.  The numbers always are chosen first before letters, so if the security debinx is numbered and someone does not want to have to number theirs, the security repo will be chosen ahead of their extra repo.  I am not sure most folks will want that behavior. 

The main, universe, multiverse, restricted are already in the main DEBINX.  And only the security one is needed to be dealt with as far as not getting in the way of the extra repos.

In fact, I will see if I can put the security repo as automatic and not part of /opt/debextra.  Then there is complete simplicity in /opt/debextra as to how folks want to arrange their files.

[Jason W]

Actually, since I would have to have the security DEBINX as a separate file apart from the main one, and it would have to be named, that naming would also be the same as naming it in /opt/debextra.   Not to mention major changes and hours of work on the code.  There is no way a zzz- prefix is going to interfere with folks normal extra repo files.  Only zzzz* could come after it. 

[sm8ps]

The main repo is always last behind the /opt/debinx files.  And the security repo would need to be second to last, hence the zzz- prefix.  Not everyone is going to want to have to number their debinx's although they can if they want.  The numbers always are chosen first before letters, so if the security debinx is numbered and someone does not want to have to number theirs, the security repo will be chosen ahead of their extra repo.  I am not sure most folks will want that behavior.

Agreed! So if someone wants to order their repos, they can use letters or numbers to their liking.

I posted in the other thread about the security repos at the same time as you answered in this one. I am going to wait for your answers. If I understand correctly then a mechanism to have the security repos out of /opt/debextra would be too much effort. So the security repos shall stay in /opt/debextra but there should be all of them then.

[Jason W]

It is not that it would be too much effort, but we just still have the same issue of debinx names and what gets priority.  Since security is hard wired in, kind of like the multiverse and such that is able to be part of the main debinx, if security is invisible and named to where it is always just above main debinx in priority yet always below the /opt/debextra entries then that would work.   I would like to have security outside of the /opt/debextra since it is not really extra, that was just a quick way of fitting it in.  I will look more into it.

[sm8ps]

I am copying your answers from the other thread here so that we can continue the discussion in this one.

Yeah, I will aim at creating two 'invisible to the user' debinx's, one main and one security.  That makes sense since security is not extra and is in preference to the main by default.  I will get that going.

And there are security repos for main, universe, multiverse, restricted for the Ubuntus.  So that would mean 4 files in /opt/debextra.  I can combine those files on our server like the main and make it much more efficient, and make the second non-extra DEBINX.

I think I did not quite get what you had said before because what you propose is exactly what I had in mind: two separate debinx-files, one for the four regular repos plus one for the corresponding security repos, both provided by Core and invisible to the user. (Is there still need for /opt/debmirror then?) Priority is security over regular but both below any debextra-repos.

Cheers!
sm

[Jason W]

/opt/debmirror is kind of like /opt/tcemirror, just there for most folks, one can modify it if desired for the main repo.

I have the server side ready, next I will modify debGetEnv to pull in the security DEBINX.  I also streamlined things in the fetching scripts on the server to need less rsyncing of the Packages files, less bandwidth and better performance for the server. 

[Jason W]

Ok, security repos are now part of the base internal DEBINX handling and not in /opt/debextra.  Please test, all dCore x86 ports updated in release_candidates.

[jls]

Do you mean
http://security.ubuntu.com/ubuntu vivid-security main multiverse restricted universe?
what about
vivid multiverse restricted universe?

[jls]

Hi Jason
I see u added /opt/securitymirror

[Jason W]

Yeah, that way we can keep the scripts portable since the Debian and Ubuntu dCore's will have different entries for the security mirror.  debGetEnv creates that file, and the other scripts can get the information from it.