sce-update, possible quick check feature

[Jason W]

Ok, thanks for testing.  Here is what I see.  Debinx needs to be run anyway to produce the /tmp/debinx but without producing a new OLDDEBINX, I have not rebooted in testing this so I overlooked it.  Will be added.

Md5sum is checked to see if the NEWDEBINX and OLDDEBINX are different in any way, and if so triggering a normal sce-update.  But a diff could be offered for those who would like to view as an expert option as to whether to proceed with sce-update.  Today dCore-wily had libvirt-bin and related packages added to the debinx that were not there an hour ago but a newly introduced package and not relevant to my install.

Viewing your diff, the + entries appear to be what is changed in the NEWDEBINX.  Correct me if I am wrong.

If the OLDDEBINX and NEWDEBINX are different at all per md5sum, then checking an individual SCE or all or a menu selected list is still just like before.  I am for the diff viewing and an informed decision for expert users, but let's first make this a stable cut after some more testing, I need to upload to a stable cut to release and then we can further add features.

[Jason W]

Fixed needed detGetEnv on first run, please check.

[nitram]

Yes as per the diff output above:
--- OLDDEBINX
+++ NEWDEBINX

I don't understand how this DEBINX diff works, however. The diff from above listed Fluxbox which i use, why not updated? Already deleted, next time will open in vi, go to line #323546 or whatever and take a closer look. Maybe because Fluxbox is dCore pre-built, overrides Debian repository. But then incorrectly, i would feel obligated to run  sce-update fluxbox.

Agree with getting stable first, hence code words 'sce-update has been under revision, so just food for thought or maybe future feature' and 'Just thinking outloud'. Certainly don't intend to disrupt your life with a new RC within hours, just something to think about while the topic is fresh. The work you do is incredible. This stuff is just nice to have...and save unnecessary CPU cycles. Don't consider myself expert, still learning, but i think most dCore users would appreciate this feature.

Just read your note, will re-test later tonight. Thanks.

[Jason W]

Ok, I see the issue with prebuilt versus Debian as far as needing updates.  This diff or md5sum check of catted debinx, prebuilt, files can only determine if something was changed, not what and where as in what repo. Is only good to determine if nothing at all was changed, then no need to run regular sce-update.  Otherwise, sce-update as usual. 

We can determine if this is really useful as if something is updated in Debian/Ubuntu but the prebuilt supercedes it, then an update is really not needed but is indicated by the change in Debian/Ubuntu.   

It's all good, because certainly nothing lost by this addition and can save a full "sce-update -a" when no changes in debinx or perbuilt have been made, but when changes in the debinx files, it is not clear whether really needed.   

With existing routines using the OLD and NEW debinx diff info, we can tell what repo packages available for update come from, and even get which SCEs are affected.  And much quicker than the current sce-update routine does as it would be grepping one word per package change rather than doing a full md5sum checking of each or every SCE.  Lets stay with this thought.

[Jason W]

Oh, and when I see new ideas and concepts I tend to go with them when they seem they may make things more efficient or useful.  Sometimes they don't work, other times they do, time and testing will tell in the end.  Everyone keep the ideas coming.

[netnomad]

hi friends,

i attached a screenshot about two missing files.
tc@box:~$ version -r
You are running dCore-jessie:2016.01.13.01.02, the latest release candidate.

thank you for your interest.

[nitram]

Sorry still something not right, excuse if netnomad indicated same - Dillo can't download attached screenshots properly.

After a fresh boot, didn't run any other sce-* commands or anything previously:

Code: [Select]

tc@box:~$ version -r
You are running dCore-jessie:2016.01.13.01.02, the latest release candidate.
tc@box:~$ sce-update -a
/usr/bin/sce-update: line 116: can't open /tmp/debinx: no such file
debian
#################### 100.0% 0.0 kBps DONE   

verifying download...checksum matches OK
used 11409408 local, fetched 0

Debian Index synced: debian_jessie_main_i386_Packages

#################### 100.0% 0.0 kBps DONE   

verifying download...checksum matches OK
used 362496 local, fetched 0

Debian Security Index synced: debian_jessie_security_i386_Packages

Using the repo http://security.debian.org jessie main
cat: /tmp/PKGEXTRAFILEMD5SUMLIST: No such file or directory
Your SCEs are up to date.


[Jason W]

Ok, I will check into it tonight.

[Jason W]

Bugs hopefully fixed, and added option to view diff of new and old debinx data.

[Jason W]

Here is an issue I see with this.  Unless "sce-update -a" is run and completed, and instead updating only one or several SCEs, the NEWDEBINX when copied to OLDDEBINX at the completion of sce-update, it gives a false impression all SCEs are up to date and therefore until a new change in DEBINX data occurs, it kind of assumes all SCEs have been updated to the latest OLDDEBINX state.  Don't know how to fix or work around.  Ideas welcome.

[nitram]

Just logged in to report this issue (stomach sinks). In OP told you my logic might be flawed. Never thought of it earlier as i always choose to  sce-update -a  , but earlier today ran regular  sce-update  just for testing.

Looks like either the entire OLDDEBINX idea should be scrapped or  sce-update -a  should be the only option. Having only update all may simplify the update script a lot. IMO dCore's strength is the ability to fairly easily run a fully up to date system. So for me personally i would never update Dillo or Iceweasel without worrying about iptables or whatever. Would want them all checked and updated.

Hopefully other regular members will provide feedback. Sorry if this idea caused a lot of work or disruption. May make things better in the end or may just be discarded. Sounded like a good idea at the time...

Other feedback that may no longer be relevant. Ran  sce-update  but decided to check all (pressed 1 multiple times). Almost half SCEs updated due to libpng update. Upon completion OLDDEBINX didn't move to NEWDEBINX, so running  sce-update  again afterwards continued to flag potential updates (perpetual updates as working off of now outdated DEBINX). Reboot still flagged the same updates, so my /debinx still looks like this (think should just be OLDDEBINX):

Code: [Select]

tc@box:/mnt/sdb4/tce/import/debinx$ ll
total 35M
-rw-rw-r-- 1 tc staff  12M Jan 13 21:31 NEWDEBINX
-rw-rw-r-- 1 tc staff  12M Jan 12 19:20 OLDDEBINX
-rw--w---- 1 tc staff  11M Jan 14  2016 debian_jessie_main_i386_Packages
-rw--w---- 1 tc staff 357K Jan 14  2016 debian_jessie_security_i386_Packages

Not sure, maybe an argument for running  sce-update -a  only. Maybe libpng bad example, but couldn't there potentially be version conflicts if multiple versions of some dependencies loaded at the same time.

In my recent libpng update all these were affected, never would have thought, but now i know they are all up to date:

Code: [Select]

SCE:  Xprogs
  Available package updates:
    None
 
 
SCE:  conky
   Available package updates: 
   libpng12-0        debian jessie main
 
 
SCE:  dillo
  Available package updates:
    libpng12-0        debian jessie main
 
 
SCE:  emelfm
  Available package updates:
    None
 
 
SCE:  fluxbox
  Available package updates:
    libpng12-0        debian jessie main

 
SCE:  graphics-3.16.6-tinycore
  Available package updates:
    None
 
                                                                               
 SCE:  gtkfind
  Available package updates:
    None
 
 
SCE:  iceweasel
  Available package updates:
    libpng12-0        debian jessie main
 
 
SCE:  iptables
  Available package updates:
    None
 
 
SCE:  netfilter-3.16.6-tinycore
    None
 
 
SCE:  xorg-intel
  Available package updates:
    libpng12-0        debian jessie main


[netnomad]

hi friends,

i attached my screenshot with a probably small cosmetic issue.

the overall functions are working now fine for me.

thank you for your work.

[Jason W]

If we use this for sce-update -a, all should be good.  But only then, of course.  Nothing is lost, nor is there any misspent time.  In fact, much gained for the sce-update -a routine.  Because with sce-update -a is where the most time and machine requirements is called upon, and is probably the most used option of sce-update, at least for me.

Let's work this into sce-update -a and this should save much time and resourse requirements.   Tomorrow I will add this to only sce-update -a and we should be good.  Sorry for the false "It may not work" alarm.  Any bugs as to moving NEWDEBINX to OLDDEBINX can be fixed.

But I just tonight noticed that it may not work for all sce-update options.

[nitram]

Think you're correct, sorry for jumping the gun. Additional feedback. Previous cat type errors appear fixed. Like ability to review DEBINX diff, worked well.

Re-ran with  sce-update -a  , duplicate 'Checking for updates of existing SCEs..' line:

Code: [Select]

Using the repo http://security.debian.org jessie main
 
Sce-update is about to proceed.  Press Enter to proceed, y to view the
difference in new and old debinx data, or Ctrl-C to abort..
Checking for updates of existing SCEs..
Checking for updates of existing SCEs..
 
Searching for available updates for all existing SCEs.
Xprogs: Checkup for updates..


After running  sce-update -a  same DEBINX issue noted above. NEWDEBINX didn't move to and overwrite OLDDEBINX upon completion, which would trigger perpetual update checks. [Edit] ...but when i run  sce-update -a  yet again, then it finally overwrites NEWDEBINX to OLDDEBINX but get this error:

Code: [Select]

tc@box:/mnt/sdb4/tce/import/debinx$ sce-update -a
debian
#################### 100.0% 24.6 kBps DONE     

verifying download...checksum matches OK
used 11409408 local, fetched 0

Debian Index synced: debian_jessie_main_i386_Packages

#################### 100.0% 0.0 kBps DONE   

verifying download...checksum matches OK
used 366592 local, fetched 0

Debian Security Index synced: debian_jessie_security_i386_Packages

Using the repo http://security.debian.org jessie main
Your SCEs are up to date.
/usr/bin/sce-update: exit: line 175: Illegal number: 0-


[Jason W]

Uploaded a fix, hopefully all is good now, the OLDDEBINX/NEWDEBINX thing is only done with "sce-update -a".