WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: 3.x repo observations  (Read 3700 times)

Offline Kingdomcome

  • Sr. Member
  • ****
  • Posts: 286
3.x repo observations
« on: May 13, 2010, 04:38:55 PM »
Ive rewritten my repo mirroring script to use zsync in hopes of conserving bandwidth.  Since doing so I have found some problems with the 3.x repo:

1) Some extensions are missing their .zsync file
Code: [Select]
alsa-modules-2.6.33.3-tinycore64.tcz
alsa-modules-2.6.33.3-tinycore.tcz
base-dev.tcz
binutils.tcz
bluetooth-2.6.33.3-tinycore64.tcz
bluetooth-2.6.33.3-tinycore.tcz
cpufreq-2.6.33.3-tinycore64.tcz
cpufreq-2.6.33.3-tinycore.tcz
doxygen-doc.tcz
filesystems-2.6.33.3-tinycore64.tcz
filesystems-2.6.33.3-tinycore.tcz
firewire-2.6.33.3-tinycore64.tcz
firewire-2.6.33.3-tinycore.tcz
firmware-iwlwifi.tcz
firmware-libertas.tcz
firmware.tcz
firmware-wimax.tcz
gcc_libs.tcz
gmp.tcz
graphics-2.6.33.3-tinycore64.tcz
graphics-2.6.33.3-tinycore.tcz
gs.tcz
gtk2_prefs.tcz
hwmon-2.6.33.3-tinycore64.tcz
hwmon-2.6.33.3-tinycore.tcz
i2c-2.6.33.3-tinycore64.tcz
i2c-2.6.33.3-tinycore.tcz
icu-dev.tcz
icu-doc.tcz
input-joystick-2.6.33.3-tinycore64.tcz
input-joystick-2.6.33.3-tinycore.tcz
input-tablet-touchscreen-2.6.33.3-tinycore64.tcz
input-tablet-touchscreen-2.6.33.3-tinycore.tcz
ipv6-2.6.33.3-tinycore64.tcz
ipv6-2.6.33.3-tinycore.tcz
irda-2.6.33.3-tinycore64.tcz
irda-2.6.33.3-tinycore.tcz
kvm-2.6.33.3-tinycore64.tcz
kvm-2.6.33.3-tinycore.tcz
leds-2.6.33.3-tinycore64.tcz
leds-2.6.33.3-tinycore.tcz
libglademm.tcz
libpulseaudio.tcz
libXtst.tcz
mpfr.tcz
mtd-2.6.33.3-tinycore64.tcz
mtd-2.6.33.3-tinycore.tcz
net-bridging-2.6.33.3-tinycore64.tcz
net-bridging-2.6.33.3-tinycore.tcz
netfilter-2.6.33.3-tinycore64.tcz
netfilter-2.6.33.3-tinycore.tcz
pavucontrol.tcz
pulseaudio-dev.tcz
pulseaudio.tcz
raid-dm-2.6.33.3-tinycore64.tcz
raid-dm-2.6.33.3-tinycore.tcz
scsi-2.6.33.3-tinycore64.tcz
scsi-2.6.33.3-tinycore.tcz
silgraphite-dev.tcz
silgraphite.tcz
usb-serial-2.6.33.3-tinycore64.tcz
usb-serial-2.6.33.3-tinycore.tcz
v4l-dvb-2.6.33.3-tinycore64.tcz
v4l-dvb-2.6.33.3-tinycore.tcz
wimax-2.6.33.3-tinycore64.tcz
wimax-2.6.33.3-tinycore.tcz
wireless-2.6.33.3-tinycore64.tcz
wireless-2.6.33.3-tinycore.tcz
Xfbdev.tcz
Xvesa.tcz

2)  Some extensions fail to download through zsync although wget downloads fine (Unknown cause, but very reproducible)
Code: [Select]
automake.tcz
bison.tcz
bluez.tcz
cloog.tcz
coreutils.tcz
e2fsprogs_apps.tcz
file.tcz
findutils.tcz
gcc.tcz
grep.tcz
gst-plugins-good.tcz
help2man.tcz
hplip.tcz
icu.tcz
libcanberra-dev.tcz
libcanberra.tcz
libmtp-dev.tcz
libmtp.tcz
m4.tcz
patch.tcz
ppl.tcz
pyqt.tcz
sed.tcz
sip.tcz

3)  A stray tczl has snuck in
Code: [Select]
help2man.tczl

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11044
Re: 3.x repo observations
« Reply #1 on: May 13, 2010, 04:54:40 PM »
Thanks, I've uploaded zsync files for the kernel and firmware extensions. Though if you already used rsync, it should be no change.

On those that won't download, seems that they have out-of-sync zsync files (tested patch.tcz).
The only barriers that can stop you are the ones you create yourself.

Offline Kingdomcome

  • Sr. Member
  • ****
  • Posts: 286
Re: 3.x repo observations
« Reply #2 on: May 13, 2010, 06:28:57 PM »
Previously I was using gnu wget with the mirroring option.  This is my first experience with a syncing app.  Although it may take a little longer to complete, I believe that the possible bandwidth savings will be worth it, especially since the script runs on a dedicated server box and during the night.  Thanks for the quick fixes.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #3 on: May 13, 2010, 08:11:24 PM »
I am in the process of getting my 3.x mirror in place, and after that it will be very easy for me to fix and upload any broken zsync files.   Maybe by tomorrow I can fix that second list that apparently has bad zsync files.

Offline Kingdomcome

  • Sr. Member
  • ****
  • Posts: 286
Re: 3.x repo observations
« Reply #4 on: May 13, 2010, 09:16:55 PM »
Jason,

  It appears that the problems I saw with the second list have disappeared.  Im not sure if anything was changed on ibiblio or not. but a resync I performed manually after Curaga's post proved totally successful.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #5 on: May 13, 2010, 10:48:36 PM »
I am still having trouble with the second set.  Tonight I will have my mirror, so corrections will be easy when needed.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #6 on: May 13, 2010, 10:51:14 PM »
I may be using the wrong test command.  Will look more into my result later.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #7 on: May 13, 2010, 10:54:12 PM »
I am seeing that some work and some don't.  Will fix soon as I can.

EDIT - Confirmed, remaking .zsync files fixes the issue with the extensions I tested.  Will remake them.
« Last Edit: May 13, 2010, 10:57:06 PM by Jason W »

Offline Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14819
Re: 3.x repo observations
« Reply #8 on: May 14, 2010, 01:30:50 AM »
On those that won't download, seems that they have out-of-sync zsync files (tested patch.tcz).

That I would be me, I updated quite a few extensions in the 3.x repo - patch being one of them...

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #9 on: May 14, 2010, 02:07:13 PM »
Mirror is complete, I will remake the zsync files tonight. 

Offline Kingdomcome

  • Sr. Member
  • ****
  • Posts: 286
Re: 3.x repo observations
« Reply #10 on: May 21, 2010, 10:55:31 PM »
Just as a follow up, Ive written a little integrity check script that checks the 2.x and 3.x repos on ibiblio.  Here are the results:
Code: [Select]
Fri May 21 18:52:08 EDT 2010
Missing infos in 2.x repo

Missing lists in 2.x repo
pigz.tcz.list

Missing md5s in 2.x repo

Missing zsyncs in 2.x repo
blueman.tcz.zsync
cutecom.tcz.zsync
doxygen-doc.tcz.zsync
doxygen-gui.tcz.zsync
GConf-doc.tcz.zsync
gnome-vfsmm.tcz.zsync
icu-40.tcz.zsync
icu-44-dev.tcz.zsync
icu-44-doc.tcz.zsync
icu-44.tcz.zsync
libatomic_ops.tcz.zsync
libglademm.tcz.zsync
libgnomecanvasmm.tcz.zsync
libgnomemm.tcz.zsync
obex-data-server.tcz.zsync
pigz.tcz.zsync
pyrex.tcz.zsync
silgraphite-dev.tcz.zsync
silgraphite.tcz.zsync
star-cups-driver.tcz.zsync

Out of sync zsyncs in 2.x repo
bluez.tcz
dbus.tcz
doxygen.tcz
hplip.tcz
libmtp-dev.tcz
libmtp.tcz
pyqt.tcz
sip.tcz


Missing infos in 3.x repo

Missing lists in 3.x repo

Missing md5s in 3.x repo

Missing zsyncs in 3.x repo
base-dev.tcz.zsync
doxygen-doc.tcz.zsync
gcc_libs.tcz.zsync
GConf-doc.tcz.zsync
gmp.tcz.zsync
gs.tcz.zsync
gtk2_prefs.tcz.zsync
icu-dev.tcz.zsync
icu-doc.tcz.zsync
libglademm.tcz.zsync
libpulseaudio.tcz.zsync
libXtst.tcz.zsync
mpfr.tcz.zsync
pavucontrol.tcz.zsync
pulseaudio-dev.tcz.zsync
silgraphite-dev.tcz.zsync
silgraphite.tcz.zsync
star-cups-driver.tcz.zsync
Xfbdev.tcz.zsync
Xvesa.tcz.zsync

Out of sync zsyncs in 3.x repo
automake.tcz
binutils.tcz
bison.tcz
bluez.tcz
cloog.tcz
coreutils.tcz
doxygen.tcz
e2fsprogs_apps.tcz
findutils.tcz
gcc.tcz
grep.tcz
gst-plugins-good.tcz
help2man.tcz
hplip.tcz
icu.tcz
libcanberra-dev.tcz
libcanberra.tcz
libmtp-dev.tcz
libmtp.tcz
m4.tcz
patch.tcz
ppl.tcz
pyqt.tcz
sed.tcz
sip.tcz

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #11 on: May 21, 2010, 11:13:41 PM »
Thanks.  I will follow up and make corrections.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: 3.x repo observations
« Reply #12 on: May 24, 2010, 11:46:49 AM »
Fixed those reported, thanks.