WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: User mirroring of the repo.  (Read 122414 times)

Offline lancer.kz

  • Newbie
  • *
  • Posts: 1
Re: User mirroring of the repo.
« Reply #15 on: May 31, 2010, 11:30:49 PM »
why don't tc forum provide DVD of whole repo?
1: reduction of bandwidth
2: one kind donation way
3: easy to use

Offline beerstein

  • Hero Member
  • *****
  • Posts: 530
Re: User mirroring of the repo.
« Reply #16 on: January 20, 2011, 12:11:51 PM »
Hi: it takes about 6-10 hours to download the whole TC 3 repo. But when you have it on DVD or in
a folder I needed to install each demendency seperately.

Could you please give me a hint how to make that easier?

Do I need to run a http server on my machine?

Is there a way to integrate that repo into the app browser or the app auditor?

Thank you so much
t(w)o be(ers) or not t(w)o be(ers) that is the question

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11048
Re: User mirroring of the repo.
« Reply #17 on: January 20, 2011, 12:42:32 PM »
Sounds like you did not download the dep files?
The only barriers that can stop you are the ones you create yourself.

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: User mirroring of the repo.
« Reply #18 on: January 20, 2011, 01:21:00 PM »
Hi: it takes about 6-10 hours to download the whole TC 3 repo.

Did you read the first post of this thread?
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline hiro

  • Hero Member
  • *****
  • Posts: 1229
Re: User mirroring of the repo.
« Reply #19 on: January 20, 2011, 02:34:36 PM »
Heh, this thread is only encouraging it :D
I don't understand why people update so often. The extensions I'm using very rarely change at all.

Offline Star Brilliant

  • Newbie
  • *
  • Posts: 5
Re: User mirroring of the repo.
« Reply #20 on: January 20, 2011, 11:21:16 PM »
I do think offering repo DVD is not suitable.
Here are the reasons in my opinion:
- The extensions updates very often
- There are a lot of extensions that are useless for most of the users (do you think it is impossible that most of you install extensions like 'aircrack-ng'?)
- It is a waste to store so many useless extensions into DVDs

My suggestion is to offer SOME useful extensions into CDs or provide an ISO for us.

Offline vinnie

  • Hero Member
  • *****
  • Posts: 1187
  • HandMace informatic works
Re: User mirroring of the repo.
« Reply #21 on: February 22, 2011, 12:30:26 AM »
My proposition: Select most popular stable extensions which are not in testing mode. Create iso image and ask people to use torrent.

but this idea has opened a way not to discard.
for the future have a repository on a torrent protocol could be revolutionary.
enough one button on the interface of type "keep active sharing" with a green/red icon (on/off).
if I remember well the tracker can be updated, md5's are not necessary with the support of hashing.
ok, I'm crazy  :P

Offline beerstein

  • Hero Member
  • *****
  • Posts: 530
Re: User mirroring of the repo.
« Reply #22 on: February 22, 2011, 03:57:26 AM »
HI:
I suggest that users should make their own sub-repo at home or in the office:

In this sub-repo they should put the extensions they need and want the most. (20-50)
is what I always need. Then, depending on further developments and requirements the users
can add new extensions to their sub-repo at home. That home repo can be built up over time,
updated and stored on HD, CD, stick, card, DVD or whatever.

At the end it should work like a "hybrid - repo" using the one from ibiblio and the home repo.
It could save substantial bandwith.

This is just a suugestioen and I would be glad if you will comment on it.
t(w)o be(ers) or not t(w)o be(ers) that is the question

Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: User mirroring of the repo.
« Reply #23 on: February 22, 2011, 04:10:58 AM »
Torrent is not revolutionary and a single tracker represents a high risk as a potential point of failure. What would be really revolutionary a distrubuted P2P DHT network. There are different opensource working models which can be used as a basis.

Just an idea, but can be a good University project for someone :)
Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline hiro

  • Hero Member
  • *****
  • Posts: 1229
Re: User mirroring of the repo.
« Reply #24 on: February 22, 2011, 08:00:21 AM »
Beerstein, I like your idea.
Currently I use tinycore on 2 pcs and 2 laptops. The tce is "cached" on multiple sd cards and usb flash drives. But there are problems in my current workflow when I change/update stuff:

1. The different tce dirs are not synced automatically
2. I don't want to keep all tces up to date, but compatible so that they work on all pcs.
3. If I update the base or extensions and everything is fine on one pc it might not work on the others.
4. If I then sync the dirs and the non-working version overwrites the old functional one it takes a lot of time to clean up the mess again. Especially if the old, working packages aren't available on the repo any more.
5. If I setup a new pc and download the extensions from ibiblio I might get newer versions than on the other pcs. The tce dirs will then potentially be incompatible even without an active upgrade.

So I haven't found a nice way to manage this yet, but a private partial repo mirror might be a good starting point. But I haven't tried it because it seems a bit more difficult then just starting a web browser in your tce dir, e.g. what about .info and .list files.?
« Last Edit: February 23, 2011, 02:51:45 PM by hiro »

Offline vinnie

  • Hero Member
  • *****
  • Posts: 1187
  • HandMace informatic works
Re: User mirroring of the repo.
« Reply #25 on: February 22, 2011, 08:27:27 AM »
Torrent is not revolutionary and a single tracker represents a high risk as a potential point of failure. What would be really revolutionary a distrubuted P2P DHT network. There are different opensource working models which can be used as a basis.

Just an idea, but can be a good University project for someone :)

The idea of app browser based on p2p is new for me, i know torrent system, but I trust your word, if you say it's better in that way

Offline beerstein

  • Hero Member
  • *****
  • Posts: 530
Re: User mirroring of the repo.
« Reply #26 on: February 23, 2011, 11:50:19 AM »
Hi hiro:
It seems to me that you do almost the same, what I am doing.
I have four (older Pcs - max 1.6 Ghz and 500 MB RAM) and three notebooks.

Once in a while I add a new extension. Then, after a while I just copy the /tce over to the other
ones. I was not able to synchronize all my /tce 's - that would be too much work.

Do you think - just copying the /tce is good enough in terms of errors and/or missing files?
Should I rather tar the /tce , copy that tar thing over and un-tar at the other machine?

Until I red you message (thank you) I thougt a lightpd or apache with all my files (the ones I already have)
should do the trick. 
I think there is a way to update the files you mentioned automatically. But how?

If we could find a few more members interested in this special thing, may be we can find a solution.

Have a good day
beerstein

PS: To all members - please let me know if you are interested in this subject
t(w)o be(ers) or not t(w)o be(ers) that is the question

Offline SamK

  • Hero Member
  • *****
  • Posts: 713
Re: User mirroring of the repo.
« Reply #27 on: February 23, 2011, 02:13:51 PM »
I think there is a way to update the files you mentioned automatically. But how?
In the past I have found this a capable app with various Ubuntu setups - Apt-Cacher NG.
http://www.unix-ag.uni-kl.de/~bloch/acng

With it running on the LAN, a software request from any other machine on the LAN is directed to ACNG.  If the package or update is found in the ACNG cache it is made available to the requesting machine.  If the software is not present in the ACNG cache it is obtained from the WAN repository, made available to the requesting machine, and added to the ACNG cache for future use.  If my memory is accurate it also concurrently handles multiple versions of the distribution.

It is primarily (but not exclusively) aimed at Debian based distributions, however it might provide ideas or a starting point for someone with coding skills.
   

Offline beerstein

  • Hero Member
  • *****
  • Posts: 530
Re: User mirroring of the repo.
« Reply #28 on: February 23, 2011, 02:23:00 PM »
Thank you SamK for this lead. I am afraid I am not that good to do a project like this, but
may be I can find a member here.
t(w)o be(ers) or not t(w)o be(ers) that is the question

Offline hiro

  • Hero Member
  • *****
  • Posts: 1229
Re: User mirroring of the repo.
« Reply #29 on: February 23, 2011, 02:33:47 PM »
Just copying is fine.
Perhaps syncing tce dirs is not that important.
To load and update stuff on the mirror first we could write a tce-cache script for the clients like this:
Code: [Select]
#!/bin/sh
ssh mirror tce-load -w $2
tce-load $1 $2
Then edit the tce-load on the mirror to include something like this:
Code: [Select]
wget `cat /opt/tcemirror`/3.x/tcz/$2.info
wget `cat /opt/tcemirror`/3.x/tcz/$2.list
and create a cache-update script, but I don't know the content yet ;)
And then we would start the httpd and echo our.mirror > /opt/tcemirror on all our clients.
« Last Edit: February 23, 2011, 02:50:53 PM by hiro »