Tiny Core Linux

Tiny Core Extensions => TCE Q&A Forum => Topic started by: GNUser on February 14, 2022, 12:38:06 PM

Title: [Solved] fast way of syncing local mirror to official TCL repository?
Post by: GNUser on February 14, 2022, 12:38:06 PM: I sync my local TCL mirror to the official repo using rsync (the remote URL is repo.tinycorelinux.net::tc/$version/$arch/tcz/). Lately the download speed for the transfers has been in the 20-30 kB/s range, which causes the mirror operations to sometimes last several hours. My actual internet connection speed (not the theoretical speed advertised by ISP) is 35 Mbps so my internet dealings are otherwise very fast.

Is the server that hosts the official repository not doing well? Either way, do you know of a faster way of syncing local mirror with the official TCL repository?
Title: Re: fast way of syncing local mirror to official TCL repository?
Post by: Rich on February 14, 2022, 01:59:56 PM: Hi GNUser
I have no information on the server, so I can't comment on that.

Maybe see if some of the mirrors are faster? Here is a partial list:
https://ftp.nluug.nl/os/Linux/distr/pub/linux/distributions/tinycorelinux/
https://mirror.cpsc.ucalgary.ca/mirror/tinycorelinux/
https://mirror.epn.edu.ec/tinycorelinux/
https://mirrors.dotsrc.org/tinycorelinux/
http://distro.ibiblio.org/tinycorelinux/
Title: Re: fast way of syncing local mirror to official TCL repository?
Post by: GNUser on February 14, 2022, 02:18:45 PM: Hi, Rich. Thanks for that, but none of these support rsync. Also, I'd rather sync directly to the official repo because I don't know how up-to-date a given third-party mirror is.
Title: Re: fast way of syncing local mirror to official TCL repository?
Post by: patrikg on February 14, 2022, 02:23:28 PM: I Don't know if this repo list is working but you have to test.

http://forum.tinycorelinux.net/index.php/topic,21478.msg134397.html#msg134397
Title: Re: fast way of syncing local mirror to official TCL repository?
Post by: GNUser on February 14, 2022, 02:27:08 PM: Thanks but again, as long as the TCL developers continue to allow local/private mirrors to be synced directly to the official repo, that would be my preference (as stated in the title of the thread).
Title: Re: fast way of syncing local mirror to official TCL repository?
Post by: GNUser on February 14, 2022, 02:48:59 PM: I tried rsync'ing to official repo from a different machine with a different internet connection. Speeds shot up to several MB/s.

The machine that serves as my local mirror accesses the internet through a VPN tunnel. When I configure firewall and route so that official TCL repo access bypasses the tunnel, problem goes away.

So it doesn't seem like there's anything wrong with the official repo, after all :) Sorry for the noise.
Title: Re: [Solved] fast way of syncing local mirror to official TCL repository?
Post by: Rich on February 14, 2022, 03:52:10 PM: Hi GNUser
It's not noise if the source of the issue can be identified and a solution can be found.
Title: Re: [Solved] fast way of syncing local mirror to official TCL repository?
Post by: GNUser on February 14, 2022, 06:22:54 PM: Quote from: Rich on February 14, 2022, 03:52:10 PM
Hi GNUser
It's not noise if the source of the issue can be identified and a solution can be found.
Well, I'm glad you see it that way :D

The solution in this case may help other users with a similar kind of problem. The solution involved bypassing the wireguard VPN tunnel for the mirror operation. (Why my VPN tunnel slows down the mirror operation is not clear to me--99% of the time the VPN tunnel does not cause any noticeable slowdown. It seems to be something specific to the TCL repo server.)

Here are the relevant parts of the shell script that I use to sync my mirror:
[Attached due to forum error]

Happy hacking!
Title: Re: [Solved] fast way of syncing local mirror to official TCL repository?
Post by: patrikg on February 15, 2022, 02:05:19 AM: You have to traceroute outside to the vpn tunnel entry point to determine and from inside to the repo to figure out where the bottleneck is.