Tiny Core Linux

Tiny Core Base => TCB Tips & Tricks => Topic started by: Sashank999 on August 02, 2020, 09:25:02 AM

Title: Apps downloading is pretty slow
Post by: Sashank999 on August 02, 2020, 09:25:02 AM
Hi !

I usually have to download some tczs from Apps so that I can compile some things. But these downloads are pretty slow. The download speed of mine is 10 Mbps and I think that is too much for a 16mb tcz(icu65-dev for vte-dev). I don't know if it is the repo's limit or my problem. What can I do to increase the network speed for repo downloads ?

The apps actually checks all the dependencies of a package so that the package loads good. But it is freaking slow. When I download a new tcz that has a lot of deps. Can we configure the Apps to maintain 5-6 subprocesses so that it can start loading or checking deps fast ?
Title: Re: Apps downloading is pretty slow
Post by: curaga on August 02, 2020, 01:09:40 PM
Pick a mirror near you.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 02, 2020, 01:12:33 PM
Hi Sashank999
Quote from: Sashank999 on August 02, 2020, 09:25:02 AM
...  Can we configure the Apps to maintain 5-6 subprocesses so that it can start loading or checking deps fast ?
Dependencies get resolved in a linear fashion, meaning the lowest dependencies get loaded first and the extension you requested
gets loaded last. Even if you could parallelize it, creating the download list is so fast it would make no difference.

That leaves your connection speed. At 10 Mbps that would be approximately 1Mbyte/sec assuming you use all of your bandwidth
for the download. So the fastest you could download a 16Mbyte  .tcz  would be 16 seconds.
If there is other activity on your network, it will take longer.
If your router limits the percentage of bandwidth a connected computer can have, it will take longer.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 02, 2020, 10:35:18 PM
@curaga I ran that in Apps browser but it shows the "repo.tinycorelinux.net" as the nearest one.

@Rich Mine is actually a 20 Mbps Connection. But in OOKLA Speed Test it showed 10Mbps which gradually increased to 15 Mbps. Hence I reported it as 10 Mbps. What I wonder is why would it be 1Mbps when my speed 10Mbps ? I have a Home Router capable of supporting upto 500 Mbps. I have checked it and confirmed that there are no restrictions on bandwidth and that my TCL Laptop is the only device connected to it (except the router itself). I want only checking the deps faster.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 02, 2020, 10:56:43 PM
Hi Sashank999
Quote from: Sashank999 on August 02, 2020, 10:35:18 PM
... What I wonder is why would it be 1Mbps when my speed 10Mbps ? ...
Read it again:
Quote from: Rich on August 02, 2020, 01:12:33 PM
... That leaves your connection speed. At 10 Mbps that would be approximately 1Mbyte/sec ...
You are confusing Mega Bits with Mega Bytes. Network bandwidth is in Mega Bits. That bandwidth includes the data in your
packet, various headers attached to that packet, routing information, and any gap time required between transmitted packets.

A byte has 8 bits but to allow for the overhead mentioned above, I divide Mega Bits by 10 to estimate Mega Bytes.
Extension sizes are given in bytes, not bits.

    [EDIT]: Replaced the word  convert  with  estimate.  Rich
Title: Re: Apps downloading is pretty slow
Post by: TinyCoreLinux on August 02, 2020, 11:45:55 PM
Code: [Select]
tce-load -wil mirrors
tcemirror.sh
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 03, 2020, 02:41:03 AM
@Rich Oh. My bad. Looks like I reported something wrong. ISPs here use Mega Bytes. Not Megabits ;D. My reports are all in Mega bytes.

@NOOB Thanks for reminding me.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 03, 2020, 10:50:26 AM
Hi Sashank999
Although possible, it sounds unlikely. Look at any networking port (on a motherboard, router, network switch, etc.) and you will find
their bandwidth is always given in  Bits Per Second.  You stated:
Quote from: Sashank999 on August 02, 2020, 10:35:18 PM
...  But in OOKLA Speed Test it showed 10Mbps which gradually increased to 15 Mbps. ...
OOKLA defaults to  Mbps  which means  Megabits Per Second.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 03, 2020, 11:05:10 PM
Oh. Ok. My bad. But 1MegaByte per second means we should get download in 16 sec. Adding some more time to ping requests, some network adapter problem or DNS Server Traffic, we should get it atleast in 30 Sec. But it took 3 mins.

Now even that "Find Fastest Mirrir" stopped working. I don't know why.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 04, 2020, 08:56:16 AM
Hi Sashank999
Install  wget.tcz.  Then:
Code: [Select]
mkdir icu
cd icu
wget --debug --output-file=icu.txt http://repo.tinycorelinux.net/11.x/x86_64/tcz/icu65-dev.tczAttach the  icu.txt  file to your next post.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 04, 2020, 09:47:22 AM
Hey, I replaced icu65-dev with bash in the URL. I can't download the 16MB tcz again. The fastest mirror is now working. It shows the tinycore official repo as fastest.

I think we are deviating away from topic. I want to make Apps verify the deps and download the tcz faster.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 04, 2020, 11:06:33 AM
Hi Sashank999
Quote from: Sashank999 on August 04, 2020, 09:47:22 AM
... I think we are deviating away from topic. I want to make Apps verify the deps and download the tcz faster.
And I wanted to gather some more information to try to see what's going on. Is there a reason you are running IPv6? I think I remember
reading at one time that just having IPv6 enabled can slow some things down.

Try it again like this:
Code: [Select]
rm bash.tcz
wget --debug --output-file=icu.txt --inet4-only http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz
According to the file you attached:
Quote
2020-08-04 13:43:59 (61.9 KB/s) - 'bash.tcz' saved [442368/442368]
Your download speed was 0.062 Mega Bytes/sec. I would expect your connection to be at least 10 times faster.

So OOKLA connects at 10Mbps.
And wget connects at 0.62Mbps.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 06, 2020, 12:35:16 AM
Here is the file.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 06, 2020, 10:23:14 AM
Hi Sashank999
Looks faster now:
Quote
2020-08-06 04:34:35 (380 KB/s) - 'bash.tcz' saved [442368/442368]
Your download speed was 0.380 Mega Bytes/sec, about 6.1 times faster than the last time.

So comparing apples to apples:
OOKLA connected at 10Mbps.
This time wget connected at 3.8Mbps.

Run this command:
Code: [Select]
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1That should disable IPv6.

Then one more speed test please:
Code: [Select]
rm bash.tcz
wget --debug --output-file=icu.txt http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 07, 2020, 10:14:57 PM
Quote from: Rich on August 06, 2020, 10:23:14 AM
Run this command:
Code: [Select]
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1That should disable IPv6.
That says :
Code: [Select]
sysctl: error: 'net.ipv6/conf/all/disable_ipv6' is an unknown key.
Another test attached.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 07, 2020, 10:24:47 PM
Hi Sashank999
Quote from: Sashank999 on August 07, 2020, 10:14:57 PM
That says :
Code: [Select]
sysctl: error: 'net.ipv6/conf/all/disable_ipv6' is an unknown key.
Those should have been periods, not slashes.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 08, 2020, 09:43:20 AM
I can't execute that command. The result is the same even when I copy paste your command.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 08, 2020, 09:43:00 PM
Hi Sashank999
Please run this:
Code: [Select]
sudo sysctl -a > sysctl.txt 2>&1Then attach  sysctl.txt  to your next post.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 10, 2020, 01:45:17 AM
Attached.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 10, 2020, 10:54:17 AM
Hi Sashank999
That's odd, I see no instances of  ipv6  mentioned in the  sysctl.txt  file.

Lets see what the busybox version of  wget  is doing:
Code: [Select]
rm bash.tcz
time busybox wget -S http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz > icu.txt 2>&1
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 10, 2020, 11:59:08 AM
Attached.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 10, 2020, 01:24:24 PM
Hi Sashank999
I don't know what is going on, but I do not think it is a misconfiguration issue with Tinycore. I want to do a brief recap of observations
and follow up with a couple of questions/causes.

This command:
Code: [Select]
wget --debug --output-file=icu.txt http://repo.tinycorelinux.net/11.x/x86_64/tcz/icu65-dev.tczAppears to show wget trying to use IPv6 addressing, possibly because it's being offered an IPv6 address to use:
Reading HSTS entries from /home/tc/.wget-hsts
--2020-08-04 13:43:51--  http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz
Resolving repo.tinycorelinux.net... 89.22.99.37, 64:ff9b::5916:6325
Caching repo.tinycorelinux.net => 89.22.99.37 64:ff9b::5916:6325
Connecting to repo.tinycorelinux.net|89.22.99.37|:80... connected.

The download used only 6% of your connections bandwidth which is unacceptable.

This command:
Code: [Select]
wget --debug --output-file=icu.txt --inet4-only http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tczThis forces  wget  to use only IPv4 addresses, which it does:
Reading HSTS entries from /home/tc/.wget-hsts
--2020-08-06 04:34:34--  http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz
Resolving repo.tinycorelinux.net... 89.22.99.37
Caching repo.tinycorelinux.net => 89.22.99.37
Connecting to repo.tinycorelinux.net|89.22.99.37|:80... connected.

The download used 30% of your connections bandwidth which sounds reasonable.

The  busybox  version, which is used by  tce-load  and  Apps:
Code: [Select]
time busybox wget -S http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz > icu.txt 2>&1Does not provide the debug information that the  wget.tcz  version does.
It does appear to be more severely effected by what's going on since it used only 1% of your networks bandwidth.

I suspect this is an IPv6 issue, yet there are no instances of  ipv6  mentioned in the  sysctl.txt  file.

So where did those IPv6 addresses come from? 2 possibilities come to mind:
1. You are running in a virtual machine that is providing it.
2. Your router (or ISP) is defaulting to IPv6.

If anyone else has any other ideas, please chime in.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 11, 2020, 03:32:03 AM
Hi !
I went to https://test-ipv6.com/ (https://test-ipv6.com/) to test IPv6. Here are the results attached.
My TC is on a laptop. Not on a VM.
My ISP doesn't have the IPv6 support. He is soon going to do it.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 11, 2020, 11:09:06 AM
Hi Sashank999
Well, this IPv6 address is coming from somewhere:
Code: [Select]
Resolving repo.tinycorelinux.net... 89.22.99.37, 64:ff9b::5916:6325
Your screen shot mentions  DNS server ..... IPv6 Internet access.  So that IPv6 test detected something.

You could try deleting the IPv6 entries from your  /etc/hosts  file to see if that helps. Those are the entries that have  ::  in them.

Are you connecting through a tunnel that could be doing some IPv6 stuff?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 11, 2020, 11:38:27 AM
Hi !

Tunneling ? Do you mean proxies or VPNs ? No, I don't use those for Tiny Core.

Once look at the /etc/hosts file. I am removing all the entries with ::. I didn't edit anything in the host file till now. I will then execute this :
Code: [Select]
wget --debug --output-file=bash-inet4.txt --inet4-only http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz
wget --debug --output-file=bash.txt http://repo.tinycorelinux.net/11.x/x86_64/tcz/bash.tcz.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 11, 2020, 01:05:38 PM
Hi Sashank999
Both of those attachment showed the same (slow) speed. So maybe it's not an IPv6 issue.

Is your ISP a cable TV provider? Does the network connection come into your house through coax (copper) or fiber optic cable?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 11, 2020, 10:40:22 PM
Hi !

Yes, my Cable Provider is also my ISP.
Network comes through Optic Fiber.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 11, 2020, 11:43:29 PM
Hi Sashank999
I'm afraid I'm running out of ideas. You should not have speed issues connected through fiber optic cable.

Post the results of:
Code: [Select]
ifconfig -a > ifconfig.txt
You might also want to Google for:
Code: [Select]
slow internet "your ISP"and:
Code: [Select]
slow internet "your MODEM"where  "your ISP"  is the company that provides your Internet connection and  "your MODEM"  is the make and model number of the MODEM
that connects you to the Internet.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 12, 2020, 03:46:36 AM
Hi !

ifconfig.txt attached.

Those Google searches did nothing.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 12, 2020, 08:06:06 AM
Hi Sashank999
That shows you are using a wireless connection, but I also see this:
eth0      Link encap:Ethernet  HWaddr 60:EB:69:E0:55:A4  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
Your wired Ethernet device is listed as being  UP  even though it hasn't been assigned an IP address.

It's a long shot, but try taking down  eth0  and see if that speeds things up:
Code: [Select]
sudo ifconfig eth0 down
Title: Re: Apps downloading is pretty slow
Post by: ashfame on August 12, 2020, 08:55:29 AM
Hi guys,

The repo has been acting slow for me as well. Using a linux box from Linode.

I ran the same command you suggested a few posts earlier:
Code: [Select]
wget --debug --output-file=icu.txt http://repo.tinycorelinux.net/11.x/x86_64/tcz/icu65-dev.tcz
I can't make out anything from it: https://pastebin.com/4nrKXum5
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 12, 2020, 10:14:16 AM
Hi ashfame
Quote from: ashfame on August 12, 2020, 08:55:29 AM
... I can't make out anything from it: https://pastebin.com/4nrKXum5

From your file:
Code: [Select]
     0K .......... .......... .......... .......... ..........  0% 42.5K 6m30s
    50K .......... .......... .......... .......... ..........  0% 31.5K 7m37s
   100K .......... .......... .......... .......... ..........  0% 40.5K 7m19s
   150K .......... .......... .......... .......... ..........  1% 18.9K 9m6s
   200K .......... .......... .......... .......... ..........  1% 14.6K 11m0s
   250K .......... .......... .......... .......... ..........  1% 9.73K 13m48s
   300K .......... .......... .......... .......... ..........  2% 12.5K 14m54sColumn 1 indicates how much of the download has completed in 50K byte increments.
Columns 2 through 6 are groups of 10 dots. Each dot represents 1K byte of data downloaded.
Column 7 indicates the percentage of the total file that has been downloaded.
column 8 gives the speed at which that 50K bytes of data was downloaded at in K bytes per second.
               Network speeds can fluctuate for a variety of reasons.
column 9 is an estimate of the time required to complete the download at the current speed.

Going to the end of the file:
Code: [Select]

 16500K .......... .......... .......... .......... .......... 99% 29.4K 5s
 16550K .......... .......... .......... .......... .......... 99% 52.4K 2s
 16600K .......... .......... .......... ..........           100% 43.9K=15m7s

2020-08-12 10:17:52 (18.3 KB/s) - 'icu65-dev.tcz' saved [17039360/17039360]Looking at the line that says 100% we find:
Columns 2 through 5 contain 10 dots each and column 6 is blank indicating the last block of data was 40K bytes in size.
Column 9 has a new meaning on this line. It now displays how long it took to download the file, 15m7s.

The last line shows the average download speed of the file, 18.3 K bytes per second.
The "saved [17039360/17039360]" means  [# of bytes saved / file size in bytes].
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 12, 2020, 10:56:25 AM
After shutting eth0 down, I got this output attached.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 12, 2020, 11:21:31 AM
Hi Sashank999
I can only think of one other thing to try. Don't initialize wlan0. Bring up eth0 instead and connect a LAN cable between your machine
and one of the modems LAN ports. See if that gets you any speed.
Title: Re: Apps downloading is pretty slow
Post by: ashfame on August 12, 2020, 04:00:35 PM
Hi Rich,

Thanks a lot for your explanation! :)
So it took 14m54s to download that single extension, why would it be this slow?

Maybe something at the server to throttle certain connections?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 12, 2020, 10:34:09 PM
Hi Rich,

I am sorry but I can't do that. Mine is a Set-Top-Box(Sorry again if you don't know that) + Modem. It has only one LAN port which is actually used to supply internet to the STB+Modem itself.

So, I think we better go to dependency checking speed now.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 12, 2020, 11:15:40 PM
Hi ashfame
Quote from: ashfame on August 12, 2020, 04:00:35 PM
... So it took 14m54s to download that single extension, ...
No it took 15m7s. I updated this line in my answer to reflect that:
Quote
Column 9 has a new meaning on this line. It now displays how long it took to download the file, 15m7s.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 12, 2020, 11:19:03 PM
Hi Sashank999
Quote from: Sashank999 on August 12, 2020, 10:34:09 PM
... Mine is a Set-Top-Box(Sorry again if you don't know that) + Modem. ...
Do you have a make and model number for that?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 13, 2020, 01:17:22 AM
Yes.
Title: Re: Apps downloading is pretty slow
Post by: curaga on August 13, 2020, 02:37:22 AM
No, shouldn't be anything at the server. I'm getting 1.5Mbps for example. Could be something in between, lots of folks Netflixing. ashfame, you should also pick your closest mirror if you haven't.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 20, 2020, 11:46:10 AM
I think I found the problem. My ISP is actually blocking some servers due to some problems. But those servers are in the routing tables to send the data to the server. Hence I can't access the repo or the forum properly(which I noticed with `traceroute` command). Hence I am now using a VPN. But it is unreliable. I have attached a report of 2 commands output.

Requesting Rich to review and say an alternative.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 22, 2020, 11:37:22 PM
Please help me. The forum loads frustatingly slow. Each page refresh takes > 1 min and posting a new message in forums say that my PHP Session ID is invalid (as it got timed out).

Atleast the repo is with moderate speed.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 23, 2020, 12:53:09 AM
Hi Sashank999
I don't think there's anything we can do if your ISP is blocking some of your traffic.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 23, 2020, 06:00:51 AM
But there might be some other way, routing traffic to forums through some other servers as this server blocks direct traffic. Do you know how to do it ?
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 23, 2020, 08:15:03 AM
Hi Sashank999
I suppose we could try changing your DNS server. What does this return:
Code: [Select]
cat /etc/resolv.conf
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 24, 2020, 01:59:19 AM
Hi !
Code: [Select]
tc@box:~$ cat /etc/resolv.conf
nameserver 192.168.55.1which makes me think that it queries my router which queries DNS given by my ISP.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 24, 2020, 02:12:53 AM
Hi Sashank999
Yup, that's what it does. Try changing the contents to this:
Code: [Select]
nameserver 8.8.4.4or:
Code: [Select]
nameserver 4.2.2.4
Then see if things run any faster.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 24, 2020, 03:06:06 AM
It didn't work as the file again changed nameserver to 192.168.55.1 .
Title: Re: Apps downloading is pretty slow
Post by: aus9 on August 24, 2020, 04:06:22 AM
Sashank999

if interested, my gateway is diff to yours. for me
Code: [Select]
cat /etc/resolv.conf
search localhost
nameserver 192.168.1.1If you log into your router....hopefully you have a page for network settings (DLINK)
LAN IP address matches my above number
and I have DNS relay enabled......and I believe that takes me to ISP DNS
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 24, 2020, 10:45:25 AM
Hi Sashank999
Change the contents of the file. Don't forget to use the save command in the editor.

Do not reboot.
Do not restart your wireless adapter.

The changes should be detected immediately. Try both of those DNS servers and see if either on improves your Internet connection.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 24, 2020, 12:07:40 PM
Hi Rich,

I have done as you said. Edited it and confirmed that it has the saved change in nameserver with cat /etc/resolv.conf. I did not restart my wifi adapter or reboot. The resolv.conf automatically changes.

@aus9

But I have a DASAN Router which is configured to use 192.168.55.1 as the router page.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 24, 2020, 12:39:07 PM
Hi Sashank999
So with the changes, is your web browsing any better? Are your downloads any faster?
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 24, 2020, 01:41:31 PM
Quote from: Sashank999 on August 24, 2020, 12:07:40 PM
The resolv.conf automatically changes.

Hi, Sashank999. I agree with Rich that trying a different DNS server is an important step in troubleshooting your issue. A slow DNS server can have a dramatic impact on page load speed. I've experienced this issue myself.

Some network managers try to be "helpful" and automatically change the contents of /etc/resolv.conf. Openvpn sometimes also changes /etc/resolv.conf without your permission. There are different ways to prevent unauthorized changes to /etc/resolv.conf. An easy way that will work regardless of your setup is to make the file immutable. Go ahead and edit /etc/resolv.conf to your liking, then do this:

Code: [Select]
$ tce-load -wi e2fsprogs
$ sudo chattr +i /etc/resolv.conf
Now the file will be left in peace (until you reboot or run sudo chattr -i /etc/resolv.conf) and you can see if the DNS server change helps your issue.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 24, 2020, 02:08:15 PM
Hi Sashank999
Quote from: Sashank999 on August 24, 2020, 12:07:40 PM
... The resolv.conf automatically changes. ...
Did I misunderstand you?

Are you saying that the  /etc/resolv.conf  file changes back to  192.168.55.1  even though you did not reboot or restart your
wireless adapter?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 25, 2020, 08:04:57 AM
@GNUser
Thank You for the suggestion. I did it. But then chattr replied with :
Code: [Select]
chattr: Inappropriate ioctl for device while reading flags on /etc/resolv.confSo, I added -f flag(help by Google). Then it replied no messages and hence I thought all was good. Then again, it changed back to its original. I think it deleted the file and made another one instead of it.
I use no network managers. I just do
Code: [Select]
sudo wifi.sh -a & in my aterm and then close aterm and open firefox.

@Rich
Yes. That is what I mean. It just changes back to its original form.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 25, 2020, 09:20:23 AM
Hi Sashank999
Does this command produce any output:
Code: [Select]
cat /opt/wlan0.sh
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 25, 2020, 09:43:13 AM
Sashank999, sorry chattr didn't work. Maybe your filesystem doesn't support it.

Try adding your desired DNS servers to the top of /usr/share/udhcpc/default.script like this, for example:

Code: [Select]
dns="1.1.1.1 1.0.0.1"
Then running wifi.sh
Title: Re: Apps downloading is pretty slow
Post by: jazzbiker on August 25, 2020, 12:13:37 PM
Hi, GNUser!

Handy tip, thanks in advance! Currently I'm not suffering from DNS losses, but who knows, what's behind the corner.
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 25, 2020, 02:14:57 PM
Hi, Shashank999. I figured out the problem with chattr: In TCL, the root file system (including /etc/resolv.conf) exists only in RAM. Apparently ext3 and ext4 filesystems (and perhaps also other filesystems on a physical harddrive) support the immutable flag, but RAM filesystem does not.

Assuming your /opt directory is persistent, this will work (if your /home is persistent but not /opt, adjust the steps accordingly):

Code: [Select]
$ cp /etc/resolv.conf /opt/resolv.conf
$ sudo rm /etc/resolv.conf
-> edit /opt/resolv.conf to taste
$ sudo chattr +i /opt/resolv.conf
$ sudo ln -s /opt/resolv.conf /etc/resolv.conf
Now your DNS settings are immutable 8) (until you reboot or run sudo chattr -i /opt/resolv.conf).
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 25, 2020, 02:36:30 PM
You're welcome, jazzbiker. I hope it comes in handy sometime.

The trick with /usr/share/udhcpc/default.script is specific to DNS issues being caused by DHCP. The chattr trick has more general application--you can use it whenever you want to prevent unauthorized edits to a file (it is stronger than making the file read-only using chmod).
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 25, 2020, 11:44:51 PM
@Rich
Code: [Select]
cat /opt/wlan0.sh
#!/bin/sh
pkill udhcpc
udhcpc -b -i wlan0 -x hostname:box -p /var/run/udhcpc.eth0.pid
@GNUser
Even my /opt dir is not persistent so chattr doesn't work :( .
Quote from: GNUser on August 25, 2020, 09:43:13 AM
Try adding your desired DNS servers to the top of /usr/share/udhcpc/default.script like this, for example:
Code: [Select]
dns="1.1.1.1 1.0.0.1"Then running wifi.sh
After also reading http://forum.tinycorelinux.net/index.php?topic=8019.0
OMG That worked !!! Now the file remained with dns at 8.8.8.8 and 8.8.4.4 but sadly, even then the forum doesn't load.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 26, 2020, 08:12:53 AM
Hi Sashank999
If you used this:
Code: [Select]
dns="8.8.8.8 8.8.4.4"It will not try  8.8.4.4  unless it can not connect to  8.8.8.8.  Those are both Google DNS servers so they probably see a lot of traffic.

I found that for me page loads were much faster with  4.2.2.4  than  8.8.4.4.

Also, when was the last time you gave your router a hard reboot. On occasion, I found if my connection slowed down, unplugging
the routers power cord for 10 seconds helped.

Quote from: Sashank999 on August 24, 2020, 12:07:40 PM
... But I have a DASAN Router ...
What's the model number?
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 26, 2020, 08:22:43 AM
Sashank999,
It would be good to confirm that this is a TCL issue and not something else (e.g., hardware issue, router issue, or ISP issue).
If you boot a different OS on the same machine, does the problem go away?
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 27, 2020, 09:24:35 AM
@Rich
I gave hard reboot. But it doesn't go away. My router model number is "H662GS", manufactured by "DASAN Network Solutions".

@GNUser
Yes, this is not only a TCL issue. I went to other OS (Windows 10 and Phoenix OS(android x86 derivative)) and the forums load slow even in them. I was actually here to propose that Apps should verify deps by running multiple subprocess for its direct dependencies(not whole tree file, only dep file) so that it can load them faster.
Title: Re: Apps downloading is pretty slow
Post by: GNUser on August 27, 2020, 09:39:10 AM
Quote from: Sashank999 on August 27, 2020, 09:24:35 AM
...this is not only a TCL issue...
Well, then you need to exclude hardware, router, and ISP issues (e.g., if other machines connected to same wireless router have same problem, then it's a router or ISP issue; if a wired connection has same problem, then it's probably an ISP issue; etc). Until you can pinpoint the problem, it's counterproductive to search for a solution.

Quote from: Sashank999 on August 27, 2020, 09:24:35 AM
I was actually here to propose that Apps should verify deps by running multiple subprocess for its direct dependencies(not whole tree file, only dep file) so that it can load them faster.
Oops, sorry. I guess I missed the point of the thread. TCL is very much a community project and encourages a do-it-yourself approach. The developers welcome patches.
Title: Re: Apps downloading is pretty slow
Post by: curaga on August 27, 2020, 12:52:30 PM
Multiple downloads in parallel from a single server is considered rude, and it may not even help your case. The throttling you see may not be per-connection but per-area.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 28, 2020, 12:31:53 AM
@Curaga
I am not saying "Multiple thread downloads". I am saying to verify the already downloaded dependency tczs in multiple processes.
For example, lets take "ffmpeg4".tcz. It contains a lot of tree dependencies. So, to verify them faster, creating subprocesses to tree dependencies may help.
Title: Re: Apps downloading is pretty slow
Post by: curaga on August 28, 2020, 02:45:52 AM
What do you mean verify? md5sum? It's quite fast...
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 28, 2020, 08:32:41 AM
I forgot to include the word "loading" beside verifying. My bad.

I mean to run subprocesses for loading and verifying. It would be really fast.
It is similar to "make -j2" - dividing the load of compiling to 2 processors. Just like that I propose running subprocesses to load dependencies recursively.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 28, 2020, 09:50:31 AM
Hi Sashank999
Doing that means those processes would be running asynchronously in the background, creating a race condition. This will cause
problems for cases when the order of extension loading matters, such as  graphics-KERNEL  and  alsa-config.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 29, 2020, 05:45:16 AM
Oh. I didn't know that.
Is there a way to add timeout option for tce-load or wget ? Or else a retry option for wget ?
Sometimes the connection to repo just breaks in the middle (happens like once or twice a month, so no bother). But when it breaks when I am downloading an extension that has like 100 tree deps (for example) it would take another 15 minutes for the Apps or tce-load to do the whole thing again. Hence I am asking this.
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 29, 2020, 09:49:31 AM
Hi Sashank999
Quote from: Sashank999 on August 29, 2020, 05:45:16 AM
... Is there a way to add timeout option for tce-load or wget ? Or else a retry option for wget ? ...
wget has both of those options (-T  and  -c):
Code: [Select]
tc@E310:~$ busybox wget --help
BusyBox v1.29.3 (2018-12-19 15:29:37 UTC) multi-call binary.

Usage: wget [-c|--continue] [--spider] [-q|--quiet] [-O|--output-document FILE]
        [--header 'header: value'] [-Y|--proxy on/off] [-P DIR]
        [-S|--server-response] [-U|--user-agent AGENT] [-T SEC] URL...

Retrieve files via HTTP or FTP

        --spider        Only check URL existence: $? is 0 if exists
        -c              Continue retrieval of aborted transfer
        -q              Quiet
        -P DIR          Save to DIR (default .)
        -S              Show server response
        -T SEC          Network read timeout is SEC seconds
        -O FILE         Save to FILE ('-' for stdout)
        -U STR          Use STR for User-Agent header
        -Y on/off       Use proxy
tc@E310:~$
If you search the  tce-*  scripts:
Code: [Select]
tc@E310:~$ grep wget /usr/bin/tce-*
/usr/bin/tce-audit:             wget -cq -P /tmp "$MIRROR"/"$TARGET" 2>/tmp/fetch_result
/usr/bin/tce-fetch.sh:  wget -cq -O- "$MIRROR"/"${1//-KERNEL.tcz/-${KERNELVER}.tcz}"
/usr/bin/tce-fetch.sh:  wget -cq "$MIRROR"/"$F"
/usr/bin/tce-load:      wget -cq "$MIRROR"/"$1".md5.txt 2>/dev/null
/usr/bin/tce-load:      wget -c "$MIRROR"/"$1"
/usr/bin/tce-load:                                      if (system("rm -f "depfile"; wget -c -P "optional" "mirror"/"name".dep 2>/dev/null") == 0 && ! SUPPRESS)
/usr/bin/tce-status:                    busybox wget --spider -q "$MIRROR"/"$E" 2>/dev/null || echo "$E not found!"
/usr/bin/tce-update:    wget -cq "$MIRROR"/"$1"
/usr/bin/tce-update:            wget -cq "$MIRROR"/"$1" # Either there was no zsync file on the server, or it failed
tc@E310:~$you'll see it uses the  -c  option for wget. You can add a  -T  option to the commands above if you think it will help you. Then add
those 4 files to your backup.

The timeout is being used to make sure the command does not hangup forever, not to try to make intermittent network connections
more tolerable. According to the  busybox  config file, the timeout for wget should default to  10 seconds.

Quote
... Sometimes the connection to repo just breaks in the middle (happens like once or twice a month, so no bother). But when it breaks when I am downloading an extension that has like 100 tree deps (for example) it would take another 15 minutes for the Apps or tce-load to do the whole thing again. ...
If  tce-load  gets interrupted, it resumes with the file that failed to download when you run it again.
Title: Re: Apps downloading is pretty slow
Post by: Sashank999 on August 30, 2020, 05:37:55 AM
I said "to do the whole thing", not downloading. It takes time in verifying tree deps from start if interrupted, not in downloading. I already know that it continues previous downloads (which helped me when I did compiletc).

I think I should study those tce-* scripts. Thanks to the devs for using ash instead of C or C++ for tce- scripts (Idk C or C++).
Title: Re: Apps downloading is pretty slow
Post by: Rich on August 30, 2020, 08:40:39 AM
Hi Sashank999
Quote from: Sashank999 on August 30, 2020, 05:37:55 AM
... Thanks to the devs for using ash instead of C or C++ for tce- scripts (Idk C or C++).
You will probably also find some  awk  in those scripts.