WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: tce-load -wi stalling out on big files?  (Read 2958 times)

Offline MikeLockmoore

  • Hero Member
  • *****
  • Posts: 525
  • Good software needn't be big!
tce-load -wi stalling out on big files?
« on: December 31, 2010, 10:38:44 AM »
Hi.  I've recently upgraded to TC 3.4 on one of my PCs.  When trying to load the extensions I normally use, I had very frequent failures in AppBrowser to fetch larger-size extension files (> 2 MB).  I downloaded a few through wget with the full path just fine.  I tried to run tce-load -wi (extensionname) on the command line, and see something like this each time:
Code: [Select]
tc@box:~$ tce-load -wi foxit_reader.tcz
Downloading: foxit_reader.tcz
Connecting to distro.ibiblio.org (152.46.7.109:80)
foxit_reader.tcz      42% |*************                  |  1699k 00:00:06 ETAwget: short write
foxit_reader.tcz: FAILED
md5sum: WARNING: 1 of 1 computed checksums did NOT match
Error on foxit_reader.tcz
If I repeat this, it seems to be happening at different points, usually between 1.5 and 2 MB into the file.  This happens a few seconds into the process.  I don't remember having any big problems like this setting up TC 3.3 several times. 

For 3.4, I started with no extensions except the wireless kernel extension and wireless_tools.tcz (so I could get online without a ethernet wire).  Maybe there is something in my backup/restore data that is interfering, but I don't know where to look.  Any ideas?
--
Mike Lockmoore

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10957
Re: tce-load -wi stalling out on big files?
« Reply #1 on: December 31, 2010, 10:42:06 AM »
Short write? Are you perhaps running out of disk space?
The only barriers that can stop you are the ones you create yourself.

Offline MikeLockmoore

  • Hero Member
  • *****
  • Posts: 525
  • Good software needn't be big!
Re: tce-load -wi stalling out on big files?
« Reply #2 on: January 02, 2011, 06:58:36 PM »
@curaga: No, I have enough space for many files of that size.  Also, I can use wget at the console to retrieve the file successfully, then successfully tce-load -i on that file after I have a copy locally.   ???  Not fatal, but also not so fun to do several times.  
« Last Edit: January 02, 2011, 07:01:59 PM by MikeLockmoore »

Offline maro

  • Hero Member
  • *****
  • Posts: 1228
Re: tce-load -wi stalling out on big files?
« Reply #3 on: January 02, 2011, 07:28:47 PM »
Hmmm, sounds like a networking problem. Maybe you could use soemthing like sudo tcpdump -i eth0 (using the 'tcpdump.tcz' extension) to get a higher-level idea where things break down.

Obviously there are options to increase the verbosity, or one could use 'wireshark'. But I just think you'll run out of memory if you try to capture any larger download with 'wireshark'.

Offline MikeLockmoore

  • Hero Member
  • *****
  • Posts: 525
  • Good software needn't be big!
Re: tce-load -wi stalling out on big files?
« Reply #4 on: January 03, 2011, 06:54:18 AM »
@maro:  I loaded the tcp filter and and used AppBrowser again to run tce-load -wi.  I'm not highly familiar with the protocols involved, but the packet log appears to contain a bunch of normal requests and acks up to the point it fails.  The last request does not seem to be satisfied.  Here is the tail end of the packet log:
Code: [Select]
09:48:30.195815 IP {{MY_IP_ADDR}}.42586 > jungle.metalab.unc.edu.www: R 2593577276:2593577276(0) win 0
09:48:30.197321 IP jungle.metalab.unc.edu.www > {{MY_IP_ADDR}}.42586: . 2329833:2331281(1448) ack 122 win 46 <nop,nop,timestamp[|tcp]>
09:48:30.197330 IP {{MY_IP_ADDR}}.42586 > jungle.metalab.unc.edu.www: R 2593577276:2593577276(0) win 0
09:48:30.200233 IP jungle.metalab.unc.edu.www > {{MY_IP_ADDR}}.42586: . 2331281:2332729(1448) ack 122 win 46 <nop,nop,timestamp[|tcp]>
09:48:30.200243 IP {{MY_IP_ADDR}}.42586 > jungle.metalab.unc.edu.www: R 2593577276:2593577276(0) win 0

UPDATE: I tried my older TC 3.3 installation with AppBrowser for the same big extension, and it also failed, so it is not an issue with TC 3.4 specifically.  I tried the command line "tce-load -wo minefield.tcz" and it failed in the same way as my original post.  I also tried wget at it succeeded to fetch the complete file.  So there is something different about the way that tce-load works and wget.  Or perhaps something different about how my ISP is handling the traffic from tce-load versus a plain instance of wget.  But tce-load uses wget.   :-\

Does the -c option used with wget in tce-load cause this to get messed up?  Maybe I'll try... nope.  Removing just the -c options in tce-load did not make it work either.  I'm still stumped.  ???

« Last Edit: January 03, 2011, 07:14:39 AM by MikeLockmoore »

Offline maro

  • Hero Member
  • *****
  • Posts: 1228
Re: tce-load -wi stalling out on big files?
« Reply #5 on: January 03, 2011, 05:12:21 PM »
A rather interesting puzzle. As 'tce-load' does in essence a 'busybox wget -c URL' (with URL e.g. 'http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/tcz/minefield4.tcz' and ignoring the possible use of the '-q' option) one wonders what version of 'wget' you are using that seems to work. I mean, could that be the GNU 'wget' from the 'wget.tcz' extension?

If I would be in this situation I'd probably install 'wget.tcz' and test the following four cases (whilst 'tcpdump' is keeping a watchful eye):
    busybox wget URL
    busybox wget -c URL
    /usr/local/bin/wget URL
    /usr/local/bin/wget -c URL

Offline MikeLockmoore

  • Hero Member
  • *****
  • Posts: 525
  • Good software needn't be big!
Re: tce-load -wi stalling out on big files?
« Reply #6 on: January 04, 2011, 06:49:15 AM »
I'm fairly sure that I am just using the wget that is included in TC Base, which would be the busybox built-in wget.  This problem was noticed when I was building up a new TC 3.4 installation.  Well, old personal backup/restore files, but a new set of extensions.  Maybe I can try the experiment later, but for the next several days, I am on a business trip and don't have that computer where I see the problem with me.  I don't think I had a problem with the computer I am typing this on, but maybe I'll try this one again too.