Re: Tiny Core v17.0 upgrade issues

[Rich]

Hi Stefann

Code: [Select]

tce-load -wi advcomp

[Stefann]

Hi Stefann
Install  advcomp.tcz.

Almost!…… This worked:

Code: [Select]

tc@hp510:~$ tce-load -wi advcomp.tczBut thanks for the tcz name. I would not have found that otherwise

But no luck…

Code: [Select]

tc@hp510:/krubo/work/TC/linux-6.18.28/arch/x86/boot$ advdef -z4 bzImage
File type not supported on bzImage [at void convert_inplace(const string&):redef.cc:498]
Note…
Probably no activity anymore today.
As said I’m on a trip.
Social obligations now and sleep after that (living in Europe/netherlands)

[Paul_123]

It wants the file name to end in .gz    But if you did not have advcomp.tcz installed when compiling the kernel, then it did not get fully compressed.

With advcomp installed just remove the old bzImage from arch/x86/boot.    Then run "make bzImage" again

It should go faster as it should not have to recompile most of the code from the last run.

[Stefann]

It wants the file name to end in .gz    But if you did not have advcomp.tcz installed when compiling the kernel, then it did not get fully compressed.

With advcomp installed just remove the old bzImage from arch/x86/boot.    Then run "make bzImage" again

It should go faster as it should not have to recompile most of the code from the last run.

Done, but not really different size

Code: [Select]

tc@hp510:/krubo/work/TC/linux-6.18.28/arch/x86/boot$ ls -l bz*
-rw-r--r--    1 tc       staff      6312448 May 28 21:43 bzImage
Ehh… I get in social trouble if I spend more time on this now :-)
Any input will be followed up tomorrow

[Paul_123]

Lets look at compressed and uncompressed kernel.  Does vmlinux.bin match your build?

Code: [Select]

tc@Devel:~/linux-6.18.28/arch/x86/boot/compressed$ ls -l vmlinux*
-rwxr-xr-x 1 tc staff  6107368 May 29 23:38 vmlinux
-rwxr-xr-x 1 tc staff 12469728 May 29 23:38 vmlinux.bin
-rw-r--r-- 1 tc staff  6046727 May 29 23:38 vmlinux.bin.gz
-rw-r--r-- 1 tc staff     3077 May 29 23:38 vmlinux.lds
-rw-rw-r-- 1 tc staff     2308 May  8 06:40 vmlinux.lds.S
-rw-r--r-- 1 tc staff   666652 May 29 23:38 vmlinux.relocs


[Stefann]

Here you go…
Looks like you are right (but please check, I don’t know whether I compare the correct files)

Compressed vmlinux and vmlinux.bin.gz are different
But uncompressed other files have same size.

That’s good!
If it’s the compressing I doubt that that would cause the crashing

Code: [Select]

tc@hp510:/krubo/work/TC/linux-6.18.28/arch/x86/boot/compressed$ ls -l vm*
-rwxr-xr-x    1 tc       staff      6308072 May 27 12:42 vmlinux
-rwxr-xr-x    1 tc       staff     12469728 May 27 12:42 vmlinux.bin
-rw-r--r--    1 tc       staff      6248050 May 27 12:42 vmlinux.bin.gz
-rw-r--r--    1 tc       staff         3077 May 27 12:42 vmlinux.lds
-rw-r--r--    1 tc       staff         2308 May  8 08:40 vmlinux.lds.S
-rw-r--r--    1 tc       staff       666652 May 27 12:42 vmlinux.relocs
On the crashing…
As said I’m now on a trip AND have a major 1wire hardware breakdown to fix on return.
So… expect things to go a little bit slower…

With that said…. I will more thorough test whether recompiled kernel with usb-serial-6.18.2-tinycore.tcz works.
If indeed my “kernel making” is identical to your “kernel making” than my custom kernel should be identical to the published kernel and should crash. In that case the crashing is related to usb-serial “included in kernel versus by kext”.
That is entirely possible because I included only a few modules in the kernel while the usb-serial-6.18.2-tinycore.tcz has much more.

[Stefann]

small update..
- I returned from my trip
- located 1wire HW issue, the usb-to-1wire interface adapter was broken so the full network was down. I luckily had a spare. It's running again (hurray!)

I picked up the TC17 trouble shooting on usb-serial:
- running on standard configuration self compiled TC17 6.18.2 kernel with usb-serial-6.18.2-tinycore.tcz
- started my application without serial read PLUS dedicated crash-test-program with 200x accelerated serial read

running under kernel-from-downloadlocation this crashed 2x on a row, after 14hr and 12hr.
So... This "should" also crash within at least 24hr
If so... usb-serial-6.18.2-tinycore.tcz would be suspect
But...... no speculation... just test...

[Stefann]

small update
After 24hr still no crash.
I was expecting a crash:
- kernel from TC17.0 download location + usb-serial-6.18.2-tinycore.tcz had crashed 2x in a row after 14hr and 12hr
- now self compiled TC17.0 kernel +usb-serial-6.18.2-tinycore.tcz had NO crash for 24hrs after which I manually stopped it.

However... I realized I had recompiled my application under TC15 because I had been running TC15 during my trip.
Already many weeks ago I had found that "application compiled under TC15 does not crash on TC17"

So...
As every little damn detail counts...
I decided to do a full clean start.

deleted self compiled kernel and made a clean start for TC17.0 linux 6.18.2:

Code: [Select]

download http://tinycorelinux.net/17.x/x86/release/src/kernel/linux-6.18.2/config-6.18.2-tinycore
download http://tinycorelinux.net/17.x/x86/release/src/kernel/linux-6.18.2-patched.tar.xz

tc@hp510:/krubo/work/TC$ tar xvf  linux-6.18.2-patched.tar.xz
tc@hp510:/krubo/work/TC$ cd linux-6.18.2
tc@hp510:/krubo/work/TC/linux-6.18.2$
tc@hp510:/krubo/work/TC/linux-6.18.2$ cp ../config-6.18.2-tinycore .config
tc@hp510:/krubo/work/TC/linux-6.18.2$ make mrproper
  CLEAN   .config
tc@hp510:/krubo/work/TC/linux-6.18.2$ cp ../config-6.18.2-tinycore .config
tc@hp510:/krubo/work/TC/linux-6.18.2$ make oldconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
#
# configuration written to .config
#
tc@hp510:/krubo/work/TC/linux-6.18.2$ make bzImage
...................lots of logging................
tc@hp510:/krubo/work/TC/linux-6.18.2$

tc@hp510:/krubo/work/TC/linux-6.18.2$ cd arch/x86/boot
tc@hp510:/krubo/work/TC/linux-6.18.2/arch/x86/boot$ ls -l bz*
-rw-r--r--    1 tc       staff      6087168 Jun  2 17:00 bzImage

tc@hp510:/krubo/work/TC/linux-6.18.2/arch/x86/boot$ cd compressed
tc@hp510:/krubo/work/TC/linux-6.18.2/arch/x86/boot/compressed$ ls -l vm*
-rwxr-xr-x    1 tc       staff      6082840 Jun  2 17:00 vmlinux
-rwxr-xr-x    1 tc       staff     12428768 Jun  2 16:55 vmlinux.bin
-rw-r--r--    1 tc       staff      6025154 Jun  2 17:00 vmlinux.bin.gz
-rw-r--r--    1 tc       staff         3106 Jun  2 16:55 vmlinux.lds
-rw-r--r--    1 tc       staff         2296 Dec 18 14:03 vmlinux.lds.S
-rw-r--r--    1 tc       staff       663872 Jun  2 16:55 vmlinux.relocs
 
After that:

Code: [Select]

- copy to production machine
- reboot towards this new kernel
- recompile application
- recompile accelerated crash program
- start application
- start accelerated crash program
And now its running again....
It should crash in 12..14hr.
if not the "self recompile" is guilty (very very strange)
Note, self compiled kernel (vmlinuz17C2x) now has identical size to kernel from download area (vmlinuz):

Code: [Select]

tc@huis:/mnt/sda1/tce/boot$ ls -l vm*
-rwxrw-r--    1 tc       staff      6087168 Feb 10 13:10 vmlinuz
-rw-rw-r--    1 tc       staff      6087168 Jun  2 17:11 vmlinuz17C2xSo... apparently the install of advcomp.tcz fixed that

Lets wait...

[Stefann]

small update....
it's weird...
I'm running under self compiled TC17.0 kernel with usb-serial-6.18.2-tinycore.tcz, linux 6.18.2
I'm running my application without serial read
In parallel I'm running a dedicated crash-program with 200x accelerated read (reading blocks of 2 bytes versus 812byte blocks with my normal application)
I made this kernel "very clean" (see last post); I fully recompiled my application and crash program under this kernel.

It's now running 22.5hr without issue:

Code: [Select]

mon day:  6  2 | hh:mm:ss: 20:18:50 | Still alive after 0k + 1 loops | 0k + 2 bytes read
mon day:  6  3 | hh:mm:ss: 18:41:05 | Still alive after 31324k + 802 loops | 62588k + 776 bytes read
May 20-23 I was running this exact same test under kernel copied from iso from TC download location and it crashed 2x in a row after 14hr and 12hr.

Comparing kernels: vmlinuz from TC download location versus self compiled vmlinux17C2x:

Code: [Select]

tc@huis:/mnt/sda1/tce/boot$ ls -l vm*
-rwxrw-r--    1 tc       staff      6087168 Feb 10 13:10 vmlinuz
-rw-rw-r--    1 tc       staff      6087168 Jun  2 17:11 vmlinuz17C2x
tc@huis:/mnt/sda1/tce/boot$ cmp vmlinuz vmlinuz17C2x
vmlinuz vmlinuz17C2x differ: char 105, line 1So: size is exactly the same but they still differ

Anyway...
- I will keep this running for max 3 extra days. If it survives 4 days I will "declare it stable"
- IF it does not crash than the problem is not in the linux source but likely in the TC make sequence
- In that case I will for TC17.0 stop trouble shooting but analyse again with beta release of TC17.1 as I expect the TC team is more active with that release.

[Rich]

Hi Stefann

... So: size is exactly the same but they still differ ...

It's possible a timestamp or some other unique identifier gets added
to vmlinuz.

That's easy enough to confirm. Compile another vmlinuz and compare
it to the one you previously compiled.

[Stefann]

Thanks @Rich
I also thought... "maybe I just have 1 corrupted byte in my downloaded vmlinuz", I actually never checked a checksum and/or redownloaded

So...
That's what I did... redownload and compare:

Code: [Select]

tc@huis:/tmp$ mkdir TC
tc@huis:/tmp$ cd TC
tc@huis:/tmp/TC$ wget --header='Accept-Language: en-us,en;q=0.5' http://tinycorelinux.net/17.x/x86/release/Core-17.0.iso
Connecting to tinycorelinux.net (128.127.66.77:80)
saving to 'Core-17.0.iso'
Core-17.0.iso        100% |***************************************************************************| 19.5M  0:00:00 ETA
'Core-17.0.iso' saved

tc@huis:/tmp/TC$ wget --header='Accept-Language: en-us,en;q=0.5' http://tinycorelinux.net/17.x/x86/release/Core-17.0.iso.md
5.txt
Connecting to tinycorelinux.net (128.127.66.77:80)
saving to 'Core-17.0.iso.md5.txt'
Core-17.0.iso.md5.tx 100% |***************************************************************************|    48  0:00:00 ETA
'Core-17.0.iso.md5.txt' saved
tc@huis:/tmp/TC$ md5sum -c Core-17.0.iso.md5.txt
Core-17.0.iso: OK

tc@huis:/tmp/TC$ mkdir tc17
tc@huis:/tmp/TC$ ls
Core-17.0.iso  tc17/
tc@huis:/tmp/TC$ sudo mount Core-17.0.iso tc17
tc@huis:/tmp/TC$ cd tc17
tc@huis:/tmp/TC/tc17$ cd boot
tc@huis:/tmp/TC/tc17/boot$ ls
core.gz   isolinux/ vmlinuz
tc@huis:/tmp/TC/tc17/boot$ cp vmlinuz /mnt/sda1/tce/boot/vmlinuz_org2
tc@huis:/tmp/TC/tc17/boot$ cd /mnt/sda1/tce/boot

tc@huis:/mnt/sda1/tce/boot$ ls -l vm*
-rwxrw-r--    1 tc       staff      6087168 Feb 10 13:10 vmlinuz
-rw-rw-r--    1 tc       staff      6087168 Jun  2 17:11 vmlinuz17C2x
-r--r--r--    1 tc       staff      6087168 Jun  4 08:35 vmlinuz_org2
tc@huis:/mnt/sda1/tce/boot$ cmp --verbose vmlinuz vmlinuz_org2
tc@huis:/mnt/sda1/tce/boot$                          // no respons so they are equalso... no... the vmlinuz that has been crashing so often is exactly equal to a freshly downloaded one.

Also checked difference between my selfmade vmlinuz17C2x and downloaded vmlinuz in some more detail:

Code: [Select]

tc@huis:/mnt/sda1/tce/boot$ cmp -b vmlinuz vmlinuz17C2x
vmlinuz vmlinuz17C2x differ: byte 105, line 1 is 271 M-9 211 M-^I

tc@huis:/mnt/sda1/tce/boot$ cmp --verbose vmlinuz vmlinuz17C2x
    105 271 211
    106 201 210
    589 363 302
    590 350 357
    613 314 234
    614 101 110
    617 234 174
    618 205 214
   9463 210 200
   9469 334 324
   9565 210 200
   9572 334 324
  10472 210 200
  10479 334 324
  10539 210 200
  10545 334 324
  10594 334 324
  10670 210 200
  10701 334 324
  10903 210 200
  10914 334 324
  11658 174 164
  11665 224 214
  11672 160 150
  11679 210 200
  11686 144 134
  12357 110 100
  13099 260 250
  13848 314 304
  15381 116 150
  15382 125 160
  15383 103  65
  .............. and it keeps going......
  ........ just skipping the middle part....
  6085719   0 123
6085721   0 145
6085723   0 145
6085725   0 144
6085729   0 123
6085731   0 145
6085733   0 143
6085735   0 165
6085737   0 162
6085739   0 145
6085741   0 102
6085743   0 157
6085745   0 157
6085747   0 164
6085751   0 123
6085753   0 145
6085755   0 164
6085757   0 165
6085759   0 160
6085761   0 115
6085763   0 157
6085765   0 144
6085767   0 145...... so... the filesize being equal is likely related to snapping to some blocksize
They are really different

Anyway...
Its still did not crash
I keep it running...

And also as said in previous post:
- If this turns out to be stable my conclusion would be "sources do not have bug", it's in the "make process". I will than keep using the custom TC17.0 kernel. There is not really a downside on that.
- I will than wait for TC17.1 beta to check further. "maybe" custom make will than match make by TC-dev-team and "if not" that would be a more appropriate time to investigate.