Tiny Core Linux
General TC => General TC Talk => Topic started by: aus9 on August 03, 2023, 10:18:20 AM
-
Hi
I did not even check until now what TC32 was seeing or giving me. I have made my dmesg a clickable text file link
https://www.mediafire.com/file/44qv22n8egdxp0a/32dmesg.txt
I would normally try a dropbox link but its not working for me at the moment...impatience is a virtue
the key thing is on TC64
free -g says I have a total of 6G and 5G available
here is the TC32 output
free -g
total used free shared buff/cache available
Mem: 1 1 0 0 0 0
Swap: 8 0 8
tc@box:~$
htop shows one memory hog is firefox which spawns multiple instances. Thats not normally problem if my sytem thought I had total of 6G
I only discovered today after building some GUI packages....and getting a memory allocation error.
thanks for reading.
PS I am aware that 32 bit may have a 3G limit but where is all my precious RAM? :-[
-
Hi aus9!
Haven't You tried Core64 configuration? It is 64-bit kernel supporting 32-bit userland. Some time ago when I was trying 32-bit TC on 64-bit machines I came to conclusion that this is much better way to run 32-bit apps on 64-bit machines.
-
Hi aus9
The -g option does not appear to be very accurate. Use -m instead.
This is from a machine with 4 Gbytes of RAM:
tc@E310:~$ free -g
total used free shared buffers cached
Mem: 2 1 1 0 0 0
-/+ buffers/cache: 1 1
Swap: 0 0 0
tc@E310:~$
tc@E310:~$ free -m
total used free shared buffers cached
Mem: 3028 1697 1331 0 4 414
-/+ buffers/cache: 1278 1750
Swap: 999 24 975
tc@E310:~$
-
Hi Rich
This is my result with a machine with 8G of RAM....confirmed on TC64 which is running on a different partition. So its not the bios nor loose sticks
tc@box:~$ free -m
total used free shared buff/cache available
Mem: 942 164 651 35 127 626
Swap: 8000 0 8000
tc@box:~$
I do not care if I only see 3 but 1 is not enough for me
Hi jazzbiker
No I had not. Does that affect any TCE submission?
-
Hi aus9
Run:
dmesg > dmesg.txtand attach the file to your next post.
-
Hi Rich
Just been back on TC64 and the key thing to me is
on TC64 dmesg | grep Memory has 6006032/6215008k available
now here is TC32 and my dmesg
dmesg | grep Memory
Memory: 952388K/985184K available (6664K kernel code, 866K rwdata, 1832K rodata, 732K init, 400K bss, 32796K reserved, 0K cma-reserved, 87196K highmem)
tc@box:~$
PS I just tried a different web browser and no difference detected so its not a TCE it must be something else
Incidently I have an AMD APU which means my graphics comes from the CPU. In the bios I give it 2G of RAM so I will never see 8G in total
hope that helps
-
I give it 2G of RAM
Hello. I once went through a similar situation. I assume these 2GB are used from the 4GB available to the system, not the remaining 4GB unallocated.
-
Hi aus9
I figured it out.
The following are the ranges of usable addresses for system RAM from dmesg.
I added the spaces to break up the addresses into groups of 4 digits.
BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000 0000 0000 0000-0x0000 0000 0009 d3ff] usable
BIOS-e820: [mem 0x0000 0000 0010 0000-0x0000 0000 09d8 1fff] usable
BIOS-e820: [mem 0x0000 0000 0a00 0000-0x0000 0000 0a1f ffff] usable
BIOS-e820: [mem 0x0000 0000 0a20 b000-0x0000 0000 0aff ffff] usable
BIOS-e820: [mem 0x0000 0000 0b02 0000-0x0000 0000 3b32 3fff] usable
BIOS-e820: [mem 0x0000 0000 3ddf f000-0x0000 0000 3eff ffff] usable
BIOS-e820: [mem 0x0000 0001 0000 0000-0x0000 0002 3f33 ffff] usable
The first 6 address ranges add up to approximately 962 Mbytes.
The last address range is approximately 6069 Mbytes (~6 Gbytes).
With 32 bits the highest address you can access is FFFF FFFF.
That last block of RAM uses addresses 1 0000 0000 through 2 3F33 FFFF
and is out of reach for a 32 bit address.
-
Hi aus9
I noticed one other thing:
Kernel command line: BOOT_IMAGE=/grub/vmlinuz tce=sda4 home=sda4 waitusb=5 nozswap
Unknown kernel command line parameters "nozswap BOOT_IMAGE=/grub/vmlinuz tce=sda4 home=sda4 waitusb=5", will be passed to user space.
The BOOT_IMAGE=/grub/vmlinuz is probably incorrect. Your boot loader
is passing it to the kernel.
-
Does that affect any TCE submission?
I suppose not. You use 32-bit extensions from x86 repo and any extension buit in Core64 mode will be 32-bit extension. In theory.
In practice I've submitted extensions a few and I did them for both x86 and x86_64 archs on the same machine. I've seen no reclamations concerning 32-bit versions. Probably because nobody use them :-) But I'm using them day by day.
-
Hi aus9,
By the way it is interesting how much memory will give 64-bit kernel for 32-bit userspace on Your box with 6G installed and 2G allowed for graphics card. I bet for 4G.
Let's not forget that modules are 64-bit too.
-
Hi
I too have figured it out sorry to take so long. The real answer was hinted by CardealRusso based on my AMD APU bios shared ram...which is not shared but reserved RAM
just then I gave my APU 512M and now I see before I LOAD hog=firefox
free -m
total used free shared buff/cache available
Mem: 2466 170 2145 35 152 2046
Swap: 8000 0 8000
some simple maths...total 2466 + 512 = 2978M which is close enough to the upper limit of a non-PAE 32 bit kernel.
Hi Rich
Please mark as solved. At your discretion change subject to
"members watch out for shared or reserved graphics RAM for x86 kernels"
Hi jazzbiker EDIT
I will not try a 64 bit kernel at this stage, see next post if interested
Hi Rich
That dir is correct.
I have MBR controlled by a debian distro.
It chainloads to the PBR for sda3 for TC64 bit such that
sudo mount /dev/sda3 /mnt/sda3
tc@box:~$ ls /mnt/sda3
boot/ home/ tce/
for TC32, I was unable to get grub2-multi (OR systemrescue usb) to embed into PBR for boot/ under sda4 so I changed it to grub and use debian 40_custom menu to boot TC32
for those interested TC32 for me is
ls /mnt/sda4
1builds/ m13.fsa
1tinycore/ m14.fsa
2023/ mp3/
MBR.MBR mx-builds/
bookmarks-2023-07-31.json tce/
grub/ v6.fsa
home/ voidlinux-files/
lost+found/
I will have to stop using firefox on TC32 unless I have display issues at certain sites like the tilda site I have already posted about...but not on TC32
thanks for reading
-
slightly offtopic to new subject but relevant to RAM usage
the internet says for firefox try this
Settings -> General -> Performance
untick use recommended
keep ticked HW acceleration
I ran htop before the change with 4 tabs open and total = 1.4G, that explains why I had a crash earlier
with new change current htop shows total of 684M
Thats enough for me to backtrack on using a 64 bit kernel on TC32, at this stage.
-
Here are some opinions by Linus Torvalds concerning virtual and physical memory amounts: http://www.realworldtech.com/forum/?threadid=76912&curpostid=76973
-
That's for link. I thought for a minute ot 2 PAE was an answer
My current setup is good enough. Jwm plus xorg plus some gtk3 apps plus firefox is around 700 megs
-
Does that affect any TCE submission?
Short answer.
No, it doesn't.
Long answer.
Does it matter ?
Does it run on older machine ?
Well, hard to tell.
I once compiled a program (that I can't remember) on a CPU that support SSE instructions.
Even though I told GCC to build for i486 like every package in TC does (-march=i486),
the product just wouldn't run on Celeron Mendocino which only support MMX.
With the help of my friend, we were able to nail down the cause.
GCC simply ignore my request and use SSE instructions for the binary.
I rebuilt the binary with exactly the same setting, exactly the same versions of extensions provided by TC, on that said Mendocino computer.
Everything was good, it ran without any trouble.
My friend checked the assembly and said there's no SSE instructions anymore.
In brief, if GCC decide to compile against a newer instruction set even though you told it not to, the only way to get a more backward compatible binary is, well, build it on an older machine. ::)
This type of trouble is bond to happen, and I think there's really not much you can do.
It's really likely that part of extensions on TC already has this problem, but no one complaining since not many sane people would stick with machine that old to run complex programs.
-
Hi Rich
I am wrong again
Your reply 7 may explain a new thing while I was attempting to compile something. I did finally compile it but it was loading pulseaudio-dev that pulls in heaps and then I got this
mount: mounting /dev/loop291 on /tmp/tcloop/libcap failed: Cannot allocate memory
root@box:/home/tc# free -m
total used free shared buff/cache available
Mem: 2466 287 1972 38 207 1899
Swap: 8000 0 8000
So I thought I had memory and dismissed your reply 7.
DOH
I have memory but clearly for above error.....it is not seen on my TC32
Thats a pita. >:(
Hi Rich can you mark it as not solved as others may be affected. I now know another member with TC32
AMD APU but he may not be compilting bloat that I do. ;D
-
Hi aus9
Your free command shows plenty of free RAM and that
your swap space hasn't been touched.
Could you recreate the error and then post the result of:
dmesg | tail -n 25
-
Hi Rich
EDIT I think I have solved it next post
thanks for looking into this for me. I have duplicated error but slightly different package but here goes
mount: mounting /dev/loop292 on /tmp/tcloop/pci-utils failed: Cannot allocate memory
tc@box:~$ dmesg | tail -n 25 # changed to larger number
loop291: detected capacity change from 0 to 272
loop292: detected capacity change from 0 to 544
vmap allocation for size 49152 failed: use vmalloc=<size> to increase size
mount: vmalloc error: size 42284, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0
CPU: 2 PID: 14481 Comm: mount Not tainted 6.1.2-tinycore #612
Hardware name: Micro-Star International Co., Ltd MS-7B86/B450 GAMING PLUS MAX (MS-7B86), BIOS H.C0 05/17/2021
Call Trace:
0xc0778e5e
0xc0778e7a
0xc023c774
0xc0238527
0xc02383e7
? 0xf78ca3fc
0xc023841b
? 0xf78ca3fc
0xf78ca3fc
0xf78c9a05
0xf78c951f
0xf78c8fa2
0xc025d5c8
? 0xf78c8c92
0xf78c8b42
0xc025c6c8
0xc0274e5f
0xc0274f46
0xc02751ba
0xc077904c
0xc078197d
EIP: 0xb7e6045e
Code: 90 66 90 66 90 66 90 66 90 66 90 90 57 56 53 8b 7c 24 20 8b 74 24 1c 8b 54 24 18 8b 4c 24 14 8b 5c 24 10 b8 15 00 00 00 cd 80 <5b> 5e 5f 3d 01 f0 ff ff 0f 83 04 92 f4 ff c3 66 90 90 57 56 53 8b
EAX: ffffffda EBX: 0893d1c0 ECX: bfb91e0d EDX: bfb91e26
ESI: 00008001 EDI: 00000000 EBP: bfb91e0d ESP: bfb90880
DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246
Mem-Info:
active_anon:78731 inactive_anon:19719 isolated_anon:0
active_file:42748 inactive_file:40843 isolated_file:0
unevictable:0 dirty:1759 writeback:0
slab_reclaimable:7700 slab_unreclaimable:17476
mapped:39999 shmem:15056 pagetables:638
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:382351 free_pcp:4102 free_cma:0
Node 0 active_anon:314924kB inactive_anon:78876kB active_file:170992kB inactive_file:163372kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:159996kB dirty:7036kB writeback:0kB shmem:60224kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 65536kB writeback_tmp:0kB kernel_stack:2440kB pagetables:2552kB sec_pagetables:0kB all_unreclaimable? no
DMA free:5424kB boost:0kB min:236kB low:292kB high:348kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15984kB managed:5500kB mlocked:0kB bounce:0kB free_pcp:76kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 828 2449 2449
Normal free:702292kB boost:0kB min:42128kB low:52660kB high:63192kB reserved_highatomic:0KB active_anon:0kB inactive_anon:176kB active_file:0kB inactive_file:10036kB unevictable:0kB writepending:28kB present:882004kB managed:860008kB mlocked:0kB bounce:0kB free_pcp:16332kB local_pcp:4524kB free_cma:0kB
lowmem_reserve[]: 0 0 12969 12969
DMA: 2*4kB (M) 3*8kB (M) 1*16kB (M) 2*32kB (M) 1*64kB (M) 1*128kB (M) 2*256kB (M) 1*512kB (M) 0*1024kB 0*2048kB 1*4096kB (M) = 5424kB
Normal: 123*4kB (UM) 229*8kB (U) 34*16kB (UE) 9*32kB (UM) 21*64kB (UE) 11*128kB (UME) 6*256kB (UE) 7*512kB (UME) 5*1024kB (UM) 5*2048kB (UM) 165*4096kB (M) = 702228kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
98647 total pagecache pages
0 pages in swap cache
Free swap = 8191996kB
Total swap = 8191996kB
639510 pages RAM
415013 pages HighMem/MovableOnly
8120 pages reserved
SQUASHFS error: Failed to allocate zlib workspace
squashfs image failed sanity checkk
at now I have
free -m
total used free shared buff/cache available
Mem: 2466 640 1389 69 437 1511
Swap: 8000 0 8000
-
I found a grub entry to change vmalloc=512M so will do a full reboot to see if error goes....ref
http://thinking-electron.blogspot.com/2015/05/how-to-increase-vmalloc-size-vmalloc.html
well that seems to be the trick!
but I noticed that when I loaded pulseaudio-dev and other stuff...my used memory was lower than what I just posted.
so I loaded an even bigger amount ffmpeg4-dev and now
free -m
total used free shared buff/cache available
Mem: 2467 630 1176 71 660 1546
Swap: 8000 0 8000
dmesg | tail -n 25
loop333: detected capacity change from 0 to 16
loop334: detected capacity change from 0 to 360
loop335: detected capacity change from 0 to 56
loop336: detected capacity change from 0 to 584
loop337: detected capacity change from 0 to 168
loop338: detected capacity change from 0 to 72
loop339: detected capacity change from 0 to 784
loop340: detected capacity change from 0 to 936
loop341: detected capacity change from 0 to 1592
loop342: detected capacity change from 0 to 88
loop343: detected capacity change from 0 to 760
loop344: detected capacity change from 0 to 40
loop345: detected capacity change from 0 to 168
loop346: detected capacity change from 0 to 392
loop347: detected capacity change from 0 to 416
loop348: detected capacity change from 0 to 408
loop349: detected capacity change from 0 to 80
loop350: detected capacity change from 0 to 104
loop351: detected capacity change from 0 to 96
loop352: detected capacity change from 0 to 9944
loop353: detected capacity change from 0 to 2208
loop354: detected capacity change from 0 to 2160
loop355: detected capacity change from 0 to 104
loop356: detected capacity change from 0 to 424
loop357: detected capacity change from 0 to 776
I notice that my loop count crashed about loop 293 ignoring sizes and have got past that and no crash
I think we have the solution
add bootcode vmalloc=512M
-
Hi aus9
Seems this has come up before:
https://forum.tinycorelinux.net/index.php/topic,19415.msg120213.html#msg120213
He fixed it in reply #46:
... Increased vmalloc to 256MB ...
-
Hi Rich,
In the thread You've mentioned vmalloc was hiding not solving. The solution was using Core64 mode on the box with a lot of RAM and lot of GPU RAM. Did I understand right?
@aus9: Are You observing this error in Core64 mode?
-
Hi jazzbiker
From https://mjmwired.net/kernel/Documentation/kernel-parameters.txt :
vmalloc=nn[KMG] [KNL,BOOT] Forces the vmalloc area to have an exact
size of <nn>. This can be used to increase the
minimum size (128MB on x86). It can also be used to
decrease the size and leave more room for directly
mapped kernel RAM.The way I read it you are setting aside a larger (in this case) contiguous
address space aside for handling memory allocations.
... The solution was using Core64 mode on the box with a lot of RAM and lot of GPU RAM. Did I understand right? ...
Yes, either Core64 or Corepure64. The issue appears to be available address space, not
available RAM. If you look at my reply #7 you'll see where I showed most of the RAM
was mapped out of reach above the 4 Gbyte address range.
-
Hi jazzbiker
The solution was using Core64 mode on the box with a lot of RAM and lot of GPU RAM. Did I understand right?
@aus9: Are You observing this error in Core64 mode?
hmm thats no help for dumbos like me ;D
How do I know what is a lot of GPU RAM. If you notice I reduced the motherboard RAM reserved for my APU to give my system more RAM and allegedly more free RAM
I have never seen this in TC64 but I am trying to stay kind of pure Tinycore x86 while compiling.
And personal taste....I do not want to run a 64 bit kernel with 32 bit apps in case my brain gets confused.
If there is a smarter way let me know.
Incidently slightly off topic.....I saw a recent post at
https://www.phoronix.com/
so I am thinking maybe I can build something related to "containers" like snap appimage flatpak etc
not good for compiling against but quite good for those wanting a quick up-to-date software
-negative is built against /usr and no stripping and a fair amount of duplication.
Hi Rich
Altho I mentioned your post 7 and thought that was the explanation to my compile issue.....I can only report what I see.
And what I am now seeing......is that my new setup is allowing me to load quite large make dependencies and still not crash
-
Hi aus9
... hmm thats no help for dumbos like me ;D ...
Let me try explaining it another way:
1. For x86, your CPU can address 4 Gbytes of stuff. It's a 4 lb sack.
2. You have 8 Gbytes of RAM. Lets call that 8 lb of shit.
3. Your peripherals (video, sound, network, disk, etc.) want another
3 Gbytes of space. That's an additional 3 lb of crap.
That takes care of the groundwork.
You can't fit 11 lb of shit and crap into a 4 lb sack.
It's first come first served, therefore:
If your 4 lb sack (x86 CPU) contains 3 lb of crap (peripherals), it can only hold 1 lb of shit (RAM).
Result:
The other 7 lb of shit (RAM) remain unused.
Note: For a metric explanation, replace each occurrence of lb with kg.
-
Hi Rich
I am not seeing 4 Kilos in total?
free -m
total used free shared buff/cache available
Mem: 2467 809 1097 72 561 1371
Swap: 8000 0 8000
-
Hi aus9
That's because your peripherals also use address space. Every address
that a peripheral uses is one less address that can be used to access RAM.
Total is installed RAM or the amount of address space space available for
RAM, whichever is less.
-
Hi Rich
Ahhh well that is good to know and me thinks that explains why I was confused....other than losing brain cells to aussie beer
OK I reverse my reluctance to go 64 bit kernel and modules. Thanks for caring
-
Hi Rich
Ok I am on TC32 with K=64bit and now we have
tc@box:~$ free -m
total used free shared buff/cache available
Mem: 7400 657 6405 53 337 6459
Swap: 8000 0 8000
Rich....Not sure if its intentional but I can not spot a difference in the files for /etc/os-release or /etc/issue
I can prove I am on TC32 using a readelf command....not that anyone cares but thought I would mention it here ;D
To anyone interested
trivia my 40_custom grub entries
TC64 is a chainload to grub in PBR for partition 3
next is TC32 running 64 bit kernel and modules
bottom is "normal for me" TC32 with 32 bit kernel
I decided to keep the virtual memory allocation vaule as I now have plenty of RAM
I now blacklist my sound module so I have a one liner in my bootlocal.sh to disable hdmi
menuentry "sda3" {
set root=(hd0,msdos3)
chainloader +1
}
menuentry "sda4K64" {
set root=(hd0,msdos4)
linux /grub/vmlinuz64 tce=sda4 home=sda4 waitusb=5 nozswap blacklist=snd_hda_intel vmalloc=512M
initrd /grub/amd-ucode.img /grub/rootfs.gz /grub/modules64.gz
}
menuentry "sda4onboot" {
set root=(hd0,msdos4)
linux /grub/vmlinuz tce=sda4 home=sda4 waitusb=5 nozswap blacklist=snd_hda_intel vmalloc=512M
initrd /grub/amd-ucode.img /grub/core.gz
}
thanks for reading
Just compiled a package and it works over reboot on TC32 kernel so I am good to go.
-
Hi aus9
... Rich....Not sure if its intentional but I can not spot a difference in the files for /etc/os-release or /etc/issue ...
You are running Kernel64 which loads Modules64+rootfs.
Aside from the base 32 bit libraries (/lib/), rootfs also contains /etc/ and
all of the other root directories you see if you run:
ls /
rootfs is the 32 bit component in this mix that allows you to run 32 bit programs.
It is the same rootfs used in 32 bit Tinycore.
-
Hi Rich
I wrote poorly (again)
I knew I had rootfs for 32 bit.
What I meant to say....those etc files in TC32 appear to be the same files as TC64.
They are only text files and are dev team created, no?
So as TC64 was the new baby to the TC distros ....maybe its os-release changes from tinycore to this?
ID=tinycore64
-
Hi aus9
Sorry, it's been a long time since I've looked at those files. I thought
they included the architecture it was compiled for. I guess only the
release version is included, so the files would be identical for all
architectures.
You can find the difference in /lib/.
TC32 (core and core64):
tc@E310:~$ ls /lib/ld-linux*
/lib/ld-linux.so.2
tc@E310:~$
TC64 (corepure64):
tc@E310:~$ ls /lib/ld-linux*
/lib/ld-linux-x86-64.so.2
tc@E310:~$
Here's a helpful tip from curaga if you're compiling for TC32 with this setup:
Oh, you can use a 64-bit kernel for that still :)
"linux32 bash" will start a bash shell that claims to be 32-bit in uname -m. This is useful for some configure scripts that check for it.