Tiny Core Linux

Tiny Core Base => TCB Talk => Topic started by: naf456 on January 30, 2013, 02:58:35 PM

Title: How does memory management work?
Post by: naf456 on January 30, 2013, 02:58:35 PM
Hello, Nathan here.

Totally in love with TC at the mo. Have my graphics all set up and my sound working.

I'm curious to how TC manages memory , if the root FS is loaded in RAM does that mean all files saved to root FS is loaded in the ram also?

for example I have got xonotic up and running , however chrome saved it to downloads initially - and thats when my RAM usage went through the roof.
I have moved it to the disk I have got TC installed now , and so the ram is much less.



Title: Re: How does memory management work?
Post by: tinypoodle on January 31, 2013, 01:45:02 AM
tmpfs (using RAM + swap) is mounted over /.
Any other filesystem can be accessed through its mount point.
Title: Re: How does memory management work?
Post by: naf456 on January 31, 2013, 05:02:23 AM
Yeah - but the core filesystem is "Virtual" - meaning that it only exists in ram , because on HDD it's actually an archive.

How can you save something to "~/Downloads" on the HDD when it doesn't exist - only in the RAM.
Download Xonotic to RAM lol.

Or does it - is "~/" inflated at boot? ie is it on the HDD? erh... so confused. too early in the morning ^_^
Title: Re: How does memory management work?
Post by: althalus on January 31, 2013, 06:08:56 AM
Options:
1. Download everything to an external drive, instead of ~/
2. Use TC's built in backup features
3. Set up a persistent home

Spend some time reading the information on the tinycore website, and on the wiki to find out more. TC is different enough from most other linuxes that the wiki and the website should probably be treated as required reading before getting started.
Title: Re: How does memory management work?
Post by: tinypoodle on January 31, 2013, 06:48:58 AM
The content of the initrd cpio archive gets extracted into initramfs (a special instance of tmpfs in case of TC - using shared mem as storage space) at boot time.
Title: Re: How does memory management work?
Post by: Rich on January 31, 2013, 11:13:01 AM
Hi naf456
The file system is basically all in RAM, however, not everything that gets attached to the file system has to be RAM
based. For example:
/mnt refers to RAM
/mnt/sda1 refers to RAM
If I execute:
Code: [Select]
mount /mnt/sda1/mnt/sda1 now refers to drive sda1 and anything copied to /mnt/sda1 goes to the root directory of that drive.
If I now execute:
Code: [Select]
umount /mnt/sda1/mnt/sda1 now refers to RAM again and anything copied to /mnt/sda1 goes to RAM.
If you don't wish to setup a persistent home, you could setup a persistent downloads directory instead and have
Chrome default to that directory.
Title: Re: How does memory management work?
Post by: tinypoodle on January 31, 2013, 03:16:18 PM
Options:
1. Download everything to an external drive, instead of ~/
Besides from that, it is also a good idea to configure web browsers so they couldn't use huge amounts of tmpfs space for cache etc.
Also, having generous swap space available whenever possible.
Title: Re: How does memory management work?
Post by: vinnie on February 01, 2013, 06:39:21 PM
I also have a question of this type:
As is explained in canonically informatic courses the programs before they are executed by the processor are loaded in ram in a load phase.
in tinycore the programs are already installed in ram, then what happens? are loaded again in ram 2 times or not?
Title: Re: How does memory management work?
Post by: gerald_clark on February 01, 2013, 07:01:32 PM
The programs are not installed in RAM.  The extensions are loop mounted into the filesystem.  They are then loaded from from disk/flash-drive to RAM for execution.
They may also reside in cache for a while, but that could be flushed at any time.

If you use copy2fs flag or list,  there will be a copy of the extension in the RAM based filesystem, but it will still need to be loaded into RAM for execution.  In this case, there will be 2 copies of a progam in RAM, and temporarily a third temporary copy in cache.  This is a good reason not to use copy2fs on systems without a lot of RAM.
Title: Re: How does memory management work?
Post by: curaga on February 02, 2013, 07:06:31 AM
IIRC tmpfs doesn't cause that kind of double copy: http://lkml.indiana.edu/hypermail/linux/kernel/0705.0/0940.html
Title: Re: How does memory management work?
Post by: gerald_clark on February 02, 2013, 09:54:07 AM
Thanks.
Corrected:  there will not be a 3rd temporary copy in cache.
Title: Re: How does memory management work?
Post by: vinnie on February 03, 2013, 05:44:51 AM
Thanks gerald, concise and comprehensive as usual.
I use the flag copy2fs so I have 2 copies.
A question yet, but copy2fs also has effect on the scm packages?
Title: Re: How does memory management work?
Post by: naf456 on February 04, 2013, 08:36:36 AM
So to clarify - every program/library/resource loaded into the operating system is loaded into the RAM - so TC is kinda crippled if you want to run something like Photoshop (if it were possible) or some other large program?

I tried to run TCX in a virtual machine with 400MHz CPU cap with 64MB or RAM (simulating that of an old laptop) - it pretty much crashed - which was to be expected - yet with 512MB - Even 256MB of ram it ran fairly stable, albeit slow.

What is also blurry is how the applications get extracted from there squash file system? where are they extracted to, in RAM or on disk?

I'm still researching and learning about Linux and Unices, but I still have tons to learn still.

tonight I'll look through some of the scripts and check out how things tick.

still find it pretty cool playing Xonotic on ultra with no considerable frame rate drops like I get in windows or ubuntu XD

 
Title: Re: How does memory management work?
Post by: vinnie on February 04, 2013, 09:37:26 AM
No, they are all copied completely in ram only if you use the flag copy2fs.flg or the command tce-load -ic or a file to specify programs to be copied entirely in RAM.
Otherwise, access is from a mass storage memory.
The main advantage is that you can umount the mass storage memory  if you copied all the extensions in ram (with the flag).
Title: Re: How does memory management work?
Post by: Rich on February 04, 2013, 09:47:16 AM
Hi naf456
Maybe some pictures will help?
http://tinycorelinux.net/architecture.html  and  http://tinycorelinux.net/arch_copymode.html
Title: Re: How does memory management work?
Post by: tinypoodle on February 04, 2013, 10:29:39 AM
So to clarify - every program/library/resource loaded into the operating system is loaded into the RAM - so TC is kinda crippled if you want to run something like Photoshop (if it were possible) or some other large program?
Such used to be the case with ramdisks, but is no longer so with tmpfs as long as you provide enough swap space, so data stored in cache can be swapped out by kernel upon mem requirement of apps. 
Quote
What is also blurry is how the applications get extracted from there squash file system? where are they extracted to, in RAM or on disk?
Nowhere, with exception of Copy Mode, in both Default and Mount Mode they are mounted and content is symlinked into tmpfs.
Quote
I'm still researching and learning about Linux and Unices, but I still have tons to learn still.
tonight I'll look through some of the scripts and check out how things tick.
Maybe you would benefit most by reading up about filesystems (initramfs, tmpfs, squashfs etc.)  ;)
Title: Re: How does memory management work?
Post by: tinypoodle on February 04, 2013, 05:19:44 PM
So to clarify - every program/library/resource loaded into the operating system is loaded into the RAM - so TC is kinda crippled if you want to run something like Photoshop (if it were possible) or some other large program?
So just a bit ago I happened to observe a condition providing proof that there is no such concern when
an app started to have heavy mem requirements:

Code: [Select]
tc@box:~$ \df |grep tmpfs
tmpfs                   462888     29976    432912   6% /
tc@box:~$ grep Cached /proc/meminfo
Cached:             8252 kB
SwapCached:        94668 kB
meaning that out of 29976kB of data in my tmpfs at most 8252kB were remaining in RAM.

Measuring again after the app was exiting and thus freeing up a lot of mem:
Code: [Select]
tc@box:~$ grep Cached /proc/meminfo
Cached:            27368 kB
SwapCached:         8032 kB
most of the data in my tmpfs was residing in RAM.

Note: above on system with vm.swappiness = 100
Title: Re: How does memory management work?
Post by: naf456 on February 07, 2013, 05:14:00 AM
Thanks for all your help.
I'm going to read into tmpfs and squashfs in more detail. ^_^

EDIT: Does TC use swap? or does it just use the partition on which it is install on? I say this because I haven't setup a swap.
Going to my root filesystem, it says that it's mounted 4.6GB or data, while my computer only has 2 GB of RAM , while top says only 1GB of my ram is being used?
Meaning 3.6GB of data is being pumped from my HDD somehow.

I read briefly about initrmfs/tmp And it states that it requires swap? :o
Title: Re: How does memory management work?
Post by: Juanito on February 07, 2013, 06:08:40 AM
If you have a swap partition and the partition is enabled (swapon), then tinycore will use the swap partition. If you don't have a swap partition and you have enough ram, tinycore will manage fine without it.

Code: [Select]
$ sudo fdisk -l..should show if there is a swap partition present