Tiny Core Linux

Tiny Core Base => Raspberry Pi => Topic started by: skyp on September 09, 2017, 04:31:56 PM

Title: how to format a partition to xfs on piCore-9.0.3?
Post by: skyp on September 09, 2017, 04:31:56 PM
Hi!

I'm quite new in linux and trying to format a partition on the SD card to xfs on my new Rpi3 B+.
I allready installed filesystems-4.9.22-piCore-v7.tcz but there's no mkfs.xfs, which other distros have so i don't know how to perfom a formatting. Could anyone help me?
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: gavinmc42 on September 09, 2017, 07:53:34 PM
I usually do all my SD card formating on another Linux Mint PC with a SD card reader and Gparted.
It comes from early days when my Pi's had no Internet connection so piCore could not update itself.
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: Juanito on September 10, 2017, 04:30:09 AM
You need xfsprogs, which is not in the piCore repos
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: Juanito on September 10, 2017, 09:05:30 AM
..but it is now - posted
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: skyp on September 10, 2017, 12:28:43 PM
Wow Juanito, thanks! :) Awesome!

I find at least 20 percent quicker to write an xfs partition on my SD card than ext3/4.... 
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on September 10, 2017, 01:08:19 PM
Quote from: skyp on September 10, 2017, 12:28:43 PM
I find at least 20 percent quicker to write an xfs partition on my SD card than ext3/4....

Interesting point, thank you for bringing up the topic. I will take a look to see is it worth to change to xfs for mmcblk0p2.

How did you benchmark it?
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: skyp on September 11, 2017, 08:56:31 PM
I've done some tests. It's a cheap Kingstone 8GB micro SD. mmcblk0p3 is formatted to xfs. As you can see, the significant difference interrestingly appeares only in write times. It's a big savior to my project because I need to capture high rate audio streams to the card without overloads. This little boost helped me out...

Code: [Select]
tc@box:~$ sudo dd if=/dev/zero of=/mnt/mmcblk0p2/test bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 87.098434 seconds, 5.7MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/dev/zero of=/mnt/mmcblk0p3/test bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 68.906372 seconds, 7.3MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/mnt/mmcblk0p2/test of=/dev/null bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 24.768939 seconds, 20.2MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/mnt/mmcblk0p3/test of=/dev/null bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 24.112068 seconds, 20.7MB/s

Btw I tested another even cheaper noname 2GB card with 3.6/4.4 MB/s results... 
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on September 12, 2017, 02:04:22 AM
Lets see my test result.

ext4 partition on SD card in RPi3 card slot:

Code: [Select]
# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 60.9158 s, 8.6 MB/s
xfs partition on SD card in RPi3 card slot:

Code: [Select]
# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 39.0545 s, 13.4 MB/s

RAM disk (root fs):

Code: [Select]
# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.95662 s, 268 MB/s
Now the interesting result, ext4 on SD card in USB external card reader:

Code: [Select]
# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 34.2842 s, 15.3 MB/s
A different SD card was used, but both cards has the same rating, Class 10. Ther must be some difference due to card differences but I think more is the poor performance of internal SD card handling.

And finally an USB stick with ext4:

Code: [Select]
# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 179.612 s, 2.9 MB/s

Result is interesting, but lets see a real life test, building xfsprogs on ext4 and xfs partition on the same card.

Code: [Select]
ext4: 8m 15s
xfs:  7m 36s
XFS benefits for write intensive applications, like databases under high load would be much higher. All together I'm considering to move to XFS as a default file system. Now only one test left. BTRFS  ;)
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on September 12, 2017, 07:03:00 AM
btrfs:

Code: [Select]
# dd if=/dev/zero of=test bs=512k count=1024
1024+0 records in
1024+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 35.761 s, 15.0 MB/s
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on September 12, 2017, 07:14:32 AM
xfstools build time on btrfs:

Code: [Select]
btrfs: 7m 30s
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on September 12, 2017, 07:58:19 AM
Apropó, most látom hogy mindketten budapestiek és UPC ügyfelek vagyunk :)
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: skyp on September 13, 2017, 08:22:53 AM
Ez így van!  ;) Nem is tudtam, hogy a piCore főhadiszállás itthon van :)! Bár a nevedből rájöhettem volna!  :o
Szép napot!  :)
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: CentralWare on January 11, 2022, 11:10:26 PM
Hello, @Béla!
I connected a pair of 512GB USB3.1 thumb drives to a RasPi4 (8GB) and in two separate SSH windows, and at roughly 5:15PM started mkfs.ext2 (I use non-journal for flash based drives out of habit) in each window and left to tend to business, expecting them to be a little while during format.

It's now 11:00PM and the format is at approximately 51% complete.  Both formats were cancelled.

After digging through the archives I finally found a copy of XFS(tools) in the 9.x repo, which seems to work just fine with 12.x filesystem-KERNEL support.  Format time: 2.27sec.  Did you ever give any additional thought to implementing XFS into core?  (It may not be a bad idea to update piCore's more recent repos with xfstools & dev?)  Any updates to your "checking out btrfs?"
Thanks!
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: bmarkus on January 13, 2022, 01:52:13 AM
Hi Centralware,

It is an interesting question, thanks. While in the past mainstream distros used ReiserFS or XFS in the last years Ext4 become the de facto standard. It is small, have a good performance and safe in case of crashing. Ext4 journaling saved my life in few cases. Formatting time is just one thing, but the operational performance is more important. Ext4 is good choice in most cases, but for specific applications other file system may be better. For example MongoDB database works much better on XFS. I tested btrfs, it offers nearly the same performance as ext4, the big advantage is the additional built-in feature set like handling virtual volumes, etc. but it is big, including tools and these extra features are not required by average use cases so I dropped to include it in piCore

The beauty of piCore its flexibility. The system itself is in the boot mmcblk0p1 partition, mmcblk0p2 is just for the preinstalled tcz packages, not required by the system. Current boot system supports multiple initrd files. You can create a small initrd with the XFS or btrfs kernel modules and add it to the boot config. In such case these partitions will be recognized during boot, you can replace the default ext4 with XFS or btrfs partitions with the same preinstalled packages.


Regards,

Béla
Title: Re: how to format a partition to xfs on piCore-9.0.3?
Post by: CentralWare on January 15, 2022, 10:01:57 PM
The flash drives here were all formatted EXT (EXT2 for what they're being used for, one of the 512GB drives is being used as a local TCL repo mirror; it's almost always read-only so safety shouldn't be too much of a concern), however, on a RasPi4 8GB it ended up taking 17.25 hours to complete formatting directly (the other unit was formatted with Lazy Init, took about five minutes for mkfs to release to the prompt, but assuming the background completion of the format still took 16+ hours.)  Here's our list: