how to format a partition to xfs on piCore-9.0.3?

[skyp]

Hi!

I'm quite new in linux and trying to format a partition on the SD card to xfs on my new Rpi3 B+.
I allready installed filesystems-4.9.22-piCore-v7.tcz but there's no mkfs.xfs, which other distros have so i don't know how to perfom a formatting. Could anyone help me?

[gavinmc42]

I usually do all my SD card formating on another Linux Mint PC with a SD card reader and Gparted.
It comes from early days when my Pi's had no Internet connection so piCore could not update itself.

[Juanito]

You need xfsprogs, which is not in the piCore repos

[Juanito]

..but it is now - posted

[skyp]

Wow Juanito, thanks! Awesome!

I find at least 20 percent quicker to write an xfs partition on my SD card than ext3/4.... 

[bmarkus]

I find at least 20 percent quicker to write an xfs partition on my SD card than ext3/4....

Interesting point, thank you for bringing up the topic. I will take a look to see is it worth to change to xfs for mmcblk0p2.

How did you benchmark it?

[skyp]

I've done some tests. It's a cheap Kingstone 8GB micro SD. mmcblk0p3 is formatted to xfs. As you can see, the significant difference interrestingly appeares only in write times. It's a big savior to my project because I need to capture high rate audio streams to the card without overloads. This little boost helped me out...

Code: [Select]

tc@box:~$ sudo dd if=/dev/zero of=/mnt/mmcblk0p2/test bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 87.098434 seconds, 5.7MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/dev/zero of=/mnt/mmcblk0p3/test bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 68.906372 seconds, 7.3MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/mnt/mmcblk0p2/test of=/dev/null bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 24.768939 seconds, 20.2MB/s
tc@box:~$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
3
tc@box:~$ sudo dd if=/mnt/mmcblk0p3/test of=/dev/null bs=500K count=1024
1024+0 records in
1024+0 records out
524288000 bytes (500.0MB) copied, 24.112068 seconds, 20.7MB/s

Btw I tested another even cheaper noname 2GB card with 3.6/4.4 MB/s results... 

[bmarkus]

Lets see my test result.

ext4 partition on SD card in RPi3 card slot:

Code: [Select]

# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 60.9158 s, 8.6 MB/s
xfs partition on SD card in RPi3 card slot:

Code: [Select]

# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 39.0545 s, 13.4 MB/s

RAM disk (root fs):

Code: [Select]

# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.95662 s, 268 MB/s
Now the interesting result, ext4 on SD card in USB external card reader:

Code: [Select]

# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 34.2842 s, 15.3 MB/s
A different SD card was used, but both cards has the same rating, Class 10. Ther must be some difference due to card differences but I think more is the poor performance of internal SD card handling.

And finally an USB stick with ext4:

Code: [Select]

# dd if=/dev/zero of=test bs=500k count=1024
1024+0 records in
1024+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 179.612 s, 2.9 MB/s

Result is interesting, but lets see a real life test, building xfsprogs on ext4 and xfs partition on the same card.

Code: [Select]

ext4: 8m 15s
xfs:  7m 36s
XFS benefits for write intensive applications, like databases under high load would be much higher. All together I'm considering to move to XFS as a default file system. Now only one test left. BTRFS 

[bmarkus]

btrfs:

Code: [Select]

# dd if=/dev/zero of=test bs=512k count=1024
1024+0 records in
1024+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 35.761 s, 15.0 MB/s

[bmarkus]

xfstools build time on btrfs:

Code: [Select]

btrfs: 7m 30s

[bmarkus]

Apropó, most látom hogy mindketten budapestiek és UPC ügyfelek vagyunk

[skyp]

Ez így van!  Nem is tudtam, hogy a piCore főhadiszállás itthon van ! Bár a nevedből rájöhettem volna! 
Szép napot! 

[CentralWare]

Hello, @Béla!
I connected a pair of 512GB USB3.1 thumb drives to a RasPi4 (8GB) and in two separate SSH windows, and at roughly 5:15PM started mkfs.ext2 (I use non-journal for flash based drives out of habit) in each window and left to tend to business, expecting them to be a little while during format.

It's now 11:00PM and the format is at approximately 51% complete.  Both formats were cancelled.

After digging through the archives I finally found a copy of XFS(tools) in the 9.x repo, which seems to work just fine with 12.x filesystem-KERNEL support.  Format time: 2.27sec.  Did you ever give any additional thought to implementing XFS into core?  (It may not be a bad idea to update piCore's more recent repos with xfstools & dev?)  Any updates to your "checking out btrfs?"
Thanks!

[bmarkus]

Hi Centralware,

It is an interesting question, thanks. While in the past mainstream distros used ReiserFS or XFS in the last years Ext4 become the de facto standard. It is small, have a good performance and safe in case of crashing. Ext4 journaling saved my life in few cases. Formatting time is just one thing, but the operational performance is more important. Ext4 is good choice in most cases, but for specific applications other file system may be better. For example MongoDB database works much better on XFS. I tested btrfs, it offers nearly the same performance as ext4, the big advantage is the additional built-in feature set like handling virtual volumes, etc. but it is big, including tools and these extra features are not required by average use cases so I dropped to include it in piCore

The beauty of piCore its flexibility. The system itself is in the boot mmcblk0p1 partition, mmcblk0p2 is just for the preinstalled tcz packages, not required by the system. Current boot system supports multiple initrd files. You can create a small initrd with the XFS or btrfs kernel modules and add it to the boot config. In such case these partitions will be recognized during boot, you can replace the default ext4 with XFS or btrfs partitions with the same preinstalled packages.


Regards,

Béla

[CentralWare]

The flash drives here were all formatted EXT (EXT2 for what they're being used for, one of the 512GB drives is being used as a local TCL repo mirror; it's almost always read-only so safety shouldn't be too much of a concern), however, on a RasPi4 8GB it ended up taking 17.25 hours to complete formatting directly (the other unit was formatted with Lazy Init, took about five minutes for mkfs to release to the prompt, but assuming the background completion of the format still took 16+ hours.)  Here's our list:

• ZFS: Never tinkered with it; rumors online it's rather fragile (UPS required?) so I've leaned away from that one.
• XFS: From experience with Buffalo NAS units here, seems solid, but online rumors claim somewhat fragile with power outages.
• AFP: Never cared to plant an apple in Linux, they're for eating, not file storage!  Anyone experienced with AFPfs?
• EXT2: (IMO) better for flash and SSD drives, "tends" to flag read-only on a dirty boot, Okay speed, better when on a stripe volume.
• EXT3/4: Good for images and HDD, Decent write speed (journaling), best option for NFS (IMO)
• RFS: Experience is limited (never "beat" on it) but noticed a lag in R/W speed on RAID-5 compared to EXT4
• FAT: Okay if being used (mmcblk0p1) between Nix and Win, Flags R/O in power outage situations more times than not.
• NTFS: Fast format, Win compliant (mostly), permissions tend to be flaky; not really a Nix FS
• SquashFS should be turned into a full bootable file system! :^)

• NFS: EXT4 is my only choice, especially if there's unexpected outages/disconnections. RFS would be my second choice.
• AoE: RFS or EXT3 (we've had some odd behavior with EXT4 here and there where fsck is needed frequently.)
• iSCSI: The jury is out; we're still beating up on this one.