WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Request: ZFS + tools  (Read 3260 times)

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 661
Request: ZFS + tools
« on: November 29, 2023, 12:37:36 AM »
Quote
When requesting an extension, please include the following information:
1. Which flavor, Tinycore or piCore.
2. Which version. Provide the result of the  version  command.
3. Which architecture. Provide the result of the  uname -a  command.
4. A link to the applications web site, if you have one.

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10992
Re: Request: ZFS + tools
« Reply #1 on: November 29, 2023, 03:05:01 AM »
IIRC ZFS had a license issue that meant it was potentially risky for distros to ship it, and only safe for the end user to compile it.
The only barriers that can stop you are the ones you create yourself.

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 661
Re: Request: ZFS + tools
« Reply #2 on: November 29, 2023, 09:46:01 AM »
It's an ancient topic, no doubt, but there's a glimmer of light through the fog...
Quote
We note that Debian's decision to place source-only ZFS in a relegated area of their archive called contrib, is an innovative solution. Debian fortunately had a long-standing policy that contrib was specifically designed for source code that, while licensed under an acceptable license for Debian's Free Software Guidelines, also has a default use that can cause licensing problems for downstream Debian users. Therefore, Debian communicates clearly to their users that this code is problematic by keeping it out of their main archive. Furthermore, Debian does not distribute any binary form of zfs.ko.

Bull : Matador...  endless fight :D

Offline hiro

  • Hero Member
  • *****
  • Posts: 1228
Re: Request: ZFS + tools
« Reply #3 on: November 30, 2023, 06:32:47 PM »
IIRC ZFS had a license issue that meant it was potentially risky for distros to ship it, and only safe for the end user to compile it.

as an extension it should be fine no? it's the user's choice whether they load it or not.
both ubuntu and alpine have it. ubuntu even on the install disk - not as an optional extension.
cddl doesn't seem super restrictive.
« Last Edit: November 30, 2023, 06:36:27 PM by hiro »

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10992
Re: Request: ZFS + tools
« Reply #4 on: December 01, 2023, 02:45:32 AM »
There are legal opinions that Ubuntu's shipping is infringement, and both Oracle and the kernel devs would have standing. Having it as an extension wouldn't absolve the builder nor the hoster. So while there is no court case, it would be prudent for us to avoid it.
The only barriers that can stop you are the ones you create yourself.

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 661
Re: Request: ZFS + tools
« Reply #5 on: December 01, 2023, 03:39:46 AM »
My initial read (both CDDL and GPL ends) led me to think in @hiro's direction (1. it's not compiled into the binary - which was the main fight initially) 2. it's not a "shipped" item (meaning, available as part of the ISO/Release) 3. it would end up an extension which has to be searched upon by the user and installed (ie: adding acceptance clause to .info file) BUT, supply 'n demand aren't going to warrant pushing the "avoid confrontation" boundaries.

"Tell ya'll what..."  If I crash into any brick walls when I go to compile it, care to lend a hand?  Releasing binaries may not be prudent, but it can't hurt to put compiling instructions/dependencies/build script(s)/etc. together for those who may want Z for themselves! :)

Offline hiro

  • Hero Member
  • *****
  • Posts: 1228
Re: Request: ZFS + tools
« Reply #6 on: December 01, 2023, 04:44:56 AM »
There are legal opinions that Ubuntu's shipping is infringement, and both Oracle and the kernel devs would have standing. Having it as an extension wouldn't absolve the builder nor the hoster. So while there is no court case, it would be prudent for us to avoid it.

should, until a courtcase, any incorrect opinion be hold without any possibility of reproach? the effect of it sounds worse than an actual sentencing to me.

here's the zfsonlinux opinion about this:
from https://openzfs.github.io/openzfs-docs/License.html
Quote
The OpenZFS software is licensed under the Common Development and Distribution License (CDDL) unless otherwise noted.

The OpenZFS documentation content is licensed under a Creative Commons Attribution-ShareAlike license (CC BY-SA 3.0) unless otherwise noted.

OpenZFS is an associated project of SPI (Software in the Public Interest). SPI is a 501(c)(3) nonprofit organization which handles the donations, finances, and legal holdings of the project.

Note

The Linux Kernel is licensed under the GNU General Public License Version 2 (GPLv2). While both (OpenZFS and Linux Kernel) are free open source licenses they are restrictive licenses. The combination of them causes problems because it prevents using pieces of code exclusively available under one license with pieces of code exclusively available under the other in the same binary. In the case of the Linux Kernel, this prevents us from distributing OpenZFS as part of the Linux Kernel binary. However, there is nothing in either license that prevents distributing it in the form of a binary module or in the form of source code.

Additional reading and opinions:

Software Freedom Law Center

Software Freedom Conservancy

Free Software Foundation

Encouraging closed source modules

Offline gadget42

  • Hero Member
  • *****
  • Posts: 670
Re: Request: ZFS + tools
« Reply #7 on: December 01, 2023, 05:00:06 AM »
...
"Tell ya'll what..."  If I crash into any brick walls when I go to compile it, care to lend a hand?  Releasing binaries may not be prudent, but it can't hurt to put compiling instructions/dependencies/build script(s)/etc. together for those who may want Z for themselves!

go for it! kudos! keep us posted.
The fluctuation theorem has long been known for a sudden switch of the Hamiltonian of a classical system Z54 . For a quantum system with a Hamiltonian changing from... https://forum.tinycorelinux.net/index.php/topic,25972.msg166580.html#msg166580

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10992
Re: Request: ZFS + tools
« Reply #8 on: December 01, 2023, 10:59:09 AM »
should, until a courtcase, any incorrect opinion be hold without any possibility of reproach? the effect of it sounds worse than an actual sentencing to me.
Well, I guess it depends on who takes the risk.

A build script or even a builder extension would be fine.
The only barriers that can stop you are the ones you create yourself.

Offline gadget42

  • Hero Member
  • *****
  • Posts: 670
Re: Request: ZFS + tools
« Reply #9 on: December 03, 2023, 07:09:08 AM »
been experimenting with some *bsd / *nas / *nix that all use *zfs and saw this:

https://news.slashdot.org/story/23/12/03/0134224/openzfs-fixes-data-corruption-issue

fortunately all we've been doing is experimenting so we're not concerned with silent data corruption at this point...whew.

20231203-0613am-cdt-usa-modified: added *nix since we're poking around in illumos / tribblix / openindiana as well
« Last Edit: December 03, 2023, 07:13:47 AM by gadget42 »
The fluctuation theorem has long been known for a sudden switch of the Hamiltonian of a classical system Z54 . For a quantum system with a Hamiltonian changing from... https://forum.tinycorelinux.net/index.php/topic,25972.msg166580.html#msg166580

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 661
Re: Request: ZFS + tools
« Reply #10 on: December 07, 2023, 07:27:48 PM »
@gadget42: After digging into the reference of data corruption, nobody has given specifics yet, based on my level of digging thus far, but all fingers seem to point to RAM issues (which I'd speculate would/could affect any kind of filesystem) and how ZFS supposedly claims ECC is virtually a mandate, though mission critical should always (IMO) be dealt in that mind-set.

@curaga: FreeNAS/TrueNAS seem to be successfully avoiding the Z FileSystem (Oracle/Sun) licensing conflict by acting as though OpenZFS is a completely separate entity.
One person went as far as claiming
Quote
OpenZFS (ZFS on Linux) is a fork and can fully be used with no license conflict.
One rebuttal (regarding Debian's inclusion) states
Quote
The CDDL and the GPL are incompatible. Debian solves this with a zfs-dkms package that builds the modules on demand, the package doesn’t actually contain the modules because license.

Common Sense...  Error...  Does Not Compute.

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10992
Re: Request: ZFS + tools
« Reply #11 on: December 08, 2023, 02:19:20 AM »
FreeNAS is BSD, aka BSD licensed. There is no conflict between BSD and CDDL.
The only barriers that can stop you are the ones you create yourself.

Offline gadget42

  • Hero Member
  • *****
  • Posts: 670
Re: Request: ZFS + tools
« Reply #12 on: December 08, 2023, 04:38:31 AM »
@gadget42: After digging into the reference of data corruption, nobody has given specifics yet, based on my level of digging thus far, but all fingers seem to point to RAM issues (which I'd speculate would/could affect any kind of filesystem) and how ZFS supposedly claims ECC is virtually a mandate, though mission critical should always (IMO) be dealt in that mind-set.
seems to go beyond a "RAM" issue, here are the links i have been following:
https://github.com/openzfs/zfs/pull/15571
https://github.com/openzfs/zfs/issues/15526

re: ECC RAM
yes, if you can't afford to lose it(or afford to fix it or work around it) then ECC is absolutely the only way to go.

ps-IMHO data integrity/security is always paramount. re:
https://github.com/openzfs/zfs/pull/15571#issuecomment-1826805735

also re: silent data corruption
(this is from fb/meta and so shouldn't be written by a bot...sigh...)
https://engineering.fb.com/2021/02/23/data-infrastructure/silent-data-corruption/

20231208-0346am-cdt-usa-modified: added additional silent data corruption information
« Last Edit: December 08, 2023, 04:48:25 AM by gadget42 »
The fluctuation theorem has long been known for a sudden switch of the Hamiltonian of a classical system Z54 . For a quantum system with a Hamiltonian changing from... https://forum.tinycorelinux.net/index.php/topic,25972.msg166580.html#msg166580

Offline gadget42

  • Hero Member
  • *****
  • Posts: 670
Re: Request: ZFS + tools
« Reply #13 on: December 08, 2023, 08:52:04 AM »
one wonders what happened to some data-integrity at alphabet/google and if they were using something anywhere similar to *zfs

https://arstechnica.com/gadgets/2023/12/google-calls-drive-data-loss-fixed-locks-forum-threads-saying-otherwise/
The fluctuation theorem has long been known for a sudden switch of the Hamiltonian of a classical system Z54 . For a quantum system with a Hamiltonian changing from... https://forum.tinycorelinux.net/index.php/topic,25972.msg166580.html#msg166580

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 661
Re: Request: ZFS + tools
« Reply #14 on: December 09, 2023, 03:17:33 PM »
one wonders what happened...
No, not really.  It's Google.  Always expanding...  always at the cost of the consumer...  [whispers] ...allllllways lisssstening...
Conglomerate attitude: "eh...  sh*t happens.  Not my problem!  It's in the TOS!"

@gadget42: Thank you for the hub links; #15571 I kind'a knew about first-hand - in fact, I blamed it on a WD RED which had been acting funky - turned out what I thought to be gunk/paper on the SATA7MALE port of the drive; I had rebuilt the pool from scratch after cleaning the port and replacing the cable - all went through fine even after blasting the datasets with about 1TB of randomness, but still found about a dozen SEEK_DATA/HOLE notices in the end but nothing that pointed at a single source.  (The data passed screening, though -- no losses, but it does show that the "issue" or issues still reside.)

"POF"
...the potential sound a person makes if punched in the gut.
...also the acronym for Point of Failure.
Same thing, if you ask me! :)

By the way, and this is pure opinion, I think ZFS based on its intended use was a wonderful invention.
What I can do with Z on Machine "A" versus RAID (mdadm) on Machine "B" is mind-blowing (and requires an engineering degree just to comprehend some of the basics it seems!)

A. I create a fresh ZRAID2 array (Five 1TB drives) on Machine "A" -- it's ready to populate and put to actual use in a matter of a few minutes with encryption and compression built in.

B. I create a fresh RAID5x5 array on Machine "B" -- ONLY 868 Minutes Left before we're supposed to be able to put it to use.  (Imagine a 48TB array -- twelve 4TB drives in a PowerEdge with 256GB-ECC...  that took forever!!!  Not to mention it was on the bench the entire time...  sounding like a damn jet engine in the building even inside a 12U enclosure with all six walls!!!)

PRO: ZRAID seems to only care about syncing actual FILES, not the empty and unused space in between.  Massive issue considering building/rebuilding times!

CON: ZRAID "costs" about 10-15% more, from what I've seen, than normal RAID5/6 in terms of parity.  This may be adjustable -- I'm still a bit new to it from the "inside."

Is Z Better than...
There's really no such thing; it's all about performance, preference and patience in the end.
« Last Edit: December 09, 2023, 03:40:58 PM by CentralWare »