[GTALUG] Which Distro is Best for Running a ZFS-on-Linux Fileserver.

[right.maple.nut via talk]

Hello All, Like the Subject Line says, I'm setting up a ZFS File Server for my Home Network. Given that I will have to go to the trouble of setting up the Distro and Migrating the Linux Install to ZFS Root, I don't want to have to do this too many times. So, which Distro are the favourite for Running ZFS-on-Linux? Also, is there such a thing as a Linux Distro that is smart enough to give you a choice if you are willing to use non-GPL'ed code in the Installer, so that I can just Install Directly on a ZFS Pool? Thank You in Advance for your Input. Regards, Amos

[Christopher Browne via talk]

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my Home
Network.
Given that I will have to go to the trouble of setting up the Distro and
Migrating the Linux Install to ZFS Root, I don't want to have to do this
too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to give
you a choice if you are willing to use non-GPL'ed code in the Installer, so
that I can just Install Directly on a ZFS Pool?

I think that if I were certain I wanted a ZFS machine on my network, I'd
look into one of the Illumos variants, most likely OmniOS.
https://omniosce.org/

That is a fork of Solaris, which is where ZFS is really native.

All the other implementations of ZFS are ports, and likely a bit less
satisfactory.

That obviously isn't Linux. But it shouldn't be ridiculously unfamiliar.

[David Collier-Brown via talk]

On Fri, Aug 24, 2018, 2:27 PM right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for
my Home Network.
Given that I will have to go to the trouble of setting up the
Distro and Migrating the Linux Install to ZFS Root, I don't want
to have to do this too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough
to give you a choice if you are willing to use non-GPL'ed code in
the Installer, so that I can just Install Directly on a ZFS Pool?
I think that if I were certain I wanted a ZFS machine on my network,
I'd look into one of the Illumos variants, most likely OmniOS.
https://omniosce.org/
That is a fork of Solaris, which is where ZFS is really native.
All the other implementations of ZFS are ports, and likely a bit less
satisfactory.
That obviously isn't Linux.  But it shouldn't be ridiculously unfamiliar.

I second Chris: most of the commands will be familiar, although
sometimes the names will merely be reminiscent of one another: strace vs
truss, for example.

I used to be in the porting business, and Solaris <-> Linux was dead easy.

--dave

[D. Hugh Redelmeier via talk]

| From: right.maple.nut via talk <***@gtalug.org>

| Hello All, Like the Subject Line says, I'm setting up a ZFS File Server
| for my Home Network. Given that I will have to go to the trouble of
| setting up the Distro and Migrating the Linux Install to ZFS Root, I
| don't want to have to do this too many times. So, which Distro are the
| favourite for Running ZFS-on-Linux? Also, is there such a thing as a
| Linux Distro that is smart enough to give you a choice if you are
| willing to use non-GPL'ed code in the Installer, so that I can just
| Install Directly on a ZFS Pool? Thank You in Advance for your Input.
| Regards, Amos

Why not run a *BSD?

Or something Solaris-related? Open Indiana????

- They more naturally run ZFS.

- there is no clash-of-licenses drama.

- the code is probably more tested.

- there are even out of the box fileserver distros based on *BSD.

- TrueOS, FreeNAS?

Is there something that you don't like about UNIX (as opposed to
Linux) for a file server?
---
Talk Mailing List
ta

[David Mason via talk]

I am interested in this question too.  I currently am running on Debian 7.7 but I’m not sure I can upgrade to a more current version, which is frustrating because I want to install Java (to run a Minecraft server) and (when I tried, so while back) I couldn’t get it to install because I had such an old version of Debian.

So I was thinking of FreeBSD, but OmniOS looks pretty interesting. I won’t be doing any upgrade for several months (because as far as I can see, I need to install a new ZFS system and copy it over. I can’t risk loosing this system). It would be good to have a backup, anyway. I could try this on a virtual system, but I don’t know long it would take to back-up over the network.

../Dave

Post by D. Hugh Redelmeier via talk
| Hello All, Like the Subject Line says, I'm setting up a ZFS File Server
| for my Home Network. Given that I will have to go to the trouble of
| setting up the Distro and Migrating the Linux Install to ZFS Root, I
| don't want to have to do this too many times. So, which Distro are the
| favourite for Running ZFS-on-Linux? Also, is there such a thing as a
| Linux Distro that is smart enough to give you a choice if you are
| willing to use non-GPL'ed code in the Installer, so that I can just
| Install Directly on a ZFS Pool? Thank You in Advance for your Input.
| Regards, Amos
Why not run a *BSD?
Or something Solaris-related? Open Indiana????
- They more naturally run ZFS.
- there is no clash-of-licenses drama.
- the code is probably more tested.
- there are even out of the box fileserver distros based on *BSD.
- TrueOS, FreeNAS?
Is there something that you don't like about UNIX (as opposed to
Linux) for a file server?
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[D. Hugh Redelmeier via talk]

| From: David Mason via talk <***@gtalug.org>

| I am interested in this question too. I currently am running on Debian
| 7.7 but I¢m not sure I can upgrade to a more current version, which is
| frustrating because I want to install Java (to run a Minecraft server)
| and (when I tried, so while back) I couldn¢t get it to install because I
| had such an old version of Debian.

Back in 2017 March 12 you mentioned a problem "Updating Wheezy to Jessie".
Did you try Stewart's and Lennart's suggestions?

This is exactly the kind of problem that a debian local users group might
address. Recently I suggested that GTALUG could function as a debian LUG.
I guess that here we have a test of this idea.

| So I was thinking of FreeBSD, but OmniOS looks pretty interesting. I
| won¢t be doing any upgrade for several months (because as far as I can
| see, I need to install a new ZFS system and copy it over. I can¢t risk
| loosing this system). It would be good to have a backup, anyway. I could
| try this on a virtual system, but I don¢t know long it would take to
| back-up over the network.

Even for the debian upgrade attempt it might be good to have a backup.

If one falls behind on updates, there is a greater incentive to stand
still and fall further behind. A vicious circle. I've had that happen.

After enough time, the old hardware becomes obsolete too and it makes
total system rebuild more sensible than update. In my most recent
example, I jumped about 15 years ahead in hardware.

[David Mason via talk]

| I am interested in this question too. I currently am running on Debian
| 7.7 but I’m not sure I can upgrade to a more current version, which is
| frustrating because I want to install Java (to run a Minecraft server)
| and (when I tried, so while back) I couldn’t get it to install because I
| had such an old version of Debian.
Back in 2017 March 12 you mentioned a problem "Updating Wheezy to Jessie".
Did you try Stewart's and Lennart's suggestions?

I put a little time into it, but didn’t get to success. Unfortunately, life intervened. It’s still important, but as you say, as you fall behind, it becomes more difficult to leap-frog into the present.

This is exactly the kind of problem that a debian local users group might
address. Recently I suggested that GTALUG could function as a debian LUG.
I guess that here we have a test of this idea.

I think it already does. What did you have in mind beyond the mailing list?

Even for the debian upgrade attempt it might be good to have a backup.

Yes, I know. :-)

After enough time, the old hardware becomes obsolete too and it makes
total system rebuild more sensible than update. In my most recent
example, I jumped about 15 years ahead in hardware.

This system is <5 years old, and at the time was kind-of leading edge. so I’m not worried about that.
It’s a 4.4Tb raidz2 at 64% and has performed flawlessly. Unfortunately I don’t really have the time to do any serious digging right now, either.

How do others backup their ZFS systems? Getting a 4T external drive doesn’t seem like the best plan, but maybe there isn’t any other choice.

../Dave

[D. Hugh Redelmeier via talk]

| From: David Mason via talk <***@gtalug.org>

| On Aug 25, 2018, 6:21 AM -0400, D. Hugh Redelmeier via talk <***@gtalug.org>, wrote:

| > | From: David Mason via talk <***@gtalug.org>

| > This is exactly the kind of problem that a debian local users group might
| > address. Recently I suggested that GTALUG could function as a debian LUG.
| > I guess that here we have a test of this idea.
|
| I think it already does. What did you have in mind beyond the mailing list?

You asked a year ago, got replies, hit problems, but didn't post them
to the list. Maybe you didn't expect the list to be of more help.
This isn't an accusation, only an observation.

| It¢s a 4.4Tb raidz2 at 64% and has performed flawlessly. Unfortunately I
| don¢t really have the time to do any serious digging right now, either.

I understand all too well.

| How do others backup their ZFS systems? Getting a 4T external drive
| doesn¢t seem like the best plan, but maybe there isn¢t any other choice.

I'm not a ZFS user, but I'll throw out a guess:

- take a snapshot (that may be the wrong terminology)

- use a dump(8) command that matches zfs.

[Giles Orr via talk]

Post by Jose A. Dias via talk
On Aug 25, 2018, 6:21 AM -0400, D. Hugh Redelmeier via talk <
| I am interested in this question too. I currently am running on Debian
| 7.7 but I’m not sure I can upgrade to a more current version, which is
| frustrating because I want to install Java (to run a Minecraft server)
| and (when I tried, so while back) I couldn’t get it to install because I
| had such an old version of Debian.
Back in 2017 March 12 you mentioned a problem "Updating Wheezy to Jessie".
Did you try Stewart's and Lennart's suggestions?
I put a little time into it, but didn’t get to success. Unfortunately,
life intervened. It’s still important, but as you say, as you fall behind,
it becomes more difficult to leap-frog into the present.
This is exactly the kind of problem that a debian local users group might
address. Recently I suggested that GTALUG could function as a debian LUG.
I guess that here we have a test of this idea.
I think it already does. What did you have in mind beyond the mailing list?
Even for the debian upgrade attempt it might be good to have a backup.
Yes, I know. :-)
After enough time, the old hardware becomes obsolete too and it makes
total system rebuild more sensible than update. In my most recent
example, I jumped about 15 years ahead in hardware.
This system is <5 years old, and at the time was kind-of leading edge. so
I’m not worried about that.
It’s a 4.4Tb raidz2 at 64% and has performed flawlessly. Unfortunately I
don’t really have the time to do any serious digging right now, either.
How do others backup their ZFS systems? Getting a 4T external drive
doesn’t seem like the best plan, but maybe there isn’t any other choice.

Actually, that sounds like a really good plan. In fact, buy two so you can
do rotating backups. Think about your alternatives - about the only one
that occurs to me is a tape drive. There used to be consumer-grade tape
backups, but they don't exist anymore and I'd argue this is no longer a
viable solution outside the data centre.

Buying external hard drives is a really good idea: they're dirt cheap (at
least compared to the alternative - failure of your primary). The first
backup will take a day or so, but after that you use rsync (and/or
rsnapshot or similar) and the backups are likely to run 30 minutes to 2
hours depending on the thrash on your drive. If you're going to use
something that does differential backups, make sure the backup drive is
larger than the source drive (rather than the default - we all assume "the
same size" is okay).

I use 2.5" spinning external 4TB drives as backups, one of which stays at
my parent's place and gets exchanged approximately weekly. This may not be
the right answer for you, but you seem to think the data on that big
machine of yours is important: stop and think about how things would be if
it failed. And then work out a backup system.

[Scott Sullivan via talk]

Post by David Mason via talk
This system is <5 years old, and at the time was kind-of leading
edge. so I’m not worried about that.
It’s a 4.4Tb raidz2 at 64% and has performed flawlessly.
Unfortunately I don’t really have the time to do any serious digging
right now, either.
How do others backup their ZFS systems? Getting a 4T external drive
doesn’t seem like the best plan, but maybe there isn’t any other choice.

In my case I built a secondary NAS and disk array, and do regular 'zfs
snapshots' and 'zfs sends'. In recent history I've started using
zfs-snap-manager to automate that.

https://github.com/khenderick/zfs-snap-manager

It's a rather coarse tool... doesn't support automate snapshots more
frequent then once a day, but will happily send over any you've made
manually (via a cron job or alternative method).

Currently the developer has only packaged it for Arch. But I've built an
rpm spec file for it. Attached.

Post by David Mason via talk
Actually, that sounds like a really good plan.  In fact, buy two so you
can do rotating backups.  Think about your alternatives - about the only
one that occurs to me is a tape drive.  There used to be consumer-grade
tape backups, but they don't exist anymore and I'd argue this is no
longer a viable solution outside the data centre.
Buying external hard drives is a really good idea: they're dirt cheap
(at least compared to the alternative - failure of your primary).

I agree with Giles. If you don't want to drop the coin on a second NAS,
this is a very usable strategy. Get a 6 or even 8TB disk, format it as a
ZFS pool and turn on zfs's block compression, and set copies to '2'.

zfs set compression=lz4 <pool>
zfs set copies=2 <pool>

Setting a number of copies, is normally not useful for a multi-disk
array, as the copies can end up on the same disk. But on a single disk,
they are an insurance policy against bad sectors.

Then you just zfs send your snapshots to it. I regularly use this as a
local backup strategy with my work laptops.

[David Mason via talk]

OK, so I have an 8TB Seagate USB disk and have created a zpool on it called
backup1. My main pool is called tank. I tried:

: ~ ; sudo zfs snapshot -r ***@2018-08-31

: ~ ; sudo zfs list

NAME USED AVAIL REFER MOUNTPOINT

backup1 508K 7.14T 136K /backup1

tank 1.66T 916G 412K /tank

tank/audio 12.1G 916G 12.1G /audio

tank/cvs 32.7M 916G 32.7M /tank/cvs

tank/etc 18.1M 916G 18.1M /tank/etc

tank/home 531G 916G 531G /home

: ~ ; sudo zfs list -t snapshot

NAME USED AVAIL REFER MOUNTPOINT

***@2018-08-31 0 - 412K -

tank/***@2018-08-31 0 - 12.1G -

tank/***@2018-08-31 0 - 32.7M -

tank/***@2018-08-31 0 - 18.1M -

tank/***@2018-08-31 0 - 531G -
and now I try (after some research):

: ~ ; sudo zfs send -R ***@2018-08-31 | sudo zfs recv -vd backup1

cannot receive new filesystem stream: destination 'backup1' exists

must specify -F to overwrite it

warning: cannot send '***@2018-08-31': Broken pipe

Any quick help?

Thanks ../Dave

Post by David Mason via talk

Post by David Mason via talk
This system is <5 years old, and at the time was kind-of leading
edge. so I’m not worried about that.
It’s a 4.4Tb raidz2 at 64% and has performed flawlessly.
Unfortunately I don’t really have the time to do any serious digging
right now, either.
How do others backup their ZFS systems? Getting a 4T external drive
doesn’t seem like the best plan, but maybe there isn’t any other

choice.
In my case I built a secondary NAS and disk array, and do regular 'zfs
snapshots' and 'zfs sends'. In recent history I've started using
zfs-snap-manager to automate that.
https://github.com/khenderick/zfs-snap-manager
It's a rather coarse tool... doesn't support automate snapshots more
frequent then once a day, but will happily send over any you've made
manually (via a cron job or alternative method).
Currently the developer has only packaged it for Arch. But I've built an
rpm spec file for it. Attached.

Post by David Mason via talk
Actually, that sounds like a really good plan. In fact, buy two so you
can do rotating backups. Think about your alternatives - about the only
one that occurs to me is a tape drive. There used to be consumer-grade
tape backups, but they don't exist anymore and I'd argue this is no
longer a viable solution outside the data centre.
Buying external hard drives is a really good idea: they're dirt cheap
(at least compared to the alternative - failure of your primary).

I agree with Giles. If you don't want to drop the coin on a second NAS,
this is a very usable strategy. Get a 6 or even 8TB disk, format it as a
ZFS pool and turn on zfs's block compression, and set copies to '2'.
zfs set compression=lz4 <pool>
zfs set copies=2 <pool>
Setting a number of copies, is normally not useful for a multi-disk
array, as the copies can end up on the same disk. But on a single disk,
they are an insurance policy against bad sectors.
Then you just zfs send your snapshots to it. I regularly use this as a
local backup strategy with my work laptops.
--
Scott Sullivan
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[Scott Sullivan via talk]

Post by David Mason via talk
OK, so I have an 8TB Seagate USB disk and have created a zpool on it
: ~ ; sudo zfs list
NAME            USED  AVAIL  REFER  MOUNTPOINT
backup1         508K  7.14T   136K  /backup1
tank           1.66T   916G   412K  /tank
tank/audio     12.1G   916G  12.1G  /audio
tank/cvs       32.7M   916G  32.7M  /tank/cvs
tank/etc       18.1M   916G  18.1M  /tank/etc
tank/home       531G   916G   531G  /home
: ~ ; sudo zfs list -t snapshot
NAME                        USED  AVAIL  REFER  MOUNTPOINT
cannot receive new filesystem stream: destination 'backup1' exists
must specify -F to overwrite it
Any quick help?
Thanks  ../Dave

That's expected Dave.

Because backup1 is a new filesystem, it is inherently not a decedent of
of your source zfs data set and is so a name collision. So -F to force
is perfectly reasonable to remove the empty dataset and replicate your
source into the pool.

In following backups, you'd use the last common snapshot and most recent
snapshot as arguments to 'zfs send -I'.

An example from my own shell history:
zfs send -I
jarvis-charlie/backups/***@20180325_224731-0400
jarvis-charlie/backups/***@20180401_121435-0400 | ssh
***@example.someplace.revident.ca "zfs receive -d jarvis-dr"

[Alex Volkov via talk]

Hey Amos,

Why not FreeBSD?

This is one of the first projects to get ZFS ported, in my experience
it's the closest thing to general-purpose linux distro in BSD land that
you can get.

All the GNU tools are available for FreeBSD and shell commands aren't
that arcane.

If you're only going to use the machine for hosting ZFS it might be
worth it.

Alex.

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my
Home Network.
Given that I will have to go to the trouble of setting up the Distro
and Migrating the Linux Install to ZFS Root, I don't want to have to
do this too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to
give you a choice if you are willing to use non-GPL'ed code in the
Installer, so that I can just Install Directly on a ZFS Pool?
Thank You in Advance for your Input.
Regards,
Amos
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[Jose A. Dias via talk]

Something like this:

https://www.canadacomputers.com/search/product.php?cPath=14_679 <https://www.canadacomputers.com/search/product.php?cPath=14_679&item_id=069771> &item_id=069771

External USB, preferable 3.0 over 2, or eSata if you have the port. If you doint' need two drive capacity then pick a single. Add the current the drive you need and let it copy. I tend to use ext4 for externals so to minimize complexity on attaching to other systems, but YMMV.


[Jose A. Dias] -----Original Message-----
From: talk [mailto:talk-***@gtalug.org]On Behalf Of David Mason via talk
Sent: Saturday, August 25, 2018 10:21 AM
To: GTALUG Talk
Subject: Re: [GTALUG] Which Distro is Best for Running a ZFS-on-Linux Fileserver.


On Aug 25, 2018, 6:21 AM -0400, D. Hugh Redelmeier via talk <***@gtalug.org>, wrote:


| From: David Mason via talk <***@gtalug.org>

| I am interested in this question too. I currently am running on Debian
| 7.7 but I’m not sure I can upgrade to a more current version, which is
| frustrating because I want to install Java (to run a Minecraft server)
| and (when I tried, so while back) I couldn’t get it to install because I
| had such an old version of Debian.

Back in 2017 March 12 you mentioned a problem "Updating Wheezy to Jessie".
Did you try Stewart's and Lennart's suggestions?



I put a little time into it, but didn’t get to success. Unfortunately, life intervened. It’s still important, but as you say, as you fall behind, it becomes more difficult to leap-frog into the present.


This is exactly the kind of problem that a debian local users group might
address. Recently I suggested that GTALUG could function as a debian LUG.
I guess that here we have a test of this idea.



I think it already does. What did you have in mind beyond the mailing list?


Even for the debian upgrade attempt it might be good to have a backup.



Yes, I know. :-)


After enough time, the old hardware becomes obsolete too and it makes
total system rebuild more sensible than update. In my most recent
example, I jumped about 15 years ahead in hardware.



This system is <5 years old, and at the time was kind-of leading edge. so I’m not worried about that.
It’s a 4.4Tb raidz2 at 64% and has performed flawlessly. Unfortunately I don’t really have the time to do any serious digging right now, either.

How do others backup their ZFS systems? Getting a 4T external drive doesn’t seem like the best plan, but maybe there isn’t any other choice.

../Dave

[Scott Sullivan via talk]

Having read through the thread to date, I'm actually a little
disappointed at the number of linux users pushing towards a Solaris or
BSD for ZFS.

My primary File servers (4 of them) are all using ZFS for their data
partitions.


Amos,

## Couple of Answers to your questions

A) Disto?

I regularly run ZFS on CentOS and Fedora on a mix of SSDs and HDDs of
both the internal and external varieties. Fedora has some caveats, only
in that sometimes the kernel releases get ahead of what the ZFS on linux
team will support. And it's just a matter of waiting on a working kernel
zfs combination a week or two for them to catch up.

But frankly, just pick your favorite distro and follow the relevant
getting started guide.

https://zfsonlinux.org/


B) Distro with ZFS root support (at install time)?

No distro install supports this yet as I've seen. Although the do it
yourself ubuntu guide is lengthy, but very well detailed.

https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS

Arch also support ZFS root, but their installation is all largely manual
to begin with.

https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS


## Couple of my own Questions

1) Why root (/) on ZFS, what is your use case / risk your trying to
mitigate?

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my Home
Network.
Given that I will have to go to the trouble of setting up the Distro and
Migrating the Linux Install to ZFS Root, I don't want to have to do this
too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to
give you a choice if you are willing to use non-GPL'ed code in the
Installer, so that I can just Install Directly on a ZFS Pool?
Thank You in Advance for your Input.
Regards,
Amos
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[Alvin Starr via talk]

At the risk of poking the bear.

Why use ZFS at all?

ext4,XFS+LVM will do most all of the same things and from what I have
read ZFS is slower than ext4,XFS and BTRFS.

Post by Scott Sullivan via talk
Having read through the thread to date, I'm actually a little
disappointed at the number of linux users pushing towards a Solaris or
BSD for ZFS.
My primary File servers (4 of them) are all using ZFS for their data
partitions.
Amos,
## Couple of Answers to your questions
A) Disto?
I regularly run ZFS on CentOS and Fedora on a mix of SSDs and HDDs of
both the internal and external varieties. Fedora has some caveats,
only in that sometimes the kernel releases get ahead of what the ZFS
on linux team will support. And it's just a matter of waiting on a
working kernel zfs combination a week or two for them to catch up.
But frankly, just pick your favorite distro and follow the relevant
getting started guide.
https://zfsonlinux.org/
B)  Distro with ZFS root support (at install time)?
No distro install supports this yet as I've seen. Although the do it
yourself ubuntu guide is lengthy, but very well detailed.
https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS
Arch also support ZFS root, but their installation is all largely
manual to begin with.
https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS
## Couple of my own Questions
1) Why root (/) on ZFS, what is your use case / risk your trying to
mitigate?

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my
Home Network.
Given that I will have to go to the trouble of setting up the Distro
and Migrating the Linux Install to ZFS Root, I don't want to have to
do this too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to
give you a choice if you are willing to use non-GPL'ed code in the
Installer, so that I can just Install Directly on a ZFS Pool?
Thank You in Advance for your Input.
Regards,
Amos
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[David Mason via talk]

I haven’t done a comparison lately, but used LVM for years, and liked it, but ZFS is so much more flexible: resize filesystems, snapshots, raidz2, sending to another ZFS system (for seamless backups and redundancy), auto-silvering, plug-and-play.

../Dave

Post by Alvin Starr via talk
At the risk of poking the bear.
Why use ZFS at all?
ext4,XFS+LVM will do most all of the same things and from what I have
read ZFS is slower than ext4,XFS and BTRFS.

Post by Scott Sullivan via talk
Having read through the thread to date, I'm actually a little
disappointed at the number of linux users pushing towards a Solaris or
BSD for ZFS.
My primary File servers (4 of them) are all using ZFS for their data
partitions.
Amos,
## Couple of Answers to your questions
A) Disto?
I regularly run ZFS on CentOS and Fedora on a mix of SSDs and HDDs of
both the internal and external varieties. Fedora has some caveats,
only in that sometimes the kernel releases get ahead of what the ZFS
on linux team will support. And it's just a matter of waiting on a
working kernel zfs combination a week or two for them to catch up.
But frankly, just pick your favorite distro and follow the relevant
getting started guide.
https://zfsonlinux.org/
B)  Distro with ZFS root support (at install time)?
No distro install supports this yet as I've seen. Although the do it
yourself ubuntu guide is lengthy, but very well detailed.
https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS
Arch also support ZFS root, but their installation is all largely
manual to begin with.
https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS
## Couple of my own Questions
1) Why root (/) on ZFS, what is your use case / risk your trying to
mitigate?

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my
Home Network.
Given that I will have to go to the trouble of setting up the Distro
and Migrating the Linux Install to ZFS Root, I don't want to have to
do this too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to
give you a choice if you are willing to use non-GPL'ed code in the
Installer, so that I can just Install Directly on a ZFS Pool?
Thank You in Advance for your Input.
Regards,
Amos
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

--
Alvin Starr || land: [phone]
Netvel Inc. || Cell: [phone]
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[Amos H. Weatherill via talk]

Hello All,

Thank you for your Feedback and Discussion.

FreeBSD is a good suggestion but I don't want to wander away from what I know and as Scott pointed out, this is a Linux User Group ...

Also, once I get More RAM, I will want to Consolidate Services running on other machines on to the NAS.

Currently, I have 8 GB of RAM, which my reading says should be enough to support a 4 TB ZFS Pool and the Necessary Samba configuration.

At the moment, my Network copy speed appears to be limited to about 25 MB/s but I suspect that this is because my Shares are on a Windows Machine with a PCI SATA Card.

So, if the NAS can do better, I'll declare Victory ...

Scott,

My reasoning for / on ZFS is pretty Simple ... the machine that is becoming my first NAS only has 4 SATA Ports, so I can't afford to Waste one on a boot drive.

For Distro, I think I'll go with Fedora, as long as the / on ZFS guide is sufficiently detailed.

Thank You All,
Amos

Sent from my android device.

-----Original Message-----
From: Scott Sullivan via talk <***@gtalug.org>
To: ***@gtalug.org
Sent: Tue, 28 Aug 2018 5:19 PM
Subject: Re: [GTALUG] Which Distro is Best for Running a ZFS-on-Linux Fileserver.

Having read through the thread to date, I'm actually a little
disappointed at the number of linux users pushing towards a Solaris or
BSD for ZFS.

My primary File servers (4 of them) are all using ZFS for their data
partitions.


Amos,

## Couple of Answers to your questions

A) Disto?

I regularly run ZFS on CentOS and Fedora on a mix of SSDs and HDDs of
both the internal and external varieties. Fedora has some caveats, only
in that sometimes the kernel releases get ahead of what the ZFS on linux
team will support. And it's just a matter of waiting on a working kernel
zfs combination a week or two for them to catch up.

But frankly, just pick your favorite distro and follow the relevant
getting started guide.

https://zfsonlinux.org/


B) Distro with ZFS root support (at install time)?

No distro install supports this yet as I've seen. Although the do it
yourself ubuntu guide is lengthy, but very well detailed.

https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS

Arch also support ZFS root, but their installation is all largely manual
to begin with.

https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS


## Couple of my own Questions

1) Why root (/) on ZFS, what is your use case / risk your trying to
mitigate?

Post by right.maple.nut via talk
Hello All,
Like the Subject Line says, I'm setting up a ZFS File Server for my Home
Network.
Given that I will have to go to the trouble of setting up the Distro and
Migrating the Linux Install to ZFS Root, I don't want to have to do this
too many times.
So, which Distro are the favourite for Running ZFS-on-Linux?
Also, is there such a thing as a Linux Distro that is smart enough to
give you a choice if you are willing to use non-GPL'ed code in the
Installer, so that I can just Install Directly on a ZFS Pool?
Thank You in Advance for your Input.
Regards,
Amos
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk

[Scott Sullivan via talk]

Post by Amos H. Weatherill via talk
Scott,
My reasoning for / on ZFS is pretty Simple ... the machine that is
becoming my first NAS only has 4 SATA Ports, so I can't afford to Waste
one on a boot drive.

Recommended best Practice is to use ZFS with whole disks. That said,
most of the arguments for that are 'because the manual says so',
'because zfs datasets are far more flexible then partitions' and
references to Solairs taking advantage of disk caches. I throw that all
out the windows in favor of doing at rest encryption, with whole luks
partitions(*).

My more practical argument is choice of MBR vs GUID partitioning. The
latter is just cleaner (and the default when ZFS manages the disk), and
works well with large disks (>2TB).

But if your booting from that disk, you either need to be:
"BIOS / CSM" + MBR + /boot
or
UEFI + GUID + "biosboot (partition)" + /boot

Either of those makes for some lopsided partitioning, compared to the
remainder of your data disks. A work around is to use a USB drive for
your /boot. But in general your creating a more complex setup to
maintain either way.

Not knowing what hardware your using, if you have PCIe slots additional
sata ports can be had for a low a $10/port.

I've been using the Syba / IOCrest cards for a variety of needs,
including ZFS arrays without issue.

https://www.newegg.ca/Product/Product.aspx?Item=N82E16816124064

Post by Amos H. Weatherill via talk
For Distro, I think I'll go with Fedora, as long as the / on ZFS guide
is sufficiently detailed.

Fedora was not one of the ones I listed as having a guide to do rootfs
on ZFS. If you found one, can you post the link?

I'd also not recommend fedora in general for a NAS. CentOS would be a
more dependable choice. LTS Ubuntu would be more reasonable as they
ship(**) ZFS and support rootfs on it.


===
* Native encryption in ZFS was added after the OpenZFS split from
Sun/Oracle. So work to re-added it has been happening for a while. We're
likely to see a stable version in the v0.8.x series.

** This is due to their adoption of a minority legal opinion about
compatibility of the CDDL and GPL licenses that has not been tested in
court.
https://blog.ubuntu.com/2016/02/18/zfs-licensing-and-linux
https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

[David Mason via talk]

If you’re building 4TB, I’t recommend raidz2 which doesn’t give you much storage (4 x 1TB disks would give you <2TB of user storage). I have 5 x 1TB raidz2 which gives me 2.82TB of user storage. This may have changed, but when I went to ZFS you couldn’t add more disks and change the structure, you could only enlarge the disks - so if you went for a 5-disk raidz2, you were stuck with a 5-disk raidz2. I really hope this has changed, but I haven’t researched it.

The reason to go with raidz2 as the pool gets larger is that with raidz if a disk dies, while it rebuilds on the replacement disk, you have no redundancy, so any error on one of the other disks will be replicated.

So I’d recommend you get an additional SATA card, and then you don’t need to do ZFS on /, and you could add another drive to get better cost-effectiveness from your ZFS. I have / on a moderate-sized SSD and 5 hard drives in a ZFS pool.

One reason to use not-whole-disks for ZFS is that you can migrate to larger disks as time goes on.  I started out with 500GB (if I remember correctly) and was able to move to 1TB and copy the partitions.

Scott knows much more about this than I, but it may (now) be that (with a raidz2) you can enlarge by simply removing a xGB disk and replacing it with a 2xGB drive, let it rebuild, then replace the next one. The problem with that is you are replacing the disks, so it’s not very incremental. So if you can enlarge by adding an additional drive, that is substantially better.

../Dave

Post by Scott Sullivan via talk

Post by Amos H. Weatherill via talk
Scott,
My reasoning for / on ZFS is pretty Simple ... the machine that is
becoming my first NAS only has 4 SATA Ports, so I can't afford to Waste
one on a boot drive.

Recommended best Practice is to use ZFS with whole disks. That said,
most of the arguments for that are 'because the manual says so',
'because zfs datasets are far more flexible then partitions' and
references to Solairs taking advantage of disk caches. I throw that all
out the windows in favor of doing at rest encryption, with whole luks
partitions(*).
My more practical argument is choice of MBR vs GUID partitioning. The
latter is just cleaner (and the default when ZFS manages the disk), and
works well with large disks (>2TB).
"BIOS / CSM" + MBR + /boot
or
UEFI + GUID + "biosboot (partition)" + /boot
Either of those makes for some lopsided partitioning, compared to the
remainder of your data disks. A work around is to use a USB drive for
your /boot. But in general your creating a more complex setup to
maintain either way.
Not knowing what hardware your using, if you have PCIe slots additional
sata ports can be had for a low a $10/port.
I've been using the Syba / IOCrest cards for a variety of needs,
including ZFS arrays without issue.
https://www.newegg.ca/Product/Product.aspx?Item=N82E16816124064

Post by Amos H. Weatherill via talk
For Distro, I think I'll go with Fedora, as long as the / on ZFS guide
is sufficiently detailed.

Fedora was not one of the ones I listed as having a guide to do rootfs
on ZFS. If you found one, can you post the link?
I'd also not recommend fedora in general for a NAS. CentOS would be a
more dependable choice. LTS Ubuntu would be more reasonable as they
ship(**) ZFS and support rootfs on it.
===
* Native encryption in ZFS was added after the OpenZFS split from
Sun/Oracle. So work to re-added it has been happening for a while. We're
likely to see a stable version in the v0.8.x series.
** This is due to their adoption of a minority legal opinion about
compatibility of the CDDL and GPL licenses that has not been tested in
court.
https://blog.ubuntu.com/2016/02/18/zfs-licensing-and-linux
https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/
--
Scott Sullivan
---
Talk Mailing List
https://gtalug.org/mailman/listinfo/talk