[GTALUG] Backups with Bacula

[Tony Fernandez via talk]

Hi All,

I just recently joined your group and was hoping for some insight or advise
on building a backup solution.

My question is related to Bacula on FreeNAS. I hope it's ok that I ask if
not I understand.

==Background==
We run a mixed environment (Windows and Linux). I've been slowly moving
services that we run over to Linux wherever possible however I've run into
an issue with our existing backup server. It is a Microsoft product. I'm
not looking for help on it, instead I am looking for advise on using bacula.

I've configured Bacula and have it working successfully and I like how I
can deploy agents onto my servers to handle backups. I love the distributed
achitecture.

So my question: I'd like to use FreeNAS as our StorageDaemon (sd) going
forward. I'd also like to do remote offsite backups by either rsyncing
files over to a remote server or by getting FreeNAS to backup to an
external HDD that I rotate weekly.

==Questions==
Does anyone see any issues with this?
What do you think about the HDD rotation?

Thanks,
Tony

[Christopher Browne via talk]

Post by Tony Fernandez via talk
Hi All,
I just recently joined your group and was hoping for some insight or advise on building a backup solution.
My question is related to Bacula on FreeNAS. I hope it's ok that I ask if not I understand.
==Background==
We run a mixed environment (Windows and Linux). I've been slowly moving services that we run over to Linux wherever possible however I've run into an issue with our existing backup server. It is a Microsoft product. I'm not looking for help on it, instead I am looking for advise on using bacula.
I've configured Bacula and have it working successfully and I like how I can deploy agents onto my servers to handle backups. I love the distributed achitecture.
So my question: I'd like to use FreeNAS as our StorageDaemon (sd) going forward. I'd also like to do remote offsite backups by either rsyncing files over to a remote server or by getting FreeNAS to backup to an external HDD that I rotate weekly.
==Questions==
Does anyone see any issues with this?
What do you think about the HDD rotation?

Bacula has always seemed to be one of the good options out there, and
running it on FreeNAS is certainly well supported.

There's nothing obviously wrong with your approach to rsync to a
remote place or copy to external HDD for rotation.

Madison Kelly did a talk on something akin back in 2004; Madison was
the first person I heard that particularly "championed" using
USB-connected HDDs as a backup medium at the time that tape drives
were only just starting to get supplanted as a backup medium.

Since then, that direction has become somewhere in between "viable"
and "preferable." And it now looks like tape drives are pretty rarely
used anymore, as rarity has made it difficult for vendors to boost
capacity as quickly as is the case for disk drives. *Everyone* wants
bigger HDDs. (Well, we're starting to glimpse a place where solid
state drives are getting sufficiently large and cheap that a lot of
computer systems now prefer SSD, and we may see HDDs go somewhat down
the road that tape drives have...)

Rotating the HDDs so that they do get spun up fairly regularly is a good idea.

[Lennart Sorensen via talk]

Post by Christopher Browne via talk
Bacula has always seemed to be one of the good options out there, and
running it on FreeNAS is certainly well supported.
There's nothing obviously wrong with your approach to rsync to a
remote place or copy to external HDD for rotation.
Madison Kelly did a talk on something akin back in 2004; Madison was
the first person I heard that particularly "championed" using
USB-connected HDDs as a backup medium at the time that tape drives
were only just starting to get supplanted as a backup medium.
Since then, that direction has become somewhere in between "viable"
and "preferable." And it now looks like tape drives are pretty rarely
used anymore, as rarity has made it difficult for vendors to boost
capacity as quickly as is the case for disk drives. *Everyone* wants
bigger HDDs. (Well, we're starting to glimpse a place where solid
state drives are getting sufficiently large and cheap that a lot of
computer systems now prefer SSD, and we may see HDDs go somewhat down
the road that tape drives have...)
Rotating the HDDs so that they do get spun up fairly regularly is a good idea.

My experience some years ago with 3 USB harddrives that were rotated
weekly was that the disks didn't last long. 3.5" HDs do not like
being moved a lot and frequently died. Moving to tape was way way
more reliable but certainly had a higher cost in terms of getting a
tape drive and for recovery you might need another tape drive while a
USB drive works with anything.

If your backup is pretty small though, USB attached SSD seems like it
could be a very reliable solution.

[Anthony de Boer via talk]

Post by Lennart Sorensen via talk
My experience some years ago with 3 USB harddrives that were rotated
weekly was that the disks didn't last long. 3.5" HDs do not like
being moved a lot and frequently died. Moving to tape was way way
more reliable but certainly had a higher cost in terms of getting a
tape drive and for recovery you might need another tape drive while a
USB drive works with anything.

You tend to only need spinning rust nowadays for the largest datasets,
and I find that splitting archival from active filesystems helps. The
archival stuff tends to be huge but write-only: media files and tarballs
and such get written and then left as-is for the life of the filesystem.
That requires a much-less-frequent backup! And that reduces the size of
the active filesystem plus any new archival files to the point it has a
chance of fitting onto flash.

Another way the world has changed is that sites are no longer such
islands connected by 56k modems and sneakernet; there's more chance of
being able to push a backup over fibre somewhere to a harddrive that's
already offsite and doesn't need to be moved further. Having another
datacentre to rsync to is ideal, as I'm not sure how much you can trust
third-party "cloud" solutions.

[Giles Orr via talk]

Post by Lennart Sorensen via talk

Post by Christopher Browne via talk
Bacula has always seemed to be one of the good options out there, and
running it on FreeNAS is certainly well supported.
There's nothing obviously wrong with your approach to rsync to a
remote place or copy to external HDD for rotation.
Madison Kelly did a talk on something akin back in 2004; Madison was
the first person I heard that particularly "championed" using
USB-connected HDDs as a backup medium at the time that tape drives
were only just starting to get supplanted as a backup medium.
Since then, that direction has become somewhere in between "viable"
and "preferable." And it now looks like tape drives are pretty rarely
used anymore, as rarity has made it difficult for vendors to boost
capacity as quickly as is the case for disk drives. *Everyone* wants
bigger HDDs. (Well, we're starting to glimpse a place where solid
state drives are getting sufficiently large and cheap that a lot of
computer systems now prefer SSD, and we may see HDDs go somewhat down
the road that tape drives have...)
Rotating the HDDs so that they do get spun up fairly regularly is a good

idea.
My experience some years ago with 3 USB harddrives that were rotated
weekly was that the disks didn't last long. 3.5" HDs do not like
being moved a lot and frequently died. Moving to tape was way way
more reliable but certainly had a higher cost in terms of getting a
tape drive and for recovery you might need another tape drive while a
USB drive works with anything.
If your backup is pretty small though, USB attached SSD seems like it
could be a very reliable solution.

I used to use a rotating set of 2TB 2.5" external USB hard drives. None of
them ever failed on me over about three years use, although three out of
four they were generally only accessed every week or two. I've now
switched to three 4TB 2.5" drives: the heavily used one of those is now
stuttering (confirming my pre-existing bias against Seagate ... the
previous set were WD). The 2.5" spinning drives are somewhat more
expensive than the 3.5", but they're much smaller - and, important in this
use case, more built for movement and frequent spin-up/spin-down.

And I have to disagree with Anthony on this one: spinning disks are going
to be in use for a while as external backups: sure, they're slow, but this
is a BACKUP. Cost per terabyte is immensely lower and speed isn't usually
the priority, and you can put multiple copies of backups (or diffs, or
whatever suits you) on one large external.

[Howard Gibson via talk]

On Mon, 29 Oct 2018 15:35:20 -0400

Post by Giles Orr via talk
I used to use a rotating set of 2TB 2.5" external USB hard drives. None of
them ever failed on me over about three years use, although three out of
four they were generally only accessed every week or two. I've now
switched to three 4TB 2.5" drives: the heavily used one of those is now
stuttering (confirming my pre-existing bias against Seagate ... the
previous set were WD). The 2.5" spinning drives are somewhat more
expensive than the 3.5", but they're much smaller - and, important in this
use case, more built for movement and frequent spin-up/spin-down.

I have been using an internal 4TB hard drive as my nightly backup. I am now on to my second hard drive. Periodically, I transfer my backup to a 50GB Blu-Ray disk. Obviously, this constrains my /home partion to an extent that may be unacceptable to you. On a couple of occasasion now, I have pulled out months old disks to recover files I unknowingly deleted. I like to permanently archive my backups.

I am getting concerned about Blu-Ray burners. Is there another cheap, high capacity medium out there?

[Anthony de Boer via talk]

Post by Giles Orr via talk
And I have to disagree with Anthony on this one: spinning disks are going
to be in use for a while as external backups: sure, they're slow, but this
is a BACKUP. Cost per terabyte is immensely lower and speed isn't usually
the priority, and you can put multiple copies of backups (or diffs, or
whatever suits you) on one large external.

Certainly you're going to use spinning rust for multi-terabyte backup
sets, but the point I was getting at is that if that's your archival set
whose files are set in stone once written then you may only need to do
backups of that monthly or less often, while a small active fileset
(including any recent archival additions) may be in the tens or hundreds
of gigabytes range so that your dailies are small and manageable and have
a good chance fitting onto flash.