Available NS7

Stephane de Labrusse at stephdl@de-labrusse.fr

Is this Module helpful to you ? Please consider donating

* First you need to install my repository, see how to do it

* You need to install the bareos repository

wget http://download.bareos.org/bareos/release/latest/CentOS_7/bareos.repo -O /etc/yum.repos.d/bareos.repo

* then you can install the rpm

yum install nethserver-bareos --enablerepo=stephdl

It's possible to access bareos the first time pointing the browser to http://your_ip/bareos-webui, restricted to your local network, access from WAN are refused.

you have a link at the Applications menu of Nethgui and cockpit

Obviously you can add more users for the UI of bareos, you simply have to create a bareos group inside the server-manager of NethServer, all members of this group will be granted to use the web UI of bareos. You have to use the short format name (user) to login to bareos, it seems the long format is not accepted(user@domain.org)

By default it is ``/var/lib/bareos/storage/`` that is used to store all your backups, either mount a storage to this path, or check the link at the bottom to use another path

The manual is intended to be the extended resource for all standard questions on using Bareos at https://docs.bareos.org/

Set up a backup client Set up a Relax-and-Recover with Bareos

Bareos uses rh-postgresql12, we need to enable scl before to use a bareos script and set the ENV custom port of postgresql (TCP 55434)

for example

- upgrade the database structure

  su - postgres  -c "export PGPORT=55434;scl enable rh-postgresql12 /usr/lib/bareos/scripts/update_bareos_tables"

- backup database to /var/lib/bareos

export PGPORT=55434;scl enable rh-postgresql12 -- /usr/lib/bareos/scripts/make_catalog_backup.pl MyCatalog

An excellent documentation comes from https://www.svennd.be/bareos-articles/ I use it as a source for all these examples

You can backup a NethServer or any other client(windows, Linux), for a client linux install `bareos-fd` and open the TCP port '9102' in the firewall, for NethServer you simply need to install nethserver-bareos-client from my repository.

* First you need to install my repository, see how to do it

* You need to install the bareos repository

wget http://download.bareos.org/bareos/release/latest/CentOS_7/bareos.repo -O /etc/yum.repos.d/bareos.repo

* then you can install the rpm

yum install nethserver-bareos-client --enablerepo=stephdl

Depending how works your network and specifically the domain name resolution, you might have in the client to fill the domain name of the backup server with its IP in the /etc/hosts, use the UI or make a custom-template. you should give the host name (host) and the Fully qualified domain name (host.domain.org)

192.168.56.14     	ns7loc13.nethservertest.org ns7loc13

once installed, go to the console on the backup server. Since the web interface can’t be used (yet?) lets fire up bconsole. Adding the client is done like this (the client here is named 'swennd'):

You need to think about good name for your client, it could be quickly a mess if you do not use relevant name.

bconsole
configure add client name=svennd address=192.168.0.2 password=SOME_PASSWORD

This should result in something like :

*configure add client name=svennd address=8.8.8.8 password=alfa123
Exported resource file "/etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf":
Director {
  Name = bareos-dir
  Password = "[md5]f7180d44d3cad452f611e2dfdfd3b007"
}
Created resource config file "/etc/bareos/bareos-dir.d/client/svennd.conf":
Client {
  Name = svennd
  Address = 8.8.8.8
  Password = alfa123
}

After that the configuration for the bareos director has been made, the only configuration that still is missing is on the client, this has been made by the bconsole, and can be found in /etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf we need to copy that to /etc/bareos/bareos-fd.d/director/ on the client machine.

This can be copied using scp to the client.

scp /etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf root@8.8.8.8:/etc/bareos/bareos-fd.d/director/ && ssh root@8.8.8.8 "systemctl restart bareos-fd"

Then on the client you have to restart the file daemon after any changes:

systemctl restart bareos-fd

And that’s it, now we are ready to start creating a backup job.

you can valid that the server/client can communicate

bconsole

then type:

*status client
The defined Client resources are:
     1: bareos-fd
     2: svennd
Select Client (File daemon) resource (1-5): 2
Connecting to Client ns7loc10 at 192.168.56.11:9102
Probing client protocol... (result will be saved until config reload)
 Handshake: Immediate TLS, Encryption: PSK-AES256-CBC-SHA

svennd-fd Version: 19.2.6 (11 February 2020)  Linux-3.10.0-1062.9.1.el7.x86_64 redhat CentOS Linux release 7.7.1908 (Core)
Daemon started 05-Apr-20 16:08. Jobs: run=0 running=0, bareos.org build binary
 Sizeof: boffset_t=8 size_t=8 debug=0 trace=0 bwlimit=0kB/s

Running Jobs:
bareos-dir (director) connected at: 05-Apr-20 20:03
No Jobs running.
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name 
======================================================================
====

In the last posts, we setup bareos and connected a client to the bareos server, there are however no backup jobs auto-magically added to the bareos server so that’s our job. At first creating a job seems difficult and complex, but most of the options only become interesting in specific applications and a simple basic job can be configured pretty easy. That’s what I am going to do now, create a backup job that will incremental backup the /etc directory of my Linux client.

First we need to understand the target of the job, this step is important, if you don’t do it you might end up with something completely different due to the options possible with bareos.

* run a backup of /etc daily during non-business hours, but before the nightly update. * To save some space, I would take incremental updates. That means I always check what changes have occurred since the last incremental update. This makes that if nothing changes 0 extra bytes are used. * Disk storage, we have a tape library, but lets start with normal disk storage

Since we have a pretty well defined time of day to run the backup job, let’s start with first creating the specific schedule. In bareos all the configurations are nicely filtered in config files. In the directory `/etc/bareos` since we are configuring jobs that will be “directed” by the director, we need to go in the director configuration : /etc/bareos/bareos-dir.d/ there we can find the schedule directory. There might already be some examples there. But let’s create a new schedule :

nano /etc/bareos/bareos-dir.d/schedule/Nightly.conf

with the content :

Schedule {
  # name (required)
  Name = "Nightly"
  
  # run time
  # allot of options are available,
  # you can create multiple "Run =" times
  # for now lets keep it simple
  # this will run once a day, every day at 21:10
  Run = daily at 21:10
}

You can make very complex schedules, but let’s start with something simple and easy. For more complex examples and options check the bareos manuel. Most important is the Run part, where we define the run frequency and the run time. Since /etc is rather small, it can be ran daily at an arbitrary time, but let’s do it outside work hours. Before you forget, all config files should be owned by the bareos user :

chown bareos.bareos /etc/bareos/bareos-dir.d/schedule/Nightly.conf

Next we need to let bareos know what we want to have backed up, by default one of the filesets is already useful for Linux : LinuxAll: /etc/bareos/bareos-dir.d/fileset/LinuxAll.conf

This fileset will backup all the root and all its subdirectory’s. This however is way to much and for our storage servers would simply not be possible. So let’s go ahead and create a new file :

nano /etc/bareos/bareos-dir.d/fileset/LinuxConfig.conf

with content :

FileSet {
  # name (required)
  Name = "LinuxConfig"
  # include directory
  Include {
    Options {
      # calculate a signature for all files
      # both MD5/SHA1 are available, this is definitly for long term storage
      # a good idea to activate, note that the hash ads a bit of storage overhead
      Signature = MD5
      
      # compress every file with compression software
      # best known are : LZ4/GZIP (see manual for the others)
      # LZ4 is super fast in both compression and decompression
      # I would activate this always.
      Compression = LZ4
      
      # if supported by the OS, the read time won't be adapted
      # this would generate a bunch of writes for no reason on the client machine.
      noatime = yes
    }
    # the directory we are including
    # note: no trailing slashes!
    File = /etc
  }
}

chown bareos.bareos /etc/bareos/bareos-dir.d/fileset/LinuxConfig.conf

Most important parts is the Include { File = /etc } this will make sure it includes /etc directory and all its subdirectory’s (this is also an option, but on by default)

notable options here are signature, a great feature, cause you don’t want to put back a corrupt backup … (it happens) While it has been proven that MD5 hashes can be broken it still is extremely difficult to do. But if you have that fear, you can use SHA1 which is a bit more CPU intense and takes up 4 more bytes per file but more difficult to break.

Another important feature is compression, I like LZ4 cause its fast, the CPU penalty is small compared to the storage you can gain. Note that this is run on the client. (file-daemon on client side)

Last option I included is noatime, while most storage systems already have atime off. Some system might still have it on, but I don’t see the point of knowing when a file has been read by a backup process.

Next part is combining all the property’s of a job that would execute the requirements we set. This is the default setup we start from when making jobs. Real jobs are called jobs, and can overwrite these defaults.

nano /etc/bareos/bareos-dir.d/jobdefs/BackupLinuxConfig.conf

JobDefs {
  # name (required)
  Name = "BackupLinuxConfig"
  
  # type can be backup/restore/verify
  Type = Backup
  
  # the default level bareos will try
  # can also be Full/Differential(since last full)/Incremental(since last incremental)
  Level = Incremental
  
  # the default client, to be overwritten by the job.conf
  Client = bareos-fd
  
  # what files to include/exclude
  FileSet = "LinuxConfig"
  
  # the schedule we just created
  Schedule = "Nightly"
  
  # where to store it
  Storage = File
  
  # the message reporting
  Messages = Standard
  
  # the pool where to store it
  Pool = Incremental
  
  # the higher the value priority the lower it will be dropped in the queue
  # so for important jobs priority=1 will run first
  Priority = 10
  
  # the bootstrap file keeps a "log" of all the backups, and gets rewritten every time a 
  # full backup is made, it can be used during recovery
  Write Bootstrap = "/var/lib/bareos/%c.bsr"
  
  # in case these value's get overwritten
  # define where would be a good pool to write 
  # note that full backup will be used atleast once because no full
  # backup will exist
  Full Backup Pool = Full
  Differential Backup Pool = Differential
  Incremental Backup Pool = Incremental
}

chown bareos.bareos /etc/bareos/bareos-dir.d/jobdefs/BackupLinuxConfig.conf

The client is here bareos-fd, which is a default configured client pointing to its own installation. The fileset and schedule we just created is now being referenced here. Since all backups start from a “full” backup, there needs to be a backup pool.

Now that we have a good “sketch” (jobdef) for our backup job, creating the real backup is easy.

Getting the final job to work is easy, just reference the jobdefs and overwrite the value’s you want changed.

nano /etc/bareos/bareos-dir.d/job/svennd-etc-backup.conf

with content :

Job {
  # required
  Name = "svennd-etc-backup"
  
  # the default settings
  JobDefs = "BackupLinuxConfig"
  
  # overwrite the client here
  Client = "svennd"
}

chown bareos.bareos /etc/bareos/bareos-dir.d/job/svennd-etc-backup.conf

And that’s it.

Obviously the job will start every day at 21:10, but as a test, its great if you can just start to run it now. Before you can start, be sure to check that all files you created are owned by bareos user if not :

chown bareos.bareos file.conf

Then we can go to the bconsole and reload the configuration, this is required. If you don’t do it only at the next restart of bareos-dir will the new configuration be picked up.

bconsole

and run reload :

*reload
reloaded

Once this worked and no errors where reported you can run the job we just made (you can also run it from the bareos-webui:

* run job=svennd-etc-backup
Using Catalog "MyCatalog"
Run Backup job
JobName:  svennd-etc-backup
Level:    Incremental
Client:   svennd
Format:   Native
FileSet:  LinuxConfig
Pool:     Incremental (From Job IncPool override)
Storage:  File (From Job resource)
When:     2017-03-10 15:48:01
Priority: 10
OK to run? (yes/mod/no): yes
Job queued. JobId=3

note : although I specify the jobname here, you can just run “run” and you will be prompted for the exact job you wish to run.

The log would report after a bit :

10-Mar 10:21 bareos-dir JobId 1: No prior Full backup Job record found.
10-Mar 10:21 bareos-dir JobId 1: No prior or suitable Full backup found in catalog. Doing FULL backup.
10-Mar 10:21 bareos-dir JobId 1: Start Backup JobId 1, Job=lungo-01-etc-backup.2017-03-10_10.21.46_43
10-Mar 10:21 bareos-dir JobId 1: Created new Volume "Full-0001" in catalog.
10-Mar 10:21 bareos-dir JobId 1: Using Device "FileStorage" to write.

As I said before the first backup would be a full one, and the next one, should start being incremental. If we run the job command a few times (you can use arrow up like in bash) and login in the web interface, under jobs.

And voila our first client is being backed up. Its should now be pretty easy to add more clients and get them backed up (just add more jobs under the job/ and overwrite the client). In the next post, we will look at how the backups are stored and how we can fine tune this setup.

By default the device used by Bareos for data archival/storage is FileStorage. Meaning data stored is stored in images on spinning disks in the file structure:

/var/lib/bareos/storage

While this might be a good place for you, its tricky with this location is in root … (full root and whatnot) You can change this in :

/etc/bareos/bareos-sd.d/device/FileStorage.conf

By changing the archive device :

Archive Device = /backup

Obviously you could make an extra device, and I’m only scratching the options you can do here, but since your root might not be terabytes in size, changing this is pretty crucial;

After changing this, you need to restart the bareos-fd :

systemctl restart bareos-fd

You can also remove the images that already are in /var/lib/bareos/storage and delete the volumes from the database :

bconsole
delete volume=Full-0001 yes

Don’t forget to physically remove them (rm -f)

https://www.svennd.be/concurrent-jobs-in-bareos-with-disk-storage/

https://www.svennd.be/bareos-fully-remove-client/

[root@ns7loc11 ~]# config show bareos-dir
bareos-dir=service
    TCPPort=9101
    access=green
    status=enabled
[root@ns7loc11 ~]# config show bareos-sd
bareos-sd=service
    TCPPort=9103
    access=green
    status=enabled
[root@ns7loc11 ~]# config show bareos-fd
bareos-fd=service
    TCPPort=9102
    access=green
    status=enabled

check logs at

/var/log/bareos/bareos.log

Please raise Issues on github