Show pagesourceOld revisionsBacklinksBack to top Share via Share via... Twitter LinkedIn Facebook Pinterest Telegram WhatsApp Yammer RedditRecent ChangesSend via e-MailPrintPermalink × Table of Contents Bareos (backup) Maintainer Installation First access Official manual Create Clients and Jobs Change default storage location Concurrent Jobs with disk storage Fully remove a client DB command log Authentication Bugs Bareos (backup) Available NS7 Maintainer Stephane de Labrusse at stephdl@de-labrusse.fr Is this Module helpful to you ? Please consider donating Installation * First you need to install my repository, see how to do it * You need to install the bareos repository wget http://download.bareos.org/bareos/release/latest/CentOS_7/bareos.repo -O /etc/yum.repos.d/bareos.repo * then you can install the rpm yum install nethserver-bareos --enablerepo=stephdl First access It's possible to access bareos the first time pointing the browser to http://your_ip/bareos-webui, restricted to your local network, access from WAN are refused. you have a link at the Applications menu of Nethgui and cockpit The admin user of nethserver is the user that can authenticate in the bareos UI, think to install either openldap (nethserver-directory) or Samba AD (nethserver-dc) to create it, after this you have to set a password inside the server-manager of NethServer. Obviously you can add more users for the UI of bareos, you simply have to create a bareos group inside the server-manager of NethServer, all members of this group will be granted to use the web UI of bareos. You have to use the short format name (user) to login to bareos, it seems the long format is not accepted(user@domain.org) Storage By default it is ``/var/lib/bareos/storage/`` that is used to store all your backups, either mount a storage to this path, or check the link at the bottom to use another path Official manual The manual is intended to be the extended resource for all standard questions on using Bareos at https://docs.bareos.org/ Set up a backup client Set up a Relax-and-Recover with Bareos Bareos uses rh-postgresql12, we need to enable scl before to use a bareos script and set the ENV custom port of postgresql (TCP 55434) for example - upgrade the database structure su - postgres -c "export PGPORT=55434;scl enable rh-postgresql12 /usr/lib/bareos/scripts/update_bareos_tables" - backup database to /var/lib/bareos export PGPORT=55434;scl enable rh-postgresql12 -- /usr/lib/bareos/scripts/make_catalog_backup.pl MyCatalog Create Clients and Jobs An excellent documentation comes from https://www.svennd.be/bareos-articles/ I use it as a source for all these examples Backup a NethServer You can backup a NethServer or any other client(windows, Linux), for a client linux install `bareos-fd` and open the TCP port '9102' in the firewall, for NethServer you simply need to install nethserver-bareos-client from my repository. On the client nethserver * First you need to install my repository, see how to do it * You need to install the bareos repository wget http://download.bareos.org/bareos/release/latest/CentOS_7/bareos.repo -O /etc/yum.repos.d/bareos.repo * then you can install the rpm yum install nethserver-bareos-client --enablerepo=stephdl Depending how works your network and specifically the domain name resolution, you might have in the client to fill the domain name of the backup server with its IP in the /etc/hosts, use the UI or make a custom-template. you should give the host name (host) and the Fully qualified domain name (host.domain.org) 192.168.56.14 ns7loc13.nethservertest.org ns7loc13 On the backup server once installed, go to the console on the backup server. Since the web interface can’t be used (yet?) lets fire up bconsole. Adding the client is done like this (the client here is named 'swennd'): You need to think about good name for your client, it could be quickly a mess if you do not use relevant name. bconsole configure add client name=svennd address=192.168.0.2 password=SOME_PASSWORD This should result in something like : *configure add client name=svennd address=8.8.8.8 password=alfa123 Exported resource file "/etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf": Director { Name = bareos-dir Password = "[md5]f7180d44d3cad452f611e2dfdfd3b007" } Created resource config file "/etc/bareos/bareos-dir.d/client/svennd.conf": Client { Name = svennd Address = 8.8.8.8 Password = alfa123 } After that the configuration for the bareos director has been made, the only configuration that still is missing is on the client, this has been made by the bconsole, and can be found in /etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf we need to copy that to /etc/bareos/bareos-fd.d/director/ on the client machine. This can be copied using scp to the client. scp /etc/bareos/bareos-dir-export/client/svennd/bareos-fd.d/director/bareos-dir.conf root@8.8.8.8:/etc/bareos/bareos-fd.d/director/ && ssh root@8.8.8.8 "systemctl restart bareos-fd" Then on the client you have to restart the file daemon after any changes: systemctl restart bareos-fd And that’s it, now we are ready to start creating a backup job. you can valid that the server/client can communicate bconsole then type: *status client The defined Client resources are: 1: bareos-fd 2: svennd Select Client (File daemon) resource (1-5): 2 Connecting to Client ns7loc10 at 192.168.56.11:9102 Probing client protocol... (result will be saved until config reload) Handshake: Immediate TLS, Encryption: PSK-AES256-CBC-SHA svennd-fd Version: 19.2.6 (11 February 2020) Linux-3.10.0-1062.9.1.el7.x86_64 redhat CentOS Linux release 7.7.1908 (Core) Daemon started 05-Apr-20 16:08. Jobs: run=0 running=0, bareos.org build binary Sizeof: boffset_t=8 size_t=8 debug=0 trace=0 bwlimit=0kB/s Running Jobs: bareos-dir (director) connected at: 05-Apr-20 20:03 No Jobs running. ==== Terminated Jobs: JobId Level Files Bytes Status Finished Name ====================================================================== ==== Create backup job In the last posts, we setup bareos and connected a client to the bareos server, there are however no backup jobs auto-magically added to the bareos server so that’s our job. At first creating a job seems difficult and complex, but most of the options only become interesting in specific applications and a simple basic job can be configured pretty easy. That’s what I am going to do now, create a backup job that will incremental backup the /etc directory of my Linux client. Job requirements First we need to understand the target of the job, this step is important, if you don’t do it you might end up with something completely different due to the options possible with bareos. * run a backup of /etc daily during non-business hours, but before the nightly update. * To save some space, I would take incremental updates. That means I always check what changes have occurred since the last incremental update. This makes that if nothing changes 0 extra bytes are used. * Disk storage, we have a tape library, but lets start with normal disk storage Schedule Since we have a pretty well defined time of day to run the backup job, let’s start with first creating the specific schedule. In bareos all the configurations are nicely filtered in config files. In the directory `/etc/bareos` since we are configuring jobs that will be “directed” by the director, we need to go in the director configuration : /etc/bareos/bareos-dir.d/ there we can find the schedule directory. There might already be some examples there. But let’s create a new schedule : nano /etc/bareos/bareos-dir.d/schedule/Nightly.conf with the content : Schedule { # name (required) Name = "Nightly" # run time # allot of options are available, # you can create multiple "Run =" times # for now lets keep it simple # this will run once a day, every day at 21:10 Run = daily at 21:10 } You can make very complex schedules, but let’s start with something simple and easy. For more complex examples and options check the bareos manuel. Most important is the Run part, where we define the run frequency and the run time. Since /etc is rather small, it can be ran daily at an arbitrary time, but let’s do it outside work hours. Before you forget, all config files should be owned by the bareos user : chown bareos.bareos /etc/bareos/bareos-dir.d/schedule/Nightly.conf Fileset Next we need to let bareos know what we want to have backed up, by default one of the filesets is already useful for Linux : LinuxAll: /etc/bareos/bareos-dir.d/fileset/LinuxAll.conf This fileset will backup all the root and all its subdirectory’s. This however is way to much and for our storage servers would simply not be possible. So let’s go ahead and create a new file : nano /etc/bareos/bareos-dir.d/fileset/LinuxConfig.conf with content : FileSet { # name (required) Name = "LinuxConfig" # include directory Include { Options { # calculate a signature for all files # both MD5/SHA1 are available, this is definitly for long term storage # a good idea to activate, note that the hash ads a bit of storage overhead Signature = MD5 # compress every file with compression software # best known are : LZ4/GZIP (see manual for the others) # LZ4 is super fast in both compression and decompression # I would activate this always. Compression = LZ4 # if supported by the OS, the read time won't be adapted # this would generate a bunch of writes for no reason on the client machine. noatime = yes } # the directory we are including # note: no trailing slashes! File = /etc } } chown bareos.bareos /etc/bareos/bareos-dir.d/fileset/LinuxConfig.conf Most important parts is the Include { File = /etc } this will make sure it includes /etc directory and all its subdirectory’s (this is also an option, but on by default) notable options here are signature, a great feature, cause you don’t want to put back a corrupt backup … (it happens) While it has been proven that MD5 hashes can be broken it still is extremely difficult to do. But if you have that fear, you can use SHA1 which is a bit more CPU intense and takes up 4 more bytes per file but more difficult to break. Another important feature is compression, I like LZ4 cause its fast, the CPU penalty is small compared to the storage you can gain. Note that this is run on the client. (file-daemon on client side) Last option I included is noatime, while most storage systems already have atime off. Some system might still have it on, but I don’t see the point of knowing when a file has been read by a backup process. Jobdefs Next part is combining all the property’s of a job that would execute the requirements we set. This is the default setup we start from when making jobs. Real jobs are called jobs, and can overwrite these defaults. nano /etc/bareos/bareos-dir.d/jobdefs/BackupLinuxConfig.conf JobDefs { # name (required) Name = "BackupLinuxConfig" # type can be backup/restore/verify Type = Backup # the default level bareos will try # can also be Full/Differential(since last full)/Incremental(since last incremental) Level = Incremental # the default client, to be overwritten by the job.conf Client = bareos-fd # what files to include/exclude FileSet = "LinuxConfig" # the schedule we just created Schedule = "Nightly" # where to store it Storage = File # the message reporting Messages = Standard # the pool where to store it Pool = Incremental # the higher the value priority the lower it will be dropped in the queue # so for important jobs priority=1 will run first Priority = 10 # the bootstrap file keeps a "log" of all the backups, and gets rewritten every time a # full backup is made, it can be used during recovery Write Bootstrap = "/var/lib/bareos/%c.bsr" # in case these value's get overwritten # define where would be a good pool to write # note that full backup will be used atleast once because no full # backup will exist Full Backup Pool = Full Differential Backup Pool = Differential Incremental Backup Pool = Incremental } chown bareos.bareos /etc/bareos/bareos-dir.d/jobdefs/BackupLinuxConfig.conf The client is here bareos-fd, which is a default configured client pointing to its own installation. The fileset and schedule we just created is now being referenced here. Since all backups start from a “full” backup, there needs to be a backup pool. Now that we have a good “sketch” (jobdef) for our backup job, creating the real backup is easy. Job Getting the final job to work is easy, just reference the jobdefs and overwrite the value’s you want changed. nano /etc/bareos/bareos-dir.d/job/svennd-etc-backup.conf with content : Job { # required Name = "svennd-etc-backup" # the default settings JobDefs = "BackupLinuxConfig" # overwrite the client here Client = "svennd" } chown bareos.bareos /etc/bareos/bareos-dir.d/job/svennd-etc-backup.conf And that’s it. Running the job Obviously the job will start every day at 21:10, but as a test, its great if you can just start to run it now. Before you can start, be sure to check that all files you created are owned by bareos user if not : chown bareos.bareos file.conf Then we can go to the bconsole and reload the configuration, this is required. If you don’t do it only at the next restart of bareos-dir will the new configuration be picked up. bconsole and run reload : *reload reloaded Once this worked and no errors where reported you can run the job we just made (you can also run it from the bareos-webui: * run job=svennd-etc-backup Using Catalog "MyCatalog" Run Backup job JobName: svennd-etc-backup Level: Incremental Client: svennd Format: Native FileSet: LinuxConfig Pool: Incremental (From Job IncPool override) Storage: File (From Job resource) When: 2017-03-10 15:48:01 Priority: 10 OK to run? (yes/mod/no): yes Job queued. JobId=3 note : although I specify the jobname here, you can just run “run” and you will be prompted for the exact job you wish to run. The log would report after a bit : 10-Mar 10:21 bareos-dir JobId 1: No prior Full backup Job record found. 10-Mar 10:21 bareos-dir JobId 1: No prior or suitable Full backup found in catalog. Doing FULL backup. 10-Mar 10:21 bareos-dir JobId 1: Start Backup JobId 1, Job=lungo-01-etc-backup.2017-03-10_10.21.46_43 10-Mar 10:21 bareos-dir JobId 1: Created new Volume "Full-0001" in catalog. 10-Mar 10:21 bareos-dir JobId 1: Using Device "FileStorage" to write. As I said before the first backup would be a full one, and the next one, should start being incremental. If we run the job command a few times (you can use arrow up like in bash) and login in the web interface, under jobs. And voila our first client is being backed up. Its should now be pretty easy to add more clients and get them backed up (just add more jobs under the job/ and overwrite the client). In the next post, we will look at how the backups are stored and how we can fine tune this setup. Change default storage location By default the device used by Bareos for data archival/storage is FileStorage. Meaning data stored is stored in images on spinning disks in the file structure: /var/lib/bareos/storage While this might be a good place for you, its tricky with this location is in root … (full root and whatnot) You can change this in : /etc/bareos/bareos-sd.d/device/FileStorage.conf By changing the archive device : Archive Device = /backup Obviously you could make an extra device, and I’m only scratching the options you can do here, but since your root might not be terabytes in size, changing this is pretty crucial; After changing this, you need to restart the bareos-fd : systemctl restart bareos-fd You can also remove the images that already are in /var/lib/bareos/storage and delete the volumes from the database : bconsole delete volume=Full-0001 yes Don’t forget to physically remove them (rm -f) Concurrent Jobs with disk storage https://www.svennd.be/concurrent-jobs-in-bareos-with-disk-storage/ Fully remove a client https://www.svennd.be/bareos-fully-remove-client/ DB command [root@ns7loc11 ~]# config show bareos-dir bareos-dir=service TCPPort=9101 access=green status=enabled [root@ns7loc11 ~]# config show bareos-sd bareos-sd=service TCPPort=9103 access=green status=enabled [root@ns7loc11 ~]# config show bareos-fd bareos-fd=service TCPPort=9102 access=green status=enabled log check logs at /var/log/bareos/bareos.log Authentication Bugs Please raise Issues on github module stephdl bareos.txt Last modified: 2020/12/21 18:26by Stephane de Labrusse