Skip to content
Snippets Groups Projects
Commit c4785ba5 authored by micah's avatar micah :speech_balloon:
Browse files

Moving to autotools standard layout

parent d58025ce
Branches
Tags
No related merge requests found
|\_
B A C K U P N I N J A /()/
`\|
a silent flower blossom death strike to lost data.
Backupninja allows you to coordinate system backup by dropping a few
simple configuration files into /etc/backup.d/. Most programs you
might use for making backups don't have their own configuration file
format. Backupninja provides a centralized way to configure and
coordinate many different backup utilities.
Features:
- easy to read ini style configuration files.
- you can drop in scripts to handle new types of backups.
- backup actions can be scheduled
- you can choose when status report emails are mailed to you
(always, on warning, on error, never).
- console-based wizard (ninjahelper) makes it easy to create
backup action configuration files.
- passwords are never sent via the command line to helper programs.
- works with Linux-Vservers (http://linux-vserver.org/)
Backup types:
- secure, remote, incremental filesytem backup (via rdiff-backup).
incremental data is compressed. permissions are retained even
with an unpriviledged backup user.
- backup of mysql databases (via mysqlhotcopy and mysqldump).
- backup of ldap databases (via slapcat and ldapsearch).
- basic system and hardware info
- encrypted remote backups (via duplicity).
- backup of subversion repositories.
The following options are available:
-h, --help This usage message
-d, --debug Run in debug mode, where all log messages are
output to the current shell.
-f, --conffile FILE Use FILE for the main configuration instead
of /etc/backupninja.conf
-t, --test Test run mode. This will test if the backup could run, without actually
preforming any backups. For example, it will attempt to authenticate
or test that ssh keys are set correctly.
-n, --now Perform actions now, instead of when they might be scheduled.
No output will be created unless also run with -d.
--run FILE Runs the specified action FILE (e.g. one of the /etc/backup.d/ files).
Also puts backupninja in debug mode.
CONFIGURATION FILES
===================
The general configuration file is /etc/backupninja.conf. In this file
you can set the log level and change the default directory locations.
You can force a different general configuration file with "backupninja
-f /path/to/conf".
To preform the actual backup, backupninja processes each configuration
file in /etc/backup.d according to the file's suffix:
.sh -- run this file as a shell script.
.rdiff -- filesystem backup (using rdiff-backup)
.dup -- filesystem backup (using duplicity)
.mysql -- backup mysql databases
.ldap -- backup ldap databases
.pgsql -- backup PostgreSQL databases
.sys -- general hardware, partition, and system reports.
.svn -- backup subversion repositories
.maildir -- incrementally backup maildirs (very specialized)
Support for additional configuration types can be added by dropping
bash scripts with the name of the suffix into /usr/share/backupninja.
The configuration files are processed in alphabetical order. However,
it is suggested that you name the config files in "sysvinit style."
For example:
00-disabled.ldap
10-runthisfirst.sh
20-runthisnext.mysql
90-runthislast.rdiff
Typically, you will put a '.rdiff' config file last, so that any
database dumps you make are included in the filesystem backup.
Configurations files with names beginning with 0 (zero) or ending with
.disabled (preferred method) are skipped.
Unless otherwise specified, the config file format is "ini style."
For example:
# this is a comment
[fishes]
fish = red
fish = blue
[fruit]
apple = yes
pear = no thanks \
i will not have a pear.
SCHEDULING
==========
By default, each configuration file is processed everyday at 01:00 (1
AM). This can be changed by specifying the 'when' option in a config
file.
For example:
when = sundays at 02:00
when = 30th at 22
when = 30 at 22:00
when = everyday at 01 <-- the default
when = Tuesday at 05:00
A configuration file will be processed at the time(s) specified by the
"when" option. If multiple "when" options are present, then they all
apply. If two configurations files are scheduled to run in the same
hour, then we fall back on the alphabetical ordering specified above.
If two configurations files are scheduled close to one another in
time, it is possible to have multiple copies of backupninja running if
the first instance is not finished before the next one starts.
Make sure that you put the "when" option before any sections in your
configuration file.
These values for 'when' are equivalent:
when = tuesday at 05:30
when = TUESDAYS at 05
These values for 'when' are invalid:
when = tuesday at 2am
when = tuesday at 2
when = tues at 02
REAL WORLD USAGE
================
Backupninja can be used to implement whatever backup strategy you
choose. It is intended, however, to be used like so:
(1) First, databases are safely copied or exported to /var/backups.
Typically, you cannot make a file backup of a database while it
is in use, hence the need to use special tools to make a safe copy
or export into /var/backups.
(2) Then, vital parts of the file system, including /var/backups, are
nightly pushed to a remote, off-site, hard disk (using
rdiff-backup). The local user is root, but the remote user is not
priviledged. Hopefully, the remote filesystem is encrypted.
There are many different backup strategies out there, including "pull
style", magnetic tape, rsync + hard links, etc. We believe that the
strategy outlined above is the way to go because: (1) hard disks are
very cheap these days, (2) pull style backups are no good, because then
the backup server must have root on the production server, and (3)
rdiff-backup is more space efficient and featureful than using rsync +
hard links.
SSH KEYS
========
In order for rdiff-backup to sync files over ssh unattended, you must
create ssh keys on the source server and copy the public key to the
remote user's authorized keys file. For example:
root@srchost# ssh-keygen -t dsa
root@srchost# ssh-copy-id -i /root/.ssh/id_dsa.pub backup@desthost
Now, you should be able to ssh from user 'root' on srchost to
user 'backup' on desthost without specifying a password.
Note: when prompted for a password by ssh-keygen, just leave it
blank by hitting return.
The included helper program "ninjahelper" will walk you through creating
an rdiff-backup configuration, and will set up the ssh keys for you.
INSTALLATION
============
Requirements:
apt-get install bash gawk
Recommended:
apt-get install rdiff-backup gzip hwinfo
Files:
/usr/sbin/backupninja -- main script
/etc/cron.d/backupninja -- runs main script nightly
/etc/logrotate.d/backupninja -- rotates backupninja.log
/etc/backup.d/ -- directory for configuration files
/etc/backupninja.conf -- general options
/usr/share/backupninja -- handler scripts which do the actual work
Installation:
There is no install script, but you just need to move files to the
correct locations. All files should be owned by root.
# tar xvzf backupninja.tar.gz
# cd backupninja
# mv backupninja /usr/sbin/backupninja
# mv ninjahelper /usr/sbin/ninjahelper
# mv etc/logrotate.d/backupninja /etc/logrotate.d/backupninja
# mv etc/cron.d/backupninja /etc/cron.d/backupninja
# mkdir /etc/backup.d/
# mv etc/backupninja.conf /etc/backupninja.conf
# mv handlers /usr/share/backupninja
VSERVERS
========
If you are using Linux-Vservers (http://linux-vserver.org/) there are some
special capabilities that different handlers have to make vserver backups easier.
Set the variable "vservers" to be "yes" in /etc/backupninja.conf and see the
example configuration files for each handler to configure the vserver specific
variables.
Additional vserver variables that can be configured in /etc/backupninja.conf. but
probably don't need to be changed:
VSERVERINFO (default: /usr/sbin/vserver-info)
VSERVER (default: /usr/sbin/vserver)
VROOTDIR (default: `$VSERVERINFO info SYSINFO |grep vserver-Rootdir | awk '{print $2}'`)
NINJAHELPER
===========
Ninjahelper is an additional script which will walk you through the process of
configuring backupninja. Ninjahelper has a menu driven curses based interface
(using dialog).
To add an additional 'wizard' to ninjahelper, follow these steps:
(1) to add a helper for the handler "blue", create the file
blue.helper in the directory where the handlers live.
(ie /usr/share/backupninja).
(2) next, you need to add your helper to the global HELPERS variable
and define the main function for your helper (the function name
is always <helper>_wizard). for example, blue.helper:
HELPERS="$HELPERS blue:description_of_this_helper"
blue_wizard() {
... do work here ...
}
(3) check the examples of the included helpers to see how they are
written. The dialog functions are defined in easydialog.sh.
version 0.9.1
fixed many bugs in rdiff helper
rdiff handler does not require 'label'
version 0.9 -- October 19 2005
*** IMPORTANT CHANGE, UPGRADE AT ONCE ***
fixed insecure temporary file creation
*****************************************
removed erroneous magic file marker in pgsql handler
fixed incorrect find positional
changed direct grep of /etc/passwd to getent passwd.
rdiff helper has much better information on failed ssh attempt
(patch from cmccallum@thecsl.org).
rdiff handler now supports remote source and local dest.
(patch from cmccallum@thecsl.org).
man pages are greatly improved.
version 0.8 -- September 15 2005
added pgsql (PostgreSQL) handler, with vservers support.
added vservers support to duplicity handler
Note: the configuration is a bit different than the rdiff
handler's one, but the default behavior is the same:
have a look at example.dup.
improved README
documented .disabled method.
corrected VROOTDIR default value.
added ninjahelper to the install instructions.
improved rdiff, dup and sys handlers' vservers support
prevent vserver-debiantools' $VROOTDIR/ARCHIVES directory
to be seen as a vserver
changes to sys handler
make use of configurable $VSERVERINFO instead of hard-coded vserver-info.
fixed dpkg existence test inside vserver.
fixed $nodpkg use.
changes to pgsql handler
now checks if the specified vserver is running.
now checks if $PGSQLDUMP/$PGSQLDUMPALL are available where needed.
now checks if "postgres" user exists on the host/vserver.
changes to ninjahelper
check_perms() does not die anymore on group/world readable
helper scripts (now consistent with the "helper scripts
must not be group or world writable!" error msg).
xedit action now tries $EDITOR, then /etc/alternatives/editor,
then nano, vim and vi, and aborts if none of these exists.
added helper for pgsql handler.
rdiff handler now does not require 'label'
changes to mysql and svn handlers' vservers support
these handlers now check if the source vserver is running
added 'ignores' for mysql handler. (thanks Daniel.Bonniot@inria.fr)
version 0.7 -- July 26 2005
added ninjahelper: a dialog based wizard for creating backupninja configs.
considerably improved and changed the log file output.
you can now disable actions with .disabled (this is new preferred method).
added makecd handler and example.makecd for backing up to cd/dvd (thanks stef).
fixed bug when specifying multiple 'when' options.
version 0.6 -- June 16 2005
ldap handler has new options: backup method to use (ldapsearch or
slapcat), restart, passwordfile and binddn. Default backup method
is set to ldapsearch as this is safer
******************************************************************
NOTE: to get the previous default behavior with the ldap handler,
you must set "method = slapcat". The new default is ldapsearch.
******************************************************************
implemented fix so that the main script will echo fatal errors rather
than being silent about them, this means an error message every hour
if there is a major configuration problem (not a handler problem)
added vserver support to main script and to the handlers: mysql, svn, sys, rdiff
changes to duplicity handler (thanks intrigeri!):
"keep = yes" now disables file cleaning on desthost
added "sign" option for backups signing
added "encryptkey" option to specify the gpg key to use
split config into [source], [gpg] and [dest] sections
added "nicelevel" option
added "testconnect" option
added "sshoptions" option (will be passed to ssh/scp)
added "bandwidthlimit" option
example.dup example config file for duplicity handler
added trac (http://trac.edgewall.com/) environment handler (thanks Charles Lepple!)
added configfile option to mysql handler
the default is /etc/mysql/debian.cnf. with this,
sqldump doesn't need dbusername. (hotcopy still does).
fixed bug in mysql handler which caused some passwords to not work.
(.my.cnf files now have double quotes around password)
can now pass options to hwinfo and sfdisk in sys handler.
version 0.5 -- April 12 2005
rdiff handler works when remote sshd has a banner
rdiff handler supports local dest
logfile is created if it doesn't exist
added "when = hourly"
added optional 'nicelevel' to rdiff handler
fixed bug where actions were not run in numeric order.
improved 'when' parsing.
version 0.4.4 -- March 18 2005
results of handlers are now read line by line.
changes to rdiff handler: added "options", and "keep" is
not necessarily days now (ie, it will pass straight through to
rdiff-backup if the keep has a unit on it, otherwise it adds the 'D').
added dup handler (still pretty beta)
added maildir handler (very specialized handler)
added --run option (runs the specified action file)
improved sys handler, now uses hwinfo
added subversion hotbackup handler, svn.
added PATH to cron.d file, which fixes file not found errors.
version 0.4.2 -- Jan 6 2005
fixed bug which caused a report email to be sent even if no actions were run.
fixed bug where multiple handler status messages were ignored
added status in the subject line of report emails
version 0.4.1 -- Jan 3 2005
added $usecolors and now more conservative about when colors are echoed.
fixed major bug, 'when' actually works now.
replaced debug function with debug, info, warning, error, fatal.
added --now option to force all actions to be performed now.
version 0.4 -- Dec 26 2004
added "when" option, so that all configs can specify when
they are to be run.
added reportsuccess and reportwarning config options
added .sys handler (hardware, packages, partitions).
version 0.3.4 -- Dec 8 2004
fixed numerical variable quoting compatibility with older wc
fixed stderr redirect bug
some comments in example.rdiff
version 0.3.3 -- Nov 10 2004
'*' (asterisk) works now in rdiff config files
works now with gawk as well as mawk
many bug fixes to ldap handler
paths to helper scripts can be configured in global config
does not require /usr/bin/stat
version 0.3.2 -- Sept 29 2004
handler scripts are no longer executable (to comply with debian policy)
handler error and warning messages are sent with the notify email
version 0.3.1 -- Sept 5 2004
added ldap handler
moved sh support to a handler script
add test mode, where no action is taken.
added --help
force only root can read /etc/backup.d/*
fixed missing equals symbols in example.rdiff
changed backupninja executable to be /usr/sbin rather than /usr/bin
version 0.3 -- Oct 20 2004
** IMPORTANT ** all config files are now ini style, not apache style
rewrote all scripts in bash for portability
added drop-in backupninja lib directory (/usr/share/backupninja)
all scripts are now run as root
version 0.2 -- Oct 14 2004
move distribution folder ./cron.d to ./etc/cron.d
fixed bug: removed printr of excludes (!)
added support for changing the user/group in rdiff sources.
added support for .mysql config files.
version 0.1 -- Oct 8 2004
initial release
File moved
File moved
File moved
File moved
File moved
File moved
File moved
File moved
File moved
File moved
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment