Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
backupninja
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Deploy
Releases
Model registry
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
nosmo
backupninja
Commits
d6d93450
Commit
d6d93450
authored
8 years ago
by
intrigeri
Browse files
Options
Downloads
Patches
Plain Diff
README.md: reformat to proper Markdown.
parent
5eb8ccb5
No related branches found
No related tags found
No related merge requests found
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
README.md
+127
-120
127 additions, 120 deletions
README.md
with
127 additions
and
120 deletions
README.md
+
127
−
120
View file @
d6d93450
...
...
@@ -6,12 +6,13 @@
a silent flower blossom death strike to lost data.
Backupninja allows you to coordinate system backup by dropping a few
simple configuration files into /etc/backup.d/. Most programs you
simple configuration files into
`
/etc/backup.d/
`
. Most programs you
might use for making backups don't have their own configuration file
format. Backupninja provides a centralized way to configure and
coordinate many different backup utilities.
Features:
# Features
- easy to read ini style configuration files.
- you can drop in scripts to handle new types of backups.
- backup actions can be scheduled
...
...
@@ -20,9 +21,10 @@ Features:
- console-based wizard (ninjahelper) makes it easy to create
backup action configuration files.
- passwords are never sent via the command line to helper programs.
-
works with Linux-Vservers (http://linux-vserver.org/)
- works with [Linux-Vservers](http://linux-vserver.org/)
# Backup types
Backup types:
- secure, remote, incremental filesytem backup (via rdiff-backup).
incremental data is compressed. permissions are retained even
with an unpriviledged backup user.
...
...
@@ -31,87 +33,92 @@ Backup types:
- encrypted remote backups (via duplicity).
- backup of subversion repositories.
# Options
The following options are available:
-h, --help This usage message
-d, --debug Run in debug mode, where all log messages are
- `
-h
`, `
--help
`: this usage message
- `
-d
`, `
--debug
`: run in debug mode, where all log messages are
output to the current shell.
-f, --conffile FILE
U
se FILE for the main configuration
instead
of /etc/backupninja.conf
-t, --test
T
est run mode. This will test if the backup could
run, without actually
preforming any backups. For example, it will
attempt to authenticate
or test that ssh keys are set correctly.
-n, --now
P
erform actions now, instead of when they might
be scheduled.
No output will be created unless also run with -d.
--run FILE
R
uns the specified action FILE (e.g. one of the
/etc/backup.d/ files).
Also puts backupninja in debug mode.
NINJAHELPER
- `
-f
`
,
`
--conffile FILE
`: u
se FILE for the main configuration
instead
of
`
/etc/backupninja.conf
`
- `
-t
`
,
`
--test
`: t
est run mode. This will test if the backup could
run, without actually
preforming any backups. For example, it will
attempt to authenticate
or test that ssh keys are set correctly.
- `
-n
`
,
`
--now
`: p
erform actions now, instead of when they might
be scheduled.
No output will be created unless also run with -d.
- `
--run FILE
`: r
uns the specified action
`
FILE
`
(e.g. one of the
`
/etc/backup.d/
` files).
Also puts backupninja in debug mode.
ninjahelper
===========
N
injahelper is an additional script which will walk you through the process of
n
injahelper is an additional script which will walk you through the process of
configuring backupninja. Ninjahelper has a menu driven curses based interface
(using dialog).
To add an additional 'wizard' to ninjahelper, follow these steps:
(1)
to add a helper for the handler "blue", create the file
blue.helper in the directory where the handlers live.
(ie /usr/share/backupninja).
1.
to add a helper for the handler "blue", create the file
`
blue.helper
`
in the directory where the handlers live.
(ie
`
/usr/share/backupninja
`
).
(2)
next, you need to add your helper to the global HELPERS variable
2.
next, you need to add your helper to the global
`
HELPERS
`
variable
and define the main function for your helper (the function name
is always
<helper>
_wizard). for example, blue.helper:
is always `
<helper>
_wizard
`). for example, `
blue.helper
`:
HELPERS="$HELPERS blue:description_of_this_helper"
blue_wizard() {
... do work here ...
}
(3)
look at the existing helpers to see how they are written. Try to re-use
functions, such as the dialog functions that are defined in easydialog.sh,
or the vserver functions defined in lib/vserver.
3.
look at the existing helpers to see how they are written. Try to re-use
functions, such as the dialog functions that are defined in
`
easydialog.sh
`
,
or the vserver functions defined in
`
lib/vserver
`
.
(4)
test, re-test, and test again. Try to break the helper by going backwards,
4.
test, re-test, and test again. Try to break the helper by going backwards,
try to think like someone who has no idea how to configure your handler
would think, try to make your helper as simple as possible. Walk like a cat,
become your shadow, don't let your senses betray you.
C
ONFIGURATION FILES
C
onfiguration files
===================
The general configuration file is /etc/backupninja.conf. In this file
The general configuration file is
`
/etc/backupninja.conf
`
. In this file
you can set the log level and change the default directory locations.
You can force a different general configuration file with
"
backupninja
-f /path/to/conf
"
.
You can force a different general configuration file with
`
backupninja
-f /path/to/conf
`
.
To preform the actual backup, backupninja processes each configuration
file in /etc/backup.d according to the file's suffix:
file in
`
/etc/backup.d
`
according to the file's suffix:
.sh --
run this file as a shell script.
.rdiff
--
filesystem backup (using rdiff-backup)
.dup
--
filesystem backup (using duplicity)
.mysql
--
backup mysql databases
.pgsql
--
backup PostgreSQL databases
.sys
--
general hardware, partition, and system reports.
.svn
--
backup subversion repositories
.maildir
--
incrementally backup maildirs (very specialized)
- `
.sh
`:
run this file as a shell script.
- `
.rdiff
`:
filesystem backup (using rdiff-backup)
- `
.dup
`:
filesystem backup (using duplicity)
- `
.mysql
`:
backup mysql databases
- `
.pgsql
`:
backup PostgreSQL databases
- `
.sys
`:
general hardware, partition, and system reports.
- `
.svn
`:
backup subversion repositories
- `
.maildir
`:
incrementally backup maildirs (very specialized)
Support for additional configuration types can be added by dropping
bash scripts with the name of the suffix into /usr/share/backupninja.
bash scripts with the name of the suffix into
`
/usr/share/backupninja
`
.
The configuration files are processed in alphabetical order. However,
it is suggested that you name the config files in "sysvinit style."
For example:
00-disabled.pgsql
10-runthisfirst.sh
20-runthisnext.mysql
90-runthislast.rdiff
Typically, you will put a
'
.rdiff
'
config file last, so that any
Typically, you will put a
`
.rdiff
`
config file last, so that any
database dumps you make are included in the filesystem backup.
Configurations files with names beginning with 0 (zero) or ending with
.disabled (preferred method) are skipped.
`
.disabled
`
(preferred method) are skipped.
Unless otherwise specified, the config file format is "ini style."
...
...
@@ -128,8 +135,7 @@ For example:
pear = no thanks \
i will not have a pear.
SCHEDULING
Scheduling
==========
By default, each configuration file is processed everyday at 01:00 (1
...
...
@@ -145,54 +151,56 @@ For example:
when = Tuesday at 05:00
A configuration file will be processed at the time(s) specified by the
"
when
"
option. If multiple
"
when
"
options are present, then they all
`
when
`
option. If multiple
`
when
`
options are present, then they all
apply. If two configurations files are scheduled to run in the same
hour, then we fall back on the alphabetical ordering specified above.
If two configurations files are scheduled close to one another in
time, it is possible to have multiple copies of backupninja running if
the first instance is not finished before the next one starts.
Make sure that you put the
"
when
"
option before any sections in your
Make sure that you put the
`
when
`
option before any sections in your
configuration file.
These values for
'
when
'
are equivalent:
These values for
`
when
`
are equivalent:
when = tuesday at 05:30
when = TUESDAYS at 05
These values for
'
when
'
are invalid:
These values for
`
when
`
are invalid:
when = tuesday at 2am
when = tuesday at 2
when = tues at 02
R
EAL WORLD USAGE
R
eal world usage
================
Backupninja can be used to implement whatever backup strategy you
choose. It is intended, however, to be used like so:
(1)
First, databases are safely copied or exported to /var/backups.
1.
First, databases are safely copied or exported to
`
/var/backups
`
.
Typically, you cannot make a file backup of a database while it
is in use, hence the need to use special tools to make a safe copy
or export into /var/backups.
or export into
`
/var/backups
`
.
(2)
Then, vital parts of the file system, including /var/backups, are
2.
Then, vital parts of the file system, including
`
/var/backups
`
, are
nightly pushed to a remote, off-site, hard disk (using
rdiff-backup). The local user is root, but the remote user is not
priviledged. Hopefully, the remote filesystem is encrypted.
There are many different backup strategies out there, including "pull
style", magnetic tape, rsync + hard links, etc. We believe that the
strategy outlined above is the way to go because: (1) hard disks are
very cheap these days, (2) pull style backups are no good, because then
the backup server must have root on the production server, and (3)
rdiff-backup is more space efficient and featureful than using rsync +
hard links.
strategy outlined above is the way to go because:
1.
hard disks are very cheap these days;
2.
pull style backups are no good, because then the backup server must
have root on the production server;
3.
rdiff-backup is more space efficient and featureful than using
rsync + hard links.
SSH KEYS
SSH keys
========
In order for rdiff-backup to sync files over ssh unattended, you must
...
...
@@ -202,13 +210,13 @@ remote user's authorized keys file. For example:
root@srchost# ssh-keygen -t rsa -b 4096
root@srchost# ssh-copy-id -i /root/.ssh/id_rsa.pub backup@desthost
Now, you should be able to ssh from user
'
root
'
on srchost to
user
'
backup
'
on desthost without specifying a password.
Now, you should be able to ssh from user
`
root
`
on
`
srchost
`
to
user
`
backup
`
on
`
desthost
`
without specifying a password.
Note: when prompted for a password by ssh-keygen, just leave it
Note: when prompted for a password by
`
ssh-keygen
`
, just leave it
blank by hitting return.
The included helper program
"
ninjahelper
"
will walk you through creating
The included helper program
`
ninjahelper
`
will walk you through creating
an rdiff-backup configuration, and will set up the ssh keys for you.
...
...
@@ -218,44 +226,43 @@ Amazon Simple Storage Service (S3)
Duplicity can store backups on Amazon S3 buckets, taking care of encryption.
Since it performs incremental backups it minimizes the number of request per
operation therefore reducing the costs. The boto Python interface to Amazon
Web Services is needed to use duplicity with S3 (Debian package: python-boto).
Web Services is needed to use duplicity with S3 (Debian package:
`
python-boto
`
).
V
SERVERS
V
servers
========
If you are using Linux-Vservers
(http://linux-vserver.org/) there are some
If you are using
[
Linux-Vservers
]
(
http://linux-vserver.org/
)
there are some
special capabilities that different handlers have to make vserver
backups easier.
Set the variable
"
vservers
"
to be
"
yes
"
in /etc/backupninja.conf and see the
Set the variable
`
vservers
`
to be
`
yes
`
in
`
/etc/backupninja.conf
`
and see the
example configuration files for each handler to configure the vserver specific
variables.
Additional vserver variables that can be configured in /etc/backupninja.conf,
Additional vserver variables that can be configured in
`
/etc/backupninja.conf
`
,
but they probably don't need to be changed:
VSERVERINFO (default: /usr/sbin/vserver-info)
VSERVER (default: /usr/sbin/vserver)
VROOTDIR (default:
`$VSERVERINFO info SYSINFO |grep vserver-Rootdir | awk '{print $2}'`
)
-
`VSERVERINFO`
(default:
`/usr/sbin/vserver-info`
)
-
`VSERVER`
(default:
`/usr/sbin/vserver`
)
-
`VROOTDIR`
(default:
`$VSERVERINFO info SYSINFO |grep vserver-Rootdir | awk '{print $2}'`
)
.sh
CONFIGURATION FILES
.sh
configuration files
=======================
Shell jobs may use the following features:
*
logging and control flow functions:
halt, fatal, error, warning, info, debug, passthru.
`
halt
`
,
`
fatal
`
,
`
error
`
,
`
warning
`
,
`
info
`
,
`
debug
`
,
`
passthru
`
.
All such functions take a list of strings a parameters.
Those strings are passed to whatever logging mechanism is enabled,
and colored if relevant.
*
Using
"
exit N
"
is useless, and has unspecified consequences.
*
Using
`
exit N
`
is useless, and has unspecified consequences.
Just don't do it.
*
when=TIME works as documented above; at may also be written
"
when = TIME
"
.
*
`
when=TIME
`
works as documented above; at may also be written
`
when = TIME
`
.
*
The $BACKUPNINJA_DEBUG environment variable is set when
backupninja is invoked with the -d option.
*
The
`
$BACKUPNINJA_DEBUG
`
environment variable is set when
backupninja is invoked with the
`
-d
`
option.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment