path: root/README.md

# Ansible Role: BorgBackup

An Ansible role that installs and configures a systemd service for BorgBackup on
a client and (delegated) server.
This allows you to very easily add backups to your hosts.

The documentation assumes basic knowledge about BorgBackup. You can get up to
speed with Borg by reading their excellent documentation on
<https://borgbackup.readthedocs.io>.

## How it works

```mermaid
sequenceDiagram
participant Ansible
participant borg-client
participant borg-server

Ansible->>borg-server: Install dependencies
Ansible->>borg-client: Install dependencies
Ansible->>borg-server: Setup Repository
Ansible->>borg-client: Setup Repository
Ansible->>borg-client: Setup backup job
borg-client->>Ansible: Store decryption keys
borg-client-->borg-server: Do Backup
```

## Installation

via ansible-galaxy

```sh
ansible-galaxy role install kliwniloc.borgbackup
```

```yaml
# requirements.yml
- src: kliwniloc.borgbackup
```

via git

```sh
ansible-galaxy role install git+https://github.com/kliwniloc/ansible-role-borgbackup.git,master
```

```yaml
# requirements.yml
- src: https://github.com/kliwniloc/ansible-role-borgbackup
  version: master
  name: kliwniloc.borgbackup
```

## Role Variables

Some configuration options were left out of this high level overview. For the
full set of configuration options, see: [`defaults/main.yml`](defaults/main.yml)

### Backup User Configuration

By default, the role runs all client-side operations as `root`. To use a
non-root user, set `borg_client_user`:

```yaml
borg_client_user: backup
```

The user must exist before running the role; it will not be created
automatically. Ensure the user can read all directories specified in
`borg_included_dirs`, or backups will fail at runtime.

### Server Configuration

You need to specify your Borg hostname (as defined in your Ansible inventory)
as well as the public SSH key from that Borg SSH server.
You can get this key by scanning the Borg server host like this:
`ssh-keyscan -t rsa borg.example.org` and removing the hostname, so the output
looks like this:
`ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA...`.

If the Ansible hostname is not reachable from the client host, you must set
`borg_server_host_url` to a host that is reachable from the Borg clients. It
will default to `borg_server_host`.

> - `borg_server_host` is used by Ansible to delegate tasks to the Borg server
> - `borg_server_host_url` is used by the client to connect to the Borg server

```yaml
borg_server_host: "" # Required
borg_server_host_ssh_key: "" # Required
borg_server_host_url: { { borg_server_host } } # Optional
```

By default, the role uses the `borg` user on the backup server and creates it
automatically. To use a different user, set `borg_server_user`:

```yaml
borg_server_user: backupserver
borg_server_user_home: /var/backups
```

The `borg_server_user_home` specifies the home directory of the borg user on
the server. All client repositories will be saved in this directory.

To disable automatic user creation (when the user is managed externally), set:

```yaml
borg_server_user_create: false
```

When disabled, the user must exist before running the role.

For the backup itself, there are a few Borg parameters you can configure.
Not all options that Borg offers are available in the Ansible role yet. For more
configuration options, see <https://borgbackup.readthedocs.io>.

`borg_repo_name` configures the name of the repository as it is created on the
Borg server in the `borg_server_user_home` directory. By default, we use the
inventory hostname, which is fine if you don't use multiple Borg repositories on
the same client and server hosts.
The individual backup names can be customized in the `borg_backup_name_format`
variable. You can use placeholders such as `{hostname}` for the backups, see:
<https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-placeholders>.

`borg_mode_append_only` restricts the deletion of the backups from the client.
This can increase security but comes at the cost of not being able to clean up
old backups from the client.

`borg_storage_quota` limits the storage space used by the repository on the
Borg server. Format: `N` (bytes), `NK` (kilobytes), `NM` (megabytes), `NG`
(gigabytes), `NT` (terabytes). Empty string means no quota.

```yaml
borg_repo_name: "{{ inventory_hostname }}"
borg_backup_name_format: "{hostname}-{now:%Y-%m-%dT%H:%M:%S}"
borg_mode_append_only: false
borg_storage_quota: "" # e.g., "100G" for 100 gigabytes
```

We use zstd compression by default, but you can change it to any of the
supported modes see:
<https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-compression>.

You can configure a passphrase for the encryption keys in `borg_passphrase`of
your Borg backups, but it is not used by default.

> [!WARNING]
> **Enable a passphrase for true end-to-end encryption.**
> If you want true end-to-end encryption, you should configure a passphrase
> Read more: [BorgBackup Security](https://borgbackup.readthedocs.io/en/stable/internals/security.html#offline-key-security)

<!-- TODO: Include keyfile backup mode -->

`borg_included_dirs` and `borg_excluded_dirs` finally picks what directories
will be backed up by Borg.

```yaml
borg_compression: zstd # -C argument
borg_create_additional_arguments: "" # Additional borg create options
borg_passphrase: "" # Env variable
borg_included_dirs: [] # Positional arguments
borg_excluded_dirs: [] # --exclude arguments
```

`borg_create_additional_arguments` lets you pass additional options to `borg
create`. These arguments are appended after all role-defined options, so
user-specified options can override defaults. For example:

```yaml
# Add verbose output
borg_create_additional_arguments: "--stats --list --filter=AME"

# Stay in one filesystem and exclude caches
borg_create_additional_arguments: "--one-file-system --exclude-caches"
```

To decrypt your backups without the client, we store the decryption keys in a
YAML file in your Ansible repository. You require the decryption keys as well as
access to the repository files on the Borg server to access the backups.
If you wish to add another layer of security for these decryption keys, consider
using git encryption tools like <https://github.com/AGWA/git-crypt> or storing
the keys in a secure location outside of git.

```yaml
borg_decryption_keys_yaml_path: "{{ inventory_dir }}/decryption_keys.yml"
```

Set to an empty string to disable exporting decryption keys entirely:

```yaml
borg_decryption_keys_yaml_path: ""
```

The scheduling of the backups is done via systemd. For this purpose a systemd
service and a corresponding timer are created.
Additionally, we create backup scripts for use in the systemd service and for
the ability to easily trigger manual backups.

To support multiple backups (with different schedules), we have the
`borg_backup_argument` which defaults to `{{ borg_server_host_url }}` and should
be unique per backup target.
The names of the systemd service and timer are:
`{{ borg_backup_timer_name }}{{ borg_backup_argument }}.service` and
`{{ borg_backup_timer_name }}{{ borg_backup_argument }}.timer`.

For the backup scripts we add `{{ borg_backup_script_location }}` for creating a
backup on all specified targets and
`{{ borg_backup_script_location }}{{ borg_backup_argument }}` for backing up to
each target.

To configure the backup schedule, we offer `borg_systemd_oncalendar` and
`borg_systemd_accuracysec`, which map to the corresponding systemd options,
see man-page systemd.timer(5).

```yaml
borg_backup_script_location: /usr/local/bin/run_borg_backup
borg_backup_timer_name: borg_backup
borg_backup_service_name: borg_backup
borg_backup_argument: "{{ borg_server_host_url }}"
borg_systemd_oncalendar: "*-*-* 02:00:00"
borg_systemd_accuracysec: 60min
```

## Example Playbooks

### Simple Playbook

```yaml
- name: >-
    Add Borg backup to the borg-client and save the
    decryption keys alongside the playbook
  hosts: borg-client
  roles:
    - role: kliwniloc.borgbackup
      borg_server_host: borg-server
      borg_server_host_ssh_key: ssh-rsa AAAAAAAA...
      borg_decryption_keys_yaml_path: "{{ playbook_dir }}/decryption_keys.yml"
      borg_included_dirs:
        - /home
        - /opt/project
```

### Multiple Backup Servers

```yaml
- name: >-
    Add Borg backup with multiple borg servers to the borg-client and
    save the decryption keys alongside the playbook
  hosts: borg-client
  vars:
    borg_included_dirs:
      - /home
      - /opt/project
  roles:
    - role: kliwniloc.borgbackup
      borg_server_host: borg-server1
      borg_server_host_ssh_key: ssh-rsa AAAAAAAA...
    - role: kliwniloc.borgbackup
      borg_server_host: borg-server2
      borg_server_host_ssh_key: ssh-rsa BBBBBBBB...
```

### Multiple Repositories on Single Server, Single Client (Multi-Instance)

> [!NOTE]
> **This is considered a beta feature**
> This feature is still being developed and has some quirks around it.
> We plan to release this properly on the next major version with some breaking
> changes to make this less of a footgun. Until then, read this section
> carefully.

When a single client host needs to back up different data sets to the same Borg
server (e.g., system configs and user data), you can run the role multiple times
with different `borg_repo_name` values.

**Important:** Each invocation must have a unique `borg_backup_argument` to create
separate systemd timer units.

<!-- TODO: (next major version): Change default borg_backup_argument to include
     borg_repo_name for uniqueness:
     "{{ borg_server_host_url }}-{{ borg_repo_name }}".
     This will be a breaking change requiring migration of existing systemd
     units. -->

```yaml
- name: Configure multi-instance backup (same host, different repos)
  hosts: borg-client
  vars:
    borg_server_host: borg-server
    borg_server_host_ssh_key: ssh-rsa AAAAAAAA...
    borg_decryption_keys_yaml_path: "{{ playbook_dir }}/decryption_keys.yml"

  roles:
    - role: kliwniloc.borgbackup
      vars:
        borg_repo_name: configs
        borg_backup_argument: configs
        borg_compression: zstd
        borg_included_dirs:
          - /etc
        borg_systemd_oncalendar: "*-*-* 02:00:00"

    - role: kliwniloc.borgbackup
      vars:
        borg_repo_name: home-data
        borg_backup_argument: home-data
        borg_compression: lz4
        borg_included_dirs:
          - /home
        borg_excluded_dirs:
          - /home/*/.cache
        borg_systemd_oncalendar: "*-*-* 04:00:00"
```

This creates:

- Two repositories on the Borg server: `/opt/borg/configs` and `/opt/borg/home-data`
- Two systemd timers: `borg_backup@configs.timer` and `borg_backup@home-data.timer`
- A single SSH keypair shared between both repositories
- A single `authorized_keys` entry with both repositories in `--restrict-to-repository`

## Important Behaviors and Limitations

### Consistent `--append-only` and `--storage-quota` Settings Required

All repositories for a given host must use the same `borg_mode_append_only` and
`borg_storage_quota` settings when sharing a single SSH key. The role will fail
with an error if you attempt to configure repositories with conflicting settings
for the same host.

This is because the SSH `authorized_keys` entry uses a single `--append-only` and
`--storage-quota` flag that applies to all repositories accessible via that key.

```yaml
# This will FAIL - conflicting append-only settings
roles:
  - role: kliwniloc.borgbackup
    vars:
      borg_repo_name: configs
      borg_mode_append_only: false # ERROR: inconsistent

  - role: kliwniloc.borgbackup
    vars:
      borg_repo_name: home-data
      borg_mode_append_only: true # ERROR: inconsistent
```

```yaml
# This will FAIL - conflicting storage quota settings
roles:
  - role: kliwniloc.borgbackup
    vars:
      borg_repo_name: configs
      borg_storage_quota: 10G # ERROR: inconsistent

  - role: kliwniloc.borgbackup
    vars:
      borg_repo_name: home-data
      borg_storage_quota: 50G # ERROR: inconsistent
```

### Per-Repo SSH Keys for Independent Settings

To use different `--append-only` or `--storage-quota` settings per repository,
enable `borg_ssh_key_per_repo: true`. This generates a unique SSH keypair for
each `(server, repo)` combination, allowing each repository to have its own
`authorized_keys` entry with independent settings.

```yaml
- name: Configure repos with independent settings using per-repo SSH keys
  hosts: borg-client
  vars:
    borg_server_host: borg-server
    borg_server_host_ssh_key: ssh-rsa AAAAAAAA...
    borg_ssh_key_per_repo: true

  roles:
    - role: kliwniloc.borgbackup
      vars:
        borg_repo_name: configs
        borg_backup_argument: configs
        borg_mode_append_only: true
        borg_storage_quota: 10G
        borg_included_dirs:
          - /etc

    - role: kliwniloc.borgbackup
      vars:
        borg_repo_name: home-data
        borg_backup_argument: home-data
        borg_mode_append_only: false # OK: different key
        borg_storage_quota: 50G # OK: different key
        borg_included_dirs:
          - /home
```

This creates:

- Two separate SSH keypairs: `id_ed25519_borgbackup_borg_server_configs` and
  `id_ed25519_borgbackup_borg_server_home_data`
- Two separate `authorized_keys` entries with independent restrictions so that
  each repository can have its own `--append-only` and `--storage-quota`
  settings

### Decryption Keys Storage Format

Decryption keys are stored in the `decryption_keys.yml` file using the format:

```yaml
hostname_repo_name: |
  To restore key use borg key import --paper /path/to/repo
  BORG PAPER KEY v1
  ...
```

For hosts with multiple repos, each repository has its own entry:

- `borg-client_configs`
- `borg-client_home-data`

### Deprovisioning Borg Repositories

Remove a repository's configuration without deleting the backup data:

```yaml
- role: kliwniloc.borgbackup
  vars:
    borg_server_host: borg-server
    borg_repo_name: my-backup
    state: absent
```

This will not delete the repository data on the backup server.

#### Deleting Repository Data

To also delete the repository data on the backup server:

```yaml
- role: kliwniloc.borgbackup
  vars:
    borg_server_host: borg-server
    borg_repo_name: my-backup
    state: absent
    borg_dangerously_delete_backups: true # WARNING destructive!
```

> [!WARNING]
> `borg_dangerously_delete_backups: true` permanently deletes all backup data
> from the server. Use with caution.

#### Multi-Instance Considerations

When removing one repository from a multi-instance setup:

- Shared SSH keys (`borg_ssh_key_per_repo: false`) are kept
- `authorized_keys` is updated to remove only the deleted repo restriction
- Other repositories remain unaffected

Example - Remove one repo from multi-instance:

```yaml
- role: kliwniloc.borgbackup
  vars:
    borg_server_host: borg-server
    borg_repo_name: configs
    borg_backup_argument: configs
    state: absent
    # Other repos using same SSH key will continue to work
```

#### Limitations

The following components are **NOT** removed by `state: absent`:

| Component                        | Reason                                     |
| -------------------------------- | ------------------------------------------ |
| `borgbackup` package             | May be required by other backups or system |
| Server user (`borg_server_user`) | May be used by other backup clients        |
| Server home directory            | Contains other repositories                |
| Shared SSH keys                  | Other repos still depend on them           |
| SSH known_hosts entries          | May be used for other purposes             |

#### Per-Repo SSH Key Behavior

When `borg_ssh_key_per_repo: true`:

- The specific SSH key for the repo is deleted
- The corresponding `authorized_keys` line is removed

When `borg_ssh_key_per_repo: false`:

- Shared SSH key is kept
- `authorized_keys` line is updated to remove repo restriction
- If the deleted repo is the last one, the entire line is removed

### Manual Deprovisioning

To remove a specific repository (applies to both single and multi-instance setups):

1. Stop and disable the systemd timer and service:

   ```bash
   systemctl stop borg_backup@ARGUMENT.timer
   systemctl disable borg_backup@ARGUMENT.timer
   systemctl stop borg_backup@ARGUMENT.service
   systemctl disable borg_backup@ARGUMENT.service
   ```

   Replace `ARGUMENT` with the `borg_backup_argument` value for that repository.

2. Remove the backup script:

   ```bash
   rm /usr/local/bin/run_borg_backup@ARGUMENT
   ```

3. _(Multi-instance only)_ Edit the base backup script to remove the repository block:

   ```bash
   # Edit /usr/local/bin/run_borg_backup
   # Remove the ANSIBLE MANAGED BLOCK for the deleted repository
   ```

4. _(If other repositories remain)_ Update `authorized_keys` on the Borg server
   to remove the repository restriction. Edit `/opt/borg/.ssh/authorized_keys`
   and remove the `--restrict-to-repository /opt/borg/REPONAME` from the
   appropriate line.

   If this is the last repository for the host, remove the entire line instead.

5. Delete the repository on the Borg server:

   ```bash
   ssh borg@SERVER "borg delete /opt/borg/REPONAME"
   # Or simply: ssh borg@SERVER "rm -rf /opt/borg/REPONAME"
   ```

6. Remove the decryption key entry from your `decryption_keys.yml`.

## Deprovisioning a Complete Host

To remove all backup configuration for a host:

1. Stop and disable all systemd timers and services:

   ```bash
   systemctl list-units --type=timer --all | grep borg_backup
   # For each relevant timer:
   systemctl stop borg_backup@*.timer
   systemctl disable borg_backup@*.timer
   ```

2. Remove all backup scripts:

   ```bash
   rm -f /usr/local/bin/run_borg_backup*
   ```

3. Remove the SSH key from the Borg server's `authorized_keys`:

   ```bash
   # On the Borg server, edit /opt/borg/.ssh/authorized_keys
   # Remove the line containing root@HOSTNAME
   ```

4. Delete all repositories for the host on the Borg server:

   ```bash
   ssh borg@SERVER "rm -rf /opt/borg/HOSTNAME"
   # For multi-instance:
   ssh borg@SERVER "rm -rf /opt/borg/HOSTNAME_repo1"
   ssh borg@SERVER "rm -rf /opt/borg/HOSTNAME_repo2"
   ```

5. Remove all decryption key entries for the host from your
   `decryption_keys.yml`.

## Dependencies

None.

## License

MIT