Provide multiple repositories via env variables

[marco-calautti]

It would be very convenient to pass to the container a list of repositories, rather than a single one. So that the same operations can be execute for multiple repositories in the same session. At the moment, one has to run instances of the container with almost identical configurations, to backup to two or more repositories.

[ThomDietrich]

... plus it would be great to run multiple syncs in sequence. Currently this can only be achieved by careful definition of the cron expressions.

I support this request

[djmaze]

Do I get this right, you want to backup the same data to multiple repositories?

[marco-calautti]

Exactly

[ThomDietrich]

Then I got it wrong 😄
I think it would be a worthy addition to be able to define multiple jobs in general. The use case would be to easily configure something that runs multiple backup jobs in sequence, e.g. at 3:00 am on workdays.

[smainz]

I have a derived image which includes the script push-to-mirror.sh (You can also mount it into the container)

#!/bin/bash

# If mirror repository and password are set on the command line, use these, otherwise rely on the environment

if [ -n "$1" ]; then
    RESTIC_MIRROR_REPOSITORY=$1
    if [ -n "$2" ]; then
        RESTIC_MIRROR_PASSWORD=$2
    fi
fi

if [ -z "$RESTIC_MIRROR_REPOSITORY" ] || [ -z "$RESTIC_MIRROR_PASSWORD" ]; then
    echo "Usage $0 <RESTIC_MIRROR_REPOSITORY> <RESTIC_MIRROR_PASSWORD>"
    echo ""
    echo "    if RESTIC_MIRROR_REPOSITORY is not given the environment variable RESTIC_MIRROR_REPOSITORY is used."
    echo "    if RESTIC_MIRROR_PASSWORD is not given the environment variable RESTIC_MIRROR_PASSWORD is used."
    echo ""
    exit 255
fi

# Try to initialize mirror repository
if RESTIC_PASSWORD=${RESTIC_MIRROR_PASSWORD} restic --repo ${RESTIC_MIRROR_REPOSITORY} cat config  > /dev/null; then
    echo "Mirror Repository exists, password is correct"
else
    echo "Trying to initialize mirror repository"
    RESTIC_FROM_PASSWORD=${RESTIC_PASSWORD} \
    RESTIC_PASSWORD=${RESTIC_MIRROR_PASSWORD} \
    restic --repo ${RESTIC_MIRROR_REPOSITORY} init --from-repo ${RESTIC_REPOSITORY} --copy-chunker-params
fi


echo "Copy local repository to mirror repository"
RESTIC_FROM_PASSWORD=${RESTIC_PASSWORD} \
RESTIC_PASSWORD=${RESTIC_MIRROR_PASSWORD} \
restic --repo ${RESTIC_MIRROR_REPOSITORY} copy --from-repo ${RESTIC_REPOSITORY}

echo  "Clean up mirror repository (purge old files)"
RESTIC_PASSWORD=${RESTIC_MIRROR_PASSWORD} \
restic --repo ${RESTIC_MIRROR_REPOSITORY} forget --prune ${RESTIC_MIRROR_FORGET_ARGS}

echo  "Unlock mirror in case ssh connection failed in above command"
RESTIC_PASSWORD=${RESTIC_MIRROR_PASSWORD} \
restic --repo ${RESTIC_MIRROR_REPOSITORY} unlock

RESTIC_MIRROR_REPOSITORY and RESTIC_MIRROR_PASSWORD need to be set as env vars like the ordinary ones.

[djmaze]

Well, I think the configuration via env variables is already complicated enough. I am unsure if it is a good idea to add even one
more layer of variables. The thing that comes to mind would be adding numbers to all variables. Like RESTIC_REPOSITORY_1, RESTIC_PASSWORD_1, RESTIC_REPOSITORY_2 and so on. But that would be an optional way to configure this. The old way should still always be available for backwards compatibility reasons.

[ThomDietrich]

@djmaze I personally think that would be a very valuable addition and I like your suggestion. Do you see yourself implementing those or are you limited to accept PRs? :)

[djmaze]

Tbh I am not really incentivized as I'd rather cope with multiple containers with different schedules rather than increasing the complexity of the service. That said, if someone wants to implement this in a clean manner, then I would be open to accept it ;)

[schewara]

As I had the same issue, I implemented a small wrapper which I then use to cleanup all repos in one central place and a different user who is allowed to perform the cleanup function.

It's quite outdated again and will have to clean it up sync it with the latest changes from here, but I am happy to create a PR for it.

https://github.com/schewara/resticker