consul-backinator does not backup ACL tokens in Consul 1.4+

[boarder981]

Hello,

Consul 1.4 introduced a new ACL system where a token's permissions are tied to a policy (or multiple policies), rather than being baked into the token itself. I just recently upgraded my Consul cluster from version 1.3.1 to 1.4.5. After upgrade, all pre-existing non-management tokens were in "legacy" mode. At this point, consul-backinator was backing up the ACLs just fine.

However, after migrating the legacy tokens to the new format, consul-backinator now backs up only the management tokens and ignores anything with a policy tied to it. I confirmed this in two ways:

1. The consul-backinator log indicates that only a handful of ACL tokens were backed up, even though I have about 50 tokens in total

   2022/03/07 15:05:02 [Success] Backed up 10 ACL tokens from consul.service.example.com:8501 to /path/to/backup/my-acls

2. I restored the above backup to a fresh test cluster and confirmed that only 10 ACL tokens exist (all of which are management tokens)

Note that KV backups appear to be working as they did previously.

Can you please implement support for backing up ACLs/Policies in Consul 1.4+? If this already exists, please let me know how to enable it. I can provide more details if needed.

Thank you!

[otterblitzar]

The code currently uses deprecated Consul API calls that only work with the legacy ACL system. For example:

consul-backinator/command/backup/backup.go

Lines 69 to 72 in d8759e6

	// get all acl tokens
	if acls, _, err = c.consulClient.ACL().List(opts); err != nil {
	return 0, err
	}

The fix would be to migrate to the new API calls. For example, replace ACL.List with ACL.TokenList. See:
https://pkg.go.dev/github.com/hashicorp/consul/api#ACL.List

Since ACLs are now composed of tokens and policies, I suggest using a struct to contain both of these items. Example:

type AclBackup struct {
    Policies []byte
    Tokens []byte
}

I'm willing to develop this code, if nobody else wants to. @nathanejohnson Are you OK with the general outline of the fix I've proposed?

[aaronhurt]

That sounds reasonable. The ACL code hasn't been touched in quite some time (6 years ago according to git-blame).

[otterblitzar]

I've made some progress on this, but ran into an interesting problem. It's not possible to restore ACL tokens in Consul 1.4.x because the Consul API does not support setting the AccessorID and SecretID properties. These are available in Consul 1.5, however, so it's only a problem for people still running 1.4.

See https://www.consul.io/api-docs/acl/tokens for details.

[aaronhurt]

The last 1.4.x release was in 2019, I think that's acceptable. Just add a note somewhere in the README that ACL functionality for consul versions >= 1.4.x require backinator <= 1.6.6 and that consul 1.5.x+ is required for the latest codebase for full functionality. Something like that.

[cherishedbrain]

I believe the issue I raised is also related. #48