Refactor for v9.0.0

[bmhughes]

Description

• Refactor cookbook
  • Remove legacy code
  • Refactor chain and rule resources
    • Complex logic moved to helper libraries
  • Refactor service resource
    • Add the full set of service actions
    • Create a default configuration to ensure starting on Redhat platform families
  • Refactor package resource
    • Add the full set of package actions
• All resource unified_mode
• Rename iptables_packages to iptables_package (old name aliased)

Issues Resolved

• n/a

Check List

• All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
• New functionality includes testing.
• New functionality has been documented in the README if applicable
• All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

[xorima]

I would like this to be reviewed with the thoughts of we will be shortly creating a PR to put this into Chef 17, so I have asked @lamont-granquist as well as @tas50 to review 👍

[detjensrobert]

This appears not to respect the cookbook property and will always use the template from this cookbook instead of a specified one.

Do we want to keep support for C6 since it is EOL?

[detjensrobert]

Was the default of :filter removed for a reason? We use this default heavily in our wrapper.

[xorima]

We have to keep support for centos6 on this one, just due to chefs customer base and us wanting to put this in core.

[bmhughes]

I had some issues with CentOS 6 testing (again!) and wanted to FIO but gathered we'd want to keep it even though it's EoL now.

[bmhughes]

Thoughts on the table default value? I'd prefer to drop the default and make it required (needs doing for rule as well if so) so that things have to be explicitly set to reduce the setting a rule on the wrong table factor.

But I'll go with the general concensus, the way I do wrapper cookbooks it doesn't matter much to me either way.

[bmhughes]

This appears not to respect the cookbook property and will always use the template from this cookbook instead of a specified one.

Hmm i'll check this out but it should do, the only change that will effect it is it'll take the setting from the first resource called (service/chain/rule) as I only initialise the template resource if it doesn't already exist. Might refactor that thinking about it but then it'll come down the last resource declaration so i'm not 100% convinced it's actually better either.

[detjensrobert]

Figured out why our wrapper template was not getting picked up: the cookbook property needs to be set on the _service resource, not just on the _chain and _rule resources. IMO the _service resource should not affect the rules template at all since it does not modify it, and let the first _chain or _rule call create it.

The examples in the documentation for _service do not match what is needed for the service to correctly restart when the template changes. The test cookbook has the delayed_action and manual subscribes, which are not mentioned as needed in the docs at all and are required to work correctly (at least from my testing). Would having the template resource notify the service be better, since that way the user does not have to remember to subscribe to the template?

[bmhughes]

Figured out why our wrapper template was not getting picked up: the cookbook property needs to be set on the _service resource, not just on the _chain and _rule resources. IMO the _service resource should not affect the rules template at all since it does not modify it, and let the first _chain or _rule call create it.

The examples in the documentation for _service do not match what is needed for the service to correctly restart when the template changes. The test cookbook has the delayed_action and manual subscribes, which are not mentioned as needed in the docs at all and are required to work correctly (at least from my testing). Would having the template resource notify the service be better, since that way the user does not have to remember to subscribe to the template?

Is this with the current release version of the cookbook or this PR? This is still a WIP so there may well be things missing/incorrect/will change so bear this in mind if you're attempting to use it.

The rulefile template resource is going to be created (and thus take it's source) by the first resource that requires it to exist (via the rulefile_resource_init method), if you create a service resource before a rule or chain then this where the template will be sourced from hence the behaviour you see. The alternative is to update on every resource call which is debatably worse. The service needs the rulefile to exist to start so it has the ability to ensure it is present.

The trouble with adding notifications to these resources internally is that is hard couples them and removes the benefit of pushing everything into resources and removes flexibility for the implementer in their wrapper. How to do this can be added to the documentation for sure, but the overall idea is someone may choose not to do that with their implementation which is why we try to make these resources as abstract as possible.

[ramereth]

@bmhughes can you please rebase and resolve the conflicts so we can properly review it?

[detjensrobert]

Is this with the current release version of the cookbook or this PR? This is still a WIP so there may well be things missing/incorrect/will change so bear this in mind if you're attempting to use it.

This was with the current version of this branch.

The rulefile template resource is going to be created (and thus take it's source) by the first resource that requires it to exist (via the rulefile_resource_init method), if you create a service resource before a rule or chain then this where the template will be sourced from hence the behaviour you see. The alternative is to update on every resource call which is debatably worse. The service needs the rulefile to exist to start so it has the ability to ensure it is present.

Ah, I see.

The trouble with adding notifications to these resources internally is that is hard couples them and removes the benefit of pushing everything into resources and removes flexibility for the implementer in their wrapper. How to do this can be added to the documentation for sure, but the overall idea is someone may choose not to do that with their implementation which is why we try to make these resources as abstract as possible.

That does make sense, even if that use case is a bit odd.

[detjensrobert]

I'm having some trouble with getting the template to restart when the recipe calling iptables_service is not the root recipe in the runlist. Our wrapper cookbooks structure follows this pattern:

node::recipe
  - wrapper_resource
    - wrapper::recipe
      - iptables_service
      - iptables_chain/_rule
    - iptables_chain/_rule

I've created a POC repo that reproduces the problem: https://github.com/detjensrobert/firewall-wrapper-test. Is there something obvious missing with this setup that is preventing the service resource from getting notified?

(testing on C7/C8)

[bmhughes]

To be perfectly honest (trying not to sound a complete dick here), there's all sorts of things 'wrong' with that cookbook and the patterns it's following so i'm not completely surprised you're having issues and I'd be surprised if they were anything to do with the iptables cookbook itself.

I'm using this internally with the same wrapper I've used for the last couple of version without issue and without too many changes so I know it does work in practice.

Looking at that example my thoughts are:

1. Why the 'wrapper' resource at all? Could be all done in a recipe and the abstraction/encapsulation a resource provides has little to no benefit there.
2. Same above for the resource, if you aren't going to call it multiple times for different cases (which the pattern would suggest not), get rid of it.
3. Calling a recipe from a resource is just.....wrong.
4. There's no timer specified to the subscribes, not that it would 100% break the subscription but a bad smell all the same.
5. From a pure Ruby perspective extending the helpers module into the service resource instance block is messy, you know what the template resource name is going to be, just use that.

[bmhughes]

For constrast, my wrapper contains only a set of recipes that:

1. installs packages
2. iterates chains ipv4
3. iterates rules ipv4
4. iterates chains ipv6
5. iterates rules ipv6
6. service ipv4/service ipv6 with relevant subscribes.

Other than those couple of recipes and some methods to set ip_family automatically there's nothing else in the cookbook.

[bmhughes]

Also if someone else has 5 can they try the debian 9/ubuntu 1604/1804 tests locally as they pass for me but fall over on actions. It's going to be related to docker/kernel/modules no doubt.

[detjensrobert]

Also if someone else has 5 can they try the debian 9/ubuntu 1604/1804 tests locally as they pass for me but fall over on actions. It's going to be related to docker/kernel/modules no doubt.

Tests pass here in Vagrant & Openstack 👍

[ramereth]

@bmhughes I can't replicate the current failures but I suspect it's related to a missing iptables kernel module not being loaded on the host system before docker runs. If you add something like the following it can probably give you some additional feedback:

https://github.com/sous-chefs/percona/blob/6e9d276cde44e96d2e848501d1fd656edc0496f8/.github/workflows/ci.yml#L130-L137

[tas50]

@bmhughes Any chance you can get the DCO slapped on here and take a look at the failures in CI?