handle reconciliation for zone creation in case the zone id did not get persisted

[rciorba]

In case zone.create fails, list all zones and match by CallerReference, if the config is the same, persist that Id, else delete the zone and create a new one.

Background:

So I've used the CallerReference for create_health_check operation to ensure my code can correctly handle transient error. So if I create the same exact HealthCheck twice, with the same CallerReference, the create is idempotent and I get back the first HealthCheck on the second create call. This allows me to ensure my code can recover in the event it called create_health_check but failed to persist the Id (*1).

However when I attempted to apply the same pattern to create_hosted_zone the second call always fails with "HostedZoneAlreadyExists".

(*1) If the second create_healthcheck has a different HealthCheckConfig the call fails as expected

[rciorba]

@pytest.mark.django_db
def test_zone_reconcile(zone, boto_client):
    original_id = zone.route53_id
    zone.route53_id = None  # simulate failed create call
    zone.save()
    zone.route53_zone._clear_cache()
    zone.route53_zone.reconcile()
    assert zone.route53_id == original_id