Pachelbel crashes during parallel execution.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x78439e]
goroutine 231 [running]:
github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi.(*Client).getJSON(0xc4200709b0, 0x892147, 0xb, 0xae5f882ea153170d, 0xf6a0465b0e40a0cf, 0xee34cb14f73d306d, 0xf6a0465b0e40a0cf, 0xe934cb14f73d306d)
/go/src/github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi/composeapi.go:127 +0xce
github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi.(*Client).GetDeploymentsJSON(0xc4200709b0, 0xef34cb14f73d306d, 0xf6a0465b0e40a0cf, 0xec34cb14f73d306d, 0xf6a0465b0e40a0cf, 0xed34cb14f73d306d)
/go/src/github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi/deployment.go:168 +0x40
github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi.(*Client).GetDeployments(0xc4200709b0, 0xc42057ea38, 0x43274b, 0x12e, 0xc42057ea48)
/go/src/github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi/deployment.go:172 +0x2f
github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi.(*Client).GetDeploymentByName(0xc4200709b0, 0xc4202e62d0, 0x2d, 0xc4202e62d0, 0x2d, 0x0, 0x0)
/go/src/github.com/benjdewan/pachelbel/vendor/github.com/benjdewan/gocomposeapi/deployment.go:206 +0x43
github.com/benjdewan/pachelbel/connection.(*Connection).GetAndAdd(0xc42006dc80, 0xc4202e62d0, 0x2d, 0xc4201cdc10, 0xf)
/go/src/github.com/benjdewan/pachelbel/connection/connection.go:137 +0x5d
github.com/benjdewan/pachelbel/runner.dryRunLookup(0xc42006dc80, 0x8ee840, 0xc4201c00d0, 0x0, 0xc4203795a0)
/go/src/github.com/benjdewan/pachelbel/runner/runner.go:88 +0x43
github.com/benjdewan/pachelbel/runner.(*Controller).Run.func1(0xc4202401e0, 0xc4201a8008, 0xc4203f23d0, 0x8ee840, 0xc4201c00d0, 0xc420358480, 0x13, 0x8b6010)
/go/src/github.com/benjdewan/pachelbel/runner/controller.go:53 +0x4e
created by github.com/benjdewan/pachelbel/runner.(*Controller).Run
/go/src/github.com/benjdewan/pachelbel/runner/controller.go:52 +0x255
The reason is that Compose.io sometimes returns 503 instead of valid json when running pachabel in parallel multiple times.
<!DOCTYPE html>
<html class="error-layout" lang="en"><head>
<title>Error 503: Service Unavailable - Compose</title>
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta charset="utf-8">
<link rel="shortcut icon" type="image/x-icon" href="https://d11f55tj5eo9e5.cloudfront.net/assets/megatron/favicon.ico">
<link rel="stylesheet" media="screen" href="https://d11f55tj5eo9e5.cloudfront.net/assets/megatron/megatron-error-pages-0.1.81.css.gz">
</head>
<body>
<div class="error-page error-503 ">
<svg xmlns="http://www.w3.org/2000/svg" width="312"><g id="group-1-cube-1"><path fill="#1D2125" d="M210.6 0l-20 11.2v22.4l20 11.2 20-11.2V11.2L210.6 0z"></path><path fill="#DADEDE" d="M210.6 5.3l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-2"><path fill="#1D2125" d="M190.6 11.2l-20 11.2v22.4l20 11.2 20-11.2V22.4l-20-11.2z"></path><path fill="#DADEDE" d="M190.6 16.5L175.3 25l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-3"><path fill="#1D2125" d="M170.6 22.4l-20 11.2V56l20 11.2 20-11.2V33.6l-20-11.2z"></path><path fill="#DADEDE" d="M170.6 27.7l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-4"><path fill="#1D2125" d="M230.6 11.2l-20 11.2v22.4l20 11.2 20-11.2V22.4l-20-11.2z"></path><path fill="#DADEDE" d="M230.6 16.5L215.3 25l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-5"><path fill="#1D2125" d="M250.6 22.4l-20 11.2V56l20 11.2 20-11.2V33.6l-20-11.2z"></path><path fill="#DADEDE" d="M250.6 27.7l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-6"><path fill="#1D2125" d="M230.6 33.6l-20 11.2v22.4l20 11.2 20-11.2V44.8l-20-11.2z"></path><path fill="#DADEDE" d="M230.6 38.9l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-7"><path fill="#1D2125" d="M210.6 44.8l-20 11.2v22.4l20 11.2 20-11.2V56l-20-11.2z"></path><path fill="#DADEDE" d="M210.6 50.1l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-8"><path fill="#1D2125" d="M270.6 33.6l-20 11.2v22.4l20 11.2 20-11.2V44.8l-20-11.2z"></path><path fill="#DADEDE" d="M270.6 38.9l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-9"><path fill="#1D2125" d="M290.6 44.8l-20 11.2v22.4l20 11.2 20-11.2V56l-20-11.2z"></path><path fill="#DADEDE" d="M290.6 50.1l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-10"><path fill="#1D2125" d="M270.6 56l-20 11.2v22.4l20 11.2 20-11.2V67.2l-20-11.2z"></path><path fill="#DADEDE" d="M270.6 61.3l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-1-cube-11"><path fill="#1D2125" d="M250.6 67.2l-20 11.2v22.4l20 11.2 20-11.2V78.4l-20-11.2z"></path><path fill="#DADEDE" d="M250.6 72.5L235.3 81l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-1"><path fill="#1D2125" d="M135.3 42.2l-20 11.2v22.4l20 11.2 20-11.2V53.4l-20-11.2z"></path><path fill="#DADEDE" d="M135.3 47.5L120 56l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-2"><path fill="#1D2125" d="M115.3 53.4l-20 11.2V87l20 11.2 20-11.2V64.6l-20-11.2z"></path><path fill="#DADEDE" d="M115.3 58.7L100 67.2l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-3"><path fill="#1D2125" d="M95.3 64.6l-20 11.2v22.4l20 11.2 20-11.2V75.8l-20-11.2z"></path><path fill="#DADEDE" d="M95.3 69.9L80 78.4 95.3 87l15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-4"><path fill="#1D2125" d="M155.3 53.4l-20 11.2V87l20 11.2 20-11.2V64.6l-20-11.2z"></path><path fill="#DADEDE" d="M155.3 58.7L140 67.2l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-5"><path fill="#1D2125" d="M175.3 64.6l-20 11.2v22.4l20 11.2 20-11.2V75.8l-20-11.2z"></path><path fill="#DADEDE" d="M175.3 69.9L160 78.4l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-6"><path fill="#1D2125" d="M115.3 75.8L95.3 87v22.4l20 11.2 20-11.2V87l-20-11.2z"></path><path fill="#DADEDE" d="M115.3 81.1L100 89.6l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-7"><path fill="#1D2125" d="M135.3 87l-20 11.2v22.4l20 11.2 20-11.2V98.2l-20-11.2z"></path><path fill="#DADEDE" d="M135.3 92.3l-15.3 8.5 15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-8"><path fill="#1D2125" d="M155.3 98.2l-20 11.2v22.4l20 11.2 20-11.2v-22.4l-20-11.2z"></path><path fill="#DADEDE" d="M155.3 103.5L140 112l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-9"><path fill="#1D2125" d="M195.3 75.8l-20 11.2v22.4l20 11.2 20-11.2V87l-20-11.2z"></path><path fill="#DADEDE" d="M195.3 81.1L180 89.6l15.3 8.6 15.3-8.6-15.3-8.5"></path></g><g id="group-2-cube-10"><path fill="#1D2125" d="M215.3 87l-20 11.2v22.4l20 11.2 20-11.2V98.2l-20-11.2z"></path><path fill="#DADEDE" d="M2
<div class="error-message">
<h1><span>Service Unavailable.</span></h1>
<p><span>But don't worry, this shouldn't affect your databases.</span></p>
</div>
<footer class="support-links">
<nav>
<a href="mailto:support@compose.io">Contact Support</a><a href="https://status.compose.io">Compose Status</a>
</nav>
<a href="https://compose.io" class="compose-logo"><span class="hidden_label">Compose</span></a>
</footer>
</div>
</body></html>
$ seq 25 | parallel -j 25 bash -c "echo 'begin{#}'; ./pachelbel --dry-run -l multi_compose.log_{#} provision example_multi_dev.yaml; echo 'end{#}'"
$ cat example_multi_dev.yaml
config_version: 1
name: postgres-XXXXXXXXX-YYYYYYYYYY
type: postgresql
cluster: some-cluster
---
config_version: 1
name: postgres-XXXXXXXXX-1
type: elasticsearch
cluster: some-cluster
---
config_version: 1
name: elasticsearch-XXXXXXXXX-1
type: elasticsearch
cluster: some-cluster
---
config_version: 1
name: elasticsearch-XXXXXXXXX-2
type: elasticsearch
cluster: some-cluster
---
config_version: 1
name: elasticsearch-XXXXXXXXX-3
type: elasticsearch
cluster: some-cluster
Pachelbel crashes during parallel execution.
The reason is that Compose.io sometimes returns 503 instead of valid json when running pachabel in parallel multiple times.
One can simulate it by GNU parallel: