letsencrypt fails with server error 503 when TARGET_SUBDIR=/

[igor-bn]

It is my first try with your docker image. I want to say thanks, it looks really pomising to me!
I tried to follow the steps described in README and I got an error while trying to generate letsecnrypt cerificate.

Here is the fail trace:

docker exec -it docker_owncloud_1 sh -c 'EMAIL=XXX@YYY DO_SSL_LETS_ENCRYPT_FETCH=true setup-apache-ssl-key'
Fetching ssl certificate files for XXX.YYY from letsencrypt.org.
This container's Apache server must be reachable from the Internet via http://XXX.YYY
Traceback (most recent call last):
  File "/usr/bin/letsencrypt", line 9, in <module>
    load_entry_point('letsencrypt==0.4.2', 'console_scripts', 'letsencrypt')()
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 1993, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 707, in obtain_cert
    _, action = _auth_from_domains(le_client, config, domains, lineage)
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 475, in _auth_from_domains
    lineage = le_client.obtain_and_enroll_certificate(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 269, in obtain_and_enroll_certificate
    certr, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 252, in obtain_certificate
    return self.obtain_certificate_from_csr(domains, csr) + (key, csr)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 225, in obtain_certificate_from_csr
    authzr = self.auth_handler.get_authorizations(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 84, in get_authorizations
    self._respond(cont_resp, dv_resp, best_effort)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 142, in _respond
    self._poll_challenges(chall_update, best_effort)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 204, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
FailedChallenges: Failed authorization procedure. XXX.YYY (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://XXX.YYY/.well-known/acme-challenge/WaXwvVvJj_KVfTZS5e5rmKQypx8LQSP_NZJXucvdTps [46.128.XXX.XXX]: 503

Looks like the 503 comes from configs/apache.conf

 <If "%{HTTPS} != 'on' && env('ALLOW_INSECURE') != 'true'">
    ErrorDocument 503 "Please use https."
    Redirect 503 /${TARGET_SUBDIR}
  </If>

I tried then to set ALLOW_INSECURE=true for my container beut ended up with HTTP 404

docker exec -it docker_owncloud_1 sh -c 'EMAIL=XXX@YYY DO_SSL_LETS_ENCRYPT_FETCH=true setup-apache-ssl-key'
Fetching ssl certificate files for XXX.YYY from letsencrypt.org.
This container's Apache server must be reachable from the Internet via http://XXX.YYY
Traceback (most recent call last):
  File "/usr/bin/letsencrypt", line 9, in <module>
    load_entry_point('letsencrypt==0.4.2', 'console_scripts', 'letsencrypt')()
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 1993, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 707, in obtain_cert
    _, action = _auth_from_domains(le_client, config, domains, lineage)
  File "/usr/lib/python2.7/site-packages/letsencrypt/cli.py", line 475, in _auth_from_domains
    lineage = le_client.obtain_and_enroll_certificate(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 269, in obtain_and_enroll_certificate
    certr, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 252, in obtain_certificate
    return self.obtain_certificate_from_csr(domains, csr) + (key, csr)
  File "/usr/lib/python2.7/site-packages/letsencrypt/client.py", line 225, in obtain_certificate_from_csr
    authzr = self.auth_handler.get_authorizations(domains)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 84, in get_authorizations
    self._respond(cont_resp, dv_resp, best_effort)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 142, in _respond
    self._poll_challenges(chall_update, best_effort)
  File "/usr/lib/python2.7/site-packages/letsencrypt/auth_handler.py", line 204, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
FailedChallenges: Failed authorization procedure. XXX.YYY (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://XXX.YYY/.well-known/acme-challenge/cN0f2Nq5sgoT1aR2R8_YZUkqPGAn5hD84MnFslQh_ec [46.128.XXX.XXX]: 404

Can you please take a closer look of what might have happenend?

[greyltc]

Hi Igor. I'll have a look at it. Did you redact some information there or did you actually use XXX.YYY?
Do you know that you have your DNS set up right and firewall ports open to the container?

[greyltc]

More thoughts here:
I'm surprised that letsencrypt is seeing my 503 error.
Are you using TARGET_SUBDIR=/ by any chance?

[igor-bn]

Hi Grey, thanks for prompt reply! :-)
Yes, I did masked actual e-mail, IP and domain with XXX, YYY and so on in the report above.
I am actually using dynamic DNS and port forwarding within my local network. The container itself runs at port 10080 and 10443. My router maps these ports to 80 and 443 for the outter world. This setup is the reason why I set TARGET_SUBDIR=/
Is it a problem?
I did check that container is really reachable from the outside for both ports using the domian specified.

[greyltc]

I bet there's a conflict I need to document between let's encrypt fetching and setting TARGET_SUBDIR=/ Try leaving that out so that your oc server is at /owncloud

[igor-bn]

Hi, I set back TARGET_SUBDIR to owncloud and it went further, thanks!
However I got next error in /usr/bin/setup-apache-ssl-key: line 46: [: missing]'`
And the lock in the browser isn't green after an hour.
Did I missed something? Should I restart the container to make the certificate effective?

Here is full log (masked) for reference:

docker exec -it docker_owncloud_1 sh -c 'EMAIL=XXX@YYY DO_SSL_LETS_ENCRYPT_FETCH=true setup-apache-ssl-key'
Fetching ssl certificate files for XXX.YYY from letsencrypt.org.
This container's Apache server must be reachable from the Internet via http://XXX.YYY

IMPORTANT NOTES:
 - If you lose your account credentials, you can recover through
   e-mails sent to XXX@YYY.
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/XXX.YYY/fullchain.pem. Your cert
   will expire on 2016-06-27. To obtain a new version of the
   certificate in the future, simply run Let's Encrypt again.
 - Your account credentials have been saved in your Let's Encrypt
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Let's
   Encrypt so making regular backups of this folder is ideal.
 - If you like Let's Encrypt, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

/usr/bin/setup-apache-ssl-key: line 46: [: missing `]'
Success! now copy your cert files out of the image and save them somewhere safe:
docker cp CONTAINER:/etc/letsencrypt/archive/XXX.YYY ~/letsencryptFor_XXX.YYY

[igor-bn]

Made closer look by myself.
It looks like line 46 is missing a whitespace after httpd.pid

 [ -f /var/run/httpd/httpd.pid] && apachectl graceful

I restarted the apache then directly from container.
Another option would be to correct copy instruction of

echo "docker cp CONTAINER:/etc/letsencrypt/archive/${HOSTNAME} ~/letsencryptFor_${HOSTNAME}"

to something like

echo "docker cp CONTAINER:/root/sslKeys ~/letsencryptFor_${HOSTNAME}"

should be much better, as the files there are ready to be used for the option (A) without any renaming

[greyltc]

Oops! That's a bug. Nice catch on my bash syntax mistake. It's fixed now.

I'd like to back up the whole letsencrypt folder rather than just what I copied. I think there's other important stuff in there than just the files apache needs to enable SSL

[greyltc]

Anyway, there is a problem that remains here:
Following my let's encrypt fetching instructions while TARGET_SUBDIR=/ is set does not work; the let's encrypt verification gets rejected by my http blocking security measure. I think I'll need to set up an exception for the folder that let's encrypt instructs apache to serve up for verification purposes: TODO