Let's Encrypt and acme-dns (or, wildcard certificates with any DNS provider)

In early 2018, Let's Encrypt began issuing wildcard certificates, after significant popular demand. One significant limitation, though, is that users must validate domain control using the DNS-01 challenge, by adding certain DNS TXT records. As a practical matter, that requires that you use a DNS provider with a supported API for automatic updates.

Many DNS providers provide APIs, but many more do not. Of the ones that do, not all are supported by popular ACME clients. And of the supported APIs, almost all are far too powerful–if an attacker were to compromise your API credentials, he could make any changes he wanted to your DNS records, potentially including stealing your domain entirely.

Enter acme-dns. acme-dns is a limited-purpose DNS server, whose only purpose is to serve the DNS TXT records needed for Let's Encrypt validation. It's a lightweight application, and offers an API that ACME clients can use to automatically create and destroy those TXT records. It automatically generates credentials that are only valid for a single subdomain. Between this and the fact that acme-dns will only serve the TXT records used for Let's Encrypt validation, in the worst case, an attacker who compromised your acme-dns API credentials would only be able to mis-issue a certificate for one of your (sub)domains. This still isn't a good thing by any means, but the exposure is far less than with most other DNS APIs.

The effect of this is that if you can create NS and CNAME records on your DNS provider, you can then use acme-dns to automate the DNS updates needed to get and renew your certificates (whether wildcard or not).

Background and Theory

acme-dns will act as the authoritative DNS server for a subdomain of your domain. If your domain is example.com, that subdomain will be acme.example.com. The acme-dns software will generate random hostnames within this subdomain (one random hostname for each FQDN you want to obtain a cer for), of the form 32f5274d-51e3-466d-bf38-eb9980e7bcf3.acme.example.com. You'll add a CNAME record for _acme-challenge.example.com, pointing to the random hostname. When Let's Encrypt tries to validate domain control over example.com, it'll look for DNS records for _acme-challenge.example.com, see the CNAME to 32f5274d-51e3-466d-bf38-eb9980e7bcf3.acme.example.com, and then query for text records for _acme-challenge.32f5274d-51e3-466d-bf38-eb9980e7bcf3.acme.example.com. At that point, acme-dns will serve the challenge that your client told it to serve, validation will succeed, and Let's Encrypt will issue the cert.

Note: Once you've set up acme-dns, you can use it for any domain you control. In other words, you can use a single instance to validate control over any number of domains or subdomains, just by setting individual CNAME records for each desired (sub)domain.

Conventions

Throughout this guide, your domain is represented as example.com. If you have more than one domain, pick one. The external IP address of your Neth server is represented as $EXTERNAL_IP. Make appropriate substitutions below.

DNS Configuration

You'll need to start by publishing the DNS records establishing your acme-dns instance as the authoritative nameserver for acme.example.com. To do this, log in to your DNS provider and add the records below:

ns1.acme.example.com	A	$EXTERNAL_IP
ns2.acme.example.com	A	$EXTERNAL_IP
acme.example.com		NS	ns1.acme.example.com
acme.example.com		NS	ns2.acme.example.com

You're done with DNS records for the time being.

Installing acme-dns

acme-dns is not available as an RPM in any of the standard repositories (to my knowledge), but its author does have a precompiled binary available for download. Download this package onto your server, extract it using tar zxf, and move the acme-dns binary to /usr/local/acme-dns/acme-dns.

Create a configuration file at /etc/acme-dns/config.cfg using your favorite text editor. Its contents should look like this:

[general]
# dns interface
listen = "$EXTERNAL_IP:1053"
# protocol, "udp", "udp4", "udp6" or "tcp", "tcp4", "tcp6"
protocol = "udp"
# domain name to serve the requests off of
domain = "acme.example.com"
# zone name server
nsname = "ns1.acme.example.com"
# admin email address, where @ is substituted with .
nsadmin = "admin.example.com"
# predefined records served in addition to the TXT
records = [
    # default A
    "acme.example.com. A $EXTERNAL_IP",
    # A
    "ns1.acme.example.com. A $EXTERNAL_IP",
    "ns2.acme.example.com. A $EXTERNAL_IP",
    # NS
    "acme.example.com. NS ns1.acme.example.com.",
    "acme.example.com. NS ns2.acme.example.com.",
]
# debug messages from CORS etc
debug = false

[database]
# Database engine to use, sqlite3 or postgres
engine = "sqlite3"
# Connection string, filename for sqlite3 and postgres://$username:$password@$host/$db_name for postgres
# Please note that the default Docker image uses path /var/lib/acme-dns/acme-dns.db for sqlite3
connection = "/etc/acme-dns/acme-dns.db"
# connection = "postgres://user:password@localhost/acmedns_db"

[api]
# domain name to listen requests for, mandatory if using tls = "letsencrypt"
api_domain = ""
# listen ip eg. 127.0.0.1
ip = "0.0.0.0"
# disable registration endpoint
disable_registration = false
# autocert HTTP port, eg. 80 for answering Let's Encrypt HTTP-01 challenges. Mandatory if using tls = "letsencrypt".
# autocert_port = "80"
# listen port, eg. 443 for default HTTPS
port = "8675"
# possible values: "letsencrypt", "cert", "none"
tls = "none"
# only used if tls = "cert"
#tls_cert_privkey = "/etc/tls/example.org/privkey.pem"
#tls_cert_fullchain = "/etc/tls/example.org/fullchain.pem"
# CORS AllowOrigins, wildcards can be used
corsorigins = [
    "*"
]
# use HTTP header to get the client ip
use_header = false
# header name to pull the ip address / list of ip addresses from
header_name = "X-Forwarded-For"

[logconfig]
# logging level: "error", "warning", "info" or "debug"
loglevel = "debug"
# possible values: stdout, TODO file & integrations
logtype = "stdout"
# file path for logfile TODO
# logfile = "./acme-dns.log"
# format, either "json" or "text"
logformat = "text"

Next, you’ll need to create a service file for systemd. Using your favorite text editor, create /etc/systemd/system/acme-dns.service with the contents below:

[Unit]
Description=acme-dns Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/acme-dns/acme-dns

[Install]
WantedBy=multi-user.target

You can then enable and start the service by running systemctl enable –now acme-dns.

Next, we’ll need to open port 53 on the Internet (red) interface and redirect it to port 1053 for acme-dns to listen to it. Create /etc/e-smith/templates-custom/etc/shorewall/rules/95acme with the following contents:

REDIRECT  net  1053  tcp  53
REDIRECT  net  1053  udp  53

Then run

config set fw_acme-dns service status enabled UDPPort 53 TCPPort 53 access red
signal-event firewall-adjust

Installation of acme-dns is now complete.

The hook script

Certbot doesn’t know, on its own, how to set DNS entries on acme-dns. Fortunately, the author of acme-dns has provided a script to handle this. Download the script by doing curl -o /etc/letsencrypt/acme-dns-auth.py https://raw.githubusercontent.com/joohoi/acme-dns-certbot-joohoi/master/acme-dns-auth.py followed by chmod 0700 /etc/letsencrypt/acme-dns-auth.py.

You’ll need to edit the script. Near the top, change ACMEDNS_URL to http://localhost:8675. No other changes need to be made.

Issuing the certificate

All the prep work is now done, and it’s time to issue the certificate. Run

certbot certonly --manual --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py \
   --preferred-challenges dns --debug-challenges --post-hook "signal-event certificate-update" \
   -d example.com -d \*.example.com

If you want more domains, add them with additional -d flags at the end. Certbot will ask you a few questions (email address, agree to TOS, share email with EFF, log IP address), and then show a message like this:

Output from acme-dns-auth.py:
Please add the following CNAME record to your main DNS zone:
_acme-challenge.example.com CNAME 32f5274d-51e3-466d-bf38-eb9980e7bcf3.acme.example.com.

Waiting for verification...

-------------------------------------------------------------------------------
Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.
-------------------------------------------------------------------------------
Press Enter to Continue

You’ll need to once again log into your DNS host and add the record specified. You'll only need to do this once for each hostname. Wait a few minutes, then press Enter.

As noted above, once you have acme-dns running, you can use it to validate any domain you control, as long as you can set the appropriate CNAME records. But if you're going to be accessing its API remotely, you should really secure its API using HTTPS.

Get a certificate

We just created a wildcard certificate above, so we could use that, but it's better practice to use a unique certificate for this service. To issue that certificate, run the following:

certbot certonly --manual --manual-auth-hook /etc/letsencrypt/acme-dns-auth.py \
   --preferred-challenges dns --debug-challenges --post-hook "systemctl restart acme-dns" \
   -d acme.example.com

Also as above, you’ll be told to create a CNAME record with your DNS host. Do that, wait a couple of minutes, then press Enter. Your certificate will be generated.

Configure acme-dns for HTTPS

You’ll need to make just a few changes to /etc/acme-dns/config.cfg:

tls = "cert"
tls_cert_privkey = "/etc/letsencrypt/live/acme.example.com/privkey.pem"
tls_cert_fullchain = "/etc/letsencrypt/live/acme.example.com/fullchain.pem"

Then restart the service: systemctl restart acme-dns.

Configure the hook script

You’ll also need to update the hook script. Edit /etc/letsencrypt/acme-dns-auth.py and change ACMEDNS_URL to https://acme.example.com:8675, then save and exit.

Adjust the firewall settings

We need to open another port in the firewall for this:

config set fw_acme-dns-api service status enabled TCPPort 8675 access red,green
signal-event firewall-adjust

You’re done. You’ve made your acme-dns API available via HTTPS over the Internet and your LAN, so you can now make requests for zone updates from other hosts. You can run certbot on those machines, or any other client that knows how to update records using acme-dns; acme.sh is another such client.

Note that, in this configuration, anyone on the Internet can access the API of your acme-dns instance. If the other hosts that might be using it are on your LAN, you might want to change the access property above to just green rather than red,green.