Manage Nginx sites and Cloudflare DNS with Ansible

In this article, using Ansible, we will

• Install and configure Nginx
• Install and configure Certbot for Cloudflare
• Create Nginx sites
• Create DNS records in Cloudflare
• Update our WAN IP in Cloudflare

Prerequisites

• Ansible
• An Ubuntu (or other Debian-based distro) host
• Package python3-pip installed in host
• A domain managed by Cloudflare
• A Clouflare API key

The Cloudflare API key can be generated in My Profile / API Tokens.

The following variables are required in the playbook. It is recommended to store them as secrets.

• cloudflare_email
• cloudflare_api_key
• cloudflare_zone_id
• cloudflare_domains

Installation and basic configuration

Install Nginx and Certbot libraries:

- name: install nginx
  apt:
    name:
      - nginx
      - nginx-extras
      - certbot
      - python3-certbot
      - python3-acme
      - python3-certbot-nginx
      - python3-certbot-dns-cloudflare
    state: latest
  notify:
    - restart nginx

Copy the Nginx configuration:

- name: copy nginx config
  copy:
    src: files/nginx/nginx.conf
    dest: /etc/nginx/nginx.conf
  notify:
    - restart nginx

Nginx SSL configuration won’t be created until we create a certificate, so let’s download it from the certbot repository:

- name: download nginx ssl configuration
  get_url:
    url: https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf
    dest: /etc/letsencrypt/options-ssl-nginx.conf

Here’s an example of a configuration file (more information here):

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
  worker_connections 768;
}

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;

  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
  ssl_prefer_server_ciphers on;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  gzip on;

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

Changes on the previous two tasks will trigger the following handler to reload the Nginx service:

handlers:
  - name: restart nginx
    service:
      name: nginx
      state: reloaded

Allow HTTPS traffic:

- name: allow related and established connections
  iptables:
    chain: INPUT
    ctstate: ESTABLISHED,RELATED
    jump: ACCEPT

- name: allow connections to port 443
  iptables:
    chain: INPUT
    protocol: tcp
    destination_port: '443'
    jump: ACCEPT

- name: allow port 443
  ufw:
    rule: allow
    port: '443'
    proto: tcp

Create sites

Only the enabled sites will be included in the configuration (in the previous config file: include /etc/nginx/sites-enabled/*;), but it is a good practice to define the sites in the sites-available folder and create symlinks to sites-enabled for the sites we want to include.

With the following template, we can create different site configurations. By passing domain and subdomain it will create a config file for the server_name <subdomain>.<domain>, access and error logs with the subdomain as part of the file name, and will expect an SSL certificate at /etc/letsencrypt/live/<domain>. Both logs and certificates files can be overwritten by passing extra variables. You can also pass local=yes to allow site access from local network only and define a different router IP than the default 192.168.1.1.

{% if router_ip is not defined %}
{% set router_ip = '192.168.1.1' %}
{% endif %}
{% set local = item.local is defined and item.local %}
{% if item.ssl_certificate is defined %}
{% set ssl_certificate = item.ssl_certificate %}
{% elif ssl_certificate is not defined %}
{% set ssl_certificate = '/etc/letsencrypt/live/'+item.domain+'/fullchain.pem' %}
{% endif %}
{% if item.ssl_certificate_key is defined %}
{% set ssl_certificate_key = item.ssl_certificate_key %}
{% elif ssl_certificate_key is not defined %}
{% set ssl_certificate_key = '/etc/letsencrypt/live/'+item.domain+'/privkey.pem' %}
{% endif %}
{% if item.access_log is defined %}
{% set access_log = item.access_log %}
{% elif access_log is not defined %}
{% set access_log = '/var/log/nginx/'+item.subdomain+'.access.log' %}
{% endif %}
{% if item.error_log is defined %}
{% set error_log = item.error_log %}
{% elif error_log is not defined %}
{% set error_log = '/var/log/nginx/'+item.subdomain+'.error.log' %}
{% endif %}
server {
    listen 443 ssl;
    server_name {{ item.subdomain }}.{{ item.domain }};

    ssl_certificate {{ ssl_certificate }};
    ssl_certificate_key {{ ssl_certificate_key }};
    include /etc/letsencrypt/options-ssl-nginx.conf;

    access_log {{ access_log }};
    error_log {{ error_log }};

    location / {
        {% if local %}allow {{ router_ip }};
        deny all;
        {% endif %}proxy_pass {{ item.proxy_pass }};
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_http_version 1.1;
    }
}

Using the template, let’s create a couple of sites in our domain mydomain.com: plex.mydomain.com and pihole.mydomain.com. Also, restrict pihole subdomain access to local network:

- name: create sites
  template:
    src: files/nginx/site.jinja2
    dest: "/etc/nginx/sites-available/{{ item.subdomain }}.{{ item.domain }}"
  loop:
    - { domain: mysite.com, subdomain: plex, proxy_pass: http://192.168.1.2:32400 }
    - { domain: mysite.com, subdomain: pihole, proxy_pass: http://192.168.1.3:8001, local: yes }    
  notify:
    - restart nginx

- name: enable sites
  file:
    src: "/etc/nginx/sites-available/{{ item.subdomain }}.{{ item.domain }}"
    dest: "/etc/nginx/sites-enabled/{{ item.subdomain }}.{{ item.domain }}"
    state: link
  loop:    
    - { domain: mysite.com, subdomain: plex }    
    - { domain: mysite.com, subdomain: pihole }    
  notify:
    - restart nginx

Certificates

To be able to generate Let’s Encrypt certificates using Certbot with the Cloudflare plugin we need to create a credentials file with our Cloudlfare email and API key:

- name: create secrets folder
  file:
    path: /root/.secrets
    state: directory
    mode: '0700'

- name: copy cloudflare configuration
  copy:
    dest: /root/.secrets/cloudflare.ini
    content: |
      dns_cloudflare_email = {{ cloudflare_email }}
      dns_cloudflare_api_key = {{ cloudflare_api_key }}
    mode: '0400'

Now, we can generate new certificates for *.mydomain from the host machine using the following command:

certbot certonly \
--dns-cloudflare \
--dns-cloudflare-credentials /root/.secrets/cloudflare.ini \
-d *.mydomain.com \
--preferred-challenges dns-01

Create subdomains in Cloudflare

Following the example, create a non-proxied CNAME record for pihole.mydomain.com. Since we wanted to restrict the access to this subdomain to local network, this record needs to be non-proxied by Cloudflare. For global access, it’s a better option to proxy our subdomain.

- name: create local subdomain dns records
  cloudflare_dns:
    domain: mydomain.com
    name: pihole
    type: CNAME
    content: mydomain.com
    proxied: no
    account_email: "{{ cloudflare_email }}"
    account_api_key: "{{ cloudflare_api_key }}"

Update WAN IP in Cloudflare

If we have a dynamic IP, our WAN IP will change, so we need to keep it up to date in Cloudflare. We can use the following script to do it. Here, cloudflare_domains must be a list, so it can be [ 'mydomain.com', '*.mydomain.com' ].

import requests


ZONE_ID = '{{ cloudflare_zone_id }}'
CLOUDFLARE_URL = \
    f'https://api.cloudflare.com/client/v4/zones/{ZONE_ID}/dns_records'
DOMAIN_NAMES = [{% for domain in cloudflare_domains %}'{{ domain }}',{% endfor%}]
USER_EMAIL = '{{ cloudflare_email }}'
USER_KEY = '{{ cloudflare_api_key }}'


def update_domain_ip(domain_name, ip):
    # Get Cloudflare domain information.
    headers = {'X-Auth-Email': USER_EMAIL, 'X-Auth-Key': USER_KEY}
    response = requests.get(CLOUDFLARE_URL,
                            {'type': 'A', 'name': domain_name},
                            headers=headers)
    domain = response.json()['result'][0]
    if domain['content'] == ip:
        print('Cloudflare already up to date.')
    else:
        # Update Cloudflare DNS.
        response = requests.put(f'{CLOUDFLARE_URL}/{domain["id"]}',
                                json={'content': ip,
                                      'type': 'A',
                                      'name': domain_name},
                                headers=headers)
        if response.ok:
            print('Cloudflare DNS updated')
        else:
            print(response.json()['errors'])


def main():
    # Get WAN IP.
    wan_ip = requests.get('http://icanhazip.com').text.strip()
    for domain_name in DOMAIN_NAMES:
        update_domain_ip(domain_name, wan_ip)


if __name__ == '__main__':
    main()

Now, we can schedule the execution of the script every 10 minutes.

- name: install requests
  pip:
    name: requests

- name: copy script to update domain ip in cloudflare
  template:
    src: files/nginx/update_cloudflare_dns.py.jinja2
    dest: update_cloudflare_dns.py

- name: setup cron job
  cron:
    name: "update cloudflare dns"
    minute: "0,10,20,30,40,50"
    job: "python3 {{ ansible_user_dir }}/update_cloudflare_dns.py 2>&1 | logger -t {{ ansible_user_dir }}-cron"