WordPress Hosting on Linux

The following is a proposal for the standardization of WordPress hosting to optimize long term maintenance and developer workflow: a simple WordPress hosting solution starting at $5 USD per month.

See the companion code repository here: wp-admin

Why Bother?

Speed: It makes design look better.™ Managed hosting platforms offer compelling solutions for high volume sites because of their advanced caching architecture, but many also suffer from poor uncached performance. The average cost starts at $25 USD.

Uncached speed is how fast the back end works, how fast developers can work, how fast the backups are restored, how fast search and sort work, how fast the inventory can be updated to match a physical store. Caching is just an aid and not a substitute for actual speed.

Our goal is to provide enough uncached speed for a low volume site to eliminate the need for complex caching solutions, at a lower price than offered by serverless providers.

While we could gain even more speed with faster hardware at Linode or LiquidWeb, the tools and documentation at Digitalocean are a better match for the web design agency this platform is intended for. We can easily, economically and sustainably achieve our speed targets using the tools and guides offered by Digitalocean.

Providers and Tools

We would like all our sites to be as fast as possible for a reasonable monthly total-cost-of-operation. The plan is to serve a basic WordPress site for $5 USD with exceptional performance and negligible maintenance fees.

Digitalocean

Digitalocean's one-click applications for LEMP and phpmyadmin on Ubuntu 16.04 are the ideal starting point for this configuration.

This provider has many easy but powerful tools including: a DNS manager, automated backups, deployment scripts to setup droplets fast, and a new monitoring feature which allows for the easy setup of alerts for system resource usage. More info.

  • LEMP stands for: Linux, Engine-X (nginx), Mysql, Php

Nginx

Nginx is a widely known and used web server that excels at handling DDOS attacks with limited system resources, and as such is my choice over apache. Each web site has its own droplet and a staging site configured in parallel, running on the live domain at port 2001. More info.

Digitalocean's tutorial on setting up WordPress on an nginx server is our starting point -> link

Standard nginx configuration includes self-renewing free https/ssl with LetsEncrypt. All unsecured requests are forwarded to the https server.

MySQL

The standard open source relational database, preferred at Digitalocean. For any site that needs a little extra power, we will create a second $5 droplet for a database server.

Networking from web servers to the db server is done on an unmetered LAN, and the database itself is not accessible to the internet. One db server provides for all web servers on the same account.

Phpmyadmin

Useful for anyone not comfortable using a SQL query directly. Digitalocean has a one-click application to deploy phpmyadmin, a user-friendly and popular tool for managing MySQL databases. More info.

Gitlab

Private git repo hosting is free at gitlab.com.

WP-CLI

The WP-CLI tool is chosen primarily because it has no impact on the performance of each site, yet offers an efficient way to manage all sites from a single location.

Digitalocean has a nice introductory guide to WP-CLI here.

These scripts use wp-cli to provide simple centralized WordPress management: wp-admin

Git-Auto-Deploy

git-auto-deploy will push code from the master branch directly to production sites, whenever it's updated.

As git-auto-deploy is the owner of the webroot, the webserver may not write changes, and therefore in-browser updates and plugin installs are disabled. dev has write permission to the web root, and manages updates using wp-cli, followed by a push back to master. See Maintenance.

WP-SendGrid-SMTP

The smallest plugin (~200kb) for email that I have found so far. SendGrid has been reliable for me thus far: https://wordpress.org/plugins-wp/wp-sendgrid-smtp/

Database-Sync

This lightweight plug-in lets you easily sync database is between any two WordPress sites. https://wordpress.org/plugins/database-sync/

CloudFlare

CloudFlare gets a reluctant inclusion here because we depend on them to handle DDOS attacks for at least one client. Hopefully we can reduce our dependence on CloudFlare by switching to nginx from apache.

Server Configuration

Every server is built the latest Ubuntu LTS release, 16.04 at the time of writing. 64-bit variants are preferred for scalability reasons. Four gigabyte swap files are standard.

User Accounts

root login is disabled. Everyone logs in as dev.

The dev user has sudo access and may therefore change the passwords for any user (including root) if needed.

Security

The ssh daemon is configured to use a non-standard port in /etc/ssh/sshd_config and root login is disabled in that file as well.

The database server opens an ssh port to the world, and then allows traffic to any port from each of the webservers, blocking all other incoming traffic. This is how you configure the ufw firewall for the db server:

sudo su  
ufw default deny incoming  
ufw allow ssh  
ufw allow from 10.10.10.10 [webserver LAN IP]  
ufw enable  

Each web server opens ports for http and https traffic in addition to ssh. The mu-plugin for fail2ban. to help the firewall see attacks on Wordpress.

Each Ubuntu server is configured to automatically install security updates. Distribution updates and repository cleanup are handled along with WP updates.

Scalability

Our basic $5 droplet takes advantage of the high speed SSD hard drives at Digitalocean to extend its memory using a swap file. While it is fast, if the swap is being used too much then it is time to upgrade.

When a site exceeds the capacity of a $5 droplet, the database should be broken out onto its own droplet. Use the one-click phpmyadmin application from Digitalocean to deploy, and change the wp-config to point to the new server using the LAN IP address. You now have twice the power and memory.

The next point of scalability is to upsize one of your droplets. By upgrading one or both to $10/month you can dramatically improve the site performance. $20 droplets have 2 processors each, and a $40 budget would host a lot of simultaneous visitors.

Web server scalability can work two ways. First of all you can give each web server more resources by scaling from $5 to $10 each. Very soon it becomes more economical to use a load balancer to distribute traffic between multiple web servers. Load balancers are a managed service available at digitalocean starting at $20 (USD) a month.

The final tier of scalability is to create a dedicated file server (or block storage service) for wp-content/uploads. This helps to free up IO resources on the web servers enabling them each to handle more traffic.

Some sites with expected bursts of traffic will require advanced caching architecture that we will not cover here. CloudFlare offers some convenient caching services.

Developer Workflow

Developers clone the repo from gitlab.com and merge to the staging or master branches which immediately deploys their code to https://www.clientsite.com:2001 and https://www.clientsite.com respectively.

Code changes must not be made directly on the production server (via terminal or ftp) except for plugin or core updates by the system administrator. The dev user has a generic git account for this purpose, so that commits originating on the production server will be easily identifiable.

The Database-Sync plugin offers a simple GUI for cloning and downloading the db.

Create a New Site

The web server droplets are configured with a script when they are created to set everything up. This script will automatically:

  • Update the server software
  • Create user dev with pre-shared key
  • Add wp-admin users to /home/dev/authorized_keys
  • Harden system, setup firewall
  • Configure nginx for WordPress
  • Set up file and folder ownership
  • Download and config WP
  • Install WP mu-plugins
  • Disable in-browser updates/plugins
  • Setup https with LetsEncrypt
  • Clone repo of maintenance shell scripts
  • Setup git-Auto-Deploy
  • Create mysql user and db
  • Initial WP configuration

The ssh keys will be rotated regularly, as each one provides access to all servers. TODO: script to rotate keys, kitestrings api/webhook?

After deploying the web droplet with cloud-config, you can go directly to the WordPress installation screen at https://yourdomain.com.

A work-in-progress cloud-config script:

#cloud-config
users:  
  - name: dev
    ssh-authorized-keys:
      - [ PUBLIC KEY GOES HERE ]
    groups: admin
    shell: /bin/bash
runcmd:  
  - sed -i -e '/^Port/s/^.*$/Port 9999/' /etc/ssh/sshd_config
  - sed -i -e '/^PermitRootLogin/s/^.*$/PermitRootLogin no/' /etc/ssh/sshd_config
  - sed -i -e '$aAllowUsers dev' /etc/ssh/sshd_config
  - service ssh restart
  - ufw default incoming disabled
  - ufw allow 9999/tcp
  - ufw allow http
  - ufw allow https
  - ufw --force enable
  - apt-get update
  - apt-get upgrade -y
  - apt-get install git php-curl php-gd php-mbstring php-mcrypt php-xml php-xmlrpc glances vim software-properties-common
  - add-apt-repository ppa:olipo186/git-auto-deploy
  - apt-get update
  - apt-get install git-auto-deploy
  - gpasswd -a dev git-auto-deploy
  - gpasswd -a www-data git-auto-deploy
  - mkdir -p /var/www/html
  - mkdir /var/www/staging
  - chown -R git-auto-deploy:git-auto-deploy /var/www
  - chown dev:dev /var/www
  - chmod g+w -R /var/www

Maintenance

Maintenance will be handled via the terminal from a central WordPress management server ($5 droplet at Digitalocean). This server will contain maintenance logs and database backups for all sites. The same configuration will run on a Mac, (possibly Windows 10 with bash) and offers the developers some powerful tools and shortcuts.

These scripts, with examples and instructions are available here: wp-admin.

Updates

Critical core updates automatically installed by WordPress itself are disabled because the web server does not have write access to its own codebase. We can update the core and plugins using WP-CLI, or by merging updated code to master.

TODO:
* test for update failure * how to roll back a failed update * automatically commit and push to master after updates

Clone Database

Most developers will prefer to use the db-sync plugin for this.

To copy the live database to the staging environment and update all URLs with the staging site port, there is a simple script that uses WP-CLI.

Backups

The codebase is backed up each time it is changed in the repository. The upload directory is backed up 4 times each month along with the whole droplet, by Digitalocean. More frequent droplet backups are best achieved with a second droplet rsync'd to the live one ever hour or so. [TODO: add that code here]

Droplet snapshots should be manually taken on a regular basis just in case more than a month passes without someone noticing that a site has been hacked and all the automated backups have been overwritten.

The database backups are written to the management server once daily​ and kept for a week, using wp-admin.

TODO: add section on restoring backups, downloading backups