Up until today (April 20, 2025) I’ve hosted this WordPress site using a variety of services and software. This site originally started out on Google Cloud Platform but then slowly transitioned to self-hosting via Raspberry Pi 4. For reasons that I do not remember, when migrating over to rpi4 I set up the site and all its components directly in the host server, without using any containerization software such as Docker. Retroactively speaking, I likely did it to brush up on my linux knowledge and make sure that I understood everything that was happening under the hood of my server and website. I’ve created and documented my journey through the various migrations in this site but finally realized recently that it is time to move on to using new tools and services. To “catch up with the times” or so people say. Now that I spent enough time building code repos from source, manually orchestrating the deployments and upgrades of WordPress and MariaDB, and manually updating PHP versions by hand, I will now leave that to the community and embrace the wonderfulness that is waiting for new software versions to arrive in docker containers 😉

About Caddy and FrankenPHP

Apache2 was (and likely still is) the go-to webserver for WordPress deployments. For WordPress users, transitioning from the traditional Apache2 setup to a more modern and efficient architecture can yield significant benefits. Caddy, with its V2 released in 2020, is a new webserver that, though young, promises high performance with an easier to use configuration. FrankenPHP is a cutting-edge PHP application server built on the Caddy web server. It consolidates the web server and PHP runtime into a single service, eliminating the need for separate PHP-FPM and Apache processes. This integration simplifies deployment and enhances performance. Notably, benchmarks have shown that FrankenPHP can process requests significantly faster than traditional setups .​

Current System Setup

• WordPress v6.7
• Apache/2.4.62
• FPM-PHP (PHP v8.2.28)
• MariaDB 10.5.26

Prerequisites

• Backup the site using your WP site migration tool of choice (I used UpdraftPlus’s Free Version)
• Backup your MySQL/MariaDB as well – if you decide to upgrade your DB version then you may potentially come across compatibility issues, and your database tables may need to be repaired in order for it to be useable
• Docker and Docker-Compose. Below is the version I’m using at the time of writing this post:
  Docker version 28.0.4, build b8034c0
Docker Compose version v2.34.0
• You will need to retrieve all of the username/passwords for the following
  • WordPress DB name
  • WordPress DB username/password
  • MariaDB root password

How to Migrate

Note: There will be site downtime while you perform this migration using this manual.

Creating the Docker Compose Configuration File

I will be using FrankenPHP’s officially (yet not official at the same time…) sponsored WordPress images created from the FrankenWP project. The list of latest images can be found here. For MySQL/MariaDB, choose the version that is closest if not equal to the version you are currently using – if you perform too big of a version jump the built-in upgrade feature may not work for your existing data. Lastly, if you use PHPMyAdmin, also grab and configure the details for that. Long story short, Caddy nor FrankenWP official pages had anything close to a work-able compose.yaml, and a lot of trial-and-error and extra configurations were necessary in order to at least properly serve the site in Production.

Below is the compose.yaml file I finally ended up with that made the whole thing work, though I’m still giving most credit to FrankenWP’s creator since I took the template from the project’s examples section. Note that the below example has sample data (denoted by ##) which will need to be replaced with info from your current WP site and DB info:

services:
  wordpress:
    image: wpeverywhere/frankenwp:latest-php8.3
    restart: always
    ports:
      - "80:80" # HTTP
      - "443:443" # HTTPS
    environment:
      SERVER_NAME: ${SERVER_NAME:-:80}
      WORDPRESS_DB_HOST: ${DB_HOST:-db}
      WORDPRESS_DB_USER: ${DB_USER:-wordpress}
      WORDPRESS_DB_PASSWORD: ${DB_PASSWORD:-##YOUR_WP_DB_PASSWORD##}
      WORDPRESS_DB_NAME: ${DB_NAME:-wordpress}
      # Currently WP_DEBUG turns on even if below is set false. To turn debug off, remove it entirely.
      # See:  https://github.com/docker-library/wordpress/issues/496
      #WORDPRESS_DEBUG: false
      WORDPRESS_TABLE_PREFIX: ${DB_TABLE_PREFIX:-wp_}
      CACHE_LOC: ${CACHE_LOC:-/var/www/html/wp-content/cache}
      TTL: ${TTL:-80000}
      PURGE_PATH: ${PURGE_PATH:-/__cache/purge}
      PURGE_KEY: ${PURGE_KEY:-}
      BYPASS_HOME: ${BYPASS_HOME:-false}
      BYPASS_PATH_PREFIXES: ${BYPASS_PATH_PREFIXES:-/wp-admin,/wp-content,/wp-includes,/wp-json,/feed}
      CACHE_RESPONSE_CODES: ${CACHE_RESPONSE_CODES:-000}
        # Enable debug to check for Caddy issues (Certificate renewal error, config error, etc)
      #CADDY_GLOBAL_OPTIONS: |
        #email ##EMAIL_ADDRESS##
        #auto_https disable_redirects
        #debug
      WORDPRESS_CONFIG_EXTRA: |
        # Add extra configs here as necessary
        define('WP_SITEURL', '##SITEURL##');
        define('WP_HOME', '##WP_HOME##');

    volumes:
      # Mount your WordPress content directory if needed
      - /var/www/html/wp-content:/var/www/html/wp-content
      # Load your own Caddyfile to configure site
      - ./Caddyfile:/etc/caddy/Caddyfile
      # Store SSL Certs so that it can be reused when containers are recreated
      # (and do avoid exceeding cert issue rate limits when debugging containers)
      - ./caddy_data:/data
      # Other root files, such as ads.txt
      - /var/www/html/ads.txt:/var/www/html/ads.txt

    depends_on:
      - db
    tty: true
    networks:
      wpnet:
        ipv4_address: 172.28.0.3

    # Below only if necessary
    #extra_hosts:
    #  - ##TLD##:172.28.0.3
    #  - www.##TLD##:172.28.0.3

  db:
    # Be careful of making too big of a version jump...
    # Slowly update db version to reduce chance of data corruption
    image: mariadb:11.7.2
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_DATABASE: ${DB_NAME:-wordpress}
      MYSQL_USER: ${DB_USER:-wordpress}
      MYSQL_PASSWORD: ${DB_PASSWORD:-##YOUR_WP_DB_PASSWORD##}
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD:-##YOUR_ROOT_PASSWORD##}
    volumes:
      - /var/lib/mysql:/var/lib/mysql
      - /etc/mysql/mariadb.conf.d:/etc/mysql/mariadb.conf.d
    networks:
      wpnet:
        ipv4_address: 172.28.0.4

  phpmyadmin:
    # using arm64 version since I am on raspberry pi
    image: arm64v8/phpmyadmin:latest
    restart: always
    ports:
      - ${LOCAL_PHPMYADMIN_PORT:-8086}:80
    environment:
      PMA_HOST: db
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD:-##YOUR_ROOT_PASSWORD##}
    depends_on:
      - db
    networks:
      wpnet:
        ipv4_address: 172.28.0.5
    volumes:
      # Use existing phpmyadmin data
      - /var/www/html/phpmyadmin:/var/www/html/phpmyadmin

networks:
  wpnet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16
          gateway: 172.28.0.1

Mix and match whatever you need from the above to get your own site working. Below is a list of some configurations which can be made differently based on your unique situation:

• Remove PHPMyAdmin if you don’t use it.
• Above config uses a bridge network. This allows for multiple sites to be hosted on the same host device via reverse proxy. Also, as you will see later, if your PHPMyAdmin also uses standard HTTP/S ports then it is not possible to run both your site (WordPress) and PHPMyAdmin on the same port in the host network configuration.
• Static addresses can be used to grant read/write privileges for the DB which is hosted in another instance and by default only allows for localhost connection. Static IP addresses can be omitted if you configure your MySQL/MariaDB to allow connections from the entire IP address range in your bridge network (example further below). It can also be omitted if you decide to host your containers using the host network configuration.
• Static IP address of PHPMyAdmin is also used in the below Caddyfile to set up reverse proxy. There’s probably a better way to do this without static IP addresses… will update later…

Creating the Caddyfile

Now that our docker configurations are set up, we need to configure Caddyfile. This Caddyfile should be placed in the same folder as your compose.yaml:

{
        # Uncomment below when experiencing redirect loops from reverse proxy/cloud load balancer/etc
        #auto_https disable_redirects
        {$CADDY_GLOBAL_OPTIONS}

        frankenphp {
                #worker /path/to/your/worker.php
                {$FRANKENPHP_CONFIG}
        }

        # https://caddyserver.com/docs/caddyfile/directives#sorting-algorithm
        order php_server before file_server
        order php before file_server
        order wp_cache before rewrite
        order request_header before wp_cache
}

{$CADDY_EXTRA_CONFIG}

##WWW_DOMAIN## {

        tls ##EMAIL_ADDRESS##

        @phpmyadmin path /phpmyadmin*
        reverse_proxy @phpmyadmin 172.28.0.5:80

        @static {
                file
                path *.ico *.css *.js *.gif *.jpg *.jpeg *.png *.svg *.woff *.txt
        }

        root * /var/www/html/
        encode br zstd gzip

        wp_cache {
                loc {$CACHE_LOC:/var/www/html/wp-content/cache}
                cache_response_codes {$CACHE_RESPONSE_CODES:2XX,404,405}
                ttl {$TTL:6000}
                purge_path {$PURGE_PATH:/__cache/purge}
                purge_key {$PURGE_KEY}
                bypass_home {$BYPASS_HOME:false}
                bypass_path_prefixes {$BYPASS_PATH_PREFIXES:/wp-admin,/wp-json}
        }

        {$CADDY_SERVER_EXTRA_DIRECTIVES}
        php_server
}

# For www redirects
###DOMAIN## {
#        redir https://www.{host}{uri} permanent
#}

Replace the ##ITEM## with configurations for your own site. At the very bottom is a redirection bit in case you want www to be redirected to your TLD+1, or vice versa. The PHPMyAdmin reverse proxy is also configured inside this Caddyfile.

Execute the Migration

Now it’s time to make the transition. Bring your webserver and DB down from your host server and start your docker configurations. If at any point in this step something goes wrong, you can revert to your existing host server setup by reversing the commands below (and in the reverse execution order):

# FYI if you do not stop Apache, you will get the following error:
#Error response from daemon: failed to set up container networking: driver failed programming external
#connectivity on endpoint frankenphp-wordpress-1
#(931a4e79170c448d3aca76617cc7be9847f227138f70a4ef085efea09e19eeb3): failed to bind host port for
#0.0.0.0:80:172.18.0.2:80/tcp: address already in use

# Also need to stop MariaDB, or else you may get corrupt data from multiple live services at the same time.
sudo systemctl stop apache2
sudo systemctl stop mariadb
docker compose up

Once the instances have started up fully, let’s go and make the change in MariaDB to allow both wordpress and phpmyadmin instance access. Below settings incorporate some wildcards in the docker network which can set both wp and phpmyadmin at the same time, but if you don’t use phpmyadmin you can just limit to the WP ip address only:

docker exec -it ##DIR##-db-1 bash
mariadb -u root -p
# enter your root password

# replace each of the data with the items you configured in the compose.yaml
# 172.28.%.% is a wildcard match to all IP addresses in your docker bridge network.
GRANT ALL PRIVILEGES ON ##WORDPRESS_DB##.* TO '##WORDPRESS_USER##'@'172.28.%.%' IDENTIFIED BY '##WORDPRESS_DB_PASSWORD##';
FLUSH PRIVILEGES;

Now your WordPress and PHPMyAdmin should be able to access your DB normally. However, you may need to wait a few minutes for Caddy to use LetsEncrypt to fetch a new certificate for you. You can use docker logs ##DIR##-db-1 -f to live tail your wordpress log to check on SSH certification creation/renewal progress. After a while you should be able to access your site from the public as normal.

Other Notes

Under the above setup, if you use Site Kit by Google, it will need to be reconfigured every time you create a new docker container for wp. This means there’s data used by Site Kit which resides outside of the wp-content directory which we bind to. Will update this guide once I figure out what else is necessary to bind to the docker containers.

Additional Troubleshooting

PHPMyAdmin – mysqli_real_connect(): (HY000/2002): No such file or directory

Change $cfg['Servers'][$i]['host'] value to 127.0.0.1

Unable to Access MySQL or MariaDB

In /etc/mysql/mariadb.conf.d/50-server.cnf, change bind-address to 0.0.0.0

One of Google Cloud’s less-known features is its Cloud CDN which leverages Google’s global network of data centers, servers, and ISP integration to deliver content to the end user in a quick manner. Its efficiency is further increased when coupled or used with other services in Google and Google Cloud, such as Google Domains, Google DNS, and Google Compute Instances. Since 2018, this site is hosted using Google’s Click-to-Deploy WordPress deployment (tutorial) using Google Domains (tutorial) with SSL encryption (tutorial) and maintenance via SSH to instance VMs (such as updating PHP version in Google Cloud VM Instance). Thus it is natural to leverage Google Cloud CDN, with the added benefit that it won’t break with updates to WordPress since the CDN configuration happens server-side and not WordPress-side.

Follow along with me as we explore how to activate Google CDN for your Google Cloud-hosted service.

Assumptions

This guide assumes the following:

• Your website uses Google Cloud VM instance(s) in an unmanaged instance group.
• You have access to Google Cloud Console and SSH privileges into your VM instance(s).
• Your website is HTTPS with a valid SSL Certificate.
• Access to DNS settings (whether Google Cloud DNS or otherwise).

Before We Begin

Your site should experience almost no downtime (with the exception of resolving DNS to the new load balancer global IP) if everything is configured and works correctly. This guide piggybacks off of Google Cloud’s official documentation on Setting up Cloud CDN with a managed instance group, but there are quite a few changes we need to make to align with the assumptions above, namely:

• In addition to HTTP/80, configuration for HTTPS/443 is also required.
• We are applying CDN to an UNmanaged instance group.
• Slightly different Health Check procedure.

Don’t worry, there will be reminders throughout this guide if we deviate from the official tutorial from Google Cloud.

Setup

Add Health Check Tag to Instance VM

The health check and firewall rules need to know which instances to apply the health check to, which can be accomplished easily by applying tags to the instance VM. Add allow-health-check as a tag to all instance VMs for your service:

Instance Group Settings

If not done so already, you will need to add your VM instances into an instance group. This tutorial assumes you already have a VM instance NOT created via an instance template and therefore can only be added into an UNmanaged instance group.

One extra setting you will need to do is to add named ports for HTTP/80 and HTTPS/443 to the instance group. Follow the section here, but don’t forget to also add HTTPS/443 as well.

Set Up Firewall

Follow the official guide section here to configure firewall rules for your health check. Note that our health check will use only HTTP/80 and so it is NOT necessary to add HTTPS/443. You are free to modify the name of the firewall rule but keep all other values (IP ranges and tag name) the same.

Set Up Load Balancer

Google Cloud CDN operates alongside a load balancer with forwarding rule applied to a global IP address. All three settings are initialized and configured at the same time. This is the hardest part of the tutorial, but don’t fret! I will walk you through the steps as detailed as I can.

Create a Health Check Page

The default settings for a health check as indicated in the official guide has a Request path of “/” which generally points to the main page of your website. The main page contains many content and is overkill to be used for a simple ping, so it is highly advised to create a separate page with a few words of text to minimize the overhead. We can actually use the sample code in Step 11 here to create a simple page. SSH into your instance VM and execute the following code:

vm_hostname="$(curl -H "Metadata-Flavor:Google" \
http://169.254.169.254/computeMetadata/v1/instance/name)"
echo "Page served from: $vm_hostname" | \
sudo tee /var/www/html/healthcheck.html

This will create a page called “healthcheck.html” on your root site directory containing the name of the VM instance. Note this page name as you will use it when configuring health check in just a moment.

Retrieve SSL Certificate Public and Private Keys

Because we are allowing HTTPS connection through the load balancer, at some point in the below guide you will be asked to provide the SSL certificate for your website. If you have configured your SSL certificate via Certbot (the same way I have as outlined in my other post), then SSH into your VM instance (using a separate browser tab) and execute the following commands to retrieve your public and private SSL certificate keys:

sudo certbot certificates

The output should give you something similar to the following, in which you can just view its contents to find all the certificate keys you need:

# Replace ##DOMAIN## with whatever is output in the above command
sudo cat /etc/letsencrypt/live/##DOMAIN##/fullchain.pem
sudo cat /etc/letsencrypt/live/##DOMAIN##/privkey.pem

Hold onto these outputs as you will need to copy-paste them later.

Configure the Backend Service

Follow the “Setting up the load balancer” section up until you finish Step 6 in its entirety. However, note that step 6c should be changed to HTTPS, 6d to “https“, and 6f to “443“. Refer to below figure for an example.

Check “Enable Cloud CDN” so that the system will automatically configure the Cloud CDN for you. For Health Check, since we haven’t created one yet, go ahead and follow the prompt to create new. Change the protocol to HTTP and use port 80 (remember our health check uses HTTP/80 and not HTTPS). Most importantly, the “Request path” should be configured to the health check page you created above (/healthcheck.html), relative to the root of the site directory.

Return to the backend settings and finish the rest of the configuration.

Configure the Frontend Service

Step 8 of the official guide assumes we already have an external IP address reserved, but I left it out until now because we can configure it here. You will need to configure TWO protocols – one for HTTP and HTTPS. When needing to specify the IP address, select “Create IP address” to create one on-the-spot.

For configuring the forwarding rule for HTTPS, a few additional steps are required, namely providing the SSL certificate keys. For certificate section, click on “Create a new certificate” to reach the UI below.

Here is where you take the outputs from “Retrieve SSL Certificate Public and Private Keys” section above and paste them here. Note that your “fullchain.pem” should actually contain TWO certificate sections, the first one should be copy pasted into the “Public key certificate” field and the second one into the “Certificate chain” field. “privkey.pem” only has one key which you can just copy the entire file contents and paste into “Private key”.

Below is a sample frontend configuration for HTTPS.

Follow the rest of the official guide until you finish setting up the load balancer.

Testing Load Balancer Configurations

Take several moments break for the load balancer to finish initialization. It should look like the below figure once finished. Note that the green check mark indicates the backend is configured correctly and that the health check is working as intended. If it is NOT green then revisit the above steps (and also using the official guide as reference) to diagnose the issue. For more concrete evidence, following “Testing traffic sent to your instance” section should direct you to the homepage of your website (ignore what step 5 says).

Modify DNS Settings

If the above load balancer test is successful, it is time to change the DNS settings “A” record from your previous single VM instance IP address to the load balancer global IP address. The global IP address can be found by clicking on the name of your load balancer in the load balancer settings page (Refer to Figure 3-1).

Enter your hostname into DNS Checker and verify that the global DNS servers return your load balancer’s IP address as the result. It may take a few moments for propagation. Once propagated, do a final check by accessing your site using your hostname (recommended to use incognito or private browser to test).

Congratulations! Your site is now configured for Google Cloud DNS! However, there are a few additional checks to make sure Google Cloud DNS is actually cacheing your website resources…

Test Caching Settings

Google Cloud DNS actually has a few rules on what resources can be cached, which can be found in detail here. After understanding the rules, use a caching checker such as this one, fill in your site, and look at the results. Your cached resources are listed in the “Cached Files” section, but there is a catch. Google Cloud DNS does NOT cache resources when cached via ETAG. Alternatively speaking, if you see a “Type: Cache-Control” tag with another tag specifying the expiry duration for a resources hosted on your domain, then that means your resource is properly cached using a CDN.

If you find your domain resources are not being cached, then check your WordPress Cache Settings and verify that caching settings are turned on. If the problem is still not resolved then follow the server-side guide here as well. For this site the WordPress cache settings were fine and my VM instance just did not have the mod_expires module enabled.

Automation is my hobby. Admittedly I never thought that I would be interested in it when I studied computer science and engineering back in my university days (my father said my personality befits a software engineer, and I blindly followed his advice). However, during my journey to become a competent software developer I invested much time in simple scripts to do repetitive work for me. While most of the above examples involve automating games (via something like Sikuli which can control your screen), lately I have been turning my attention to scraping websites and integrating APIs for data aggregation. This post contains some of the tips and tricks I have used in order to make your web crawler quicker and more robust.

Before We Start – Requests vs Webdrivers

Many who write programs to crawl webpages often use Webdrivers such as Selenium to retrieve HTML contents. However, it can be a bit of an overkill if the pages you want to crawl are more static. For webpages with little to no dynamic content, using simple CURL libraries (such as Requests for python) for GET/POSTS will be sufficient for your needs.

Webdrivers are recommended for times when it is infeasible to purely use CURL for driving dynamic sites and content. This is especially true for users who aren’t scraping for information but merely wish to automate a series of actions on a website.

The remaining sections below contain advice for both CURL-based and Webdriver-based solutions. Parts that can be better explained with examples will contain sample code for reference – though a word of caution: Browsers and Webdriver API specs change frequently so they are not guaranteed to work in the future.

Tips and Advices

Python Libraries for Parsing HTML Documents

My goto libraries for processing HTML documents are a combination of html5lib, lxml, and xml.etree.ElementTree.

The following code’s “buildHtmlTree()” retrieves the HTML document from a URL and converts it into a tree ready for element traversal:

import xml.etree.ElementTree
from lxml import etree
from lxml import html
import html5lib
import requests

class RequestTooManyTimesException(Exception):
    pass

class Html5Scraper(object):
    def __init__(self):
        self.current_url = ""
        self.session = requests.Session()

    def buildHtmlTree(self, url, encoding='UTF-8'):
        self.current_url = url
        html_elements = self.parseHtmlContent(self.getResponseText(url, encoding))
        html_tree = etree.ElementTree(html_elements)
        return html_elements

    def parseHtmlContent(self, content_string):
        # html5lib supports tags that are not closed, so parse twice to
        # convert from html5lib's xml.etree.ElementTree to lxml's
        # lxml.html.HtmlElement.
        html5_etree = html5lib.parse(content_string)
        html5_string = xml.etree.ElementTree.tostring(html5_etree, encoding='UTF-8')
        return html.fromstring(html5_string, parser=html.HTMLParser(encoding='UTF-8'))

    def getResponseText(self, url, encoding='UTF-8'):
        response = None
        retries = 0
        while not response:
            try:
                response = self.session.get(url)
            except requests.exceptions.SSLError:
                retries += 1
                if retries > 10:
                    raise RequestTooManyTimesException("Max connection exceeded when accessing: %s" % url)
                time.sleep(60)
        response.encoding = encoding
        return response.text

Once you have the HTML document parsed into a tree, finding elements on the page becomes simple as follows, using xpath:

scraper = Html5Scraper()
tree = scraper.buildHtmlTree('https://www.google.com')
element = tree.xpath("//div[@id='mngb']")

[WebDrivers] Run in Headless or Disable Images

PhantomJSDriver is a selenium module which will emulate a headless (ie no Window) browser. It is particular useful in cases where only a terminal can be used (and no screen emulator software is installed). Because it is headless there is no overhead with regards to graphics and loading images, so unless you are doing UI testing and need to take screenshots of what webpages look like, it is not likely that you absolutely need to have a visual browser.

However, that said, PhantomJSDriver is not without its faults. This particular webdriver is not of the IE/FireFox/Chrome type, and therefore there may be compatibility issues when visiting some sites. When PhantomJSDriver couldn’t parse a website correctly or otherwise just doesn’t do the trick, more common browsers such as Chrome (via ChromeDriver) may handle that site well. Chrome is non-headless, but you may still turn a few things off – image loading included – to increase its performance. Below is a sample Python code of how to set up Chromedriver without image loading, plus a few other settings turned off:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
preferences = {"profile.managed_default_content_settings.images": 2}
chrome_options.add_experimental_option("prefs", preferences)
chrome_options.add_argument("--ignore-certificate-errors");
chrome_options.add_argument("--disable-logging");
chrome_options.add_argument("--no-proxy-server");
chrome_options.add_argument("--disable-client-side-phishing-detection");
chrome_options.add_argument("--disable-sync");
chrome_options.add_argument("--disable-component-update");
chrome_options.add_argument("--disable-default-apps");
chrome_options.add_argument("--disable-infobars");
chrome_options.add_argument("--disable-web-security");
chrome_options.add_argument("--safebrowsing-disable-auto-update");
driver = webdriver.Chrome(chrome_options=chrome_options)

Refer to this page for information about each of the above arguments.

Search For Elements in ID -> Name -> Class -> CSS/XPath Order

A well-structured website should have most of its important elements tagged with a unique ID – thus it is most optimal to scan an element using its ID, followed by its name, followed by class, and lastly its other attributes. The further away from ID you search for an element the more brittle your program gets, and can lead to much time debugging whether the element you picked is what you are actually looking for. In addition, even a small change in the website UI can cause your program to fail to find the correct element.

Assuming “driver” is the object holding your webdriver, elements may be searched using the following sample methods:

driver.find_element_by_id("id")
driver.find_element_by_name("user")
driver.find_element_by_class_name("class")
driver.find_element_by_css_selector("#root > div > div.css")
driver.find_element_by_xpath("//div[@style='color: red']")

Refer to this page for information about locating elements. There are other methods to search for elements and I encourage you to learn and figure out which method is best for you. Also, there are separate methods for selecting multiple elements fitting a certain criteria – this is useful for parsing information from tabular structures such as tables and lists.

Don’t Click if You can “GET”

Webdriver users may get into the habit of clicking a link to move between pages. The advantage of clicking is that one does not have to worry about how to build the GET or POST request, which can save a lot of time when wanting to create a quick-and-easy solution. However, clicking an element increases risk and overhead because changes to the element may cause the program to fail to find it, and there may be unnecessary Javascript or other tasks the browser may run upon clicking a link.

If it is the simple matter of moving to another page and not having to worry about anything else, then GET-ing the URL is quicker and less prone to error. For POST, an inspection of the network when submitting the form can give you an idea of what to include in the form parameters, as illustrated in the below figure:

requests.session() to Store Cookies

Python’s requests API, in its simplest use case, can be thought of as a wrapper for CURL. Each call of requests.get() simply retrieves the source of the HTML document but ignores any session variables and cookies in the response. Fortunately requests also allows for sessions which can be used for exactly this purpose! The following is a simpler code of what was provided in the “Python Libraries for Parsing HTML Documents” to illustrate the simplicity of how sessions work – this is especially useful when parsing sites which require users to login:

import requests

session = requests.Session()
r = session.get('https://www.google.com')

There are many blogs – especially the article from WordPress themselves – which go into details about why updating PHP is important. Since I would be spitting out the exact same details I will not write about it here. However, pages I have visited would simply ask WordPress administrators to contact their webmaster/sysadmins to update the PHP version, which will not fly for an amateur such as myself. In addition, building from source often results in faster performance, so I took on that challenge as well. To serve as future reference, I have created a manual in the process of updating this site from PHP7.0 to PHP7.4.5. Continue on for steps on how to do it yourself!

Environment

This article uses the following environment from the Google Cloud Platform:

• Debian 9 (Stretch)
• WordPress 5.4.1 (/var/www/html/)
• Apache 2.4.25 (/etc/apache2)
• PHP 7.0.33 –> PHP 7.4.5 (/etc/php/)
• phpmyadmin 4.6.6 (just FYI)

Make sure to check the above and make the necessary changes to directories or versioning before beginning the below manual.

Before You Start

DISCLAIMER

I can NOT guarantee there are no issues with this manual!

It is imperative that you make all of the necessary backups to both WordPress and the server before you begin (Even though the previous version of PHP will not be removed from the server). For WordPress database and settings backup I use the free UpdraftPlus (which saved this site for me, as I have noted in my other post). For the VM part, if using GCP, follow the manual here to create a disk snapshot you can use to restore if things go kaput. The last section in this post, “Rollback”, details the steps to switch back to the previous version of PHP in the Apache environment without re-loading the disk – in case somebody needs to look at it in the future.

Server RAM Requirements

Because we are building PHP from scratch there needs to be sufficient RAM in your server for “make”-ing the source:

Some website sources mention that at least 2GB of RAM is necessary to compile and build PHP 7.4.5, so do make sure that your resources are secured at least for the “Build/Install PHP7.4.5 From Source” of this guide. For those who have less traffic or otherwise normally doesn’t require so much resource, it is alright to increase the RAM temporarily and then revert to original settings once upgrading of PHP is complete.

Manual

Updating Repositories

Open up terminal or SSH into your WordPress server. Type the following to get a list of currently installed PHP extensions:

php -m

Note all of the extensions in the output – you will need to install them on the fresh version of PHP later. Some extensions – especially mysql and mbscript, are absolutely required to operate WordPress and phpmyadmin.

Once all noted down, proceed with setting up repositories and installing some initial packages:

sudo apt install apt-transport-https lsb-release
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg # Download the signing key
sudo sh -c 'echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list'
sudo apt update
sudo apt install -y build-essential pkg-config libxml2-dev sqlite3 libsqlite3-dev libapache2-mod-php7.4

Build/Install PHP7.4.5 From Source

Proceed with downloading and extracting the new PHP version from the official site.

wget https://www.php.net/distributions/php-7.4.5.tar.gz
tar xf php-7.4.5.tar.gz
cd php-7.4.5
./configure
make
sudo make install

The “make” command above will take some time, so grab a coffee or stretch for 5-10 minutes as a break! When everything above is finished invoking PHP from command line should now show the new version:

php -v

Install PHP Extensions

Remember when I asked you to note all of the extensions in the previous PHP? Now is the time to install them in your new PHP version. Some extensions may have either been removed or combined with each other into a new extension – the prompt will tell you so if you ever come into this situation. The command containing the extensions I installed is as follows:

sudo apt install -y php7.4-common php7.4-xml php7.4-mysql php7.4-pdo php7.4-phar php7.4-simplexml php7.4-curl php7.4-mbstring php7.4-imagick php7.4-zip php7.4-gd

Activating New PHP Version in Apache

Since you didn’t remove the old PHP, Apache is still using the old PHP version. Update and restart Apache configurations with the following code:

sudo a2dismod php7.0
sudo a2enmod php7.4
sudo service apache2 restart

… And done! Your WordPress should now be using the new PHP version, and can be confirmed by going to Admin Settings -> Tools -> Site Health -> Info.

Rollback

Rollback is simple – it is just the “Activating New PHP Version in Apache” section’s code with the two PHP versions switched:

sudo a2dismod php7.4
sudo a2enmod php7.0
sudo service apache2 restart

Did you notice this fade-in??

Many of the UI we see today, whether it’s on TV, slideshow presentations, mobile apps and many other kinds of visuals, all incorporate some form of transition to keep the user engaged. A few days ago I was designing a mock e-commerce website for a local business and spent quite some time stuck on how to implement this fade-in effect. While there are a lot of ways to incorporate fade-in effects and there are many webpages showing you how to do so, I’ve found that there are no articles which matched the exact effect I was looking for, and thus I’m putting it here for the sake of documentation and knowledge sharing.

Read on to learn more about how to incorporate fade-in of elements the moment it is partly visible in the viewport.

Why Incorporate Transition Effects??

Before we go into the actual implementation details, readers may be inclined to understand why fade-in and other transition effects are important in keeping users engaged. The following are some of my observations.

As a form of guidance

Have you ever sat in a meeting with slideshow presentations and the presenter changes into the next slide with so much information scattered around that you have no idea where to look?? It happens very often for me and if the presenter does not use a laser pointer device it would require much more of my attention to simultaneously listen to the presenter and follow the material.

Incorporating some transition effects throughout the UI serves to prevent the user from being distracted by other content and keeps them focused on the current topic.

More natural feel

Flipping pages of a book, swiping right on a smartphone app, scrolling down a webpage… all of these smooth movements ease and prepare the user for new content. Everything in the perceivable real world takes time to execute – nothing switches from 0 to 1 in an instant. While incorporating transition effects DO have impact on performance (Windows’s UI is a prime example… disabling Windows animations reduces CPU load and enables a faster UI), sacrificing some performance or resources to engage the visitor is a small price worth paying.

Better than static

The official Google Pixel 4 product page is a prime, one-of-a-kind example (although possibly overkill for most) of different effects to wow the user. I would never have imagined such a webpage could exist, and the amount of effort that went into developing this page must have been substantial. In the context of adding transitions to a webpage or other form of UI, I strongly believe in a form of the Pareto Principle, where twenty percent of your effort equates to eighty percent of the result. In other words, there is no need to spent so much energy to make your webpage look extravagant like Google Pixel’s or Apple IPhone’s product page, but small transition elements definitely do breathe life into your UI.

Implementing Fade-In On Scroll Into Viewport

CSS

Let’s start with the CSS:

The following code initiates settings for a class and subclass to perform the fade-in effect:

.fade-in {
  opacity: 0;
  transform: translateY(20vh);
  visibility: hidden;
  transition: opacity 0.6s ease-out, transform 1.2s ease-out;
  will-change: opacity, visibility;
}
.fade-in.is-visible {
  opacity: 1;
  transform: none;
  visibility: visible;
}

The combination of 0 opacity and hidden visibility causes all elements with class “fade-in” to appear invisible by default. The transform: translateY(20vh) provides the upwards movement associated with an element fading into view. Those who don’t want the upwards movement can remove the transform parts in the class. Lastly, the transition line sets the parameters for how fast the fade-in should occur – feel free to adjust and experiment accordingly.

Next, the “is-visible” subclass provides the mechanism for the dynamic fade-in effect. In essence, when conditions for the animation are met the “is-visible” classname is appended to element with the “fade-in” class which will then cause the animation to begin. Adding “is-visible” to an element withOUT the “fade-in” class will do nothing. The appending of the subclass is done via JQuery, thus every event compatible with JQuery can be used to trigger the appending of “is-visible” (scrolling, button click, element change, etc).

JavaScript/JQuery

The next part deals with the JQuery. Here’s the code:

<script>
$(window).scroll(function() {
    $('.fade-in').each(function() {
        var top_of_element = $(this).offset().top;
        var bottom_of_element = $(this).offset().top + $(this).outerHeight();
        var bottom_of_screen = $(window).scrollTop() + $(window).innerHeight();
        var top_of_screen = $(window).scrollTop();

        if ((bottom_of_screen > top_of_element) && (top_of_screen < bottom_of_element) && !$(this).hasClass('is-visible')) {
            $(this).addClass('is-visible');
        }
    });
});
</script>

At the top is where the “event” is designated. In this situation we are detecting all scrolling of the page, but it can be modified to another event such as button click or detecting changes in some other element. Next, the $(‘.fade-in’).each(function() { checks for all elements with the “fade-in” class and runs the code afterwards accordingly. This is especially important to control fade-in effects for individual elements, because without this code all elements will fade-in if a single one is partially visible in the viewport.

The rest of the code is based off of a wonderful StackOverflow answer in which the element position and viewport dimensions are scanned and then compared to see if the element exists in the viewport. A !$(this).hasClass(‘is-visible’) is added as an additional condition to prevent the element from fading in twice.

This jQuery code can be added anywhere in the code, but do make sure that the jQuery library has already been imported and that the code itself has finished loading before the user can scroll. In a faster, optimized website this shouldn’t be such an issue, but otherwise it is suggested to place the code at the end of the “head” element.

HTML

The HTML part is easy – just add the “fade-in” class to all elements you wish to animate. For example, the following shows the class for the top paragraph of this webpage which has the fade-in animation:

Note that the JQuery code triggers for EACH element with the fade-in class, thus performance may be significantly impacted for a huge amount of elements. Consider grouping elements under a single parent element and setting the class for that top-level element to simulate fade-in effect for multiple elements at the same time.

Note to WordPress Users

Careful of the jQuery code!

For WordPress users, note that “$” is substituted for “jQuery” because WordPress runs jQuery in safe mode and therefore uses a different syntax to avoid conflict with other potential libraries which also use “$”. Refer to the below code block for an updated version for WordPress users:

<script>
jQuery(window).scroll(function() {
    jQuery('.fade-in').each(function() {
        var top_of_element = jQuery(this).offset().top;
        var bottom_of_element = jQuery(this).offset().top + jQuery(this).outerHeight();
        var bottom_of_screen = jQuery(window).scrollTop() + jQuery(window).innerHeight();
        var top_of_screen = jQuery(window).scrollTop();

        if ((bottom_of_screen > top_of_element) && (top_of_screen < bottom_of_element) && !jQuery(this).hasClass('is-visible')) {
            jQuery(this).addClass('is-visible');
        }
    });
});
</script>

Make Settings via the WordPress UI

For the CSS portion of the code, all WordPress themes have a UI setting for “Additional CSS”. If you don’t feel like fiddling with the theme code, setting it via the UI is absolutely painless:

Next is the jQuery code. While there is the option to paste the code via the Theme code editor, there is a separate WordPress add-on called Header and Footer Scripts which allows you to set header and footer code via the admin console (Settings -> Header and Footer Scripts):

Lastly, for the HTML itself, the new “Block” interface of WordPress posts has an advanced section for Additional CSS classes. Just throw the “fade-in” class to that setting as shown below:

Update 2023-02-11 : Added an additional check (modify “apache2.conf” file) at the end of the article in the case the existing method doesn’t work.

As I was reviving this site after its two-year hiatus, I noticed that I neglected to post an article detailing how to set up WordPress permalinks. Since it requires some modifications on the server side it took a bit of trial-and-error to re-enable it. If even I could forget how to make the settings after two years, then it is definitely reason enough to make an article out of it (for my own sake!).

Read on to learn more about why and how to set WordPress permalinks.

As this article serves as more of an appendix to the previous wordpress articles, I will NOT be linking this article to previous WordPress tutorials (for now… in the future I’m considering to link articles by keywords/hashtags). However, I will provide the previous tutorials for how to get a WordPress site up-and-running using Google Cloud below, so do check them out if you’re interested:

Why Permalink?

There are many pages scattered around the internet explaining why permalinks are so important – almost to the point where it should be necessary – so I will try not to repeat what others are saying and add my own spin to the importance of permalinks.

Everyone who has initialized a WordPress site knows that the default page linking is via the “p” query parameter which points to the row number in the WordPress database where the article exists. There are a number of problems with the second half of the previous sentence (other than the grammar issues… I know!), but below are some reasons why:

“?p=123”

What does the query parameter “p” stand for? “Page”? “Post”? “Parameter”? What significance does it have with the contents of the page (By the way, the answer is “None Whatsoever”).

This parameter naming is quite ambiguous and doesn’t leave any impact to the user. Nowadays browser bookmarks or URL autofill technologies are also searchable by Page Title which diminishes the issue, but a sole “p” parameter doesn’t tell the reader anything about the category nor the contents of the page. 

Query Parameters Are Growing Weaker

Query parameters are becoming the subject of growing debate over the past few years. Some sites pass tokens or keys as query parameters which cause bloating and cluttering of the URL. SEO performance also takes a hit when you are using the default permalink structure WordPress provides (“p” parameter above). Apple’s mission to protect Safari users’ privacy included a release which strips query parameters when navigating from a site flagged by ITP’s model used for detecting whether the site tracks users in an invasive manner. Lastly, analytics tools such as Google Analytics may strip query parameters in order to make reports cleaner – this means that if you control your page content via query parameters such as “p”, then they may all be analyzed and clumped together as one single entity.

URL Path vs Query Parameters

There is a common best practice (at least for developing REST APIs) that the URL should contain the resource to be accessed while the query parameter be used to filter its results. If you consider the WordPress database as a collection of articles, then it would seem alright to use query parameters to filter the relevant articles. However, WordPress by default has no easy way to implement this since, AFAIK, there are no other parameters other than “p” for filtering posts from the WordPress database. Thus, as an alternative, it would make more sense to use URL paths structure the URL to something like “www.example.com/post/category/topic“.

Setting Up WordPress Permalinks

In the case of GCP’s WordPress Click-To-Deploy feature, WordPress is hosted using Apache, and therefore Apache needs to be reconfigured to allow rules for WordPress permalinks. The guide below breaks down the process into two parts – server and client (WordPress Admin) side. But before we get started, this post is made as of April 24, 2020, with the GCP Compute VM instantiated the day before (April 23, 2020). Thus the VM will have the following configurations, and you will need to adjust accordingly for any future changes in VM configuration:

• Debian 9.11 (“Stretch”)
• Apache 2.4.25
• WordPress 5.2.3

Server-Side : Modify Apache Configurations

SSH into your VM instance.
The following screenshots are all taken from GCP’s SSH interface.

For GCP’s click-to-deploy instance, there should be a “wordpress.conf” in apache2 subdirectory. Check that it exists with the following command:

ls -al /etc/apache2/sites-available/wordpress.conf

If it exists, then this is the file you will need to modify. Go ahead and open it up with vim (or whatever editor you prefer):

sudo vim /etc/apache2/sites-available/wordpress.conf

If it the above DOESN’T exist, it’s okay, the above command will create a new file for you so just paste in the “final result” section just down below and save.

You will need to add the following parameters under the “Directory” section:

Options FollowSymLinks
AllowOverride All

Your final result for the file contents should look something like as follows:

<Directory /var/www/html>
  Options FollowSymLinks
  Options -Indexes
  AllowOverride All
</Directory>

Save and exit (using “:x” for vim). Afterwards, run the following command to restart Apache:

sudo service apache2 restart

WordPress Admin : Change Permalink Settings

Admin settings are very easy: head on over to the Permalinks settings using the following link (replace hostname with the hostname of your site):

${{HOSTNAME}}/wp-admin/options-permalink.php

I prefer to use “Post name” as it is the most simple and gets the point across. Whatever change you decide to make, make sure to hit “Save Changes” at the bottom to reflect the changes.

If it doesn’t work, check apache2.conf

Some folks may have another setup or use updated versions of some software to run their WordPress site. If the above guide doesn’t work, then you may be able to fix it by modifying the /etc/apache2/apache2.conf configuration. If this file exists, change the AllowOverride setting inside of the “<Directory /var/www/>” to AllowOverride All. Below is what I have for my apache2.conf:

<Directory /var/www/>
  AddType application/x-httpd-php .htm
  Options Indexes FollowSymLinks
  AllowOverride All
  Require all granted
</Directory>

Done!

Your permalinks settings should now be active and all of your pages should now reflect its permalinked version. There are multiple ways to test whether it’s working, such as simply clicking on a post on your website, or checking “preview mode” of a post you are currently editing. If it is not to your liking, you can always just change it at any time via the Permalinks setting. However, a word of warning: changing your permalink will require all your pages to be re-indexed, and as a consequence your SEO performance will fall considerably. It is wise to consider and set your permalink preferences in stone so that you will not have to change it in the future and risk your site performance.

Wow, it’s been a while!

While neglecting to publish some new posts due to being busy with work, I also overlooked the fact that after the GCP trial period ends, there is only a 30-day grace period for you to update to non-free status before they TERMINATE your compute VM. This means that while your deployment history and GCP project will still be kept, everything else – including the things you installed/changed on your VM – will be wiped.

Lately I have been speaking to some friends and relatives about the Covid-19 situation and whether there is anything I can do to help my family or the local community. Especially in these times when there is little freedom for people to go outside and enjoy brick-and-mortar establishments, it suddenly became ever-so-important to have an online presence in order to maintain connection to your customer base. Many companies’ research concludes that the average user is spending MUCH more time shopping online, playing online games, and watching online videos, which is a given.

In my resolve to offer my technical and digital assistance to potential local businesses I thought back to the time when I wrote a full tutorial on how to build a self-hosted WordPress site using your own secure domain. However, upon my disappointment I found that my website was no longer operational and I had not taken the time to make a backup image of my VM before GCP pulled the plug.

When all seemed lost, yesterday I looked back on my Google Drive folders and noticed an UpdraftPlus directory inconspicuously sitting there, and it suddenly clicked to me that I actually installed and setup a WordPress backup add-on which saved all my posts, media, and WordPress admin settings. My entire site was saved!! All it took was three hours of my time to spin up a GCP instance with WordPress already deployed, access the admin console, re-install UpdraftPlus, and feed in the backup files in order to restore all the posts and settings I made two years prior. It was a excruciatingly painless task, and I have the developers at UpdraftPlus to thank for creating such a great FREE plugin which saved me several weeks of tinkering and fiddling with GCP + WordPress settings and writing articles.

For those who are reading this and are WordPress site owners, I strongly urge you to give UpdraftPlus a chance. While this site is still in its infancy and modest at best, I do plan to monetarily support UpdraftPlus in the future if this site takes off in viewer count or in the event that I will be building sites for local businesses. 

If you’re seeing this post then you must have been referred to here by me personally, clicked on a link from my social accounts, or got really lucky and came here from organic search engine results (just kidding, you are a good samaritan and it had nothing to do with luck, but it might have some to do with my SEO skills 🙂 ).

This post is an introduction to this site and outlines some of the goals and motivations of this site. Personally, this post serves as a guideline for me so that I don’t stray from the original plan for this blog. You can read my profile in a separate About page, though I will likely go into small tangents about my personal life in the posts I will write. If you went and came back from that page you would notice that I am, in fact, a Taiwanese who grew up in the States and now work in Tokyo. While I can speak, read, and write in three languages, English is my strongest point (and, consequently, a strong selling point as a talent here in Japan), and thus posts in this blog will be largely written in English. While there may be times in which I will post in Japanese or Mandarin Chinese, my grasp of these two languages are not strong enough to produce a prose in which readers will find interesting – and to be honest, I wasn’t necessarily the top of my class in the English department, neither, so I hope to become more poetic at English first before engaging in others.

At the time of this post, I live in a somewhat small city in Tokyo called Jiyuugaoka (自由が丘) of the Meguro ward district. It is a place relatively known for youngsters and lavish dessert shops, but is also an important place for train transfers along the Toyoko and Oimachi lines. While not cheap, it is a very convenient area to live in and therefore many expats choose to live in or around this area. The name of this blog, “JiyuuLife,” bears semblance to the area I call home now. “Jiyuu,” or “自由” in Japanese and Mandarin Chinese, translates to “freedom,” which to me symbolizes the beginning of my journey to financial and spiritual freedom – roughly a month before the inception of this site I had quit my previous job as a technical solutions consultant in the finance industry to move back to the internet services industry which I currently work in now. Since this company also has housing assistance, I also opted in to move to a bigger and more vibrant area. But while I only moved two stations south (I have been to Jiyuugaoka many times before in my life) the day I decided on the apartment I will move into I was overcome with a sense of liberation and the determination that I will better myself, for myself, by finally breaking free of the financial and mental chains holding me back from being my best. To me, this blog serves as both a tracker – so that I not deviate from my path to personal greatness – but also a memento of such journey, so that in the years or decades to come I can come back to this public diary and reminisce about the wonderful memories I made.

Night picture of rotary outside of Jiyuugaoka Station

Alright, now let’s get on with some administrative topics. Since I’m new to blogging, I will take this time to try out some formatting features. I promise, I’ll get better at deciding what to use for which situations in the future 🙂

Motivation

In addition to self-development and the vague goals I outlined above, the following are what I hope to explore and try out in this blog to gain a better understanding of how it works and become more professional at:

Advertising – Advertisement and marketing are major topics in the world today and it is said that no businesses can get by without some form of marketing. While I do apologize for displaying advertisements in this site, it is a necessary evil as I am hosting, publishing, and maintaining everything by myself. The main costs for maintaining this blog are the hosting fees (variable depending on site traffic) and well as domain hosting fees, and though it doesn’t amount to much in a year-on-year basis, I do hope to generate enough royalty for this site to be self-sustainable. In this way I can continue to provide more DIY and how-to guides for everyone to follow along in the future.

Web Design – I’ve never been great at web design as I’ve always handled the backend in all the salaried positions I engaged in. Fortunately, self-hosting WP comes with many unique templates for me to use as a base, and adding small things on top of it is much simpler than starting from scratch. Expect this blog to become more and more lifelike as time goes by, and I hope it becomes beautiful enough that other sources will refer to this place as an example of a great blog.

System Administration – Load balancing, seamlessly adding/subtracting virtual machines, error logging and monitoring… the list goes on. There are many ways to approach it and is utterly overwhelming as a result. While there are many tools which can be leveraged to assist in backend management, the orchestration and decisions ultimately come down to human intelligence. While not necessary now, it will be a delightful burden for me to have the opportunity to incorporate many of these concepts in the future.

Linguistics – As stated above, I’m not the best at expressing myself in any language. I hope to fix that.

Exploring New Concepts – This place serves as a log of sorts for me to remember what I have tried – what worked and what didn’t, how I could improve next time, et cetera. As engineers we tend to have do things in a methodical and scientific manner, and keeping track of our processes is a time-consuming but necessary part of organizing our achievements.

Below is a bullet point list, just because 🙂

Posts will be mainly centered around:

• Data aggregation
• DIY
• Shop/Item Reviews
• Other Know-Hows

Aaaand, that’s it! I assume if you made it this far then you must either be very bored or very infatuated with me. Regardless of which, while this blog is not specific nor niche in content, I hope that every viewer will be able to take something away from here and use for their own self-betterment. It is an exciting time for me to start my own blogging “service,” and I do hope I have the strength and persistence to nurture this site for the years and decades to come. Thank you, and thank you in advance for your continuing support.