Convert a PHP/MySQL site to Jamstack and host on Github Pages

One of the first items on my sabbatical to do list last year was to convert the Camp La Jolla Military Park website to a static Jamstack site hosted at Github Pages. This post documents that process.

I originally created the site as a research tool using PHP and a MySQL database for data entry, to record, organize, and ultimately publish research about the military history of UCSD. Moving the project to Github Pages would simplify maintenance and hosting costs, but was no small task. Technically it didn’t cost me extra to host, but the domain was $20/year, which adds up. Further, maintaining a PHP/MySQL website with a custom admin tool requires a bit of overhead for security and updates. Additionally, I wanted to continue having the benefit of using template engines or some way to include files (like PHP’s require()) to save time coding the site.

There are a plethora of tools available to publish a database-driven site on a static host. They mostly can be categorized as follows (ordered by most to least dynamic data sources):

  • Move the database to a remote service like Firebase, Atlas, et al. — ⚠️ Ultimately creates vendor lock-in, potential hidden costs, and probably more maintenance issues in the future.
  • Create a static version of the database by converting MySQL to flat files (JSON, CSV, etc.), which still allows for querying using one of various tools — 🤔 This method is interesting, because it still decouples the data from the presentation and allows for potentially more options later, including a remote database. I am already using this in various forms but worry again about maintenance, or issues with the client’s ability to manipulate the data as needed.
  • Generate a static version of the site, including all its pages which contain the data formerly dynamically inserted from MySQL. — 🤔 I like this concept, and have been using it to auto-publish my class lectures and tutorials using Markdown and Grunt with Marp. Plus, there are many site generators to choose from, which I explore below.
  • Scrape the site and save pages as static files. After all, everything is static once the HTML is rendered on the frontend. — ⚠️ While fast, this requires the site be built first and would make updates difficult.

Essential Concepts

Here I outline some concepts on rendering. See SSR vs CSR vs ISR vs SSG for a more in-depth look:

  • Client-Side Rendering (CSR) (e.g. Single-Page Application (SPA)) – JS renders pages in client’s browser with empty HTML files + raw data from the server. (e.g. React, Vue, Angular)
  • Server-Side Rendering (SSR) (e.g. Fullstack) – Generates pages on server with dynamic data before delivering to client. Improves SEO and loading, decreases server performance. (e.g. EJS, PHP, or Next.js)
  • Static Site Generation (SSG) (e.g. JAMStack) – Pre-renders all pages as static HTML during build process. Fast loading times, enhanced security, and dynamic content can still be achieved via “islands”.
  • Incremental Static Regeneration (ISR) – Combines SSR and SSG to pre-render static pages during build time while periodically regenerating specific pages with updated data.
  • Edge-Side Rendering (ESR) – Rendering process is pushed to the CDN’s edge servers which handles requests and generates HTML. Requires Nitro engine.

Requirements

Requirements for most of the projects I mention above.

  1. Javascript-based
  2. The data will be static, but editable using Git
  3. The data should be searchable using a tool like chosen
  4. The site design will be simple, but I should have creative control
  5. The site will be public, and be presented mostly using the same design as the original site
  6. HTML/CSS validation + Accessibility checks
  7. Convert from Google Maps to Leaflet
  8. Host the site on Github Pages

While planning the project I also wanted to select a method I could use for other potential static (or nearly static) sites like:

  1. A new site documenting the hundreds of examples and overlapping categories of Critical Web Design and internet art I’ve been collecting for an upcoming book. Update: I built it with SvelteKit
  2. Converting tallysavestheinternet.com to a static site (currently SSR) and unlinking the extension from the server to use local data only
  3. Moving away from WordPress more generally (and its security problems) and blogging with Markdown instead, starting with this blog (NO MORE TYPING IN A WEB FORM!!)

It’s nice to learn new tools. I already have some experiences with React, Next.js, and Vue.js, but wouldn’t say I’m confident in any. I found that publishing apps using React Native is very frustrating and unreliable, and my general distrust of Facebook makes me lean towards Vue. Still, I went in with with an open attitude.

Tool Comparison

A breakdown of some tools I considered. Some of the icons are clickable…

Framework SPA SSR SSG MD Themes Search Cons
Jekyll ? ❌ Ruby
Hugo ? ❌ Go
Astro React, Vue, Svelte in dynamic islands, simple routing
Vue.js ? ✅ vuepress
Nuxt.js ? Built on Vue
SvelteKit MD rendering not native but supported via MDX
Next.js MD rendering not native but supported via MDX

Next

My experience with Next.js SSG was painful.

  • Constantly running into subtle changes in syntax across versions
  • Adding an image is ridiculously difficult.
  • Exporting the SSG ultimately created dead links, packaged up in Next code that was hard to understand.
  • So much automation to make a static site in the end you can’t do anything unless you have the exact right code, in the right version, in the right paths.
  • Error messages are rarely helpful or pinpoint the line number causing the issue in the stack.
  • I followed so many tutorials that ultimately ended in dead-ends, incorrect versions, or that completely didn’t work.

Astro

I ended up choosing Astro for the final project, which can be seen here https://omundy.github.io/camplajolla/. While they appear to have many similarities, I found Astro soooooo much easier than Next.js. It’s fairly simple to install, even using a starter theme, and it didn’t take long torip out tailwind to add bootstrap.

Processing.js – Missing Documentation

The Processing.js project is really cool. It allows you to run Processing .pde files inside (HTML5 compatible) web browsers using Javascript. You can pass data back and forth between the two programs, access the DOM with Processing, and you don’t need any plugins or Java.

Click here to run this Processing sketch in your browser

One caveat… When importing a Processing .pde file into the HTML5 canvas you must access the files on a web server or by using localhost (a server running on your computer, e.g. MAMP) because most (all modern?) web browsers don’t allow file:/// access for security reasons. Unfortunately this is not intuitive as Javascript should run in the browser regardless of file:/// access. Nor is it mentioned in any of the Processing.js Quick Start documentation. I found it by testing, and then confirmed it in their README. Darn, need to remember to read (all of) the instructions.

UPDATE: A friend pointed out that the problem accessing the .pde file could be due to the same origin policy. Though not explicitly stated on the Github page for Processing.js, they do mention that disabling same origin setting in your browser is a(n undesirable) workaround.

Some web browsers (e.g., Chrome) require secondary files to be loaded from a web server for security reasons. This means loading a web page that references a Processing.js sketch in a file via a file:/// URL vs. http:// will fail. You are particularly likely to run into this problem when you try to view your webpage directly from file, as this makes all relatively links file:/// links.

New yourarthere.net website is live

After 4 months the new yourarthere.net website and member-run content management system is now live. Thanks to Braylin and Brittany Morales, Beth Lee, and Chris Cumbie for all their hard work.

The site is valid XHTML/CSS and runs on PHP/MySQL using the Codeigniter framework. All the details from our research from inception onward are archived here.

This site is based around the idea that members should have control of the content on the website. Every member has a profile where they can add images, text, tags, and events to promote their artwork or group. Members can create a new profile for every domain they host with yourarthere.nets.

Picture 6

Picture 8

Fading an LED with PWM and a Potentiometer

Using a potentiometer and PWM on an Arduino to fade an LED.

  1.  
  2. /* POT to LED test -> by Owen Mundy March 11, 2010
  3.    from: http://itp.nyu.edu/physcomp/Labs/AnalogIn
  4. —————————————————————*/
  5.  
  6. int potPin = 0;    // Analog input pin that the potentiometer is attached to
  7. int potValue = 0;  // value read from the pot
  8. int led = 9;      // PWM pin that the LED is on.  n.b. PWM 0 is on digital pin 9
  9.  
  10. void setup() {
  11.   // initialize serial communications at 9600 bps:
  12.   Serial.begin(9600);
  13.   // declare the led pin as an output:
  14.   pinMode(led, OUTPUT);
  15. }
  16.  
  17. void loop() {
  18.   potValue = analogRead(potPin); // read the pot value
  19.   analogWrite(led, potValue/4);  // PWM the LED with the pot value (divided by 4 to fit in a byte)
  20.   Serial.println("hello");      // print the pot value back to the debugger pane
  21.   delay(10);                     // wait 10 milliseconds before the next loop
  22. }
  23.  

Here is the schematic for the above project.

Using PWM and a potentiometer to fade an LED and drive a stepper motor, powered by a Boarduino RBBB.

  1.  
  2. /*
  3.   Owen Mundy
  4.  July 29, 2009
  5.  
  6.  p. 262 of Physical Computing
  7.  Using BBB to run stepper motor by manually moving steppers
  8.  
  9.  */
  10.  
  11. int pin1 = 3;                 // PWM
  12. int pin2 = 5;                 // PWM
  13. int pin3 = 6;                 // PWM
  14. int pin4 = 9;                 // PWM
  15. int ledpin = 13;              // LED
  16. int led = false;              // LED monitor
  17. int motor_time_lapse = 80;
  18.  
  19. int potPin = 0;      // Analog input pin that the potentiometer is attached to
  20. int potValue = 0;    // value read from the pot
  21. int ledPotPin = 11;  // PWM pin that the LED is on.  n.b. PWM 0 is on digital pin 9
  22.  
  23.  
  24. void setup()
  25. {
  26.   pinMode(pin1, OUTPUT);      // sets the pin as output
  27.   pinMode(pin2, OUTPUT);      // sets the pin as output
  28.   pinMode(pin3, OUTPUT);      // sets the pin as output
  29.   pinMode(pin4, OUTPUT);      // sets the pin as output
  30.   pinMode(ledpin, OUTPUT);    // sets the pin as output
  31.  
  32.   // initialize serial communications at 9600 bps:
  33.   Serial.begin(9600);
  34.   // declare the led pin as an output:
  35.   pinMode(ledPotPin, OUTPUT);
  36. }
  37.  
  38. void loop()
  39. {
  40.   potValue = analogRead(potPin); // read the pot value
  41.   analogWrite(ledPotPin, potValue/4);  // PWM the LED with the pot value (divided by 4 to fit in a byte)
  42.   Serial.println(potValue);
  43.  
  44.   digitalWrite(pin1, HIGH);   // on
  45.   digitalWrite(pin2, LOW);    // off
  46.   digitalWrite(pin3, HIGH);   // on
  47.   digitalWrite(pin4, LOW);    // off
  48.   delay(motor_time_lapse);    // wait
  49.  
  50.  
  51.   digitalWrite(pin1, LOW);    // off
  52.   digitalWrite(pin2, HIGH);   // on
  53.   digitalWrite(pin3, HIGH);   // on
  54.   digitalWrite(pin4, LOW);    // off
  55.   delay(motor_time_lapse);    // wait
  56.  
  57.   digitalWrite(pin1, LOW);    // off
  58.   digitalWrite(pin2, HIGH);   // on
  59.   digitalWrite(pin3, LOW);    // off
  60.   digitalWrite(pin4, HIGH);   // on
  61.   delay(motor_time_lapse);    // wait
  62.  
  63.  
  64.   digitalWrite(pin1, HIGH);   // on
  65.   digitalWrite(pin2, LOW);    // off
  66.   digitalWrite(pin3, LOW);    // off
  67.   digitalWrite(pin4, HIGH);   // on
  68.   delay(motor_time_lapse);    // wait
  69.  
  70.   blink();
  71. }
  72.  
  73. void blink()
  74. {
  75.   if (led == false)
  76.   {
  77.     led = true;
  78.     digitalWrite(ledpin, HIGH); // on  
  79.   }
  80.   else
  81.   {
  82.     led = false;
  83.     digitalWrite(ledpin, LOW); // on  
  84.   }
  85. }