Lessons Learned: A Browser Test That Found a Live Outage

How a browser setting turned into an incident

I set out to change my default browser and ended up diagnosing a live outage on this blog. The whole thing started, ran, and was resolved on September 29, 2026.

My Debian desktop had quietly made Zen its default browser, and I wanted Microsoft Edge. Two commands fixed that:

xdg-settings set default-web-browser microsoft-edge.desktop
xdg-mime default microsoft-edge.desktop text/html

To prove the change worked, I opened a link from the terminal with xdg-open. The link I happened to pick was my own blog. Edge opened it correctly, and showed a bare Apache page: Internal Server Error.

The browser change had nothing to do with it. The test just happened to point at a site that was already broken. Without that test, I might have found out from a reader.

Narrowing it down to PHP

Three curl requests showed the web server was fine and PHP was failing. curl skips the browser cache, cookies, and extensions, so it shows what any visitor gets.

curl -sI https://ny2honolulu.com | head -n 1              # HTTP/2 500
curl -sI https://ny2honolulu.com/wp-login.php | head -n 1 # HTTP/2 500
curl -sI https://ny2honolulu.com/license.txt | head -n 1  # HTTP/2 200

Request	Runs PHP?	Result	What it told me
Homepage	Yes	500	The site is down for everyone, not just my browser
Login page	Yes	500	WordPress core fails too, not just the theme
license.txt	No	200	Apache and .htaccess work; PHP is the problem

The license.txt request was the control. It is a plain text file that ships with WordPress, so Apache serves it without running PHP. A broken .htaccess would have failed it too.

Two more clues pointed the same way. The error was Apache’s bare default page, not WordPress’s own “critical error” page, so the failure happened before WordPress started. And HTTP/2 in the reply meant TLS and the connection were fine, which ruled out a repeat of an earlier HTTPS outage I wrote about.

One small trap: the .htaccess I first inspected was a local copy from July, not the live file. ls -l showed its date. Always check the timestamp on a file before drawing conclusions from it.

The SSH warning I did not click past

When I connected over SFTP to read the server logs, SSH refused with its loudest warning:

@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Host key verification failed.

The warning offers a one-line command to delete the old key and move on. That is exactly what an attacker in the middle would want, so I verified first.

Nothing sensitive had leaked. SSH checks the server’s identity before it asks for a password, so the connection stopped before my password was ever typed.

There were two possible explanations: the host legitimately changed servers, or something was intercepting my connection. The checks I ran:

1. Recorded both fingerprints. ssh-keygen -lf ~/.ssh/known_hosts printed the saved key; ssh-keyscan fetched the one the server presents now. Both were 4096-bit RSA keys.
2. Checked DNS. dig +short returned the same IP address my hosting dashboard lists, so traffic was reaching the real server.
3. Asked the host. Support confirmed they had migrated my account to a cloud server, which replaced the host key.
4. Accepted the new key only when it matched. After removing the old entry with ssh-keygen -R, I typed yes only because the fingerprint shown matched the one I had recorded.

One event explained both problems at once: a server migration changes the SSH key and can break a site’s PHP setup. A WordPress-level compromise could not have changed the host’s SSH key, which also pointed away from an attack.

A tip I learned: at the “Are you sure you want to continue connecting” prompt, you can paste the expected fingerprint instead of typing yes. SSH then does the comparison for you.

Working with support: evidence first

Leading with test results skipped the usual “clear your cache” script. My first two chat messages were the symptom and the evidence: every PHP page returns 500, a static file returns 200, DNS matches the dashboard, and .htaccess has no PHP handler lines. The agent went straight to the server.

The chat ran about 26 minutes, starting at 11:30 a.m. Honolulu time. The case was escalated to a senior technical team with a 24 to 48 hour estimate. The site was actually back that same afternoon.

The emailed transcript showed something I missed live. Asked to confirm the new host key fingerprint, the agent wrote that it was correct, then two minutes later wrote that they could not confirm it. The same chat also said my SFTP hostname had changed, then gave me the exact hostname I had always used.

Neither was bad faith; front-line agents often cannot see host keys. But it is a clear lesson: a support agent’s “yes” is not verification. My own checks were what made the new key trustworthy.

Two small practical notes. Ask for a case number at the start, and ask for the transcript by email. And after the chat closed, an automated sales pitch appeared; a quick “yes” to it would have routed me to a sales rep, not support.

Auditing what the fix left behind

The site was back, but the fix had changed two files, and one change exposed debug output to the public. An ls -la in the site root showed .htaccess and wp-config.php were both modified at 21:40 and 21:41 UTC, during my chat while I was on hold. That timing matched support’s work, not an intruder’s.

The .htaccess diff. I compared the live file with my July copy:

diff .htaccess htaccess-today.txt

Every changed line began with < and there were no > lines, so lines were only removed, never added. An attacker usually adds something, like a hidden redirect. Two meaningful lines were gone: a Content-Security-Policy: upgrade-insecure-requests header and the rewrite rule that passes HTTP_AUTHORIZATION through to WordPress.

The wp-config.php settings. I listed only the settings, with secrets filtered out:

grep -n "define" wp-config.php | grep -viE "PASSWORD|KEY|SALT"

Three lines were new: WP_DEBUG, WP_DEBUG_LOG, and WP_DEBUG_DISPLAY, all set to true. Display on means PHP warnings print on public pages, revealing server paths and plugin details. Log on writes them to wp-content/debug.log, and that URL returned HTTP 200 to anyone who asked.

The fix. I backed up the file, then changed only those three lines and checked the syntax before uploading:

cp wp-config.php wp-config-backup.php
sed -i '69,71s/true/false/' wp-config.php
php -l wp-config.php

After uploading, the site still returned 200 and the debug log URL returned 404, the same as a control URL for a file that never existed.

My own mistake. While editing, I pasted an editor screen that included my WordPress security keys and salts into a chat. Those keys sign login cookies, so I replaced them with fresh ones from WordPress’s official generator at api.wordpress.org, swapping the eight lines with sed and verifying by line counts rather than printing them. Rotating keys logs every session out, which is the point.

Old copies as evidence. I found two July copies of wp-config.php in my home folder. Before deleting them, I used them to see what existed before the migration. The http:// site address lines and DISABLE_WP_CRON were mine from July, not support’s. Only the debug lines were new.

Backups, and what is still open

My backups ran every night right up to the outage, and a locked full backup from the day before is a clean restore point. UpdraftPlus showed nightly database backups from September 23 to 28, full-site backups on the 25th and 28th, and a Google Drive copy of each.

That schedule works even with DISABLE_WP_CRON set because I run an external cron pinger that calls WordPress on a timer. The first scheduled backup on the new cloud server is the real test, so I will check that it appears.

Still open after this incident:

• Confirm the first scheduled backup completes on the new server
• Upgrade PHP from 7.4.33 (unsupported since late 2022) to 8.3, tested on my staging site first
• Ask the host whether the removed security header can be restored on the new server
• Consider tighter permissions on wp-config.php once the host confirms how PHP runs

The migration did not upgrade PHP. WordPress still warns that it will soon stop updating on this version, so this is next.

Lessons learned

1. Test with something real. A throwaway test link found a production outage.
2. Use a control. One static file separated “the server is down” from “PHP is down” in a single request.
3. Never click past a host key warning. Verify with fingerprints, DNS, and the provider before trusting a new key.
4. Bring evidence to support. Status codes and a control test get you past the script and into the fix.
5. Support’s yes is not verification. The transcript showed two contradictory answers two minutes apart.
6. Audit after someone else fixes your system. The repair left public debug output that would have stayed on.
7. Diff against a known-good copy. Old files told me which changes were theirs and which were mine.
8. Filter secrets before you share output. I learned this one the hard way and rotated my keys.

A note on redaction: hostnames, usernames, IP addresses, fingerprints, and database details are deliberately left out of this post. None of them help a reader, and all of them help an attacker.