Tag Archives: apache

WordPress & SSL Fixes

I’ve been using WordPress for this blog for a number of years now – at some point I realised that whilst writing my own code is fun, there’s no need to reinvent yet-another-fucking-blog-platform and ended up selecting WordPress to use for my content, on the basis of it’s strong and active development and community.

Generally it’s pretty good, but there are times it disappoints, such as WordPress expecting servers to have FTP for unpacking updates and plugins (it’s 2013 guys, SFTP at least!), excessively setting cookies which makes caching layers more complex and doing stupid stuff with storing full URLs inside the database for page links and image resources.

The latter has been impacting me in particular. Visitors to my site have had the option of using HTTP or HTTPS (SSL secured) access methods for some time, but annoyingly whenever I posted an article with images, WordPress includes all the images using http://. This mixed content type prevents browsers from showing the lock icon (best case) or throws up a nasty error (worst case) depending on the browser and it’s level of concern for user safety for mismatched content.

Dubious Firefox is dubious about this site.

Dubious Firefox is dubious about this site, no lock icon of security here!

Despite having accessed the site on https://, WordPress still uses http:// for my images.

Despite having accessed the site on https://, WordPress still uses http:// for my images.

I could work around this by setting the WordPress base URL for my site to be https://www.jethrocarr.com, but then images served at the unsecured http:// site would also be served via SSL, which is just adding pointless load to the server (not that SSL termination really adds much load these days, but damnit, I’m being a purist here!).

I was hoping that it was a misconfiguration of my WordPress setup, but reading online it seems that this is a known issue with WordPress and a whole bunch of modules, hacks and themes have sprung up to fix/workaround the issue…

Of course there’s an easier way – fix it at the webserver layer! Both Nginx and Apache have modules to do substitutions in page content on load, for Nginx there’s HttpSubModule and for Apache there is mod_substitute. In my case with stock Apache 2.2 on CentOS 5, I was able to fix the whole issue by adding the following to my SSL vhost configuration:

# Fix SSL URLs thanks to WordPress hardcoding http:// links to images :'(
<Location />
    AddOutputFilterByType SUBSTITUTE text/html
    Substitute "s|http://www.example.com|https://www.example.com|"
</Location>

Following this, things look much better:

The lock icon of browser approval!

The lock icon of browser approval!

All media files are now https://, not http://

All media files are now https://, not http://

Technically this substitution will have some level of performance impact, as it has to process the generated HTML content and check for strings to replace, but the impact is so low that I wasn’t able to measure it amongst the usual variation of page response times – and it’s not going to be anywhere as slow as mod_php and WordPress itself anyway. ;-)

Finally, if you haven’t already, you probably want to change the following in wp-config.php:

define('FORCE_SSL_ADMIN', true);

This forces all WordPress logins and wp-admin activities to take place under HTTPS which is a pretty good idea if you ever post to your blog from an unsecured network.

The Apache that wanted to be root

I’ve run into an issue a couple of times where some web applications on my server have broken following a restart of Apache when the application in question calls external programs..

What seems to happen is that when an administrator restarts Apache during general maintenance of that server, Apache picks up some of the unwanted environmental settings from the root user account, in particular the variable HOME ends up getting set to the home directory of the root user account (/root).

Generally it won’t be an issue for web applications, but if they call an external application (in my case, Git), that external application may use the HOME environment to try and read or write configuration files.

# tail -n1 error.log
fatal: unable to access '/root/.config/git/config': Permission denied

In my case, Git kept dying with a fatal error, which lead to a very confused sysadmin wondering why a process running as Apache is trying to read from the root user’s account…

By looking at the environmental settings for the Apache worker processes, we can see what’s happening. After a normal boot, the environmental variables look something like the below:

# ps aux | grep httpd
root     10173  0.0  1.6  27532  8496 ?        Ss   22:42   0:00 /usr/sbin/httpd
apache   10175  0.1  2.8  37560 14692 ?        S    22:42   0:01 /usr/sbin/httpd
apache   10176  0.1  2.8  37836 14952 ?        S    22:42   0:01 /usr/sbin/httpd
apache   10177  0.1  2.8  37332 14876 ?        S    22:42   0:01 /usr/sbin/httpd
apache   10178  0.1  2.8  37560 14692 ?        S    22:42   0:01 /usr/sbin/httpd

# cat /proc/10175/environ
TERM=dumbPATH=/sbin:/usr/sbin:/bin:/usr/binPWD=/LANG=CSHLVL=2_=/usr/sbin/httpd

Because Apache has been started by init, it has a nice clean environment. But after a restart by the root user, it’s clear that some cruft from the root user account has been pulled into the application environment variables:

# cat /proc/10175/environ

HOSTNAME=localhostSHELL=/bin/bashTERM=xtermHISTSIZE=1000USER=root:
MAIL=/var/spool/mail/rootPATH=/sbin:/usr/sbin:/bin:/usr/bin
INPUTRC=/etc/inputrcPWD=/rootLANG=CSHLVL=3HOME=/rootLOGNAME=root
LESSOPEN=|/usr/bin/lesspipe.sh %sG_BROKEN_FILENAMES=1_=/usr/sbin/httpd

Because of these settings, external programs relying on the value of HOME will try to read/write to a directory that they aren’t permitted to use.

Debian-based systems fix this issue by unsetting certain environmentals (including HOME) in the bootscript for Apache, based on the rules in /etc/apache2/envvars.

To fix the issue on a RHEL/CentOS host, you can instead just append a replacement HOME setting into /etc/sysconfig/httpd. This particular configuration file is read at server startup and isn’t overwritten when Apache gets upgraded.

cat >> /etc/sysconfig/httpd << "EOF"
# Correct Apache's home directory
HOME=/var/www
EOF

Following a restart, Apache should now show the correct HOME environmental variable and your application should function as expected.

Apache Redirect & Varnish

Apache’s mod_rewrite is an awesome module providing all sorts of impressive capabilities, however it’s sometimes overkill when all you want is to redirect an entire site or path to a different one. In these situations, the mod_alias’s Rewrite function is often the ideal solution, with it’s simple syntax.

Redirect permanent /  http://newhost.example.com/
Redirect permanent /test.html http://newhost.example.com/test.php

Unfortunately I’ve found a situation where certain syntax can cause failures in some environments – consider the following:

Redirect permanent /  http://newhost.example.com

This example will work happily when a user is accessing Apache directly, however it will fail horribly when accessing via a Varnish cache, eg:

$ wget oldhost.example.com
 --2012-11-27 11:26:54--  http://oldhost.example.com/
 Resolving oldhost.example.com (oldhost.example.com)... 172.16.1.1
 Connecting to oldhost.example.com (oldhost.example.com)|172.16.1.1|:80... connected.
 HTTP request sent, awaiting response... 301 Moved Permanently
 Location: unspecified
 ERROR: Redirection (301) without location.

This was the source of a lot of headaches since the site still *worked* correctly when connecting directly to it, but kept failing whenever accessed via the varnish caches.

The cause is simply a missing trailing / in the redirect – configuration should actually look like:

Redirect permanent /  http://newhost.example.com/

It’s a nasty trap for the unwary sysadmin, since it works perfectly when accessing the server directly – I’m not entirely sure whether it’s an issue with Apache allowing this syntax, or whether it’s an issue with Varnish not understanding how to proceed – maybe a bit of both.

RFC 2616 isn’t clear, it states that if the abs_path is empty, it’s the same as abs_path being equal to /, and that when requesting a URL with an undefined abs_path, it should be treated as /… but it doesn’t clarify whether an application serving a URL should or shouldn’t append the trailing /.

I guess ideally Apache should be patched to serve up URLs with a trailing / and Varnish should accept redirects with or without it to protect both applications from blame.