Constant Shift

PHP Sessions with Memcached

After setting up PHP 5.3.0 on a Debian virtual machine recently, I had some trouble implementing XCache, an opcode cacher for PHP. I’ve used XCache with Lighttpd in the past however now I am working with Nginx and using a compiled version of PHP instead of the Debian package. The issue is that the current XCache release has not been adapted to support late static binding and namespaces of PHP 5.3.0. I’ve even compiled the 2.0 beta build of XCache however this was still problematic.

I decided instead to focus on session management first and build up a caching daemon to store PHP sessions. Sessions, in many regards, would seem ideal to cache because they are transient, relatively small, and accessed frequently. I’m not overly concerned with disk I/O but I’m always interested in finding new, or better, ways to improve server performance. Enter memcached.

Memcached is a general-purpose distributed memory caching system that was originally developed by Danga Interactive for LiveJournal, but is now used by many other sites. It is often used to speed up dynamic database-driven websites by caching data and objects in memory to reduce the number of times an external data source must be read. The system is used by several very large, well-known sites including YouTube, Wikipedia, Facebook, Digg, and Twitter. For a great article on how memcached was implemented into Facebook’s architecture, check out Scaling memcached at Facebook.

So, to get started, I’ll outline how to install memcached from source and also the aptitude package manager. To install from source, use the following for reference:

$ wget http://pecl.php.net/get/memcache-3.0.4.tgz
$ tar xvfz memcache-3.0.4.tgz
$ cd memcache-3.0.4
$ phpize
$ ./configure
$ make && make test
$ make install

Note: When you install from source, it will provide the memcached extension directory, which is usually something like the following: extension_dir = “/usr/local/lib/php/extensions/no-debug-non-zts-20090626/”

If you are on a Debian based system, you can also use aptitude instead of installing from source:

$ aptitude install memcached

Start the daemon:

$ /etc/init.d/memcached start

The default port is 11211 so if you want to verify that everything has been set up properly, use netstat:

$ netstat -an | grep :11211

The default configuration for memcached was adequate in my case however if you want to modify the connection port or memory limit, you can edit the memcached.conf — /etc/memcached.conf. Now we need the memcached daemon to talk to PHP, and vice versa. For that we need the memcached PHP module. Again, this can be done with PECL or the package manager:

$ aptitude install php5-memcache

If the memcached extension hasn’t already been loaded, make sure the following is in the php.ini:

display_startup_errors = On
extension = memcache.so
memcache.hash_strategy = "consistent"
memcache.max_failover_attempts = 100
memcache.allow_failover = 1
extension_dir = "/usr/local/lib/php/extensions/no-debug-non-zts-20090626/"
session.save_handler = memcache
session.save_path = "tcp://127.0.0.1:11211?persistent=1&weight=1&timeout=1&retry_interval=15"

The extension_dir may vary depending on which version you are running so make sure the path is valid. You can also see we’ve defined the session.save_handler and session.save_path to use the memcached. Once you’ve saved the changes to the php.ini, restart your server and php if it runs as it’s own process (php-fcgi or php-fpm):

$ /etc/init.d/nginx restart && php-fpm restart

To verify that everything has been set up properly, load a phpinfo page in the document root. You should have the memcached PHP module displayed:

PHP Module

The session settings should be changed as well:

PHP Sessions

To test the new PHP session configuration, I’ve found 2 great scripts that can be used to track PHP sessions. I’d also recommend checking out the official memcached documentation as this is only a fraction of what memcached is capable of doing.

Hide a WordPress plugin version

Often times with WordPress plugins, developers will design the plugin to display the plugin name and version number within the header output. This is great for the developers to promote their work however given some of the security implications, it’s not always a great idea, especially given the nature of third-party plugins. A good example of this is the All in One SEO Pack.

Continue reading…

Virtual Hosts in Lighttpd using mod_evhost

I’ve seen a few different ways to set up virtual hosts in Lighttpd recently and tried a few different variations based on what I’ve found. The simplest solution for me was to use the mod_evhost module. The mod_evhost module essentially builds the document-root based on a pattern which contains wildcards. So, to begin, uncomment (or add if the module is not present) the mod_evhost from the server.modules section of the lighttpd.conf:

server.modules = (
...
"mod_evhost",
...
)

Then, define the virtual host base directory and path pattern. Depending if you’ve installed from source, the var.basedir may be different:

var.basedir = "/var/www"
evhost.path-pattern = var.basedir + "/%0/"

Save the lighttpd.conf and make sure that mod_evhost has been enabled:

$ lighty-enable-mod evhost

Before restarting Lighttpd, create your virtual host directories in /var/www:

$ mkdir /var/www/domain1.com
$ mkdir /var/www/domain2.com

Once you’ve added content in the domain directories for domain1.com and domain2.com, you can restart Lighttpd:

$ /etc/init.d/lighttpd restart

You should now be serving virtual hosts for all domains located within /var/www. You can do some further tweaking to set up per domain access and error logs as well. Please check the official documentation for further information.

Enable gzip compression in Nginx

As with many modern web servers, enabling gzip compression can be fairly easy. If you’re using Nginx, you can modify your nginx.conf to enable gzip compression with several different directives. Open the nginx.conf file and add the following:

gzip  on;
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

To make sure that gzip doesn’t lose bigger gzipped javascript or css files, you can set buffers as well. This will assign the number and the size of the buffers into which to store the compressed response. If unset, the size of one buffer is equal to the size of page, depending on platform this either 4K or 8K. :

gzip_buffers 16 8k;

You’ll also want to disable compression for some older browsers. To do this, we can use some regex magic to make an exception for IE6 SP1:

gzip_disable "MSIE [1-6]\.(?!.*SV1)";

One of the best features is the ability to set the compression level. The compression level can be set between 1 and 9, where 1 is the least compression (fastest) and 9 is the most (slowest). For further information on this directive and others, please check out the official Nginx module page.

Unknown user in statoverride file

When trying to install a new package dependency (flex) for Monit 5, I ran into this error:

dpkg: unknown group 'munin' in statoverride file
E: Sub-process /usr/bin/dpkg returned an error code (2)

I recognized the munin user from a previous munin install. Since I had already purged the Munin package, I simply removed all references of the munin user from the statoverride file in /var/lib/dpkg/statoverride. I also checked other munin references by using grep:

$ grep -ril 'munin' /var/lib/*

Once all Munin references were removed, I was able to install flex successfully.

Installing Monit 5 on Ubuntu 9.04

Monit is a free open source utility for managing and monitoring, processes, files, directories and filesystems on a UNIX system. Monit also conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. Since the Monit package for Ubuntu 9.04 is pretty old (4.x), we’re going to apt-get the package and then install from source. This way we can use the init scripts that come with the package. First, install the older Monit package:

$ apt-get install monit

Make sure all of the monit files are in place — /etc/default/monit and /etc/init.d/monit — and then we can remove the package:

$ apt-get remove monit

Now, the newer version of Monit has a few different dependencies that the 4.x version did not have. Although your system may be different, I needed to install bison and flex:

$ apt-get install bison flex

Continue reading…

Running php-fcgi independently from Lighttpd

As of Lighttpd 1.4.x, the spawn-fcgi binary is included in the package when you install Lighttpd. When Lighttpd starts, it will launch php-cgi processes under Lighttpd defined by the fastcgi.server value in the lighttpd.conf. The problem is that when you need to restart Lighttpd, the FastCGI processes don’t die. This also makes it difficult to monitor php-cgi as a service because it does not have a pid file. The solution is to remove spawn-fcgi and proxy all requests to a php-fastcgi daemon to keep it independent of Lighttpd altogether.

The first step was to create a php-fcgi default file, /etc/default/php-fcgi:

START=yes
EXEC_AS_USER=www-data
FCGI_HOST=localhost
FCGI_PORT=9000
PHP_FCGI_CHILDREN=2
PHP_FCGI_MAX_REQUESTS=1000

Once this file has been created, now we want to make sure that php-fcgi starts when the server starts:

$ update-rc.d php-fcgi defaults

Now, edit the lighttpd.conf to proxy all php requests to port 9000:

fastcgi.server = (
".php" => (
( "host" => "127.0.0.1",
"port" => 9000,
"max-procs" => 1,
"idle-timeout" => 15,
"broken-scriptfilename" => "enable" )))

Continue reading…

Varnish Broke My Logs

After setting up Varnish as a reverse proxy, I was really impressed with the results. A few kinks came up however after I implemented Varnish with Lighttpd. One major problem was that all remote requests logged in my log files were from 127.0.0.1. This isn’t very helpful. One way around this is to define the accesslog.format in the lighttpd.conf file so that remote IP addresses are not shown as localhost. Here is the snip from my lighttpd.conf:

accesslog.format = "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b /\"%{Referer}i\" \"%{User-Agent}i\""

Before force reloading the new configuration, test it first by running:

$ lighttpd -t -f /etc/lighttpd/lighttpd.conf

If the syntax checks out, reload Lighttpd:

$ /etc/init.d/lighttpd force-reload

Now you can check your access logs and it should be showing the IP addresses correctly:

$ tail -f /var/log/lighttpd/access.log

Varnish + Lighttpd

For a while now, I’ve wanted to implement a reverse proxy or http accelerator with Lighttpd. Two popular offerings are Squid and Varnish. After reading some of the architect notes, I decided to go with Varnish. Instead of installing from source, I used apt-get:

$ apt-get install varnish

The key is then setting up Varnish to listen on port 80 which will then proxy to your web server, in this case Lighttpd. Open up your lighttpd.conf file and change the server.port value:

server.port = 8080

Then force reload the new Lighttpd configuration:

$ /etc/init.d/lighttpd force-reload

Now you’ll have to stop varnishd, and reload it with the following command:

$ varnishd -a :80 -b 127.0.0.1:8080 -T 127.0.0.1:6082

This will proxy all requests from port 80 to 8080 which Lighttpd is now listening on. The -T flag is for the management interface. You can then manage Varnish by sending commands over this port to update the VCL file or just find cache status information. There are serveral flags you can use to define many options with Varnish; I recommend reading the man page for all the good stuff. Since I am using WordPress, I also found a great article with an example VCL file specifically for WordPress.

If you want to check if Varnish is doing it’s thing, you can take a look at the http header. You should see two obvious entries:

X-Varnish => 620759461
Via => 1.1 varnish

XCache: No space left on device

Recently, I’ve been trying to implement XCache on this server though I’ve run into a problem when trying to raise the xcache.size size in the php.ini file. On my previous server, I had this variable set to 64MB and had no problem. On this installation, I am unable to raise this above 16MB. If I do, I receive the following error:

$ php-cgi -v
/dev/zero: No space left on device
Failed creating file mappping
PHP Fatal error:  Failed creating file mappping in Unknown on line 0
PHP Fatal error:  XCache: Cannot create shm in Unknown on line 0
PHP 5.2.6-3ubuntu4.1 with Suhosin-Patch 0.9.6.2 (cgi-fcgi) (built: Apr 23 2009 14:30:49)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
with XCache v1.2.2, Copyright (c) 2005-2007, by mOo
Segmentation fault

My current xcache.count is set to 8; this is defined by the output from:

$ cat /proc/cpuinfo | grep -c processor

I am also using the 64-bit version of PHP when I call the zend extension — I have used both 32 and 64-bit versions with no luck:

zend_extension = /usr/lib64/php5/20060613/xcache.so

I am still able to use XCache at the moment with the current xcache.size however I am concerned this will break at some point. If I don’t find an obvious answer, I’ll likely end up using APC for op-code caching.

Update: After setting the xcache.mmap_path = “/tmp/xcache”, the output from php-cgi -v looked much better.