Unknown user in statoverride file

When trying to install a new package dependency (flex) for Monit 5, I ran into this error:

dpkg: unknown group 'munin' in statoverride file
E: Sub-process /usr/bin/dpkg returned an error code (2)

I recognized the munin user from a previous munin install. Since I had already purged the Munin package, I simply removed all references of the munin user from the statoverride file in /var/lib/dpkg/statoverride. I also checked other munin references by using grep:

$ grep -ril 'munin' /var/lib/*

Once all Munin references were removed, I was able to install flex successfully.

Installing Monit 5 on Ubuntu 9.04

Monit is a free open source utility for managing and monitoring, processes, files, directories and filesystems on a UNIX system. Monit also conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. Since the Monit package for Ubuntu 9.04 is pretty old (4.x), we’re going to apt-get the package and then install from source. This way we can use the init scripts that come with the package. First, install the older Monit package:

$ apt-get install monit

Make sure all of the monit files are in place — /etc/default/monit and /etc/init.d/monit — and then we can remove the package:

$ apt-get remove monit

Now, the newer version of Monit has a few different dependencies that the 4.x version did not have. Although your system may be different, I needed to install bison and flex:

$ apt-get install bison flex

Continue reading…

Running php-fcgi independently from Lighttpd

As of Lighttpd 1.4.x, the spawn-fcgi binary is included in the package when you install Lighttpd. When Lighttpd starts, it will launch php-cgi processes under Lighttpd defined by the fastcgi.server value in the lighttpd.conf. The problem is that when you need to restart Lighttpd, the FastCGI processes don’t die. This also makes it difficult to monitor php-cgi as a service because it does not have a pid file. The solution is to remove spawn-fcgi and proxy all requests to a php-fastcgi daemon to keep it independent of Lighttpd altogether.

The first step was to create a php-fcgi default file, /etc/default/php-fcgi:

START=yes
EXEC_AS_USER=www-data
FCGI_HOST=localhost
FCGI_PORT=9000
PHP_FCGI_CHILDREN=2
PHP_FCGI_MAX_REQUESTS=1000

Once this file has been created, now we want to make sure that php-fcgi starts when the server starts:

$ update-rc.d php-fcgi defaults

Now, edit the lighttpd.conf to proxy all php requests to port 9000:

fastcgi.server = (
".php" => (
( "host" => "127.0.0.1",
"port" => 9000,
"max-procs" => 1,
"idle-timeout" => 15,
"broken-scriptfilename" => "enable" )))

Continue reading…

Varnish Broke My Logs

After setting up Varnish as a reverse proxy, I was really impressed with the results. A few kinks came up however after I implemented Varnish with Lighttpd. One major problem was that all remote requests logged in my log files were from 127.0.0.1. This isn’t very helpful. One way around this is to define the accesslog.format in the lighttpd.conf file so that remote IP addresses are not shown as localhost. Here is the snip from my lighttpd.conf:

accesslog.format = "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b /\"%{Referer}i\" \"%{User-Agent}i\""

Before force reloading the new configuration, test it first by running:

$ lighttpd -t -f /etc/lighttpd/lighttpd.conf

If the syntax checks out, reload Lighttpd:

$ /etc/init.d/lighttpd force-reload

Now you can check your access logs and it should be showing the IP addresses correctly:

$ tail -f /var/log/lighttpd/access.log

Varnish + Lighttpd

For a while now, I’ve wanted to implement a reverse proxy or http accelerator with Lighttpd. Two popular offerings are Squid and Varnish. After reading some of the architect notes, I decided to go with Varnish. Instead of installing from source, I used apt-get:

$ apt-get install varnish

The key is then setting up Varnish to listen on port 80 which will then proxy to your web server, in this case Lighttpd. Open up your lighttpd.conf file and change the server.port value:

server.port = 8080

Then force reload the new Lighttpd configuration:

$ /etc/init.d/lighttpd force-reload

Now you’ll have to stop varnishd, and reload it with the following command:

$ varnishd -a :80 -b 127.0.0.1:8080 -T 127.0.0.1:6082

This will proxy all requests from port 80 to 8080 which Lighttpd is now listening on. The -T flag is for the management interface. You can then manage Varnish by sending commands over this port to update the VCL file or just find cache status information. There are serveral flags you can use to define many options with Varnish; I recommend reading the man page for all the good stuff. Since I am using WordPress, I also found a great article with an example VCL file specifically for WordPress.

If you want to check if Varnish is doing it’s thing, you can take a look at the http header. You should see two obvious entries:

X-Varnish => 620759461
Via => 1.1 varnish

XCache: No space left on device

Recently, I’ve been trying to implement XCache on this server though I’ve run into a problem when trying to raise the xcache.size size in the php.ini file. On my previous server, I had this variable set to 64MB and had no problem. On this installation, I am unable to raise this above 16MB. If I do, I receive the following error:

$ php-cgi -v
/dev/zero: No space left on device
Failed creating file mappping
PHP Fatal error:  Failed creating file mappping in Unknown on line 0
PHP Fatal error:  XCache: Cannot create shm in Unknown on line 0
PHP 5.2.6-3ubuntu4.1 with Suhosin-Patch 0.9.6.2 (cgi-fcgi) (built: Apr 23 2009 14:30:49)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
with XCache v1.2.2, Copyright (c) 2005-2007, by mOo
Segmentation fault

My current xcache.count is set to 8; this is defined by the output from:

$ cat /proc/cpuinfo | grep -c processor

I am also using the 64-bit version of PHP when I call the zend extension — I have used both 32 and 64-bit versions with no luck:

zend_extension = /usr/lib64/php5/20060613/xcache.so

I am still able to use XCache at the moment with the current xcache.size however I am concerned this will break at some point. If I don’t find an obvious answer, I’ll likely end up using APC for op-code caching.

Update: After setting the xcache.mmap_path = “/tmp/xcache”, the output from php-cgi -v looked much better.

Hide your Lighttpd version

With Lighttpd 1.5 not officially released yet, it might be worthwhile to disclose the version you’re running as spawn-fcgi is still included with the current stable version. One easy way to modify this is to change the server.tag in the lighttpd.conf file. Use the following to keep the version hidden:

server.tag = "lighttpd"

If you use a Firefox extension like Domain Details or Live HTTP Headers, you’ll notice that the version is missing after Lighttpd has been restarted.

Easy WordPress permalinks in Lighttpd

With most WordPress installations, the permalink structure is defined with some simple .htaccess mod_rewrite rules. In Lighttpd, the structure can be defined really easily in the lighttpd.conf file:

server.error-handler-404 = "/index.php"

Now reload the new configuration changes:

$ /etc/init.d/lighttpd force-reload

Use ETags, mod_expire and mod_compress with Lighttpd

If you’ve ever run YSlow to see how your site performs, you may have noticed the ETags component. Entity tags (ETags) are a mechanism that web servers and browsers use to determine whether the component in the browser’s cache matches the one on the origin server. ETags were added to provide a mechanism for validating entities that is more flexible than the last-modified date.

To set this up in Lighttpd, simply uncomment the mod_expire module (make sure this is listed before mod_compress) in the server.modules section of the lighttpd.conf file. Then down at the bottom of the file, add the following to enable ETags:

$HTTP["url"] =~ "(css|js|png|jpg|ico|gif)$" {
expire.url = ( "" => "access 7 days" )
}
etag.use-inode = "enable"
etag.use-mtime = "enable"
etag.use-size = "enable"
static-file.etags = "enable"

Then you’ll add another section to define the cache directory and file types to cache:

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ("text/plain", "text/html", "application/x-javascript", "text/css")

Create the directory defined in the compress.cache-dir value and chown the directory so Lighttpd can read and write to it (note: the owner and group may differ depending on what has been defined for the server.username and server.groupname in the lighttpd.conf:

$ mkdir /var/cache/lighttpd/compress/
$ chown www-data:www-data /var/cache/lighttpd/compress/

Save and restart Lighttpd:

$ /etc/init.d/lighttpd restart

You’ll likely want to set up a cron to empty the compress directory at a given interval depending on the needs of the application. I’ve included mine here:

00   */6 *    *   *   find /var/cache/lighttpd/compress/ -type f -mmin +120 -exec rm -rf {} \;

This cron for example will run every six (6) hours and clear out any file or directory that is more than (2) hours old. To test, simply curl an image or css file from the command line:

$ curl -I http://constantshift.com/wp-content/themes/constant-shift/style.css | grep Cache

You should see an output like this:

Cache-Control: max-age=604800

Installing phpMyAdmin for Lighttpd

To begin, you must have Lighttpd and MySQL already installed. Log in via SSH and run following from the command line:

$ apt-get install phpmyadmin

After the package manager installs phpMyAdmin, you must add an alias to the Lighttpd configuration. Open the lighttpd.conf file in /etc/lighttpd and add the following line:

alias.url += ( "/phpmyadmin" => "/usr/share/phpmyadmin/" )

Reload your new lighttpd configuration:

$ /etc/init.d/lighttpd force-reload

Continue reading…