Nginx Syntax Highlighting For Vim

I found this little gem by Evan Miller in the Vim scripts archive a few days ago and love it so far. If you’re working with Nginx configurations, even on a casual basis, it’s nice to have syntax highlighting. Although the Vim plugin directions are fairly straightforward, I thought I’d outline the complete process here. Log into your box and running the following commands:

$ /bin/mkdir -p ~/.vim/syntax/
$ cd ~/.vim/syntax/
$ wget http://www.vim.org/scripts/download_script.php?src_id=12990
$ mv download_script.php?src_id=12990 nginx.vim
$ touch ~/.vim/filetype.vim
$ echo "au BufRead,BufNewFile /etc/nginx/* set ft=nginx" >> ~/.vim/filetype.vim

Continue reading…

Install PHP-FPM 5.3.2 on Ubuntu 10.04 (Lucid Lynx)

A few weeks ago, we went through installing PHP-FPM on Ubuntu 9.10 (Karmic Koala). Now that (mt) Media Temple began offering Ubuntu 10.04 (Lucid Lynx) on their (ve) servers, I figured it was time to see if the installation process for Lucid was the same. It isn’t but fortunately, it’s even easier.

If you’re working on a new (ve) server, we’ll only need 1 dependency met before we get started.

$ aptitude install python-software-properties

This package provides an abstraction of the used apt repositories. It allows you to easily manage your distribution and independent software vendor software sources. Since we’re going to be using a Launchpad PPA, we’ll need this tool to make adding packages easier. Personal Package Archives (PPA) allow you to upload Ubuntu source packages to be built and published as an apt repository by Launchpad. The one we’ll be using was brought to us by Brian Mercer.

$ add-apt-repository ppa:brianmercer/php

You should see some gpg output. This is normal. Now that we’ve added the new apt repository, we need to update our sources list:

$ aptitude -y update

Now, install the preferred PHP packages. These are simply examples that I frequently use. The one you’ll need for this is example is php5-fpm:

$ aptitude -y install php5-cli php5-common php5-mysql php5-suhosin php5-gd

$ aptitude -y install php5-fpm php5-cgi php-pear php5-memcache php-apc

$ service php5-fpm start

Once you’ve started the PHP daemon, make sure the service is listening on port 9000:

$ netstat -plan | grep :9000

APC Plugin For Munin

If you’re like me, you want to monitor and graph every little piece of your server. Until recently, I’ve been relying on a simple PHP script to display APC statistics. Although the tool worked well, I wanted to integrate the statistics into Munin, as most all other server properties and system services are monitoring with it. The problem was that finding a suitable APC plugin for Munin proved to be very difficult. Fortunately I stumbled across this German gem (Vielen Dank, dass Sie meine deutsche Freundin!) that met my needs. To set this up on your system, just follow the directions below:

First, let’s set up the apcinfo.php file in the document root of your web server and retrieve the file from a private gist:

$ cd /var/www/html
$ wget http://gist.github.com/raw/403135/8cbf9c71c5f82c2cda5af864c46f91910552e9f7/apcinfo.php

Now, make the file executable:

$ chmod +x apcinfo.php

The script itself is very simple. It simply ouputs the total cache memory, memory available and memory used:

<?php
$mem = apc_sma_info();
$mem_size = $mem['num_seg']*$mem['seg_size'];
$mem_avail= $mem['avail_mem'];
$mem_used = $mem_size-$mem_avail;
$out = array(
    'size: ' . sprintf("%.2f", $mem_size),
    'used: ' . sprintf("%.2f", $mem_used),
    'free: ' . sprintf("%.2f", $mem_avail)
    );

echo implode(' ', $out);

To confirm that the script is working, check out /apcinfo.php on your site. You should see something like the following:

size: 31457200.00 used: 19478104.00 free: 11979096.00

Now, let’s set up the Munin plugin that will use that information:

$ cd /usr/share/munin/plugins
$ wget http://gist.github.com/raw/403127/51c7bf078d2ffd5b70da8b2380881648b3cc3481/apc
$ chmod +x apc
$ ln -s /usr/share/munin/plugins/apc /etc/munin/plugins/apc

What that does is create the apc plugin, makes it executable and then symlinks it to the proper directory. Now, we’ll need to make sure Munin loads the new plugin. To do this, we’ll edit the /etc/munin/plugin-conf.d/munin-node to add the following:

[apc*]
user root
env.url http://example.com/apcinfo.php

Obviously, you’ll want to replace example.com with the host you intend to use. Once the file has been updated and saved, restart Munin:

$ /etc/init.d/munin-node restart

Once the node is running, it’ll take a few minutes for Munin to update so don’t worry if you don’t see any new graphs right away. One thing to keep in mind is that you will need the LWP::UserAgent Perl module. If you are on a Debian/Ubuntu OS, just run the following to install it:

$ aptitude install libwww-perl

Now, go check your pretty graphs:

What an interesting graph!

Simple Load Balancing Using Nginx

While the concept of load balancing has been around for a while, using Nginx to do this is fairly new to me. Other common load balancers in use today are LVS, HAProxy, Perlbal and Pound. In this example, I am using 3 (ve) servers from Media Temple, each running on Ubuntu 9.10. To get started, log into the server you want to set up as the load balancer and install Nginx:

$ aptitude install nginx

Then we’ll need to make a new Nginx default virtual host, /etc/nginx/sites-available/default. In the two server directives under the upstream backend section, be sure to put in the IP addresses or hosts you are balancing:

upstream backend  {
  server 123.123.123.123;
  server 123.123.123.124;
}

server {
  location / {
    proxy_pass  http://backend;
  }
}

This is the simplest configuration possible. After you’ve completed the proxy configuration, test and restart Nginx:

$ nginx -t
$ /etc/init.d/nginx restart

At this point, requests handled by the load balancer will serve equal requests from each upstream server. The really nice thing is that if one of the upstream servers is not responding, the load balancer will automagically stop routing requests to it. So although the configuration has the unavailable server loaded, Nginx sees that it is down and then routes to all other available upstream servers. If all upstreams are down, Nginx will halt the proxy, simply showing a 502 http error. This also makes it a very useful tool for balancing mongrels for those running rails.

It’s also worth noting that you can add as many backend nodes as you want; I’m just using two as an example. This makes using a (ve) server an ideal choice; spin up a new (ve) and simply add it to the upstream pool. There are a few other configuration options that provide the ability to add weight to backends to force an uneven load distribution. You can also use proxy_set_header to make sure the user’s IP address is logged instead of localhost:

upstream backend  {
  server john.constantshift.com;
  server paul.constantshift.com;
  server ringo.constantshift.com;
  server george.constantshift.com weight=30; # George was always my favorite
}

server {
  server_name www.constantshift.com constantshift.com;
  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass  http://backend;
  }
}

Nginx provides many other configuration directives so I would recommend checking the official Nginx documentation on the NginxHttpUpstreamModule for more information. In the next post, we’ll look at using a few file synchronization tools like Unison and rsync for data replication between backends.

Install PHP-FPM 5.3.2 on Ubuntu 9.10

Until PHP-FPM is included in PHP core, installing PHP-FPM is kind of a pain. Fortunately, it looks like it will be included in PHP 5.4. For now, installing PHP-FPM now can be done in a few different ways:

• Using SVN. This version is based off of the 0.6.x series in Launchpad. This method is considered experimental and has a few known issues reported.

$ svn co http://svn.php.net/repository/php/php-src/branches/PHP_5_3_FPM php_5_3_fpm
$ cd php_5_3_fpm
$ ./buildconf --force
$ ./configure --enable-fpm --with-foo --enable-bar...
$ make && make install

• The stable patch method works well however it still requires you to compile PHP manually. This is considered the most reliable method.

• The easiest method is to use apt-get. This is what we’ll be using here.

$ cd /tmp
$ wget http://us.archive.ubuntu.com/ubuntu/pool/main/k/krb5/libkrb53_1.6.dfsg.4~beta1-5ubuntu2_amd64.deb
$ wget http://us.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu38_3.8-6ubuntu0.2_amd64.deb
$ dpkg -i *.deb
$ echo "deb http://php53.dotdeb.org stable all" >> /etc/apt/sources.list
$ apt-get --force-yes update
$ apt-get --force-yes install php5-cli php5-common php5-mysql php5-suhosin
$ apt-get --force-yes install php5-fpm php5-cgi php-pear php5-memcache

Use the –force-yes flag in the commands above to roll past the package warnings. Now, start the daemon and clean up the /tmp directory:

$ /etc/init.d/php5-fpm start
$ rm -rf /tmp/lib*

To check if the install was successful, use the following:

$ php -v

You should see something like the following output:

PHP 5.3.2-0.dotdeb.1 with Suhosin-Patch (cli) (built: Mar  9 2010 11:42:01)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
    with Suhosin v0.9.29, Copyright (c) 2007, by SektionEins GmbH

Notes:

• To meet the required dependencies, you’ll need to install libxml2-dev and libevent-dev before you get started.

• If you are running Karmic in 32-bit, replace amd64 with i386 in the dependencies used above.

Nginx Error: Increase server_names_hash_bucket_size

During an initial server build with Nginx, I was having trouble starting the web server. Testing the Nginx configuration, the reported error was:

$ nginx -t
2010/02/05 17:18:23 [emerg] 12299#0: could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
2010/02/05 17:18:23 [emerg] 12299#0: the configuration file /etc/nginx/nginx.conf test failed

The server_names_hash_bucket_size is one of the directives in the core HTTP module for Nginx. This directive assigns the size of basket in the hash-tables of the names of servers. The hash bucket size is always equal to, or in multiples of, the size of the line of processor cache. As you can see from the error, the default on my Debian Lenny virtual machine was 32. After checking out the official documentation, I found that I simply had to increase the default bucket size from 32 to 128 in the nginx.conf. Test the configuration again and make sure the output is clean:

$ nginx -t
2010/02/05 17:22:57 [info] 21732#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2010/02/05 17:22:57 [info] 21732#0: the configuration file /etc/nginx/nginx.conf was tested successfully

Fix PECL Error: '/bin/sh: bad interpreter: Permission denied'

Continuing off my last post on PECL issues, I’ve run into additional problems installing PECL extensions on my CentOS 5.4 server. Now that PECL is able to at least run, I was seeing permission issues as it finished compiling:

$ pecl install apc

downloading APC-3.0.19.tgz …
Starting to download APC-3.0.19.tgz (115,735 bytes)
…………………….done: 115,735 bytes
47 source files, building
running: phpize
Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20060613
Zend Extension Api No: 220060519
/usr/local/bin/phpize: /tmp/pear/temp/APC/build/shtool: /bin/sh: bad interpreter: Permission denied
Cannot find autoconf. Please check your autoconf installation and the $PHP_AUTOCONF environment variable. Then, rerun this script.

ERROR: ‘phpize’ failed

The problem is that for security reasons, the /tmp file system is mounted with the noexec flag. You can check this by executing the following command:

$ mount | grep \/tmp
simfs on /tmp type simfs (rw,noexec)
simfs on /var/tmp type simfs (rw,noexec)

To fix this, you can temporary remount the directory to make it executable. To do this, issue the following command:

$ mount -o remount,exec /tmp

Once you’ve installed the extension, remount the directory again to set it back to noexec:

$ mount -o remount,noexec /tmp

Fixing 'Unsupported Protocol' Error with PECL

After installing PHP 5.2.10, I wanted to setup a few additional modules using PECL. I found soon after that I was unable to do so because of the following error:

pear.php.net is using a unsupported protocal – This should never happen. install failed

The problem seems to exhibit itself in corrupted PEAR installations with PHP 5.2.9 and 5.2.10. To fix this, we’ll update the channels:

$ mv /usr/local/lib/php/.channels /usr/local/lib/php/.channels.bak

$ pear update-channels

After that, you should be good to go.

WordPress Permalinks in Nginx

Setting up permalinks in WordPress is very easy in Apache. A few lines of mod_rewrite in the .htaccess and you’ve got pretty URLs. If you plan on using Nginx, the process is a bit different however the idea is the same. Essentially, you’ll want to add rewrite request directive in the virtual host configuration. This may vary depending on how you set up Nginx but this is the code you’ll need:

if (!-e $request_filename) {
rewrite ^/(.*)$ /index.php?q=$1 last;
break; }

You can place the rewrite directive in the location section. Here is what I am using with this blog:

location / {
    root   /var/www/constantshift.com/public/;
    index  index.php;
    if (!-e $request_filename) {
       rewrite ^/(.*)$ /index.php?q=$1 last;
       break;
    }
}

Configuring Munin Plugins for Varnish with Git

I’ve been using Varnish as a proxy to Nginx for some time and have really been impressed with some of the performance benefits I’ve seen. Although this site doesn’t generate enough traffic to warrant this type of setup, I’ve wanted to at least monitor and track cache hits and misses for Varnish. Enter Munin.

Munin is a networked resource monitoring tool that can help analyze resource trends and “what just happened to kill our performance?” problems. It is designed to be very plug and play. A default installation provides a lot of graphs with almost no work. For my setup, I’ve got Munin, the grapher/gatherer, running on another (dv) Dedicated Virtual Server from (mt) Media Temple that does all the leg work. Then I’ve got Munin nodes running on a few different servers. The default resources monitored are network throughput, disk usage, MySQL, load, ram, entropy, etc.

To start monitoring Varnish, we’ll need to setup a Munin plugin that is available on GitHub. For this, you’ll need Git:

$ aptitude install git-core

Navigate to your Munin’s plugin directory, usually /usr/share/munin/plugins:

$ cd /usr/share/munin/plugins

Now clone the repository with Git:

$ git clone git://github.com/basiszwo/munin-varnish.git

Now, let’s make things executable:

$ chmod a+x /usr/share/munin/plugins/munin-varnish/varnish_*

Then symlink the new plugins to the configuration directory, usually in /etc/munin:

$ ln -s /usr/share/munin/plugins/munin-varnish/varnish_* /etc/munin/plugins/

For most, this should be your final step. Change your Munin node configuration by adding the following to /etc/munin/plugin-conf.d/munin-node:

[varnish*]
user root

Restart the Munin node daemon and check the munin-node.log for errors:

$ /etc/init.d/munin-node restart
$ tail -f /var/log/munin/munin-node.log

Not so fast. You’re missing a Perl module:

Can't locate Net/Telnet.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at /etc/munin/plugins/varnish_varnishstat_ line 2.

No problem. Let’s install the Net::Telnet Perl module:

$ wget http://search.cpan.org/CPAN/authors/id/J/JR/JROGERS/Net-Telnet-3.03.tar.gz
$ tar -zxvf Net-Telnet-3.03.tar.gz
$ cd Net-Telnet-3.03
$ perl Makefile.PL
$ make
$ make test
$ make install

If you are on a Debian/Ubuntu OS, you can simply also use the following to install the Net::Telnet Perl module

$ aptitude install libnet-telnet-perl

If the module is installed successfully, you should restart Munin node again and check for pretty graphs:

Varnish Graph