Archived entries for Technology

Media Temple (gs) Grid Service Domain & MySQL Backups

Posted: October 25, 2009 | By: TJ | In Technology | 4 comments

Having been on many shared hosting platforms before, I am fortunate enough to have a few (gs) Grid Services with (mt) Media Temple. Although no native backup tool currently exists for the (gs), I’ve created some simple backup scripts to backup the domain directories and MySQL databases. Both are very simple bash scripts that can be set up as cron jobs from with the AccountCenter. The first backup we’ll create is for domain backups:

1. If you haven’t done so already, enable SSH from the (mt) Media Temple AccountCenter.

2. Once you’ve logged in via SSH, navigate to the /data directory. Keep in mind you’ll need to add your site number in the ##### for this to work. You can find your site number in the Server Guide > Access Domain section of the AccountCenter.

cd /home/#####/data

3. Run the following commands to create and set permissions of the backup directory:

$ mkdir backup
$ chmod 777 backup

4. Create a file called backup.sh and make it executable:

$ touch backup.sh
$ chmod +x backup.sh

5. Open the file in a text editor or vim and paste in the following:

#!/bin/bash
today=$(date '+%d_%m_%y')
echo "* Performing domain backup..."
tar czf /home/#####/data/backup/example.com_"$today".tar.gz -C / home/#####/domains/example.com
# Remove backups older than 7 days:
MaxFileAge=7
find /home/#####/data/backup/ -name '*.gz' -type f -mtime +$MaxFileAge -exec rm -f {} \;
echo "* Backed up..."

This really simple script creates a tar.gz backup of the example.com directory, stamps it with the current date and places it in the the backup folder we just created. It will also prune any backup older than 7 days old. Before saving the file, be sure to make the necessary adjustments for your site number after the /home directory. You’ll also need to modify example.com for the actual domain you’ll be backing up. Before adding it as a cron, run the script via SSH — the output should look like this:

example.com@n10:/home/#####/data$ ./backup.sh
* Performing domain backup...
* Backed up...

Depending on the size, the backup may hesitate after the initialization while the domain directory is being compressed. You can now verify that the new backup is in the /home/#####/data/backup/ directory with the today’s date: example.com_25_10_09.tar.gz. Once you’ve confirmed the backup works, add the backup.sh file as a cron within the AccountCenter following this KnowledgeBase article. You can schedule the cron as frequent as you would like however the General Settings should look like this:

Cron Settings

Click to enlarge

Now, you’ve probably got at least 1 MySQL database that should be backed up as well. This can be accomplished just as easily.

1. SSH back into the /data directory and create a new bash script file:

$ touch db-backup.sh
$ chmod +x db-backup.sh

2. Now, open the file in a text editor or vim and paste in the following:

#!/bin/sh
#############################
SQLHOST="internal-db.s#####.gridserver.com"
SQLDB="db#####_dbname"
SQLUSER="db#####"
SQLPASS="db_user_password"
SQLFILE="db#####_example.com_$(date '+%d_%m_%y').sql"
LOCALBACKUPDIR="/home/#####/data/backup"
#############################
echo "* Performing SQL dump..."
cd $LOCALBACKUPDIR
mysqldump -h $SQLHOST --add-drop-table --user="$SQLUSER" --password="$SQLPASS" $SQLDB > $SQLFILE
# Remove backups older than 7 days:
MaxFileAge=7
find /home/#####/data/backup/ -name '*.sql' -type f -mtime +$MaxFileAge -exec rm -f {} \;
echo "* Backed up..."
exit

This bash script uses mysqldump to dump a MySQL database which stamps it with the current date and places it in the the backup folder we created before. This will also prune any database backups that are older than 7 days. Before saving the file, be sure to make the necessary adjustments for your site number and database credentials — all of this can be found under the Manage Databases section of the AccountCenter. Before adding it as a cron, run the script via SSH — the output should look like this:

example.com@n10:/home/#####/data$ ./db-backup.sh
* Performing SQL dump...
* Backed up...

Once you’ve confirmed the script completes without errors, add it as a cron job so you’ll never have to worry about losing your MySQL data again!

Installing APC for PHP 5.3

Posted: September 2, 2009 | By: TJ | In Technology | 7 comments

After attempting to install APC, an opcode cacher on my Lenny server, I found that the version included in the Lenny package isn’t compatible with PHP 5.3. APC has released a version that is compatible so you’ll need install the PHP extension from source. The pecl error with version 3.0.19 was:

/tmp/pear/temp/APC/php_apc.c:959: error: duplicate 'static'
make: *** [php_apc.lo] Error 1
ERROR: 'make' failed

Luckily, installing from source is pretty easy:

$ cd /usr/local/src
$ wget http://pecl.php.net/get/APC-3.1.3p1.tgz
$ tar xzvf APC-3.1.3p1.tgz
$ cd APC-3.1.3p1
$ phpize
$ ./configure --enable-apc --enable-mmap
$ make
$ make install

After the extension successfully compiles, you’ll need to move the apc.so into your extension directory — this may differ depending on your system:

$ cp /usr/local/src/APC-3.1.3p1/modules/apc.so /usr/local/lib/php/extensions/no-debug-non-zts-20090626/

Copy the following into the php.ini:

extension = apc.so
apc.enabled=1
apc.shm_size=30

Restart the web server and if applicable, the php daemon:

$ /etc/init.d/nginx restart && php-fpm restart

To verify that everything has been set up properly, load a phpinfo page in the document root. You should have the APC PHP module displayed:

APC Module

Click to enlarge

Nginx Error: 413 Request Entity Too Large

Posted: August 14, 2009 | By: TJ | In Technology | One comment

While working in phpMyAdmin, I found that making a very simple table change was causing the following error:

413 Request Entity Too Large

The problem was that the client_max_body_size was not set properly in the nginx.conf. The directive is part of the Nginx core HTTP module and is set to 1MB by default. I set the value to 4MB and restarting fixed the issue. Essentially, the client_max_body_size is the maximum accepted body size of client request, indicated by the line “Content-Length” in the header of request. If size exceeds this value, the client gets sent the error “Request Entity Too Large” (413). If you expect to receive files uploaded to your server through the POST request method you’ll need to increase this value.

Compiling PHP 5.3.0 with FPM

Posted: July 21, 2009 | By: TJ | In Technology | 9 comments

PHP-FPM (PHP FastCGI Process Management) is a patch for PHP to improve PHP’s FastCGI capabilities and administration. Prior to FPM, I was using the spawn-fcgi library from Lighttpd to manage PHP processes for Nginx. Although this works as it should, I was curious to try FPM given some of the configuration options. Some of the key features over using FastCGI are:

1. Process Management. Using PHP-FPM provides the ability to gracefully stop and start PHP workers without losing any queries. You also have a log and pid file as well.

2. Restrict IP addresses from which requests can come from.

3. Start the workers with different uid/gid/chroot/environment and different php.ini option. You do not need a safe mode.

4. Just like MySQL, PHP-FPM provides the ability to track the slow execution of scripts and record them in a log file along with the backtrace.

The most recent patch provided is for PHP 5.3.0. The following will outline how to install and patch PHP 5.3.0 with FPM:

$ cd /usr/local/src/
$ wget http://us.php.net/get/php-5.3.0.tar.bz2/from/this/mirror
$ tar -jxvpf php-5.3.0.tar.bz2
$ wget http://php-fpm.org/downloads/php-5.3.0-fpm-0.5.12.diff.gz
$ gzip -cd php-5.3.0-fpm-0.5.12.diff.gz | patch -d php-5.3.0  -p1
$ cd php-5.3.0

Now, this part is really up to you. I’ve set this up on a development VM with VirtualBox so if you want to compile PHP with different extensions, check php.net for availability:

$ ./configure --enable-fastcgi --enable-fpm --with-mcrypt --with-zlib --enable-mbstring --with-openssl --with-mysql --with-mysql-sock --with-curl --with-gd --enable-gd-native-ttf --without-sqlite --disable-pdo --disable-reflection

Now, make and install:

$ make all install
$ strip /usr/local/bin/php-cgi

Edit the php-fpm.conf so that PHP runs as www-data instead of nobody. You’ll need to uncomment lines 63 and 66 as well:

$ vim /usr/local/etc/php-fpm.conf
:63
www-data
:66
www-data

To create the init script, we’ll symlink the one provided and update the rc.d:

$ cd /etc/init.d/
$ ln -s /usr/local/sbin/php-fpm php-fpm
$ /usr/sbin/update-rc.d -f php-fpm defaults

Start it up:

$ /etc/init.d/php-fpm start

WordPress + Memcached = Awesome

Posted: July 19, 2009 | By: TJ | In Technology | One comment

If you’re already running memcached and using WordPress, you can provide an object cache directive to help speed things up. All you need to do is place this object-cache.php script in the /wp-content directory (not the plugins folder). Then add the following to the wp-config.php:

/** Memcached Support */
global $memcached_servers;
$memcached_servers = array('default' => array('127.0.0.1:11211'));

If memcached is running on another port besides 11211, make sure to change the port value in the memcached_servers line. You can verify that caching is taking place by tailing the memcached log:

$ tail -f /var/log/memcached.log

You should see some pretty verbose output in the log. For the more documentation, check out the SVN branch.

PHP Sessions with Memcached

Posted: July 19, 2009 | By: TJ | In Technology | One comment

After setting up PHP 5.3.0 on a Debian virtual machine recently, I had some trouble implementing XCache, an opcode cacher for PHP. I’ve used XCache with Lighttpd in the past however now I am working with Nginx and using a compiled version of PHP instead of the Debian package. The issue is that the current XCache release has not been adapted to support late static binding and namespaces of PHP 5.3.0. I’ve even compiled the 2.0 beta build of XCache however this was still problematic.

I decided instead to focus on session management first and build up a caching daemon to store PHP sessions. Sessions, in many regards, would seem ideal to cache because they are transient, relatively small, and accessed frequently. I’m not overly concerned with disk I/O but I’m always interested in finding new, or better, ways to improve server performance. Enter memcached.

Memcached is a general-purpose distributed memory caching system that was originally developed by Danga Interactive for LiveJournal, but is now used by many other sites. It is often used to speed up dynamic database-driven websites by caching data and objects in memory to reduce the number of times an external data source must be read. The system is used by several very large, well-known sites including YouTube, Wikipedia, Facebook, Digg, and Twitter. For a great article on how memcached was implemented into Facebook’s architecture, check out Scaling memcached at Facebook.

So, to get started, I’ll outline how to install memcached from source and also the aptitude package manager. To install from source, use the following for reference:

$ wget http://pecl.php.net/get/memcache-3.0.4.tgz
$ tar xvfz memcache-3.0.4.tgz
$ cd memcache-3.0.4
$ phpize
$ ./configure
$ make && make test
$ make install

Note: When you install from source, it will provide the memcached extension directory, which is usually something like the following: extension_dir = “/usr/local/lib/php/extensions/no-debug-non-zts-20090626/”

If you are on a Debian based system, you can also use aptitude instead of installing from source:

$ aptitude install memcached

Start the daemon:

$ /etc/init.d/memcached start

The default port is 11211 so if you want to verify that everything has been set up properly, use netstat:

$ netstat -an | grep :11211

The default configuration for memcached was adequate in my case however if you want to modify the connection port or memory limit, you can edit the memcached.conf — /etc/memcached.conf. Now we need the memcached daemon to talk to PHP, and vice versa. For that we need the memcached PHP module. Again, this can be done with PECL or the package manager:

$ aptitude install php5-memcache

If the memcached extension hasn’t already been loaded, make sure the following is in the php.ini:

display_startup_errors = On
extension = memcache.so
memcache.hash_strategy = "consistent"
memcache.max_failover_attempts = 100
memcache.allow_failover = 1
extension_dir = "/usr/local/lib/php/extensions/no-debug-non-zts-20090626/"
session.save_handler = memcache
session.save_path = "tcp://127.0.0.1:11211?persistent=1&weight=1&timeout=1&retry_interval=15"

The extension_dir may vary depending on which version you are running so make sure the path is valid. You can also see we’ve defined the session.save_handler and session.save_path to use the memcached. Once you’ve saved the changes to the php.ini, restart your server and php if it runs as it’s own process (php-fcgi or php-fpm):

$ /etc/init.d/nginx restart && php-fpm restart

To verify that everything has been set up properly, load a phpinfo page in the document root. You should have the memcached PHP module displayed:

Memcached

PHP Module

The session settings should be changed as well:

PHP Sessions

PHP Sessions

To test the new PHP session configuration, I’ve found 2 great scripts that can be used to track PHP sessions. I’d also recommend checking out the official memcached documentation as this is only a fraction of what memcached is capable of doing.

Hide a WordPress plugin version

Posted: July 5, 2009 | By: TJ | In Technology | 2 comments

Often times with WordPress plugins, developers will design the plugin to display the plugin name and version number within the header output. This is great for the developers to promote their work however given some of the security implications, it’s not always a great idea, especially given the nature of third-party plugins. A good example of this is the All in One SEO Pack.

Continue reading…

Virtual Hosts in Lighttpd using mod_evhost

Posted: July 1, 2009 | By: TJ | In Technology | No comments yet

I’ve seen a few different ways to set up virtual hosts in Lighttpd recently and tried a few different variations based on what I’ve found. The simplest solution for me was to use the mod_evhost module. The mod_evhost module essentially builds the document-root based on a pattern which contains wildcards. So, to begin, uncomment (or add if the module is not present) the mod_evhost from the server.modules section of the lighttpd.conf:

server.modules = (
...
"mod_evhost",
...
)

Then, define the virtual host base directory and path pattern. Depending if you’ve installed from source, the var.basedir may be different:

var.basedir = "/var/www"
evhost.path-pattern = var.basedir + "/%0/"

Save the lighttpd.conf and make sure that mod_evhost has been enabled:

$ lighty-enable-mod evhost

Before restarting Lighttpd, create your virtual host directories in /var/www:

$ mkdir /var/www/domain1.com
$ mkdir /var/www/domain2.com

Once you’ve added content in the domain directories for domain1.com and domain2.com, you can restart Lighttpd:

$ /etc/init.d/lighttpd restart

You should now be serving virtual hosts for all domains located within /var/www. You can do some further tweaking to set up per domain access and error logs as well. Please check the official documentation for further information.

Enable gzip compression in Nginx

Posted: June 29, 2009 | By: TJ | In Technology | No comments yet

As with many modern web servers, enabling gzip compression can be fairly easy. If you’re using Nginx, you can modify your nginx.conf to enable gzip compression with several different directives. Open the nginx.conf file and add the following:

gzip  on;
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

To make sure that gzip doesn’t lose bigger gzipped javascript or css files, you can set buffers as well. This will assign the number and the size of the buffers into which to store the compressed response. If unset, the size of one buffer is equal to the size of page, depending on platform this either 4K or 8K. :

gzip_buffers 16 8k;

You’ll also want to disable compression for some older browsers. To do this, we can use some regex magic to make an exception for IE6 SP1:

gzip_disable "MSIE [1-6]\.(?!.*SV1)";

One of the best features is the ability to set the compression level. The compression level can be set between 1 and 9, where 1 is the least compression (fastest) and 9 is the most (slowest). For further information on this directive and others, please check out the official Nginx module page.

Unknown user in statoverride file

Posted: June 16, 2009 | By: TJ | In Technology | No comments yet

When trying to install a new package dependency (flex) for Monit 5, I ran into this error:

dpkg: unknown group 'munin' in statoverride file
E: Sub-process /usr/bin/dpkg returned an error code (2)

I recognized the munin user from a previous munin install. Since I had already purged the Munin package, I simply removed all references of the munin user from the statoverride file in /var/lib/dpkg/statoverride. I also checked other munin references by using grep:

$ grep -ril 'munin' /var/lib/*

Once all Munin references were removed, I was able to install flex successfully.



Copyright © 2010. All rights reserved.

This blog is powered by WordPress and proudly hosted by (mt) Media Temple.