How to setup Varnish Cache a web application accelerator on CentOS 6.3

December 6, 2013

This is a how to article on setting up Varnish Cache on CentOS 6.3 64-bit as a single server web application accelerator (caching HTTP reverse proxy). Varnish Cache can increase the speed of your web application by 300 – 1000 times when used properly in the proper architecture.

First install your Varnish dependencies:

bash-shell# yum install -y gcc
bash-shell# yum install -y make
bash-shell# yum install -y automake
bash-shell# yum install -y autoconf
bash-shell# yum install -y libtool
bash-shell# yum install -y ncurses-devel
bash-shell# yum install -y libxslt
bash-shell# yum install -y groff
bash-shell# yum install -y pcre-devel
bash-shell# yum install -y pckgconfig
bash-shell# yum install -y libedit
bash-shell# yum install -y libedit-devel

Get your Varnish source code at the time of this blog Varnish 3.0.5 stable:

bash-shell# wget http://repo.varnish-cache.org/source/varnish-3.0.5.tar.gz

Extract and compile Varnish

bash-shell# gunzip -c varnish-3.0.5.tar.gz | tar -xvf -
bash-shell# cd varnish-3.0.5
bash-shell# ./autogen.sh
bash-shell# ./configure
bash-shell# make
bash-shell# make check
bash-shell# make install

Your Varnish configuration file will be locate at /usr/local/etc/varnish/default.vcl

Below is a example default.vcl which configures Varnish to cache only what you determine and not everything else. The configuration is commented to explain the settings:

default.vcl

## Your backend http application server info as well as health checking
backend website {
  .host = "app.server.hostname";   # backend http application server or vip
  .port = "80";                    # port your backend application listens on
  .probe = {                       # health check setting 
    .request =
      "GET / HTTP/1.0"             # what to health check
      "Host: app.server.hostname"  # what host to health check
      "Connection: close";         # make sure to close the connection of the check
    .interval = 5s;                # how long time to wait between polls
    .window = 5;                   # how many of the latest polls to consider if its healthy
    .threshold = 2;                # how many of the .window polls must be good to be healthy
    .timeout = 1s;                 # how fast the probe must finish
    .expected_response = 200;      # expected http response of the health check
  }
}
 
## Example cache only url beginning with /xyz and not everything else
sub vcl_recv {
 
  remove req.http.X-Forwarded-For;             # allowing client ip pass through
  set req.http.X-Forwarded-For = client.ip;
 
# cache url beginning with xyz and things ending in jpg css etc..  
  if (! req.url ~ "^/xyz/" || req.url ~ "\.(jpg|jpeg|css|js|png)$" ) {  
    return(pass);
  }
 
  unset req.http.Cookie;   # disregard caching cookies
 
  set req.grace = 1h;      # accept objects that are up to 1 hour old in case of backend failure
 
  return(lookup);
}
 
sub vcl_fetch {
 
  set beresp.do_esi = true;  # enable esi includes
 
  set beresp.ttl = 2m;       # time to live for cache that is not explict set from the header info
 
  set beresp.grace = 1h;     # deliver objects that are up to 1 hour old in case of backen failure
 
  return(deliver);
}

Now you can start Varnish with the below command, in this case it says to use the default.vcl with 3 Gigs of memory for cache, with administrative ip localhost and port 2000 and listen for any request on port 80

bash-shell#  /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80

Some other commands to help you tune Varnish and debug it are:

# show varnish statistics
bash-shell# varnishstat
 
# show varnish logs
bash-shell# varnishlog
 
# clear all varnish cache
bash-shell# varnishadm "ban req.url ~ /"

To test it out put the url of your application site substituting the hostname for your new Varnish server, on your second load it should be a lot faster serving data from cache. You should use firebug to make sure or varnishstat command to see your cache hits vs misses. Remember in this configuration the things you explicitly state are only cached, and are what you will see the performance increase in.

Example:

http://app.server.hostname/xyz/my/app/is/awesome.php

vs

http://varnish.server.hostname/xyz/my/app/is/awesome.php (should be faster after on second load)

Hope you find this helpful.

Comments for “How to setup Varnish Cache a web application accelerator on CentOS 6.3”

  1. me commented on April 3, 2014

    Hi,
    I follow your instructions everything goes fine until
    # ./autogen.sh

    I receive the following message:
    + libtoolize –copy –force
    You should update your `aclocal.m4′ by running aclocal.
    + aclocal -I m4
    + autoheader
    + automake –add-missing –copy –foreign
    + autoconf

    after that….

    # make
    make all-recursive
    make[1]: Entering directory `/root/varnish-3.0.5′
    Making all in include
    make[2]: Entering directory `/root/varnish-3.0.5/include’
    make all-am
    make[3]: Entering directory `/root/varnish-3.0.5/include’
    make[3]: Leaving directory `/root/varnish-3.0.5/include’
    make[2]: Leaving directory `/root/varnish-3.0.5/include’
    Making all in lib
    make[2]: Entering directory `/root/varnish-3.0.5/lib’
    Making all in libvarnishcompat
    make[3]: Entering directory `/root/varnish-3.0.5/lib/libvarnishcompat’
    if /bin/sh ../../libtool –tag=CC –mode=compile gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I. -I../.. -I../../include -g -O2 -pthread -Wextra -Wno-missing-field-initializers -Wno-sign-compare -MT daemon.lo -MD -MP -MF “.deps/daemon.Tpo” -c -o daemon.lo daemon.c; \
    then mv -f “.deps/daemon.Tpo” “.deps/daemon.Plo”; else rm -f “.deps/daemon.Tpo”; exit 1; fi
    ../../libtool: line 851: X–tag=CC: command not found
    ../../libtool: line 884: libtool: ignoring unknown tag : command not found
    ../../libtool: line 851: X–mode=compile: command not found
    ../../libtool: line 1017: *** Warning: inferring the mode of operation is deprecated.: command not found
    ../../libtool: line 1018: *** Future versions of Libtool will require –mode=MODE be specified.: command not found
    ../../libtool: line 1161: Xgcc: command not found
    ../../libtool: line 1161: X-std=gnu99: command not found
    ../../libtool: line 1161: X-DHAVE_CONFIG_H: command not found
    ../../libtool: line 1161: X-I.: command not found
    ../../libtool: line 1161: X-I.: command not found
    ../../libtool: line 1161: X-I../..: No such file or directory
    ../../libtool: line 1161: X-I../../include: No such file or directory
    ../../libtool: line 1161: X-g: command not found
    ../../libtool: line 1161: X-O2: command not found
    ../../libtool: line 1161: X-pthread: command not found
    ../../libtool: line 1161: X-Wextra: command not found
    ../../libtool: line 1161: X-Wno-missing-field-initializers: command not found
    ../../libtool: line 1161: X-Wno-sign-compare: command not found
    ../../libtool: line 1161: X-MT: command not found
    ../../libtool: line 1161: Xdaemon.lo: command not found
    ../../libtool: line 1161: X-MD: command not found
    ../../libtool: line 1161: X-MP: command not found
    ../../libtool: line 1161: X-MF: command not found
    ../../libtool: line 1161: X.deps/daemon.Tpo: No such file or directory
    ../../libtool: line 1161: X-c: command not found
    ../../libtool: line 1213: Xdaemon.lo: command not found
    ../../libtool: line 1218: libtool: compile: cannot determine name of library object from `’: command not found
    make[3]: *** [daemon.lo] Error 1
    make[3]: Leaving directory `/root/varnish-3.0.5/lib/libvarnishcompat’
    make[2]: *** [all-recursive] Error 1
    make[2]: Leaving directory `/root/varnish-3.0.5/lib’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/root/varnish-3.0.5′
    make: *** [all] Error 2

    and it is impossible to continue, any idea why??

    Thanks in advance

  2. Make sure you successfully installed libtool, autoconf, automake. Usually these errors happen if the dependencies are missing or versions are to old. Hope this helps.

  3. I fixed that error by running
    export echo=echo on centos 5.7

Leave a Reply