September 2002

BSD Newsletter

Get BSD

Contact Us

Search BSD

Setting Up Squid on FreeBSD

Andi Salimun <asalimun@andix.info>

Introduction
This article will explain how to configure Squid and tweak a FreeBSD box for web caching and
conserving bandwidth. Usually, Squid will reduce the traffic 30% or more from normal usage
(without squid), and enhance response time.

FAQ
Search

Monthly Ezine
Search

Get BSD Stuff

This configuration and setting has been tested and the Squid FreeBSD boxes are running at
several colleges in Sydney, Australia. Each Squid FreeBSD box serves about 100+ nodes.

Requirements
Squid needs a lot of memory. More is better but 128 MB of memory is good to start with.
Squid also needs fast disk storage. Use SCSI drives if you can, though IDE drives will do the
job. Also have a FreeBSD 4.5 box setup and running! You will need to recompile the kernel
with additional options and components.

1. Installing Squid

You can install Squid using Ports but you can't play with configure options, so I'll cover the steps to install from a t

You can download the Squid source from http://www.squid-cache.org. The lastest version is squid-2.4.STABLE6Run the following commands as root.
#
#
#
#

cd /path/to/tarball
tar zxvf squid-2.4.STABLE6-src.tar.gz
cd squid-2.4.STABLE6
./configure --enable-delay-pools --enable-ipf-transparent \
--enable-storeio=diskd,ufs --enable-storeio=diskd,ufs \
--disable-ident-lookups --enable-snmp --enable-removal-policies
# make all
# make install

The explaination of configure script options are below:

--enable-delay-pools - Enable delay pools to limit bandwidth usage.

You need to enable the option in order to use Squid to limit bandwith usage. It will give fair ban
for everybody. In my case, I don't want one person sucking all of the available bandwidth by do

big movie, causing others to suffer.

--enable-ipf-transparent - Enable Transparent Proxy support for systems using IP Filter network address r

With this option, you don't have to configure the client's browser proxy setting. Also it is a good
force the client to use the proxy everytime.
--enable-storeio=diskd,ufs - Enable diskd

Improve disk I/O performance. According to the Squid FAQ, if you enable diskd you can gain a 4
increase of perfomance. However, you would need to recompile the kernel because your operati
must support message queues and shared memory.
--enable-removal-policies - Build support for the list of removal policies.

By default, Squid uses LRU, but there are two better policies: GDSF and LFUDA. See the Squid c
more detailed explanation.
--disable-ident-lookups - This allows you to remove code that performs Ident (RFC 931) lookups.

Not really important. By the way, if you do transparent proxy, ident lookups won't work.
--enable-snmp

Optional: enable this and you can monitor Squid with mrtg or rrdtool. How to do this is outside o
article's scope. Perhaps in my next one.

2. Edit Squid Configuration File /usr/local/squid/etc/squid.conf

# Need for transparent proxy
# You need to --enable-ipf-transparent
http_port 3128
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

# Physical memory / 3
cache_mem 128 MB
# Max out Squid I/O perfomance, 15 GB cache and use Squid special diskd but you need to reco
# To use disk you need to --enable-storeio=diskd,ufs
# Reasonable values for Q1 and Q2 are 72 and 64, respectively.
# Q1 value must bigger Q2
cache_dir diskd /usr/local/squid/cache 15360 16 256 Q1=72 Q2=64
# You can use normal ufs instead
#cache_dir ufs /usr/local/squid/cache 15360 16 256
# I dont want to log anything
# The reason is to save some expensive I/O operation.
cache_access_log /dev/null
cache_store_log none

cache_log /dev/null
# Cache replacement policy
# The heap GDSF policy optimizes object-hit rate by keeping smaller popular
# objects in cache, so it has a better chance of getting a hit. It achieves a
# lower byte hit rate than LFUDA, though, since it evicts larger (possibly popular)
# objects.
# The heap LFUDA ( Least Frequently Used with Dynamic Aging ) policy keeps
# popular objects in cache regardless of their size and thus optimizes byte hit
# rate at the expense of hit rate since one large, popular object will prevent
# many smaller, slightly less popular objects from being cached.
# You need to --enable-removal-policies
cache_replacement_policy GDSF
# Standard Access List
# I have two subnets, one for student and another one for admin
# Modify this according to your network
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl outgoing src 192.168.10.2/255.255.255.255
acl student src 192.168.0.0/255.255.255.0
acl admin src 192.168.1.0/255.255.255.0
acl SSL_ports port 443 563
acl Safe_ports port 80
# http
acl Safe_ports port 21
# ftp
acl Safe_ports port 443 563
# https, snews
acl Safe_ports port 70
# gopher
acl Safe_ports port 210
# wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280
# http-mgmt
acl Safe_ports port 488
# gss-http
acl Safe_ports port 591
# filemaker
acl Safe_ports port 777
# multiling http
acl CONNECT method CONNECT
http_access
http_access
http_access
http_access
http_access
http_access
http_access
http_access
icp_access
icp_access
icp_access
icp_access

allow manager
allow localhost
allow outgoing
allow student
allow admin
deny !Safe_ports
deny CONNECT !SSL_ports
deny all
allow localhost
allow student
allow admin
deny all

# Avoid caching cgi scripts

acl QUERY urlpath_regex cgi-bin
no_cache deny QUERY
acl magic_words1 url_regex -i 192.168

acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar .avi .mpeg .mpe
# Delay Pool
# For delay pool, you need to --enable-delay-pools
delay_pools 2
# I have ADSL 2Mbits line
# 2 mbits == 256 kbytes per second
# 256 KB/s, 5 KB/s
# It means 256 KB/s bandwith for the whole network, but 5 KB/s for each node, which is fair
delay_class 1 2
delay_parameters 1 256000/256000 5000/256000
delay_access 1 allow magic_words2
delay_access 1 allow student
delay_access 1 allow admin
# -1/-1 means that there are no limits for local traffic.
delay_class 2 2
delay_parameters 2 -1/-1 -1/-1
delay_access 2 allow magic_words1
# Cancel download if file is bigger than 1MB
reply_body_max_size 1024 KB
# snmp stuff
acl snmppublic snmp_community public
snmp_access allow snmppublic localhost
snmp_access deny all
# Change to your domain
# visible_hostname yourdomain.domain.com
# cache_mgr yourname@youremail.com

3. Create cache dir and create swap

#
#
#
#

mkdir /usr/local/squid/cache
chown nobody:nogroup cache
/usr/local/squid/bin/squid -k parse
/usr/local/squid/bin/squid -z

4. Configure transparent proxy with ipfilter

4.1 Edit /etc/rc.conf
# add these lines to enable ipfilter
ipfilter_enable="YES"
ipnat_enable="YES"
ipmon_enable="YES"
ipfs_enable="YES"

4.2 Edit /etc/ipnat.rules

# add this line

# I assume rl0 is your internal nic

# Redirect everything else to squid on port 3128
rdr rl0 0/0 port 80 -> 127.0.0.1 port 3128 tcp

5. Reconfigure kernel for squid diskd support

Consult the Freebsd Handbook for recompiling the kernel and add the following lines. Your kernel must have:
options

SYSVMSG

You can set the parameters in the kernel as follows. This is just an example. Make sure the values are appropriate
system:
options
options
options
options
options

MSGMNB=8192
MSGMNI=40
MSGSEG=512
MSGSSZ=64
MSGTQL=2048

#
#
#
#
#

max # of bytes in a queue

number of message queue identifiers
number of message segments per queue
size of a message segment
max messages in system

The following is the explanation of the kernel options from the Squid FAQ:

The messages between Squid and diskd are 32 bytes for 32-bit CPUs and 40 bytes for 64-bit CPUs. Thus, MSGSSZ
32 or greater. You may want to set it to a larger value, just to be safe.

We'll have two queues for each cache_dir, one in each direction. So, MSGMNI needs to be at least two times the n
cache_dir's.

I've found that 75 messages per queue is about the limit of decent performance. If each diskd message consists o
segment (depending on your value of MSGSSZ), then MSGSEG should be greater than 75.

MSGMNB and MSGTQL affect how many messages can be in the queues at one time. Diskd messages shouldn't
than 40 bytes, but let's use 64 bytes to be safe. MSGMNB should be at least 64*75. I recommend rounding up to
power of two, or 8192.
MSGTQL should be at least 75 times the number of cache_dir's that you'll have.

Also you can tweak the kernel by commenting out unnecessary lines in the kernel config to gain extra perfomance
recompile the kernel.

6. Create start-up script /usr/local/etc/rc.d/squid.sh

#!/bin/sh
echo -n ' Squid '
case "$1" in
start)
/usr/local/squid/bin/squid -D
;;
stop)
/usr/local/squid/bin/squid -k shutdown
;;
restart)
/usr/local/squid/bin/squid -k reconfigure

;;
*)
echo "Usage: `basename $0` {start|stop|restart}"
;;
esac

7. All Done!
References
FreeBSD - http://www.freebsd.org/
Squid Web Proxy Cache - http://www.squid-cache.org/
Configuration Manual for Squid 2.4 Stable x - http://squid.visolve.com/
Linux Documentation Project - Bandwidth Limiting HOWTO - http://www.linuxdoc.org
Do-It-Yourself Caching: Squid 2.3 - http://www.bsdtoday.com/2000/February/Tutorials28.html
IP Filter Based Firewalls HOWTO - http://www.obfuscation.org/ipf/ipf-howto.html
Andi Salimun
18 Jul 2002

Author maintains all copyrights on this article.

Images and layout Copyright 1998-2003 Dmon News. All Rights Reserved.