I use an external squid transparent proxy for my clients.

We recently switched to a RouterOS firewall

and I discovered a great way to route traffic to my proxy without having to configure each client's
proxy settings.
I dug around on the internet and found this lovely guide that describes pretty close to what i'm looking
for.
The way we handled it in the past was to redirect port 80 traffic from the firewall to the ip and port
number of our squid proxy. MikroTik offers ways I can do this while still preserving the source ip
address. Before i setup my new config anyone who didn't have their client proxy setup was listed as
the filewall's ip address in squid's logs.
Here's my setup
Firewall/Mikrotik 192.168.1.1
Squid Proxy 192.168.1.2 (Proxy port 8080)
Clients 192.168.100-192.168.1.199
I already have the mangle and masquerading setup for my main firewall so I'm only going to list the
settings that affect and redirect http traffic to my proxy server.
First I setup an address list of the client ip addresses i wanted to be redirected to my proxy server. In
my case its my dhcp pool addresses

Code: Select all

/ip firewall address-list
add address=192.168.1.100-192.168.1.199 list=Proxy_Clients

Then i setup mangle to tag web traffic for rerouting later

Code: Select all

/ip firewall mangle
chain=prerouting action=mark-routing new-routing-mark=to_proxy passthrough=yes protocol=tcp
src-address-list=Proxy_Clients dst-port=80

Next i setup a custom route for the marked packets

Code: Select all

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.2 routing-mark=to_proxy
scope=30 target-scope=10

What we're doing now is routing all traffic from clients 192.168.1.100-192.168.1.199 on port 80 to the
proxy server.
The last piece of the puzzle is to setup the proxy server (squid) to redirect all traffic on port 80 to the
proxy port (8080). This is easily done with iptables.

Code: Select all

iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to 192.168.1.2:8080

That should do it. Now you are forwarding all port 80 traffic to the squid proxy. Most importantly is you
are preserving the source ip address. This way if you want to use squid reporting you can track your
users usage by ip address.

Most of the information i used to get this solution was borrowed from this website. I changed up the
syntax a little to fit my needs. My next project is to figure out how to exclude streaming services
(netflix) from being forwarded to the proxy server. I've already toyed with using ip blocks. The biggest
problem i've run into there is adding all netflix's ip ranges to an address list. I keep finding new ones.
For those interested: Here's my modified mangle rule for not forwarding netflix traffic to the proxy, but
forwarding everything else.

Code: Select all

/ip firewall mangle
chain=prerouting action=mark-routing new-routing-mark=to_proxy passthrough=yes protocol=tcp
src-address-list=Proxy_Clients dst-address-list=!Netflix dst-port=80

My address list for netflix

Code: Select all

/ip firewall address-list
add address=69.164.0.0/18 list=Netflix
add address=208.111.128.0/18 list=Netflix
add address=68.142.64.0/18 list=Netflix
add address=108.175.32.0/20 list=Netflix

I found a better way to handle netflix traffic. The address list is fine, but after more digging i found a
way to dynamically generate the list. This guide shows a queueing system for prioritizing or throttling

video sites. I took his example and made it fit into my proxy forwarding solution.
I'm still using my route forward

Code: Select all

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.2 routing-mark=to_proxy
scope=30 target-scope=10

Remember my external squid proxy ip address is 192.168.1.2 and my firewall address is 192.168.1.1
What I've changed is the way I generate my address-list. Instead of adding entire ip blocks I added 2
mangle rules in *front* of my proxy forwarding one. Here's what they look like (I've listed them in the
order required. The proxy routing mark mangle rule must be the last.)

Code: Select all

/ip firewall mangle
chain=prerouting action=add-dst-to-address-list protocol=tcp address-list=Netflix address-listtimeout=1d dst-port=80 content=nflxvideo.net
chain=prerouting action=add-dst-to-address-list protocol=tcp address-list=Netflix address-listtimeout=1d dst-port=80 content=netflix.com
chain=prerouting action=add-dst-to-address-list protocol=tcp address-list=Netflix address-listtimeout=1d layer7-protocol="Netflix llnwd" dst-port=80

In the guide the guy didn't set a timeout for the address list. That means that the list will eventually
grow to include every ip address netflix uses. In theory this sounds good, but you have to remember
its going to happen 1 ip address at a time, NOT one subnet block at a time. I thought 1d was plenty of
time.
I'm still using my mangle rule to add a route tag to traffic with the specified mark. I added a dstaddress-list negative to exclude my newly created Netflix address list that is being generated. (This
mangle rule must be last.)

Code: Select all

/ip firewall mangle
chain=prerouting action=mark-routing new-routing-mark=to_proxy passthrough=yes protocol=tcp
src-address-list=Proxy_Clients dst-address-list=!Netflix dst-port=80

I also have a layer7 protocol. I'm not great with regexp but the one i'm using works better than all the
other examples i've found out there

Code: Select all

/ip firewall layer7-protocol
name="Netflix llnwd" regexp="netflix-[0-9][0-9][0-9]\.vo\.llnwd\.net"

I'm sure the regex statement could be tightened up a little. I ran a few packet sniffers and tried to see
exactly what the router was looking at when it ran regex statement against the packets. This was the
best i could do. At least i'm not excluding all of llnwd.net from the proxy. I only wanted netflix's content
they host.
What this does is create a dynamic entry in the Netflix address list every time someone tries to access
netflix.com, nflxvideo.net, or one of the many content servers they have netflix-####.vo.llnwd.net.
Since the address list "Netflix" is excluded from proxy tagging my firewall doesn't try to forward netflix
traffic to my external squid proxy server. There is another netflix domain i didn't include in this.
nflximg.com Netflix doesn't seem to mind if i cache the images associated with the videos so i left it
out. I'd rather have that traffic proxied and save a little bandwidth. If you wanted to include it all that's
needed is to add another mangle rule. Simply copy the first mangle rule and modify the content value

to nflximg.com (don't forget the order is important). So far its working fairly well. The router load
doesn't seem to spike even with 6 different devices running traffic at the same time. I think since i'm
marking connections for the router part that helps a lot.