Using Squidguard and Pfsense to Url Filter with Domain Blacklists from Squidblacklist.org

Using Squidguard and Pfsense to Url Filter with Domain Blacklists.

pfsense Logo

URL filtering is one strategy used to filter access to websites based the domain name and/or url. There are several commercial products available for URL or domain content filtering, but you could easily build a very reliable system on your own using SquidGuard and pfSense. SquidGuard is a useful add on package for the Squid proxy server and can be used to filter or redirect web requests on the network.

SquidGuard has a long list of features that can be tailored to fit your needs. It’s also rather fast and does’nt slow down the internet for your clients. If you do need to block access to a list of unwanted websites or only allow access to a whitelist of specific web sites, SquidGuard can certainly assist with this.

SquidGuard is also very flexible, and it is easy to adapt to different applications. If you intend to do basic URL filtering on your home network or if you need to create some complicated rules for a large private or public network SquidGuard can do it.

Before you can put a web filtering proxy under pfSense into production, some configuraation is required. If you are new to pfSense I might recommend reading through the instructions that shit with pfSense.

Install the package SquidGuard Package

SquidGuard & Squid proxy can both be installed using the pfSense package manager. To access the pfSense package manager, click packs on the system menu. Select the tab available packages and scroll down where you will find SquidGuard and Squid proxy individually, click the plus sign next to each item to begin the installation.

Once the installations are complete you will have a new menu item called proxy services/filter.

Blacklists

To set up domain blacklist, open the general settings page ‘Filter Services & Proxy’. Click the checkbox to activate the domain blacklist.

You can use one of several different domain black lists publicly available on the web. You can also find a list of several blacklists from http://www.squidblacklist.org. We have our blacklists available in multiple formats, but likely, youll want the standard directory formatted archives located at the following url. http://www.squidblacklist.org/downloads/squidblacklists/squidblacklist.tar.gz


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 3 Year Membership Option now available.
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.


Excluding URL blacklist

There may be some places that you need to allow your users to access. To prevent these sites from locking can create a new category of destination and add a list of domains or URLs that should not be blocked.

To do this click the target categories tab, and then click the plus sign to add a new category. You must assign a unique name to the new category, the name you choose can not contain spaces.

The target category can filter by domain name, URL, or an expression. Add a domain site will grant access to the main site and all its sub-pages. Entering a URL allows access only to that exact website. Expressions allow you to grant based on certain keywords access.

When finished, click Save, and then back to the common ACL tab or group (wherever that created the rule) and select and action whitelist for your new category.

You can also use this same method to add additional sites to its blacklist.

Filtering by Expression

In addition to the domain and URL filtering SquidGuard can create filters using regular expressions. These types of filters are great when you want to search for specific text strings in a URL to make a decision for this search. If you are unfamiliar with regular expressions can be a bit confusing at first, but there are many online resources on the subject, so I will not go into much detail about them in this article.

To create a filter that uses an expression, click the target categories tab, or create a new category or edit an existing one. Enter the expression you want to filter in the expression box and then click Save. Then go back to the common or group ACL tab and select the action (deny, permit, etc.) for your target category.

Here are some examples of filter expressions are presented. These can be edited according to what to filter. For more useful information about filtering regular expressions http://www.squidguard.org/Doc/Examples review.

Downloads based on file extension block

(* \ /.* \ (Zip | .. Rar | exe | msi | mpeg | avi))

Block certain TLDs

(.gov | .xxx | Mil | .net)

Block search “bypass proxy” on Google and Yahoo

(.*(google|yahoo).*(search_query|keywords|search|query|q|p)=.*(\+|\%20)*(proxy|bypass).*(\-|\+|\%20).*(proxy|bypass).*)

Programming rules & Time-based rules

SquidGuard also allows you to apply URL filtering based on schedules. Times are useful for applying rules at different times during the day, or only on certain days of the week.

For example, you could apply URL filtering rules strict office hours and automatically disable the rules after 17:00. If you are filtering your home network you may not want the children to visit certain sites during the school week, this is another example in which a time-based rule would be used.

To create a rule-based time, click the time tab and then click the plus sign to create a new schedule. You can create as many different times as you need.

Schedules can be applied using the ACL Groups tab. Create a new ACL or edit an existing group, then click the “time” drop-down box select the schedule you created.

Do not forget to click Apply on the General tab for the settings to take effect.

Conclusion

Commercial Web filtering devices can be very expensive and difficult to handle. PfSense SquidGuard and are completely free and very powerful. SquidGuard offers many other features that are not covered in this center. For more detailed information, visit SquidGuard.org and check out the documentation section. Also be sure to check out some of my other centers to learn about more ways to use pfSense on your network.
Guidelines pfSense

pfSense Bandwidth – Setting Traffic Shaping
Heavy users wide band can slow the entire network. This center will show you how to use pfSense to set traffic shaping to prioritize Internet traffic.
Dual Wan Router – How to load balance with pfSense
Dual WAN Routers allow you to increase the bandwidth of the Internet on your network by combining two Internet connections. Using pfSense can turn an old computer into a powerful multi WAN router.
How to set up a transparent proxy using squid pfSense
Proxy servers can be very useful for improving the speed of an Internet connection by caching, log Internet usage, or filter traffic. Learn how to set up a transparent proxy using pfSense.

2 Replies to “Using Squidguard and Pfsense to Url Filter with Domain Blacklists from Squidblacklist.org”

  1. Nice post, but if may have a question, is there any way to completely block certain web browser – say MS Edge for a group of ADS users by utilizing squidGuard expression list or other blacklist?

Leave a Reply

Your email address will not be published. Required fields are marked *

*