Category Archives: pfsense

New Blacklist: A List of Terrorist & Extremist Domains.

We have been working to compile the initial release of a new blacklist, which we have now released in the members area of Squidblacklist.org. Available for immediate download, our new and unique “Terrorism and Extremism” Domain Blacklist. Available now for all current and new subscribers. As is always, we have formatted this blacklist for use with multiple platforms.

DNS Zone format, RouterOS DNS and Web Proxy *RSC import script, Squid Proxy Native ACL format, as well as plain text Dansguardian/SquidGuard Compatible format for use with all other web filtering platforms and applications.

Subscribe today and find out why so many have switched. Shouldnt you be using a higher class blacklist?

We are committed to continually evolve the technology we are using on the backend to produce ever increasingly better publications, We are proud to consistently bring ever more refined and simply the highest quality blacklists available.


Subscribe Now For Zone Access.

Flat rate subscription. For full access to all of our works, select a membership option & subscribe today.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.


New Option Available for Urlblacklist/Shallalist Compatible Platforms!


We heard you loud and clear, you wanted our enhanced blacklists in a similar archive/file structure as shallalist & urlblacklist for your web filtering platform, so we finally did it!

pfSense Blacklist Update
pfSense Blacklist Update

Available now to all squidblacklist.org members is the new “Universal Archive Structure Format” for any platform coded for shallalist or urlblacklist file structured archives!
Untitled

This new option is available for immediate download to all Squidblacklist.org visitors in the members area of the website, a subscription is required.

Untitled2

Enhance your web filtering strategy with enhanced blacklists and subscribe today.


Blacklisting has Evolved. Subscribe Now!

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.


Music website/domain blacklist for content filtering published.

We have had a video, and an image blacklist for a while now, so it only seemed right to include a music blacklist. This became clear as it was requested by one of our current members who is responsible for content filtering at an educational facility.

The new Music domain blacklist for web filtering purposes has been added to our existing line up and is available in the ‘all’ archive as well as standalone download, just as you would expect any other blacklist that we offer, this blacklist is also available in multiple formats for broad compatibility with most content filters and url filtering platforms and applications.

Enhance your web filtering strategy with enhanced blacklists and subscribe today.


Blacklisting has Evolved. Subscribe Now!

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.


Using Squidguard and Pfsense to Url Filter with Domain Blacklists from Squidblacklist.org

Using Squidguard and Pfsense to Url Filter with Domain Blacklists.

pfsense Logo

URL filtering is one strategy used to filter access to websites based the domain name and/or url. There are several commercial products available for URL or domain content filtering, but you could easily build a very reliable system on your own using SquidGuard and pfSense. SquidGuard is a useful add on package for the Squid proxy server and can be used to filter or redirect web requests on the network.

SquidGuard has a long list of features that can be tailored to fit your needs. It’s also rather fast and does’nt slow down the internet for your clients. If you do need to block access to a list of unwanted websites or only allow access to a whitelist of specific web sites, SquidGuard can certainly assist with this.

SquidGuard is also very flexible, and it is easy to adapt to different applications. If you intend to do basic URL filtering on your home network or if you need to create some complicated rules for a large private or public network SquidGuard can do it.

Before you can put a web filtering proxy under pfSense into production, some configuraation is required. If you are new to pfSense I might recommend reading through the instructions that shit with pfSense.

Install the package SquidGuard Package

SquidGuard & Squid proxy can both be installed using the pfSense package manager. To access the pfSense package manager, click packs on the system menu. Select the tab available packages and scroll down where you will find SquidGuard and Squid proxy individually, click the plus sign next to each item to begin the installation.

Once the installations are complete you will have a new menu item called proxy services/filter.

Blacklists

To set up domain blacklist, open the general settings page ‘Filter Services & Proxy’. Click the checkbox to activate the domain blacklist.

You can use one of several different domain black lists publicly available on the web. You can also find a list of several blacklists from http://www.squidblacklist.org. We have our blacklists available in multiple formats, but likely, youll want the standard directory formatted archives located at the following url. http://www.squidblacklist.org/downloads/squidblacklists/squidblacklist.tar.gz


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.


Excluding URL blacklist

There may be some places that you need to allow your users to access. To prevent these sites from locking can create a new category of destination and add a list of domains or URLs that should not be blocked.

To do this click the target categories tab, and then click the plus sign to add a new category. You must assign a unique name to the new category, the name you choose can not contain spaces.

The target category can filter by domain name, URL, or an expression. Add a domain site will grant access to the main site and all its sub-pages. Entering a URL allows access only to that exact website. Expressions allow you to grant based on certain keywords access.

When finished, click Save, and then back to the common ACL tab or group (wherever that created the rule) and select and action whitelist for your new category.

You can also use this same method to add additional sites to its blacklist.

Filtering by Expression

In addition to the domain and URL filtering SquidGuard can create filters using regular expressions. These types of filters are great when you want to search for specific text strings in a URL to make a decision for this search. If you are unfamiliar with regular expressions can be a bit confusing at first, but there are many online resources on the subject, so I will not go into much detail about them in this article.

To create a filter that uses an expression, click the target categories tab, or create a new category or edit an existing one. Enter the expression you want to filter in the expression box and then click Save. Then go back to the common or group ACL tab and select the action (deny, permit, etc.) for your target category.

Here are some examples of filter expressions are presented. These can be edited according to what to filter. For more useful information about filtering regular expressions http://www.squidguard.org/Doc/Examples review.

Downloads based on file extension block

(* \ /.* \ (Zip | .. Rar | exe | msi | mpeg | avi))

Block certain TLDs

(.gov | .xxx | Mil | .net)

Block search “bypass proxy” on Google and Yahoo

(.*(google|yahoo).*(search_query|keywords|search|query|q|p)=.*(\+|\%20)*(proxy|bypass).*(\-|\+|\%20).*(proxy|bypass).*)

Programming rules & Time-based rules

SquidGuard also allows you to apply URL filtering based on schedules. Times are useful for applying rules at different times during the day, or only on certain days of the week.

For example, you could apply URL filtering rules strict office hours and automatically disable the rules after 17:00. If you are filtering your home network you may not want the children to visit certain sites during the school week, this is another example in which a time-based rule would be used.

To create a rule-based time, click the time tab and then click the plus sign to create a new schedule. You can create as many different times as you need.

Schedules can be applied using the ACL Groups tab. Create a new ACL or edit an existing group, then click the “time” drop-down box select the schedule you created.

Do not forget to click Apply on the General tab for the settings to take effect.

Conclusion

Commercial Web filtering devices can be very expensive and difficult to handle. PfSense SquidGuard and are completely free and very powerful. SquidGuard offers many other features that are not covered in this center. For more detailed information, visit SquidGuard.org and check out the documentation section. Also be sure to check out some of my other centers to learn about more ways to use pfSense on your network.
Guidelines pfSense

pfSense Bandwidth – Setting Traffic Shaping
Heavy users wide band can slow the entire network. This center will show you how to use pfSense to set traffic shaping to prioritize Internet traffic.
Dual Wan Router – How to load balance with pfSense
Dual WAN Routers allow you to increase the bandwidth of the Internet on your network by combining two Internet connections. Using pfSense can turn an old computer into a powerful multi WAN router.
How to set up a transparent proxy using squid pfSense
Proxy servers can be very useful for improving the speed of an Internet connection by caching, log Internet usage, or filter traffic. Learn how to set up a transparent proxy using pfSense.

Domain Whitelist For Content Filtering Published

Our domain whitelist for content filtering purposes is now available for public access.

 

This whitelist does not contain any torrent or porn sites and should provide a good baseline  whitelist for general audiences.

 

It can be downloaded here.


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Free blacklists suck , and heres why.

Squidblacklist.org – Blacklisting Has Evolved.

Normally I would believe free is great, free as in, gratis. However, when it comes to the blacklists freely available online, specifically, blacklists tailored for consumption as an important web filtering component used in various content filter hardware platforms and software applications. The free solutions just dont cut it. And If you as an administrator can find as many holes within just a few simple Google searches using the free blacklists, just imagine how easy it will be for a determined end user on your network to find and access unauthorized content.

snakeoil
One can easily load up their content control platform of choice with any of the freely available blacklists and with just a few simple Google searches, one can easily find many websites not included in those free blacklists. Many websites will easily load, exposing the fact that many of these blacklists are poorly maintained, and result in an unreliably poor degree of quality.

Not only are these blacklists highly inadequate, they are generally, all filled with errors, erroneous characters, blank spaces, formatting issues, capitalization mistakes,  etc, etc.

It is not uncommon for many administrators who fail to achieve an effective degree of content control using open source platforms, combined with low cost solutions, such as free domain blacklists. Many times concluding their efforts in frustration at the lack of efficacy and are forced to turn to much more expensive content control solutions from large vendors. This is why we believe that the world is primed and ready for a higher quality value added blacklist solution, and Squidblacklist.org intend’s to fill this gap.

This is why we founded Squidblacklist.org, as a paid service, which allows us to allocate our resources to pursue and continue producing blacklists of a much higher quality, with a level of sophistication in our production and validation processes that is seriously lacking in other, competing solutions..

Regardless of whether it is paid or gratis, we intend to beat our competition, as we continue to uphold our reputation as The Worlds Leading Publisher of Blacklists Tailored Specifically for use with Content Filering platforms.


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Gaming Blacklist For Content Filtering

Gaming Blacklist.

 

So you want a blocklist to use to control access to gaming related websites and content? Maybe you need to prevent students or employees from accessing such content to keep productivity up.

No problem, our gaming blacklist contains over 165,000+ gaming domain names.  Everything you could consider a game, from online poker websites, to video game consoles, board games, and everything in between our gaming blacklist is ready for production.


 

This list and others are currently available from Squidblacklist.org.  A valid username and password is required to download.

http://www.squidblacklist.org/downloads/squidblacklists/squid-gaming.tar.gz

 

SAMPLE:

This is just a small sample of the gaming blacklist, this port is formatted for use with Squid Proxy.

# Squid-Gaming: Blacklist compiled by SquidBlacklist.org                    -MADE IN USA-
#
# Note: This list is fully compatible and will work fine in combination with our other lists.
#
# FEATURES:
# Any sort of website that hosts online game or video game related content.
#
# Additions or amendments that should be made to this list, email us at submit@squidblacklist.org
#
# Blacklists by Squidblacklist.org  licensed under  Creative Commons Attribution 3.0 License.
# Based on a works located at http://www.squidblacklist.org .

.xboxonekaufen.net
.xboxonekeys.net
.xboxonekinect.net
.xboxonekinectone.net
.xboxonekopen.net
.xboxonelife.net
.xboxonelive.net
.xboxonemagazine.net
.xboxone-magazin.net
.xboxonemicrosoft.net
.xboxone.mobi
.xboxonemod.net
.xboxonemods.net
.xboxonemusic.net
.xbox-one.net
.xboxone.net
.xbox-one-news.net
.xboxoneofficial.net
.xboxoneombouw.net
.xboxone.org
.xboxoneoyunlari.net
.xboxoneoyun.net
.xbox-one-paris.net
.xboxoneplus.net
.xboxonepoints.net
.xboxone.pro
.xboxoneproducts.net
.xboxone-promotion.net
.xboxonerelease.net
.xboxonerepair.net
.xboxonerepairs.net
.xboxonerevealed.net
.xboxonereview.net
.xbox-onereviews.net
.xboxone-reviews.net
.xboxonereviews.net
.xboxone.ru
.xboxonesales.net
.xboxonescene.net
.xboxonesearch.net
.xboxonesecondterms.net
.xboxonesmartglass.net


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

HowTo: Using Domain Blacklists with PfSense & Squid Proxy for easy Content Filtering.

In the following blog entry we will be describing how to use Squid proxy Native ACL Blacklists from Squidblacklist.org in combination with Squid3x and PFsense Release 2.2.2.

pfsense Logo

Step 1.

Install Requisite Packages into Pfsense.

1. Install Squid3

2 Install Cron – We will be using this to automate update downloads for the blacklists using cron and fetch.

 

Step 2.

Next we will open a terminal and manually download a blacklist from Squidblacklist.org using fetch, we will download this file directly into the working directory for the Squid3 proxy installation using fetch. We are doing this to help you become more familiar with what is going on here.

 

Open a terminal on the pfsense box and navigate to the directory, which will be similar to the following location.

[2.2.2-RELEASE][admin@pfSense.local] cd /usr/pbi/squid-amd64/etc/squid

 

Now we will fetch the blacklist. ( you must have a valid Squidblacklist.org user account to download )

[2.2.2-RELEASE][admin@pfSense.local] fetch http://username:password@www.squidblacklist.org/downloads/squidblacklists/squid-porn.tar.gz

 

Now we will decompress the blacklist file with the following command string.

[2.2.2-RELEASE][admin@pfSense.local] tar -xvf squid-porn.tar.gz

 

Now that we have downloaded, and decompressed the blacklist file, squid-porn.acl, it is time to log into the pfSense administration panel and configure squid proxy to use the acl we just downloaded, and complete the process.

First, do not forget to allow access to the proxy for all subnets you intend to allow to have access. In the squid proxy configuration, navigate to the section titled acls and add the subnets you wish to allow access, and click apply, you will be required to manually restart the squid proxy for the changes to take effect using the buttons on the top right of this image in the administration panel.

pfsense-config-1

Next . and final step is to add the ACL rules to the admin panel for Squid proxy.

# acl porn dstdomain “/usr/pbi/squid-amd64/etc/squid/squid-porn.acl”
# http_access deny porn

pfsense web filter squid proxy blacklist

Now,  squid proxy will automatically begin to load the ACL blacklist, which is quite large, and may consume a considerable amount of cpu while the reload process is taking place, I recommend monitoring the cpu load on your pfSense router during this process so that you know when it is completed.  Dependong on how powerful your cpu is, it may take several minutes to load.

 

It is important to note that browsing through the proxy may be interrupted and temporarily unavailable during this loading process. This can be avoided by using a parent proxy to bypass the local when it becomes unavailable or unresponsive.  Other work arounds exist, such as updating during late night hours using a cron job similar to the next step we are about to describe.

To eliminate downtime, there are also many plugins available for Squid proxy  such as DansGuardian which also can be leveraged to elminate this problem, as Dansguardian is still available during reloading of blacklist acl files.  However, this howto is focused on Squid proxy Native ACL use with pfSense and Squid proxy, so we will save those options for another howto.

 

Final Step. – Automated Blacklist Updates for pfSense and Squid proxy  using Cron and Fetch.

 

Assuming you have already installed cron via the pfSense package manager, you can now open the administration panel for cron in the pfsense web administration panel.

pfsense web filter squid proxy blacklist acl

Enter the following scheduled task.

cd /usr/pbi/squid-amd64/etc/squid/ ; fetch http://username:password@www.squidblacklist.org/downloads/squidblacklists/squid-porn.tar.gz ; tar -xvf /usr/pbi/squid-amd64/etc/squid/squid-porn.tar.gz

 

pfsense-config-4

Hit save and voila,  updates scheduled for  download every 30 minutes, which  is a bit absurd, you should schedule these updates once per day, preferably after midnight.

 

 

You are now filtering content with pfsense and squid proxy using the worlds largest adult domain blacklist,  squid-porn.acl from Squidblacklist.org

 

squid proxy content filtering, k9 filter, adult block

 

NOTES AND ADVISORY:

 

Depending on your system resources, in particular , Processing power, this may add considerable time to rebooting/start ups because Squid will have to reload the blacklist every time you restart your pfSense box.  Something you should be aware of when using Squid Native ACL blacklists, it is dirty, but it does work. Using our lists with third party plugins under pfSense, such as Dansguardian or SquidGuard can easily be done with the same cron + fetch method, however, for other alternative methods, please see the following relevant links and material.
Update script that is mentioned in the legoclan howto can be downloaded at the link below.
http://squidblacklist.org/downloads/sblorg_updater.tar.gz


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Updating pfSense and Squidguard Blacklists with Squidblacklist.org

 

pfSense Logo

For a while we have been informing the public that yes, our lists are compatible with pfSense, DansGuardian, SquidGuard, HAProxy and just about a jillion other web filtering platforms you could possiblely think of. Its as easy as subscribing, and inserting the link to our shalla/urlblacklist compatible format archive found here http://www.squidblacklist.org/downloads/squidblacklists/squidblacklist.tar.gz

Untitled2

Authentication:
You will be required to authenticate with a valid username and password to download our works, which is no big deal really, most of these platforms are using wget or fetch to pull the blacklists, and both of these programs support basic authentication. Figuring it out should be trivial for the average user who knows how to google.


Alternatively, if you really wanted to, enabling automated updates for our standard acl formatted blacklists can also be done, but does require a little additional work, just some simple scripting really.  One of our members recently shared a link and a series of scripts with the intention that his works be shared with the community.

If you are interested in pursuing this method, below is a link to his original blog entry describing his efforts as well as direct links to download the scripts he has drafted. But for most people using the shalla/urlblacklist compatible formatted archive will be much easier because many of these programs and platforms are coded to accept that particular file structure.

Thank you to Paul Goreman for submitting his time and research.

http://paulgorman.org/blog/1395259268
http://legoclan.com/pfsense/
We have also hosted a backup copy of his update script.
http://blog.squidblacklist.org/downloads/sblorg_updater.tar.gz


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.