Web filtering with RouterOS web proxy leveraging domain blacklists – how to.

 

Web filtering with a RouterOS enabled device leveraging domain blacklists from Squidblacklist.org is not difficult to achieve.  By simply converting our existing blacklists that we publish for other platforms, into a Mikrotik *.rsc script , we can then reliably filter web traffic using a RouterOS enabled device, as well as many low cost RouterBoard devices.

 

 

What does a RouterOS web proxy blacklist look like?

Piracy rsc blacklist.

Piracy rsc blacklist script.

 

I will quickly walk you through a brief illustrated description of how one would upload a blacklist script via winbox and schedule its execution on an RB750gl. Download  and test drive a copy of this  blacklist from the links we have provided at the bottom of this article.

 

Upload the blacklist via winbox (drag n drop)

Upload blacklist rsc

Uploading the piracy blacklist rsc via winbox.

 

Of course in production you could and likely would be using automated procedures to do this, using ftp, ssh, or whatever you like for file transfer,  but for this demonstration we are just going to drop and drop.

 

 Open the system scheduler and create a task.

Create task in system scheduler

Create a task.

Here we create a task in system scheduler and pick a time to execute it.  In this case, we are running the following  ‘import tik-piracy.rsc’  command.

 

RB750gl CPU load during import and duration.

Cpu load during run, RB750g

Cpu load during tik-piracy run, RB750g.

 

It is important for me to point out that the cpu on this RB750GL was under 100% load during this import process, and lasted approximately 2-3 minutes in duration.  Around 14,000 domains were loaded from this particular blacklist.

 

System log after blacklist script run completion.

RouterOS Syslog after loading blacklist

RouterOS syslog after loading piracy blacklist.

 

RouterOS Web Proxy access list after blacklist import.

Access list after Loading.

RouterOS web proxy access list after loading piracy blacklist.

Here we can see over 14,000 domains successfully imported into the RB750gl’s web proxy access list, which is now ready to filter against piracy related domains.

System Requirements – Memory Limitations

tikchart

Blacklist compatibility chart.

It is important for us to mention that the system memory, ram usage, is very high using our blacklists, in this example we used one of our smaller size lists, and most of our blacklists are comparable in size and line count, however. The most sought after lists that we publish will not run on a low cost Routerboard device such as the RB750gl, and would result in memory exhaustion, a kernel panic  and thereafter a resulting watchdog timer reboot.  If you attempted to load more than one of these blacklists on a small device such as an RB411/532/133/112/950x type device, this failure would likely be the result. Therefore, we recommend that an x86/x64 PC based platform be used for serious web filtering purposes using RouterOS web proxy.  Alternatively,  one of the more sophisticated, higher end Mikrotik RouterBoard platforms might also suffice.  RouterOS has been proven to have an inherent flaw, or set of flaws that make running our larger lists, porn, proxies, malicious, and prime difficult.  A minimum of 6gb ram is required before attempting to run these lists, and a recommended 8gb. Running these four blacklists on a RouterOS device should be considered experimental until Mikrotik fixes the problems challenging RouterOS.

Obtaining blacklists for RouterOS Web proxy.

These blacklists and more are available for download to our members immediately. A subscription to squidblacklist.org is required.

 Footnotes:


 

  • Blacklist immediate availability from squidblacklist.org for all subscribed members.
  • A Routerboard compatibility chart can be found here.
  • A sample RouterOS web proxy blacklist is available for download here.
  • Mikrotik blacklist conversion tool for windows is available for download gratis.
  • Update script examples are available for download.
  • If you have any questions contact us.

Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 3 Year Membership Option now available.
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

4 Replies to “Web filtering with RouterOS web proxy leveraging domain blacklists – how to.”

  1. Pingback: idiotonline

Leave a Reply

Your email address will not be published. Required fields are marked *

*