How To Update Blacklists For DansGuardian – Update Script

I just tossed together a simple bash script for updating Blacklists from Squidblacklist.org for DansGuardian under GNU/Linux or I suppose any platform with a bash shell would suffice, it isnt really that sophisticated, it is rather crude and gets the job done.

The blacklist update script for DansGuardian can be downloaded here. Remember that a valid username and password will be required for basic http authentication. To download these files one must subscribe. Get your own username and password today by subscribing to Squidblacklist.org here..

Here is the blacklist update script in plain text form.


#! /bin/bash
#
# DansGuardian Blacklist Updater Script v0.1 - Squidblacklist.org
# Thoughts or suggestions can be emailed to webmaster@squidblacklist.org
# You will need to change the wget line with your user account for authentication
#
# EXAMPLE:
# wget --http-user=USERNAME --http-password=PASSHERE --auth-no-challenge http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-prime.tar.gz
#
# Use this script to download and update blacklists for squid3 proxy
# You may need to edit this for your specific environment.
# you may optionaly wish to crontab this job so it will update at regular intervals
#
# Create the directories you want
# EXAMPLE: 'mkdir /etc/dansguardian/lists/blacklists/porn'
#
# Place this script within /etc/dansguardian/ directory
# Next, chmod +x squid-update.sh .That will make the script executable.
#
# nano /etc/crontab and add the following line
# 01 0 * * * root /etc/dansguardian/dg-update.sh
#
# URLS for DG ACL blacklists Are listed below.
#
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-all.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-prime.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-malicious.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-usg.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-piracy.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-porn.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-dating.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-ads.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-proxies.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-gaming.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-smedia.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-gambling.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-cp.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-blasphemy.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-file.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-image.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-video.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-pharma-rx.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-new-tlds.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-chanology.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-dyn.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-freeweb.tar.gz
# http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-racism.tar.gz
#
# Happy Updating.

echo Beginning squidblacklist.org Dansguardian Blacklist Update procedure... ;
cd /etc/dansguardian/ ;
echo Downloading blacklists...
# Easy to download blacklists, decompress and move to the proper directory.
wget --http-user=USERNAME --http-password=PASSHERE --auth-no-challenge http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-porn.tar.gz ;
tar -xvf dg-porn.tar.gz ;
mv dg-porn.acl /etc/dansguardian/lists/blacklists/porn/domains ;
wget --http-user=USERNAME --http-password=PASSHERE --auth-no-challenge http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-ads.tar.gz ;
tar -xvf dg-ads.tar.gz ;
mv dg-ads.acl /etc/dansguardian/lists/blacklists/ads/domains ;
wget --http-user=USERNAME --http-password=PASSHERE --auth-no-challenge http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-malicious.tar.gz ;
tar -xvf dg-malicious.tar.gz ;
mv dg-malicious.acl /etc/dansguardian/lists/blacklists/malicious/domains ;
wget --http-user=USERNAME --http-password=PASSHERE --auth-no-challenge http://www.squidblacklist.org/downloads/squidblacklists/dg/dg-chanology.tar.gz ;
tar -xvf dg-chanology.tar.gz ;
mv dg-chanology.acl /etc/dansguardian/lists/blacklists/chanology/domains ;
echo Cleaning up temporary files... ;
rm *.tar.gz ;
echo Reloading Dansguardian Service... ;
service dansguardian stop ;
service dansguardian start ;
echo Done.


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Mikrotik RouterOS Blacklist Validation Testing & Compatibility Chart Update Posted.

logo_new800

CCR1036-12G-4S & Porn Blacklist
CCR1036-12G-4S & Porn Blacklist

We have retooled our backend logic which has had a dramatic impact on the size and contents of our blacklists, just another reason for us to retest the RouterBoards we have available to see which blacklists are able to run on them and which ones cannot.  We posted the results of our findings in the following chart.

Mikrotik 951G-2HnD
Mikrotik 951G-2HnD

We also added two new routers to the chart, the map2n and 951G-2HnD.  Which we recently received for testing. We have prepared a compatibility chart for your review.

Mikrotik RBmAP2n
Mikrotik RBmAP2

 

 

Mikrotik compatible conversions of our blacklists are available to all subscribers in .rsc import script formats for both Mikrotik Web Proxy Access Blacklist and Mikrotik DNS Access Blacklist. Subscribe Now.

Test it for yourself! Download this sample Mikrotik Web Proxy blacklist tik-ads.rsc


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

India Government Porn Blacklist Leaked.

Internet-Censorship-IndiaA Botched Attempt to block Porn and  CP.  Files Leaked…. 


 

The Indian Government ordered local internet service providers to begin blocking a list of 857 adult websites that purportedly hosted or linked illegal adult material as well as otherwise legal porn. The broad blocking order went further than targeting dedicated porn sites alone though. Many websites that should not have been a part of the blacklist block were. Including  torrent sites like kickass.to & h33t.to were listed, as well as 9Gag, Liveleak and CollegeHumor.

Untitled

In  July 2015,  millions of Indian Internet users starting to notice that favorite sites werent  accessible. On a Friday, the Government ordered local ISP’s to filter content access to a the blacklist of 857 websites, including many of the top adult sites.

“Your requested URL has been blocked as per the directions received from Department of Telecommunications, Government of India,” was the warning banner users got to see instead of the pages they intended to visit.

The move has sparked outrage among the public, who condemn the Government for censoring the Internet without proper cause. According to the court order the sites are being blocked because they threaten the morality and decency of Indians, which a local official has now confirmed.

“Free and open access to porn websites has been brought under check. We don’t want them to become a social nuisance,” a spokesman at the Department of Telecommunications told Reuters.

The Government order is quite broad, and not just because of the high number of domain names involved. A leaked copy which list all of the affected domains reveals some unsuspected entries.

For example, the list contains two of the largest torrent sites, Kickass.to and H33t.to. The first is now operating under the new Kat.cr domain name and the latter site is down, so the effects of the blockade are minimal.

 

While blocking these torrent sites may be justified as both sites do link to pornographic content, the same can’t really be said for CollegeHumor and 9Gag, which are also on the blacklist.

The same goes for Liveleak, which has plenty of ‘immoral’ videos but isn’t really known for its vast amounts of porn. Finally, the list also includes nonvegjokes.com, a site specializing in dirty jokes.

The blocking order was issued under Rule 12 of the local Information Technology Rules, which allows the Government to block access to sites that are deemed to violate the integrity or security of India.

The Government still has to justify its blocking request before the end of the month. If those arguments prove insufficient, the court order may be overturned again. In the meantime, the interest in circumvention tools such as VPN services and proxy sites is expected to skyrocket.

Update August 5: The porn block has been lifted already. But the blacklist was leaked, see below for a link to download the pdf.

 

Leaked Indian Government Blacklist file is available for download –> 2015-07-31-dot-block-order-decency.pdf


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Free blacklists suck , and heres why.

Squidblacklist.org – Blacklisting Has Evolved.

Normally I would believe free is great, free as in, gratis. However, when it comes to the blacklists freely available online, specifically, blacklists tailored for consumption as an important web filtering component used in various content filter hardware platforms and software applications. The free solutions just dont cut it. And If you as an administrator can find as many holes within just a few simple Google searches using the free blacklists, just imagine how easy it will be for a determined end user on your network to find and access unauthorized content.

snakeoil
One can easily load up their content control platform of choice with any of the freely available blacklists and with just a few simple Google searches, one can easily find many websites not included in those free blacklists. Many websites will easily load, exposing the fact that many of these blacklists are poorly maintained, and result in an unreliably poor degree of quality.

Not only are these blacklists highly inadequate, they are generally, all filled with errors, erroneous characters, blank spaces, formatting issues, capitalization mistakes,  etc, etc.

It is not uncommon for many administrators who fail to achieve an effective degree of content control using open source platforms, combined with low cost solutions, such as free domain blacklists. Many times concluding their efforts in frustration at the lack of efficacy and are forced to turn to much more expensive content control solutions from large vendors. This is why we believe that the world is primed and ready for a higher quality value added blacklist solution, and Squidblacklist.org intend’s to fill this gap.

This is why we founded Squidblacklist.org, as a paid service, which allows us to allocate our resources to pursue and continue producing blacklists of a much higher quality, with a level of sophistication in our production and validation processes that is seriously lacking in other, competing solutions..

Regardless of whether it is paid or gratis, we intend to beat our competition, as we continue to uphold our reputation as The Worlds Leading Publisher of Blacklists Tailored Specifically for use with Content Filering platforms.


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now vvailable.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.

Survey finds the largest single group of malicious domains, about one-third of the total, fall under the .biz TLD.

 

Domains and domain names are fundamental to the operation of the Internet. They provide a hierarchy of unique identifiers that guide traffic across the Web and identify websites, servers and other resources. However, in the form of malicious domains, they are a basic tool in the hands of cybercriminals.

As with other aspects of computer security, there are no silver bullets for protecting against malicious domains. However, understanding domain names can help firms and individual employees guard themselves against attacks.

Domain names form a hierarchy of domains and subdomains. For example, marketing.companyname.com is a subdomain of companyname.com. In turn, this is one of the many subdomains of the familiar top-level domain (TLD) com. It is typical to type a period in front of TLD names, as in .com, though the period is technically a separator, not part of the TLD itself.

Looking at the TLD Landscape

A recent IBM security intelligence and research report, “The .Bizness Behind Malicious Domain Names,” looks at malicious domains and their distribution across the overall domain name structure, particularly the various TLDs, such as .com, .net and .org. The report builds on research from IBM security partner CrowdStrike, which keeps track of ongoing malicious activity online.

CrowdStrike’s survey found that the largest single group of malicious domains, about one-third of the total, fall under the TLD .biz. This TLD was created specifically for business use in 2000 to alleviate overcrowding within the original .com TLD (which dates back to the 1980s).

It should be emphasized that most .biz websites are perfectly legitimate businesses. However, the difficulty of policing an entire global TLD has let cybercriminals register domain names that often mimic well-known, legitimate domains, such as the websites of major firms. Most other malicious domains fall under the long-established .org, .com and .net TLDs. Some have country-specific TLDs, often to either target victims in those countries or disguise their own origins.

Protecting Against Malicious Domains

The best protection against malicious domains is user awareness. For example, a domain name such as companyname.com.biz should trigger immediate suspicion. It is deceptively trying to masquerade as a subdomain of the .com TLD when, in fact, it is a subdomain of .biz.

Overly clever spellings, such as wind0wsupdates.com, should also raise a red flag. Unfortunately, all too many users have “domain blindness” and pay little or no attention to where they are actually going online. Moreover, mobile devices such as smartphones may hide address bars in order to conserve limited screen space.

Firms and other organizations can use a brute-force method to protect against some malicious domains by blocking entire TLDs. If, for example, a company has no business partners with a .biz subdomain, it can bar all connections to .biz. Individual exceptions can then be white-listed.

However, this is not practical for TLDs such as .com or .org. Along with encouraging user awareness, the best protection is provided by a security partner that can provide up-to-date listings of malicious domains to avoid.


Subscribe Today – Paypal or Credit Card Accepted.

Flat rate subscription. Select a membership option & subscribe.



Select Payment Option



  • You will be issued a username and password.
  • You will be granted access to our member area.
  • 5 Year Membership Option now available.
  • For lifetime membership options click here.clipart
  • Contact us if you would like a pre order invoice.

Disclaimer: All sales are final, we do not issue refunds. Cancel your subscription anytime.