1 # dns based ad/abuse domain blocking
4 A lot of people already use adblocker plugins within their desktop browsers, but what if you are using your (smart) phone, tablet, watch or any other wlan gadget...getting rid of annoying ads, trackers and other abuse sites (like facebook ;-) is simple: block them with your router. When the dns server on your router receives dns requests, you will sort out queries that ask for the resource records of ad servers and return a simple 'NXDOMAIN'. This is nothing but **N**on-e**X**istent Internet or Intranet domain name, if domain name is unable to resolved using the dns server, a condition called the 'NXDOMAIN' occurred.
7 * support of the following domain block list sources (free for private usage, for commercial use please check their individual licenses):
8 * [adaway](https://adaway.org)
9 * => infrequent updates, approx. 400 entries (enabled by default)
10 * [adguard](https://adguard.com)
11 * => numerous updates on the same day, approx. 12.000 entries
13 * => static local blacklist, located by default in '/etc/adblock/adblock.blacklist'
14 * [disconnect](https://disconnect.me)
15 * => numerous updates on the same day, approx. 6.500 entries (enabled by default)
16 * [dshield](http://dshield.org)
17 * => daily updates, approx. 4.500 entries
18 * [feodotracker](https://feodotracker.abuse.ch)
19 * => daily updates, approx. 0-10 entries
20 * [hphosts](https://hosts-file.net)
21 * => monthly updates, approx. 50.000 entries
22 * [malwaredomains](http://malwaredomains.com)
23 * => daily updates, approx. 16.000 entries
24 * [malwaredomainlist](http://www.malwaredomainlist.com)
25 * => daily updates, approx. 1.500 entries
26 * [openphish](https://openphish.com)
27 * => numerous updates on the same day, approx. 1.800 entries
28 * [ransomware tracker](https://ransomwaretracker.abuse.ch)
29 * => daily updates, approx. 150 entries
30 * [reg_cn](https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt)
31 * => regional blocklist for China, daily updates, approx. 1.600 entries
32 * [reg_id](https://easylist-downloads.adblockplus.org/abpindo+easylist.txt)
33 * => regional blocklist for Indonesia, daily updates, approx. 800 entries
34 * [reg_nl](https://easylist-downloads.adblockplus.org/easylistdutch+easylist.txt)
35 * => regional blocklist for the Netherlands, weekly updates, approx. 1300 entries
36 * [reg_pl](http://adblocklist.org)
37 * => regional blocklist for Poland, daily updates, approx. 50 entries
38 * [reg_ro](https://easylist-downloads.adblockplus.org/rolist+easylist.txt)
39 * => regional blocklist for Romania, weekly updates, approx. 600 entries
40 * [reg_ru](https://code.google.com/p/ruadlist)
41 * => regional blocklist for Russia, weekly updates, approx. 2.000 entries
42 * [securemecca](http://www.securemecca.com)
43 * => infrequent updates, approx. 25.000 entries
44 * [shallalist](http://www.shallalist.de) (categories "adv" "costtraps" "spyware" "tracker" "warez" enabled by default)
45 * => daily updates, approx. 32.000 entries (a short description of all shallalist categories can be found [online](http://www.shallalist.de/categories.html))
46 * [spam404](http://www.spam404.com)
47 * => infrequent updates, approx. 5.000 entries
48 * [sysctl/cameleon](http://sysctl.org/cameleon)
49 * => weekly updates, approx. 21.000 entries
50 * [whocares](http://someonewhocares.org)
51 * => weekly updates, approx. 12.000 entries
52 * [winhelp](http://winhelp2002.mvps.org)
53 * => infrequent updates, approx. 15.000 entries
54 * [winspy](https://github.com/crazy-max/WindowsSpyBlocker)
55 * => infrequent updates, approx. 120 entries
56 * [yoyo](http://pgl.yoyo.org/adservers)
57 * => weekly updates, approx. 2.500 entries (enabled by default)
58 * [zeus tracker](https://zeustracker.abuse.ch)
59 * => daily updates, approx. 440 entries
60 * zero-conf like automatic installation & setup, usually no manual changes needed
61 * simple but yet powerful adblock engine: adblock does not use error prone external iptables rulesets, http pixel server instances and things like that
62 * supports five different dns backends / block list formats: dnsmasq, unbound, named (bind), kresd and dnscrypt-proxy
63 * automatically selects uclient-fetch or wget as download utility (other tools like curl or aria2c are supported as well)
64 * provides 'http only' mode without installed ssl library for all non-SSL block list sources
65 * supports a wide range of router modes, even AP modes are supported
66 * full IPv4 and IPv6 support
67 * provides top level domain compression ('tld compression'), this feature removes thousands of needless host entries from the block list and lowers the memory footprint for the dns backends
68 * each block list source will be updated and processed separately
69 * block list source parsing by fast & flexible regex rulesets
70 * overall duplicate removal in central block list (adb_list.overall)
71 * additional whitelist for manual overrides, located by default in /etc/adblock/adblock.whitelist
72 * quality checks during block list update to ensure a reliable dns backend service
73 * minimal status & error logging to syslog, enable debug logging to receive more output
74 * procd based init system support (start/stop/restart/reload/suspend/resume/query/status)
75 * procd network interface trigger support or classic time based startup
76 * suspend & resume adblock actions temporarily without block list reloading
77 * output comprehensive runtime information via LuCI or via 'status' init command
78 * query function to quickly identify blocked (sub-)domains, e.g. for whitelisting
80 * optional: force dns requests to local resolver
81 * optional: force overall sort / duplicate removal for low memory devices (handle with care!)
82 * optional: 'manual mode' to re-use blocklist backups during startup, get fresh lists only via manual reload or restart action
83 * optional: automatic block list backup & restore, they will be used in case of download errors or during startup in manual mode
84 * optional: add new adblock sources on your own via uci config
87 * [LEDE project](https://www.lede-project.org), tested with latest stable release (LEDE 17.01) and with current LEDE snapshot
88 * a usual setup with an enabled dns backend at minimum - dump AP modes without a working dns backend are _not_ supported
90 * to support all blocklist sources a full version (with ssl support) of 'wget', 'uclient-fetch' with one of the 'libustream-*' ssl libraries, 'aria2c' or 'curl' is required
91 * for limited devices with real memory constraints, adblock provides also a 'http only' option and supports wget-nossl and uclient-fetch (without libustream-ssl) as well
92 * for more configuration options see examples below
94 ## Installation & Usage
95 * install 'adblock' (_opkg install adblock_)
96 * at minimum configure the appropriate dns backend ('dnsmasq' by default) and enable the adblock service in _/etc/config/adblock_
97 * control the adblock service manually with _/etc/init.d/adblock_ start/stop/restart/reload/suspend/resume/status or use the LuCI frontend
99 ## LuCI adblock companion package
100 * for easy management of the various block list sources and all other adblock options you should use the provided LuCI frontend
101 * install 'luci-app-adblock' (_opkg install luci-app-adblock_)
102 * the application is located in LuCI under 'Services' menu
105 * **runtime information:** the adblock status is available via _/etc/init.d/adblock status_ (see example below)
106 * **debug logging:** for script debugging please set the config option 'adb\_debug' to '1' and check the runtime output with _logread -e "adblock"_
107 * **storage expansion:** to process and store all block list sources at once it might helpful to enlarge your temp directory with a swap partition => see [openwrt wiki](https://wiki.openwrt.org/doc/uci/fstab) for further details
108 * **add white- / blacklist entries:** add domain white- or blacklist entries to always-allow or -deny certain (sub) domains, by default both lists are empty and located in _/etc/adblock_. Please add one domain per line - ip addresses, wildcards & regex are _not_ allowed (see example below)
109 * **backup & restore block lists:** enable this feature, to restore automatically the latest compressed backup of your block lists in case of any processing error (e.g. a single block list source is not available during update). Please use an (external) solid partition and _not_ your volatile router temp directory for this
110 * **scheduled list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below)
111 * **change startup behaviour:** by default the startup will be triggered by the 'wan' procd interface trigger. Choose 'none' to disable automatic startups, 'timed' to use a classic timeout (default 30 sec.) or select another trigger interface.
112 * **suspend & resume adblocking:** to quickly switch the adblock service 'on' or 'off', simply use _/etc/init.d/adblock [suspend|resume]_
113 * **domain query:** to query the active block list for a specific domain, please run _/etc/init.d/adblock query `<DOMAIN>`_ (see example below)
114 * **add new list sources:** you could add new block list sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
115 * **disable active dns probing in windows 10:** to prevent a yellow exclamation mark on your internet connection icon (which wrongly means connected, but no internet), please change the following registry key/value from "1" to "0" _HKLM\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet\EnableActiveProbing_
117 ## Further adblock config options
118 * usually the pre-configured adblock setup works quite well and no manual config overrides are needed, all listed options apply to the 'global' config section:
119 * adb\_enabled => main switch to enable/disable adblock service (default: '0', disabled)
120 * adb\_debug => enable/disable adblock debug output (default: '0', disabled)
121 * adb\_dns => select the dns backend for your environment: 'dnsmasq', 'unbound', 'named', 'kresd' or 'dnscrypt-proxy' (default: 'dnsmasq')
122 * adb\_dnsdir => target directory for the generated blocklist 'adb_list.overall' (default: not set, use dns backend default)
123 * adb\_trigger => set the startup trigger to a certain interface, to 'timed' or to 'none' (default: 'wan')
124 * adb\_triggerdelay => additional trigger delay in seconds before adblock processing begins (default: '1')
125 * adb\_fetch => full path to a dedicated download utility, see example below (default: not set, use wget default)
126 * adb\_fetchparm => options for the download utility, see example below (default: not set, use wget default options)
127 * adb\_forcedns => force dns requests to local resolver (default: '0', disabled)
128 * adb\_forcesrt => force overall sort on low memory devices with less than 64 MB RAM (default: '0', disabled)
129 * adb\_manmode => do not automatically update block lists during startup, use backups instead (default: '0', disabled)
132 **change default dns backend to 'unbound':**
134 Adblock deposits the sorted and filtered block list (adb_list.overall) in '/var/lib/unbound' where unbound can find them in its jail.
135 If you use manual configuration for unbound, then just include the following line in your 'server' clause:
137 include: "/var/lib/unbound/adb_list.overall"
140 **change default dns backend to 'named' (bind):**
142 Adblock deposits the sorted and filtered block list (adb_list.overall) in '/var/lib/bind' where bind can find them.
143 To use the block list please modify the following bind configuration files:
145 change '/etc/bind/named.conf', in the 'options' namespace add:
146 response-policy { zone "rpz"; };
148 and at the end of the file add:
151 file "/etc/bind/db.rpz";
152 allow-query { none; };
153 allow-transfer { none; };
156 create the new file '/etc/bind/db.rpz' and add:
159 @ SOA localhost. root.localhost. (1 6h 1h 1w 2h)
162 $INCLUDE /var/lib/bind/adb_list.overall
165 **change default dns backend to 'kresd':**
167 The knot-resolver (kresd) is only available on turris omnia devices. Currently there's no package for kresd in the official LEDE / OpenWrt package repository.
168 Adblock deposits the sorted and filtered block list (adb_list.overall) in '/etc/kresd' where kresd can find them.
169 To use the block list please create/modify the following kresd configuration files:
172 edit '/etc/config/resolver' and change / uncomment the following options:
174 list rpz_file '/etc/kresd/adb_list.overall'
177 edit '/etc/config/resolver' and change / uncomment the following options:
179 option include_config '/etc/kresd/custom.conf'
181 create '/etc/kresd/custom.conf' and add:
182 policy.add(policy.rpz(policy.DENY, '/etc/kresd/adb_list.overall'))
183 policy.add(policy.all(policy.FORWARD({'8.8.8.8', '8.8.4.4'})))
186 **change default dns backend to 'dnscrypt-proxy':**
188 Adblock deposits the sorted and filtered block list (adb_list.overall) by default in '/tmp' where DNSCrypt-Proxy can find them.
189 The blacklist option is not supported by default, because DNSCrypt-Proxy is compiled without plugins support.
190 Take a custom LEDE build with plugins support to use this feature:
192 edit '/etc/config/dnscrypt-proxy' and add the following option per dnscrypt-proxy instance:
193 list blacklist 'domains:/tmp/adb_list.overall'
196 **configuration for different download utilities:**
199 option adb_fetch '/usr/bin/wget'
200 option adb_fetchparm '--quiet --no-cache --no-cookies --max-redirect=0 --timeout=10 --no-check-certificate -O'
203 option adb_fetch '/usr/bin/aria2c'
204 option adb_fetchparm '-q --timeout=10 --allow-overwrite=true --auto-file-renaming=false --check-certificate=false -o'
207 option adb_fetch '/bin/uclient-fetch'
208 option adb_fetchparm '-q --timeout=10 --no-check-certificate -O'
211 option adb_fetch '/usr/bin/curl'
212 option adb_fetchparm '-s --connect-timeout 10 --insecure -o'
215 **receive adblock runtime information:**
217 /etc/init.d/adblock status
218 ::: adblock runtime information
219 + adblock_status : enabled
220 + adblock_version : 3.0.0
221 + blocked_domains : 37406
222 + fetch_utility : wget (built-in)
223 + dns_backend : kresd (/etc/kresd)
224 + last_rundate : 08.09.2017 21:21:21
225 + system_release : OpenWrt omnia 15.05
228 **cronjob for a regular block list update (/etc/crontabs/root):**
230 0 06 * * * /etc/init.d/adblock reload
233 **blacklist entry (/etc/adblock/adblock.blacklist):**
237 This entry blocks the following (sub)domains:
238 http://ads.example.com/foo.gif
239 http://server1.ads.example.com/foo.gif
240 https://ads.example.com:8000/
242 This entry does not block:
243 http://ads.example.com.ua/foo.gif
247 **whitelist entry (/etc/adblock/adblock.whitelist):**
251 This entry removes the following (sub)domains from the block list:
255 This entry does not remove:
260 **query active block list for a certain (sub-)domain, e.g. for whitelisting:**
262 /etc/init.d/adblock query example.www.doubleclick.net
263 root@turris:~# /etc/init.d/adblock query example.www.doubleclick.net
264 ::: max. ten results for domain 'example.www.doubleclick.net'
266 ::: max. ten results for domain 'www.doubleclick.net'
268 ::: max. ten results for domain 'doubleclick.net'
271 The query function checks against the submitted (sub-)domain and recurses automatically to the upper top level domain.
272 For every (sub-)domain it returns the first ten relevant results.
273 In the example above whitelist "doubleclick.net" to free the submitted domain.
276 **add a new block list source:**
279 example: https://easylist-downloads.adblockplus.org/rolist+easylist.txt
280 adblock already supports an easylist source, called 'ruadlist'. To add the additional local easylist
281 as a new source, copy the existing config source 'ruadlist' section and change only
282 the source name, the url and the description - that's all!
284 config source 'rolist'
286 option adb_src 'https://easylist-downloads.adblockplus.org/rolist+easylist.txt'
287 option adb_src_rset '{FS=\"[|^]\"} \$0 ~/^\|\|([A-Za-z0-9_-]+\.){1,}[A-Za-z]+\^$/{print tolower(\$3)}'
288 option adb_src_desc 'focus on romanian ad related domains plus generic easylist additions, weekly updates, approx. 600 entries'
291 to add a really new source with different domain/host format you have to write a suitable
292 awk one-liner on your own, so basic awk skills are needed. As a starting point check the already
293 existing awk strings (adb_src_rset) in adblock config, maybe you need only small changes for your individual list.
294 Download the desired list and test your new awk string locally with:
295 cat new.list | awk 'fs__individual search__search core__result'
296 'fs' => field separator (optional)
297 'individual search' => individual search part to filter out needless list information
298 'search core' => always '([A-Za-z0-9_-]+\.){1,}[A-Za-z]+', this is part of all list sources and should be unchanged
299 'result' => always '{print tolower(\$n)}', only the output column 'n' may vary
300 the output result should be a sequential list with one domain/host per line - nothing more.
302 If your awk one-liner works quite well, add a new source section in adblock config and test your new source
306 Please join the adblock discussion in this [forum thread](https://forum.lede-project.org/t/adblock-2-x-support-thread/507) or contact me by mail <dev@brenken.org>
309 * stop all adblock related services with _/etc/init.d/adblock stop_
310 * optional: remove the adblock package (_opkg remove adblock_)