Merge pull request #20467 from tobiaspc/njalla
[feed/packages.git] / net / banip / files / README.md
1 <!-- markdownlint-disable -->
2
3 # banIP - ban incoming and outgoing IP addresses/subnets via sets in nftables
4
5 ## Description
6 IP address blocking is commonly used to protect against brute force attacks, prevent disruptive or unauthorized address(es) from access or it can be used to restrict access to or from a particular geographic area — for example. Further more banIP scans the log file via logread and bans IP addresses that make too many password failures, e.g. via ssh.
7
8 ## Main Features
9 * banIP supports the following fully pre-configured domain blocklist feeds (free for private usage, for commercial use please check their individual licenses).
10 **Please note:** the columns "WAN-INP", "WAN-FWD" and "LAN-FWD" show for which chains the feeds are suitable in common scenarios, e.g. the first entry should be limited to the LAN forward chain - see the config options 'ban\_blockinput', 'ban\_blockforwardwan' and 'ban\_blockforwardlan' below.
11
12 | Feed | Focus | WAN-INP | WAN-FWD | LAN-FWD | Information |
13 | :------------------ | :----------------------------- | :-----: | :-----: | :-----: | :----------------------------------------------------------- |
14 | adaway | adaway IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
15 | adguard | adguard IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
16 | adguardtrackers | adguardtracker IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
17 | antipopads | antipopads IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
18 | asn | ASN IPs | | | x | [Link](https://asn.ipinfo.app) |
19 | backscatterer | backscatterer IPs | x | x | | [Link](https://www.uceprotect.net/en/index.php) |
20 | bogon | bogon prefixes | x | x | | [Link](https://team-cymru.com) |
21 | country | country blocks | x | x | | [Link](https://www.ipdeny.com/ipblocks) |
22 | cinsscore | suspicious attacker IPs | x | x | | [Link](https://cinsscore.com/#list) |
23 | darklist | blocks suspicious attacker IPs | x | x | | [Link](https://darklist.de) |
24 | debl | fail2ban IP blacklist | x | x | | [Link](https://www.blocklist.de) |
25 | doh | public DoH-Provider | | | x | [Link](https://github.com/dibdot/DoH-IP-blocklists) |
26 | drop | spamhaus drop compilation | x | x | | [Link](https://www.spamhaus.org) |
27 | dshield | dshield IP blocklist | x | x | | [Link](https://www.dshield.org) |
28 | edrop | spamhaus edrop compilation | x | x | | [Link](https://www.spamhaus.org) |
29 | feodo | feodo tracker | x | x | x | [Link](https://feodotracker.abuse.ch) |
30 | firehol1 | firehol level 1 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level1) |
31 | firehol2 | firehol level 2 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level2) |
32 | firehol3 | firehol level 3 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level3) |
33 | firehol4 | firehol level 4 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level4) |
34 | greensnow | suspicious server IPs | x | x | x | [Link](https://greensnow.co) |
35 | iblockads | Advertising IPs | | | x | [Link](https://www.iblocklist.com) |
36 | iblockspy | Malicious spyware IPs | x | x | x | [Link](https://www.iblocklist.com) |
37 | myip | real-time IP blocklist | x | x | | [Link](https://myip.ms) |
38 | nixspam | iX spam protection | x | x | | [Link](http://www.nixspam.org) |
39 | oisdbig | OISD-big IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
40 | oisdnsfw | OISD-nsfw IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
41 | oisdsmall | OISD-small IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
42 | proxy | open proxies | x | | | [Link](https://iplists.firehol.org/?ipset=proxylists) |
43 | ssbl | SSL botnet IPs | x | x | x | [Link](https://sslbl.abuse.ch) |
44 | stevenblack | stevenblack IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
45 | talos | talos IPs | x | x | | [Link](https://talosintelligence.com/reputation_center) |
46 | threat | emerging threats | x | x | x | [Link](https://rules.emergingthreats.net) |
47 | threatview | malicious IPs | x | x | x | [Link](https://threatview.io) |
48 | tor | tor exit nodes | x | x | x | [Link](https://github.com/SecOps-Institute/Tor-IP-Addresses) |
49 | uceprotect1 | spam protection level 1 | x | x | | [Link](http://www.uceprotect.net/en/index.php) |
50 | uceprotect2 | spam protection level 2 | x | x | | [Link](http://www.uceprotect.net/en/index.php) |
51 | uceprotect3 | spam protection level 3 | x | x | | [Link](http://www.uceprotect.net/en/index.php) |
52 | urlhaus | urlhaus IDS IPs | x | x | | [Link](https://urlhaus.abuse.ch) |
53 | urlvir | malware related IPs | x | x | x | [Link](https://iplists.firehol.org/?ipset=urlvir) |
54 | webclient | malware related IPs | x | x | x | [Link](https://iplists.firehol.org/?ipset=firehol_webclient) |
55 | voip | VoIP fraud blocklist | x | x | | [Link](https://voipbl.org) |
56 | yoyo | yoyo IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
57
58 * zero-conf like automatic installation & setup, usually no manual changes needed
59 * all sets are handled in a separate nft table/namespace 'banIP'
60 * full IPv4 and IPv6 support
61 * supports nft atomic set loading
62 * supports blocking by ASN numbers and by iso country codes
63 * supports local allow- and blocklist (IPv4, IPv6, CIDR notation or domain names)
64 * auto-add the uplink subnet to the local allowlist
65 * provides a small background log monitor to ban unsuccessful login attempts in real-time
66 * auto-add unsuccessful LuCI, nginx, Asterisk or ssh login attempts to the local blocklist
67 * fast feed processing as they are handled in parallel as background jobs
68 * per feed it can be defined whether the wan-input chain, the wan-forward chain or the lan-forward chain should be blocked (default: all chains)
69 * automatic blocklist backup & restore, the backups will be used in case of download errors or during startup
70 * automatically selects one of the following download utilities with ssl support: aria2c, curl, uclient-fetch or wget
71 * supports an 'allowlist only' mode, this option restricts internet access from/to a small number of secure websites/IPs
72 * deduplicate IPs accross all sets (single IPs only, no intervals)
73 * provides comprehensive runtime information
74 * provides a detailed set report
75 * provides a set search engine for certain IPs
76 * feed parsing by fast & flexible regex rulesets
77 * minimal status & error logging to syslog, enable debug logging to receive more output
78 * procd based init system support (start/stop/restart/reload/status/report/search)
79 * procd network interface trigger support
80 * ability to add new banIP feeds on your own
81
82 ## Prerequisites
83 * **[OpenWrt](https://openwrt.org)**, latest stable release or a snapshot with nft/firewall 4 support
84 * a download utility with SSL support: 'wget', 'uclient-fetch' with one of the 'libustream-*' SSL libraries, 'aria2c' or 'curl' is required
85 * a certificate store like 'ca-bundle', as banIP checks the validity of the SSL certificates of all download sites by default
86 * for E-Mail notifications you need to install and setup the additional 'msmtp' package
87
88 **Please note the following:**
89 * Devices with less than 256Mb of RAM are **_not_** supported
90 * Any previous installation of ancient banIP 0.7.x must be uninstalled, and the /etc/banip folder and the /etc/config/banip configuration file must be deleted (they are recreated when this version is installed)
91
92 ## Installation & Usage
93 * update your local opkg repository (_opkg update_)
94 * install banIP (_opkg install banip_) - the banIP service is disabled by default
95 * edit the config file '/etc/config/banip' and enable the service (set ban\_enabled to '1'), then add pre-configured feeds via 'ban\_feed' (see the feed list above) and add/change other options to your needs (see the options reference below)
96 * start the service with '/etc/init.d/banip start' and check check everything is working by running '/etc/init.d/banip status'
97
98 ## banIP CLI interface
99 * All important banIP functions are accessible via CLI. A LuCI frontend will be available in due course.
100 ```
101 ~# /etc/init.d/banip
102 Syntax: /etc/init.d/banip [command]
103
104 Available commands:
105 start Start the service
106 stop Stop the service
107 restart Restart the service
108 reload Reload configuration files (or restart if service does not implement reload)
109 enable Enable service autostart
110 disable Disable service autostart
111 enabled Check if service is started on boot
112 report [text|json|mail] Print banIP related set statistics
113 search [<IPv4 address>|<IPv6 address>] Check if an element exists in the banIP sets
114 running Check if service is running
115 status Service status
116 trace Start with syscall trace
117 info Dump procd service info
118 ```
119
120 ## banIP config options
121
122 | Option | Type | Default | Description |
123 | :---------------------- | :----- | :---------------------------- | :------------------------------------------------------------------------------------ |
124 | ban_enabled | option | 0 | enable the banIP service |
125 | ban_nicelimit | option | 0 | ulimit nice level of the banIP service (range 0-19) |
126 | ban_filelimit | option | 1024 | ulimit max open/number of files (range 1024-4096) |
127 | ban_loglimit | option | 100 | the logread monitor scans only the last n lines of the logfile |
128 | ban_logcount | option | 1 | how many times the IP must appear in the log to be considered as suspicious |
129 | ban_logterm | list | regex | various regex for logfile parsing (default: dropbear, sshd, luci, nginx, asterisk) |
130 | ban_autodetect | option | 1 | auto-detect wan interfaces, devices and subnets |
131 | ban_debug | option | 0 | enable banIP related debug logging |
132 | ban_loginput | option | 1 | log drops in the wan-input chain |
133 | ban_logforwardwan | option | 1 | log drops in the wan-forward chain |
134 | ban_logforwardlan | option | 0 | log rejects in the lan-forward chain |
135 | ban_autoallowlist | option | 1 | add wan IPs/subnets automatically to the local allowlist |
136 | ban_autoblocklist | option | 1 | add suspicious attacker IPs automatically to the local blocklist |
137 | ban_allowlistonly | option | 0 | restrict the internet access from/to a small number of secure websites/IPs |
138 | ban_reportdir | option | /tmp/banIP-report | directory where banIP stores the report files |
139 | ban_backupdir | option | /tmp/banIP-backup | directory where banIP stores the compressed backup files |
140 | ban_protov4 | option | - / autodetect | enable IPv4 support |
141 | ban_protov6 | option | - / autodetect | enable IPv4 support |
142 | ban_ifv4 | list | - / autodetect | logical wan IPv4 interfaces, e.g. 'wan' |
143 | ban_ifv6 | list | - / autodetect | logical wan IPv6 interfaces, e.g. 'wan6' |
144 | ban_dev | list | - / autodetect | wan device(s), e.g. 'eth2' |
145 | ban_trigger | list | - | logical startup trigger interface(s), e.g. 'wan' |
146 | ban_triggerdelay | option | 10 | trigger timeout before banIP processing begins |
147 | ban_deduplicate | option | 1 | deduplicate IP addresses across all active sets |
148 | ban_splitsize | option | 0 | split ext. sets after every n lines/members (saves RAM) |
149 | ban_cores | option | - / autodetect | limit the cpu cores used by banIP (saves RAM) |
150 | ban_nftexpiry | option | - | expiry time for auto added blocklist members, e.g. '5m', '2h' or '1d' |
151 | ban_nftpriority | option | -200 | nft banIP table priority (default is the prerouting table priority) |
152 | ban_feed | list | - | external download feeds, e.g. 'yoyo', 'doh', 'country' or 'talos' (see feed table) |
153 | ban_asn | list | - | ASNs for the 'asn' feed, e.g.'32934' |
154 | ban_country | list | - | country iso codes for the 'country' feed, e.g. 'ru' |
155 | ban_blockinput | list | - | limit a feed to the wan-input chain, e.g. 'country' |
156 | ban_blockforwardwan | list | - | limit a feed to the wan-forward chain, e.g. 'debl' |
157 | ban_blockforwardlan | list | - | limit a feed to the lan-forward chain, e.g. 'doh' |
158 | ban_fetchcmd | option | - / autodetect | 'uclient-fetch', 'wget', 'curl' or 'aria2c' |
159 | ban_fetchparm | option | - / autodetect | set the config options for the selected download utility |
160 | ban_fetchinsecure | option | 0 | don't check SSL server certificates during download |
161 | ban_mailreceiver | option | - | receiver address for banIP related notification E-Mails |
162 | ban_mailsender | option | no-reply@banIP | sender address for banIP related notification E-Mails |
163 | ban_mailtopic | option | banIP notification | topic for banIP related notification E-Mails |
164 | ban_mailprofile | option | ban_notify | mail profile used in 'msmtp' for banIP related notification E-Mails |
165 | ban_resolver | option | - | external resolver used for DNS lookups |
166
167 ## Examples
168 **banIP report information**
169 ```
170 ~# /etc/init.d/banip report
171 :::
172 ::: banIP Set Statistics
173 :::
174 Timestamp: 2023-02-25 08:35:37
175 ------------------------------
176 auto-added to allowlist: 0
177 auto-added to blocklist: 4
178
179 Set | Elements | WAN-Input (packets) | WAN-Forward (packets) | LAN-Forward (packets)
180 ---------------------+--------------+-----------------------+-----------------------+------------------------
181 allowlistvMAC | 0 | - | - | OK: 0
182 allowlistv4 | 15 | OK: 0 | OK: 0 | OK: 0
183 allowlistv6 | 1 | OK: 0 | OK: 0 | OK: 0
184 torv4 | 800 | OK: 0 | OK: 0 | OK: 0
185 torv6 | 432 | OK: 0 | OK: 0 | OK: 0
186 countryv6 | 34282 | OK: 0 | OK: 1 | -
187 countryv4 | 35508 | OK: 1872 | OK: 0 | -
188 dohv6 | 343 | - | - | OK: 0
189 dohv4 | 540 | - | - | OK: 3
190 firehol1v4 | 1670 | OK: 296 | OK: 0 | OK: 16
191 deblv4 | 12402 | OK: 4 | OK: 0 | OK: 0
192 deblv6 | 41 | OK: 0 | OK: 0 | OK: 0
193 adguardv6 | 12742 | - | - | OK: 161
194 adguardv4 | 23183 | - | - | OK: 212
195 adguardtrackersv6 | 169 | - | - | OK: 0
196 adguardtrackersv4 | 633 | - | - | OK: 0
197 adawayv6 | 2737 | - | - | OK: 15
198 adawayv4 | 6542 | - | - | OK: 137
199 oisdsmallv6 | 10569 | - | - | OK: 0
200 oisdsmallv4 | 18800 | - | - | OK: 74
201 stevenblackv6 | 11901 | - | - | OK: 4
202 stevenblackv4 | 16776 | - | - | OK: 139
203 yoyov6 | 215 | - | - | OK: 0
204 yoyov4 | 309 | - | - | OK: 0
205 antipopadsv4 | 1872 | - | - | OK: 0
206 urlhausv4 | 7431 | OK: 0 | OK: 0 | OK: 0
207 antipopadsv6 | 2081 | - | - | OK: 2
208 blocklistvMAC | 0 | - | - | OK: 0
209 blocklistv4 | 1174 | OK: 1 | OK: 0 | OK: 0
210 blocklistv6 | 40 | OK: 0 | OK: 0 | OK: 0
211 ---------------------+--------------+-----------------------+-----------------------+------------------------
212 30 | 203208 | 12 (2173) | 12 (1) | 28 (763)
213 ```
214
215 **banIP runtime information**
216 ```
217 ~# /etc/init.d/banip status
218 ::: banIP runtime information
219 + status : active
220 + version : 0.8.1-2
221 + element_count : 206644
222 + active_feeds : allowlistvMAC, allowlistv4, allowlistv6, torv4, torv6, countryv6, countryv4, dohv4, dohv6, firehol1v4, deblv4, deblv6,
223 adguardv6, adguardv4, adguardtrackersv6, adguardtrackersv4, adawayv6, adawayv4, oisdsmallv6, oisdsmallv4, stevenblack
224 v6, stevenblackv4, yoyov6, yoyov4, antipopadsv4, urlhausv4, antipopadsv6, blocklistvMAC, blocklistv4, blocklistv6
225 + active_devices : eth2
226 + active_interfaces : wan, wan6
227 + active_subnets : 91.61.199.218/24, 2a02:910c:0:80:e542:4b0c:846d:1d33/128
228 + run_info : base_dir: /tmp, backup_dir: /mnt/data/banIP-backup, report_dir: /mnt/data/banIP-report, feed_file: /etc/banip/banip.feeds
229 + run_flags : proto (4/6): ✔/✔, log (wan-inp/wan-fwd/lan-fwd): ✔/✔/✔, deduplicate: ✔, split: ✘, allowed only: ✘
230 + last_run : action: restart, duration: 1m 6s, date: 2023-02-25 08:55:55
231 + system_info : cores: 2, memory: 1826, device: Turris Omnia, OpenWrt SNAPSHOT r22125-52ddb38469
232 ```
233
234 **banIP search information**
235 ```
236 ~# /etc/init.d/banip search 221.228.105.173
237 :::
238 ::: banIP Search
239 :::
240 Looking for IP 221.228.105.173 on 2023-02-08 22:12:48
241 ---
242 IP found in set oisdbasicv4
243 ```
244
245 **allow-/blocklist handling**
246 banIP supports local allow and block lists (IPv4, IPv6, CIDR notation or domain names), located in /etc/banip/banip.allowlist and /etc/banip/banip.blocklist.
247 Unsuccessful login attempts or suspicious requests will be tracked and added to the local blocklist (see the 'ban\_autoblocklist' option). The blocklist behaviour can be further tweaked with the 'ban\_nftexpiry' option.
248 Furthermore the uplink subnet will be added to local allowlist (see 'ban\_autowallowlist' option).
249 Both lists also accept domain names as input to allow IP filtering based on these names. The corresponding IPs (IPv4 & IPv6) will be extracted in a detached background process and added to the sets.
250
251 **allowlist-only mode**
252 banIP supports an "allowlist only" mode. This option restricts the internet access from/to a small number of secure websites/IPs, and block access from/to the rest of the internet. All IPs and Domains which are _not_ listed in the allowlist are blocked.
253
254 **redirect Asterisk security logs to lodg/logread**
255 banIP only supports logfile scanning via logread, so to monitor attacks on Asterisk, its security log must be available via logread. To do this, edit '/etc/asterisk/logger.conf' and add the line 'syslog.local0 = security', then run 'asterisk -rx reload logger' to update the running Asterisk configuration.
256
257 **tweaks for low memory systems**
258 nftables supports the atomic loading of rules/sets/members, which is cool but unfortunately is also very memory intensive. To reduce the memory pressure on low memory systems (i.e. those with 256-512Mb RAM), you should optimize your configuration with the following options:
259
260 * point 'ban_reportdir' and 'ban_backupdir' to an external usb drive
261 * set 'ban_cores' to '1' (only useful on a multicore system) to force sequential feed processing
262 * set 'ban_splitsize' e.g. to '1000' to split the load of an external set after every 1000 lines/members
263
264 **tweak the download options**
265 By default banIP uses the following pre-configured download options:
266 ```
267 * aria2c: --timeout=20 --allow-overwrite=true --auto-file-renaming=false --log-level=warn --dir=/ -o
268 * curl: --connect-timeout 20 --silent --show-error --location -o
269 * uclient-fetch: --timeout=20 -O
270 * wget: --no-cache --no-cookies --max-redirect=0 --timeout=20 -O
271 ```
272 To override the default set 'ban_fetchparm' manually to your needs.
273
274 **send E-Mail notifications via 'msmtp'**
275 To use the email notification you must install & configure the package 'msmtp'.
276 Modify the file '/etc/msmtprc', e.g.:
277 ```
278 [...]
279 defaults
280 auth on
281 tls on
282 tls_certcheck off
283 timeout 5
284 syslog LOG_MAIL
285 [...]
286 account ban_notify
287 host smtp.gmail.com
288 port 587
289 from <address>@gmail.com
290 user <gmail-user>
291 password <password>
292 ```
293 Finally add a valid E-Mail receiver address.
294
295 **change existing banIP feeds or add a new one**
296 The banIP blocklist feeds are stored in an external JSON file '/etc/banip/banip.feeds'.
297 A valid JSON source object contains the following required information, e.g.:
298 ```
299 [...]
300 "tor": {
301 "url_4": "https://raw.githubusercontent.com/SecOps-Institute/Tor-IP-Addresses/master/tor-exit-nodes.lst",
302 "url_6": "https://raw.githubusercontent.com/SecOps-Institute/Tor-IP-Addresses/master/tor-exit-nodes.lst",
303 "rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
304 "rule_6": "/^(([0-9A-f]{0,4}:){1,7}[0-9A-f]{0,4}:?(\\/(1?[0-2][0-8]|[0-9][0-9]))?)$/{printf \"%s,\\n\",$1}",
305 "focus": "tor exit nodes",
306 "descurl": "https://github.com/SecOps-Institute/Tor-IP-Addresses"
307 },
308 [...]
309 ```
310 Add an unique object name (no spaces, no special chars) and make the required changes: adapt at least the URL the regex to the new feed.
311 **Please note:** if you're going to add new feeds, **always** make a backup of your work, cause this file is always overwritten with the maintainers version on every banIP update.
312
313 ## Support
314 Please join the banIP discussion in this [forum thread](https://forum.openwrt.org/t/banip-support-thread/16985) or contact me by mail <dev@brenken.org>
315
316 ## Removal
317 * stop all banIP related services with _/etc/init.d/banip stop_
318 * remove the banip package (_opkg remove banip_)
319
320 Have fun!
321 Dirk