| voip | VoIP fraud blocklist | x | x | | [Link](https://voipbl.org) |
| yoyo | yoyo IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
-* zero-conf like automatic installation & setup, usually no manual changes needed
-* all sets are handled in a separate nft table/namespace 'banIP'
-* full IPv4 and IPv6 support
-* supports nft atomic set loading
-* supports blocking by ASN numbers and by iso country codes
-* supports local allow- and blocklist (IPv4, IPv6, CIDR notation or domain names)
-* auto-add the uplink subnet to the local allowlist
-* provides a small background log monitor to ban unsuccessful login attempts in real-time
-* auto-add unsuccessful LuCI, nginx, Asterisk or ssh login attempts to the local blocklist
-* fast feed processing as they are handled in parallel as background jobs
-* per feed it can be defined whether the wan-input chain, the wan-forward chain or the lan-forward chain should be blocked (default: all chains)
-* automatic blocklist backup & restore, the backups will be used in case of download errors or during startup
-* automatically selects one of the following download utilities with ssl support: aria2c, curl, uclient-fetch or wget
-* supports an 'allowlist only' mode, this option restricts internet access from/to a small number of secure websites/IPs
-* deduplicate IPs accross all sets (single IPs only, no intervals)
-* provides comprehensive runtime information
-* provides a detailed set report
-* provides a set search engine for certain IPs
-* feed parsing by fast & flexible regex rulesets
-* minimal status & error logging to syslog, enable debug logging to receive more output
-* procd based init system support (start/stop/restart/reload/status/report/search)
-* procd network interface trigger support
-* ability to add new banIP feeds on your own
+* Zero-conf like automatic installation & setup, usually no manual changes needed
+* All sets are handled in a separate nft table/namespace 'banIP'
+* Full IPv4 and IPv6 support
+* Supports nft atomic set loading
+* Supports blocking by ASN numbers and by iso country codes
+* Supports local allow- and blocklist (IPv4, IPv6, CIDR notation or domain names)
+* Auto-add the uplink subnet to the local allowlist
+* Provides a small background log monitor to ban unsuccessful login attempts in real-time
+* Auto-add unsuccessful LuCI, nginx, Asterisk or ssh login attempts to the local blocklist
+* Fast feed processing as they are handled in parallel as background jobs
+* Per feed it can be defined whether the wan-input chain, the wan-forward chain or the lan-forward chain should be blocked (default: all chains)
+* Automatic blocklist backup & restore, the backups will be used in case of download errors or during startup
+* Automatically selects one of the following download utilities with ssl support: aria2c, curl, uclient-fetch or wget
+* Supports an 'allowlist only' mode, this option restricts internet access from/to a small number of secure websites/IPs
+* Deduplicate IPs accross all sets (single IPs only, no intervals)
+* Provides comprehensive runtime information
+* Provides a detailed set report
+* Provides a set search engine for certain IPs
+* Feed parsing by fast & flexible regex rulesets
+* Minimal status & error logging to syslog, enable debug logging to receive more output
+* Procd based init system support (start/stop/restart/reload/status/report/search/survey)
+* Procd network interface trigger support
+* Ability to add new banIP feeds on your own
## Prerequisites
-* **[OpenWrt](https://openwrt.org)**, latest stable release or a snapshot with nft/firewall 4 support
-* a download utility with SSL support: 'wget', 'uclient-fetch' with one of the 'libustream-*' SSL libraries, 'aria2c' or 'curl' is required
-* a certificate store like 'ca-bundle', as banIP checks the validity of the SSL certificates of all download sites by default
-* for E-Mail notifications you need to install and setup the additional 'msmtp' package
+* **[OpenWrt](https://openwrt.org)**, latest stable release or a snapshot with nft/firewall 4 support
+* A download utility with SSL support: 'wget', 'uclient-fetch' with one of the 'libustream-*' SSL libraries, 'aria2c' or 'curl' is required
+* A certificate store like 'ca-bundle', as banIP checks the validity of the SSL certificates of all download sites by default
+* For E-Mail notifications you need to install and setup the additional 'msmtp' package
**Please note the following:**
* Devices with less than 256Mb of RAM are **_not_** supported
* Any previous installation of ancient banIP 0.7.x must be uninstalled, and the /etc/banip folder and the /etc/config/banip configuration file must be deleted (they are recreated when this version is installed)
## Installation & Usage
-* update your local opkg repository (_opkg update_)
-* install banIP (_opkg install banip_) - the banIP service is disabled by default
-* edit the config file '/etc/config/banip' and enable the service (set ban\_enabled to '1'), then add pre-configured feeds via 'ban\_feed' (see the feed list above) and add/change other options to your needs (see the options reference below)
-* start the service with '/etc/init.d/banip start' and check check everything is working by running '/etc/init.d/banip status'
+* Update your local opkg repository (_opkg update_)
+* Install banIP (_opkg install banip_) - the banIP service is disabled by default
+* Install the LuCI companion package 'luci-app-banip' (opkg install luci-app-banip)
+* It's strongly recommended to use the LuCI frontend to easily configure all aspects of banIP, the application is located in LuCI under the 'Services' menu
+* If you're going to configure banIP via CLI, edit the config file '/etc/config/banip' and enable the service (set ban\_enabled to '1'), then add pre-configured feeds via 'ban\_feed' (see the feed list above) and add/change other options to your needs (see the options reference below)
+* Start the service with '/etc/init.d/banip start' and check check everything is working by running '/etc/init.d/banip status'
## banIP CLI interface
-* All important banIP functions are accessible via CLI. A LuCI frontend will be available in due course.
+* All important banIP functions are accessible via CLI.
```
~# /etc/init.d/banip
Syntax: /etc/init.d/banip [command]
| ban_autoallowlist | option | 1 | add wan IPs/subnets automatically to the local allowlist |
| ban_autoblocklist | option | 1 | add suspicious attacker IPs automatically to the local blocklist |
| ban_allowlistonly | option | 0 | restrict the internet access from/to a small number of secure websites/IPs |
+| ban_basedir | option | /tmp | base working directory while banIP processing |
| ban_reportdir | option | /tmp/banIP-report | directory where banIP stores the report files |
| ban_backupdir | option | /tmp/banIP-backup | directory where banIP stores the compressed backup files |
| ban_protov4 | option | - / autodetect | enable IPv4 support |
```
~# /etc/init.d/banip status
::: banIP runtime information
- + status : active
- + version : 0.8.1-2
- + element_count : 206644
- + active_feeds : allowlistvMAC, allowlistv4, allowlistv6, torv4, torv6, countryv6, countryv4, dohv4, dohv6, firehol1v4, deblv4, deblv6,
- adguardv6, adguardv4, adguardtrackersv6, adguardtrackersv4, adawayv6, adawayv4, oisdsmallv6, oisdsmallv4, stevenblack
- v6, stevenblackv4, yoyov6, yoyov4, antipopadsv4, urlhausv4, antipopadsv6, blocklistvMAC, blocklistv4, blocklistv6
+ + status : active (nft: ✔, monitor: ✔)
+ + version : 0.8.1-3
+ + element_count : 180596
+ + active_feeds : allowlistvMAC, allowlistv4, allowlistv6, adawayv4, adawayv6, adguardv4, cinsscorev4, adguardv6, countryv6, countryv4,
+ deblv4, deblv6, dohv4, dohv6, firehol1v4, oisdsmallv6, oisdsmallv4, urlvirv4, webclientv4, blocklistvMAC, blocklistv4,
+ blocklistv6
+ active_devices : eth2
+ active_interfaces : wan, wan6
- + active_subnets : 91.61.199.218/24, 2a02:910c:0:80:e542:4b0c:846d:1d33/128
- + run_info : base_dir: /tmp, backup_dir: /mnt/data/banIP-backup, report_dir: /mnt/data/banIP-report, feed_file: /etc/banip/banip.feeds
- + run_flags : proto (4/6): ✔/✔, log (wan-inp/wan-fwd/lan-fwd): ✔/✔/✔, deduplicate: ✔, split: ✘, allowed only: ✘
- + last_run : action: restart, duration: 1m 6s, date: 2023-02-25 08:55:55
- + system_info : cores: 2, memory: 1826, device: Turris Omnia, OpenWrt SNAPSHOT r22125-52ddb38469
+ + active_subnets : 91.64.168.218/24, 2a02:710c:0:80:e342:4b0c:725d:1d43/128
+ + run_info : base: /tmp, backup: /mnt/data/banIP-backup, report: /mnt/data/banIP-report, feed: /etc/banip/banip.feeds
+ + run_flags : auto: ✔, proto (4/6): ✔/✔, log (wan-inp/wan-fwd/lan-fwd): ✔/✔/✔, dedup: ✔, split: ✘, allowed only: ✘
+ + last_run : action: restart, duration: 0m 58s, date: 2023-03-06 13:50:27
+ + system_info : cores: 2, memory: 1831, device: Turris Omnia, OpenWrt SNAPSHOT r22151-1d82a47b49
```
**banIP search information**
IP found in set oisdbasicv4
```
+**banIP survey information**
+```
+~# /etc/init.d/banip survey cinsscorev4
+:::
+::: banIP Survey
+:::
+ List the elements of set cinsscorev4 on 2023-03-06 14:07:58
+ ---
+1.10.187.179
+1.10.203.30
+1.10.255.58
+1.11.67.53
+1.11.114.211
+1.11.208.29
+1.12.75.87
+1.12.231.227
+1.12.247.134
+1.12.251.141
+1.14.96.156
+1.14.250.37
+1.15.40.79
+1.15.71.140
+1.15.77.237
+[...]
+```
+
**allow-/blocklist handling**
banIP supports local allow and block lists (IPv4, IPv6, CIDR notation or domain names), located in /etc/banip/banip.allowlist and /etc/banip/banip.blocklist.
Unsuccessful login attempts or suspicious requests will be tracked and added to the local blocklist (see the 'ban\_autoblocklist' option). The blocklist behaviour can be further tweaked with the 'ban\_nftexpiry' option.
**tweaks for low memory systems**
nftables supports the atomic loading of rules/sets/members, which is cool but unfortunately is also very memory intensive. To reduce the memory pressure on low memory systems (i.e. those with 256-512Mb RAM), you should optimize your configuration with the following options:
- * point 'ban_reportdir' and 'ban_backupdir' to an external usb drive
+ * point 'ban_basedir', 'ban_reportdir' and 'ban_backupdir' to an external usb drive
* set 'ban_cores' to '1' (only useful on a multicore system) to force sequential feed processing
* set 'ban_splitsize' e.g. to '1000' to split the load of an external set after every 1000 lines/members
By default banIP uses the following pre-configured download options:
```
* aria2c: --timeout=20 --allow-overwrite=true --auto-file-renaming=false --log-level=warn --dir=/ -o
- * curl: --connect-timeout 20 --silent --show-error --location -o
+ * curl: --connect-timeout 20 --fail --silent --show-error --location -o
* uclient-fetch: --timeout=20 -O
* wget: --no-cache --no-cookies --max-redirect=0 --timeout=20 -O
```
export PATH="/usr/sbin:/usr/bin:/sbin:/bin"
ban_basedir="/tmp"
-ban_backupdir="${ban_basedir}/banIP-backup"
-ban_reportdir="${ban_basedir}/banIP-report"
+ban_backupdir="/tmp/banIP-backup"
+ban_reportdir="/tmp/banIP-report"
ban_feedfile="/etc/banip/banip.feeds"
+ban_allowlist="/etc/banip/banip.allowlist"
+ban_blocklist="/etc/banip/banip.blocklist"
+ban_mailtemplate="/etc/banip/banip.tpl"
ban_pidfile="/var/run/banip.pid"
+ban_rtfile="/var/run/banip_runtime.json"
ban_lock="/var/run/banip.lock"
-ban_blocklist="/etc/banip/banip.blocklist"
-ban_allowlist="/etc/banip/banip.allowlist"
ban_fetchcmd=""
ban_logreadcmd="$(command -v logread)"
ban_logcmd="$(command -v logger)"
ban_mailreceiver=""
ban_mailtopic="banIP notification"
ban_mailprofile="ban_notify"
-ban_mailtemplate="/etc/banip/banip.tpl"
ban_nftpriority="-200"
ban_nftexpiry=""
ban_loglevel="warn"
ban_autoblocklist="1"
ban_deduplicate="1"
ban_splitsize="0"
-ban_autodetect=""
+ban_autodetect="1"
ban_feed=""
ban_blockinput=""
ban_blockforwardwan=""
: >"${ban_pidfile}"
}
+# get nft/monitor actuals
+#
+f_actual() {
+ local nft monitor
+
+ if "${ban_nftcmd}" -t list set inet banIP allowlistvMAC >/dev/null 2>&1; then
+ nft="$(f_char "1")"
+ else
+ nft="$(f_char "0")"
+ fi
+ if pgrep -f "logread" -P "$(cat "${ban_pidfile}" 2>/dev/null)" >/dev/null 2>&1; then
+ monitor="$(f_char "1")"
+ else
+ monitor="$(f_char "0")"
+ fi
+ printf "%s" "nft: ${nft}, monitor: ${monitor}"
+}
+
# get wan interfaces
#
f_getif() {
# nft header (tables and chains)
#
printf "%s\n\n" "#!/usr/sbin/nft -f"
- if "${ban_nftcmd}" -t list table inet banIP >/dev/null 2>&1; then
+ if "${ban_nftcmd}" -t list set inet banIP allowlistvMAC >/dev/null 2>&1; then
printf "%s\n" "delete table inet banIP"
fi
printf "%s\n" "add table inet banIP"
return ${feed_rc}
}
+# handle downloads
+#
f_down() {
local log_input log_forwardwan log_forwardlan start_ts end_ts tmp_raw tmp_load tmp_file split_file input_handles forwardwan_handles forwardlan_handles handle
local cnt_set cnt_dl restore_rc feed_direction feed_rc feed_log feed="${1}" proto="${2}" feed_url="${3}" feed_rule="${4}" feed_flag="${5}"
local tmp_del table_sets input_handles forwardwan_handles forwardlan_handles handle sets feed feed_log feed_rc
tmp_del="${ban_tmpfile}.final.delete"
- table_sets="$("${ban_nftcmd}" -t list table inet banIP 2>/dev/null | "${ban_awkcmd}" '/^[[:space:]]+set [[:alnum:]]+ /{printf "%s ",$2}' 2>/dev/null)"
+ table_sets="$("${ban_nftcmd}" -tj list table inet banIP 2>/dev/null | jsonfilter -qe '@.nftables[*].set.name')"
input_handles="$("${ban_nftcmd}" -t --handle --numeric list chain inet banIP wan-input 2>/dev/null)"
forwardwan_handles="$("${ban_nftcmd}" -t --handle --numeric list chain inet banIP wan-forward 2>/dev/null)"
forwardlan_handles="$("${ban_nftcmd}" -t --handle --numeric list chain inet banIP lan-forward 2>/dev/null)"
# generate status information
#
f_genstatus() {
- local object duration nft_table nft_feeds cnt_elements="0" split="0" status="${1}"
+ local object duration nft_feeds cnt_elements="0" split="0" status="${1}"
[ -z "${ban_dev}" ] && f_conf
if [ "${status}" = "active" ]; then
ban_endtime="$(date "+%s")"
duration="$(((ban_endtime - ban_starttime) / 60))m $(((ban_endtime - ban_starttime) % 60))s"
fi
- nft_table="$("${ban_nftcmd}" -t list table inet banIP 2>/dev/null)"
- nft_feeds="$(f_trim "$(printf "%s\n" "${nft_table}" | "${ban_awkcmd}" '/^[[:space:]]+set [[:alnum:]]+ /{printf "%s ",$2}')")"
+ nft_feeds="$("${ban_nftcmd}" -tj list table inet banIP 2>/dev/null | jsonfilter -qe '@.nftables[*].set.name')"
for object in ${nft_feeds}; do
cnt_elements="$((cnt_elements + $("${ban_nftcmd}" -j list set inet banIP "${object}" 2>/dev/null | jsonfilter -qe '@.nftables[*].set.elem[*]' | wc -l 2>/dev/null)))"
done
f_system
[ ${ban_splitsize:-"0"} -gt "0" ] && split="1"
- : >"${ban_basedir}/ban_runtime.json"
+ : >"${ban_rtfile}"
json_init
- json_load_file "${ban_basedir}/ban_runtime.json" >/dev/null 2>&1
+ json_load_file "${ban_rtfile}" >/dev/null 2>&1
json_add_string "status" "${status}"
json_add_string "version" "${ban_ver}"
json_add_string "element_count" "${cnt_elements}"
fi
json_close_array
json_add_string "run_info" "base: ${ban_basedir}, backup: ${ban_backupdir}, report: ${ban_reportdir}, feed: ${ban_feedfile}"
- json_add_string "run_flags" "protocol (4/6): $(f_char ${ban_protov4})/$(f_char ${ban_protov6}), log (wan-inp/wan-fwd/lan-fwd): $(f_char ${ban_loginput})/$(f_char ${ban_logforwardwan})/$(f_char ${ban_logforwardlan}), deduplicate: $(f_char ${ban_deduplicate}), split: $(f_char ${split}), allowed only: $(f_char ${ban_allowlistonly})"
+ json_add_string "run_flags" "auto: $(f_char ${ban_autodetect}), proto (4/6): $(f_char ${ban_protov4})/$(f_char ${ban_protov6}), log (wan-inp/wan-fwd/lan-fwd): $(f_char ${ban_loginput})/$(f_char ${ban_logforwardwan})/$(f_char ${ban_logforwardlan}), dedup: $(f_char ${ban_deduplicate}), split: $(f_char ${split}), allowed only: $(f_char ${ban_allowlistonly})"
json_add_string "last_run" "${runtime:-"-"}"
json_add_string "system_info" "cores: ${ban_cores}, memory: ${ban_memory}, device: ${ban_sysver}"
- json_dump >"${ban_basedir}/ban_runtime.json"
+ json_dump >"${ban_rtfile}"
}
# get status information
#
f_getstatus() {
- local key keylist type value index_value
+ local key keylist type value index_value actual="${1}"
[ -z "${ban_dev}" ] && f_conf
- json_load_file "${ban_basedir}/ban_runtime.json" >/dev/null 2>&1
+ json_load_file "${ban_rtfile}" >/dev/null 2>&1
if json_get_keys keylist; then
printf "%s\n" "::: banIP runtime information"
for key in ${keylist}; do
json_get_var value "${key}" >/dev/null 2>&1
- if [ "${key%_*}" = "active" ]; then
+ if [ "${key}" = "status" ]; then
+ value="${value} ($(f_actual))"
+ elif [ "${key%_*}" = "active" ]; then
json_select "${key}" >/dev/null 2>&1
index=1
while json_get_type type "${index}" && [ "${type}" = "object" ]; do
done
json_select ".."
fi
- value="$(
- printf "%s" "${value}" |
- awk '{NR=1;max=118;if(length($0)>max+1)while($0){if(NR==1){print substr($0,1,max)}else{printf"%-24s%s\n","",substr($0,1,max)}{$0=substr($0,max+1);NR=NR+1}}else print}'
- )"
+ value="$(printf "%s" "${value}" |
+ awk '{NR=1;max=118;if(length($0)>max+1)while($0){if(NR==1){print substr($0,1,max)}else{printf"%-24s%s\n","",substr($0,1,max)}{$0=substr($0,max+1);NR=NR+1}}else print}')"
printf " + %-17s : %s\n" "${key}" "${value:-"-"}"
done
else
f_log "debug" "f_lookup ::: name: ${feed}, cnt_domain: ${cnt_domain}, cnt_ip: ${cnt_ip}, duration: ${duration}"
}
-# banIP table statistics
+# table statistics
#
f_report() {
local report_jsn report_txt set tmp_val nft_raw nft_sets set_cnt set_input set_forwardwan set_forwardlan set_cntinput set_cntforwardwan set_cntforwardlan output="${1}"
printf "%s\n%s\n%s\n" ":::" "::: banIP Set Statistics" ":::"
printf "%s\n" " Timestamp: ${timestamp}"
printf "%s\n" " ------------------------------"
- printf "%s\n" " auto-added to allowlist: ${autoadd_allow}"
- printf "%s\n\n" " auto-added to blocklist: ${autoadd_block}"
+ printf "%s\n" " auto-added to allowlist today: ${autoadd_allow}"
+ printf "%s\n\n" " auto-added to blocklist today: ${autoadd_block}"
json_select "sets" >/dev/null 2>&1
json_get_keys nft_sets >/dev/null 2>&1
if [ -n "${nft_sets}" ]; then
esac
}
-# banIP set search
+# set search
#
f_search() {
local nft_sets ip proto run_search search="${1}"
f_system
run_search="/var/run/banIP.search"
-
if [ -n "${search}" ]; then
ip="$(printf "%s" "${search}" | "${ban_awkcmd}" 'BEGIN{RS="(([0-9]{1,3}\\.){3}[0-9]{1,3})+"}{printf "%s",RT}')"
[ -n "${ip}" ] && proto="v4"
rm -f "${run_search}"
}
+# set survey
+#
+f_survey() {
+ local set_survey set="${1}"
+
+ f_system
+ if [ -n "${set}" ]; then
+ if "${ban_nftcmd}" -jt list set inet banIP "${set}" >/dev/null 2>&1; then
+ set_survey="$("${ban_nftcmd}" -j list set inet banIP "${set}" 2>/dev/null | jsonfilter -qe '@.nftables[*].set.elem[*]')"
+ else
+ printf "%s\n%s\n%s\n" ":::" "::: unknown banIP set (single banIP set name)" ":::"
+ return
+ fi
+ else
+ printf "%s\n%s\n%s\n" ":::" "::: no valid survey input (single banIP set name)" ":::"
+ return
+ fi
+ printf "%s\n%s\n%s\n" ":::" "::: banIP Survey" ":::"
+ printf "%s\n" " List the elements of set ${set} on $(date "+%Y-%m-%d %H:%M:%S")"
+ printf "%s\n" " ---"
+ printf "%s\n" "${set_survey}"
+}
+
# send status mails
#
f_mail() {
projects / feed / packages.git / commitdiff