ACZoom  ·  Archive-2016

Either hosts.allow features or blockhosts.py command-line arguments can be used to whitelist or blacklist IP addresses.

• Using hosts.allow. To stop blockhosts.py from being called at all for whitelisted/blacklisted addresses, use the hosts.allow whitelist/blacklist feature.

  Edit /etc/hosts.allow, and before the line where blockhost section starts, add
  -- your permanent whitelist and blacklist of IP addresses (this is for hosts.allow white/blacklisting)

  Here's an example hosts.allow:

  # permanent whitelist addresses - this should always be allowed access
  
  ALL: 127.0.0.1 : allow
  ALL: 10. : allow
  
  # permanent blacklist addresses - this should always be denied access
  ALL: 192.168.1.1 : deny
  ALL: 192.168.1.2 : deny
  

In addition to the above, to prevent blockhosts.py from ever entering a block rule for the whitelisted addresses, the --whitelist argument should be used, as described below.

• If not using hosts.allow, or when it is necessary to have blockhosts manage the blacklisting or whitelisting, use the --blacklist or --whitelist options of blockhosts.py. These take regular expressions or IP addresses as arguments. Here's an example:

  /usr/bin/blockhosts.py --verbose  --mail --ipblock=iptables --whitelist="10\..*,127.0.0.1" --blacklist="192.168.1.1,192.168.1.2"
  

  The above example will cause blockhosts to whitelist any address beginning with 10., and the address 127.0.0.1 and thus never block these addresses even if their count of failed accesses goes above the threshold.

  Secondly, the above example will cause blockhosts to immediately blacklist the two addresses listed - 192.168.1.1 and 192.168.1.2 when blockhosts.py is run.
  If regular expressions are provided to the --blacklist option, then as soon as the first failed access is seen, even if the count has not crossed threshold, that IP address will be blocked. Note that since blockhosts.py runs after the connection attempt, using regular expressions may not immediately block a blacklisted IP address. See blockhosts.py --help for more information.
  -------------------------------------------------------------------------

BlockHosts uses the host control facility and related files such as hosts.access to enable blocking at the service level.

But with the --ipblock="ip route" or --ipblock=iptables options, BlockHosts can be used to protect any service, by allowing the blocking to happen at the network level. This requires no assistance from the TCP_WRAPPER enabled host control facility.

The requirements are:

1. there has to be some way to scan a log file and there needs to be way to match the IP address of an abusive host
2. there has to some way to invoke blockhosts.py whenever there is failed access

As an example, from the forum topic blockhost-for-apache, there is the need to scan the Apache web server log files, detect abusive accesses, and then block the abusive hosts. For this example, this is the pattern to use to scan the Apache log file:

    "APACHE-FAIL": r"""\[client (::ffff:)?(?P<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\].+authentication failure""",

Here are the steps to enable blocking of hosts based on Apache web server log information:

1. Add the required pattern to the configuration file /etc/blockhosts.cfg.
2. Configure BlockHosts as usual, still using hosts.allow as the block file to maintain the list of currently watched and blocked hosts.
3. Find a way to run blockhosts.py for this service - the forum topic blockhost-for-apache uses a php script in a Error document or in a web page for failed access redirects, the line to add in PHP is:
   

   exec('/usr/bin/blockhosts.py --ipblock="ip route"');

   Use either the --ipblock="ip route" or --ipblock=iptables options to blockhosts.py. This will enable blocking at the network level, so those blocked hosts will be unable to connect to any service running on your server.

Now, there will no longer be any connects from that host in your Apache log files.

For BlockHosts versions 2.3 or newer:

Example pattern:

    "sshd-Invalid":
        r'{LOG_PREFIX{sshd}} (Invalid|Illegal) user .* from {HOST_IP}',

See comments in the configuration file /etc/blockhosts.cfg for more patterns. The short-forms {LOG_PREFIX{}} and {HOST_IP} are used to make the patterns easier to write.

Pattern must contain {HOST_IP} keyword to identify remote host IP address.
Pattern may contain {LOG_PREFIX{service_name}} keyword to denote syslog/metalog/multilog prefix at the beginning of a log line.
Service_name is a string representing name of service in the prefix of the log lines in syslog, for example service_name may be sshd or vsftpd(pam_unix).
The log prefix does not include the trailing space, and always matches at beginning of line.

# Examples of LOG_PREFIX matches (the / delimiter is not part of match):
#   matched by {LOG_PREFIX{vsftpd(pam_unix)}}  :
#    /Dec 25 21:18:58 host vsftpd(pam_unix)[21989]:/
#   matched by {LOG_PREFIX{xinted}}  :
#    /Oct 18 04:21:52 host xinetd[3316]:/
#   matched by {LOG_PREFIX{sshd}}  :
#    /Oct  4 12:04:50 host.example.com sshd[1110]: [ID 800047 auth.info]/

For BlockHosts versions 2.2 or older:
BlockHosts uses regular expressions to detect repeated probes into your system, by trying to match the regular expressions to lines in the system log files.

This list of regular expressions can be updated in the configuration file, which by default is /etc/blockhosts.cfg.

The name of the configuration variable is: ALL_REGEXS_STR
The value for ALL_REGEXS_STR is a python dictionary. The key is a string to label the regular expression (choose any unique string) and the value is the regular expression string.

The regexps should contain a P<host> to make a named match for the IP address. P<host> is essential, and required in every pattern.

In some rare cases, the process id pattern also needs to be matched, and named as P<pid>. Process-id needs to be identified when the pattern may be matched multiple times in the system log files, and adding a process-id match prevents double or triple or multiple counting for a single probe from a rogue host. But - this is just a nicety, there is no harm in double/triple counting, the only effect is that some host may be blocked even when the actual number of attacks is less than the COUNT_THRESHOLD configuration value. Most services do not need this, so safe to leave this out.

Here's the pattern that is the most widely used -- it detects SSH attacks:

    "SSHD-Invalid": r"""sshd\[(?P<pid>\d+)\]:.*?(Invalid|Illegal) user .*? from (::ffff:)?(?P<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})""",

Follow the instructions in /etc/blockhosts.cfg to make sure the Python language syntax is followed, incorrect editing or errors in this file will cause blockhosts.py to fail to startup. Always test by running blockhosts.py --dry-run --verbose after editing blockhosts.cfg.

Pattern Testing

Since it is not easy to write a regular expression, it is important to test out the regular expression and matching it on one of the lines from the log. A very good test tool for this is kodos - The Python Regular Expression Debugger. Install kodos, and then run it. Enter your test regular expression in the top window, the log line in the second window named "Search String", and the bottom results window should show the groups that are matched - there must be a group named "host" for blockhosts.py to work, and this "host" group should display the IP (IPv4) address to be blocked.

Use the --ipblock="ip route" or --ipblock=iptables options to enable null routing or packet filtering blocking.

Example command to use in hosts.allow:

sshd, proftpd, vsftpd: ALL: spawn /usr/bin/blockhosts.py --ipblock="ip route" \
    --logfiles="/var/log/secure,/var/log/vsftpd.log" \
    --echo "%c-%s" --mail --check-ip "%h" & \
: allow

Use --ipblock="ip route" if the ip route command null routing is desired.
Use --ipblock=iptables if iptables packet level filtering is desired.

Root permission for the run of blockhosts.py script is needed, since only root can change routing tables or install iptables rules. This works properly with correct permissions, if using hosts.access/hosts.deny to run this script.

Full path for the "ip" and "iptables" can also be provided in the --ipblock option if these are not in PATH, for example: --ipblock="/sbin/ip route" or --ipblock=/sbin/iptables.

1. Linux-like systems. Not usable on Microsoft Windows.
2. Python 2.3 or later, with the optparse module.
3. Primarily uses host control facility and related files such as hosts.access. If not using TCP/IP blocking (--iproute or --iptables options to blockhosts.py), then the extensions to the access control language as described in the man 5 hosts_options page are required, which allow use of :allow and :deny keywords. ["...extensions are turned on at program build time by building with -DPROCESS_OPTIONS..." for services like ssh, ftp, etc]
4. If not using host control facilities (tcpd, hosts.access, etc),
   then there needs to be a way to trigger the run of blockhosts.py, or blockhosts.py should be run periodically using cron. Secondly, there must be some way to update a file to list the blocked ip (for example, hosts.access file, or Apache .htaccess file, etc). Alternately, all TCP/IP communication can be blocked by using the --iproute or --iptables options of blockhosts.py

1. Run the installer:
   Extract source, and run
     python setup.py install
   OR use rpm command (change n.n.n to version used):
     rpm -iv BlockHosts-n.n.n-1.noarch.rpm
2. Edit /etc/blockhosts.cfg to configure BlockHosts as required at the site
3. Edit /etc/hosts.allow, this is the hosts block file used to store the watched and blocked hosts.
   Add following sections, in this order:
   -- your permanent whitelist and blacklist of IP addresses (this is for hosts.allow white/blacklisting)
   -- blockhosts marker lines - two lines
   -- execute command to kick off blockhosts.py on connects to services
   Here's an example hosts.allow:

   #
   # hosts.allow	This file describes the names of the hosts which are
   #		allowed to use the local INET services, as decided
   #		by the '/usr/sbin/tcpd' server.
   #
   # ----
   # see "man 5 hosts_access" for details of the format of IP addresses,
   #services, allow/deny options. Also see "man hosts_options"
   #
   # permanent whitelist addresses - this should always be allowed access
   
   ALL: 127.0.0.1 : allow
   
   # permanent blacklist addresses - this should always be denied access
   # ALL: 10. : deny
   # ----------------------------------------
   # next section is the blockhosts section - it will add/delete entries in
   # between the two marker lines (#---- BlockHosts Additions)
   
   #---- BlockHosts Additions
   #---- BlockHosts Additions
   
   # ----------------------------------------
   # finally, the command to execute the blockhosts script, based on
   # connection to particular service or services:
   
   sshd, proftpd, vsftpd: ALL: spawn /usr/bin/blockhosts.py --verbose \
        --mail --ipblock=iptables \
        --whitelist="10\..*,127.0.0.1" --blacklist="192.168.1.1,192.168.1.2" \
        --echo "%c-%s" & \
   : allow
   
   # add    --verbose >> /var/log/blockhosts.log 2>&1 
   #    instead of just the "&" above if extra logging is needed, to the /var/log/blockhosts.log file
   # leave out whitelist/blacklist/mail/ipblock options if not needed - see 
   #           blockhosts.py --help        for more information.
   # -------------------------------------------------------------------------
   

4. For the first run, execute the program manually. As root user, run:
     blockhosts.py --dry-run --verbose
   (--dry-run just shows what will be added to the list of blocked and watched hosts).

   The first run will scan through the entire system logs, so it will add a number of hosts to the block list. Most or all of these hosts will be removed after the AGE_THRESHOLD configuration value (12 hours by default) has elapsed.

   Assuming everything looks fine, run the command without --dry-run:
     blockhosts.py --verbose

This completes the installation. Now wait, and watch hosts being added and removed from the list, as BlockHosts goes about its business.

1: save existing site configuration file, this will have to merged later:
/etc/blockhosts.cfg

2: run the installer, it will overwrite all old files.
Extract source, and run
  python setup.py install --force
OR use rpm command (change n.n.n to version used):
  rpm -Uv BlockHosts-n.n.n-1.noarch.rpm

3: edit /etc/blockhosts.cfg and if needed, update /etc/hosts.allow, to configure BlockHosts as required at the site