Furkan İkkan Türkçe
Back to posts
Post

· Network & System Administrator

Fail2ban + LLM: Auto-Block SSH Brute Force IPs from Logs

Use LLMs to analyze Fail2ban logs and auto-generate iptables blocks for abnormal SSH login attempts — reduces noise and catches evasion.

Yapay ZekaFail2banSSHLLMiptableslog analysis

I’ve been running Fail2ban on public SSH endpoints for years. It works well for basic brute force — too many failed attempts from one IP, and it gets jailed. But attackers have adapted. They spread attempts across dozens of IPs, slow down the rate, or use credential stuffing from compromised residential proxies. Fail2ban’s regex-based filters miss these low-and-slow patterns because they look for repetition in tight windows. That’s where I started experimenting with LLMs to analyze log context and flag abnormal behavior that rule-based systems ignore.

My goal wasn’t to replace Fail2ban but to augment it. I feed aggregated auth logs into a local LLM (I use Mistral 7B via Ollama) and ask it to score login attempts for anomaly: time clustering, username patterns, geographic impossibility, or session anomalies. If the score crosses a threshold, the LLM outputs a candidate iptables rule. I then validate it and apply it via a cron-triggered script — never auto-apply without review.

Here’s how the pipeline works in my environment.

Collecting and Preparing Log Data

Fail2ban already monitors /var/log/auth.log. I extended this by copying relevant lines to a staging file every five minutes via cron. I focus on SSH daemon logs: failed passwords, invalid users, and disconnects before authentication.

I extract the last 20 minutes of relevant entries:

journalctl -u sshd --since "20 minutes ago" | \
  grep -E "Failed password|Invalid user|disconnected" > /var/log/ssh-anomaly-staging.log

This keeps the input small enough for the LLM to process quickly. I avoid feeding full logs — only the raw lines that indicate potential risk.

Prompting the LLM for Anomaly Scoring

I built a simple Python wrapper that reads the staging log and sends it to Ollama with a focused prompt. The prompt asks the model to:

  • Identify IPs with abnormal login behavior
  • Consider time clustering, username reuse, and failed patterns
  • Output only IPs that warrant blocking, one per line
  • If none, output "none"

Here’s the core of the script:

import subprocess
import ollama

def analyze_ssh_log(log_path):
    with open(log_path, 'r') as f:
        log_content = f.read()
    
    prompt = f"""Analyze the following SSH daemon log entries for signs of brute force or credential stuffing.
    Look for:
    - Multiple failed attempts from the same IP over time
    - Attempts with non-existent users (e.g., 'admin', 'test', 'oracle')
    - Patterns suggesting credential stuffing (e.g., common passwords across users)
    - Geographic or timing impossibility (if implied by username/time)
    Output only the IP addresses that should be blocked, one per line.
    If no IPs meet the threshold, output exactly: none
    
    LOG:
    {log_content}
    """
    
    response = ollama.generate(model='mistral', prompt=prompt)
    return response['response'].strip()

ips = analyze_ssh_log('/var/log/ssh-anomaly-staging.log')
if ips and ips.lower() != 'none':
    with open('/var/log/ssh-block-candidates.txt', 'w') as f:
        f.write(ips)

The model isn’t perfect — it sometimes flags shared NAT IPs or misses slow drips — but it catches patterns Fail2ban overlooks, like 10 attempts over 2 hours from the same IP with rotating usernames.

Validating and Applying Blocks

I never apply LLM output directly. A separate validation script checks the candidate IPs against:

  • Current Fail2ban jails (to avoid duplicates)
  • Whitelisted networks (my office, monitoring)
  • ASN reputation (via local cache of known bad ASNs)

If an IP passes, it adds a temporary iptables rule with a 2-hour timeout:

while read ip; do
    if ! iptables -C INPUT -s $ip -j DROP 2>/dev/null; then
        iptables -I INPUT -s $ip -j DROP -m comment --comment "LLM-SSH-Anomaly-$(date +%s)"
        echo "$(date): Blocked $ip via LLM analysis" >> /var/log/ssh-llm-block.log
    fi
done < /var/log/ssh-block-candidates.txt

I use a comment to track the source and timestamp. A cleanup cron job removes rules older than 2 hours to prevent accumulation.

Tuning and False Positives

Early on, the LLM flagged IPs from my internal monitoring probes because they tested SSH banners. I added those networks to the whitelist. Another time, it blocked a cloud NAT used by developers — turned out they were rotating credentials during a migration. I now exclude known dev/cloud ranges unless the anomaly score is very high.

I also adjusted the prompt to ignore single failed attempts — the model should look for patterns, not isolated errors. This cut noise by ~70%.

Why Not Just Lower Fail2ban’s Threshold?

You might ask: why not just reduce Fail2ban’s maxretry or findtime? I tried that. Lowering thresholds caused lockouts for legitimate users on flaky connections or those using PAM modules that log multiple failures per attempt (like Duo). The LLM approach adds context — it doesn’t just count, it interprets.

As I mentioned before in my post on AI-generated iptables rules from logs, the value isn’t in full automation — it’s in augmenting human judgment with pattern recognition that scales beyond regex.

Conclusion

This isn’t a silver bullet. The LLM adds latency, requires maintenance, and needs tuning. But for public-facing SSH, where attackers constantly evolve, it’s a useful layer. I’ve seen it catch slow credential stuffing campaigns that flew under Fail2ban’s radar for days.

If you run SSH exposed to the internet and want to reduce noise while catching subtle attacks, consider pairing Fail2ban with an LLM log analyzer. Start with manual review, validate outputs, and tighten the prompt over time. The goal isn’t to replace rules — it’s to make them smarter.

Cover image: Jefferson Lab · PDM (Openverse / kamu malı) · https://www.flickr.com/photos/53950384@N02/55166403733