Detecting Suspicious sudo Usage with LLM from syslog and Blocking via iptables

I’ve been seeing more attempts where attackers compromise a low-privilege account and then probe for sudo misconfigurations. In my environment, I centralize syslog via rsyslog to a dedicated log server, which gives me a clean stream to work with. Instead of relying on static sudoers rules or simple keyword matching, I wanted to see if an LLM could help identify truly anomalous sudo behavior—like a user suddenly running sudo passwd root or sudo visudo after only ever using sudo apt update.

Here’s how I set it up.

Parsing sudo events from syslog

First, I ensure sudo logs are being captured. On Ubuntu/Debian, sudo writes to auth.log, which rsyslog forwards. I filter for sudo entries using a simple rsyslog rule:

authpriv.*   /var/log/sudo.log
& ~

Then I restart rsyslog. This gives me a clean file with lines like:

May 10 09:15:22 host sudo: furkan : TTY=pts/0 ; PWD=/home/furkan ; USER=root ; COMMAND=/bin/systemctl restart nginx May 10 09:16:01 host sudo: furkan : TTY=pts/0 ; PWD=/home/furkan ; USER=root ; COMMAND=/usr/bin/passwd root

I tail this file and extract the username, command, and timestamp.

Feeding sudo commands to an LLM for anomaly detection

I wrote a small Python script that runs continuously, reading new lines from /var/log/sudo.log. For each sudo event, it extracts the user and the full command, then sends a prompt to a local LLM (I use Ollama with mistral):

""" Analyze the following sudo command for signs of privilege escalation, system tampering, or credential access. Respond with only 'ANOMALY' or 'NORMAL'.

User: furkan Command: /usr/bin/passwd root """

If the LLM returns 'ANOMALY', the script triggers an action. I’ve tuned the prompt over time—adding examples of dangerous commands like sudo chmod 4755 /bin/bash or sudo echo 'ALL ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers—to reduce false positives. It’s not perfect, but it catches things rule-based systems miss, especially novel attack patterns.

Automatically blocking with iptables

When an anomaly is detected, I don’t just alert—I block the source IP at the firewall level. Since sudo runs locally, the relevant IP is the SSH source from the session. I extract that from the journal or from who/w output tied to the TTY.

Here’s the core action:

IP=$(who | grep "$TTY" | awk '{print $5}' | tr -d '()')
if [ -n "$IP" ]; then
    iptables -I INPUT -s $IP -j DROP
    logger -t sudo-llm-block "Blocked $IP after anomalous sudo: $COMMAND"
fi

I wrap this in a script that runs via systemd, so it survives reboots. I also add a timeout—after 20 minutes, the rule is removed automatically using iptables with recent module or a systemd timer that calls a cleanup script.

Reducing false positives with user baselines

One issue I ran into early: the LLM flagged rare but legitimate admin tasks, like a monthly certificate renewal script using sudo openssl. To fix this, I added a lightweight user baseline. The script keeps a local SQLite DB of previously seen (user, command) pairs. If a command has been seen before for that user, it’s assumed benign unless the LLM flags it with high confidence.

This hybrid approach—LLM for novelty detection, baseline for known behavior—cut false positives by nearly 80% in my tests.

Important limitations and warnings

This isn’t a replacement for proper sudoers configuration, auditd, or SELinux. It’s a complementary layer. Also, never expose the LLM prompt or logging socket to untrusted users. I run the Ollama instance locally, no network exposure.

And be careful with automatic blocking: if you misidentify the source IP (e.g., due to proxies or NAT), you could lock yourself out. Always test with a non-critical user first, and keep console or out-of-band access available.

Final thoughts

Using LLMs for log analysis isn’t about replacing engineers—it’s about augmenting detection for low-frequency, high-impact events that rule-based systems miss. In my case, it’s helped catch sudo abuse attempts that would’ve otherwise gone unnoticed until post-incident review.

If you’re already centralizing logs and experimenting with LLMs, this is a practical way to start. As I mentioned before in my post about [LLM-based sudo detection](https://furkanikkan.com/urun/llm-ile-syslog-tan-gecersiz-sudo-komutlarini-tespit-etme-ve-iptables-engelleme-122), the goal isn’t perfection—it’s raising the cost for attackers.

Stay safe, and keep your logs weird.

Cover image: Unknown · CC0 (Openverse / kamu malı) · https://www.rawpixel.com/image/11176762/computer-chip