Use case

First question in any Linux incident: what changed?

When an alert fires on a Linux server — credential abuse, unexpected outbound traffic, suspicious process — the on-call engineer’s first useful question is almost always the same: what’s different about this host since the last time we trusted it? Blackglass answers that in seconds.

Why baselines beat ad-hoc forensics for the first 30 minutes

Without a baseline, the first half hour of Linux IR is reconstructive: tail every log, pull bash history, run rkhunter, compare process lists against memory… and you still don’t know whether the SUID binary in /usr/local/bin was always there or appeared at 03:47.

With a Blackglass baseline, the answer is one click: every drift event since approval, in order, with severity, before/after diff, and the timestamp the change appeared. The on-call goes from “is this host compromised?” to “here are the four things that changed since Tuesday, two were ours, two weren’t” in under a minute.

What the IR-specific drift view shows

  • New SSH authorized_keys — per user, per fingerprint, with the timestamp the key first appeared.
  • New sudoers entries — every file in /etc/sudoers.d/that wasn’t in the baseline, including NOPASSWD grants.
  • New SUID/SGID binaries anywhere on the filesystem — full enumeration vs baseline. New SUIDs are a rare and high-confidence signal.
  • New listeners— any TCP/UDP socket binding that wasn’t in the baseline, with the binding process.
  • New systemd units & cron entries — persistence mechanisms that appeared since approval.
  • Loaded kernel modules — new modules vs baseline, with module path and signature status.
  • Package install / removal — anything that appeared from apt, dpkg, or rpm since the baseline, with install timestamp.
  • Hosts file & resolver tampering— common persistence-and-redirection trick that’s easy to miss without a hash diff.

IR runbook — how teams actually use this

  1. Page fires. Suspicious behaviour on prod-app-07.
  2. Open the host in Blackglass. Drift tab shows every change since the last approved baseline, ordered newest-first.
  3. Triage by severity. HIGH events first — new SUIDs, sudoers grants, new authorized_keys, sshd config flips. These are the high-signal ones that distinguish a compromise from a deployment.
  4. Cross-reference with deployment log. For each HIGH event: is there a corresponding approved change ticket? If yes → tag and move on. If no → keep digging.
  5. Decide.If the unexplained drift looks malicious, isolate the host (firewall rule, security group change, depending on your environment) and start formal IR. If it looks benign, acknowledge each event with the rationale so the next on-call doesn’t re-walk the same path.
  6. Capture the new baseline (optionally).Once the host is back to a known-good state, capture a new approved baseline so you don’t re-alert on yesterday’s changes.

What this looks like in practice

A real customer scenario from a recent walkthrough: alerts fire on a host running a third- party billing daemon. Blackglass drift view shows two things in the last 36 hours: a package upgrade matching the engineering team’s scheduled patch window (acknowledged), and a new entry in /root/.ssh/authorized_keys that doesn’t match any team member’s key inventory (escalated). Total time from page to escalation: 4 minutes.

Pairing with your existing IR tooling

Blackglass doesn’t replace your EDR, your SIEM, or your forensic imaging tools — it sits at the front of the funnel. The drift view is what you look at beforedeciding whether to spend the next two hours pulling a memory image and running deep forensics. For the majority of IR pages on a Linux host, baseline diff alone is enough to decide “benign” or “escalate”.

Drift events can be forwarded to your SIEM via webhook (CEF / JSON envelope) so the IR timeline already has them when an analyst opens the case.

Related use cases

14-day trial · up to 10 hosts · no card required