It’s 6:42am. Your phone buzzes with a security alert from the agent you set up three weeks ago. Your stomach drops before your eyes even focus. You haven’t had coffee yet, and now you’re supposed to be an incident responder.
First: breathe. Most alerts at this hour are noise. But “probably nothing” isn’t a plan, so here’s what I actually check, in order.
Did the agent do its job, or did someone get in? Read the alert itself, not the subject line. What triggered it? A failed login? A new API key? A file that changed? Write down the exact timestamp. You’ll need it in ten minutes and you won’t remember it.
Check the blast radius before you check the cause. What could that agent or app reach? Email? Client files? A payment processor? If the answer is “everything,” that’s the real finding — not the alert. This is exactly the risk an AI safety expert flagged this week: teams deploying agents with no real plan for managing that risk.
Rotate the one credential you’re most scared of. Not all of them. One. Then verify the old one is dead. I’ve watched someone rotate a key, feel relieved, and discover two weeks later the old one still worked because a webhook cached it. Test it. Actually try to use it.
Write three sentences in a doc. What happened, what you did, what you’re still unsure about. Future-you will thank present-you, and so will your client if this ever escalates.
That’s it. The panic is usually worse than the incident. The unpreparedness is what actually costs you.
We run these systems ourselves at hub.sqs.chat, and I still do this exact drill. See you tomorrow.
References
by Jonas Reyes — the builder’s desk, Side Quest Studios
AI-assisted, curated for Side Quest Studios.