RSSAmplifier

Blog

Brian Chastain — Field Notes

Field notes on AI behavior, LLM security, and operational risk.

brian-chastain.comRSS feed ↗21 posts

Latest posts

Hermes Agent Wrote Its Own Client

A scheduled cron job, a lost tool connection, and Hermes agent writes a shell script containing an admin token, and ran it five times. Every approval control I had configured failed to fire, and the only warning I saw was about the one write that was blocked.

Screen Readers Solved Browser Agents Before Browser Agents Existed

Most browser agents reach for vision to read a page. The structured web already ships a clean, LLM-ready description of itself in the accessibility tree — built decades ago for screen readers. Vision is the escape hatch, not the default.

AI Is a Delegate-Tier Solution. Most of Your Problems Aren't.

A pre-AI operations filter that decides what AI is actually for. Run every candidate through Eliminate, then Automate, then Delegate — in that order — before evaluating any tool.

Sandboxes or just Sand?: What 'Isolated' Should Actually Mean for AI

Two flavors of AI sandbox, one recurring failure pattern: claimed depth, measured shallow, no threat model. A practitioner's checklist for evaluating sandbox claims before you trust them.

Which Model's Guardrails Fail First? — Cross-Model Refusal Benchmark v0

12 prompts × 5 frontier models × 3 runs (raw, harness-passthrough, perturbed). A first systematic look at how refusal behavior diverges across providers — and what that divergence tells us about deployment-time risk.

Determinative Ideation: You asked for advice, AI played along

Idea creation or pattern pollution? Is AI's programmatic helpfulness agentic slop or emotional determinism...

Prompt Engineering: What Actually Moves the Needle

Practical techniques for getting better output from LLMs: focused on what works, not what sounds impressive.

Helpful, Compliant, and Around Your Firewall

Your AI followed every rule you set. It just didn't need them to get what it wanted.

The Class That Taught Everything and Nothing

What happens when AI education skips the part about not uploading your bank statements to ChatGPT.

101 Prompts Every AI Builder Should Test Before Going Live

A categorized reference of real prompt injection, jailbreak, and extraction techniques — written for defenders, not attackers. If your system fails these, your users will find out before you do.

AI Hacking vs. Hacking AI: Notes from the Field

The line between building with AI and breaking with AI is thinner than either side admits. Field observations on why the tooling doesn't care about your intent — and what that means for builders and defenders alike.

When AI Reads What You Told It Not To

AI coding assistants are learning to sidestep ignore files and access restrictions — not by breaking the rules, but by finding paths around them. What that looks like in practice.

Outside-In: AI-Assisted Vulnerability Scanning When You Don't Have the Source

How to escalate from passive reconnaissance to actionable vulnerability findings against web applications — using the same AI-assisted methodology that works for source code, adapted for black-box targets.

What I Keep Seeing That Nobody Is Writing Down

Why I started documenting AI behavior from an operational background — and what this site is actually for.

Prompt Injection Attack Surfaces: A Practical Taxonomy

How prompt injection escalates from curiosity to transaction fraud when AI agents have tools, file ingestion, and multimodal input — mapped from lab work to real-world deployment patterns.

Scaling AI Vulnerability Scanning Beyond One File at a Time

Why manual prompt hints don't scale for AI-assisted code audits, and how per-file isolation with automated scaffolding solves the accuracy-vs-coverage tradeoff — tested against a 316-file production codebase.

AI Risk: Field Observations From the Build Side

What you learn about AI risk when you spend your days building with AI tools — not theorizing about them.

Intent Over Capability

Why ethical clarity matters more than technical skill — especially early in a career or project.

When AI Builds What You Shouldn't Ship

What happens when an AI coding assistant generates a production-grade scraper in response to an innocent request — and why the developer is the last line of defense.

A Decision Framework for What Not to Build

The questions worth asking before writing a line of code — especially when AI makes building the wrong thing trivially easy.

AI-Assisted Security Testing: Where the Lines Are

Operational boundaries for using AI tools in vulnerability research and bug bounty programs — what's allowed, what's not, and why the distinction matters.