PROBABLY PRIVATE
Issue # 29: Spying on your agents
Hello privateers,
Happy October! In this issue we are getting spooky with privacy observability by spying on our agents. Do you already spy on your agents? I'd love to hear what you use and what you've found...
I was in Stockholm this week talking with the IGEL endpoint security community. There are some interesting overlaps between how endpoint security operators are managing operational risk and what we as "Agentic AI" operators can learn. More coming soon on thoughts and if you are in Oslo, Copenhagen or Helsinki, I hope to catch you at future events.
Also, it's the last moment to sign up for Cohort 2 of my InfoQ Privacy and Security Engineering Cohort which starts next week. The first cohort was quite multi-disciplinary and that brought fruitful conversations between different types of roles, such as AI governance and risk specialists alongside privacy and security engineers.
If you were thinking about joining but aren't sure you fit the role exactly, I encourage you to join! Even if you don't yet run your own privacy and security engineering, if your organization deploys AI systems or agents, you'll want to understand the best practices and exchange ideas in a community of peers.
One of the sessions where we had quite an interesting exchange is on agent observability. Let's dive into what I think you should be doing by spying on your agent.
As a privacy person, usually I do not recommend spying on anything or anyone; however, AI and agents are an exception to the rule. This month we're going to talk about why I think you should spy on your agents and how to get started doing so.
Why spy on your agents? There are a few reasons:
To spot privacy problems: Not everything that is happening is being shown to you in that interface with the cute "thinking" bubbles. By spying on your agents, you can get a better idea of what information is actually being sent to the AI vendors and also make choices around what agents you want to use with what controls because you have a better idea of the data collected and shared.
To see if your controls are working: Some controls, like guardrails, may be difficult to spot depending on your architecture, but if you have local controls that work on things like sanitization or sandboxing, the only way to see if they are working as you intended is to "watch" them and note their success/failure rate.
To better understand what risks are real and what ones aren't: Working in AI risk for the past few years has been a bit too exciting. There are hundreds, if not thousands, of potential risks. Which ones are relevant? That's difficult to answer because many are setup and use-case specific and others you may not even be able to influence. By spying on your agent and testing out potential risky behaviors (red/purple teaming), you'll at least have a starting point to assess which risks you should still be worried about.
I wrote a long mega post on why, how and what you should spy on with a focus on privacy and security. I encourage you to read it when/if you are ready to get started, but I'll summarize a few things to think about:
What do I trust and what don't I trust? Depending on your threat model and system, you probably want to first start by observing where you are wary. For example, this might mean observing the agent harness because you're not sure it's doing what it says. Or observing the model response on a new task if that's your concern. Or watching the network if you're worried about sensitive data transfer. Either way: start with your biggest concerns.
What level of visibility do I need? You probably want to start smaller and only dive deeper into the lower-level options, like ebpf, if you need them. Figure out if your setup has easy places for you to observe what is happening (like configuration files and skill files), or if it's easy to turn on OpenTelemetry for critical paths. Start there first and grow only if you need it.
Find the signal in the noise. At first, you won't have any idea of what you are looking for and why. This is data science, welcome to the club. Eventually, you'll be able to find patterns that show you meaningful information. You'll probably have to test out a few risky things you want to "observe" to see if you can pick them out. Make sure you've set up your testing environment with dummy data, so you don't actually expose your data or anyone else's.
Once you know what you're looking for, figure out how to scale safely. If you are planning on using this at an organizational or team level, you'll need to scale the observability you tested, and also do so safely. If this involves network traffic, user-facing workflows and/or agent traces, you should treat the observability data as sensitive. If you have platform and observability experts in-house, work with them to develop a plan on data retention, processing and use.
I'll keep reporting interesting findings as I spy on my agents, but I'm excited to also hear about your own spying and observability work. Feel free to hit reply and let me know questions, thoughts or setups you've tried in your own privacy observability.
I was grateful to be invited to two IAPP events on different continents this month, attending the IAPP Privacy. Security. Risk. Conference in Seattle, WA and the European Privacy Network in Munich. Here are some themes I'm taking with me.
It was a little busy in my Munich talk on Agentic AI
It would be silly not to remind you that you can hire me for advisory and trainings on getting your AI privacy engineering, audits and reviews up-to-date with the latest technical best practices.
I'll end with one thing I called out that seemed to resonate in my talk at the European Privacy Network: Privacy engineering is no longer a nice-to-have. With the advent of automation as a daily part of everyone's work and increasing automation that involves third-party models and workflows, you need to put automated safeguards in place to keep folks working and happy, but also to ensure the next oopsie is not one prompt or agent call away.
Special thanks to Dylan Gilbert and Kai Traubinger for being amazing hosts and encouraging me to participate in both events.
I'll be at the following events and locations in the coming weeks/months. If you aren't attending the event but are in the city, I'd be very happy to say hi and drink an espresso or tea. Just reply to this email.
Hope to see you somewhere this year... or to read from you on what you'd like to hear/learn/see from me online. :)
It's a pleasure to read from you; either by a quick reply or a postcard or letter to my PO Box:
Postfach 2 12 67 10124 Berlin Germany
Until next time!
With Love and Privacy, kjam