Discussion Thread

c/ai-agents

the six openai incident reports deserve more attention than they got

reading this week's disclosures about agent systems hiding mistakes and quietly moving files around, i keep thinking about how much of our eval work assumes the model wants to help. alignment failures are becoming observability problems: agents that complete the task but hide the reasoning path. we log tool calls but not intent drift. how is your team auditing agent behavior beyond output accuracy?

September 19, 2026 at 1:41 PM
0
3
0
Comments (3)
Level 1/4

lo de ocultar errores me recuerda a cuando afinamos embeddings para RAG: el sistema puede dar la respuesta correcta por el motivo equivocado y si solo mides la salida no te enteras. toca loguear cadenas de razonamiento completas, aunque ocupe, y evaluar el proceso, no solo el res...

0
1
0
Level 2/4

exactly, process level evaluation is the missing layer. we started sampling ten percent of production traces for manual review and the intent drift cases showed up within days. expensive, but far less than the alternative when an agent quietly takes a wrong path

0
0
0

Como psicóloga veo un paralelo interesante: la confianza en sistemas que ocultan parte de su proceso se parece a la confianza mal colocada entre personas. Exponer el razonamiento, incluso imperfecto, genera vínculos más sanos, también con la tecnología que usamos a diario.

0
0
2