AI Agents Need an Audit Trail, Not Just an Activity Log

AI agents are starting to do more than produce text. They can inspect files, call business systems, update records and carry a task through several steps. That makes them more useful, but it also creates a practical problem when the outcome is wrong: somebody has to work out what the agent actually did.

GitHub has added OpenTelemetry support to the GitHub Copilot app, allowing enterprise administrators to send agent activity to compatible monitoring tools. A team can follow a Copilot session, see requests to AI models, inspect the tools the agent used and review a step-by-step trace when behaviour is unexpected. The settings can be managed centrally rather than configured separately by each developer.

It is a technical release, but the operating principle applies well beyond software development. If an AI system can take actions across a business, its work needs to leave evidence that is useful when a customer complains, a record changes unexpectedly or a task simply disappears between systems. A timestamp saying that the automation ran is rarely enough.

What GitHub Has Added

OpenTelemetry is an open-source framework for collecting and exporting operational data such as traces, metrics and logs. A trace follows one piece of work across the components involved, while the individual steps within it are recorded as spans. Instead of treating an agent session as one opaque event, a monitoring tool can present the route it took and the time spent at each stage.

GitHub says its Copilot app can now export this agent activity through an endpoint set in an enterprise-managed configuration file. The trace can show the flow of a session, including interactions with models and tools, and administrators can use monitoring systems they already operate. Prompt and response content is excluded by default, with a separate content-capture setting available for organisations that decide they need it.

That default matters. Useful monitoring does not require copying every customer message, document or model response into another system. An organisation can begin with structured facts: which workflow ran, which tools were called, whether each call succeeded, how long it took, what record identifiers were involved and where a person intervened. More detailed content should be collected only when there is a defined need, appropriate access control and a sensible retention period.

A Successful Result Can Still Hide a Bad Route

Many automations already produce logs, but logs tend to describe individual applications. The CRM may record that a contact was updated. The email service may record that a message was sent. The AI provider may report a successful response. Each component can look healthy while the overall job has gone wrong.

A trace joins those events around the piece of work the business cares about. It can show that an enquiry arrived, an AI model classified it, a customer record was found, a quote was prepared and an email was handed to the delivery service. When the result is wrong, the operator has a route to investigate rather than several unrelated logs and a vague suspicion that the AI made a mistake.

This is especially important because the final output does not always reveal the source of the failure. A model may have interpreted a request correctly but received an old price from a connected system. The CRM search may have returned two similar customers and the workflow selected the wrong record. A tool call may have timed out after creating an order but before reporting success, causing an automatic retry and a duplicate. Blaming the model in each case would send the repair work in the wrong direction.

Following One Job From Enquiry to Outcome

Consider a commercial equipment supplier that uses an AI-assisted workflow to handle service enquiries. A customer emails to report a failed refrigeration unit at one of several sites. The workflow reads the message, identifies the customer and location, checks the service agreement, creates a priority job in the field-service system and sends the customer a confirmation with the expected response time.

At 9:10, the customer receives a confirmation for the wrong site. The email itself gives the support team no clue about how that happened. With a useful trace, the operator can open the run linked to the enquiry and follow it from end to end. The trace shows that the model extracted the site name correctly. The following CRM lookup returned two branches with similar names, and the tool selected the first result because the workflow did not require an exact site code. The job was then created successfully and the email used the address returned by that lookup.

The business can correct the live job, contact the customer and fix the actual weakness. The lookup should require an exact identifier when one is present, and ambiguous results should go to a service coordinator rather than proceeding automatically. A regression test can replay examples involving similar site names and confirm that the workflow now stops safely.

The same trace can expose a different kind of problem. Perhaps the field-service API created the job but returned an error before sending its response. The workflow retried, creating two engineer visits. A trace that records the request identifier, retry and returned status makes that sequence visible. The repair may be an idempotency rule that allows repeated requests to produce only one job. Better prompting would not solve it.

Monitoring Can Create Its Own Risk

Capturing more data feels helpful during an investigation, but unrestricted collection can turn the monitoring system into another sensitive repository. Prompts may contain personal details, contract information, source code or credentials pasted in by mistake. Tool responses can include far more of a customer record than the workflow needed. If all of that is copied into a broadly accessible dashboard and retained indefinitely, observability has widened the exposure it was meant to control.

The monitoring design should begin with the decisions it must support. A service owner may need to know why a job was assigned, which system supplied the customer details and whether a retry occurred. They may not need the complete email body or the model's full response. Access should reflect operational responsibility, sensitive fields should be redacted, and retention should match a real investigation or compliance need.

Traces also need a common identifier that survives system handovers. If the AI service, CRM, job system and email platform each invent their own unrelated reference, the evidence remains fragmented. A workflow or correlation ID should travel with the job, while business identifiers such as a customer or order number are recorded carefully enough to support investigation without becoming an excuse to expose whole records.

There is a cost dimension too. Detailed telemetry consumes storage and can produce a great deal of noise. Routine successful work may need less detail than a failed, unusually slow or high-value transaction. Sensible sampling, alerts for defined failure conditions and a clear owner are more useful than a dashboard nobody has time to watch.

Build the Evidence Into the Workflow

GitHub's release is a useful sign of maturity in agent tooling. Attention is moving from whether an agent can call a tool to whether an organisation can understand that call afterwards. Businesses adopting AI-assisted operations should make that part of the design rather than waiting for the first costly error.

For each important workflow, decide what a successful outcome looks like, which handovers could fail and what evidence would distinguish one cause from another. Record model and tool activity in a connected trace, keep sensitive content out unless it is genuinely required, and route exceptions to somebody who can act on them. Then test the awkward cases, including ambiguous records, timeouts, partial success and retries.

Birdcage Tech helps businesses build AI integrations and automations that remain understandable after they go live. If a workflow crosses several systems and nobody can explain why it produced a particular result, we can map the complete route, add practical monitoring and strengthen the failure handling around it.

Image: GitHub Copilot conversation followed by a trace in Splunk Observability Cloud. Source: GitHub.