Barry "Baz" Shaw

Cisco Live AMER 2026: Using LLMs and Endace Full Packet Capture for Incident Response

Original Entry by : Barry "Baz" Shaw

By Barry “Baz” Shaw, Senior Engineering Manager, Technology Partner Programs, Endace

At Cisco Live AMER 2026 Endace debuted an MCP (Model Context Protocol) server for the integration of EndaceVault into the emergent agentic Security Operations Center (SOC), providing access to 200TB of full packet data and on-demand packet analysis to all agents present in the SOC.

Going into Cisco Live we had key questions about the utility of PCAP data to agents, how to manage making terabytes of data available without overflowing their context windows, and whether the models could parse, analyze and make judgements from the PCAP data comparable to that of an analyst. Results from some simple tests are extremely encouraging and indicate a bright future for agentic AI in the SOC.

Endace Always-on Packet Capture

Two of Endace’s newest EP94C8 EndaceProbes were installed in the GovWare SOC to provide full packet capture to support the conference SOC directives of Protect, Educate, and Innovate. Full packet capture provides unique insight into all activities on the network, delivering critical context and evidence for Incident Response and Threat Hunting teams.

The EndaceProbes each hosted three VMs that were running Zeek and delivering critical log data into Splunk. Custom Zeek script additions provided additional valuable details around clear text passwords in email and HTTP, and the use of insecure protocols. Zeek was also used for file-carving and automated submission of objects to Splunk Attack Analyzer and a beta of Endace’s new Vault API used in a Cisco XDR automated workflow was field tested.

PCAP and AI – a key question answered

In this scenario we wanted to test the ability of a reasonably new on-prem model to make incident judgements based on PCAP data. Using the qwen-3.6 model and Open WebUI, a DNS buffer overflow attempt flagged by Cisco Secure Firewall, and raised as an incident in Cisco XDR, was chosen as the candidate.

Cisco Live USA 2026: Possible Buffer Overflow Attempt Flagged — Cisco Live AMER 2026: Possible Buffer Overflow Attempt Flagged

Using the Endace XDR workflow integration (which automatically creates Endace Pivot-to-Vision and EndaceVault links directly in the worklog) a quick pivot from Cisco XDR to EndaceVision shows DNS traffic present:

Investigating the suspect DNS traffic in EndaceVision

After pivoting from EndaceVision into Wireshark, we can see DNS activity and some large query responses. This verification served as the analyst judgement to be compared with the LLM findings.

Examining the actual packets in Wireshark lets us validate the AI agent's analysis. — Examining the actual packets in Wireshark lets us validate the AI agent’s analysis.

In Open WebUI a new chat was opened, the Endace MCP connected as a tool, and the following prompt was issued:

There was a firewall alert called ‘PROTOCOL-DNS Microsoft SMTP excessive answer records buffer overflow attempt’ recorded for the traffic between IP addresses <redacted> and <redacted> at 2026-06-03T15:24:01.000Z and 2026-06-03T15:22:33.000Z. Can you get a packet decode from Endace and either confirm or deny the existence of DNS responses that would match this alert.

The model’s reasoning is shown in the following screenshot:

The AI agent's reasoning before starting the analysis. — The AI agent’s reasoning before starting the analysis.

The Endace MCP tools contain detailed annotations that provide context to the model. Above we can see this context helping the model choose between different time inputs and correctly identifying that the start and stop times provided in the prompt were (accidentally) transposed in the prompt.

The AI agent prepares to retrieve and analyze the packet data.

Once the AI issued the tool call to the Endace Vault API via the MCP, the extraction of the related PCAP data began. To make this data more consumable by AI, it is converted to JSON by the Endace Vault API and streamed back in manageable sized chunks. The AI can request additional chunks as required, which protects the context window from being swamped by a large PCAP download.

An example of the returned JSONified PCAP data is shown below, this is the raw data that the AI has to make its determination.

A sample of the packet data returned in json format

After consuming the JSONified PCAP data the AI began its analysis, clearly having parsed the PCAP data and displaying an understanding of the DNS protocol. From the name of the alert, it understood the specific details pertaining to the alert and was able to look for these in the DNS packets:

The AI agent shows its reasoning and the resulting analysis. — The AI agent shows its reasoning and the resulting analysis – Part 1.

The AI agent completes its analysis and outlines its conclusion. — The AI agent shows its reasoning and the resulting analysis – Part 2.

The reasoning of the model makes for interesting reading. It can comprehend the nature and specifics of the alert and use this to reason about the DNS packets. It successfully found an oversized DNS response and provided unprompted enrichment by determining the hosting provider of the DNS server.

So far, the model has performed admirably, validating the firewall alert and the analyst judgement. To take things further, we asked the model to do some extra analysis over and above the human analysis we’d done. While this chat was user driven, we expect that this is the kind of autonomous activity AI agents will perform on their own.

The AI was asked to confirm the hosting provider and look at the contents of the large DNS response to determine if it contained a malicious payload or was in fact just a verbose and unusually large response.

Can you confirm that the destination host <redacted> is in fact Microsoft hosted? Is there anything in the large DNS response that could be malicious or is it just a DNS response?

The model has told us that the source of the DNS response is a reputable domain and that the payload of the response was just a large number of regular AAAA records. In a pre-AI SOC, the analyst would need to spend time inspecting the PCAP in Wireshark (if they had access to full packet capture in the first place) to determine this was a false positive.

Conclusion

While this example was intentionally not an autonomous agentic finding, the model was able to determine that this alert was a false positive by analyzing the PCAP. The fact that relatively small open-source models can successfully extract findings from PCAP data is a major validation of their utility in the agentic SOC. We expect that frontier models will provide even richer analysis of PCAP data when available in the SOC if the token cost can be justified. A possible architecture is an on-prem first approach, escalating to a frontier model in the event on-prem cannot make a confident determination.

By using an agent to investigate events like these, and having access to full PCAP data, alerts can be triaged by the agent, requiring only a cursory review by a human analyst to definitively close cases in record time. This validates the core concept that the agentic SOC will improve overall efficiency and accuracy by optimizing incident triage and freeing up time for analysts to work on the more complex incidents that threaten enterprise operations.

Acknowledgements

Our thanks go to the Cisco SOC team, led by Jessica Oppenheimer and Ivan Berlinson, for the opportunity to integrate EndaceProbes with the Cisco Live Agentic SOC architecture.

The SOC team is a collection of Cisco and Splunk experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco Security and Splunk tools.

The Endace and Cisco teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read more from Endace about the Cisco Live USA 2026 SOC:

Elliott Hinson: Supporting Encryption vs. Using Encryption – when the best laid plans go astray
Stephen Donnelly: Never underestimate Cisco Live!

Read related Cisco Team Blogs from the Cisco Live USA 2026 SOC:
https://blogs.cisco.com/security/clamer-soc-2026

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/

Event SOC Website
Visit Cisco’s Event SOC website for full details of the SOC setup, and download the whitepaper written by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html

Govware 2025: Full packet capture reveals suspicious traffic masquerading as MPEG in HTTP POSTS

Original Entry by : Barry "Baz" Shaw

By Barry “Baz” Shaw, Senior Engineering Manager, Technology Partner Programs, Endace

At GovWare 2025, Suspicious Video traffic was investigated using Full Packet Capture, revealing the traffic was in fact not video at all, but instead a potential threat masquerading as video to evade detection. Without packet capture this traffic would have gone unnoticed.

In this blog we reveal what led us to be suspicious, how we analyzed the traffic, and the clues that lead us to believe it may be threat traffic to a concerning destination.

Endace Always-on Packet Capture

SOC Findings and Lessons Learned

During a cursory review of the file mime-types submitted to Splunk Attack Analyzer, a number of video files were observed. To ascertain the contents of these files, an interactive session was launched in Splunk Attack Analyzer and VLC used to open them. Nothing was observed in these media files, and no codec information was available, which lead to the question “Why did Zeek carve these files in the first place?”

A custom search in Splunk to identify more information about files with mime-type video/mpeg or video/mpv showed a curious pattern emerging. A number of these files were approximately 500 bytes in length – far too small to be a legitimate media file. All these small files were associated with various conference WIFI addresses and all being sent to the same remote IP address.

A bespoke investigation into these IP addresses in Cisco XDR indicated that there were connections to known and suspected malicious sites:

Using EndaceVision the relationship between these addresses could also be seen in the Chords chart where 5 infected hosts were communicating with a single external address. The thicker lines indicated larger data transfers:

Pivoting to the hosted Wireshark instance available on all EndaceProbes quickly revealed the data flows from the conference WIFI to the remote address to be in the form of HTTP POSTs:

Following the HTTP streams in Wireshark then revealed an interesting User-Agent:

A search of the web indicated that these Host and User-Agent strings were used by a now-defunct windows application called SIM Instant Messenger. The domain associated with this application was also defunct and indications are that the project was discontinued in 2008.

On the face of it, this could be an old instant messaging client contacting a central server, but the age of the application and the fact the contents were impossibly small MPEG files continued to raise suspicion. Information relevant to the destination IP address also raised doubts that this was legitimate messaging traffic, even within the context of being embedded inside another application.

The question about how these MPEG files were carved was soon answered by Wireshark:

The first eight bytes of each of these POSTs were associated with different MPEG magic numbers, a few of which are shown below:

Notice that the bytes following the magic numbers in the screenshot above are the same, and there was a lot of repetition of content across all the traffic to the external address – could this be compromised endpoints obfuscating their phone home connections to a C2? The short, directional and periodic nature of these connections could also be seen to support this theory.

Unfortunately, no further context was able to be extracted from this small sample of connections, but further monitoring of small POSTs with MPEG contents external sites would be warranted on a corporate network – we will certainly be looking for similar patterns at future events.

It’s important to note that this threat investigation could not have been conducted without continuous full packet capture provided by EndaceProbe. At no point was this traffic flagged or alerted on, therefore a triggered PCAP or metadata only solution would not have captured these packets and their presence on the network and the risk associated with them would have gone unseen by the network operators. When securing the most important networks in the world it’s critical to have full packet capture for visibility to defend against attackers that will go to extreme lengths to obfuscate their activity.

Acknowledgements

Once again, our thanks go to the Cisco team lead by @Jessica Oppenheimer for the opportunity to include EndaceProbes in the GovWare SOC architecture. The SOC team is a collection of Cisco experts across many domains who were a pleasure to work and innovate with and we came away with a great appreciation for power of the Cisco Security tools. The Endace team was able to prove out integration innovations from previous SOC events and test these in earnest in a real-world environment in preparation for making them generally available to the market.

About GovWare

GovWare Conference and Exhibition is the region’s premier cyber information and connectivity platform, offering multi-channel touchpoints to drive community intel sharing, training, and strategic collaborations.

A trusted nexus for over three decades, GovWare unites policymakers, tech innovators, and end-users across Asia and beyond, driving pertinent dialogues on the latest trends and critical information flow. It empowers growth and innovation through collective insights and partnerships.

Its success lies in the trust and support from the cybersecurity and broader cyber community that it has had the privilege to serve over the years, as well as organisational partners who share the same values and mission to enrich the cyber ecosystem.

For more blogs in our Endace SOC series, see here:
https://blog.endace.com/tag/soc/

Threat actors are recording PCAPs, maybe you should too?

Original Entry by : Barry "Baz" Shaw

By Barry “Baz” Shaw, Senior Engineering Manager, Technology Partner Programs, Endace

On Aug 29th, Government Cybersecurity agencies from around the world released a joint advisory detailing how nation-state threat actors are compromising networks across the world, particularly in the US, Australia, Canada, New Zealand and the UK: www.cisa.gov/news-events/cybersecurity-advisories/aa25-239a www.darkreading.com/cybersecurity-operations/cisa-fbi-nsa-warn-chinese-global-espionage-system

These attacks are primarily focussed on telecommunications, government, lodging and military networks, and the tactics, techniques, and procedures (TTP) overlaps with APT (Advanced Persistent Threat) actors linked to multiple China-based entities. These threat actors are exploiting well-known vulnerabilities in VPN servers and web user interfaces on switches and routers. Even devices not owned by targets of interest are being compromised in order to provide additional attack pathways to the intended targets. Upon gaining a foothold in a network, persistence is achieved by modifying ACLs, opening services on non-standard ports to avoid detection, and tunnelling C2 and exfiltrated data to obfuscate malicious activity.

Of note in this particular instance is the use of PCAP collection on the target network by the threat actors. Once they’ve gained a foothold on the network infrastructure, the native capability of some routers to record PCAPs is then used to capture TACACS+ (authentication) traffic. When transmitted in clear text (or weakly encrypted) this authentication traffic exposes users credentials which can then be used to elevate the attacker’s access and enable them to move laterally across the network.

The use of network sniffing to extract credentials in authentication traffic is a common technique of threat actors (attack.mitre.org/versions/v17/techniques/T1040/). As we continue to see the stubborn use of unencrypted and weakly-encrypted protocols on networks, these insecure communications remain prime targets for credential gathering. Additionally, the uses of maliciously collected PCAP is evolving, with the ArcaneDoor campaign taking this a step further and exfiltrating captured PCAPs for remote analysis (attack.mitre.org/campaigns/C0046/). Exfiltrated PCAPs may contain anything from authentication data to file objects.

PCAP data is the ground-truth for what is happening on the network, and it is the source that all other network and security telemetry is derived from. Threat actors know this and value the raw unfiltered and unsampled intel that it provides about the target. This begs the question: if your adversaries see value in collecting PCAPs off your network, shouldn’t you be capturing full PCAP too!?

If you are not recording your network traffic, your security team has less visibility into network activity than your attackers – which makes the job of protecting your network impossibly difficult. With PCAP at their fingertips, SOC analysts can see exactly what’s happening on the network, making for faster, more accurate investigation and resolution of security incidents – as this excellent blog post from Cisco’s Steve Nowell describes.

That full PCAP data is so valuable to attackers also highlights a stark warning that wise defenders should heed. PCAP data must be protected and secured to the highest standards. Can you trust packet capture solutions that aren’t FIPS and Common Criteria certified? Or packet capture sources that can’t be properly locked down and protected from access by attackers?

For more blogs in our Endace SOC series, see here:
https://blog.endace.com/tag/soc/

Cisco Live US 2025 SOC – PCAP or it didn’t happen!

Original Entry by : Barry "Baz" Shaw

By Barry “Baz” Shaw, Senior Engineering Manager, Technology Partner Programs, Endace

Elevating Incident Response with the Ultimate Network Forensics

After a successful SOC @ RSAC2025, our team was stoked to be invited to help Cisco run the SOC at Cisco Live US (CLUS). We jumped at the chance to work with the Cisco team again. It was a great opportunity to innovate while helping Cisco protect and educate the attendees of the conference. Plus, the Cisco team is a lot of fun to hang out with—there’s a very infectious vibe in the SOC that has everyone buzzing for the entire week.

Packet capture is essential in the SOC. It provides an indelible record of all network activity, which is invaluable to the SOC team when investigating threats or security risks—hence the phrase, “PCAP or it didn’t happen.” For CLUS, we deployed two EndaceProbes with a combined storage of 864TB to continuously record all network activity delivered via 2 x 10GbE SPAN ports. This gave us the capacity to record at least several weeks of full network packet data—covering more than the entire duration of the show.

Endace Fusion integrations provided the glue between the Cisco Security suite and the packet data on the EndaceProbes, enabling analysts to quickly pivot from Splunk Enterprise Security/Splunk Cloud, Cisco XDR, Cisco Firepower, and Cisco Secure Network Analytics (SNA) through to EndaceVision and hosted Wireshark. When access to historical full PCAP is available in a seamless integration, security analysts are empowered with the ease of contextual access, and this simplifies the use of PCAP data, where previously it could be seen to be cumbersome to use.

The EndaceProbes each hosted two VMs that were running Zeek and delivering critical log data into Splunk. Custom Zeek script additions provided additional valuable detail around clear text passwords in email and HTTP, shining a light on the heavy use of insecure protocols, and ultimately driving automation to manage the unexpected volume. File-carving was enabled, and over 750,000 files were reconstructed from packet data, with over 40,000 samples submitted to Splunk Attack Analyzer (SAA) via Endace’s automatic submission software. SAA then sent over 12,000 files to Secure Malware Analytics (formerly Threat Grid) for dynamic analysis of the behavior.

SOC Findings and Lessons Learned

The SOC team was surprised and initially overwhelmed at the volume of unencrypted traffic on the network. Logging of passwords was coupled with a Cisco XDR automation that created an incident on each detection. This resulted in a heavy workload identifying and notifying users to educate and protect them in the future. The Splunk team developed a creative automated solution to notify users that the SOC detected their use of insecure protocols.

We even found a version of POP that was news to us all—APOP. This hashes the server timestamp in the response header with the user’s password to create a password digest. While this obscures the password, it only delays its inevitable retrieval, all the while the actual message bodies are still transferred in plain text!

In the theme of plain text passwords, reviewing the connections associated with one of these sessions showed a large number of file downloads in the Zeek log generated on EndaceProbe. This was one of many clients that used the free conference Wi-Fi to download Windows update files, but after filtering out the cab files in a Splunk search, we found a suspicious-looking file:

A search on this filename in SAA confirmed the presence of a malware download by this unfortunate user, whom the SOC team made every effort to identify.

There were also a few notable occasions where Secure Firewall alerts indicated intrusion attempts, which, after a pivot to EndaceVision, were pulled up in Wireshark for further analysis. One alert of note was a “BROWSER-IE Microsoft Internet Explorer Chakra.dll Array.filter type confusion attempt” indicating a malicious web server was trying to exploit a vulnerability in IE.

A review of the PCAP noted that the target client was, in fact, running Safari on MacOS X.

This indicated that even if the web server was launching a legitimate attack, the client was not vulnerable to this attempt and therefore no further action was required. This highlights the value of full PCAP, packets in related sessions that don’t trigger alerts can offer valuable insight and context to security analysts. This allows rapid determinations to be made with confidence.

Read more about the SOC at CLUS 25 on Cisco’s Blog here:

https://blogs.cisco.com/security/cisco-live-san-diego-2025-soc

Acknowledgements

Our thanks to the Cisco team, led by Jessica Bair-Oppenheimer and Steve Fink, for the opportunity to include EndaceProbes in the Cisco Live SOC architecture. The SOC team is a collection of Cisco experts across many Cisco solutions who were a pleasure to work with and innovate with. We came away with a great appreciation for the power and ease of use of the Cisco Security tools. The close collaboration resulted in the Endace team leaving with not only ideas for further integrations and workflow improvements, but also prototype integration extensions that were developed and proved out during the SOC.

To learn more about all the ways Endace integrates with Cisco, check out: https://www.endace.com/cisco.

For more blogs in our Endace SOC series, see here:
https://blog.endace.com/tag/soc/

Solutions

Cybersecurity

Cybersecurity

Threat Hunting

Network Performance Monitoring

Network Performance Monitoring

Application Performance Monitoring

Application Performance Monitoring

Industries

Learn

About Endace

Author: Barry "Baz" Shaw

Cisco Live AMER 2026: Using LLMs and Endace Full Packet Capture for Incident Response

Endace Always-on Packet Capture

PCAP and AI – a key question answered

Conclusion

Acknowledgements

More from Endace, Splunk and Cisco in the SOC series

Govware 2025: Full packet capture reveals suspicious traffic masquerading as MPEG in HTTP POSTS

Endace Always-on Packet Capture

SOC Findings and Lessons Learned

Acknowledgements

About GovWare

Read more

Threat actors are recording PCAPs, maybe you should too?

Cisco Live US 2025 SOC – PCAP or it didn’t happen!