Splunk .conf26: Letting the Agentic SOC Handle the Drudgery So the Humans Can Dig Deeper

Original Entry by : Elliott Hinson

By Elliott Hinson, Integrations Engineer, Endace


Elliot Hinsen, Integrations Engineer, EndaceWorking at the Agentic Security Operations Center (SOC) for Splunk .conf26 was a very exciting experience. The strangest part was that I did not spend most of my time doing any of the Tier 1 style SOC analyst work. Sure, there were tickets to close, a few devices trying to phone home to any TOR exit node they could reach, some interesting alerts related to gaming software installed on corporate devices that should not have anything like that, and many other discoveries that my colleagues and I had to take care of.

But the most interesting part to me was the time made available, now that the bulk of the drudge work was offloaded to AI agents. By the time you managed to look at an incident ticket the agent had already done a bunch of data collection that you would have had to do manually, so all we really had to do was assess the evidence provided, make a judgement call on how to proceed, and get back to doing anything more interesting than basic or repetitive triage.

Why be a Tier 1 analyst when you can be a Tier 3 threat hunter?

So, we had all this free time now, how did we spend it? The Event Agentic SOC has three objectives: Protect, Educate, and Innovate. The Educate part was being taken care of by the 30-plus SOC tours that were going on, and now that the agents were making the Protect part of the job less time-consuming, there was plenty of time for us to focus on Innovate during the conference.

The team at work at Splunk .conf26
The team at work at Splunk .conf26

One innovation I had the most fun working on was the newly added JA3/4+ style traffic fingerprinting. For readers not familiar with the SOC architecture, I will skip most of it and focus on the parts that matter for JA3/4+ style scanning. The SOC at Splunk .conf26 had three main components that enabled our network fingerprinting innovation work. First, was the incoming SPAN port that mirrored all of the conference traffic, from every subnet and access point, into a single fiber that came into the SOC. This was a twin set of 10Gbit links that pumped terabytes of data into the second of the three core components of the SOC: two EndaceProbe packet capture appliances. These EndaceProbes recorded every single packet that traversed the network from the moment the SPAN was turned on until we shut them down when the conference ended.

The EndaceProbes not only provide massive high-speed storage capacity, and multiple packet capture interfaces, they also support integration with a whole suite of 3rd-party monitoring solutions, and enough onboard compute power and Application Dock™ hosting capacity (think virtual machines) to host a wide range of network monitoring security software, from Snort, to Zeek, to Suricata, and more.

At Splunk .conf26 we thought “why choose between Zeek and Suricata when we can run both”? With both tools generating massive volumes of log data, it all has to go somewhere. Where should it go? If only there was a vendor with robust enough log ingestion systems to keep up with the network speeds our EndaceProbes can operate at? If only there was a vendor with a trusted, reliable search and query system that could sift through massive volumes of logging data to provide a budding threat hunter with all of the intel they needed to really see the big picture?

Oh wait… this is Splunk .conf and Splunk Enterprise Security is the platform!

Splunk + Endace: pairing the wine with the cheese

Splunk and Endace are uniquely synergistic. Splunk searches and bubbles up logs and alerts, and with the Endace Pivot2Vision integration, you can go from the searchability of Splunk logs to the authority and clarity of Endace packets with a single click.  While we were tuning the SOC during setup, we took the approach of “turn every single bit of logging and scanning on that we can and separate the signal from the noise later”. After all, Endace was reading every packet, and it could turn that into logs, detections, etc. And Splunk is well known for being able to drink from the firehose. Plus, how often do you get a Splunk Cloud instance with essentially a blank check on how much data you can push into it? So, we opened up the fire hose and let Splunk drink up, and oh boy did it.  Splunk has a very rich search and dashboarding interface and a powerful query language that can tear through log data at ridiculous speeds. However, dumping every single bit of data you can get out of your firewall or IDS system into Splunk is both computationally and financially wasteful. That is why it is so important to tune your logs so that you are ingesting useful stuff and skipping the noise.

With the Endace hardware the economics are reversed. You buy an EndaceProbe, put it in your network, and let it fill up with as much packet data as it can hold. It becomes an exercise of “I paid for the whole hard drive, I’m going to use the whole hard drive”. And while EndaceProbes can hold massive pools of packet data and can filter down that data to what you are looking for with the EndaceVision web interface, it can’t hold a candle to the immense power of the Splunk query language. Need to find all the traffic coming from a single IP? EndaceVision. Need to see a specific IP conversation between two hosts? EndaceVision again. Need to look at the exact contents of a specific TCP session that happened 3 days ago? Those EndaceProbes won’t even break a sweat. Need to do a complex query involving multiple lookups, regex across several plain text fields, generating custom columns that are calculated by generating a risk score based on an amalgamation of other plain text fields? That is squarely a Splunk job.

But what if you are looking for something that wouldn’t generate an alert? Not something dangerous, just something odd. Well, that line of questioning leads right into the new traffic fingerprinting work we implemented during this conference.

JA3? JA4? NPF? Whose fingerprints are these?

We just established that it doesn’t make sense to load raw packet data into Splunk. EndaceProbes provide amazingly quick search and data mining of packet data across terabytes of packet data – for both human and AI agents. What they don’t provide is the broader capability that Splunk offers to query across multiple telemetry sources and run complex queries. But, when we pair them, we get an amazing combination that lets us see the 30,000 foot view, and quickly correlate data, as well as being able to dive right down to definitive packet level evidence to understand individual sessions and packet sequences.

An example of the power of combining the full packet recording capability of EndaceProbes with the powerful search, querying and data correlation capability of Splunk, is using EndaceProbes to generate fingerprints from encrypted traffic sessions. You can then configure your Splunk instance to ingest these fingerprint logs so that you can do connection level analysis without having to have all the connection data itself loaded into Splunk.

 Framework  

 Best At                                                                                  

 JA3        

 Fingerprinting TLS clients using a simple, widely adopted hash.                          

 JA4        

 Correlating client behavior across multiple protocols and network layers.                

 Mercury NPF

 Identifying devices and applications from encrypted traffic using rich protocol metadata.

We ended up going with the JA4+ series of fingerprints because it offered more detection versatility than the older JA3 series of fingerprints. We didn’t employ Cisco’s Mercury NPF to analyze encrypted traffic because there was already an existing Cisco encrypted traffic analysis engine running in the SOC as part of Cisco Secure Firewall Firepower Threat Defense Encrypted Visibility Engine. So, it did not make sense to duplicate this. However, we could have used Mercury NPF via a Zeek plugin if that had not been already available to us.

What is JA4+ fingerprinting?

The list below shows various JA4 style fingerprinting styles. The key takeaway is that different types of fingerprints reveal different types of information about the data traversing the network. 

Fingerprinting data can be generated continuously, as traffic is recorded on EndaceProbes, and sent to Splunk where it provides very useful insight into encrypted traffic sessions. Generating this data continuously – rather than relying on a trigger to detect suspected malicious activity and then send a log to Splunk – means you (and your agentic tools) always have the fingerprint data at hand when it’s needed, even when there was no triggered detection. As we’ll see below, this can enable you to do detections and investigations that you might not have initially envisaged would be useful.

 Fingerprint 

 Protocol / Layer      

 Purpose                                                                                                                    

JA4

 TLS Client            

 Identifies client applications and TLS libraries based on ClientHello characteristics.                                     

JA      

 TLS Server            

 Fingerprints TLS server responses and server-side TLS implementations.                                                     

JA4H

 HTTP                  

 Identifies browsers, applications, and tools based on HTTP request behavior and headers.                                   

JA4L

 Network Latency       

 Estimates network distance and latency characteristics between hosts.                                                      

JA4LS

 Network Latency Server

 Server-side counterpart to JA4L for latency and path analysis.                                                             

JA

 X.509 Certificates    

 Fingerprints certificate generation patterns to identify common tooling, infrastructure, or certificate issuance behaviors.

JA4SSH

 SSH                   

 Identifies SSH clients, servers, and automation frameworks from SSH negotiation behavior.                                  

JA4

 TCP                   

 Fingerprints TCP/IP stack behavior for operating system and networking stack identification.                               

JA4TS

 TCP Server            

 Server-side TCP stack fingerprinting.                                                                                      

JA4TScan

 TCP Scanning          

 Provides TCP fingerprints useful for identifying and classifying scanned hosts and services.                               

JA4

 DHCP                  

 Identifies devices and operating systems from DHCP request characteristics.                                                 

JA4D6

 DHCPv6                

 IPv6 version of DHCP fingerprinting for device identification.                                                             

Now that is a long table! We are going to specifically focus on the JA4SSH fingerprints for brevity, but you can imagine all the fun ways in which you could combine and cross reference these fingerprints to find interesting outliers or even build detections.

A JA4SSH fingerprint looks like this: `c64s64_c31s149_c64s0` which may look like just a string of letters and numbers, but there is actually a lot of information here. Each of the segments delineated by the underscore describes a different SSH behavior that was detected on the wire. `c64s64` is a description of the client and server payload sizes, `c31s149` is the description of the Packet counts and `c64s0` is a description of the ACK behavior.  If we decode these a little further into a nice table, we can start to see a clearer pattern.

 Metric        

 Value

 Client payload

64

 Server payload

64

 Client packets

31

 Server packets

149

 Client ACKs   

64

 Server ACKs   

0

You can deduce from the balanced size of payloads and the relatively balanced and low size of packets that this is an interactive session. If we saw a lot of unbalanced payload sizes or packet counts, we might suspect that this was a bulk download of some kind, perhaps using something like SCP. 

Now suppose we wanted to detect bulk downloads or uploads, and possibly even filter for a specific IP address that might be important on the network? For example, probably no one should be SSHing into the firewall and uploading or downloading large quantities of data. The best part about having fingerprint data available is that with a very small amount of context describing the syntax for these network fingerprinting technologies, the Agentic SOC can pick up on this and use it to find outliers that it would not have thought to look for before.

For example, below is an image of an agent generated Splunk query, searching for any large bulk downloads or uploads:

Agent-generated Splunk query to search for large bulk uploads or downloads.
Agent-generated Splunk query to search for large bulk uploads or downloads.

With a short description of how the fingerprints we had in place could be broken up, the agent could ascertain several instances of large SSH traffic sessions and categorize them by what is probably happening inside that SSH tunnel – using heuristics that would normally not have been available without the fingerprinting data.

It’s always good to double-check your AI’s work! Using the EndaceVision web GUI, we can easily go back and review the SSH traffic from that period… I wonder when this bulk download happened?

Using EndaceVision to analyze the selected traffic.
Using EndaceVision to analyze the selected traffic.

What started as a strange-looking JA4SSH fingerprint turned into a hypothesis. The fingerprint suggested behavior more consistent with a bulk transfer than an interactive shell. A quick Splunk search surfaced the candidate traffic. A single Pivot-to-Vision click later, and we were looking at the packet capture that explained exactly what happened. For the curious, the first two spikes in the graph above were a vulnerability demo put on at Splunk .conf26 for training, and the second two large spikes were a user uploading a large amount of encrypted data to a cloud VPS.

This sequence of events is what I found most interesting about working in the Agentic SOC. None of this investigation started with an alert. Nobody woke up that morning wondering what the JA4SSH fingerprints of the conference network looked like. There wasn’t a ticket demanding that the work be completed by the end of the shift. This was curiosity-driven threat hunting enabled by the fact that the repetitive work had already been handled for us.

The agents were closing the loop on the obvious incidents. They were gathering enrichment data. They were collecting context. They were doing the sort of repetitive investigative tasks that traditionally consume the bulk of a Tier 1 analyst’s day. And because of that, analysts were free to spend time asking questions that normally fall into the category of “that would be nice to investigate if I ever had a spare afternoon.”

It turns out, when you free up analysts’ time and give them a chance to dig deeper, spare afternoons can turn into innovations that provide interesting (and useful) results.

Acknowledgements

Our thanks go to the Splunk .conf26 SOC team, led by Jessica Oppenheimer and Paul Pelletier, for the opportunity to integrate EndaceProbes with the Splunk .conf26 Agentic SOC architecture. 

The SOC team is a collection of Cisco, Splunk and Endace experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco and Splunk tools.

The teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read all about the Agentic Security Operations Center at Splunk .conf26, including an overview from Jessica Oppenheimer, and posts and use cases from Splunk and Cisco SOC team members:

Read about the Endace team’s experience working in the SOC at Splunk .conf26:

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Cisco Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC-in-a-Box setup, and download the detailed architecture blueprint and report by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Splunk .conf26: Richer Data for the Agentic SOC

Original Entry by : Andreas Löf

By Andreas Löf, Software Manager, Endace


Dr. Andreas Löf, Software Manager, EndaceEndace supported Splunk in two different ways in the Agentic Security Operations Center (SOC) at Splunk .conf26. Firstly, capturing and recording every packet transmitted across the Spunk .conf network to create a source of ground truth so all events can be verified. Secondly, creating rich metadata ingested by Splunk and used for detection and investigation in the Agentic SOC.

During Splunk.conf26 we expanded on the data the EndaceProbe Appliances delivered to Splunk by enabling full Suricata detections and expanding the pre-existing Zeek detections we had from previous events.

These expanded rulesets allowed us to cross-correlate events between Zeek, Suricata, and Cisco Secure Firewall Threat Intelligence. Each one of these sources provides a part of the picture. Correlating them gave us better quality detections and enabled more robust investigations. EndaceVision™️ and embedded Wireshark™️ then provided the ability to evaluate findings by analyzing the actual packet evidence.

Real-world Investigation Scenario

During .conf26, there was an analyst’s queue in Splunk Enterprise Security (ES) of alerts that had been generated. The AI agents would then investigate the different events, classify them, and suggest further steps for the analysts to follow up for the event.

At .conf26, the AI agent flagged and investigated multiple instances of hosts connecting to a specific external IP address that was deemed potentially risky. We verified the agent’s findings and discovered the hosts were connecting to Baidu, often known as the Google of China. As .conf26 is attended by people from all over the world, there’s nothing inherently risky with visitors connecting to Baidu. As a part of the investigation, we verified that the connections came only from hosts on the attendee network – no instances originated from conference infrastructure – and that there was no sign of C2 traffic. We also confirmed the IP address was genuinely registered to Baidu and wasn’t spoofing a Baidu host.

Further examination of the traffic showed us there were a multitude of hosts connecting to the IP address flagged by the AI agent. Analyzing all the related traffic on the EndaceProbe showed that it was genuine traffic, primarily encrypted.

Analyzing the suspicious traffic in EndaceVision
Analyzing the suspicious traffic in EndaceVision

 

The EndaceProbes were also able to identify most of the traffic in the above screenshot as Baidu traffic, through detection rules built into the EndaceProbe’s Deep Packet Inspection (DPI) engine.

We then further verified the traffic by looking at the raw packets in Wireshark, where we could see that the traffic was to a server presenting itself as baidu.com, as can be seen from the Wireshark screenshot below.

Inspecting the traffic using embedded Wireshark hosted on the EndaceProbes
Inspecting the traffic using embedded Wireshark hosted on the EndaceProbes

This then allowed us to confidently move the investigation back into Splunk and look at all alerts that had been generated around these systems. We verified that all the events were going to eitherwww[.]baidu.com or sp1[.]baidu.com./

Checking all www.baidu.com alerts in Splunk
Checking all www.baidu.com alerts in Splunk

 

 

 

 

 

Checking all sp1.baidu.com alerts in Splunk
Checking all sp1.baidu.com alerts in Splunk

We could then confidently close these events as false positives, having verified the premise that the traffic was benign in nature.

Richer Data – More Certainty

During the preparation phase of the Agentic SOC for .conf26 we enabled more metadata collectors running on the hosted DockOS VMs on the EndaceProbes. These ran both Suricata™ and Zeek™ and performed deep packet inspection of all network traffic, without any interruptions to the capture and recording.

Comparing Zeek and Suricata shows that they have a different detection and reporting rate when operating across the same traffic.

System

Event Findings

Suricata

208

Zeek

461

Further investigation into the specific events (TLS inspection) shows that Zeek uncovered more events and more hosts than Suricata. However, they are different types of systems and detect different things.

This highlights the usefulness of running both systems in tandem and processing the same data. This generates richer metadata for both AI Agents and Humans to investigate and react to, as well as more detections. By correlating events across multiple sources, we get stronger evidence as to whether the event is a true positive, or a false positive. We can also more easily determine whether it is benign.

Detection Tuning

Over 30 minutes, Suricata generated 5.3 million events and Zeek about 3.1 million. For both systems, this excluded the stats reporting.

Suricata

Suricata was configured to capture traffic from eight Endace Virtual Data Acquisition and Generation (DAG) cards that received a flow-coherent, balanced traffic stream from the EndaceProbe hosting the virtual machine on which the system was running.

The events were configured to be logged into eve.json, which was then ingested into Splunk via the Splunk Log Forwarder. The logs were aggressively rotated after ingestion to manage disk space.

We enabled the open Emerging Threats rule set (approximately 55k rules) as well as most of the protocol dissectors. There’s overlap with the configuration we applied to Zeek, but the example above shows a difference in the system’s effectiveness, which makes the overlap valuable.

Zeek

Like Suricata, Zeek was also configured to capture network traffic from eight Endace Virtual DAG cards with the same setup as for Suricata.

Unlike Suricata, Zeek typically writes output from each analyzer to a separate log file. The Splunk Log Forwarder ingested these files, and they were then rotated to conserve disk space. Again, like Suricata, we enabled most of the dissectors. In addition to the protocol dissections, we had the Endace custom file extraction code and sample uploader enabled. Extracted uploaded files were then analyzed and used to generate events for any suspicious files.

Conclusions

Endace DockOS provides the ability to run both Suricata and Zeek in tandem and ingest exactly the same set of data. Both systems have some overlapping functionality but are fundamentally different in their use cases and types of alerting.

Suricata is rule and alert-based, designed to fill a role similar to Snort. Zeek, on the other hand, is designed to analyze the network traffic and generate rich metadata from it.

There is some overlap in the data Zeek and Suricate can provide, but there are also large areas where each system can create an alert/event that the other does not. We can also observe how the event rates differ from the same traffic across Suricata and Zeek. By running both systems in tandem and feeding them the same traffic from our EndaceProbes we get a better overview of the network activity, and we can pivot back to the raw packets where necessary to confirm findings or do more in-depth investigation.

As discussed in The Zero-Day Blind Spot: Why Your Agentic SOC needs Retrospective Packet Replay, it’s also possible to replay existing traffic against new detection rules to validate whether any systems have been affected by a zero-day attack. That can be extremely useful to determine whether a successful attack took place (and you have been impacted and need to respond) or, conversely, prove that no successful attacks happened.

Preparing for the next Agentic SOC

During .conf26 the human analysts made a great amount of use of the expanded event logs from Suricata and Zeek to validate AI agent investigations. In future SOCs we are looking to expand on the AI agent capabilities and the base events to make greater use of event correlation, when possible, to provide a stronger evidence chain for investigations.

This will allow AI agents to draw conclusions more quickly and easily when investigating events, and human analysts to more quickly and easily validate an AI agent’s conclusions.

Making Zeek and Suricata evidence available alongside Splunk Events gives both human analysts and AI agents much greater certainty for fast investigation and effective response. While both tools provide some common data, together they offer a very rich set of evidence, with the ability to pivot to Endace’s full packet data when needed, that gives both AI agents and human analysts deep visibility into all network activity, backed by primary, definitive, packet evidence.

Acknowledgements

Our thanks go to the Splunk .conf26 SOC team, led by Jessica Oppenheimer and Paul Pelletier, for the opportunity to integrate EndaceProbes with the Splunk .conf26 Agentic SOC architecture. 

The SOC team is a collection of Cisco, Splunk and Endace experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco and Splunk tools.

The teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read all about the Agentic Security Operations Center at Splunk .conf26, including an overview from Jessica Oppenheimer, and posts and use cases from Splunk and Cisco SOC team members:

Read about the Endace team’s experience working in the SOC at Splunk .conf26:

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Cisco Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC-in-a-Box setup, and download the detailed architecture blueprint and report by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Splunk .conf26: Why Endace Full Packet Capture Becomes Mission-Critical in the Agentic SOC

Original Entry by : Michael Morris

By Michael Morris, Director of Global Business Development, Endace


Michael Morris, Director of Global Business Development, EndaceWhy Full Packet Capture Matters More as AI Becomes a Security Analyst

The security industry is entering a new operational era. For years, Security Operation Center (SOC) modernization focused on improving detection. Better analytics. Better threat intelligence. Better correlation. More automation. Today, the focus has shifted.

The emergence of Agentic SOC architectures is changing the fundamental question from: “Can we detect threats?” to “Can we trust the conclusions generated by autonomous investigations?“

As AI agents take on larger portions of triage, enrichment, evidence gathering, and investigation, one reality becomes increasingly apparent: The success of the Agentic SOC is not determined by AI alone. It is determined by the quality of evidence available to AI.

At Splunk .conf26, Endace demonstrated why full packet capture is rapidly becoming one of the most important sources of evidence in modern security operations.

Security Teams Have Solved the Visibility Problem

For most organizations, the lack of data is no longer the problem. SOCs are flooded with telemetry:

  • Security logs
  • DNS activity
  • Endpoint events
  • Firewall records
  • NetFlow
  • Application telemetry
  • Identity data
  • Threat intelligence feeds
  • Cloud activity

The challenge today is not collecting data. The challenge is determining which data is trustworthy enough to support increasingly autonomous security decisions. As investigations become more automated, the distinction between telemetry and evidence becomes critical. Telemetry tells you something happened. Evidence tells you exactly what happened. That distinction separates packet capture from virtually every other data source in the SOC.

The Agentic SOC Needs Ground Truth

One of the most important architectural principles emerging from modern security operations is the concept of ground truth.


AI systems consume evidence. They assess confidence. They eliminate noise. They prioritize risk. They recommend actions. But every recommendation ultimately depends on the underlying data. If the evidence is incomplete, conclusions become uncertain. If the evidence is delayed, investigations slow down. If the evidence is ambiguous, analysts lose confidence.

Full packet capture solves this problem by preserving the original network conversation. Instead of relying solely on summaries, metadata, or derived analytics, analysts and AI agents can continuously return to the authoritative source. In a world increasingly driven by machine reasoning, packet capture provides something rare: A verifiable record of reality.

Splunk .conf26 Demonstrated the Scale of the Challenge

The event network operated at a scale that most enterprise security teams would recognize immediately.

Endace captured:

  • 30 TB of packet data
  • 31.6 billion packets
  • 154.7 million unique sessions
  • 9,443 unique clients

Beyond pure scale, packet analysis revealed operational insights that would have been difficult to uncover through logs alone:

  • 111,062 files extracted
  • 5,839 files submitted for advanced analysis
  • 3,589 plaintext password sightings
  • 83 unique accounts transmitting credentials in clear text
Statistics from Splunk .conf26
Statistics from Splunk .conf26

These are not simply interesting statistics. They illustrate why packet capture remains unique. Every file transfer. Every protocol exchange. Every credential exposure. Every network session. Every byte of communication. All available for validation long after the original event occurred.

Packet Capture Is No Longer a Forensics Tool

Historically, packet capture occupied a specialized role. A major security incident occurred. An analyst opened packet data. Investigators reconstructed what happened. A report was written. The incident was closed. That model no longer reflects how modern SOCs operate. The Agentic SOC requires evidence continuously, not occasionally. Investigations happen in real time. AI agents gather context automatically. Analysts expect immediate answers. Executive stakeholders demand traceable decisions. Packet capture has therefore evolved from a forensic platform into an operational system. Instead of helping explain yesterday’s incident, packet data actively informs today’s decision-making. That shift represents a fundamental change in the value proposition of full packet capture.

The Network Still Tells the Truth

One challenge every security team faces is fragmented visibility. Endpoint data may be unavailable. Cloud telemetry may be incomplete. Third-party systems may lack instrumentation. Logs may be sampled. Alerts may be suppressed. Users may operate devices outside administrative control. The network remains the common denominator. Every system communicates. Every application transmits data. Every attack generates traffic. Every compromise leaves traces. Packet capture therefore provides visibility that does not depend upon deployment models, operating systems, agents, or administrative ownership.

For environments with large populations of unmanaged devices, such as conferences, campuses, hospitals, universities, manufacturing facilities, and guest networks, packet capture is especially valuable because it remains available even when other telemetry sources – such as end point telemetry – are not.

What AI Learns from Packets

Discussions around the use of AI in cybersecurity typically focus on its ability to increase the speed of response. Speed is undeniably important, but the greater value is context.

Full Packet data provides context that is unavailable from any other source of data:
• Complete session reconstruction
• Application behavior
• File movement
• Protocol misuse
• Credential exposure
• The detail of command-and-control communications
• User activity patterns
• Traffic timelines

When these elements are combined with threat intelligence, security analytics, and behavioral detections, AI systems gain access to evidence rather than indicators alone. That changes the quality of investigation outcomes. Instead of reporting: “a suspicious event occurred” the system can determine: “this event occurred, these systems communicated, these files were transferred, this protocol was used, and this is the evidence that supports the conclusion.” The result is not simply faster investigations. The result is more trustworthy investigations.

Building Explainable Security Operations

One of the greatest risks facing security organizations is the emergence of black-box decision making. The more autonomous systems become, the more important explainability becomes.

Security leaders need to know:

  • Why a finding was escalated
  • Why an event was closed
  • Why an investigation was prioritized
  • Why a response action was recommended

Answers require evidence. Evidence requires records. Records require preservation.

That is the role packet capture fulfills. Endace provides the evidentiary foundation that connects AI reasoning, analyst decision-making, and organizational accountability. It allows every conclusion to be traced back to observable network activity.

The Future of the Agentic SOC

Over the next several years, most security operations centers will adopt some level of agentic capability. AI-driven triage will become standard. Autonomous investigations will become common. Machine-generated recommendations will become expected. What will differentiate successful organizations from unsuccessful ones will not be solely the intelligence of the agents. It will be the quality of the evidence available to those agents. The organizations that achieve the greatest value from AI will be those that provide their AI tools with the richest, most complete, and most trustworthy data foundation. That foundation increasingly requires full packet capture.

At Splunk .conf26, Endace demonstrated that packet capture is no longer just about retrospective forensics. It is about enabling trustworthy AI, explainable investigations, and evidence-based security operations. As the industry moves toward the Agentic SOC, one conclusion becomes clear:

AI may become the analyst’s co-pilot, but packet capture remains the system of record.

Acknowledgements

Our thanks go to the Splunk .conf26 SOC team, led by Jessica Oppenheimer and Paul Pelletier, for the opportunity to integrate EndaceProbes with the Splunk .conf26 Agentic SOC architecture. 

The SOC team is a collection of Cisco, Splunk and Endace experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco and Splunk tools.

The teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read all about the Agentic Security Operations Center at Splunk .conf26, including an overview from Jessica Oppenheimer, and posts and use cases from Splunk and Cisco SOC team members:

Read about the Endace team’s experience working in the SOC at Splunk .conf26:

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Cisco Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC-in-a-Box setup, and download the detailed architecture blueprint and report by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Episode #67 Erik Dove from Cisco talks Agentic SOC and Security Operations

Original Entry by : Michael Morris

In the Packet Forensic Files, Episode 67, Michael talks to Cisco’s Erik Dove.

By Michael Morris, Senior Director of Global Business Development, Endace


How Agentic AI is changing Security Operations.

Michael Morris, Director of Global Business Development, Endace

Artificial intelligence is rapidly reshaping Security Operations Centers (SOCs), helping security teams investigate incidents faster while reducing alert fatigue. But as AI becomes more capable, what role do human analysts and packet data play in an increasingly automated SOC?

In Packet Forensics Files Episode 67, I sit down with Cisco Security Sales Engineer and Incident Response expert Erik Dove to discuss how Agentic AI is changing incident response, where packet data fits into modern SOC workflows, and why experienced analysts remain essential.

Erik’s first point was that AI’s biggest benefit isn’t simply speed. It’s improving the accuracy of incident triage. By helping analysts identify which alerts genuinely deserve attention, AI reduces time spent on false positives and lets teams focus on real threats.

He also explained that effective AI starts with strong foundations. Organizations need clear priorities and service level agreements so AI can accurately identify the alerts that matter most.

We explored where always-on packet capture fits into an AI-assisted SOC. While SIEMs, XDR platforms and firewalls all provide valuable telemetry, packet data gives investigators the evidence to validate alerts, reconstruct communications and build accurate timelines. Rather than relying solely on logs, analysts can quickly determine whether activity is truly malicious and understand the “why” behind an alert.

One of my favourite parts of the discussion was Erik’s perspective on the future role of analysts. Rather than replacing people, he sees AI acting as a team of specialists that can summarize incidents, assist investigations, support orchestration and document findings. Human judgement remains essential because cybersecurity investigations still require context, experience and decision-making that AI cannot provide on its own.

To wrap up, I asked Erik what a fully optimized Agentic AI SOC might look like. He compared modern investigations to Law & Order. The tools continue to improve and the cases become more complex, but investigators still rely on evidence. In cybersecurity, those digital fingerprints are preserved in packet data, giving analysts the visibility they need to understand exactly what happened.

If you’re interested in how AI, automation and packet capture are shaping the future of security operations, I encourage you to watch the full episode. It’s a fascinating discussion with plenty of practical advice for organizations looking to build more effective SOCs.

Other episodes in the Secure Networks video/audio podcast series are available here. Or listen to the podcast here, or on your favorite podcast platform.


Cisco Live AMER 2026: Never underestimate Cisco Live!

Original Entry by : Stephen Donnelly

By Dr Stephen Donnelly, Chief Technical Officer, Endace


Dr Stephen Donnelly, CTO, Endace

Cisco Live AMER is a huge event, attracting more than 20,000 attendees to see the latest products, technology, and innovations from Cisco and its partners. In 2026, Cisco Live AMER had a dedicated SOC for the second time, out on the main show floor for everyone to see. The SOC mission is threefold:

  • To Protect: The primary goal of the SOC is to protect attendees, partners, and staff using the Cisco Live network from threats from outside or within the network
  • To Educate: The SOC educates attendees about modern Agentic SOC operation, current threats, and best practices. Over a dozen new analysts were trained in the SOC, empowered by agentic workflows.
  • To Innovate: The live environment is an opportunity to pressure test how Cisco Security, Splunk Security, and Endace can work together in a next-generation SOC where AI assists, evidence grounds the decision, and humans remain in control

Endace (a Cisco Solutions Plus Partner) was invited to collaborate in the SOC again this year.

We have a great relationship with the SOC team and have assisted at multiple events by providing our always-on full packet capture capability. Capturing all the network traffic at the event not only provides incontrovertible evidence to confirm incident investigations but allows SOC analysts to ‘zoom out’ from a single alert to see the full context of the event.

Visibility of the network activity before and after an incident helps determine the attack chain, and any indications of compromise, such as malware downloads, C2 beaconing, reconnaissance, or lateral movement. A complete record of traffic for the event also enables systematic threat hunting, as well as providing the ability to help train AI on real-world traffic for the Agentic SOC.

The SOC dashboards on show outside the Cisco Live USA 2026 SOC
The SOC dashboards on show outside the Cisco Live AMER 2026 SOC

Endace’s Always-On Packet Capture

To capture all the traffic at the event, Endace supplied two EndaceProbe appliances for use in Cisco’s ‘SOC-in-a-Box’. Each capture appliance features 100 Gigabit Ethernet capture interfaces and NVMe storage, capable of sustained capture at 100Gbps. Each appliance also supports multiple virtual machines which have access to the live captured data in real-time and can host analytics tools. An Endace InvestigationManager VM provides federated search across both appliances, with integrations for Cisco XDR and Splunk, and an MCP server for AI agents.

As incidents are raised in XDR or Splunk Enterprise Security, they are automatically enriched with links to the Endace InvestigationManager. These links allow AI agents and SOC Analysts to easily access full PCAP captures for the related traffic, or pivot to wider searches as needed.

Cisco's SOC-in-a-Box, used at a wide range of events globally.
Cisco’s SOC-in-a-Box, used at a wide range of events globally, is installed in the Cisco Live NOC.

How much is enough?

At 2025 Cisco Live AMER in San Diego, the EndaceProbes captured a total of 78.9TB of network data during the whole event: a record volume of traffic at the time. We expected 2026 to be even bigger, so we configured 160TB of packet storage across the two appliances.

Full packet capture started as soon as the SOC was stood up on the Friday before the show, capturing the network traffic as the show floor and booths were constructed. By the end of the second full day of Cisco Live we had already captured 100TB, and it was looking like our 160TB allocation might be insufficient. This would have meant overwriting traffic captured at the start of the event, so we reconfigured the storage to increase the total packet storage to 192TB. Metadata and search indexes are on top of this figure, allowing retrieval of any captured flow in seconds.

By the end of the show, the EndaceProbe appliances had captured more than 200 billion packets, consuming 199TB of storage, without dropping any packets from the SPAN or overwriting any data since the start of the event. How was that possible with only 192TB of storage configured?

The EndaceProbe appliances support hardware compression of packet data prior to storage. Of the 199TB of traffic collected, 37% was unencrypted, which compresses well. The overall compression ratio for the packet data was 1.25:1, so our 199TB of traffic consumed only about 159TB of storage space. Our original 160TB configuration was pretty close after all!

A dashboard at Cisco Live USA 2026 shows Endace statistics for the show's duraction.
A dashboard at Cisco Live AMER 2026 shows Endace statistics for the show’s duraction.

Why was so much traffic unencrypted, isn’t it a security risk? In some cases yes, the SOC detected 685 unique accounts exposing login credentials in the clear, mostly because of misconfigured email clients using SMTP, IMAP, and POP3 without encryption. The SOC even automated a playbook to contact affected users and warn them of their exposure, inviting them to the SOC for a demonstration and help with remediation.

A surprising amount of traffic still uses unencrypted http. This is commonly used for downloading unencrypted (but signed) software update packages such as Microsoft .cab and Android .apk files, plus EDR rule updates etc. The contents of these files are freely available and frequently cached so transport encryption is not necessary, provided package signatures are verified to confirm authenticity.

The Cisco Live SPAN was configured to capture not only the attendee WiFi network, but also a large amount of infrastructure traffic including CiscoTV, one of the largest users of the network. Some of that traffic was also unencrypted, such as syslog data from IDS appliances. This also contributed to the relatively high percentage of unencrypted traffic seen.

Packet Loss

If the EndaceProbe appliances captured all the traffic from the SPAN without drops and did not overwrite any data, did we get everything? Looking at the bandwidth arriving on the 10GE SPAN port the traffic was reaching 10Gbps early in the day, suggesting the SPAN may be saturated and dropping. Without access to the SPAN port statistics this can be difficult to quantify, but one approach is to look at the sequence numbers seen in TCP flows.

Analyzing traffic flows from Cisco Live USA 2026 in InvestigationManager
Analyzing traffic flows from Cisco Live AMER 2026 in InvestigationManager

When a TCP flow encounters packet loss in the network, a retransmission will occur to fill in the missing data. When this flow is captured by analysis tools they may or may not see the initial lost packet, but they can see the retransmission and infer the loss. Conversely, if the analysis tools see many TCP flows with missing segments but no retransmissions, this can indicate multi-path routing (not applicable at Cisco Live), or packet loss at the SPAN port.

Examining these indicators showed estimated packet loss exceeding 30% during peak times, or around 13-14Gbps of traffic. This could be easily solved in future by upgrading the SPAN port to 25G or 40G Ethernet, a definite learning opportunity.

File Reconstruction

The EndaceProbe appliances also extracted files from unencrypted connections. Over Cisco Live Americas 2026 a total of 2,392,180 files were extracted. These files were filtered by type and deduplicated, with 23,368 submitted to Splunk Attack Analyzer and 5,427 to Cisco Secure Malware Analytics for deeper inspection.

Examining the rate of file extraction over time, we noticed it did not follow the overall bandwidth curve. This is counterintuitive, as more files should be transferred and extracted as the network bandwidth increases during the day.

This anomaly may be caused by the SPAN being overloaded and dropping packets before they reached the EndaceProbes. Files cannot be successfully reconstructed and extracted when packets are lost, and a packet loss rate exceeding 10% implies that most file transfers would have at least one packet missing. With a higher bandwidth SPAN to the SOC we are confident the file extraction counts would have been significantly higher.

Conclusion

Never underestimate Cisco Live! We will be back next year with a faster SPAN and more storage to support the SOC in 2027 (and at Cisco Live APCJ in Melbourne this November), helping protect, educate, and innovate. Hope to see you there!

Acknowledgements

Our thanks go to the Cisco SOC team, led by Jessica Oppenheimer and Ivan Berlinson, for the opportunity to integrate EndaceProbes with the Cisco Live Agentic SOC architecture. 

The SOC team is a collection of Cisco and Splunk experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco Security and Splunk tools.

The Endace and Cisco teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read more from Endace about the Cisco Live USA 2026 SOC:

Read related Cisco Team Blogs from the Cisco Live USA 2026 SOC: 
https://blogs.cisco.com/security/clamer-soc-2026

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC setup, and download the whitepaper written by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Cisco Live AMER 2026: Using LLMs and Endace Full Packet Capture for Incident Response

Original Entry by : Barry "Baz" Shaw

By Barry “Baz” Shaw, Senior Engineering Manager, Technology Partner Programs, Endace


Barry

At Cisco Live AMER 2026 Endace debuted an MCP (Model Context Protocol) server for the integration of EndaceVault into the emergent agentic Security Operations Center (SOC), providing access to 200TB of full packet data and on-demand packet analysis to all agents present in the SOC.

Going into Cisco Live we had key questions about the utility of PCAP data to agents, how to manage making terabytes of data available without overflowing their context windows, and whether the models could parse, analyze and make judgements from the PCAP data comparable to that of an analyst. Results from some simple tests are extremely encouraging and indicate a bright future for agentic AI in the SOC.

Endace Always-on Packet Capture

Two of Endace’s newest EP94C8 EndaceProbes were installed in the GovWare SOC to provide full packet capture to support the conference SOC directives of Protect, Educate, and Innovate.  Full packet capture provides unique insight into all activities on the network, delivering critical context and evidence for Incident Response and Threat Hunting teams.

The EndaceProbes each hosted three VMs that were running Zeek and delivering critical log data into Splunk.  Custom Zeek script additions provided additional valuable details around clear text passwords in email and HTTP, and the use of insecure protocols.  Zeek was also used for file-carving and automated submission of objects to Splunk Attack Analyzer and a beta of Endace’s new Vault API used in a Cisco XDR automated workflow was field tested.   

PCAP and AI – a key question answered

In this scenario we wanted to test the ability of a reasonably new on-prem model to make incident judgements based on PCAP data.  Using the qwen-3.6 model and Open WebUI, a DNS buffer overflow attempt flagged by Cisco Secure Firewall, and raised as an incident in Cisco XDR, was chosen as the candidate.

Cisco Live USA 2026: Possible Buffer Overflow Attempt Flagged
Cisco Live AMER 2026: Possible Buffer Overflow Attempt Flagged

Using the Endace XDR workflow integration (which automatically creates Endace Pivot-to-Vision and EndaceVault links directly in the worklog) a quick pivot from Cisco XDR to EndaceVision shows DNS traffic present:

Investigating the suspect DNS traffic in EndaceVision
Investigating the suspect DNS traffic in EndaceVision

After pivoting from EndaceVision into Wireshark, we can see DNS activity and some large query responses.  This verification served as the analyst judgement to be compared with the LLM findings.

Examining the actual packets in Wireshark lets us validate the AI agent's analysis.
Examining the actual packets in Wireshark lets us validate the AI agent’s analysis.

In Open WebUI a new chat was opened, the Endace MCP connected as a tool, and the following prompt was issued:

There was a firewall alert called ‘PROTOCOL-DNS Microsoft SMTP excessive answer records buffer overflow attempt’ recorded for the traffic between IP addresses <redacted> and <redacted> at 2026-06-03T15:24:01.000Z and 2026-06-03T15:22:33.000Z. Can you get a packet decode from Endace and either confirm or deny the existence of DNS responses that would match this alert.

The model’s reasoning is shown in the following screenshot:

The AI agent's reasoning before starting the analysis.
The AI agent’s reasoning before starting the analysis.

The Endace MCP tools contain detailed annotations that provide context to the model. Above we can see this context helping the model choose between different time inputs and correctly identifying that the start and stop times provided in the prompt were (accidentally) transposed in the prompt.

The AI agent prepares to retrieve and analyze the packet data.
The AI agent prepares to retrieve and analyze the packet data.

Once the AI issued the tool call to the Endace Vault API via the MCP, the extraction of the related PCAP data began. To make this data more consumable by AI, it is converted to JSON by the Endace Vault API and streamed back in manageable sized chunks. The AI can request additional chunks as required, which protects the context window from being swamped by a large PCAP download.

An example of the returned JSONified PCAP data is shown below, this is the raw data that the AI has to make its determination.

A sample of the packet data returned in json format
A sample of the packet data returned in json format

After consuming the JSONified PCAP data the AI began its analysis, clearly having parsed the PCAP data and displaying an understanding of the DNS protocol. From the name of the alert, it understood the specific details pertaining to the alert and was able to look for these in the DNS packets:

The AI agent shows its reasoning and the resulting analysis.
The AI agent shows its reasoning and the resulting analysis – Part 1.
The AI agent completes its analysis and outlines its conclusion.
The AI agent shows its reasoning and the resulting analysis – Part 2.

The reasoning of the model makes for interesting reading. It can comprehend the nature and specifics of the alert and use this to reason about the DNS packets.  It successfully found an oversized DNS response and provided unprompted enrichment by determining the hosting provider of the DNS server.

The AI Agent completes its analysis and outlines its conclusion.
The AI Agent completes its analysis and outlines its conclusion.

So far, the model has performed admirably, validating the firewall alert and the analyst judgement.  To take things further, we asked the model to do some extra analysis over and above the human analysis we’d done. While this chat was user driven, we expect that this is the kind of autonomous activity AI agents will perform on their own.

The AI was asked to confirm the hosting provider and look at the contents of the large DNS response to determine if it contained a malicious payload or was in fact just a verbose and unusually large response.

Can you confirm that the destination host <redacted> is in fact Microsoft hosted?  Is there anything in the large DNS response that could be malicious or is it just a DNS response?

The AI Agent elaborates on its initial conclusion when asked for additional analysis.
The AI Agent elaborates on its initial conclusion when asked for additional analysis.

The model has told us that the source of the DNS response is a reputable domain and that the payload of the response was just a large number of regular AAAA records.  In a pre-AI SOC, the analyst would need to spend time inspecting the PCAP in Wireshark (if they had access to full packet capture in the first place) to determine this was a false positive.

Conclusion

While this example was intentionally not an autonomous agentic finding, the model was able to determine that this alert was a false positive by analyzing the PCAP.  The fact that relatively small open-source models can successfully extract findings from PCAP data is a major validation of their utility in the agentic SOC. We expect that frontier models will provide even richer analysis of PCAP data when available in the SOC if the token cost can be justified. A possible architecture is an on-prem first approach, escalating to a frontier model in the event on-prem cannot make a confident determination.

By using an agent to investigate events like these, and having access to full PCAP data, alerts can be triaged by the agent, requiring only a cursory review by a human analyst to definitively close cases in record time. This validates the core concept that the agentic SOC will improve overall efficiency and accuracy by optimizing incident triage and freeing up time for analysts to work on the more complex incidents that threaten enterprise operations.

Acknowledgements

Our thanks go to the Cisco SOC team, led by Jessica Oppenheimer and Ivan Berlinson, for the opportunity to integrate EndaceProbes with the Cisco Live Agentic SOC architecture. 

The SOC team is a collection of Cisco and Splunk experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco Security and Splunk tools.

The Endace and Cisco teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read more from Endace about the Cisco Live USA 2026 SOC:

Read related Cisco Team Blogs from the Cisco Live USA 2026 SOC: 
https://blogs.cisco.com/security/clamer-soc-2026

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC setup, and download the whitepaper written by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Cisco Live USA 2026: Supporting Encryption vs. Using Encryption – when the best laid plans go astray

Original Entry by : Elliott Hinson

By Elliott Hinson, Integrations Engineer, Endace


Elliot Hinsen, Integrations Engineer, Endace

Are you actually using encryption, or does your software simply support it?

The answer may seem silly, but there lies a hidden amount of complexity behind this question that many users never have the need or opportunity to approach.

At the Cisco Live USA 2026 Security Operations Center (SOC) we found a great number of instances of users using technology that supported encryption. However, through some form of misconfiguration or a sheer lack of knowledge, they had configured their system to not use the encryption that their software did, in fact, support.

Sony DSLR FTP uploads

The first incident that I would like to discuss regards a user operating a Sony DSLR camera with automated photo upload enabled. Most DSLR cameras these days offer a Wi-Fi or Bluetooth connectivity option to offload photos for later processing.

While sifting through log data in Splunk, we discovered one user at the conference had configured automated image uploading to an FTP server in the cloud directly from their Sony DSLR camera.

Because this was configured to an FTP cloud server, and because we had the full packet data recorded via the EndaceProbe, we were able to inspect the username and password for the login credentials of this FTP server, which were being broadcasted in the clear over the network. Furthermore, thanks to Endace’s file carving ability, we were even able to reconstruct the original file uploads that the user had sent to the cloud over the Wi-Fi network, also unencrypted.

This represented one of the lower risk level incidents of supporting encryption without using encryption because the photos were intended to be used at the Cisco Live Conference, and they were not particularly private. One could imagine an incident, however, where much more private photos had been taken on a similar camera system, and those photos could be gathered by any nefarious actor with access to the Wi-Fi network.

This problem is particularly unfortunate, since a quick perusal of the Sony documentation for that particular model of camera shows that they very clearly do support encryption in the form of SFTP or FTPS. However, the user had not configured these and was unaware that they had been uploading their photos and password in the clear all week.

Recorded data upload dates and times shown in EndaceVision.
Recorded data upload dates and times shown in EndaceVision.
Extracted images were reassembled from the packet data using Zeek.
Extracted images were reassembled from the packet data using Zeek.

POP3 email, using the wrong port

This flavor of incident resulted in several users being invited into the SOC for us to inform them of the dangers of using unencrypted email. The POP3 protocol, as well as the SMTP protocol, do support encryption. However, once again we found users were connecting to their email servers with mail clients that had not been configured to enable encryption. It had merely asked them for a port number, and the user, not knowing which port number to use or why that may be important, had simply selected the default.

This was quite convenient for us, since their username for their accounts was their email address (and their email login password was also exposed). So, all we had to do was use the email address that we had seen passing, unencrypted, over the network to send them an email inviting them to the SOC to improve their security posture. For one of the users, it was as simple as changing the mail client to use the POP3 port that supported encryption.

This represented a much higher risk level of incident, since not only was the username and password for their email accounts in the clear, but we could also reconstruct emails for several users with this problem. That allowed us to uncover an additional problem. Some of these users were using a network monitoring solution that also did have encryption enabled and was sending email reports containing sensitive information about the state of their laptop computers in the clear. The issue was fixed by digging through a nested menu to check a box that enabled encryption.

What was interesting was that this network monitoring software was attempting to send unencrypted email reports to service providers that were actually returning error codes indicating that they did not support receiving unencrypted email. However, while the service providers were attempting to stop this unencrypted email system from being received, this did not preventing the user’s client software from sending their username and password in the clear for anyone to see.

Almost encrypted office-suite

This next incident was particularly tragic because of how much effort the user had clearly put into attempting to ensure their security by supporting encryption at multiple layers of their software stack.

Once again sifting through Splunk logs and corroborating those Splunk logs with the original packet data – available to us from the EndaceProbe in the Cisco Live SOC – we discovered a user who had configured a self-hosted Office 365 alternative on their Android smartphone so they could synchronize and manage contacts, images, and important documents using a self-hosted office suite server.

Initially, we thought this was a case of some user setting up an office suite on a home network and port forwarding that instance to the open internet using their home router. This obviously is not a recommended way to expose a private service to a remote user, but is still very commonplace due to users not fully understanding the risks of such a setup.

In this case, however, upon further inspection we discovered this office suite server was running behind a Cloudflare TLS termination gateway. We also saw that encryption for the WebUI had been configured correctly so attempts to access a non-encrypted version of the login page would automatically redirect to the encrypted page.

The one weak link in the system was that this particular Android client had been configured to use the unencrypted webDAV endpoint and was not following redirects because it was not a full web browser.  It was at this point that we realized, tragically, that we were observing a user who:

  • had properly put their self-hosted office suite system behind a Cloudflare DDOS protection gateway
  • had properly configured TLS for the service
  • had properly HTTPS redirection in place.

However, since they did not have port 80 configured to only allow HTTPS redirection and not allow any other access to the services via port 80, and because the android client was defaulting to using port 80 but not being redirected, the user was leaking sensitive credentials and business information for anyone to see.

Authentication data transmitted in the clear because of a simple, overlooked, mis-configuration.
Authentication data transmitted in the clear because of a simple, overlooked, mis-configuration.

Conclusion

To wrap all this up, and provide a moral to this collection of stories: just because your software stack supports encryption does not guarantee that it is necessarily using that encryption. You have to make sure that all software is configured correctly to be using the encryption that it will most likely already support. Furthermore, most firewalls and security systems won’t flag unencrypted traffic as it is not necessarily malicious nor is it inherently risky to the network operator.

The only way we discovered this was thanks to the use of Splunk logging and its integration with Endace’s always-on packet capture system. This allowed us to reconstruct the full packets for these incidents – which a smart PCAP system or a firewall triggered PCAP system would never have seen.

Acknowledgements

Our thanks go to the Cisco SOC team, led by Jessica Oppenheimer and Ivan Berlinson, for the opportunity to integrate EndaceProbes with the Cisco Live Agentic SOC architecture. 

The SOC team is a collection of Cisco and Splunk experts across many domains who were a pleasure to work and innovate with, and we came away with a great appreciation for the power of the Cisco Security and Splunk tools.

The Endace and Cisco teams were able to prove out integration innovations and test them in earnest in a real-world environment in preparation for making them generally available to the market.

More from Endace, Splunk and Cisco in the SOC series

Read more from Endace about the Cisco Live USA 2026 SOC:

Read related Cisco Team Blogs from the Cisco Live USA 2026 SOC: 
https://blogs.cisco.com/security/clamer-soc-2026

For more Endace blogs in our SOC series, see here:
https://blog.endace.com/tag/soc/ 

Event SOC Website 
Visit Cisco’s Event SOC website for full details of the SOC setup, and download the whitepaper written by Jessica Oppenheimer:
https://www.cisco.com/site/us/en/products/security/event-soc-report.html


Episode #66 Cody Spooner from Corelight talks Incident Response and Threat Hunting

Original Entry by : Michael Morris

In the Packet Forensic Files, Episode 66, Michael talks to Corelight’s Cody Spooner.

By Michael Morris, Senior Director of Global Business Development, Endace


Michael Morris, Director of Global Business Development, Endace

The Increasing Complexity of Incident Response
and Threat Hunting

In this episode of the Endace Packet Forensic Files, I sat down with Cody Spooner, Principal Sales Engineer and DFIR expert at Corelight, to discuss a really interesting topic: the subtleties and differences of “Enablers” vs “Behaviors” of a cybersecurity compromise.

Cody explains that when most people think of threat hunting or incident response investigations, they picture analysts looking for signs of malicious activity. In reality there are critical subtle differences between the “behavior of a compromise” and the underlying “enabler of a compromise” that often go unnoticed or overlooked. He highlights how organizations tend to focus heavily on detecting malicious behaviors – such as data exfiltration or unauthorized logins – but often miss identifying the enabling conditions – such as misconfigurations or legacy protocols – that led to those compromises in the first place.

Cody shares examples of seemingly harmless issues that can become the doorway to a full compromise, such as configuration issues or outdated or deprecated protocols like NTLMv1 and SMBv1. These often persist in modern environments and Cody suggests that incident responders and threat hunters can usefully focus on identifying and eliminating these enablers to reduce the organisation’s risk profile.

Cody gives advice for security teams on how to shift their mindset from focusing only on behaviors to focusing on enablers as well in their threat hunting activity. He also provides insights into how IR teams should interpret and contextualize indicators of compromise and discusses how the “why” behind an attack can often change or influence the response strategy.

Finally, Cody shares his thoughts on what emerging technologies or architectural trends will create new classes of enablers that defenders need to start paying attention to now.

Other episodes in the Secure Networks video/audio podcast series are available here. Or listen to the podcast here or on your favorite podcast platform.