The Importance of Network Data to Threat Hunting (Part 3)

Original Entry by : Robert Salier

Frameworks and Regulations

By Robert Salier, Product Manager, Endace


Robert Salier, Product Manager, EndaceIn this, the third article in our series on threat hunting (see here for Part 1 and Part 2), we explore the frameworks and regulations most relevant to threat hunting.

These tend to fall into two categories: those that address cybersecurity at a governance level, and those that facilitate insight into individual attacks and help formulate appropriate defense actions.

Governance Level Frameworks and Regulations

The regulatory environment influences threat hunting, and cyber defense in general. In many countries, regulations impose obligations on disclosure of breaches, including what information must be provided, when, and to which stakeholders. This influences the information that an organization needs to know about a breach, and hence its choice of strategies, policies, processes and tools. These regulations generally require companies to disclose a breach to all customers that have been affected. However if an organization cannot ascertain which customers were affected, or even if any customers were affected, then they may need to contact every customer. The only thing worse than having to disclose a breach is having to disclose a breach without being able to provide the details your customers expect you to know.

There are a also a number of frameworks addressing cybersecurity at the governance level, which in some cases overlap with regulations, dealing with many of the same issues and considerations. Collectively, these frameworks and regulations help to ensure organizations implement good strategies, policies, processes and tools, e.g. …

  • Which systems and data is most important to the organization
  • What Information security policies should be in place
  • How cybersecurity should be operationalized (e.g. what organizational structure, security architecture and systems are most appropriate for the organization)
  • Incident management processes
  • Best practice guidelines

Prevalent frameworks and regulations include…

  • ISO 27000 Series of Information Security Standards
    A comprehensive family of standards for information security management, providing a set of best practices for information security management. Maintained by the International Standards Organization, it has been broadly adopted around the globe.
  • NIST Special Publication 800-53
    A catalogue of security and privacy controls for all U.S. federal organizations except those related to national security.
  • NIST Cybersecurity Framework
    A policy framework for private sector organizations to assess and improve their ability to prevent, detect, and respond to cyber attacks. It was developed for the USA, but has been adopted in a number of countries.
Frameworks to Characterize Attacks and Facilitate Responses

A number of frameworks have been developed to help describe and characterize attacker activity, and ultimately facilitate defense strategies and tactics.

Prevalent frameworks include…

  • Cyber Kill Chain
    Developed by Lockheed Martin, this framework was developed from a “kill chain” framework developed for military attack and defense. It decomposes a cyber attack into seven generic stages, providing a framework for characterising and responding to attacks. Refer to this Dark Reading article for some discussion on the benefits and limitations of this framework.
  • Diamond Model
    This model describes attacks decomposing an attack into four key aspects, i.e. details of the adversary, their capabilities, the infrastructure they used, and the victim(s). Multiple attack diamonds can be plotted graphically in various ways including timelines and groupings, facilitating deeper insight.
  • Mitre Att&ck
    Developed by Mitre, Att&ck stands for “Adversarial Tactics, Techniques, and Common Knowledge”. It is essentially a living, growing knowledge base capturing intelligence gained from millions of attacks on enterprise networks. It consists of a framework that decomposes a cyber attack into eleven different phases, a list of techniques used in each phase by adversaries, documented real-world use of each technique, and a list of known threat actor groups. Att&ck is becoming increasingly popular, used by and contributed to by many security vendors and consultants.
  • OODA Loop
    Describes a process cycle of “Observe – Orient – Decide – Act”. Originally developed for military combat operations, it is now being applied to commercial operations.

The Importance of Network Data to Threat Hunting (Part 2)

Original Entry by : Robert Salier

Threat Hunting in Practice

By Robert Salier, Product Manager, Endace


Robert Salier, Product Manager, EndaceHunting for security threats involves looking for traces of attackers in an organization’s IT environment, both past and present. It involves creativity combined with (relatively loose) methodologies and frameworks, focused on outsmarting an attacker.

Threat Hunting relies on a deep knowledge of the Tactics, Techniques and Procedures (TTP’s) that adversaries use, and a thorough knowledge of the organization’s IT environment. Well executed threat hunts provide organizations with deeper insight into their IT environment and into where attackers might hide.

This, the second article in our series of blog posts on threat hunting (read Part 1 here), looks at how leading organizations approach threat hunting, and the various data, resources, systems, and processes required to threat hunt effectively and efficiently.

Larger organizations tend to have higher public profiles, more valuable information assets, and complex and distributed environments that present a greater number of opportunities for criminals to infiltrate, hide, and perform reconnaissance without detection. When it comes to seeking out best practice, it’s not surprising that large organizations are the place to look.

Large organizations recognize that criminals are constantly looking for ways to break in undetected and that it is only a matter of time before they succeed if they haven’t already. While organizations of all sizes are being attacked, larger organizations are the leaders in this proactive approach to hunting down intruders, i.e. “threat hunting”. They have recognized that active threat hunting increases detection rates over-relying on incident detection alone – i.e. waiting for alerts from automated intrusion detection systems that may never come.

Best practice involves formulating a hypothesis about what may be occurring, then seeking to confirm it. There are three general categories of hypothesis:

  • Driven by threat intelligence from industry news, reports, and feeds.
    e.g. newsfeeds report a dramatic increase in occurences of a specific ransomware variant targeting your industry. So a threat hunt is initiated with the hypothesis that your organization is being targeted with this ransomware
  • Driven by situational awareness, i.e. focus on infrastructure, assets and data most important to the organization.
    e.g. a hypothesis that your customers’ records are the “crown jewels”, so hackers will be trying to gain access to exfiltrate this data

Having developed a hypothesis as a starting point, leading organizations rely on a range of tools and resources to threat hunt efficiently and effectively:

Historic Data from Hardware, Software and the Network
  • Infrastructure Logs from the individual components of hardware and software that form your IT environment, e.g. firewalls, IDS, switches, routers, databases, and endpoints. These logs capture notable events, alarms and other useful information, which when pieced together can provide valuable insight into historic activity in your environment. They’re like study notes that you take from a text book, i.e. highly useful, but not a full record, just a summary of what is considered notable. Also, be wary that hackers often delete or modify logs to remove evidence of their malicious activity.
  • Summarized network data (a.k.a. “packet metadata”, “network telemetry”). Traffic on network links can be captured and analysed in real time to generate a feed of summary information characterizing the network activity. The information that can be obtained goes well beyond the flow summaries that Netflow provides, e.g. by identifying and summarizing activity and anomalies up to and including layer 7 such as email header information and expired certificates. This metadata can be very useful in hunts and investigations, particularly to correlate network traffic with events and activity from infrastructure logs, and users. Also, unlike logs, packet metadata cannot be easily deleted or modified.
  • Packet level network history. By capturing and storing packets from a network link, you have a verbatim copy of the communication over that link, allowing you to see precisely what was sent and received, with zero loss of fidelity. Some equipment such as firewalls and IDS’s capture small samples of packets, but these capture just a fraction of a second of communications, and therefore must be automatically triggered by a specific alarm or event. Capturing and storing all packets (“full packet capture”, “100% packet capture”) is the only way to obtain a complete history of all communications. Historically, the barriers to full packet capture have been the cost of the required storage and the challenge of locating the packets of interest, given the sheer volume of data. However, recent advances in technology are now breaking down those barriers.
Baselines

Baselines are an understanding of what is normal and what is anomalous.
Threat hunting involves examining user, endpoint, and network activity, searching for IoA’s and IoC’s – i.e. “clues” pointing to possible intrusions and malicious activity. The challenge is knowing which activity is normal, and which is anomalous. Without knowing that, in many cases, you will not know whether certain activity is to be expected in your environment, or whether it should be investigated.

A Centralized Location for Logs and Metadata

Because there are so many disparate sources of logs, centralized collection and storage is a practical necessity for organizations with substantial IT infrastructure. Most organizations use a SIEM (Security Information and Event Manager), which may have a dedicated database for storage of logs and metadata, or may use an enterprise data lake. SIEMs can correlate data from multiple sources, support rule-based triggers, and can feature Machine Learning algorithms able to learn what activity is normal (i.e. “baselining”). Having learned what is normal, they can then identify and flag anomalous activity.

Threat Intelligence

Threat intelligence is knowledge that helps organizations protect themselves against cyber attacks. It encompasses both business level and technical level detail. At a business level this includes general trends in malicious activity, individual breaches that have occurred, and how organizations are succeeding and failing to protect themselves. At a technical level, threat intelligence provides very detailed information on how individual threats work, informing organizations how to detect, block, and remove these threats. Generally this comes in the form of articles intended for consumption by humans, but also encompasses machine-readable intelligence that can be directly ingested by automated systems, e.g. updates to threat detection rules.

Frameworks and Regulations

The regulatory environment influences threat hunting, and cyber defense in general. In many countries, regulations impose obligations on disclosure of breaches, including what information must be provided, when, and to which stakeholders. There are a also a number of frameworks addressing cyber security at the governance level, which in some cases overlap with regulations, dealing with many of the same issues and considerations. Collectively, these frameworks and regulations help to ensure organizations implement good strategies, policies, processes and tools.

In the next article in this series, we explore the frameworks and regulations that apply to threat hunting, and which ensure organizations implement appropriate strategies, policies, processes and tools.


The Importance of Network Data to Threat Hunting (Part 1)

Original Entry by : Robert Salier

Introduction to Threat Hunting

By Robert Salier, Product Manager, Endace


Robert Salier, Product Manager, EndaceCriminal hackers are stealthy. They put huge efforts into infiltrating without triggering intrusion detection systems or leaving traces in logs and metadata … and often succeed. So you need to actively go searching for them. That’s why SecOps teams are increasingly embracing threat hunting.

This is the first in a series of blog articles where we discuss various aspects of threat hunting, and how visibility into network traffic can increase the efficiency and effectiveness of threat hunting. This visibility is often the difference between detecting an intruder, or not, and collecting the conclusive evidence you need to respond to an attack, or not.

In December 2015 Ukraine suffered from a power grid cyber attack that disrupted power distribution to the nation’s citizens. Thirty substations were switched off and damaged leaving 230,000 without power.

This attack was meticulously planned and executed, with the attackers having first gained access over six months before they finally triggered the outage. There were many stages of intrusion and attack, leaving traces that were only identified in subsequent investigations. Well planned and executed threat hunting would probably have uncovered this intruder activity, and averted the serious outages that took place.

This is a good example of why, in the last few years, threat hunting has been gaining substantial momentum and focus amongst SecOps teams, with increasing efforts to better define and formalize it as a discipline. You’ll see a range of definitions with slightly different perspectives, but the following captures the essence of Threat Hunting:

The process of proactively and iteratively searching through IT infrastructure to detect and isolate advanced threats that evade existing security solutions.

There’s also some divergence in approaches to threat hunting, and in the aspects that individual organizations consider most important, but key themes are:

  • To augment automated detection, increasing the likelihood that threats will be detected.
  • To provide insight into attackers’ Tactics, Techniques and Procedures (TTP’s) and hence inform an organization where they should focus their resources and attention.
  • To identify if, and where, automated systems need updating – e.g. with new triggers.

So, threat hunting involves proactively seeking out attacks on your IT infrastructure that are not detected by automated systems such as IDS’s, firewalls, DLP and EDR solutions. It’s distinct from incident response, which is reactive. It may, however, result in an incident response being triggered.

Although threat hunting can be assisted by machine-based tools, it is fundamentally an activity performed by people, not machines, heavily leveraging human intelligence, wisdom and experience.

In the next article, we explore how leading organizations approach threat hunting, and the various data, resources, systems, and processes required to threat hunt effectively and efficiently.

In the meantime, feel free to browse the Useful References page in our Theat Hunting Section on endace.com, which contains both a glossary and useful links to various pages related to threat hunting. Below are some additional useful references.

References

(1) Threat Hunting Report (Cyber Security Insiders), p22

(2) 2018 Threat Hunting Survey Results (SANS), p13

(3) 2018 Threat Hunting Survey Results (SANS), p5

(4) Improving the Effectiveness of the Security Operations Center (Ponemon Institute), p10

(5) The Ultimate Guide To Threat Hunting, InfoSec Institute

 


Watch Endace on Cisco ThreatWise TV from RSA 2019

Original Entry by : Endace

It was a privilege to attend this year’s RSA cybersecurity event in San Francisco, and one of our top highlights was certainly the opportunity to speak to Cisco’s ThreatWise TV host Jason Wright. Watch the video on Cisco’s ThreatWise TV (or below) as Jason interviews our very own Michael Morris to learn more about how Cisco and Endace integrate to accelerate and improve cyber incident investigations.

In this short 4 minute video, Michael demonstrates how Cisco Firepower and Stealthwatch can be used together to investigate intrusion events, using Cisco dashboards and EndaceVision to drill down into events by priority and classification to show where threats come from, who has been affected and whether any lateral movement occurred, as well as conversation history and traffic profiles. Michael also explains how Cisco and Endace work together to ‘find a needle in a haystack’ across petabytes of network traffic.

A big thanks to Cisco and to Jason for giving us this spotlight opportunity. If you have any questions about how Cisco and Endace integrations can accelerate and improve cyber incident investigation, visit our Cisco partner page.


Endace Scoops Award Hat-trick at Info Security Products Guide’s 2019 Global Excellence Awards

Original Entry by : Sebastian Mackay

Endace Scoops Triple at Info Security Products Guide Global Excellence Awards 2019It’s been a great start to the year for Endace, with a triple win at Info Security Products Guide 2019 Global Excellence Awards.

The EndaceProbe Series 9200 was announced as a Gold Winner in the Best Security Hardware Product (New or Updated Version) category, Silver Winner in the Security Investigation category, and Bronze Winner in Network Security and Management category.

The global awards, now in its 15th year, recognize cybersecurity and information technology vendors with advanced, ground-breaking products, solutions, and services that are helping set the bar higher for others in all areas of security and technologies.

The new 9200 Series is the latest model EndaceProbe which has a significant increase in capability; setting new industry benchmarks for speed, density and storage capacity. By introducing built-in hardware compression and patented Smart TruncationTM Endace has quadrupled the storage capacity, doubled the sustained recording speed and tripled the hosting capacity of this model compared to previous models – resulting in the world’s first petabyte network recorder in a single, 4-RU footprint.

“We are extremely proud that our EndaceProbe 9200 Series Analytics Platform has been recognized as a winner by Info Security Products Guide,” said Stuart Wilson, CEO of Endace. “And to receive not just one award but three, amongst such fantastic company, is truly tremendous.”

“This success is a direct result of our relentless drive to stay customer-focused and make packet capture affordable for all enterprises, and above all a fantastic effort from a very talented team here at Endace. The awards further validate our commitment to our customers and their security needs and to the fantastic collaborative relationship we have with our Fusion Partners. It was great to see two of these partners – Darktrace and Ixia – also recognised in these awards: congratulations team.”

 


Investigate Threats Faster than Ever Before

Original Entry by : Sebastian Mackay

The EndaceProbe Analytics Platform allows analysts to capture, store, and analyze petabytes of Network History in real-time.OSm - Operating System for Monitoring

By going back-in-time, analysts can search recorded network traffic and find the precise “needle-in-the-haystack” packets that relate to a security threat, breach or outage, and quickly and accurately reconstruct exactly what took place.

InvestigationManager, released as part of OSm 6.5 for Endace appliances, allows analysts to conduct searches in seconds across petabytes of distributed Network History recorded by the EndaceProbe Analytics Platforms on their network.

Designed for conducting centralized, network-wide investigations, InvestigationManager is built for speed and efficiency and maintains the same ultra-fast response times whether it’s searching a single EndaceProbe or multiple EndaceProbes simultaneously. It does this by parallelizing search and data-mining across all the EndaceProbes being searched, simultaneously.

InvestigationManager is a standalone virtual server application that has a no-cost license. Multiple instances of InvestigationManager can be deployed as needed to manage or control access to Network History by region, network segment, job function or security clearance level.


Endace Selected as SC Media 2019 Trust Award Finalist; Company Recognized in Best Computer Forensics Solution Category

Original Entry by : Sebastian Mackay

Endace, a specialist in high-speed network recording, traffic playback and analytics hosting, today announced that its new, ultra-high-capacity, 9200 Series EndaceProbe™ Analytics Platform has been recognized as a Trust Award finalist in the Best Computer Forensic Solutions category for the 2019 SC Awards. The finalists and winners for the Trust Awards are chosen by an expert panel of judges with extensive knowledge and experience in the cybersecurity industry. Winners will be announced at the SC Awards ceremony on March 5, 2019 in San Francisco.

“Every new year brings with it an unpredictable mix of adversity and opportunity for information security professionals,” said Illena Armstrong, VP, editorial, SC Media. “In 2018, we watched as ransomware took down entire city governments, popular online platforms were accused of mishandling user data, and technology giants announced an unprecedented industry-wide effort to solve the Spectre and Meltdown CPU vulnerabilities. Through it all, this year’s SC Awards finalists found ways to break boundaries, overcome challenges and contribute fresh new ideas to the world of cybersecurity.”

Now in its 22nd year, SC Awards is recognized as the industry gold standard of accomplishment for cybersecurity professionals, products and services. With the awards, SC Media recognizes the achievements of cybersecurity professionals in the field, the innovations happening in the vendor and service provider communities, and the vigilant work of government, commercial and nonprofit entities. Vendors and service providers who offer a product and/or service for the commercial, government, educational, nonprofit or other industries are eligible for the SC Awards’ Trust Award category.

“We are honored that SC Media has recognized the breakthrough accomplishments of our new 9200 Series EndaceProbe Analytics Platform,” said Stuart Wilson, Endace CEO. “With 100% accurate packet capture and up to a petabyte of packet storage capacity on each appliance, the new 9200’s rapid search capability lets security analysts quickly find and analyze specific traffic of interest from within weeks or months of network history recorded by the EndaceProbes on their network.”

“The platform’s ability to simultaneously host third-party and open-source security solutions means customers can deploy their chosen security tools where and when they need to, giving them the agility to keep up with today’s constantly evolving threat landscape without having to change hardware to deploy new security functions,” Wilson said.

“Nobody understands the cybersecurity battle better than the cybersecurity professionals who work day in and day out to clean up and protect businesses from malicious attacks,” added Armstrong of SC Media. “Endace is one of a select few to receive this tremendous recognition of a Trust Award finalist, and they should be proud of the work this represents.”


New OSm 6.5 brings ultra-fast, network-wide search to all EndaceProbe models

Original Entry by : Sebastian Mackay

OSm - Operating System for Monitoring

We are really excited to announce the release of OSm 6.5

This significant new release incorporates some major architectural changes and introduces a truly revolutionary feature – ultra-fast, network-wide search and data-mining – with the brand-new InvestigationManager™ application

Customers are always telling us how important it is to accelerate the investigation of security threats and performance issues so they can respond to them more quickly and more accurately.

InvestigationManager is a game-changer for analysts involved in the investigation process, allowing them to search across petabytes of globally-distributed Network History for specific “packets-of-interest” at lightning-speed, putting definitive evidence at their fingertips when they need it.

New Groundbreaking EndaceFabric Architecture 

Watch this short video for an overview of the architectural changes that OSm 6.5 introduces and how this new architecture underpins the amazing new, ultra-fast search capability that InvestigationManager brings to all EndaceProbe models.

InvestigationManager’s Ultra-Fast Search in Action

Watch this demo to see just how fast InvestigationManager can find specific “needle-in-the-haystack” packet from within more than a petabyte of Network History distributed across multiple EndaceProbes deployed around the world.

(Tip: prepare to be impressed!).

Want to Find Out More?

OSm 6.5 includes a number of other updates including:
• Real-time visualizations in both InvestigationManager and EndaceProbes (“Play Mode”)
• The ability to trigger, collect and export system and RAID dumps from one or more EndaceProbes at a time.

You can read more about the new features of OSm and the new InvestigationManager application on endace.com.

Or watch the video below for a deep-dive into the new features of OSm 6.5.2 and InvestigationManager and what the new ultra-fast search capability of InvestigationManager means for Threat Hunting.

How do I get hold of OSM 6.5?

OSm 6.5 is supported by all current EndaceProbe models.

The downloadable image and documentation for OSm will be available on the Endace Support Portal from early February, 2019.

If you wish to install this new release earlier, please contact your Endace account team.


Packets Are The Ultimate Forensic Evidence, says Cisco’s Doug Hurd

Original Entry by : Sebastian Mackay

Most organizations are overwhelmed by alerts, and 93% are unable to triage all relevant threats. On average, organizations are unable to sufficiently investigate 25% of their alerts.

(McAfee Lab’s Dec 2016 Quarterly Threat Report)

Organizations everywhere are flooded with alerts and many security teams are drowning under the deluge.

So what can security teams do to get ahead of this flood of alerts and keep their heads above water?

We posed that question – and others – to Cisco’s Alliances and Integrations Manager, Doug Hurd.

Watch this short video to hear what he had to say about the value of full packet data and why it is such a powerful complement to Cisco Firepower and Stealthwatch and how integrating Endace’s Network History can help security teams resolve alerts faster and more accurately.

Learn More


Endace Back at Black Hat Europe

Original Entry by : Mark Evans

Well it’s that time of year again. It seems like such a short time ago we were at Black Hat USA, and already Black Hat Europe is here again.

Black Hat Europe, taking place from the 3rd to 6th December, brings together more than 2,000 InfoSec professionals for networking, training and briefings. As a silver sponsor and exhibitor at this year’s event, we are looking forward to engaging again with the Black Hat cybersecurity community—including our Fusion partners, and fellow sponsors, Darktrace and Splunk – and seeing what’s new.

Cyber Skills Shortage: the perennial challenge

One of the interesting things about sponsoring Black Hat in different regions is seeing the common themes that emerge.

At Black Hat USA, back in August at Mandalay Bay, Las Vegas there was much discussion about what we, as an industry, can do to combat the extreme shortage of skilled cyber professionals.

This is perhaps not surprising, given Black Hat’s strong focus on practical skills building, training and workshops. But it was interesting to talk to attendees about the challenges that this shortage of skilled people is causing, and what tools and strategies they’re implementing to help address those challenges.

A key theme was how tools like AI can help to reduce the burden on overworked analysts, leaving them free to focus on high-priority threats, and on proactive, rather than reactive, response. We talked to a number of attendees about why packet capture is such an ideal complement to AI tools because it provides the context that enables security teams to quickly prioritize, investigate and respond to the threats that their AI-based tools detect.

It will be interesting to see what attendees in London have to say about how their companies are addressing the cyber skills shortages in Europe.

What we’ll be talking about

We’ll be talking about our recently launched 9200 Series EndaceProbe Analytics Platform, the world’s first Petabyte Network Recording appliance.

We’ll also be showing how the unique architecture of the EndaceFabric allows customers to connect multiple EndaceProbes together to form single logical stacks of probes with multi-petabyte storage capacity that can monitor high-speed links of 100Gbps and beyond. This “stacking” approach, combined with the breakthrough density and price of the new 9200 Series EndaceProbes, gives organizations the ability to record and store weeks of full packet data.

Come and visit

We’re looking forward to catching up at Black Hat. So if you are attending, do drop in and see us. We’re at Stand 306.