Apache Spot Advantages
Apache Spot (Incubating)
A Community Approach to Fighting Cyber Threats
Help Spot expand his hunting ground to see deeper into the darkness of cyber threats.
Apache Spot at a Glance
Apache Spot is a community-driven cybersecurity project, built from the ground up, to bring advanced analytics to all IT Telemetry data on an open, scalable platform. Spot expedites threat detection, investigation, and remediation via machine learning and consolidates all enterprise security data into a comprehensive IT telemetry hub based on open data models. Spot’s scalability and machine learning capabilities support an ecosystem of ML-based applications that can run simultaneously on a single, shared, enriched data set to provide organizations with maximum analytic flexibility. Spot harnesses a diverse community of expertise from Centrify, Cloudera, Cybraics, Endgame, Intel, Jask, Streamsets, and Webroot.
Apache Spot is functional after just one day and just keeps improving through feedback and machine learning.
How It Works
Apache Spot uses machine learning as a filter for separating bad traffic from benign and to characterize the unique behavior of network traffic. A proven process, of context enrichment, noise filtering, whitelisting and heuristics, is also applied to network data to produce a shortlist of most likely security threats.
Key Features
Suspicious DNS packets
Apache Spot is capable of performing deep-packet inspection of DNS traffic to build a profile of probable and improbable DNS payloads. After visualizing, normalizing, and conducting pattern searches, the analyst has a shortlist of the most likely threats present in DNS traffic.
Threat Incident and Response
Given an IP address, Apache Spot gathers all the characteristics about the communication associated with it – the “social network” of that IP address. Then Apache Spot builds a timeline of the conversations that originated with that IP.
Suspicious Connects
Apache Spot uses advanced machine learning to build a model of the machines on the network and their communication patterns. The connections between the machines that are the lowest probability are then visualized, filtered for noise, and searched for known patterns. The result is the most likely threat patterns in the data, a few hundred flows picked from billions.
Storyboard
After an analyst has investigated a threat, the need still exists to communicate the event up and across the organization. A “dashboard” gives quick answers to the questions you already know to ask. What the analyst requires is a “storyboard,” something that tells who, what, where, and how of the story in words and interactive visualizations.
Open Data Models
Spot provides common open data model for network, endpoint, and user – Open Data Models. These Open Data Models provide a standard format of enriched event data that makes it easier to integrate cross application data to gain complete enterprise visibility and develop net new analytic functionality. Spot’s Open Data Models helps organizations quickly share new analytics with one another as new threats are discovered.
Collaboration
Spot’s Open Data Models help organizations quickly share new analytics with one another as new threats are discovered. And, with Hadoop, organizations able to run these analytics against comprehensive historic data sets, helping organizations identify past threats that have slipped through the cracks. With this capability, Spot aims to give security professionals the ability to collaborate like cybercriminals do.
Apache Spot Open Data Models (ODM)
The primary use case initially supported by Spot includes Network Traffic Analysis for network flows (Netflow, sflow, etc.), DNS and Proxy. The Spot open data model strategy aims to extend Spot capabilities to support a broader set of cybersecurity use cases.
ODM at a Glance
- Includes a growing catalog of packaged ingestion pipelines for common data sources
- Enriched events provide full context leading to better analytics and faster incident response
- Organizations maintain and control a single copy of their security data
Spot Fosters a Rich Application Ecosystem
Spot accelerates the development of cybersecurity applications by providing a cybersecurity analytics framework. This means more solutions can be created faster. This is because Spot allows organizations to focus developing the analytics and visualizations for applications that discover cybercrime rather than spending time building systems to ingest, integrate, store, and process myriad volumes or varieties of security data.
Join the Apache Spot community and collaborate with us using a common framework.
Use Case
Network Traffic Analytics
Spot allows organizations to detect potentially malicious activity by identifying suspicious network connections, by analyzing large amounts of netflow, DNS, proxy data with algorithms that are available out of the box.
Reduction of mean time to incident detection &resolution (MTTR)
Spot provides the capability for a central data store that houses ALL the data needed to facilitate an investigation, returning investigative query results in seconds and minutes (vs. hours and days), which effectively reduces incident MTTR and minimizes the adverse impacts of a breach.
Threat Hunting
Spot improves the efficacy of threat hunting by providing the analytic flexibility to perform ad-hoc searches and queries over vast amounts of data, as well as applying ad-hoc algorithms to detect the needle in the haystack.
Cybersecurity Data Management
Offloading data from legacy cybersecurity systems (e.g., SIEMs) to Spot delivers immediate economic value, because of the cost of data storage and processing with Hadoop. It also opens up future value as organizations deploy one of the many analytics use cases on their newly formed security data hub.
Identify the needle in the haystack with patterns that provide insight into potential threats.
Apache Spot is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
The contents of this website are © 2016 Apache Software Foundation under the terms of the Apache License v2. Apache Spot and its logo are trademarks of the Apache Software Foundation.