Awesome Detection Engineering

Awesome Detection Engineering: A Practical, Comprehensive Guide to Building Proactive Defenses

Introduction

Detection engineering is the tactical backbone of a modern cybersecurity defense program. It is the deliberate design, implementation, and operation of detective controls with the explicit goal of proactively identifying malicious or unauthorized activity before it causes harm. In practice, detection engineering blends threat intelligence, security telemetry, and disciplined engineering to create a living library of detection content that maps to real-world adversary behaviors. This guide distills the core concepts, frameworks, content sources, data sources, and general resources that detection engineers rely on to build, measure, and improve detection coverage across an organization.

Table of contents

• Concepts & Frameworks
• Detection Content & Signatures
• Logging, Monitoring & Data Sources
• General Resources

Concepts & Frameworks

A solid detection program rests on a clear understanding of adversary behavior, how it is observed, and how to detect it efficiently. The following frameworks and concepts form the backbone of detection strategy and maturation.

• MITRE ATT&CK: The foundational framework of adversary tactics, techniques, and procedures based on real-world observations. MITRE ATT&CK provides a common language for describing how attackers operate and what telemetry is most likely to reveal their presence. Detecting against ATT&CK techniques helps ensure coverage across the attack lifecycle, from initial access to impact.

• Alerting and Detection Strategies (ADS) Framework | Palantir: A blueprint for creating and documenting effective detection content. The ADS framework emphasizes clarity, traceability, and actionable alerts, helping teams describe the what, why, and how of detections in a way that aligns with investigations and response.

• Detection Engineering Maturity Matrix | Kyle Bailey: A detailed matrix that serves as a tool to measure the overall maturity of an organization's Detection Engineering program. It guides improvement from basic signal generation to automated, scalable detection pipelines, emphasizing governance, quality, and measurable outcomes.

• Detection Maturity Level (DML) Model | Ryan Stillions: Defines and describes eight different levels of an organization's threat detection program maturity. The DML model provides a ladder of capability, from ad hoc detections to an integrated, threat-informed security program with continuous feedback loops.

• The Pyramid of Pain | David J Bianco: A model used to describe various categorizations of indicators of compromise and their level of effectiveness in detecting threat actors. The pyramid guides prioritization of detection investments—from infrastructure-level signals to high-context, attacker-centric indicators.

• Cyber Kill Chain | Lockheed Martin: A framework outlining the seven stages commonly observed in a cyber attack. Understanding the kill chain helps detection teams target the most effective points for early warning and disrupt adversary progression.

• MaGMa Use Case Definition Model | (Use Case Framework): A business-centric approach for defining threat detection use cases. MaGMa emphasizes aligning detection work with business risk, value, and measurable outcomes, ensuring detection content supports real-world protective goals.

• Synthetic Adversarial Log Objects (SALO) | Splunk: A framework for the generation of log events without the need for infrastructure or actions to initiate the event. SALO enables testing and validation of detections against synthetic, realistic log data, reducing blind spots in coverage.

• The Zen of Security Rules | Justin Ibarra: Outlines 19 aphorisms that serve as universal principles for the creation of high-quality detection content. These aphorisms emphasize clarity, maintainability, and threat-informed design.

• Blue-team-as-Code — The Spiral of Joy | Den Iuzvyk, Oleg Kolesnikov: Lessons From Real-world Red Team Detection Automation Using Logs. This perspective highlights iterative, code-like treatments of defensive content, promoting automation, versioning, and collaboration.

• Detection Development Lifecycle | Haider Dost et al.: Snowflake’s implementation of the Detection Development Lifecycle. This lifecycle covers discovery, design, development, testing, deployment, and evaluation of detections, ensuring repeatable, auditable processes.

• Threat Detection Maturity Framework | Haider Dost of Snowflake: A maturity matrix to measure the success of your threat detection program. It complements the DML by focusing on the practical outcomes and operational health of detection activities.

• Elastic's Detection Engineering Behavior Maturity Model: Elastic’s qualitative and quantitative approach to measuring threat detection program maturity. This model provides concrete behavior-based milestones across detection, analytics, and response.

• Detection Engineering AI Maturity Framework | Brendan Chamberlain: A community framework with four maturity levels across ten dimensions for assessing how organizations apply AI and LLMs across a detection engineering program, from foundations through the detection lifecycle.

• Prioritizing Detection Engineering | Ryan McGeehan: A longtime detection engineer outlines how a detection program should be built from the ground up. This piece emphasizes prioritization, pragmatic scope, and early wins to demonstrate value.

• Detection Engineering Field Manual | Zack Allen: A series of posts exploring the various foundational components of Detection Engineering. The Field Manual format provides practical guidance, checklists, and reference implementations.

• Open Threat Informed Detection Engineering (OpenTide) | OpenTideHQ: An all-in-one Detection Engineering Operations framework created and maintained by the European Commission. OpenTide converts CTI into an actionable detection coverage graph that aligns threat vectors with detection objectives, and manages the entire detection library in a central repository with a detection-as-code deployment system. It supports multi-platform deployment and cross-team interoperability, measuring and expanding coverage as threats evolve.

• ThreatMapper | Andrey Pautov: CTI-to-detection workbench for mapping threat reports to ATT&CK, comparing TTP overlap with groups and campaigns, identifying detection gaps, and exporting analyst-ready outputs. ThreatMapper helps translate threat intelligence into concrete, testable detections.

• ZettelForge | ZettelForge: An agentic memory system that treats Sigma and YARA rules as first-class memory entities, with an LLM rule explainer, STIX 2.1 knowledge graph of CTI entities, and offline-first RAG to connect rules to the actors and techniques they detect. Python, MIT. ZettelForge provides a framework for connecting detection rules with CTI context, enabling more intelligent rule reasoning and maintenance.

Detection Content & Signatures

A robust detection program needs a curated library of detection content and rules that can be deployed, evaluated, and refined. The following sources provide ready-made content, templates, and mappings to help teams accelerate coverage and reduce duplication of effort.

• Rulehound: An index of publicly available and open-source threat detection rulesets. Rulehound aggregates and points to community-driven detections, enabling teams to discover, compare, and reuse rules across tools and environments.

• MITRE Cyber Analytics Repository (CAR): MITRE's well-maintained repository of detection content. CAR hosts analytics and detection stories aligned with ATT&CK techniques, providing practical examples for defenders to implement and test.

• CAR Coverage Comparison: A matrix of MITRE ATT&CK technique IDs and links to available Splunk Security Content, Elastic detection rules, Sigma rules, and CAR content. This matrix helps teams understand where coverage exists and where gaps remain for specific techniques.

• Sigma Rules: Sigma's repository of turnkey detection content. Sigma rules are platform-agnostic detections that can be converted to run on many SIEMs, enabling portability and cross-platform consistency.

• Sigma Rule Converter: An open-source tool that can convert detection content for use with most SIEMs. This converter reduces the friction of migrating detections between platforms and keeps rule logic portable.

• AttackRuleMap: Mapping of open-source detection rules and atomic tests. AttackRuleMap provides a reference to understand how detections align with ATT&CK techniques and sub-techniques.

• Splunk Security Content: Splunk's open-source and frequently updated detection content that can be tweaked for use in other tools. This content is a practical starting point for building detections in Splunk, with ongoing community contributions.

• Elastic Detection Rules: Elastic's detection rules written natively for the Elastic SIEM. These rules cover a wide range of threat behaviors and can be adapted for other SIEMs using conversion tools like Uncoder.

• Elastic Endpoint Behavioral Rules: Elastic's endpoint behavioral (prevention) rules written in EQL, designed for the Elastic endpoint agent. They offer proactive protection and detection at the endpoint level.

• Elastic Yara Signatures: Elastic's YARA signatures, which run on the Elastic endpoint agent. YARA rules help classify and detect malicious files and artifacts on endpoints.

• Elastic Endpoint Ransomware Artifact: Elastic's ransomware artifact, which runs on the Elastic endpoint agent to detect ransomware behaviors and artifacts.

• Chronicle (GCP) Detection Rules: Chronicle's detection rules written natively for the Chronicle Platform. Chronicle aims to provide scalable, cloud-native detection coverage for Google Cloud environments.

• Exabeam Content Library: Exabeam's out-of-the-box detection content compatible with the Exabeam Common Information Model (CIM). This library provides ready-made detections aligned with CIM structures.

• Panther Labs Detection Rules: Panther Lab's native detection rules. Panther’s rules support detection analytics and can be adapted to other platforms as needed.

• Anvilogic Detection Armory: Anvilogic's open-source and publicly available detection content. The Armory offers detections, templates, and examples to accelerate development.

• AWS GuardDuty Findings: A list of all GuardDuty findings, their descriptions, and associated data sources. Understanding GuardDuty findings helps map cloud telemetry to detections and correlate with other data sources.

• GCP Security Command Center Findings: A list of all Security Command Center findings, their descriptions, and associated data sources. This resource helps map cloud findings to detection coverage and response workflows.

• Azure Defender for Cloud Security Alerts: A list of all Defender for Cloud alerts, their descriptions, and associated data sources. It provides telemetry mapping for detection content in Azure environments.

• Center for Threat-Informed Defense Security Stack Mappings: Describes cloud platform built-in detection capabilities and their mappings to MITRE ATT&CK. This resource helps align native detections with a standardized framework.

• Detection Engineering with Splunk: A GitHub repository dedicated to sharing detection analytics in SPL. This repository is a practical hub for learning, replicating, and adapting Splunk-specific analytics.

• Google Cloud Security Analytics: A community-driven list of sample security analytics for auditing cloud usage and detecting threats in Google Cloud. It provides practical analytics patterns and examples tailored to GCP environments.

• KQL Advanced Hunting Queries & Analytics Rules: A list of endpoint detections and hunting queries for Microsoft Defender for Endpoint, Defender for Identity, and Defender for Cloud Apps. This resource is particularly useful for defenders using Microsoft security products.

• Sigma2KQL: A repository of all Sigma rules converted to KQL that runs on a weekly schedule to stay up to date with the latest Sigma rules repository. This helps bridge the gap between Sigma rules and Microsoft Defender environments.

• TerraSigma: A repository of all Sigma rules converted to Microsoft Sentinel Terraform Scheduled analytic resources. TerraSigma ensures consistency and automation when deploying detections to Microsoft Sentinel.

• Proper entity mapping is completed for the rules to ensure the repo is plug-and-play.

• Detections Digest | Sergey Polzunov: A newsletter featuring updates from many popular detection content sources. Detections Digest helps practitioners stay informed about new detections, rule updates, and evolving best practices.

Logging, Monitoring & Data Sources

Detection content is only as good as the data that feeds it. The following data sources, logging practices, and monitoring frameworks describe where detections come from, how to normalize data, and how to turn telemetry into actionable insights.

• Windows Logging Cheatsheets: A set of Windows event logging recommendations at varying granularity. These cheatsheets guide how to instrument Windows in a way that yields high-value telemetry for detections.

• Linux auditd Detection Ruleset: A ruleset for Linux auditd that provides telemetry required for threat detection use cases. This set of rules helps capture kernel events, file access, and other critical Linux activities.

• MITRE ATT&CK Data Sources Blog Post: MITRE describes various data sources and how they relate to TTPs found in the ATT&CK framework. This background helps teams decide which data sources to instrument for a given technique.

• MITRE ATT&CK Data Sources List: Data source objects added to MITRE ATT&CK as part of v10. This list keeps defenders aligned with the evolving data source taxonomy.

• Splunk Common Information Model (CIM): Splunk's proprietary model used as a framework for normalizing security data. CIM provides a common vocabulary and schema for diverse data.

• Elastic Common Schema (ECS): Elastic's framework for normalizing security data. ECS helps achieve consistent fields across telemetry from different sources.

• Exabeam Common Information Model (CIM): Exabeam's CIM, used to normalize telemetry within Exabeam workflows. This supports consistent detections across data sources.

• Open Cybersecurity Schema Framework (OCSF): An open-source security data source and event schema. OCSF provides a vendor-agnostic schema for interoperability.

• osquery | Facebook: A SQL-powered OS instrumentation, monitoring, and analytics framework exposing OS data as relational tables for querying and detection.

• Loghub | Logpai: Open-source security data sources for research and testing. Loghub aggregates sources to facilitate experimentation and rule testing.

• ElastAlert | Yelp: ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch. ElastAlert helps teams implement time-series-based detections with ease.

• Matano: An open-source cloud-native security lake platform (SIEM alternative) for threat hunting, Python detections-as-code, and incident response on AWS. Matano supports detections-as-code and scalable threat hunting in cloud environments.

• Microsoft XDR Advanced Hunting Schema: Helps with multi-table queries in Defender XDR for event information, devices, alerts, identities, and other entities. Advanced hunting schemas enable richer investigations and faster detections.

• InnerWarden: An autonomous security agent for Linux with real-time threat detection and response via 38 eBPF hooks, 48 detectors, and 23 correlation rules. This provides a high-fidelity, low-overhead detection surface.

• Rustinel | Karib0u: Open-source endpoint detection engine for Windows and Linux that collects ETW/eBPF telemetry and evaluates Sigma, YARA, and IOC detections. Rustinel demonstrates a modular, telemetry-driven approach to detection on endpoints.

General Resources

Beyond frameworks and content, a detection program relies on ongoing guidance, collaboration, and learning. The following resources help teams plan, execute, and mature detection efforts with best practices, community wisdom, and practical tooling.

• ATT&CK Navigator | MITRE: MITRE’s open-source tool used to map detection coverage, visibility, and other efforts to the ATT&CK framework. Navigator is a practical way to visualize coverage gaps and plan improvements.

• Detection Engineering Weekly | Zack Allen: A newsletter dedicated to news and how-tos for Detection Engineering. Regular reading helps practitioners stay current with trends, techniques, and tooling.

• Detection Engineering Twitter List | Zack Allen: A curated list of Detection Engineering thought leaders. Following this list provides quick access to insights, discussions, and new resources from practitioners.

• DETT&CT — Mapping Your Blue Team to MITRE ATT&CK: Outlines a methodology for measuring security data visibility and detection coverage against the MITRE ATT&CK framework. This mapping helps quantify how well an environment can observe attacker techniques.

• Awesome Kubernetes Threat Detection: An Awesome List dedicated to Kubernetes threat detection. This resource focuses on cloud-native defense patterns and detections.

• Detection and Response Pipeline: A curated list of tools for each component of a detection and response pipeline, including real-world examples. This helps teams assemble end-to-end capabilities.

• Living Off the Land: A collection of resources for thriving off the land. This term highlights detection opportunities that leverage legitimate tools and living off the land techniques to detect abuse.

• Detection at Scale Podcast | Jack Naglieri: A detection-engineering-focused podcast featuring thought leaders in the specialization. Listening to experts discusses scalable detection approaches and real-world practice.

• Cloud Threat Landscape | Wiz: A cloud detection engineering-focused database listing threat actors, their tools, techniques, and preferred targets in cloud environments. It helps align cloud detections with current attacker behavior.

• CTI Analyst Field Manual | Andrey Pautov: Practical CTI-to-detection reference with evidence labels, source reliability, ATT&CK mapping guidelines, hunting hypotheses, and detection backlog workflow. A practical guide for converting intelligence into detections.

• Splunk ES Correlation Searches Best Practices | OpsTune: A detailed guide to producing high-quality detection content in the Splunk Enterprise Security app. It emphasizes robust correlation logic, alerting discipline, and investigative value.

• How Google Does It: Making Threat Detection High-Quality, Scalable, and Modern | Anton Chuvakin, Tim Nguyen: Google’s perspective on modernizing threat detection, with key principles and actionable guidance for large-scale programs.

• SOCLabs: A lab for blue teamers and detection engineers, offering real threat data and support for popular SIEM query languages. SOCLabs supports hands-on practice and experimentation with detections.

Practical guidance for building and maturing a detection program

• Start with a threat-informed baseline: Map detection coverage to MITRE ATT&CK techniques most relevant to your business, data sources, and threat model. A prioritized baseline ensures early wins while maintaining focus on high-impact techniques.

• Build detections as code: Treat detections as code, maintain versioning, peer reviews, and automated testing. Use detection development lifecycles to ensure repeatability and quality.

• Leverage open content responsibly: Use community detections to accelerate development, but validate, tailor, and test them against your environment before production.

• Quantify coverage and quality: Use dashboards and metrics to measure detection coverage, false positive rates, mean time to detect (MTTD), and mean time to respond (MTTR). Regularly reassess and re-prioritize.

• Integrate CTI and data sources: Continuously map new threat intelligence to existing detections and data sources. Keep a backlog of enhancements that align with evolving adversary behavior.

• Foster collaboration: Encourage blue-team collaboration across security operations, incident response, engineering, and risk management. A detection program thrives when it is embedded within the broader security and business context.

• Embrace automation and AI judiciously: Apply automation where it scales detections and reduces toil, while maintaining human oversight for tuning, testing, and investigations. Use AI-assisted rule authoring and risk-aware validation to avoid drift.

• Operationalize detection content: Ensure detections feed into reliable alerting, investigation playbooks, and response actions. A detection program should produce actionable signals that improve incident outcomes.

• Plan for cloud and on-premises parity: Cloud environments bring unique telemetry and data sources. Build detections that span hybrid environments and align with cloud-native security services when appropriate.

• Continuously improve: Treat detection as an ongoing lifecycle. Collect feedback from investigations, measure false positives, and iterate on detection logic, data quality, and coverage.

Closing thoughts

A robust Detection Engineering program is a disciplined blend of theory, practical content, and data-driven practice. By anchoring work in established frameworks like MITRE ATT&CK, following a rigorous detection development lifecycle, and leveraging a broad ecosystem of detection content and data sources, security teams can raise the fidelity, coverage, and speed of detection. The open landscape of community resources, best practices, and tooling provides a rich toolkit for defenders to adapt to evolving threats while maintaining a defensible, auditable, and scalable detection program.

If you’re just starting, begin with a core set of detections tied to high-risk techniques, instrument essential data sources, and adopt a detection-as-code approach. As your program matures, expand coverage to cover more techniques, platforms, and cloud services, guided by a clear maturity roadmap and the indicators of a threat-informed defense. The journey is iterative, collaborative, and ultimately centered on turning signals into insight, and insight into secure, resilient operations.