Sentinel

---
name: sentinel
description: "Expert agent for Microsoft Sentinel. Provides deep expertise in KQL query development, analytics rules, ASIM normalization, data connectors, UEBA, automation rules, playbooks, content hub, multi-workspace architecture, and cost optimization with log tiers. WHEN: \"Sentinel\", \"Microsoft Sentinel\", \"KQL\", \"analytics rule\", \"ASIM\", \"Fusion\", \"UEBA\", \"content hub\", \"Sentinel playbook\", \"Defender portal\", \"Log Analytics workspace\"."
license: MIT
---

# Microsoft Sentinel

This skill covers Microsoft Sentinel, Azure's cloud-native SIEM and SOAR platform. It has deep knowledge of:

- Kusto Query Language (KQL) for threat detection and investigation
- Analytics rules (scheduled, NRT, Fusion, ML, anomaly)
- Advanced Security Information Model (ASIM) normalization
- Data connectors (350+) and ingestion architecture
- UEBA (User and Entity Behavior Analytics)
- Automation rules and playbooks (Logic Apps)
- Content hub (solutions, analytics rules, workbooks, playbooks)
- Multi-workspace and multi-tenant architecture
- Cost optimization (basic logs, analytics logs, commitment tiers, data collection rules)
- Integration with Microsoft Defender XDR (unified security operations portal)

**Key architectural note:** Microsoft Sentinel is migrating to the unified Defender portal (security.microsoft.com). This migration is mandatory by July 2026. New deployments should use the unified portal.

## How to Approach Tasks

1. **Classify** the request:
   - **Detection engineering** -- KQL analytics rules, ASIM-normalized detections
   - **Investigation** -- KQL hunting queries, entity investigation, incident triage
   - **Architecture** -- Workspace design, data connector configuration, multi-tenant setup
   - **Automation** -- Automation rules, playbook design, SOAR integration
   - **Cost management** -- Log tier optimization, DCR filtering, commitment tiers
   - **SOAR-specific** -- Read the `sentinel-playbooks` skill

2. **Gather context** -- Azure subscription model, Microsoft 365 licensing, existing Defender products, log volume, compliance requirements

3. **Check ASIM compatibility** -- For cross-source detections, verify ASIM parsers exist for the data sources

4. **Recommend** actionable guidance with KQL examples and Azure portal steps

## Core Expertise

### KQL Fundamentals

KQL (Kusto Query Language) is the query language for Sentinel and Log Analytics:

```kql
// Basic threat hunting: brute force detection
SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType != "0"  // Non-success
| summarize FailureCount = count(), 
    DistinctUsers = dcount(UserPrincipalName),
    Users = make_set(UserPrincipalName, 10)
    by IPAddress, bin(TimeGenerated, 5m)
| where FailureCount > 20
| sort by FailureCount desc
```

**KQL key operators:**

| Operator | Purpose | Example |
|---|---|---|
| `where` | Filter rows | `where Status == "Failed"` |
| `summarize` | Aggregate | `summarize count() by User` |
| `extend` | Add calculated column | `extend Duration = EndTime - StartTime` |
| `project` | Select columns | `project TimeGenerated, User, Action` |
| `join` | Combine tables | `T1 \| join kind=inner T2 on Key` |
| `union` | Merge tables | `union Table1, Table2` |
| `let` | Declare variables | `let threshold = 10;` |
| `render` | Visualize | `render timechart` |
| `has` / `contains` | String search | `where Field has "error"` (word) vs `contains` (substring) |
| `has_any` / `has_all` | Multi-value match | `where Field has_any ("err","warn")` |
| `between` | Range filter | `where Count between (10 .. 100)` |
| `ago` | Relative time | `where TimeGenerated > ago(24h)` |

**KQL performance tips:**
- Use `has` over `contains` (word boundary search is indexed; substring is not)
- Filter with `where` before `join` or `summarize` to reduce data volume
- Use `project` early to drop unnecessary columns
- Use `materialize()` to cache intermediate results referenced multiple times
- Avoid `*` in `summarize` -- specify only needed aggregations
- Use `bin()` for time-based grouping (not `floor()`)

### Analytics Rules

Sentinel analytics rules detect threats and create incidents:

**Rule types:**

| Type | Trigger | Latency | Use Case |
|---|---|---|---|
| **Scheduled** | Runs on cron (every 5m to 24h) | 5-15 minutes | Standard detection rules |
| **NRT (Near Real-Time)** | Runs every minute on latest data | ~1 minute | Time-sensitive detections |
| **Fusion** | ML-based multi-stage attack detection | Minutes | Advanced persistent threat chains |
| **ML Behavior Analytics** | Anomaly detection models | Hours | Baseline deviation detection |
| **Anomaly** | Built-in anomaly detection | Hours | Pre-built anomaly detections |
| **Threat Intelligence** | IOC matching against TI feeds | Minutes | Known-bad indicator detection |

**Scheduled rule example:**
```kql
// Analytics rule: Suspicious PowerShell command
let lookback = 1h;
DeviceProcessEvents
| where TimeGenerated > ago(lookback)
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any (
    "-EncodedCommand", "-enc ", "Invoke-Expression",
    "IEX(", "Net.WebClient", "DownloadString",
    "Invoke-Mimikatz", "Invoke-Obfuscation"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine, InitiatingProcessFileName
| extend AccountCustomEntity = AccountName, HostCustomEntity = DeviceName
```

**Rule configuration:**
- **Query frequency** -- How often the rule runs (e.g., every 5 minutes)
- **Lookup period** -- How far back the query looks (must overlap with frequency)
- **Alert threshold** -- Minimum results to trigger (usually > 0)
- **Entity mapping** -- Map query fields to Sentinel entities (Account, Host, IP, URL)
- **Incident grouping** -- Group related alerts into a single incident
- **Automated response** -- Trigger automation rules or playbooks

### ASIM (Advanced Security Information Model)

ASIM normalizes data from different sources into a common schema at query time:

```kql
// Without ASIM: must query each table separately
SigninLogs | where ResultType != "0" | project TimeGenerated, UserPrincipalName, IPAddress
| union (
    AADNonInteractiveUserSignInLogs | where ResultType != "0" | project TimeGenerated, UserPrincipalName, IPAddress
)

// With ASIM: single normalized query
imAuthentication
| where EventResult == "Failure"
| project TimeGenerated, TargetUsername, SrcIpAddr
```

**ASIM schemas:**
- `imAuthentication` -- Login/logout events across all sources
- `imNetworkSession` -- Network connection events
- `imDns` -- DNS query/response events
- `imProcessEvent` -- Process creation/termination
- `imFileEvent` -- File creation/modification/deletion
- `imRegistryEvent` -- Windows registry changes
- `imWebSession` -- HTTP/HTTPS sessions

**ASIM parsers:**
- **Source-specific parsers** -- `vimAuthenticationAADSigninLogs`, `vimAuthenticationWindowsEvent`
- **Unifying parsers** -- `imAuthentication` (unions all source-specific parsers)
- **Custom parsers** -- Create for unsupported data sources

### Data Connectors

Sentinel ingests data through 350+ connectors:

| Category | Key Connectors | Volume Impact |
|---|---|---|
| **Microsoft 365** | Defender XDR, Entra ID, Office 365, Purview | High (often free with M365 E5) |
| **Azure** | Activity logs, Diagnostics, NSG flow logs | Medium-High |
| **Security products** | CrowdStrike, Palo Alto, Fortinet, Zscaler | High |
| **Cloud** | AWS CloudTrail/S3, GCP Pub/Sub | Medium |
| **Linux/Network** | Syslog, CEF, ASIM | Variable |
| **Custom** | Log Analytics API, Azure Monitor Agent, Codeless connectors | Variable |

**Free data sources (no ingestion charge):**
- Azure Activity logs
- Microsoft Defender XDR incidents and alerts (not raw data)
- Office 365 audit logs (limited)
- Security alerts from Microsoft Defender products

### Multi-Workspace Architecture

| Scenario | Pattern | Reason |
|---|---|---|
| **Single tenant, compliance** | Workspace per region | Data residency requirements |
| **Multi-tenant (MSSP)** | Azure Lighthouse + workspace per customer | Tenant isolation |
| **High volume** | Workspace per data tier | Cost optimization (basic vs analytics logs) |
| **Dev/Test** | Separate workspace | Isolation from production |

**Cross-workspace queries:**
```kql
workspace("other-workspace").SecurityEvent
| where EventID == 4625
| union (SecurityEvent | where EventID == 4625)
| summarize count() by Computer
```

### Cost Optimization

Sentinel costs = Log Analytics ingestion + Sentinel analytics charge + retention.

**Log tiers:**

| Tier | Cost | Search | Retention | Use Case |
|---|---|---|---|---|
| **Analytics logs** | Full price | Full KQL | 90 days free, up to 2 years | Active detection and investigation |
| **Basic logs** | ~60% cheaper | Limited KQL (8-day window) | 30 days, then archive | High-volume, low-query data |
| **Archive** | Storage-only pricing | Search jobs (async) | Up to 12 years | Compliance, forensic |

**Cost optimization strategies:**
1. **Commitment tiers** -- Pre-pay for daily ingestion (100 GB/day = ~50% discount vs pay-as-you-go)
2. **Data Collection Rules (DCR)** -- Filter events before ingestion (e.g., drop informational events)
3. **Basic logs for high-volume, low-value data** -- Firewall allowed traffic, DNS queries, success-only auth
4. **Archive for compliance** -- Move beyond-retention data to archive tier
5. **Free data sources** -- Maximize use of data sources that don't incur Sentinel charges
6. **Summary rules** -- Pre-aggregate statistics; query summaries instead of raw data

**DCR filtering example:**
```json
{
  "transformKql": "source | where EventID != 4688 or CommandLine has_any ('powershell', 'cmd', 'wscript')"
}
```

### UEBA (User and Entity Behavior Analytics)

UEBA builds behavioral baselines and detects anomalies:

- **Entity pages** -- Consolidated view of user/host activity, anomalies, and risk
- **Anomaly detection** -- ML-based detection of unusual behavior (impossible travel, unusual resource access, abnormal data transfer)
- **Investigation priority** -- Entities ranked by anomaly score for analyst prioritization
- **Timeline** -- Chronological view of entity activity across all data sources

```kql
// Query UEBA anomalies
BehaviorAnalytics
| where TimeGenerated > ago(7d)
| where ActivityInsights has "anomaly"
| project TimeGenerated, UserPrincipalName, ActivityType, ActivityInsights, InvestigationPriority
| sort by InvestigationPriority desc
```

### Defender XDR Integration

Sentinel integrates deeply with Microsoft Defender XDR:

- **Unified incident queue** -- Incidents from Defender XDR and Sentinel in one view
- **Advanced hunting** -- KQL across both Defender XDR and Sentinel data
- **Bi-directional sync** -- Status, assignments, comments sync between portals
- **Unified portal** -- security.microsoft.com consolidates Sentinel + Defender XDR (mandatory migration by July 2026)

## Common Pitfalls

1. **KQL `contains` vs `has`** -- `contains` does substring search (slow, not indexed). `has` does word-boundary search (fast, indexed). Always prefer `has` unless you need substring matching.
2. **Ingestion delay in analytics rules** -- Set lookup period longer than query frequency to avoid gaps. E.g., query every 5 min, look back 10 min.
3. **Basic logs limitations** -- Basic logs support only simple KQL (no join, limited summarize). Cannot be used in analytics rules. Plan tier assignment carefully.
4. **Entity mapping omission** -- Analytics rules without entity mapping produce incidents that can't be investigated with UEBA or entity pages. Always map entities.
5. **Content hub drift** -- Solutions installed from content hub may be overwritten on update. Customize by cloning rules, not editing originals.
6. **Workspace sprawl** -- Every workspace has a fixed overhead cost and operational complexity. Minimize workspace count unless required by compliance or isolation.

## Reference Files

Load these for deep knowledge:
- `references/architecture.md` -- Log Analytics workspace internals, data connectors, ASIM, Fusion ML, multi-workspace patterns
- `references/best-practices.md` -- KQL optimization, analytics rule tuning, UEBA, cost management, content hub strategy