Log and Monitor

Purpose

This page covers notes, labs, personal setup items, deviations, etc. covered in the Design a solution to log and monitor Azure resources Learn module.

Azure Services Overview

Topic/Need Recommendation
Logging Azure Monitor Log Analytics Workspace
Routing Logs Azure Event Hubs
Monitoring Azure Monitor
Authentication Azure Entra ID
Identity Management Azure Entra ID
Authorization Azure Role-Based Access Control (RBAC)
Secret Management Azure Key Vault
Hierarchy for Management Groups Azure Management Groups, Subscriptions, Resource Groups
Managing Compliance Azure Policy
Identity Governance Azure Active Directory Identity Governance features

Design For Azure Monitor Data Sources

  • Virtually every deployment should have the Baseline Alerts enabled
  • Another amazing resource is the Azure Monitor Community repo
  • Azure Monitor Logs: data about and from Azure resources and applications (Event log, platform logs, syslog, etc.)
  • Azure Monitor Metrics: stores data in a time based database and collected at specific intervals.
    • Real-time insights, diagnostics and alerting provide proactive monitoring
  • Setup alert rules based on logs and other metric data to proactively receive notifications
    • There are many pre-defined alert rules - all you have to do is configure and enable them
  • Cost for alert rules is pretty minimal. It will grow with the expansion of resources, complexity of alert rules (dynamic thresholds are more) etc. but revenue should be growing as monitored items decrease.

Azure Event Hubs

  • Ingest and route high volumes of data
  • Supports stream processing and integrates with analytics and storage services for real-time processing

Example Example

Design For Azure Monitor Logs (Log Analytics) Workspaces

  • You can micromanage data access via RBAC, scope and assign as appropriate
  • Design for the appropriate deployment model
    • Centralized: Central spot for all logs for one team
    • Decentralized: Each time has their own workspace with data targeted appropriately
      • Cross correlating log data around events can prove difficult
    • Hybrid: not really worth mentioning
  • Access mode can be based on workspace context (scope data to the workspace –> to the worker) or resources vie the specific resource’s log menu

An example image example workspace model example workspace model

Design For Azure Workbooks and Azure Insights

  • The Azure Monitor Community repo also has a number of pre-canned workbooks
  • Visually represent data, metrics, usage, health, etc. from most of the Azure stack
  • Azure insights provide a customized monitoring experience for particular applications and services.
  • Azure insights collect and analyze both logs and metrics.
  • A significant amount of insights are included in Azure Monitor

Design For Azure Data Explorer

Basically a tool to ingest additional data sources (vs Azure Monitor) run queries against the combined data and present them with the same workbooks, log queries, etc. It also:

  • Provides more real-time(ish) data
  • Application trace logs, Machine learning, anomaly detection, flexible queries, longer data retention and monitoring