3.6 KiB
title, type, source-type, category, tags, date-added, video-source, audio-source, status
| title | type | source-type | category | tags | date-added | video-source | audio-source | status | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CTP Topic 54 ESM SaaS Log Analytics | cloud-learning | video | DevOps & SRE/04_EKS |
|
2026-04-14 | nas:///volume2/work/Public Cloud Learning Sessions/CTP _ Topic 54_ ESM SaaS Log Analytics.mp4 | summarized (Gemini 摘要) |
CTP Topic 54 ESM SaaS Log Analytics
Source: NAS /volume2/work/Public Cloud Learning Sessions/CTP _ Topic 54_ ESM SaaS Log Analytics.mp4
Type: VIDEO | Category: 04_EKS
Status: 🟡 Awaiting Whisper transcription → Summary
摘要
ESM SAS Log Analytics
Jackie, an ITOM ESM SAS architect, discusses Log Analytics, covering concepts, architecture, regional setup, provisioning, security, and a demo of a counter solution. He also briefly compares different solutions.
The presentation begins with an overview of the ELK stack (Elasticsearch, Logstash, Kibana) and its open-source alternative, OpenSearch. Applications collect logs via BEATS, which are then aggregated and processed by Logstash to give meaning to each column, before being stored in Elasticsearch or OpenSearch. Kibana is used as a front-end for log file visualization and analysis.
The application collects your log, it's called the BEATS. The architecture involves two VPCs: one for the application and another for logging. Filebeat, running as a container, continuously ships logs from the application VPC to the logging VPC. Logstash processes these logs, and OpenSearch stores them. End users can view logs via Kibana, connecting from a specified network. Redis is used as an optional buffer to prevent Logstash overload.
Due to legal reasons like GDPR, farms are split regionally, with farms in Oregon, the US, and Europe. Provisioning is done via CloudFormation or Terraform, but security hardening and continuous optimization pose challenges. Security measures include encryption at rest (using encrypted nodes and hardware-level encryption on NVMe devices) and in transit (using TLS 1.2). Traffic between VPCs is private, not over the internet. Index-based access control and RBAC are implemented for different user roles.
A demo shows how to search for specific IDs or services within the logs. A comparison of solutions like Logz.io, AWS OpenSearch, self-hosted ELK, and Microfocus OBA is provided. Logz.io is a managed ELK solution, while OBA offers more mature commercial options with automated clustering. ELK is easy to configure but complex to manage, while OBA is more mature with commercial options. ELK supports fine-grained access control, while OBA supports column-level access control.
Cost estimates are provided based on a single farm usage with 14 days retention and 100GB processed daily. Logz.io costs around $4,000, while AWS OpenSearch costs around $1,500 or less. Self-hosted options can be very low cost but require more maintenance. Availability SLAs vary, with Logz.io offering 99.8% and AWS OpenSearch offering 99.9%. Disaster recovery is covered by the vendor for Logz.io, while AWS OpenSearch automatically captures snapshots.
Recommendations for starting with Log Analytics include beginning with Logz.io for its trial period, then transitioning to AWS OpenSearch or self-hosted options for more control. The presentation concludes with a Q&A session covering GDPR requirements, log acquisition, cost details, scaling, and comparisons to other solutions. We have already built up all the farms.
关键概念
行动项
相关视频
配对视频笔记链接（生成后填入）
最后更新: 2026-04-14