How Rakbank built centralized observability at scale with Amazon OpenSearch Service
Loading article
Cloud Computing/Data Engineering/Observability
How Rakbank built centralized observability at scale with Amazon OpenSearch Service
AWS Architecture Blog·Yesterday·11 min read
By Identity and cross-account authorization flow
Open SourceKubernetesObservabilityGoogle CloudAzureSoftware ArchitectureLogging
AI summary
Rakbank implemented a tiered telemetry architecture on Amazon OpenSearch Service to handle 150 GB of daily logs from their EKS microservices. By leveraging an append-only data model, they utilized hot/warm/cold storage to achieve 55 percent cost savings while maintaining seven years of compliance-driven retention. The architecture uses Fluent Bit for dual-path ingestion, simultaneously routing logs to a searchable OpenSearch cluster and a raw S3 archive.
Key points
Implement hot/warm/cold storage tiers from the outset to avoid complex retrofitting as data volume grows.
Append-only log data allows for safe force-merging of shards before migration, eliminating update-related overhead.
Dual-write ingestion patterns to both a search cluster and a raw S3 bucket ensure both high performance and durable, cost-effective long-term retention.
Provision observability infrastructure with significant headroom to accommodate the growth of microservices-based telemetry.
Evaluate availability SLAs carefully; 99.99 percent availability requires doubling index replicas and reduces active search capacity by a third.