Autoscaling Elasticsearch for Logs on Kubernetes - Radu Gheorghe & Ciprian Hacman скачать в хорошем качестве

Autoscaling Elasticsearch for Logs on Kubernetes - Radu Gheorghe & Ciprian Hacman

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects. Autoscaling Elasticsearch for Logs on Kubernetes - Radu Gheorghe, Sematext Group & Ciprian Hacman, polypoly Elasticsearch (and its fork, OpenSearch) is the go-to storage for logs. As with any storage, the cluster likely needs to scale to keep up with the change of load. But autoscaling Elasticsearch isn't trivial: indices and shards need to be well sized and well balanced across nodes. Otherwise the cluster will have hotspots and scaling it further will be less and less efficient. This talk focuses on two aspects: - best practices around scaling Elasticsearch for logs and other time-series data - how to apply them when deploying Elasticsearch on Kubernetes. In the process, a new (open-source) operator will be introduced (yes, there will be a demo!). This operator will autoscale Elasticsearch while keeping a good balance of load. It does so by changing the number of shards in the index template and rotating indices when the number of nodes changes.