Image for post
Image for post

This is the second part of High scale web server with Kubernetes. We will go over Kubernetes Horizontal Pod Autoscale and how we are using it at Dynamic Yield.

Overview

While serving a huge amount of requests, we can easily observe that our traffic graph looks like a sine wave with a high rate at midday and a lower rate at night. The difference is relatively big, around 2–3 times more requests in the rush hours. Moreover, there are special occasions such as Black Friday, Cyber Monday, sale campaigns, etc, that our traffic can raise up to x3.

Using Kubernetes’s elasticity…


Image for post
Image for post

Here at Dynamic Yield we are serving thousands of HTTP requests per second. Moving our serving services from EC2 to EKS required some tuning to ensure we could serve efficiently without losing any request. I summarized some tips and I hope it might help others with a smooth transition.

Overview

Kubernetes can decide to proactively replace pods (replicas) of our application, whenever:

  • Kubernetes node is replaced (if the node is spot instance). If you have running pods on that node — they will be terminated and new pods will start in another node instead.
  • Kubernetes nodes scale-in. Pods in this node…

Image for post
Image for post

Celery is a simple and reliable distributed system to process tasks. At Dynamic Yield, we are using Celery to distribute work across pods in Kubernetes.

Celery communicates via messages, usually using a broker to mediate between clients and workers. We are using RabbitMQ as the broker.

Our system we have ~30 different celery tasks such as:

  • Reading data from Elasticsearch.
  • Indexing data to Elasticsearch.
  • Reading data from S3.
  • Writing data to Redis.
  • etc

There’s a huge variety between those tasks: some of them can run for seconds while others can take hours, depends on the data (size) being processed and…


Image for post
Image for post

Elasticsearch is being used in a lot of companies as a great search-engine thanks to its speed and scale. At Dynamic Yield, we are using Elasticsearch as part of our recommendations engine and handle thousands of requests per second.

Terminology

Elasticsearch cluster contains several nodes that can play one or more roles: master, data, ingestion, etc.

Indices are being stored in the data nodes.

Each index can have one or more shards, usually determined by the index size.

Each shard holds part of the index data (documents), so if for example, we have an index with two shards, each shard holds…

Itay Bittan

Principal Engineer at Dynamic Yield

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store