Ask HN: How to scale long-lived connections with Kubernetes?

7 points by k888sss 12 days ago

My application serves users via long lived TCP connections and users are quite volatile. They come and go without any obvious patterns. And my current approach of scaling based on CPU/memory usage are too damn barbaric. I should not break active connections, but I also need to control the cost. On top my head, I can think of writing a custom middleware to manage connections and pods. Are there any ready-to-use solutions or approaches to tackle similar problems? Thanks

wmf 12 days ago

This kind of issue comes up in game development where I think the standard answer is a tier of proxies that can hold open the connection to the clients while reconnecting to different backends.

  • k888sss 12 days ago

    Unfortunately, client code is not upgradable, and I have to solve this issue from server side only.

    • neonlex 11 days ago

      I don’t understand your answer, that’s basically what wmf described. Put a proxy or load balancer in front of your service.

      • k888sss 11 days ago

        Thanks. I can't upgrade clients to add reconnect functionality, and I can only work on the server side to solve the scaling issue. I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

        • dominotw 9 days ago

          > I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

          thats exactly what the original suggestion seemed to be was.

          • k888sss 9 days ago

            Yupp. I think I misunderstood at first. Now I get it. Thanks all.