Quantcast
Channel: Consumer rebalance per topic
Browsing latest articles
Browse All 6 View Live

Consumer rebalance per topic

Hello We are starting to use Kafka in production but we found an unexpected (at least for me) behavior with the use of partitions. We have a bunch of topics with a few partitions each. We try to...

View Article


Consumer rebalance per topic

Pablo, Currently, partition is the smallest unit that we distribute data among consumers (in the same consumer group). So, if the # of consumers is larger than the total number of partitions in a Kafka...

View Article


Consumer rebalance per topic

Pablo, That is a good suggestion. Ideally, the partitions across all topics should be distributed evenly across consumer streams instead of a per-topic based decision. There is no particular advantage...

View Article

Consumer rebalance per topic

Thank you Jun and Neha I was trying to avoid adding more partitions. I have enough partitions if you count all partitions in all topics. I understand the problem with different data load per topic but...

View Article

Consumer rebalance per topic

Jira ticket https://issues.apache.org/jira/browse/KAFKA-687 2013/1/7 Pablo Barrera González < [email protected] > -- Pablo Barrera González

View Article


Consumer rebalance per topic

(From http://kafka.apache.org/design.html) one potential benefit of the existing rebalancing logic is to reduce the number of connections to brokers per consumer instance. However, if you have a large...

View Article
Browsing latest articles
Browse All 6 View Live