consumer - Kafka rebalance minimisation -
in simple scenario have: 1 kafka broker n topics no topic partitioning
i want , example, separate topics between consumers.
i load k consumers(streams), each stream process n/k topics.
- i can use same consumer group
- i can load each consumer in own group
can confirm, rebalance in 2-d case not happen if new consumer added? good, because avoid rebalance.
what hidden disadvantages of both approaches?
rebalance shouldnt happen when adding consumer different group.
when using same consumer group, if 1 consumer dies, others take topics/partitions , continue processing offset died. if use different consumer groups you'll have restart consumer right place yourself.
Comments
Post a Comment