Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIP-120: Enable client memory limit by default #13306

Closed
merlimat opened this issue Dec 14, 2021 · 0 comments · Fixed by #13344
Closed

PIP-120: Enable client memory limit by default #13306

merlimat opened this issue Dec 14, 2021 · 0 comments · Fixed by #13344
Assignees
Labels
Milestone

Comments

@merlimat
Copy link
Contributor

Motivation

In Pulsar 2.8, we have introduced a setting to control the amount of memory
used by a client instance.

interface ClientBuilder {
    ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit);
}

By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit is being
enforced.

I think it's a good time for 2.10 to enable this setting by default and,
correspondingly, to disable by default the producer queue size limit.

This will simplify a lot the configuration that a producer application will
have to come up with, when publishing with many topic/partitions or when messages
are bigger than expected.

Proposed changes

In 2.10 release, for the ClientBuilder, change
  * memoryLimit: 0 -> 64 MB

For the ProducerBuilder, changes
  * maxPendingMessages: 1000 -> 0

64MB is picked because it's a small enough memory size that will guarantee
a very high producer throughput, irrespective of the individual messages size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 participant