This partially addresses the roadmap item
LSM: Runtime-configurable NodePool size, with a sane default. (Currently it is constant and too small.)
It adds a CLI flag, --lsm-manifest-memory, that says how many bytes to allocate for the node pool. It does not address the matter of fixing the default size of the node pool. If this PR is merged I will update the roadmap to split these items in two and indicate one is not complete.
I'm open to other names for this flag. I chose this one since it seems relatively understandable, though it requires understanding the concept of a "manifest". This argument is similar in function to --grid-cache though, and the name of the flag doesn't follow the same format, e.g. it could be called --lsm-manifest.
I am unsure about the help text
"When the number or size of LSM-trees would become too large for their
manifests to fit into memory, such requests are rejected."
In particular: "number or size" - is this true? Additionally, from comments I think it is not the case that such requests are currently rejected, but that is the intent.
The value of this flag must be a large multiple of 16KB (the default is 64MB), which is perhaps not the easiest for users to reason about without help, and the error text currently displays this 16KB multiplier in bytes, which reads poorly.
The minimum value of this flag is equal to one node, which in practice would seemingly be a poor choice. The maximum value of this flag is the largest multiple of the node size which won't cause overflows elsewhere (slightly less than maxInt(u32)).
To match the grid cache, I have added this value to the initialization log sequence, splitting the current line about allocations into two, as one line was looking long to me on my console at 107 characters.