Max threads needs set for systemd

[qgriffith-zz]

Getting an error when starting elasticsearch

max number of threads [1813] for user [elasticsearch] is too low

Doing some research I increased this in /etc/security/limits. However it appears systemd ignores that and LimitNPROC=2048 needs to be set in the systemd service script.

I followed this http://unix.stackexchange.com/questions/324504/how-to-raise-max-threads-for-elasticsearch and got ES started. I will try to get a pull request together to correct this soon.

[martinb3]

Hi there @qgriffith -- is this an Amazon Linux issue? From what I can tell in elastic/elasticsearch#21796, one of the Amazon Linux AMIs had a bug where it wasn't respecting limits but should be now. Could you confirm if you're on the same platform, and if so, if upgrading fixes it?

[qgriffith-zz]

I ran into the issue on Ubuntu 16, but I assume it will be a issue for any linux that uses systemd. I don't think Amazon LInux uses systemd though so it shouldn't be the same issue.

[jeusdi]

I'm getting this error message on a centos/7 ghest. How could I fix this value?

[martinb3]

@jeusdi the initial comment says:

Doing some research I increased this in /etc/security/limits. However it appears systemd ignores that and LimitNPROC=2048 needs to be set in the systemd service script.

That's the potential workaround.

[jeusdi]

I know that perhaps it's not related with elasticsearch cookbook.

Please, let me ask yor for how to get this workaround using chef.

Do I need to use another cookbook?

[jeusdi]

So, I mean is it podsible to change elasticsearch.service in order to add this line using chef? I'd like to have not to do anything manually...

I've take a look on your scripts ans I've seen you are using ulimit...

[martinb3]

@jeusdi We're shipping the files from Elastic's project directly and trying to avoid making fixes here when they could be fixed upstream. I know that means we're keeping consumers from getting fixes as quickly -- I'm sorry about that!

You can override the systemd file provided by this cookbook / provided by Elasticsearch (we're using the upstream files), or use one of the line editing cookbooks to make this edit in the existing file. I believe systemd also has an override mechanism for settings, but I don't know the details offhand.

[tylerjl]

Hey @martinb3, I wanted to give you a heads-up that I raised this internally and we do think we should be controlling the NPROC limit within the upstream systemd unit (see the related elasticsearch repo PR), so making this downstream change in chef/puppet/etc. should be a consistent change with the stock unit that the package is shipping. 👍

[martinb3]

Thanks @tylerjl! I'll keep an eye on elastic/elasticsearch#24970.

[kylegoch]

@martinb3 Following up on this. So it looks like elastic/elasticsearch#24970 was merged, but not released? Or has it been released? I noticed the puppet repo has a similar change rolled in already via voxpupuli/puppet-elasticsearch#837. Do we need to do the same here?

[martinb3]

Hi @kylegoch -- it hasn't been released just yet; you can check out this FAQ question we've recently added for what is probably the best way to override this setting right now: https://github.com/elastic/cookbook-elasticsearch/blob/master/FAQ.md#how-do-i-override-settings-in-the-systemd-unit-file

[kylegoch]

Thanks! Ill keep an eye out then and do one the suggestions from the FAQ.

And correct me if I am wrong, this is taking care of this correct (still new to systemd)?
https://www.elastic.co/guide/en/elasticsearch/reference/current/max-number-of-threads.html