Disable OOM killer for process?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Disable OOM killer for process?

Casey Jordan
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Dannes Wessels-2
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Casey Jordan
Hi Dannes,

Thanks for the input but I have already explored options not using overcommit, or turning off the OOM killer entirely and from what I can tell the best thing for me to do is tell the OOM killer not to kill eXist.

So before I evaluate other options I really need to know how to add this functionality to the eXist startup.

Thanks


On Sun, Sep 27, 2015 at 1:58 PM Dannes Wessels <[hidden email]> wrote:
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Leif Mortenson-3
Casey,
This would need to be done after the JVM is launched.
You can get the PID from within the JVM with WrapperManager.getJavaPID() or by using the wrapper.java.pidfile property.

But the easiest way to do this is to use the "jvm_start" event in the Professional Edition to launch the required external command at the appropriate time.
This can be setup as follows:
---
wrapper.event.jvm_start.command.argv.1=/usr/bin/myscript
wrapper.event.jvm_start.command.argv.2=%WRAPPER_EVENT_JVM_PID%
---
The following will help with debugging and understanding the timing of various events:
---
wrapper.event.default.loglevel=INFO
wrapper.event.jvm_start.command.loglevel=INFO
---
Your "myscript" script would then handle the pid argument and run your command.

See the following for details:

Cheers,
Leif





On Mon, Sep 28, 2015 at 3:10 AM, Casey Jordan <[hidden email]> wrote:
Hi Dannes,

Thanks for the input but I have already explored options not using overcommit, or turning off the OOM killer entirely and from what I can tell the best thing for me to do is tell the OOM killer not to kill eXist.

So before I evaluate other options I really need to know how to add this functionality to the eXist startup.

Thanks


On Sun, Sep 27, 2015 at 1:58 PM Dannes Wessels <[hidden email]> wrote:
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
Leif Mortenson
President & CEO
Tanuki Software, Ltd.
6-18-10-4F Nishi-Kasai, Edogawa-ku
Tokyo 134-0088 Japan
Tel: +81-3-3878-3211
Fax: +81-3-3878-0313
モートンソン リーフ
代表取締役
タヌキソフトウェア有限会社
〒134-0088 東京都江戸川区西葛西6-18-10 SKセントラルビル4階
Tel: 03-3878-3211
Fax: 03-3878-0313
http://www.tanukisoftware.com
[hidden email]

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Casey Jordan
Thanks Leif,

So I the main thing I need to ensure is that this gets run as root, since the command I need to run will only be effective as root. It seems to me that the first method won't allow me to do that, since I will already be inside the JVM executing that code right?

Will the event script method allow me to do this?

Thanks!



On Sun, Sep 27, 2015 at 9:11 PM Leif Mortenson <[hidden email]> wrote:
Casey,
This would need to be done after the JVM is launched.
You can get the PID from within the JVM with WrapperManager.getJavaPID() or by using the wrapper.java.pidfile property.

But the easiest way to do this is to use the "jvm_start" event in the Professional Edition to launch the required external command at the appropriate time.
This can be setup as follows:
---
wrapper.event.jvm_start.command.argv.1=/usr/bin/myscript
wrapper.event.jvm_start.command.argv.2=%WRAPPER_EVENT_JVM_PID%
---
The following will help with debugging and understanding the timing of various events:
---
wrapper.event.default.loglevel=INFO
wrapper.event.jvm_start.command.loglevel=INFO
---
Your "myscript" script would then handle the pid argument and run your command.

See the following for details:

Cheers,
Leif





On Mon, Sep 28, 2015 at 3:10 AM, Casey Jordan <[hidden email]> wrote:
Hi Dannes,

Thanks for the input but I have already explored options not using overcommit, or turning off the OOM killer entirely and from what I can tell the best thing for me to do is tell the OOM killer not to kill eXist.

So before I evaluate other options I really need to know how to add this functionality to the eXist startup.

Thanks


On Sun, Sep 27, 2015 at 1:58 PM Dannes Wessels <[hidden email]> wrote:
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
Leif Mortenson
President & CEO
Tanuki Software, Ltd.
6-18-10-4F Nishi-Kasai, Edogawa-ku
Tokyo 134-0088 Japan
Tel: +81-3-3878-3211
Fax: +81-3-3878-0313
モートンソン リーフ
代表取締役
タヌキソフトウェア有限会社
〒134-0088 東京都江戸川区西葛西6-18-10 SKセントラルビル4階
Tel: 03-3878-3211
Fax: 03-3878-0313
http://www.tanukisoftware.com
[hidden email]
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Leif Mortenson-3
Casey,
No.  The Wrapper will also run the event script as the user the Wrapper is running as.

If it needs to be run by an elevated process, then one idea would be to have another cron job running as root look for the pid file written by the wrapper.java.pidfile property and then take action based on that.  You would need to write out another state file or something so you know when you already did it.

Cheers,
Leif


On Mon, Sep 28, 2015 at 12:20 PM, Casey Jordan <[hidden email]> wrote:
Thanks Leif,

So I the main thing I need to ensure is that this gets run as root, since the command I need to run will only be effective as root. It seems to me that the first method won't allow me to do that, since I will already be inside the JVM executing that code right?

Will the event script method allow me to do this?

Thanks!



On Sun, Sep 27, 2015 at 9:11 PM Leif Mortenson <[hidden email]> wrote:
Casey,
This would need to be done after the JVM is launched.
You can get the PID from within the JVM with WrapperManager.getJavaPID() or by using the wrapper.java.pidfile property.

But the easiest way to do this is to use the "jvm_start" event in the Professional Edition to launch the required external command at the appropriate time.
This can be setup as follows:
---
wrapper.event.jvm_start.command.argv.1=/usr/bin/myscript
wrapper.event.jvm_start.command.argv.2=%WRAPPER_EVENT_JVM_PID%
---
The following will help with debugging and understanding the timing of various events:
---
wrapper.event.default.loglevel=INFO
wrapper.event.jvm_start.command.loglevel=INFO
---
Your "myscript" script would then handle the pid argument and run your command.

See the following for details:

Cheers,
Leif





On Mon, Sep 28, 2015 at 3:10 AM, Casey Jordan <[hidden email]> wrote:
Hi Dannes,

Thanks for the input but I have already explored options not using overcommit, or turning off the OOM killer entirely and from what I can tell the best thing for me to do is tell the OOM killer not to kill eXist.

So before I evaluate other options I really need to know how to add this functionality to the eXist startup.

Thanks


On Sun, Sep 27, 2015 at 1:58 PM Dannes Wessels <[hidden email]> wrote:
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!


------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user
Reply | Threaded
Open this post in threaded view
|

Re: Disable OOM killer for process?

Adam Retter
In reply to this post by Casey Jordan

You could setup sudoers to allow you to exec just that as root or even use setuid on your app that makes the changes

On 28 Sep 2015 04:20, "Casey Jordan" <[hidden email]> wrote:
Thanks Leif,

So I the main thing I need to ensure is that this gets run as root, since the command I need to run will only be effective as root. It seems to me that the first method won't allow me to do that, since I will already be inside the JVM executing that code right?

Will the event script method allow me to do this?

Thanks!



On Sun, Sep 27, 2015 at 9:11 PM Leif Mortenson <[hidden email]> wrote:
Casey,
This would need to be done after the JVM is launched.
You can get the PID from within the JVM with WrapperManager.getJavaPID() or by using the wrapper.java.pidfile property.

But the easiest way to do this is to use the "jvm_start" event in the Professional Edition to launch the required external command at the appropriate time.
This can be setup as follows:
---
wrapper.event.jvm_start.command.argv.1=/usr/bin/myscript
wrapper.event.jvm_start.command.argv.2=%WRAPPER_EVENT_JVM_PID%
---
The following will help with debugging and understanding the timing of various events:
---
wrapper.event.default.loglevel=INFO
wrapper.event.jvm_start.command.loglevel=INFO
---
Your "myscript" script would then handle the pid argument and run your command.

See the following for details:

Cheers,
Leif





On Mon, Sep 28, 2015 at 3:10 AM, Casey Jordan <[hidden email]> wrote:
Hi Dannes,

Thanks for the input but I have already explored options not using overcommit, or turning off the OOM killer entirely and from what I can tell the best thing for me to do is tell the OOM killer not to kill eXist.

So before I evaluate other options I really need to know how to add this functionality to the eXist startup.

Thanks


On Sun, Sep 27, 2015 at 1:58 PM Dannes Wessels <[hidden email]> wrote:
Hi,

over committing memory sounds like a bad thing to do? You should make sure anyway that the JVM will not be swapped out due to memory restrictions.

regards

Dannes


On Sun, Sep 27, 2015 at 12:32 AM, Casey Jordan <[hidden email]> wrote:
Hi all,

We are having an issue where the out of memory killer is killing out process due to the out of the box overcommit memory settings on CentOS.

After researching this quite a bit, I think the best thing to do is to tell the OOM killer to never kill the process, as described here: http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

To do this, we need to execute the following command after startup:

echo -17 > /proc/{pid}/oom_adj

I was thinking this could be done as part of the *.sh script that gets executed via the java service wrapper, but I am not sure exactly how to do this (mainly where in this script I can put the valid logic). I am hoping someone might have done this before that could help.

Thanks!

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user




--
Leif Mortenson
President & CEO
Tanuki Software, Ltd.
6-18-10-4F Nishi-Kasai, Edogawa-ku
Tokyo 134-0088 Japan
Tel: <a href="tel:%2B81-3-3878-3211" value="+81338783211" target="_blank">+81-3-3878-3211
Fax: <a href="tel:%2B81-3-3878-0313" value="+81338780313" target="_blank">+81-3-3878-0313
モートンソン リーフ
代表取締役
タヌキソフトウェア有限会社
〒134-0088 東京都江戸川区西葛西6-18-10 SKセントラルビル4階
Tel: 03-3878-3211
Fax: 03-3878-0313
http://www.tanukisoftware.com
[hidden email]
------------------------------------------------------------------------------
_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user

------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user


------------------------------------------------------------------------------

_______________________________________________
Wrapper-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/wrapper-user