PermGen Memory Leak in Tomcat

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

PermGen Memory Leak in Tomcat

DaveCrystal
Hello, I am using Jersey 1.9.1 with spring to create a rest web service hosted in Tomcat.

The application follows the pattern in this example. http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring 

When the web application is undeployed Tomcat logs a warning that the application has created a ThreadLocal and not removed it:

SEVERE: The web application [/sandbox] created a ThreadLocal with key of type [null] (value [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4]) and a value of type [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.

After deploying and undeploying the app several times Tomcat runs out of PermGen space.

Any advice on how I can cause Jersey to clean up the Thread Locals on undeploy would be much appreceiated.

Thanks a lot.
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

Petr Jurák
Hi,

IMHO it's not an issue of Jersey but Tomcat. We had many problems like
that (after few redeployments we need to restart Tomcat because of
PermGenSpace) in production though we don't use Jersey in these
production applications. We "solved" it with sheduled restarts in
regular service breaks. :(
Please try to search Tomcat forum for suggestion.

Br, Petr

2011/9/29 DaveCrystal <[hidden email]>:

> Hello, I am using Jersey 1.9.1 with spring to create a rest web service
> hosted in Tomcat.
>
> The application follows the pattern in this example.
> http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring
>
> When the web application is undeployed Tomcat logs a warning that the
> application has created a ThreadLocal and not removed it:
>
> SEVERE: The web application [/sandbox] created a ThreadLocal with key of
> type [null] (value
> [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4])
> and a value of type
> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value
> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983]) but
> failed to remove it when the web application was stopped. This is very
> likely to create a memory leak.
>
> After deploying and undeploying the app several times Tomcat runs out of
> PermGen space.
>
> Any advice on how I can cause Jersey to clean up the Thread Locals on
> undeploy would be much appreceiated.
>
> Thanks a lot.
>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/PermGen-Memory-Leak-in-Tomcat-tp6843479p6843479.html
> Sent from the Jersey mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

Philippe Marschall
On 09/29/2011 07:22 PM, Petr Jurák wrote:
> Hi,
>
> IMHO it's not an issue of Jersey but Tomcat. We had many problems like
> that (after few redeployments we need to restart Tomcat because of
> PermGenSpace) in production though we don't use Jersey in these
> production applications. We "solved" it with sheduled restarts in
> regular service breaks. :(
> Please try to search Tomcat forum for suggestion.

What makes you sure it's a Tomcat issue? Libraries not cleaning up
thread locals usually is a library issue, not a Tomcat issue.

You can try running the latest Tomcat with
ThreadLocalLeakPreventionListener [1].

 [1]
http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#ThreadLocal_Leak_Prevention_Listener_%28org.apache.catalina.core.ThreadLocalLeakPreventionListener%29

Cheers
Philippe

Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

Petr Jurák
First of all I'm not sure, just guessing. And thanks for tip.
In our case we use Tomcat 6. The problem was related to redeploy war
in this container wich lead to memory leak. I didn't spend such long
time to solve this. Basically something locks web.xml and remains in
memory.

Br, Petr



2011/9/30 Philippe Marschall <[hidden email]>:

> On 09/29/2011 07:22 PM, Petr Jurák wrote:
>> Hi,
>>
>> IMHO it's not an issue of Jersey but Tomcat. We had many problems like
>> that (after few redeployments we need to restart Tomcat because of
>> PermGenSpace) in production though we don't use Jersey in these
>> production applications. We "solved" it with sheduled restarts in
>> regular service breaks. :(
>> Please try to search Tomcat forum for suggestion.
>
> What makes you sure it's a Tomcat issue? Libraries not cleaning up
> thread locals usually is a library issue, not a Tomcat issue.
>
> You can try running the latest Tomcat with
> ThreadLocalLeakPreventionListener [1].
>
>  [1]
> http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#ThreadLocal_Leak_Prevention_Listener_%28org.apache.catalina.core.ThreadLocalLeakPreventionListener%29
>
> Cheers
> Philippe
>
>
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

japod
In reply to this post by DaveCrystal
Hi Stephen,

I am afraid Jersey does not remove it's thread locals
and there is currently no way to force Jersey to clean up properly.
We need to fix that.

Could you please file a bug report [1] to track this?

Thanks,

~Jakub

[1]http://java.net/jira/browse/JERSEY/

On 29.9.2011 12:56, DaveCrystal wrote:

> Hello, I am using Jersey 1.9.1 with spring to create a rest web service
> hosted in Tomcat.
>
> The application follows the pattern in this example.
> http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring
>
> When the web application is undeployed Tomcat logs a warning that the
> application has created a ThreadLocal and not removed it:
>
> SEVERE: The web application [/sandbox] created a ThreadLocal with key of
> type [null] (value
> [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4])
> and a value of type
> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value
> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983]) but
> failed to remove it when the web application was stopped. This is very
> likely to create a memory leak.
>
> After deploying and undeploying the app several times Tomcat runs out of
> PermGen space.
>
> Any advice on how I can cause Jersey to clean up the Thread Locals on
> undeploy would be much appreceiated.
>
> Thanks a lot.
>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/PermGen-Memory-Leak-in-Tomcat-tp6843479p6843479.html
> Sent from the Jersey mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

japod
Looks like Tomcat >= 7.0.6 fixes the issue (see [1]).

Any chance you can update your Tomcat?

~Jakub

[1]http://wiki.apache.org/tomcat/MemoryLeakProtection#customThreadLocal

On 11.10.2011 16:59, Jakub Podlesak wrote:

> Hi Stephen,
>
> I am afraid Jersey does not remove it's thread locals
> and there is currently no way to force Jersey to clean up properly.
> We need to fix that.
>
> Could you please file a bug report [1] to track this?
>
> Thanks,
>
> ~Jakub
>
> [1]http://java.net/jira/browse/JERSEY/
>
> On 29.9.2011 12:56, DaveCrystal wrote:
>> Hello, I am using Jersey 1.9.1 with spring to create a rest web service
>> hosted in Tomcat.
>>
>> The application follows the pattern in this example.
>> http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring
>>
>> When the web application is undeployed Tomcat logs a warning that the
>> application has created a ThreadLocal and not removed it:
>>
>> SEVERE: The web application [/sandbox] created a ThreadLocal with key of
>> type [null] (value
>> [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4])
>>
>> and a value of type
>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value
>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983]) but
>>
>> failed to remove it when the web application was stopped. This is very
>> likely to create a memory leak.
>>
>> After deploying and undeploying the app several times Tomcat runs out of
>> PermGen space.
>>
>> Any advice on how I can cause Jersey to clean up the Thread Locals on
>> undeploy would be much appreceiated.
>>
>> Thanks a lot.
>>
>> --
>> View this message in context:
>> http://jersey.576304.n2.nabble.com/PermGen-Memory-Leak-in-Tomcat-tp6843479p6843479.html
>> Sent from the Jersey mailing list archive at Nabble.com.
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

Holger Hoffstaette
On Thu, 13 Oct 2011 15:18:07 +0200, Jakub Podlesak wrote:

> Looks like Tomcat >= 7.0.6 fixes the issue (see [1]).

No, it *tries to work around broken software*. That's different.

> Any chance you can update your Tomcat?

How does this "fix" the actual problem when Jersey will still leak
everyhwere else?

-h


Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

John G. Lussmyer
In reply to this post by japod
Just how much will this matter to most jersey users?
How often is an undeploy normally done?
(I'm trying to figure out if this will impact my products noticeably.)

On 10/11/2011 7:59 AM, Jakub Podlesak wrote:

> Hi Stephen,
>
> I am afraid Jersey does not remove it's thread locals
> and there is currently no way to force Jersey to clean up properly.
> We need to fix that.
>
> Could you please file a bug report [1] to track this?
>
> Thanks,
>
> ~Jakub
>
> [1]http://java.net/jira/browse/JERSEY/
>
> On 29.9.2011 12:56, DaveCrystal wrote:
>> Hello, I am using Jersey 1.9.1 with spring to create a rest web service
>> hosted in Tomcat.
>>
>> The application follows the pattern in this example.
>> http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring
>>
>> When the web application is undeployed Tomcat logs a warning that the
>> application has created a ThreadLocal and not removed it:
>>
>> SEVERE: The web application [/sandbox] created a ThreadLocal with key of
>> type [null] (value
>> [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4])
>>
>> and a value of type
>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value
>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983]) but
>>
>> failed to remove it when the web application was stopped. This is very
>> likely to create a memory leak.
>>
>> After deploying and undeploying the app several times Tomcat runs out of
>> PermGen space.
>>
>> Any advice on how I can cause Jersey to clean up the Thread Locals on
>> undeploy would be much appreceiated.
>>
>> Thanks a lot.
>>
>> --
>> View this message in context:
>> http://jersey.576304.n2.nabble.com/PermGen-Memory-Leak-in-Tomcat-tp6843479p6843479.html
>> Sent from the Jersey mailing list archive at Nabble.com.
>>
>


--
--
John G. Lussmyer  mailto:[hidden email]
Electric Vehicle Battery Monitoring Systems, http://www.CasaDelGato.com


Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

japod
In reply to this post by Holger Hoffstaette
Hi Holger,

On 15.10.2011 15:25, Holger Hoffstaette wrote:
> On Thu, 13 Oct 2011 15:18:07 +0200, Jakub Podlesak wrote:
>
>> Looks like Tomcat>= 7.0.6 fixes the issue (see [1]).
> No, it *tries to work around broken software*. That's different.

I just tried to suggest a temporary fix for the original problem
reporter and whoever else is experiencing
similar issue. And in the given scenario the upgrade actually works as a
fix, as Tomcat will actually
clean up the thread locals when the application has been redeployed.


>> Any chance you can update your Tomcat?
> How does this "fix" the actual problem when Jersey will still leak
> everyhwere else?

See above, this is only applicable as a temporary fix for Tomcat users.

~Jakub

> -h
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: PermGen Memory Leak in Tomcat

japod
In reply to this post by John G. Lussmyer
Hi John,

It depends on what container you use, what message body readers/writers
and how many re-deployments you do. The original reporter run out of
PermGen space,
which is of course a serious issue, but this should only happen
if you rely on particular XML related stuff and redeploy feature.
Thread local variables used by Jersey in the request scope are being
cleaned up properly,
so if you just deploy once and keep running, the issue should not have a
great impact on you.
It also should not impact you at all when running on Grizzly and other
low level embedded
containers.

HTH,

~Jakub



On 15.10.2011 17:34, John G. Lussmyer wrote:

> Just how much will this matter to most jersey users?
> How often is an undeploy normally done?
> (I'm trying to figure out if this will impact my products noticeably.)
>
> On 10/11/2011 7:59 AM, Jakub Podlesak wrote:
>> Hi Stephen,
>>
>> I am afraid Jersey does not remove it's thread locals
>> and there is currently no way to force Jersey to clean up properly.
>> We need to fix that.
>>
>> Could you please file a bug report [1] to track this?
>>
>> Thanks,
>>
>> ~Jakub
>>
>> [1]http://java.net/jira/browse/JERSEY/
>>
>> On 29.9.2011 12:56, DaveCrystal wrote:
>>> Hello, I am using Jersey 1.9.1 with spring to create a rest web service
>>> hosted in Tomcat.
>>>
>>> The application follows the pattern in this example.
>>> http://blogs.oracle.com/enterprisetechtips/entry/jersey_and_spring
>>>
>>> When the web application is undeployed Tomcat logs a warning that the
>>> application has created a ThreadLocal and not removed it:
>>>
>>> SEVERE: The web application [/sandbox] created a ThreadLocal with
>>> key of
>>> type [null] (value
>>> [com.sun.jersey.core.impl.provider.xml.ThreadLocalSingletonContextProvider$1@15ce9c4])
>>>
>>> and a value of type
>>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl] (value
>>> [com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl@f76983])
>>> but
>>> failed to remove it when the web application was stopped. This is very
>>> likely to create a memory leak.
>>>
>>> After deploying and undeploying the app several times Tomcat runs
>>> out of
>>> PermGen space.
>>>
>>> Any advice on how I can cause Jersey to clean up the Thread Locals on
>>> undeploy would be much appreceiated.
>>>
>>> Thanks a lot.
>>>
>>> --
>>> View this message in context:
>>> http://jersey.576304.n2.nabble.com/PermGen-Memory-Leak-in-Tomcat-tp6843479p6843479.html
>>> Sent from the Jersey mailing list archive at Nabble.com.
>>>
>>
>
>