CLOSE_WAIT sockets on client disconnect

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

CLOSE_WAIT sockets on client disconnect

rpujar
Hello experts,

I am using a Northbound client to access the Opendaylight SDN
Controller (ODL) RESTful api which uses JAX-RS javax.ws.rs v2.0 and its
implementation Jersey API v1.17.
When a device connects, ODL exposes the REST services through port
8443. Thus, on startup the Northbound client tries to access the device
using RESTConf at port 8443.

I have many ~50 devices that come up. The socket count for port 8443
increases and when it reaches ~90 the devices are brought down. This
ends up in many 8443 sockets in CLOSE_WAIT state on the ODL (server)
side.

Is there any outstanding issue/bug for Jersey or javax.ws.rs closing
the connections that could possibly explain these CLOSE-WAIT sockets?

More context:
[1]: Resource class and method definition. The PUT REST service is
where I see the issue.
[2]: REST Service implementation for PUT request, where the Response
object is returned.

I would appreciate any leads that would help me with this issue. Please
let me know if you need me to add more information.

[1]
https://github.com/opendaylight/netconf/blob/ede6d9b71393a3058904feafc8
b7ba0dd691627a/restconf/sal-rest-connector/src/main/java/org/opendaylig
ht/netconf/sal/rest/api/RestconfService.java

[2]
https://github.com/opendaylight/netconf/blob/87082c4351ffa19202f4368f33
41974ca3bade1d/restconf/sal-rest-connector/src/main/java/org/opendaylig
ht/netconf/sal/restconf/impl/RestconfImpl.java#L651

Thanks,
~rashmi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CLOSE_WAIT sockets on client disconnect

Rallavagu
In general, CLOSE_WAIT issues arise from http client not closing the
connection. In your case, make sure you are actually closing connections
after accessing RESTful API. As you mentioned, what is the actual http
client used in Northbound client?

On 8/2/16 12:33 PM, [hidden email] wrote:

> Hello experts,
>
> I am using a Northbound client to access the Opendaylight SDN
> Controller (ODL) RESTful api which uses JAX-RS javax.ws.rs v2.0 and its
> implementation Jersey API v1.17.
> When a device connects, ODL exposes the REST services through port
> 8443. Thus, on startup the Northbound client tries to access the device
> using RESTConf at port 8443.
>
> I have many ~50 devices that come up. The socket count for port 8443
> increases and when it reaches ~90 the devices are brought down. This
> ends up in many 8443 sockets in CLOSE_WAIT state on the ODL (server)
> side.
>
> Is there any outstanding issue/bug for Jersey or javax.ws.rs closing
> the connections that could possibly explain these CLOSE-WAIT sockets?
>
> More context:
> [1]: Resource class and method definition. The PUT REST service is
> where I see the issue.
> [2]: REST Service implementation for PUT request, where the Response
> object is returned.
>
> I would appreciate any leads that would help me with this issue. Please
> let me know if you need me to add more information.
>
> [1]
> https://github.com/opendaylight/netconf/blob/ede6d9b71393a3058904feafc8
> b7ba0dd691627a/restconf/sal-rest-connector/src/main/java/org/opendaylig
> ht/netconf/sal/rest/api/RestconfService.java
>
> [2]
> https://github.com/opendaylight/netconf/blob/87082c4351ffa19202f4368f33
> 41974ca3bade1d/restconf/sal-rest-connector/src/main/java/org/opendaylig
> ht/netconf/sal/restconf/impl/RestconfImpl.java#L651
>
> Thanks,
> ~rashmi
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CLOSE_WAIT sockets on client disconnect

rpujar
In reply to this post by rpujar
Hi,
Doesn't CLOSE_WAIT state mean that the server already received a FIN
from the client and the server did not issue a close()? Could you
please clarify why you say: make sure you are actually closing
connections after accessing RESTful API.

Also, the client being used is: Javax.net.ssl.HttpsURLConnection

Thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CLOSE_WAIT sockets on client disconnect

John MacAuley
Rashmi,

I believe HttpsURLConnection will use connection pooling if you just call close().  You need to explicitly call disconnect() for the socket to be shutdown.

I have run into CLOSE_WAIT issues before and it was typically caused by error conditions that dod not result in me properly cleaning up the connection.

John

On 2016-08-02, at 5:25 PM, [hidden email] wrote:

> Hi,
> Doesn't CLOSE_WAIT state mean that the server already received a FIN
> from the client and the server did not issue a close()? Could you
> please clarify why you say: make sure you are actually closing
> connections after accessing RESTful API.
>
> Also, the client being used is: Javax.net.ssl.HttpsURLConnection
>
> Thanks

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CLOSE_WAIT sockets on client disconnect

rpujar
In reply to this post by rpujar
Hi John, thank you for your insight.
The client indeed uses disconnect() to close the socket. The client
also has a 15sec timeout. So if the client is able to access the port
but does not receive any response within 15s it disconnects.
Would there be any other possibility that I could verify to make sure I
avoid running into such file descriptor leak issues?
Thank you..
Loading...