Finalizer slowed by resteasy BaseClientResponse.finalize

Environment

• JBoss Enterprise Application Platform (EAP) 7.x
• RestEasy

Issue

• Our heap becomes heavily consumed by Finalizers to the point of OOME unless we force GCs earlier and often to release them. At the issue time, the Finalizer thread is shown spending long amounts of time in a socket read from org.jboss.resteasy.client.core.BaseClientResponse.finalize:

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00000000028c6000 nid=0x44ed runnable [0x00007fcc7c4c7000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:593)
	at sun.security.ssl.InputRecord.read(InputRecord.java:532)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
	- locked <0x00000005f678bd00> (a java.lang.Object)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	- locked <0x00000005f678c5d0> (a sun.security.ssl.AppInputStream)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:198)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:200)
	at org.apache.http.impl.io.ContentLengthInputStream.close(ContentLengthInputStream.java:103)
	at org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:168)
	at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
	at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:174)
	at java.io.BufferedInputStream.close(BufferedInputStream.java:483)
	at org.jboss.resteasy.client.core.executors.ApacheHttpClient4Executor$1.performReleaseConnection(ApacheHttpClient4Executor.java:269)
	at org.jboss.resteasy.client.core.BaseClientResponse.releaseConnection(BaseClientResponse.java:607)
	at org.jboss.resteasy.client.core.BaseClientResponse.finalize(BaseClientResponse.java:616)
	- locked <0x00000005f678c690> (a org.jboss.resteasy.client.core.BaseClientResponse)
	at java.lang.System$2.invokeFinalize(System.java:1270)
	at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:102)
	at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:217)

Resolution

• It seems that somewhere you are creating clients and/or responses and not closing them. You should not be relying solely on finalizers.
  According to Content from docs.jboss.org is not included.docs, ApacheHttpClient4Engine.finalize() will close any open sockets, but only if it created the HttpClient it has been using. If an HttpClient has been passed into the ApacheHttpClient4Executor, then the user is responsible for closing the connections:

HttpClient httpClient = new DefaultHttpClient();
ApacheHttpClient4Engine executor = new ApacheHttpClient4Engine(httpClient);
...
httpClient.getConnectionManager().shutdown();

• Please note that if ApacheHttpClient4Engine has created its own instance of HttpClient, it is not necessary to wait for finalize() to close open sockets. The ClientHttpEngine interface has a close() method for this purpose. Finally, if you are using JAX-RS 2.0 syntax the javax.ws.rs.client.Client class has created the engine automatically for you, you should call Client.close() and this will clean up any socket connections. If you are using JAX-RS 1.0 syntax you are likely using ClientResponse and should call ClientResponse.close() when you are done.

Root Cause

• RestEasy clients are not being closed by the app, resulting in excessive finalizer work.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.