Jersey 2 Client that would write the stream of data to POST call chunk by chunk

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Jersey 2 Client that would write the stream of data to POST call chunk by chunk

Naseer Sargi

I wanted to write a Jersey 2 Client that would write the stream of data to POST call chunk by chunk.

Why? , this will help me to avoid to keep the whole inputstream request data to store in disk memory before sending via a POST call.

I have searched over the net and looked into Jersey2 API as well but did not find any solution , however there are solution for server side which sends the huge response in stream and reads the same in Jersey Client by doing a GET call at Client , but I wanted send the huge payload say 1 GB of XML data as stream vis POST call.

I tried using solution given in here , but this solution again uses the system memory.

I do not want to store 1GB of data in disk , instead create on-fly the 1GB request stream of data / write the 1GB data to POST call directly chunk by chunk.

Any help highly appreciated. Thanks in Advance.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jersey 2 Client that would write the stream of data to POST call chunk by chunk

Pavel Bucek-2

Hi Naseer,

have you seen this test?

https://github.com/jersey/jersey/blob/2.x/tests/e2e/src/test/java/org/glassfish/jersey/tests/e2e/client/ClientBufferingDisabledTest.java

Regards,
Pavel


On 15/08/16 12:53, Naseer Sargi wrote:

I wanted to write a Jersey 2 Client that would write the stream of data to POST call chunk by chunk.

Why? , this will help me to avoid to keep the whole inputstream request data to store in disk memory before sending via a POST call.

I have searched over the net and looked into Jersey2 API as well but did not find any solution , however there are solution for server side which sends the huge response in stream and reads the same in Jersey Client by doing a GET call at Client , but I wanted send the huge payload say 1 GB of XML data as stream vis POST call.

I tried using solution given in here , but this solution again uses the system memory.

I do not want to store 1GB of data in disk , instead create on-fly the 1GB request stream of data / write the 1GB data to POST call directly chunk by chunk.

Any help highly appreciated. Thanks in Advance.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jersey 2 Client that would write the stream of data to POST call chunk by chunk

Naseer Sargi
Hi All ,

After doing much research I found the solution and here it goes :

First we need to implements javax.ws.rs.core.StreamingOutput as mentioned below:

 public class JerseyStreamingOutput implements StreamingOutput {


    /**
     * Overriding the write method to write request data directly to Jersey outputStream .
     * @param outputStream
     * @throws IOException
     * @throws WebApplicationException
     */
    @Override
    public void write(OutputStream outputStream) throws IOException, WebApplicationException {
        
        // Write to Jersey OutputStream chunk by chunk here 
    }


Jersey Cient code :

JerseyStreamingOutput jerseyStreamingOutput =
                    new JerseyStreamingOutput(10240,1024);
            WebTarget target = client.target("http://localhost:8080");
            response = target.path("/somepath").
                    request().post(Entity.entity(jerseyStreamingOutput, MediaType.APPLICATION_OCTET_STREAM_TYPE));

So the above solution helps in saving hard disk memory by writing the request chunk by chunk to outputStream , without this approach we would end up keeping the request xml file say 1GB in hard disk.



On Mon, Aug 15, 2016 at 5:07 PM, Pavel Bucek <[hidden email]> wrote:

Hi Naseer,

have you seen this test?

https://github.com/jersey/jersey/blob/2.x/tests/e2e/src/test/java/org/glassfish/jersey/tests/e2e/client/ClientBufferingDisabledTest.java

Regards,
Pavel


On 15/08/16 12:53, Naseer Sargi wrote:

I wanted to write a Jersey 2 Client that would write the stream of data to POST call chunk by chunk.

Why? , this will help me to avoid to keep the whole inputstream request data to store in disk memory before sending via a POST call.

I have searched over the net and looked into Jersey2 API as well but did not find any solution , however there are solution for server side which sends the huge response in stream and reads the same in Jersey Client by doing a GET call at Client , but I wanted send the huge payload say 1 GB of XML data as stream vis POST call.

I tried using solution given in here , but this solution again uses the system memory.

I do not want to store 1GB of data in disk , instead create on-fly the 1GB request stream of data / write the 1GB data to POST call directly chunk by chunk.

Any help highly appreciated. Thanks in Advance.



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jersey 2 Client that would write the stream of data to POST call chunk by chunk

Gili
By "hard disk space", I assume you mean RAM ;) because this solution won't do anything for hard disk space.

Gili

On 2016-08-23 8:37 AM, Naseer Sargi wrote:
Hi All ,

After doing much research I found the solution and here it goes :

First we need to implements javax.ws.rs.core.StreamingOutput as mentioned below:

 public class JerseyStreamingOutput implements StreamingOutput {


    /**
     * Overriding the write method to write request data directly to Jersey outputStream .
     * @param outputStream
     * @throws IOException
     * @throws WebApplicationException
     */
    @Override
    public void write(OutputStream outputStream) throws IOException, WebApplicationException {
        
        // Write to Jersey OutputStream chunk by chunk here 
    }


Jersey Cient code :

JerseyStreamingOutput jerseyStreamingOutput =
                    new JerseyStreamingOutput(10240,1024);
            WebTarget target = client.target("http://localhost:8080");
            response = target.path("/somepath").
                    request().post(Entity.entity(jerseyStreamingOutput, MediaType.APPLICATION_OCTET_STREAM_TYPE));

So the above solution helps in saving hard disk memory by writing the request chunk by chunk to outputStream , without this approach we would end up keeping the request xml file say 1GB in hard disk.



On Mon, Aug 15, 2016 at 5:07 PM, Pavel Bucek <[hidden email]> wrote:

Hi Naseer,

have you seen this test?

https://github.com/jersey/jersey/blob/2.x/tests/e2e/src/test/java/org/glassfish/jersey/tests/e2e/client/ClientBufferingDisabledTest.java

Regards,
Pavel


On 15/08/16 12:53, Naseer Sargi wrote:

I wanted to write a Jersey 2 Client that would write the stream of data to POST call chunk by chunk.

Why? , this will help me to avoid to keep the whole inputstream request data to store in disk memory before sending via a POST call.

I have searched over the net and looked into Jersey2 API as well but did not find any solution , however there are solution for server side which sends the huge response in stream and reads the same in Jersey Client by doing a GET call at Client , but I wanted send the huge payload say 1 GB of XML data as stream vis POST call.

I tried using solution given in here , but this solution again uses the system memory.

I do not want to store 1GB of data in disk , instead create on-fly the 1GB request stream of data / write the 1GB data to POST call directly chunk by chunk.

Any help highly appreciated. Thanks in Advance.




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jersey 2 Client that would write the stream of data to POST call chunk by chunk

Naseer Sargi
Hi Gili ,

No , it is not RAM :-)

Let me explain in detail :

I was having an requirement where in I should use limited hard disk memory.

Now coming to my use case is that I was using Jersey Client to send a request xml of size 1GB .

Earlier I was generating the 1GB of request xml data as file i.e. .xml, so I was reading the file as InputStream and passing to

post method as mentioned below :

            File requestTemplateXml = new File("file directory");
 FileInputStream fis = FileUtils.openInputStream(requestTemplateXml);

            WebTarget target = client.target("http://localhost:8080");
            response = target.path("/somepath").
                    request().post(Entity.entity(fis, MediaType.APPLICATION_OCTET_STREAM_TYPE));
Now above approach has serious memory consumption , if I have different scenario with different request xml data size , say 1GB , 500MB , 10MB etc.
so all above file will consume the hard disk space , in order to avoid storing the request xml file , I thought to generate the request data as bytes 
on-fly and write to OutputStream chunk by chunk .
Hope you got the solution , Let me know if you need more info .

On Tue, Aug 23, 2016 at 8:35 PM, cowwoc <[hidden email]> wrote:
By "hard disk space", I assume you mean RAM ;) because this solution won't do anything for hard disk space.

Gili


On 2016-08-23 8:37 AM, Naseer Sargi wrote:
Hi All ,

After doing much research I found the solution and here it goes :

First we need to implements javax.ws.rs.core.StreamingOutput as mentioned below:

 public class JerseyStreamingOutput implements StreamingOutput {


    /**
     * Overriding the write method to write request data directly to Jersey outputStream .
     * @param outputStream
     * @throws IOException
     * @throws WebApplicationException
     */
    @Override
    public void write(OutputStream outputStream) throws IOException, WebApplicationException {
        
        // Write to Jersey OutputStream chunk by chunk here 
    }


Jersey Cient code :

JerseyStreamingOutput jerseyStreamingOutput =
                    new JerseyStreamingOutput(10240,1024);
            WebTarget target = client.target("http://localhost:8080");
            response = target.path("/somepath").
                    request().post(Entity.entity(jerseyStreamingOutput, MediaType.APPLICATION_OCTET_STREAM_TYPE));

So the above solution helps in saving hard disk memory by writing the request chunk by chunk to outputStream , without this approach we would end up keeping the request xml file say 1GB in hard disk.



On Mon, Aug 15, 2016 at 5:07 PM, Pavel Bucek <[hidden email]> wrote:

Hi Naseer,

have you seen this test?

https://github.com/jersey/jersey/blob/2.x/tests/e2e/src/test/java/org/glassfish/jersey/tests/e2e/client/ClientBufferingDisabledTest.java

Regards,
Pavel


On 15/08/16 12:53, Naseer Sargi wrote:

I wanted to write a Jersey 2 Client that would write the stream of data to POST call chunk by chunk.

Why? , this will help me to avoid to keep the whole inputstream request data to store in disk memory before sending via a POST call.

I have searched over the net and looked into Jersey2 API as well but did not find any solution , however there are solution for server side which sends the huge response in stream and reads the same in Jersey Client by doing a GET call at Client , but I wanted send the huge payload say 1 GB of XML data as stream vis POST call.

I tried using solution given in here , but this solution again uses the system memory.

I do not want to store 1GB of data in disk , instead create on-fly the 1GB request stream of data / write the 1GB data to POST call directly chunk by chunk.

Any help highly appreciated. Thanks in Advance.





Loading...