Entity streams

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Entity streams

Gili
Hi,

I'd like to define a @GET method whose entity is an OutputStream that gets populated slowly over time. Imagine for example, a response that gets populated with a line of text every second, for a total of one minute. The client would like to act on the reply every time a line is added (once a second) as opposed to waiting for the complete response to be written (once a minute).

I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput on the server-side thanks to http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey but it's not clear what I'm supposed to do on the client-side.

How can I get access the client-side InputStream before the Response is complete?

Thanks,
Gili
Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

zzantozz
I've never done it, but my first guess would be:
InputStream in = resource. ... .get(InputStream.class);

On Thu, Apr 28, 2011 at 2:33 PM, Gili <[hidden email]> wrote:
Hi,

I'd like to define a @GET method whose entity is an OutputStream that gets
populated slowly over time. Imagine for example, a response that gets
populated with a line of text every second, for a total of one minute. The
client would like to act on the reply every time a line is added (once a
second) as opposed to waiting for the complete response to be written (once
a minute).

I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput on
the server-side thanks to
http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
but it's not clear what I'm supposed to do on the client-side.

How can I get access the client-side InputStream before the Response is
complete?

Thanks,
Gili

--
View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
Sent from the Jersey mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Gili
I believe WebResource.get() only returns once the Response is complete; otherwise, how can it return the HTTP return-code?

I think we need to register some sort of hook for processing the InputStream before invoking get() but no such method seems to exist. Am I overlooking something or should I file a RFE?

Gili

zzantozz wrote
I've never done it, but my first guess would be:
InputStream in = resource. ... .get(InputStream.class);

On Thu, Apr 28, 2011 at 2:33 PM, Gili <[hidden email]> wrote:

> Hi,
>
> I'd like to define a @GET method whose entity is an OutputStream that gets
> populated slowly over time. Imagine for example, a response that gets
> populated with a line of text every second, for a total of one minute. The
> client would like to act on the reply every time a line is added (once a
> second) as opposed to waiting for the complete response to be written (once
> a minute).
>
> I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput on
> the server-side thanks to
>
> http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
> but it's not clear what I'm supposed to do on the client-side.
>
> How can I get access the client-side InputStream before the Response is
> complete?
>
> Thanks,
> Gili
>
> --
> View this message in context:
> http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
> Sent from the Jersey mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Pavel Bucek-2
On 4/29/11 4:21 AM, Gili wrote:
> I believe WebResource.get() only returns once the Response is complete;
> otherwise, how can it return the HTTP return-code?
not true; actually http return code is the first information which comes
back from server (it is the very first "line")
> I think we need to register some sort of hook for processing the InputStream
> before invoking get() but no such method seems to exist. Am I overlooking
> something or should I file a RFE?
no, please don't. You can't get access to InputStream before calling
get, because the request wasn't send yet and there is nothing what you
could get at that moment.

Zzantozz is right in this case, WebResource.get(InputStream.class) is
what you are looking for.

Regards,
Pavel

> Gili
>
>
> zzantozz wrote:
>> I've never done it, but my first guess would be:
>> InputStream in = resource. ... .get(InputStream.class);
>>
>> On Thu, Apr 28, 2011 at 2:33 PM, Gili&lt;[hidden email]&gt;
>> wrote:
>>
>>> Hi,
>>>
>>> I'd like to define a @GET method whose entity is an OutputStream that
>>> gets
>>> populated slowly over time. Imagine for example, a response that gets
>>> populated with a line of text every second, for a total of one minute.
>>> The
>>> client would like to act on the reply every time a line is added (once a
>>> second) as opposed to waiting for the complete response to be written
>>> (once
>>> a minute).
>>>
>>> I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput
>>> on
>>> the server-side thanks to
>>>
>>> http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
>>> but it's not clear what I'm supposed to do on the client-side.
>>>
>>> How can I get access the client-side InputStream before the Response is
>>> complete?
>>>
>>> Thanks,
>>> Gili
>>>
>>> --
>>> View this message in context:
>>> http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
>>> Sent from the Jersey mailing list archive at Nabble.com.
>>>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315471.html
> Sent from the Jersey mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Gili

    Excellent. I'm happy to be wrong in this case :)

    Next question, seeing how Javascript does not support multitasking is there any way for me to implement the following in Javascript?

1. Client issues a request
2. The connection is kept open for a minute. Every second, the server writes a URI into the response body (and flushes) without closing the response.
3. As the JS client receives each line, it should issue an HTTP GET to the URI and process the information, then proceed to the next line in the original connection.

    Is it possible to implement #3 in Javascript? Is there a limit to the number of concurrent HTTP connections?

    Bigger question: what's the best way to send real-time information from the server to the client (potentially 300ms between each event) while minimizing the impact on scalability? I am trying to build a hybrid of the methods mentioned at http://stackoverflow.com/questions/568897/event-based-interaction-style-in-rest

Thanks,
Gili

On 29/04/2011 3:31 AM, Pavel Bucek-2[via Jersey] wrote:
On 4/29/11 4:21 AM, Gili wrote:
> I believe WebResource.get() only returns once the Response is complete;
> otherwise, how can it return the HTTP return-code?
not true; actually http return code is the first information which comes
back from server (it is the very first "line")
> I think we need to register some sort of hook for processing the InputStream
> before invoking get() but no such method seems to exist. Am I overlooking
> something or should I file a RFE?
no, please don't. You can't get access to InputStream before calling
get, because the request wasn't send yet and there is nothing what you
could get at that moment.

Zzantozz is right in this case, WebResource.get(InputStream.class) is
what you are looking for.

Regards,
Pavel

> Gili
>
>
> zzantozz wrote:
>> I've never done it, but my first guess would be:
>> InputStream in = resource. ... .get(InputStream.class);
>>
>> On Thu, Apr 28, 2011 at 2:33 PM, Gili&lt;[hidden email]&gt;
>> wrote:
>>
>>> Hi,
>>>
>>> I'd like to define a @GET method whose entity is an OutputStream that
>>> gets
>>> populated slowly over time. Imagine for example, a response that gets
>>> populated with a line of text every second, for a total of one minute.
>>> The
>>> client would like to act on the reply every time a line is added (once a
>>> second) as opposed to waiting for the complete response to be written
>>> (once
>>> a minute).
>>>
>>> I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput
>>> on
>>> the server-side thanks to
>>>
>>> http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
>>> but it's not clear what I'm supposed to do on the client-side.
>>>
>>> How can I get access the client-side InputStream before the Response is
>>> complete?
>>>
>>> Thanks,
>>> Gili
>>>
>>> --
>>> View this message in context:
>>> http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
>>> Sent from the Jersey mailing list archive at Nabble.com.
>>>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315471.html
> Sent from the Jersey mailing list archive at Nabble.com.
>




If you reply to this email, your message will be added to the discussion below:
http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315953.html
To unsubscribe from Entity streams, click here.

Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

japod
Hi Gili,

Please see in-line...

On 04/29/2011 02:17 PM, Gili wrote:

    Excellent. I'm happy to be wrong in this case :)

    Next question, seeing how Javascript does not support multitasking is there any way for me to implement the following in Javascript?

1. Client issues a request
2. The connection is kept open for a minute. Every second, the server writes a URI into the response body (and flushes) without closing the response.
3. As the JS client receives each line, it should issue an HTTP GET to the URI and process the information, then proceed to the next line in the original connection.

    Is it possible to implement #3 in Javascript? Is there a limit to the number of concurrent HTTP connections?

I believe with any browser you are only guaranteed to get 2 concurrent connections to one host at a time.
There are some browser, which give you more, but 2 is the number you can rely on by default.
Then to solve the above problem, you would need to use the first connection (the one kept open,
point 2 above) to actually serve some JavaScript code including the commands
to trigger the other request (point 3).


    Bigger question: what's the best way to send real-time information from the server to the client (potentially 300ms between each event) while minimizing the impact on scalability? I am trying to build a hybrid of the methods mentioned at http://stackoverflow.com/questions/568897/event-based-interaction-style-in-rest


Another problem here is even the StreamingOutput does not save you from running out of the Java threads.
If you want a scalable solution, you would need to somehow re-use the threads to be able to serve
all the long open connections. Some work in this area was done by Paul Sandoz and Jean-Francois Arcand
with project Jersey and project Atmosphere. An example is available at [1].

~Jakub

[1]http://atmosphere.java.net/nonav/xref/org/atmosphere/samples/twitter/TwitterResource.html#80

Thanks,
Gili

On 29/04/2011 3:31 AM, Pavel Bucek-2[via Jersey] wrote:
On 4/29/11 4:21 AM, Gili wrote:
> I believe WebResource.get() only returns once the Response is complete;
> otherwise, how can it return the HTTP return-code?
not true; actually http return code is the first information which comes
back from server (it is the very first "line")
> I think we need to register some sort of hook for processing the InputStream
> before invoking get() but no such method seems to exist. Am I overlooking
> something or should I file a RFE?
no, please don't. You can't get access to InputStream before calling
get, because the request wasn't send yet and there is nothing what you
could get at that moment.

Zzantozz is right in this case, WebResource.get(InputStream.class) is
what you are looking for.

Regards,
Pavel

> Gili
>
>
> zzantozz wrote:
>> I've never done it, but my first guess would be:
>> InputStream in = resource. ... .get(InputStream.class);
>>
>> On Thu, Apr 28, 2011 at 2:33 PM, Gili&lt;[hidden email]&gt;
>> wrote:
>>
>>> Hi,
>>>
>>> I'd like to define a @GET method whose entity is an OutputStream that
>>> gets
>>> populated slowly over time. Imagine for example, a response that gets
>>> populated with a line of text every second, for a total of one minute.
>>> The
>>> client would like to act on the reply every time a line is added (once a
>>> second) as opposed to waiting for the complete response to be written
>>> (once
>>> a minute).
>>>
>>> I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput
>>> on
>>> the server-side thanks to
>>>
>>> http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
>>> but it's not clear what I'm supposed to do on the client-side.
>>>
>>> How can I get access the client-side InputStream before the Response is
>>> complete?
>>>
>>> Thanks,
>>> Gili
>>>
>>> --
>>> View this message in context:
>>> http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
>>> Sent from the Jersey mailing list archive at Nabble.com.
>>>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315471.html
> Sent from the Jersey mailing list archive at Nabble.com.
>




If you reply to this email, your message will be added to the discussion below:
http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315953.html
To unsubscribe from Entity streams, click here.



View this message in context: Re: Entity streams
Sent from the Jersey mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Gili
Jakub,

Thank you for that link! Atmosphere is exactly what I had in mind. This brings up the question: does this "Reverse-Ajax" model suffer from the economies of scale problem mentioned by Roy Fielding in [1]?

[1] http://roy.gbiv.com/untangled/2008/economies-of-scale

Thanks,
Gili

Jakub Podlesak-2 wrote
Another problem here is even the StreamingOutput does not save you from
running out of the Java threads.
If you want a scalable solution, you would need to somehow re-use the
threads to be able to serve
all the long open connections. Some work in this area was done by Paul
Sandoz and Jean-Francois Arcand
with project Jersey and project Atmosphere. An example is available at [1].

~Jakub

[1]http://atmosphere.java.net/nonav/xref/org/atmosphere/samples/twitter/TwitterResource.html#80
Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Tauren Mills
Gili,

Something else to consider is that there might be better tools for
doing what you need to do. It sounds like WebSockets might be a better
match for you. Unfortunately, I'm not at all familiar with using this
in the Java world, so I can't provide any Java-specific pointers.

But lately I've been dabbling with Node.JS [1] and Socket.IO [2].
Might be worth looking into. The great thing about Socket.IO is that
it supports Javascript clients all the way back to IE5.5, and could
save you a bunch of client-side work.

[1] http://nodejs.org/
[2] http://socket.io/

Tauren


On Fri, Apr 29, 2011 at 12:16 PM, Gili <[hidden email]> wrote:

> Jakub,
>
> Thank you for that link! Atmosphere is exactly what I had in mind. This
> brings up the question: does this "Reverse-Ajax" model suffer from the
> economies of scale problem mentioned by Roy Fielding in [1]?
>
> [1] http://roy.gbiv.com/untangled/2008/economies-of-scale
>
> Thanks,
> Gili
>
>
> Jakub Podlesak-2 wrote:
>>
>> Another problem here is even the StreamingOutput does not save you from
>> running out of the Java threads.
>> If you want a scalable solution, you would need to somehow re-use the
>> threads to be able to serve
>> all the long open connections. Some work in this area was done by Paul
>> Sandoz and Jean-Francois Arcand
>> with project Jersey and project Atmosphere. An example is available at
>> [1].
>>
>> ~Jakub
>>
>> [1]http://atmosphere.java.net/nonav/xref/org/atmosphere/samples/twitter/TwitterResource.html#80
>>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6317967.html
> Sent from the Jersey mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Entity streams

Gili
In reply to this post by japod

    Thanks for the head's up.

Gili

On 29/04/2011 11:55 AM, Jakub Podlesak-2[via Jersey] wrote:
Hi Gili,

Please see in-line...

On 04/29/2011 02:17 PM, Gili wrote:

    Excellent. I'm happy to be wrong in this case :)

    Next question, seeing how Javascript does not support multitasking is there any way for me to implement the following in Javascript?

1. Client issues a request
2. The connection is kept open for a minute. Every second, the server writes a URI into the response body (and flushes) without closing the response.
3. As the JS client receives each line, it should issue an HTTP GET to the URI and process the information, then proceed to the next line in the original connection.

    Is it possible to implement #3 in Javascript? Is there a limit to the number of concurrent HTTP connections?

I believe with any browser you are only guaranteed to get 2 concurrent connections to one host at a time.
There are some browser, which give you more, but 2 is the number you can rely on by default.
Then to solve the above problem, you would need to use the first connection (the one kept open,
point 2 above) to actually serve some JavaScript code including the commands
to trigger the other request (point 3).


    Bigger question: what's the best way to send real-time information from the server to the client (potentially 300ms between each event) while minimizing the impact on scalability? I am trying to build a hybrid of the methods mentioned at http://stackoverflow.com/questions/568897/event-based-interaction-style-in-rest


Another problem here is even the StreamingOutput does not save you from running out of the Java threads.
If you want a scalable solution, you would need to somehow re-use the threads to be able to serve
all the long open connections. Some work in this area was done by Paul Sandoz and Jean-Francois Arcand
with project Jersey and project Atmosphere. An example is available at [1].

~Jakub

[1]http://atmosphere.java.net/nonav/xref/org/atmosphere/samples/twitter/TwitterResource.html#80

Thanks,
Gili

On 29/04/2011 3:31 AM, Pavel Bucek-2[via Jersey] wrote:
On 4/29/11 4:21 AM, Gili wrote:
> I believe WebResource.get() only returns once the Response is complete;
> otherwise, how can it return the HTTP return-code?
not true; actually http return code is the first information which comes
back from server (it is the very first "line")
> I think we need to register some sort of hook for processing the InputStream
> before invoking get() but no such method seems to exist. Am I overlooking
> something or should I file a RFE?
no, please don't. You can't get access to InputStream before calling
get, because the request wasn't send yet and there is nothing what you
could get at that moment.

Zzantozz is right in this case, WebResource.get(InputStream.class) is
what you are looking for.

Regards,
Pavel

> Gili
>
>
> zzantozz wrote:
>> I've never done it, but my first guess would be:
>> InputStream in = resource. ... .get(InputStream.class);
>>
>> On Thu, Apr 28, 2011 at 2:33 PM, Gili&lt;[hidden email]&gt;
>> wrote:
>>
>>> Hi,
>>>
>>> I'd like to define a @GET method whose entity is an OutputStream that
>>> gets
>>> populated slowly over time. Imagine for example, a response that gets
>>> populated with a line of text every second, for a total of one minute.
>>> The
>>> client would like to act on the reply every time a line is added (once a
>>> second) as opposed to waiting for the complete response to be written
>>> (once
>>> a minute).
>>>
>>> I figured out that I'm supposed to use javax.ws.rs.core.StreamingOutput
>>> on
>>> the server-side thanks to
>>>
>>> http://stackoverflow.com/questions/3496209/input-and-output-binary-streams-using-jersey
>>> but it's not clear what I'm supposed to do on the client-side.
>>>
>>> How can I get access the client-side InputStream before the Response is
>>> complete?
>>>
>>> Thanks,
>>> Gili
>>>
>>> --
>>> View this message in context:
>>> http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6314573.html
>>> Sent from the Jersey mailing list archive at Nabble.com.
>>>
> --
> View this message in context: http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315471.html
> Sent from the Jersey mailing list archive at Nabble.com.
>




If you reply to this email, your message will be added to the discussion below:
http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6315953.html
To unsubscribe from Entity streams, click here.



View this message in context: Re: Entity streams
Sent from the Jersey mailing list archive at Nabble.com.




If you reply to this email, your message will be added to the discussion below:
http://jersey.576304.n2.nabble.com/Entity-streams-tp6314573p6317401.html
To unsubscribe from Entity streams, click here.