Using SSE without 'chunked' Transfer-Encoding

[Behrooz Nobakht]

Hi,

I’ve followed the documentation on
SSE<https://jersey.java.net/documentation/latest/sse.html>to implement
a RESTful resource in combination with a JavaScript client.
Simplifying the code, at the level of Jersey resources, I have:

@***@Produces(SseFeature.SERVER_SENT_EVENTS)@Path("out")public EventOutput get(
@HeaderParam(SseFeature.LAST_EVENT_ID_HEADER)
@DefaultValue("-1") String lastEventId) {
final EventOutput output = new EventOutput();
// publish an async mechanism to build the output in a separate thread
return output;
}

and in a separate thread, the event output is being built as:

OutboundEvent.Builder oeb = new OutboundEvent.Builder();
List<MyPojo> events = // retrieve my pojos
listoeb.name("event");oeb.id("someID");
oeb.reconnectDelay(500);
oeb.mediaType(MediaType.APPLICATION_JSON_TYPE);
oeb.data(MyPojo.class, events);try {
OutboundEvent event = oeb.build();
output.write(event);
logger.info("Flushed SSE event: {}", event);
} catch (Exception x) {
logger.error("Failed flushing SSE event: {}", x);
} finally {
try {
output.close();
} catch (Exception x2) {
logger.error("Failed closing output: {}", x2);
}
}

and at the client side, I have a simple JavaScript code to use
EventSource<https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-sent_events>as:

var source = new EventSource('//myserver/events/out');
source.addEventListener('event', function(e) {
var jsonData = JSON.parse(e.data);
// use the jsonData
}, false);

The problem starts with the observation that only the first event is
processed and all subsequent events remain in a “pending” state of HTTP
request (using Chrome developer console).

Investigating further shows indeed that the header Transfer-Encoding is set
to “chunked”. By its own, I think this should not be a problem. However,
there are a number of resources (such as
this<http://stackoverflow.com/q/13590755/248082>,
that<http://www.whatwg.org/specs/web-apps/current-work/multipage/comms.html#authoring-notes>,
or even <https://github.com/http-kit/http-kit/issues/56>) suggest that
Transfer-Encoding should be set to “identity” or even removed. Is this
possibly from Jersey server side? I’m using Jersey with embedded Jetty HTTP
container factory.

It is also an interesting observation based on the above code that I see
consecutive log lines saying “Flushed SSE Event 
” but the client side
remains pending and nothing happens further.

I appreciate any feedback or solution to this issue as it seems not to be
so documented or explored.

Thanks in advance,
Behrooz

[Michal Gajdos]

Hi Behrooz,

the problem here is the finally block in code executed in your separate
thread. After first sent OutboundEvent you're closing the EventOutput
which means no more OutboundEvents can be sent to that client. Remove
the closing of EventOutput and all events should be sent to your client.
With this approach you need to handle event outputs closed by clients
and release the resources by yourself. The other possibility is to start
using SseBroadcaster ([1]) and store created EventOutputs in there (see
[2]). SseBroadcaster will take care of releasing resources of closed
event outputs for you. Then, in your separate thread, you should sent
new events via SseBroadcaster.broadcast method which delivers the events
to all connected clients.

[1]
https://jersey.java.net/apidocs/latest/jersey/org/glassfish/jersey/media/sse/SseBroadcaster.html
[2]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L166
[3]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L216

HTH,
Michal

Hi,
I’ve followed the documentation on SSE
<https://jersey.java.net/documentation/latest/sse.html> to implement a
RESTful resource in combination with a JavaScript client. Simplifying
@Produces(SseFeature.SERVER_SENT_EVENTS)
@Path("out")
public EventOutput get(
@HeaderParam(SseFeature.LAST_EVENT_ID_HEADER)@DefaultValue("-1") String lastEventId) {
final EventOutput output =new EventOutput();
// publish an async mechanism to build the output in a separate thread
return output;
}
|
|OutboundEvent.Builder oeb =new OutboundEvent.Builder();
List<MyPojo> events =// retrieve my pojos list
oeb.name <http://oeb.name>("event");
oeb.id <http://oeb.id>("someID");
oeb.reconnectDelay(500);
oeb.mediaType(MediaType.APPLICATION_JSON_TYPE);
oeb.data(MyPojo.class,events);
try {
OutboundEvent event = oeb.build();
output.write(event);
logger.info <http://logger.info>("Flushed SSE event: {}", event);
}catch (Exception x) {
logger.error("Failed flushing SSE event: {}", x);
}finally {
try {
output.close();
}catch (Exception x2) {
logger.error("Failed closing output: {}", x2);
}
}
|
and at the client side, I have a simple JavaScript code to use
EventSource
<https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-sent_events>
|var source =new EventSource('//myserver/events/out');
source.addEventListener('event',function(e) {
var jsonData = JSON.parse(e.data);
// use the jsonData
},false);
|
The problem starts with the observation that only the first event is
processed and all subsequent events remain in a “pending” state of
HTTP request (using Chrome developer console).
Investigating further shows indeed that the header Transfer-Encoding
is set to “chunked”. By its own, I think this should not be a problem.
However, there are a number of resources (such as this
<http://stackoverflow.com/q/13590755/248082>, that
<http://www.whatwg.org/specs/web-apps/current-work/multipage/comms.html#authoring-notes>,
or even <https://github.com/http-kit/http-kit/issues/56>) suggest that
Transfer-Encoding should be set to “identity” or even removed. Is this
possibly from Jersey server side? I’m using Jersey with embedded Jetty
HTTP container factory.
It is also an interesting observation based on the above code that I
see consecutive log lines saying “Flushed SSE Event 
” but the client
side remains pending and nothing happens further.
I appreciate any feedback or solution to this issue as it seems not to
be so documented or explored.
Thanks in advance,
Behrooz

[cowwoc]

On a side-note, the whole concept of SSE seems unRESTful in the sense
that you are moving from stateless to stateful connections. This reduces
server scalability by maintaining at least one connect per client, even
though the connection is idle 99.99% of the time.

Here is what Roy Fielding had to say about it:
http://roy.gbiv.com/untangled/2008/economies-of-scale

Gili

Post by Michal Gajdos
Hi Behrooz,
the problem here is the finally block in code executed in your
separate thread. After first sent OutboundEvent you're closing the
EventOutput which means no more OutboundEvents can be sent to that
client. Remove the closing of EventOutput and all events should be
sent to your client. With this approach you need to handle event
outputs closed by clients and release the resources by yourself. The
other possibility is to start using SseBroadcaster ([1]) and store
created EventOutputs in there (see [2]). SseBroadcaster will take care
of releasing resources of closed event outputs for you. Then, in your
separate thread, you should sent new events via
SseBroadcaster.broadcast method which delivers the events to all
connected clients.
[1]
https://jersey.java.net/apidocs/latest/jersey/org/glassfish/jersey/media/sse/SseBroadcaster.html
[2]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L166
[3]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L216
HTH,
Michal

Hi,
I’ve followed the documentation on SSE
<https://jersey.java.net/documentation/latest/sse.html> to implement
a RESTful resource in combination with a JavaScript client.
@Produces(SseFeature.SERVER_SENT_EVENTS)
@Path("out")
public EventOutput get(
@HeaderParam(SseFeature.LAST_EVENT_ID_HEADER)@DefaultValue("-1") String lastEventId) {
final EventOutput output =new EventOutput();
// publish an async mechanism to build the output in a separate thread
return output;
}
|
|OutboundEvent.Builder oeb =new OutboundEvent.Builder();
List<MyPojo> events =// retrieve my pojos list
oeb.name <http://oeb.name>("event");
oeb.id <http://oeb.id>("someID");
oeb.reconnectDelay(500);
oeb.mediaType(MediaType.APPLICATION_JSON_TYPE);
oeb.data(MyPojo.class,events);
try {
OutboundEvent event = oeb.build();
output.write(event);
logger.info <http://logger.info>("Flushed SSE event: {}", event);
}catch (Exception x) {
logger.error("Failed flushing SSE event: {}", x);
}finally {
try {
output.close();
}catch (Exception x2) {
logger.error("Failed closing output: {}", x2);
}
}
|
and at the client side, I have a simple JavaScript code to use
EventSource
<https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-sent_events>
|var source =new EventSource('//myserver/events/out');
source.addEventListener('event',function(e) {
var jsonData = JSON.parse(e.data);
// use the jsonData
},false);
|
The problem starts with the observation that only the first event is
processed and all subsequent events remain in a “pending” state of
HTTP request (using Chrome developer console).
Investigating further shows indeed that the header Transfer-Encoding
is set to “chunked”. By its own, I think this should not be a
problem. However, there are a number of resources (such as this
<http://stackoverflow.com/q/13590755/248082>, that
<http://www.whatwg.org/specs/web-apps/current-work/multipage/comms.html#authoring-notes>,
or even <https://github.com/http-kit/http-kit/issues/56>) suggest
that Transfer-Encoding should be set to “identity” or even removed.
Is this possibly from Jersey server side? I’m using Jersey with
embedded Jetty HTTP container factory.
It is also an interesting observation based on the above code that I
see consecutive log lines saying “Flushed SSE Event 
” but the client
side remains pending and nothing happens further.
I appreciate any feedback or solution to this issue as it seems not
to be so documented or explored.
Thanks in advance,
Behrooz

[Behrooz Nobakht]

Hi,

Thanks Michal for your hints. Indeed that was the problem and also using
SseBroadcaster fits better with event-driven design.

Thanks Gili, for your argument and perspective and I also agree with that.
However, the use for me here is really different. In my case, the server
does not go idle for event short times. I'm using this features in
integration with d3 to be able to visualize graph properties through time.
The source is available at: https://github.com/nobeh/visactor

Cheers,
Behrooz

Post by cowwoc
On a side-note, the whole concept of SSE seems unRESTful in the sense
that you are moving from stateless to stateful connections. This reduces
server scalability by maintaining at least one connect per client, even
though the connection is idle 99.99% of the time.
http://roy.gbiv.com/untangled/2008/economies-of-scale
Gili
Hi Behrooz,
the problem here is the finally block in code executed in your separate
thread. After first sent OutboundEvent you're closing the EventOutput which
means no more OutboundEvents can be sent to that client. Remove the closing
of EventOutput and all events should be sent to your client. With this
approach you need to handle event outputs closed by clients and release the
resources by yourself. The other possibility is to start using
SseBroadcaster ([1]) and store created EventOutputs in there (see [2]).
SseBroadcaster will take care of releasing resources of closed event
outputs for you. Then, in your separate thread, you should sent new events
via SseBroadcaster.broadcast method which delivers the events to all
connected clients.
[1]
https://jersey.java.net/apidocs/latest/jersey/org/glassfish/jersey/media/sse/SseBroadcaster.html
[2]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L166
[3]
https://github.com/jersey/jersey/blob/master/examples/sse-item-store-webapp/src/main/java/org/glassfish/jersey/examples/sseitemstore/ItemStoreResource.java#L216
HTH,
Michal
Hi,
I’ve followed the documentation on SSE<https://jersey.java.net/documentation/latest/sse.html>to implement a RESTful resource in combination with a JavaScript client.
@***@Produces(SseFeature.SERVER_SENT_EVENTS)@Path("out")public EventOutput get(
@HeaderParam(SseFeature.LAST_EVENT_ID_HEADER) @DefaultValue("-1") String lastEventId) {
final EventOutput output = new EventOutput();
// publish an async mechanism to build the output in a separate thread
return output;
}
OutboundEvent.Builder oeb = new OutboundEvent.Builder();
List<MyPojo> events = // retrieve my pojos listoeb.name("event");oeb.id("someID");
oeb.reconnectDelay(500);
oeb.mediaType(MediaType.APPLICATION_JSON_TYPE);
oeb.data(MyPojo.class, events);try {
OutboundEvent event = oeb.build();
output.write(event);
logger.info("Flushed SSE event: {}", event);
} catch (Exception x) {
logger.error("Failed flushing SSE event: {}", x);
} finally {
try {
output.close();
} catch (Exception x2) {
logger.error("Failed closing output: {}", x2);
}
}
var source = new EventSourc
e('//myserver/events/out');
source.addEventListener('event', function(e) {
var jsonData = JSON.parse(e.data);
// use the jsonData
}, false);
The problem starts with the observation that only the first event is
processed and all subsequent events remain in a “pending” state of HTTP
request (using Chrome developer console).
Investigating further shows indeed that the header Transfer-Encoding is
set to “chunked”. By its own, I think this should not be a problem.
However, there are a number of resources (such as this<http://stackoverflow.com/q/13590755/248082>,
that<http://www.whatwg.org/specs/web-apps/current-work/multipage/comms.html#authoring-notes>,
or even <https://github.com/http-kit/http-kit/issues/56>) suggest that
Transfer-Encoding should be set to “identity” or even removed. Is this
possibly from Jersey server side? I’m using Jersey with embedded Jetty HTTP
container factory.
It is also an interesting observation based on the above code that I see
consecutive log lines saying “Flushed SSE Event 
” but the client side
remains pending and nothing happens further.
I appreciate any feedback or solution to this issue as it seems not to be
so documented or explored.
Thanks in advance,
Behrooz