In part one we had a look at a JAX-RS endpoint that streams its content to the requesting client. Now I’d like to show how the fetch-API can be used to consume that streamed content in a web component.

When talking about streamed content here we’re actually refering to chunked transfere encoding where the complete content of a response is spread accross several transmissions. In order to consume our streaming endpoint we’ll use the fetch-API - a more flexible and powerful approach to retrieve resources over the network than the former used XMLHttpRequest.

Notice: In order to use fetch in our web-component we have to rely on a feature called Streaming response body which is marked as experimental at the time of this writing (although supported by all major browsers).

Let’s start with a simple index.html that loads our app.js-module and uses the sfm-stream-output-component defined by it.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
<body>
    <sfm-stream-output></sfm-stream-output>
    <script type="module" src="app.js"></script>
</body>
</html>

For the actual component let’s start with the followin' skeleton:

class SfmStreamOutput extends HTMLElement {

    constructor() {
        super();
        this.root = this.attachShadow({mode: 'open'});
        this.elements = [];
    }

    connectedCallback() {
        customElements
            .whenDefined('sfm-stream-output')
            .then(_ => this.render());
    }

    render() {
        this.root.innerHTML = this.elements
            .map(e => `<div>${e.index}. -- ${e.content}</div>`)
            .join('')
    }
}

Nothing fancy so far - the purpose of the component is to render a <div>-tag for each entry in the components elements-array. Right now this array is empty - so let’s fill it with the response-data from our streaming endpoint using fetch.

A call to fetch will return a Promise which resolves to the Response of that request. Now, it’s actually the Body mixin which provides a bunch of methods to read the responses content. Most of the time you’ll see usages of methods like response.text() or response.json() which actually consume the underlying stream for you and read it to completion. But you can also access the stream and consume it yourself by using response.body:

fetch('/streamoutput/api/resource/stream')  // call the endpoint
    .then(response => response.body)        // get the stream 
    .then(body => { 
        const reader = body.getReader();    // get a reader 
        reader.read()
            .then(({done, value}) => console.log(done, value));
    });

When executing the above code you’ll notice several things:

  1. your console.log will be called only once … so apparently you’re not consuming the complete response
  2. the value is of type Uint8Array … so a binary representation you’ll have to decode before it can be used

Let’s fix the decoding first:

fetch('/streamoutput/api/resource/stream')  // call the endpoint
    .then(response => response.body)        // get the stream 
    .then(body => { 
        const reader = body.getReader();    // get a reader 
        reader.read().then(({done, value}) => 
            console.log(done, new TextDecoder('utf-8').decode(value)));
    });

We still got only one call to console.log - but this time we can actually see the content of our first chunk

...
{"index":36,"content":"Some random text which actually can be generated in order to be really random"}
{"index":37,"content":"Some random text which a

which reveals that it get’s tricky to parse that chunk since it was split arbitrarily.

But now we know at least everything we need in order to process the response.

  1. make sure we call read() several times to consume all chunks of the stream (done in line 17 and 28).
  2. split each junk at our newline-delimiter to get parsable portions (so a valid JSON-object - done via the regex in line 8)
  3. buffer incomplete, trailing lines and prepend ‘em to the next chunk (in order to yield a valid JSON again - line 13 ‘til 22).
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

fetch('/streamoutput/api/resource/stream')
    .then(response => response.body)
    .then(body => {
        const reader = body.getReader();
        const process = async ({done, value}) => {
        if (done) {console.log('I\'m done with you ...');return;}

        let regexp = /\n|\r|\r\n/gm;
        let startIndex = 0;
        let chunk = new TextDecoder('utf-8').decode(value);
        for (;;) {
            let match = regexp.exec(chunk);
            if (!match) {
                if (done) {break;}

                let remainder = chunk.substr(startIndex);
                ({value, done} = await reader.read());

                chunk = remainder + (chunk ? new TextDecoder('utf-8').decode(value) : "");
                startIndex = regexp.lastIndex = 0;
                this.render();
                continue;
            }
            this.elements.push(JSON.parse(chunk.substring(startIndex, match.index)));
            startIndex = regexp.lastIndex;
        }
    };
    reader.read().then(process);
});

When placed in the connectedCallback-method of our web-component it’ll issue the request when it gets added to the DOM and it’ll render each chunk of the response as soon as it was received. You can find all sources in this repository - clone it, compile and run it using the buildAndRun.sh-script (prerequisite is maven and docker) and open http://localhost:8080/streamoutput/index.html in Chrome where you should see a steadily growin UI.