Incremental Stream Pattern
The Incremental Stream Pattern allows a server to divide the execution of a GraphQL Operation into multiple steps and send the results of each step back to the client as soon as they are available. This allows the client to start processing partial results before the entire operation has completed.
Problem
GraphQL Operations can be long running and resource intensive, even more so when they are executed in a distributed environment, or in combination with GraphQL Federation.
It might take multiple seconds to resolve an entire GraphQL Operation, so it might be desirable if the server could partially resolve a request and send the results back to the client as a stream. This would allow the client to start processing or rendering at least some of the results, otherwise the user would have to wait in front of a blank screen until the entire operation has completed.
Solution
To solve this problem, the two directives @stream and @defer emerged.
They allow the client to "defer" the execution of a fragment,
or to stream the results of a list.
Let's first have a look at the @defer directive.
The @defer Directive
query {
me {
id
name
email
... FriendsFragment @defer(label: "friends")
}
}
fragment FriendsFragment on User {
friends {
id
name
email
}
}In this case, the @defer directive "defers" the execution of the FriendsFragment fragment.
The server will first resolve the id, name and email fields of the me field and send the result as the first chunk.
Then, it will resolve the Friends fragment and send the result as the second chunk.
The client can already show user information while it is waiting for the friends to be resolved. But what if the friends list is very long, like 1000 friends? We would have to wait until the entire list has been resolved, transferred over the network and parsed by the client.
The @stream Directive
That's a problem that the @stream directive can solve.
Let's change our Query to use the @stream directive instead of @defer.
query {
me {
id
name
email
friends @stream(initialCount: 10, label: "friends") {
id
name
email
}
}
}With this change, the server will send the fields id, name and email of the me field and the first 10 friends as the first chunk.
Once the first chunk is sent, the server will resolve the remaining friends and send them as additional chunks.
The client could be able to show some user info and the first 10 friends, which might be enough to render enough data above the fold to offer a snappy experience to the user while the rest of the friends are still loading.
The Good, the Bad and the Ugly of the @defer and @stream Directives
The benefits of both directives are quite obvious. We're able to start rendering data much earlier than before, which can lead to a much better user experience.
That said, there are some downsides to using these directives.
The @defer and @stream Directives break code generation
Imagine you're applying 3 @defer directives to a query.
How should the code generator account for the fact that the response might have 4 different shapes? (1 initial response + 3 deferred responses)
As deferred Fragments are being sent to the client as separate (labeled) chunks, how should a code generator expose a simple enough API to the developer that is capable of representing the different shapes of the response?
The result will be that the use of @defer will lead to a lot of optional fields in the generated code,
so the client code will have to deal with a lot of null checks.
However, if a field is null, is it actually null or is it just not yet resolved?
The client might know that an Operation is not yet fully resolved,
but it doesn't know if a field needs resolving or if it is actually null.
The @defer and @stream Directives break caching
When you make a request to a GraphQL server, and you get a single response back, it's straight forward to cache the response, independent of the cache implementation.
You might be using a normalized cache, a simple HTTP cache, or a cache that understands GraphQL. What all of them have in common is that they rely on the fact that you get a single response back.
A chunked response cannot be cached in the same way,
at least not without the cache understanding the semantics of the @defer and @stream directives.
The @stream Directive is quite unpredictable
When a @stream directive is applied to a field,
the server will send the desired number of items as the first chunk,
this is the predictable part.
The unpredictable part is that the server will send additional chunks as soon as they are available. How large should these chunks be? How many chunks will the server send? What's the optimal chunk size? How long should the client wait for additional chunks?
As you can see, it's pretty hard to predict what the client will receive when using the @stream directive.
A list of 1000 friends in chunks of 1 could take forever to resolve,
while two chunks, one with 10 friends and one with 990 might also not be ideal.
But there's an even bigger problem than that.
Developers might (accidentally) use @defer and @stream the wrong way
Accidental or not, what if a client developer uses the directives in a way that is not optimal? E.g. what if you defer fields that the resolver resolves in a single roundtrip to the database? What if you stream a list that is actually never longer than 10 items? Is it actually smart to stream a list if the resolver always just makes a single roundtrip to the database? What if deferring a field actually makes the resolver slower because the execution plan is more complex now?
All of these cases have one thing in common:
Just from looking at the GraphQL Schema, a client developer doesn't understand when it is smart to use @defer or @stream and when it is not.
If a client developer doesn't know anything about the implementation of the resolvers,
how can they make an informed decision about when to use @defer or @stream?
On the other hand, backend developers might be confronted with Operations that use @defer and @stream in a way that they didn't expect.
They might have written their resolvers in a way to optimize for a single roundtrip to the database,
but now the use of @defer and @stream might make the resolvers slower or more complex.
In another blog post, I wrote about how using these directives might be overkill in some cases (opens in a new tab).
Conclusion
The @defer and @stream directives are a great addition to the GraphQL specification.
They have the potential to improve the user experience significantly when used correctly.
However, they can also lead to more complex implementations on the client and the server, make code generation harder and sometimes the execution less predictable.
I personally think that they are a complex solution to a problem that is very rare. Like I mentioned in the other blog post, there might be simpler solutions in such edge cases.