The
Stream type represents a producer of a sequence of values.
Its dual,
Fold, represents a consumer. While both types support
similar transformations, the key difference is that only
Stream
can compose multiple producers, and only
Fold can compose
multiple consumers.
Console Echo Example
To get you started, here is an example of a program which reads lines
from console and writes them back to the console.
>>> import Data.Function ((&))
>>> :{
echo =
Stream.repeatM getLine -- Stream IO String
& Stream.mapM putStrLn -- Stream IO ()
& Stream.fold Fold.drain -- IO ()
:}
This is a simple example of a declarative representation of an
imperative loop using streaming combinators. In this example,
repeatM generates an infinite stream of
Strings by
repeatedly performing the
getLine IO action.
mapM then
applies
putStrLn on each element in the stream converting it to
stream of
(). Finally,
drain folds the stream
to IO discarding the () values, thus producing only effects.
This gives you an idea about how we can program declaratively by
representing loops using streams. Compare this declarative loopless
approach with an imperative approach using a
while loop for
writing the same program. In this module, you can find all
Data.List-like functions and many more powerful combinators to
perform common programming tasks.
Static Stream Fusion
The
Stream type represents streams as state machines. When
composed statically, these state machines fuse together at compile
time, eliminating intermediate data structures and function calls.
This results in the generation of tight, efficient loops comparable to
those written in low-level languages like C. For instance, in the
earlier example, operations like
repeatM and
mapM are
written as separate fragments but fuse into a single, optimized loop.
The primary goal of the
Stream type is to build highly
efficient streams via compile-time fusion of modular loop fragments.
However, this technique comes with trade-offs and should be used with
care. Stream
construction operations such as
cons,
append,
interleave,
mergeBy, and
zipWith
work extremely well at a small scale. But at a large scale, their
performance degrades due to O(n^2) complexity, where
n is the
number of compositions.
Therefore, it's best to generate a fused stream in one go, if
possible. While using a small number of composition operations is
absolutely fine, avoid using large number of composition operations.
For example, do not try to construct a fused
Stream by using
cons rescursively. However, you can use
cons and any
other construction operations on the CPS
StreamK type without
any problem. The CPS construction operations have linear (O(n))
performance characteristics and scale much better, though they are not
as efficient as fused streams due to function call overhead at each
step.
When used correctly, the fused
Stream type can be 10x to 100x
faster than CPS-based streams, depending on the use case.
Rule of Thumb: Use the fused
Stream type when the number
of compositions is small and they are static or compile-time. Use the
CPS-based
StreamK type when the number of compositions is
large or potentially infinite, and they are dynamic or composed at
runtime. Both types are fully interconvertible, allowing you to choose
the best tool for each part of your pipeline.
Better and Effectful Lists
This module offers operations analogous to standard Haskell lists from
the
base package. Streams can be viewed as a generalization
of lists — providing all the functionality of standard lists, plus
additional capabilities such as effectful operations and improved
performance through stream fusion. They can easily replace lists in
most contexts, and go beyond where lists fall short.
For instance, a common limitation of lists is the inability to perform
IO actions (e.g., printing) at arbitrary points during processing.
Streams naturally support such effectful operations.
As discussed in the fusion section above, while the
Stream type
is not consable and appendable at scale, the
StreamK type is
consable and appendable at scale.
Non-determinism and List Transformers
Streamly does not provide a
ListT like Monad instance but it
provides all the equivalent functionality and more. We do not provide
a Monad instance for streams, as there are many possible ways to
define the bind operation. Instead, we offer bind-style operations
such as
concatFor,
concatForM, and their variants (e.g.
fair interleaving and breadth-first nesting). These can be used for
convenient ListT-style stream composition. Additionally, we provide
applicative-style cross product operations like
cross and its
variants which are many times faster than the monad style operations.
Logic Programming
Streamly does not provide a
LogicT-style Monad instance, but
it offers all the equivalent functionality—and more. Operations like
fairCross and
fairConcatFor nest outer and inner streams
fairly, ensuring that no stream is starved when exploring cross
products.
This enables balanced exploration across all dimensions in
backtracking problems, while also supporting infinite streams. It
effectively replaces the core functionality of
LogicT from
the
logict package, with significantly better performance. In
particular, it avoids the quadratic slowdown seen with
observeMany, and the applicative
fairCross runs many
times faster, achieving loop nesting performance comparable to C.