This module provides functions to run operating system processes as stream producers, consumers or stream transformation functions. Thus OS processes can be used in the same way as Haskell functions and all the streaming combinators in streamly can be used to combine them. This allows you to seamlessly integrate external binary executables into your Haskell program. However, we recommend native Haskell functions with Streamly threads over using system processes whenever possible. This approach offers a simpler programming model compared to system processes, which also have a larger performance overhead. Prefer Streamly.System.Command module as a higher level wrapper over this module.

Executables as functions

Processes can be composed in a streaming pipeline just like a Posix shell command pipeline. Moreover, we can mix processes and Haskell functions seamlessly in a processing pipeline. For example:

>>> :{
Process.toBytes "echo" ["hello world"]
& Process.pipeBytes "tr" ["[a-z]", "[A-Z]"]
& Stream.fold Stdio.write
:}
HELLO WORLD

Of course, you can use a Haskell function instead of "tr":

>>> :{
Process.toBytes "echo" ["hello world"]
& Unicode.decodeLatin1 & fmap toUpper & Unicode.encodeLatin1
& Stream.fold Stdio.write
:}
HELLO WORLD

Shell commands as functions

Using a shell as the command interpreter we can use shell commands in a data processing pipeline:

>>> :{
Process.toBytes "sh" ["-c", "echo hello | tr [a-z] [A-Z]"]
& Stream.fold Stdio.write
:}
HELLO

Running Commands Concurrently

We can run executables or commands concurrently as we would run any other functions in Streamly. For example, the following program greps the word "to" in all the files in the current directory concurrently:

>>> :{
grep file =
Process.toBytes "grep" ["-H", "pattern", file]
& Stream.handle (\(_ :: Process.ProcessFailure) -> Stream.nil)
& Stream.foldMany (Fold.takeEndBy (== 10) Array.write)
:}

>>> :{
pgrep =
Dir.readFiles "."
& Stream.parConcatMap id grep
& Stream.fold Stdio.writeChunks
:}

Experimental APIs

See Streamly.Internal.System.Process for unreleased functions.