The
Get monad. A monad for efficiently building structures from
encoded lazy ByteStrings.
Primitives are available to decode words of various sizes, both big
and little endian.
Let's decode binary data representing illustrated here. In this
example the values are in little endian.
+------------------+--------------+-----------------+
| 32 bit timestamp | 32 bit price | 16 bit quantity |
+------------------+--------------+-----------------+
A corresponding Haskell value looks like this:
data Trade = Trade
{ timestamp :: !Word32
, price :: !Word32
, qty :: !Word16
} deriving (Show)
The fields in
Trade are marked as strict (using
!)
since we don't need laziness here. In practise, you would probably
consider using the UNPACK pragma as well.
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#unpack-pragma
Now, let's have a look at a decoder for this format.
getTrade :: Get Trade
getTrade = do
timestamp <- getWord32le
price <- getWord32le
quantity <- getWord16le
return $! Trade timestamp price quantity
Or even simpler using applicative style:
getTrade' :: Get Trade
getTrade' = Trade <$> getWord32le <*> getWord32le <*> getWord16le
There are two kinds of ways to execute this decoder, the lazy input
method and the incremental input method. Here we will use the lazy
input method.
Let's first define a function that decodes many
Trades.
getTrades :: Get [Trade]
getTrades = do
empty <- isEmpty
if empty
then return []
else do trade <- getTrade
trades <- getTrades
return (trade:trades)
Finally, we run the decoder:
lazyIOExample :: IO [Trade]
lazyIOExample = do
input <- BL.readFile "trades.bin"
return (runGet getTrades input)
This decoder has the downside that it will need to read all the input
before it can return. On the other hand, it will not return anything
until it knows it could decode without any decoder errors.
You could also refactor to a left-fold, to decode in a more streaming
fashion, and get the following decoder. It will start to return data
without knowing that it can decode all input.
incrementalExample :: BL.ByteString -> [Trade]
incrementalExample input0 = go decoder input0
where
decoder = runGetIncremental getTrade
go :: Decoder Trade -> BL.ByteString -> [Trade]
go (Done leftover _consumed trade) input =
trade : go decoder (BL.chunk leftover input)
go (Partial k) input =
go (k . takeHeadChunk $ input) (dropHeadChunk input)
go (Fail _leftover _consumed msg) _input =
error msg
takeHeadChunk :: BL.ByteString -> Maybe BS.ByteString
takeHeadChunk lbs =
case lbs of
(BL.Chunk bs _) -> Just bs
_ -> Nothing
dropHeadChunk :: BL.ByteString -> BL.ByteString
dropHeadChunk lbs =
case lbs of
(BL.Chunk _ lbs') -> lbs'
_ -> BL.Empty
The
lazyIOExample uses lazy I/O to read the file from the
disk, which is not suitable in all applications, and certainly not if
you need to read from a socket which has higher likelihood to fail. To
address these needs, use the incremental input method like in
incrementalExample. For an example of how to read
incrementally from a Handle, see the implementation of
decodeFileOrFail in
Data.Binary.