tokens isSpace = words
>>> putDoc (tupled (words "Lorem ipsum dolor")) (Lorem, ipsum, dolor)
ghci> is <- Streams.fromList ["Hello, world!"] >>= Streams.words ghci> replicateM 3 (Streams.read is) [Just "Hello,", Just "world!", Nothing]Note that this may increase the chunk size if the input contains extremely long words.
words :: Text -> [Text]but it was given a more complex type to provide friendlier compile time errors.
>>> words ""
[]
>>> words "one line"
["one","line"]
>>> words "   >_<   "
[">_<"]
>>> words ("string words" :: String)
...
... 'words' works with 'Text', not 'String'.
Possible fixes:
1. Make sure OverloadedStrings extension is enabled.
2. Apply 'toText' to a single value.
3. Apply 'map toText' to the list value.
...
>>> words True
...
... 'words' works with 'Text'
But given: 'Bool'
...
> words "abc def ghi" ["abc","def","ghi"]
>>> words "Lorem ipsum\ndolor" ["Lorem","ipsum","dolor"]
words "Hello Foundation"