tokens isSpace = words
>>> putDoc (tupled (words "Lorem ipsum dolor")) (Lorem, ipsum, dolor)
ghci> is <- Streams.fromList ["Hello, world!"] >>= Streams.words ghci> replicateM 3 (Streams.read is) [Just "Hello,", Just "world!", Nothing]Note that this may increase the chunk size if the input contains extremely long words.
words :: Text -> [Text]but it was given a more complex type to provide friendlier compile time errors.
>>> words ""
[]
>>> words "one line"
["one","line"]
>>> words "   >_<   "
[">_<"]
>>> words ("string words" :: String)
...
... 'words' works with 'Text', not 'String'.
Possible fixes:
1. Make sure OverloadedStrings extension is enabled.
2. Apply 'toText' to a single value.
3. Apply 'map toText' to the list value.
...
>>> words True
...
... 'words' works with 'Text'
But given: 'Bool'
...
> words "abc def ghi" ["abc","def","ghi"]
>>> words "Lorem ipsum\ndolor" ["Lorem","ipsum","dolor"]
words = S.words A.write
>>> Stream.fold Fold.toList $ Unicode.words $ Stream.fromList "A newline\nis considered white space?" [fromList "A",fromList "newline",fromList "is",fromList "considered",fromList "white",fromList "space?"]
>>> words = Stream.wordsBy Char.isSpaceUsage:
>>> Stream.toList $ Unicode.words Fold.toList (Stream.fromList " ab cd ef ") ["ab","cd","ef"]Pre-release
words "Hello Foundation"