bitonic · GitHub

@bitonic

@bitonic

Chains of `B.append`s were being created by repeated calls to
`demandInput`.
Try the following program, which writes and read 100MB, to appreciate
the difference:
```
import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as BSL
import Data.Binary (encode, decode)
import Data.Char (ord)
main :: IO ()
main = do
  let inBs = BS.replicate 100000000 (fromIntegral $ ord 'a')
  BSL.writeFile "bs.bin" (encode inBs)
  putStrLn "writing done"
  bin <- BSL.readFile "bs.bin"
  -- This takes around 30 seconds and causes more than 10GB to be
  -- allocated.
  let outBs = decode bin
  print $ inBs == outBs
```

@bitonic

...by making the common case (when the input is big enough) non-recursive.

@kolmodin

hvr added a commit to ghc/ghc that referenced this pull request

Jun 1, 2015
Quoting the changelog, this pulls in the following fixes:
binary-0.7.5.0
--------------
- Fix performance bug that was noticable when you get a big strict ByteString
  and the input to the decoder consists of many small chunks.
    - haskell/binary#73
    - haskell/binary#76
- Fix memory leak when decoding Double and Float.
    - Commit 497a181c083fa9faf7fa3aa64d1d8deb9ac76ecb
- We now require QuickCheck >= 2.8. Remove our version of arbitrarySizedNatural.
binary-0.7.4.0
--------------
- Some invalid UTF-8 strings caused an exception when decoded. Those errors will
  now now fail in the Get monad instead. See issue 70.
  Patch contributed by @ttuegel.

hvr added a commit to ghc/ghc that referenced this pull request

Jun 9, 2015
Quoting the changelog, this pulls in the following fixes:
binary-0.7.5.0
--------------
- Fix performance bug that was noticable when you get a big strict ByteString
  and the input to the decoder consists of many small chunks.
    - haskell/binary#73
    - haskell/binary#76
- Fix memory leak when decoding Double and Float.
    - Commit 497a181c083fa9faf7fa3aa64d1d8deb9ac76ecb
- We now require QuickCheck >= 2.8. Remove our version of arbitrarySizedNatural.
binary-0.7.4.0
--------------
- Some invalid UTF-8 strings caused an exception when decoded. Those errors will
  now now fail in the Get monad instead. See issue 70.
  Patch contributed by @ttuegel.
(cherry picked from commit  7dd0ea7 )

Closed

Read the original on github.com ↗