如何压缩迭代器？

Question

我有一个生成数据的迭代器，我想将其解压缩。import gziph = open('myfile.gz', 'rb')data = iter(lambda: h.read(1024), b'')gzip.decompress(data)我得到：Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/usr/lib/python3.6/gzip.py", line 531, in decompress    with GzipFile(fileobj=io.BytesIO(data)) as f:TypeError: a bytes-like object is required, not 'callable_iterator'如何解压迭代器？数据无法加载到内存中。

Answer 1

如何解压迭代器？

你不知道。gzip.decompress()不适用于任意迭代器。您需要将迭代器转换为字节流，以便gzip.decompress(). 我会先看看BytesIO.

Answer 2

import zlib

def unzip_iterable(data):

  decompressor = zlib.decompressobj(wbits=zlib.MAX_WBITS | 16)  # gzip format

  for chunk in data:

    yield decompressor.decompress(chunk)

h = open('myfile.gz', 'rb')

data = iter(lambda: h.read(1024), b'')

for chunk in unzip_iterable(data):

    print(len(chunk))