在 Windows 上读取 snappy parquet 文件导致 python 崩溃

Question

我无法在 Windows 上通过 pyarrow 读取活泼的镶木地板文件。import dask.dataframe as ddimport pandas as pdimport numpy as npdf = pd.DataFrame(np.random.randint(0,100,size=(15, 4)), columns=list('ABCD'))dd_df = dd.from_pandas(df, npartitions=1)dd_df.to_parquet("my_df.snappy.parquet", engine="pyarrow", compression="snappy")dd_df_copy = dd.read_parquet("my_df.snappy.parquet", engine="pyarrow")dd_df_copy.compute() #<--- This is where it crashes我已经使用 Python 3.8 在干净的 Anaconda 环境中复制了这个问题。创建环境后，我跑pip install "dask[complete]"了pip install pyarrow错误是：Problem signature:  Problem Event Name:   APPCRASH  Application Name: python.exe  Application Version:  3.8.3150.1013  Application Timestamp:    5ed53446  Fault Module Name:    arrow.dll  Fault Module Version: 0.0.0.0  Fault Module Timestamp:   5ebd3029  Exception Code:   c000001d  Exception Offset: 00000000007abfc7  OS Version:   6.3.9600.2.0.0.16.7  Locale ID:    1033  Additional Information 1: d8e4  Additional Information 2: d8e42c04b828d96accf490cd13472bea  Additional Information 3: aebe  Additional Information 4: aebe917bfb5c1b58e884baa1f9c3d3d2当我尝试使用时出现类似版本的崩溃conda -c conda-forge dask pyarrow：Problem signature:  Problem Event Name:   APPCRASH  Application Name: python.exe  Application Version:  3.8.3150.1013  Application Timestamp:    5ed53446  Fault Module Name:    arrow.dll  Fault Module Version: 0.0.0.0  Fault Module Timestamp:   5ecf56ac  Exception Code:   c000001d  Exception Offset: 0000000000521587  OS Version:   6.3.9600.2.0.0.16.7  Locale ID:    1033  Additional Information 1: e863  Additional Information 2: e8638a01b9fb70505b0604ef9b98f3c6  Additional Information 3: 1e47  Additional Information 4: 1e47c852f479606e071f3ea8f80878a1

Answer 1

从 2020 年 7 月 1 日起，更新软件包解决了这个问题。我认为是pyarrow更新解决了这个问题。