我有一个包含 2 列的数据框。第一列 (content_cleaned) 包含保存句子的行。第二列(有意义的)包含相关联的二进制标签。当我尝试标记 content_cleaned 列中的文本时,我遇到的问题是空格。到目前为止,这是我的代码:df = pd.read_csv(pathname, encoding = "ISO-8859-1")df = df[['content_cleaned', 'meaningful']]df = df.sample(frac=1)#Transposed columns into numpy arrays X = np.asarray(df[['content_cleaned']])y = np.asarray(df[['meaningful']])#Split into training and testing setX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=21) # Create tokenizertokenizer = Tokenizer(num_words=100) #No row has more than 100 words.#Tokenize the predictors (text)X_train = tokenizer.sequences_to_matrix(X_train.astype(np.int32), mode="binary")X_test = tokenizer.sequences_to_matrix(X_test.astype(np.int32), mode="binary")#Convert the labels to the binaryencoder = LabelBinarizer()encoder.fit(y_train) y_train = encoder.transform(y_train)y_test = encoder.transform(y_test)错误突出显示的代码行是:X_train = tokenizer.sequences_to_matrix(X_train.astype(np.int32), mode="binary")错误信息是:invalid literal for int() with base 10: "STX's better than reported quarter is likely to bode well for WDC results."“base 10:”之后的句子是包含文本的列中的行之一的示例。那将是我试图标记的例句。我被认为这是 NumPy 的问题,但我也确信这可能是我标记此文本数组的方法中的错误。任何帮助都会很棒!
添加回答
举报
0/150
提交
取消