如何使用 ktrain for NER Offline？

Question

我按照这个笔记本 ( https://nbviewer.jupyter.org/github/amaiya/ktrain/blob/master/tutorials/tutorial-06-sequence-tagging.ipynb ) 训练了我的英语模型。我能够保存我的预训练模型并毫无问题地运行它。但是，我需要再次运行它但离线并且它不起作用，我知道我需要下载文件并执行类似于此处所做的操作。https://github.com/huggingface/transformers/issues/136但是，我无法理解我需要在哪里更改 ktrain 的设置。我运行这个：ktrain.load_predictor('Functions/my_english_nermodel')这是我得到的错误：Traceback (most recent call last):  File "Z:\Functions\NER.py", line 155, in load_bert    reloaded_predictor= ktrain.load_predictor('Z:/Functions/my_english_nermodel')  File "C:\Program Files\Python37\lib\site-packages\ktrain\core.py", line 1316, in load_predictor    preproc = pickle.load(f)  File "C:\Program Files\Python37\lib\site-packages\ktrain\text\ner\anago\preprocessing.py", line 76, in __setstate__    if self.te_model is not None: self.activate_transformer(self.te_model, layers=self.te_layers)  File "C:\Program Files\Python37\lib\site-packages\ktrain\text\ner\anago\preprocessing.py", line 100, in activate_transformer    self.te = TransformerEmbedding(model_name, layers=layers)  File "C:\Program Files\Python37\lib\site-packages\ktrain\text\preprocessor.py", line 1095, in __init__    self.tokenizer = self.tokenizer_type.from_pretrained(model_name)  File "C:\Program Files\Python37\lib\site-packages\transformers\tokenization_utils.py", line 903, in from_pretrained    return cls._from_pretrained(*inputs, **kwargs)  File "C:\Program Files\Python37\lib\site-packages\transformers\tokenization_utils.py", line 1008, in _from_pretrained    list(cls.vocab_files_names.values()),Process finished with exit code 1

Answer 1

更一般地，基于transformers的预训练模型被下载到<home_directory>/.cache/torch/transformers. 例如，在 Linux 上，这将是/home/<user_name>/.cache/torch/transformers.

如上面的答案所示，要在无法访问互联网的机器上重新加载ktrain （对于使用库中模型的模型），您需要将该文件夹中的模型文件复制到新机器上的相同位置。predictorktraintransformers

Answer 2

当 ktrain 通过互联网连接运行时，我找到了一个解决方案，它创建了一个文件夹： ''' C:\Users\lemolina.cache\torch\transformers ''' 我需要在没有访问权限的机器中复制相同的文件夹上网