Firstly, I appreciate your work that is really useful. It is not difficult to set up. However, I found an issue while running at JupyterNotebook. You can see below the typed codes then following by error messages.
from deepcut import DeepcutTokenizer
tokenizer = DeepcutTokenizer(ngram_range=(1,1),
max_df = 1.0, min_df=0.0)
X = tokenizer.fit_tranform(['ฉันบินได้','ฉันกินข้าว','ฉันอยากบิน'])
print(tokenizer.vocabulary_)
X_test = tokenizer.transform(['ฉันกิน','ฉันไม่อยากบิน'])
print(X_test.shape)
tokenizer.save_model('tokenizer.pickle')
{'บิน': 0, 'ข้าว': 1, 'ได้': 2, 'อยาก': 3, 'ฉัน': 4, 'กิน': 5}
(2, 6)
AttributeError Traceback (most recent call last)
in
10 print(X_test.shape)
11
---> 12 tokenizer.save_model('tokenizer.pickle')
AttributeError: 'DeepcutTokenizer' object has no attribute 'save_model'
I tried several times it would be a typo. But it does not. I am a newbie in ML/DL. Please guide me to fix this problem. Thank you.
Regards,
Teddy
Firstly, I appreciate your work that is really useful. It is not difficult to set up. However, I found an issue while running at JupyterNotebook. You can see below the typed codes then following by error messages.
from deepcut import DeepcutTokenizer
tokenizer = DeepcutTokenizer(ngram_range=(1,1),
max_df = 1.0, min_df=0.0)
X = tokenizer.fit_tranform(['ฉันบินได้','ฉันกินข้าว','ฉันอยากบิน'])
print(tokenizer.vocabulary_)
X_test =
tokenizer.transform(['ฉันกิน','ฉันไม่อยากบิน'])print(X_test.shape)
tokenizer.save_model('tokenizer.pickle')
{'บิน': 0, 'ข้าว': 1, 'ได้': 2, 'อยาก': 3, 'ฉัน': 4, 'กิน': 5}
(2, 6)
AttributeError Traceback (most recent call last)
in
10 print(X_test.shape)
11
---> 12 tokenizer.save_model('tokenizer.pickle')
AttributeError: 'DeepcutTokenizer' object has no attribute 'save_model'
I tried several times it would be a typo. But it does not. I am a newbie in ML/DL. Please guide me to fix this problem. Thank you.
Regards,
Teddy