in this func:
func (seg *Segmenter) Reader(reader *bufio.Reader, files ...string) error
those code lines:
if fsErr != nil {
if fsErr == io.EOF {
...
}
must put after :
...
seg.Dict.AddToken(token)
Otherwise,last line of dictory file will be missed.exclude the last line is empty.
in this func:
func (seg *Segmenter) Reader(reader *bufio.Reader, files ...string) error
those code lines:
if fsErr != nil {
if fsErr == io.EOF {
...
}
must put after :
...
seg.Dict.AddToken(token)
Otherwise,last line of dictory file will be missed.exclude the last line is empty.