Pronoun lemmas inconsistent with docs

[nickdavidhaynes]

Unclear whether this is a bug or an issue with the docs, but when I lemmatize a pronoun in 1.3.0, I'm not seeing the -PRON- token a la https://spacy.io/docs/api/annotation#lemmatization:

>>> sentence = nlp('He is the man.')
>>> sentence[0].orth_
'He'
>>> sentence[0].lemma_
'he'
>>> sentence[0].pos_
'PRON'

My Environment

• El Capitan
• Python 3.5.2
• spaCy 1.3.0

[honnibal]

Thanks! Looks like I broke something in the morphological analysis.

[honnibal]

Turns out this has been the behaviour for a long time. Interesting.

The good news is that this means the model is correct — I was worried that I broke the calculation of these lemmas, and that meant the weights in the model we've all been using were wrong. That's not the case.

We'll fix this bug in version 2.0, because we don't want to switch the behaviour in the code, and invalidate the model.

[nickdavidhaynes]

Thanks! Definitely glad to hear the model wasn't wrong.

[honnibal]

Fixed in v1.7

[drorata]

Should I be surprised that sentence[0].lemma_ yields 'Er' given:

nlp = spacy.load('de')
sentence = nlp('Er is wieder da.')

I was expecting '-PRON-' as well.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.