Fix #1196 - Gensim error when loading FastText

[jaksmid]

Gensim can load large fasttext model on Mac
Loading of binary fasttext models is faster
Fasttext vector size is correctly set
Vector_size is also saved when loading just the vector_file, which can be useful if you are not interested in training

[tmylk]

Travis tests re-ran after smart_open update

[tmylk]

What is the purpose of adding a new attribute vector_size to all KeyedVectors? isn't it always syn0.shape[0]? If necessary, maybe just add it to FastText class?

[jaksmid]

Thanks @tmylk for the comment. To my understanding, you would use FastText in the case you want to load both the vec and bin files of the fasttext. In case you just want to load the vectors you can use FastTextKeyedVectors. As you pointed out, you can use syn0.shape[0] which may not be that straightforward if you are not familiar with the gensim internals. So I thought it can be a good idea to add vector_size explicitly so people can figure out the right name of the attribute more easily.

By adding it to the FastText class you meant FastTextKeyedVectors class, right?
Thanks!

EDIT: It seems like it should be self.syn0.shape[1] as seen in keyedvectors - save_word2vec_format which kind of proves my points that it is not that staightforward to get it right :)

Aslo, there is currenty no override of load_word2vec_format in the FastTextKeydVector, so unless we increase the complexity of FastTextKeyedVector by adding this override we have no place to initialize vector_size is FastTextKeyedVector.

Looking forward to your suggestions.

[jaksmid]

Furthermore, you cannot just use FastTextKeyedVectors without FastText initialization (which needs both vec and bin) as __contains__ in FastTextKeyedVectors references self.min_n which is actually set in FastText load_model_params.

[tmylk]

Just indent changes requested. Glad it became a comprehensive fix!
Please add a note in the changelog.md as well.

[tmylk]

Please use hanging indent

[tmylk]

Hanging indent preferred

[jaksmid]

Thanks for reviewing the code.
I incorporated the code review and merged with the current develop.

It seems like the build for python 2 stalled, could you rerun it @tmylk please?

[tmylk]

These tests are know to occasionally fail but it's the first time they fail constantly. Will disable them in the main branch soon.

[jaksmid]

Ok. Let me know if I can help with anything.