Uninitialised weights warning when loading with Sentence Transformers
Hi all,
Thanks for the great work on this model, amazing work and performance for its size and MRL support is fantastic.
I have a question about a warning being emitted by sentence transformers when loading the model. I am getting the following:
Some weights of BertModel were not initialized from the model checkpoint at Snowflake/snowflake-arctic-embed-m-v1.5 and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
I think this may be a harmless warning but I wanted to check if this is expected behaviour.
Thanks.
It should be harmless but also should not happen. Are you perhaps initializing the model differently from the example snippet in the readme? We do suggest explicitly disabling the pooling layer in that example. I believe the v1.0 model may already have this setting in its config files, too. I'll look at migrating that setting from the sentence bert config json file for the 1.0 model to v1.5 when I get the chance, but also feel free to open a PR if getting this option automatically applied is urgent for you.
I get the same warning and I'm pretty sure I followed the example in the readme, including disabling the pooling layer.
The warning can indeed be safely ignored. However, it is odd that you still get the warning even with model_kwargs=dict(add_pooling_layer=False).
#5 should make it a bit easier: you won't have to manually specify model_kwargs=dict(add_pooling_layer=False) anymore, and you shouldn't get any warning.
- Tom Aarsen
Oh, it was my mistake, a base class was initialising the model with the wrong arguments, just to be overwritten with the correct ones later. Thank you!