Missing tie_word_embeddings in config.json causes incorrect weight tying

by louis-jan - opened Feb 12

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

-2

louis-jan

Feb 12

•

edited Feb 12

This is to fix issue with transformers 4.54+..<5 where it sets tie_word_embeddings=True which is not expected.

Update config.jsonc60e8083

louis-jan changed pull request title from Update config.json to Missing tie_word_embeddings in config.json causes incorrect weight tying Feb 12

jan-hq changed pull request status to merged Feb 12

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment