What is the function of the weights in out_llm.safetensors?
Hi and thank you for the release. Just wondering, what is the function of the weights in out_llm.safetensors? I can't see any description or documentation about this anywhere. I'm asking because I am curious if this bit needs to be tuned for fine tuning.
Thank you again
Hi @deathknight0 Thank you for the interest in our model and great question. The out_llm.safetensors is the BPE classification head of the CTC encoder and is used if you want to get the CTC hypothesis directly (bypassing the adapter and LLM) as was done for speculative decoding here: https://colab.research.google.com/github/ibm-granite/granite-speech-models/blob/main/notebooks/speculative_decoding_bpe.ipynb You don't need it for finetuning and inference with the full granite speech model.