The config.json seems incorrect
#7
by shoubing35 - opened
The config.json seems incorrect.
"model_type": "llama" should be "falcon"
"architectures": ["LlamaForCausalLM"] should be "architectures": ["FalconForCausalLM"]
shoubing35 changed discussion status to closed
shoubing35 changed discussion status to open
weights seem to be llama weights as well:
model.embed_tokens.weight
@shoubing35 - Falcon3 Architecture is compatible with llama architecture. We leverage modelling_llama.py of Hugging face for Falcon3 models. Hope it addresses your concern.