How can I run this model on a more recent version of transformers?
Hey guys first of all I'd like to thank you for this wonderful model which from my pov is the absolute greatest OCR type model at parsing financial tables specifically, literally no other model comes even close in accuracy when it comes to extracting large numbers so thanks again for that.
So far I've been able to run this model on transformers 4.51.3 (using the recommended versions of all other various packages), as well as the more recent versions of vllm, but neither of these options is ideal for my workflow since vllm has extremely long startup times on my hardware and the recommended version of transformers is quite old and does not support the "transformers serve" feature .
Ideally I'd like to use this model as an openai compatible api endpoint while avoiding the usage of vllm, but unfortunately after testing multiple recent version of transformers from 4.53 to 5.2.0 I always end up getting various errors such as
"Parse_hyphen_v1_dot_1_hyphen_TC/77d31557a2b9ca4a9caadf685bad38e7d4edd10b/hf_nemotron_parse_modeling.py", line 199, in forward
past_key_values_length = past_key_values[0][0].shape[2] if past_key_values is not None else 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'shape'"
So I'm wondering if this model has been tested successfully on a recent transformers build and what environment do you recommend I use to get it working properly? Thanks a bunch.