about gpu usage in CoreML
Hello, thank you for this great work! It works really well in macOS M3 CPU (3 seconds). I tried to run it locally via CoreML, but it's not using GPU. Is this related to ONNX? Have you tested it with CoreML?
My concern is that fastVLM's ONNX operations might not be supported by CoreML's neural engine
FYI: my code for test app. CUDA works fine though
Okay, I tried to create logs about that issue. Maybe give some additional insights
The most critical log messages indicating the problem are:
For vision_encoder.onnx:
INFO ort::logging: CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 794 number of nodes supported by CoreML: 0
For embed_tokens.onnx, we see a specific reason for the failure:
WARN ort::logging: CoreML does not support input dim > 16384. Input:weight, shape: {151680,896}
INFO ort::logging: CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 1 number of nodes supported by CoreML: 0
For decoder_model_merged.onnx:
INFO ort::logging: CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 445 number of nodes supported by CoreML: 0
Full logs:
(Cant upload my txt file but you can find it on here)
https://github.com/altunenes/calcarine/pull/2#issuecomment-3254128556
Creating vertex buffer
Creating render pipeline
[2m2025-09-04T14:30:04.861767Z[0m [33m WARN[0m [2megui_wgpu::renderer[0m[2m:[0m Detected a linear (sRGBA aware) framebuffer Bgra8UnormSrgb. egui prefers Rgba8Unorm or Bgra8Unorm
🔧 Initializing FastVLM...
🔍 Searching for FastVLM models...
[2m2025-09-04T14:30:04.862363Z[0m [32m INFO[0m [2mcalcarine[0m[2m:[0m 🔍 Searching for FastVLM models...
🔍 Checking path "/Users/enes/Library/Caches/Calcarine/models": ✅ Found all models
[2m2025-09-04T14:30:04.862757Z[0m [32m INFO[0m [2mcalcarine[0m[2m:[0m 🔍 Checking path "/Users/enes/Library/Caches/Calcarine/models": ✅ Found all models
[2m2025-09-04T14:30:04.862761Z[0m [32m INFO[0m [2mcalcarine[0m[2m:[0m Found FastVLM data directory at: "/Users/enes/Library/Caches/Calcarine/models"
[2m2025-09-04T14:30:04.863516Z[0m [32m INFO[0m [2mcalcarine::fastvlm::fastvlm[0m[2m:[0m Initializing FastVLM...
[2m2025-09-04T14:30:04.863670Z[0m [32m INFO[0m [2mcalcarine::fastvlm::fastvlm[0m[2m:[0m Using CoreML execution provider.
[2m2025-09-04T14:30:05.010340Z[0m [34mDEBUG[0m [2mort::environment[0m[2m:[0m Environment created [3menv_ptr[0m[2m=[0m"0x60001945daf0"
[2m2025-09-04T14:30:05.010370Z[0m [32m INFO[0m [2mcalcarine::fastvlm::fastvlm[0m[2m:[0m Loading model: vision_encoder.onnx
[2m2025-09-04T14:30:05.010699Z[0m [32m INFO[0m [2mort::execution_providers[0m[2m:[0m Successfully registered CoreMLExecutionProvider [3msource[0m[2m=[0menvironment
[2m2025-09-04T14:30:05.011374Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Session Options { execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath:"" enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:-1 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:3 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str: set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str: set_denormal_as_zero: 0 } use_per_session_threads:1 thread_pool_allow_spinning:1 use_deterministic_compute:0 ep_selection_policy:0 config_options: { ep.coremlexecutionprovider.RequireStaticInputShapes: 1 ep.coremlexecutionprovider.MLComputeUnits: CPUAndGPU } }
[2m2025-09-04T14:30:05.011383Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Flush-to-zero and denormal-as-zero are off
[2m2025-09-04T14:30:05.011387Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Creating and using per session threadpools since use_per_session_threads_ is true
[2m2025-09-04T14:30:05.011391Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Dynamic block base set to 0
[2m2025-09-04T14:30:05.167608Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Initializing session.
[2m2025-09-04T14:30:05.167631Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Adding default CPU execution provider.
[2m2025-09-04T14:30:05.167977Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
[2m2025-09-04T14:30:05.177278Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m This model does not have any local functions defined. AOT Inlining is not performed
[2m2025-09-04T14:30:05.177352Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
[2m2025-09-04T14:30:05.177593Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer Level1_RuleBasedTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.177644Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer DoubleQDQPairsRemover modified: 0 with status: OK
[2m2025-09-04T14:30:05.177763Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConstantSharing modified: 0 with status: OK
[2m2025-09-04T14:30:05.178062Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer CommonSubexpressionElimination modified: 0 with status: OK
[2m2025-09-04T14:30:05.244464Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConstantFolding modified: 1 with status: OK
[2m2025-09-04T14:30:05.246784Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.3.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246797Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.0.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246802Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.3.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246806Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.1.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246812Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.2.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246817Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.0.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246821Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.0.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246825Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.1.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246828Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.0.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246833Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'mm_projector.0.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246837Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.1.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246841Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.2.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246844Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.1.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246847Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.0.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246850Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.1.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246854Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.1.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246856Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.1.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246860Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'mm_projector.2.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246872Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.0.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246874Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.10.1.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246877Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.2.token_mixer.proj.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246880Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.3.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246882Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.0.norm.bias'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246885Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'node_Unsqueeze_996_axes'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246888Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.2.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246891Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.3.token_mixer.qkv.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.246893Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'vision_model.vision_tower.model.network.7.0.norm.weight'. It is no longer used by any node.
[2m2025-09-04T14:30:05.248596Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulAddFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.251394Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ReshapeFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.251406Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer FreeDimensionOverrideTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.251412Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GeluFusionL1 modified: 0 with status: OK
[2m2025-09-04T14:30:05.251488Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer LayerNormFusionL1 modified: 0 with status: OK
[2m2025-09-04T14:30:05.251535Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQPropagationTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.251588Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer WeightBiasQuantization modified: 0 with status: OK
[2m2025-09-04T14:30:05.251629Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
[2m2025-09-04T14:30:05.251667Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer RocmBlasAltImpl modified: 0 with status: OK
[2m2025-09-04T14:30:05.252305Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer TransposeOptimizer modified: 1 with status: OK
[2m2025-09-04T14:30:05.254559Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_36_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254563Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_32_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254566Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_29_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254570Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_27_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254573Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_19_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254581Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_18_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254584Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_15_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254587Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_6_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254589Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_5_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254592Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_2_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254595Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_13_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254599Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_20_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254602Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_33_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254605Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_26_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254609Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_8_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254615Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_11_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254619Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_4_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254624Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_25_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254627Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_39_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254631Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_22_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254636Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254638Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_12_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254642Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_34_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254646Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing NodeArg 'Transpose_token_0_out0'. It is no longer used by any node.
[2m2025-09-04T14:30:05.254792Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer Level1_RuleBasedTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.254833Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer DoubleQDQPairsRemover modified: 0 with status: OK
[2m2025-09-04T14:30:05.255116Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer CommonSubexpressionElimination modified: 0 with status: OK
[2m2025-09-04T14:30:05.255166Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConstantFolding modified: 0 with status: OK
[2m2025-09-04T14:30:05.255207Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulAddFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.255251Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ReshapeFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.255254Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer FreeDimensionOverrideTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.255261Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GeluFusionL1 modified: 0 with status: OK
[2m2025-09-04T14:30:05.255305Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer LayerNormFusionL1 modified: 0 with status: OK
[2m2025-09-04T14:30:05.255349Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQPropagationTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.255398Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer WeightBiasQuantization modified: 0 with status: OK
[2m2025-09-04T14:30:05.255438Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
[2m2025-09-04T14:30:05.255477Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer RocmBlasAltImpl modified: 0 with status: OK
[2m2025-09-04T14:30:05.257149Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 794 number of nodes supported by CoreML: 0
[2m2025-09-04T14:30:05.259573Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer Level2_RuleBasedTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.259824Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer TransposeOptimizer_CPUExecutionProvider modified: 0 with status: OK
[2m2025-09-04T14:30:05.259956Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQSelectorActionTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.259995Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GemmActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.260042Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulIntegerToFloatFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.260080Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer DynamicQuantizeMatMulFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.260170Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConvActivationFusion modified: 1 with status: OK
[2m2025-09-04T14:30:05.262507Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GeluFusionL2 modified: 0 with status: OK
[2m2025-09-04T14:30:05.262511Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer LayerNormFusionL2 modified: 0 with status: OK
[2m2025-09-04T14:30:05.262556Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer SimplifiedLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.262596Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer AttentionFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.262790Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer EmbedLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.262912Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GatherSliceToSplitFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.262952Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GatherToSliceFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.263064Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatmulTransposeFusion modified: 1 with status: OK
[2m2025-09-04T14:30:05.265447Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265489Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GroupQueryAttentionFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265528Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer SkipLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265598Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer FastGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265659Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QuickGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265708Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasSoftmaxFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265756Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasDropoutFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.265819Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulScaleFusion modified: 1 with status: OK
[2m2025-09-04T14:30:05.267981Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Removing initializer 'val_670'. It is no longer used by any node.
[2m2025-09-04T14:30:05.268081Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268137Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulNBitsFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268179Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQFinalCleanupTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.268222Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer Level2_RuleBasedTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.268353Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQSelectorActionTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.268393Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GemmActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268439Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulIntegerToFloatFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268478Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer DynamicQuantizeMatMulFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268557Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConvActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268636Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GeluFusionL2 modified: 0 with status: OK
[2m2025-09-04T14:30:05.268639Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer LayerNormFusionL2 modified: 0 with status: OK
[2m2025-09-04T14:30:05.268685Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer SimplifiedLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268725Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer AttentionFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268764Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer EmbedLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268861Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GatherSliceToSplitFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268899Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GatherToSliceFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268949Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatmulTransposeFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.268998Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269038Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer GroupQueryAttentionFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269077Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer SkipLayerNormFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269145Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer FastGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269205Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QuickGeluFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269255Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasSoftmaxFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269295Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer BiasDropoutFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269351Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulScaleFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269390Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269446Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MatMulNBitsFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.269488Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer QDQFinalCleanupTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.269654Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer NhwcTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.269717Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer ConvAddActivationFusion modified: 0 with status: OK
[2m2025-09-04T14:30:05.270021Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer RemoveDuplicateCastTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.270030Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer CastFloat16Transformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.270050Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m GraphTransformer MemcpyTransformer modified: 0 with status: OK
[2m2025-09-04T14:30:05.272922Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Use DeviceBasedPartition as default
[2m2025-09-04T14:30:05.274741Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Saving initialized tensors.
[2m2025-09-04T14:30:05.274792Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:9 (requested) num_bytes: 147456 (actual) rounded_bytes:147456
[2m2025-09-04T14:30:05.274796Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 1048576 bytes.
[2m2025-09-04T14:30:05.274799Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 1048576
[2m2025-09-04T14:30:05.274802Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x158320000 to 0x158420000
[2m2025-09-04T14:30:05.275024Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:0 (requested) num_bytes: 8 (actual) rounded_bytes:256
[2m2025-09-04T14:30:05.275027Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 2097152 bytes.
[2m2025-09-04T14:30:05.275030Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 3145728
[2m2025-09-04T14:30:05.275033Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x158428000 to 0x158628000
[2m2025-09-04T14:30:05.275043Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:15 (requested) num_bytes: 9437184 (actual) rounded_bytes:9437184
[2m2025-09-04T14:30:05.275047Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 16777216 bytes.
[2m2025-09-04T14:30:05.275049Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 19922944
[2m2025-09-04T14:30:05.275051Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x11d404000 to 0x11e404000
[2m2025-09-04T14:30:05.276170Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:13 (requested) num_bytes: 2359296 (actual) rounded_bytes:2359296
[2m2025-09-04T14:30:05.276175Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 16777216 bytes.
[2m2025-09-04T14:30:05.276177Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 36700160
[2m2025-09-04T14:30:05.276180Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x11b900000 to 0x11c900000
[2m2025-09-04T14:30:05.278033Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:15 (requested) num_bytes: 9437184 (actual) rounded_bytes:9437184
[2m2025-09-04T14:30:05.278043Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 33554432 bytes.
[2m2025-09-04T14:30:05.278046Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 70254592
[2m2025-09-04T14:30:05.278057Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x16ec20000 to 0x170c20000
[2m2025-09-04T14:30:05.281624Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:13 (requested) num_bytes: 2359296 (actual) rounded_bytes:2359296
[2m2025-09-04T14:30:05.281639Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 67108864 bytes.
[2m2025-09-04T14:30:05.281642Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 137363456
[2m2025-09-04T14:30:05.281645Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x170c24000 to 0x174c24000
[2m2025-09-04T14:30:05.291192Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:17 (requested) num_bytes: 37748736 (actual) rounded_bytes:37748736
[2m2025-09-04T14:30:05.291209Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 134217728 bytes.
[2m2025-09-04T14:30:05.291212Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 271581184
[2m2025-09-04T14:30:05.291215Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x174c28000 to 0x17cc28000
[2m2025-09-04T14:30:05.308075Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:13 (requested) num_bytes: 2359296 (actual) rounded_bytes:2359296
[2m2025-09-04T14:30:05.308094Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 268435456 bytes.
[2m2025-09-04T14:30:05.308096Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 540016640
[2m2025-09-04T14:30:05.308099Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x300000000 to 0x310000000
[2m2025-09-04T14:30:05.347649Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extending BFCArena for Cpu. bin_num:4 (requested) num_bytes: 6144 (actual) rounded_bytes:6144
[2m2025-09-04T14:30:05.347671Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Extended allocation by 536870912 bytes.
[2m2025-09-04T14:30:05.347674Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Total allocated bytes: 1076887552
[2m2025-09-04T14:30:05.347678Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Allocated memory at 0x310004000 to 0x330004000
[2m2025-09-04T14:30:05.353026Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Done saving initialized tensors
[2m2025-09-04T14:30:05.355137Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 2359296
[2m2025-09-04T14:30:05.356035Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 2359296
[2m2025-09-04T14:30:05.356881Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 2359296
[2m2025-09-04T14:30:05.357694Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 2359296
[2m2025-09-04T14:30:05.358631Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 9437184
[2m2025-09-04T14:30:05.362045Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 9437184
[2m2025-09-04T14:30:05.365504Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 3211264
[2m2025-09-04T14:30:05.366590Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 7077888
[2m2025-09-04T14:30:05.369604Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 7077888
[2m2025-09-04T14:30:05.372650Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 7077888
[2m2025-09-04T14:30:05.375564Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 7077888
[2m2025-09-04T14:30:05.378469Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 28311552
[2m2025-09-04T14:30:05.389013Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 28311552
[2m2025-09-04T14:30:05.398988Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Reserving memory in BFCArena for Cpu size: 11010048
[2m2025-09-04T14:30:05.403423Z[0m [32m INFO[0m [2mort::logging[0m[2m:[0m Session successfully initialized.
[2m2025-09-04T14:30:05.403455Z[0m [32m INFO[0m [2mcalcarine::fastvlm::fastvlm[0m[2m:[0m Loading model: embed_tokens.onnx
[2m2025-09-04T14:30:05.403498Z[0m [32m INFO[0m [2mort::execution_providers[0m[2m:[0m Successfully registered CoreMLExecutionProvider