eval_custom_0.5.log

2026-03-25 17:23:23.070164: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2026-03-25 17:23:23.087786: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.10/dist-packages/torch/lib:/usr/local/lib/python3.10/dist-packages/torch_tensorrt/lib:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2026-03-25 17:23:23.087841: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2026-03-25 17:23:30.251344: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-03-25 17:23:30.287869: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2026-03-25 17:23:30.287965: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: 24c964746905d416ce09d045f9a06f23-taskrole1-0
2026-03-25 17:23:30.287986: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: 24c964746905d416ce09d045f9a06f23-taskrole1-0
2026-03-25 17:23:30.288155: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 580.82.7
2026-03-25 17:23:30.288179: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 580.82.7
2026-03-25 17:23:30.288188: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 580.82.7
2026-03-25 17:23:30.805683: W tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
warming up TensorFlow...

  0%|          | 0/1 [00:00<?, ?it/s]2026-03-25 17:23:31.768251: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled

100%|██████████| 1/1 [00:01<00:00,  1.78s/it]
100%|██████████| 1/1 [00:01<00:00,  1.78s/it]
computing reference batch activations...

  0%|          | 0/157 [00:00<?, ?it/s]
  1%|          | 1/157 [00:01<04:56,  1.90s/it]
  1%|▏         | 2/157 [00:03<04:35,  1.78s/it]
  2%|▏         | 3/157 [00:05<04:36,  1.80s/it]
  3%|▎         | 4/157 [00:07<04:59,  1.96s/it]
  3%|▎         | 5/157 [00:09<05:10,  2.05s/it]
  4%|▍         | 6/157 [00:11<04:50,  1.92s/it]
  4%|▍         | 7/157 [00:13<04:36,  1.85s/it]
  5%|▌         | 8/157 [00:14<04:27,  1.80s/it]
  6%|▌         | 9/157 [00:16<04:35,  1.86s/it]
  6%|▋         | 10/157 [00:18<04:36,  1.88s/it]
  7%|▋         | 11/157 [00:20<04:31,  1.86s/it]
  8%|▊         | 12/157 [00:22<04:39,  1.93s/it]
  8%|▊         | 13/157 [00:24<04:32,  1.89s/it]
  9%|▉         | 14/157 [00:26<04:30,  1.89s/it]
 10%|▉         | 15/157 [00:28<04:19,  1.83s/it]
 10%|█         | 16/157 [00:29<04:12,  1.79s/it]
 11%|█         | 17/157 [00:31<04:04,  1.75s/it]
 11%|█▏        | 18/157 [00:33<03:59,  1.72s/it]
 12%|█▏        | 19/157 [00:34<03:56,  1.72s/it]
 13%|█▎        | 20/157 [00:36<03:51,  1.69s/it]
 13%|█▎        | 21/157 [00:38<03:49,  1.68s/it]
 14%|█▍        | 22/157 [00:39<03:48,  1.69s/it]
 15%|█▍        | 23/157 [00:41<03:43,  1.67s/it]
 15%|█▌        | 24/157 [00:43<03:42,  1.67s/it]
 16%|█▌        | 25/157 [00:44<03:38,  1.66s/it]
 17%|█▋        | 26/157 [00:46<03:37,  1.66s/it]
 17%|█▋        | 27/157 [00:48<03:45,  1.73s/it]
 18%|█▊        | 28/157 [00:49<03:42,  1.72s/it]
 18%|█▊        | 29/157 [00:51<03:40,  1.72s/it]
 19%|█▉        | 30/157 [00:53<03:37,  1.72s/it]
 20%|█▉        | 31/157 [00:55<03:38,  1.74s/it]
 20%|██        | 32/157 [00:56<03:36,  1.73s/it]
 21%|██        | 33/157 [00:58<03:32,  1.71s/it]
 22%|██▏       | 34/157 [01:00<03:27,  1.69s/it]
 22%|██▏       | 35/157 [01:02<03:29,  1.72s/it]
 23%|██▎       | 36/157 [01:03<03:30,  1.74s/it]
 24%|██▎       | 37/157 [01:05<03:24,  1.70s/it]
 24%|██▍       | 38/157 [01:07<03:22,  1.70s/it]
 25%|██▍       | 39/157 [01:08<03:20,  1.70s/it]
 25%|██▌       | 40/157 [01:10<03:18,  1.70s/it]
 26%|██▌       | 41/157 [01:12<03:17,  1.70s/it]
 27%|██▋       | 42/157 [01:13<03:15,  1.70s/it]
 27%|██▋       | 43/157 [01:15<03:11,  1.68s/it]
 28%|██▊       | 44/157 [01:17<03:09,  1.68s/it]
 29%|██▊       | 45/157 [01:18<03:11,  1.71s/it]
 29%|██▉       | 46/157 [01:20<03:09,  1.71s/it]
 30%|██▉       | 47/157 [01:22<03:07,  1.70s/it]
 31%|███       | 48/157 [01:24<03:05,  1.70s/it]
 31%|███       | 49/157 [01:25<03:01,  1.68s/it]
 32%|███▏      | 50/157 [01:27<03:03,  1.71s/it]
 32%|███▏      | 51/157 [01:29<03:01,  1.71s/it]
 33%|███▎      | 52/157 [01:30<02:58,  1.70s/it]
 34%|███▍      | 53/157 [01:32<02:53,  1.67s/it]
 34%|███▍      | 54/157 [01:34<02:51,  1.66s/it]
 35%|███▌      | 55/157 [01:35<02:47,  1.64s/it]
 36%|███▌      | 56/157 [01:37<02:46,  1.65s/it]
 36%|███▋      | 57/157 [01:39<02:44,  1.64s/it]
 37%|███▋      | 58/157 [01:40<02:43,  1.66s/it]
 38%|███▊      | 59/157 [01:42<02:43,  1.66s/it]
 38%|███▊      | 60/157 [01:44<02:40,  1.65s/it]
 39%|███▉      | 61/157 [01:45<02:42,  1.69s/it]
 39%|███▉      | 62/157 [01:47<02:38,  1.67s/it]
 40%|████      | 63/157 [01:49<02:35,  1.65s/it]
 41%|████      | 64/157 [01:50<02:34,  1.66s/it]
 41%|████▏     | 65/157 [01:52<02:33,  1.67s/it]
 42%|████▏     | 66/157 [01:54<02:32,  1.68s/it]
 43%|████▎     | 67/157 [01:55<02:32,  1.69s/it]
 43%|████▎     | 68/157 [01:57<02:31,  1.70s/it]
 44%|████▍     | 69/157 [01:59<02:29,  1.70s/it]
 45%|████▍     | 70/157 [02:00<02:27,  1.70s/it]
 45%|████▌     | 71/157 [02:02<02:25,  1.69s/it]
 46%|████▌     | 72/157 [02:04<02:21,  1.67s/it]
 46%|████▋     | 73/157 [02:05<02:20,  1.67s/it]
 47%|████▋     | 74/157 [02:07<02:17,  1.66s/it]
 48%|████▊     | 75/157 [02:09<02:14,  1.64s/it]
 48%|████▊     | 76/157 [02:10<02:13,  1.65s/it]
 49%|████▉     | 77/157 [02:12<02:11,  1.64s/it]
 50%|████▉     | 78/157 [02:14<02:13,  1.69s/it]
 50%|█████     | 79/157 [02:15<02:09,  1.66s/it]
 51%|█████     | 80/157 [02:17<02:08,  1.67s/it]
 52%|█████▏    | 81/157 [02:19<02:04,  1.63s/it]
 52%|█████▏    | 82/157 [02:20<02:03,  1.64s/it]
 53%|█████▎    | 83/157 [02:22<02:00,  1.63s/it]
 54%|█████▎    | 84/157 [02:23<02:00,  1.65s/it]
 54%|█████▍    | 85/157 [02:25<01:59,  1.66s/it]
 55%|█████▍    | 86/157 [02:27<01:59,  1.68s/it]
 55%|█████▌    | 87/157 [02:29<01:57,  1.68s/it]
 56%|█████▌    | 88/157 [02:30<01:56,  1.69s/it]
 57%|█████▋    | 89/157 [02:32<01:54,  1.69s/it]
 57%|█████▋    | 90/157 [02:34<01:53,  1.69s/it]
 58%|█████▊    | 91/157 [02:35<01:50,  1.67s/it]
 59%|█████▊    | 92/157 [02:37<01:48,  1.68s/it]
 59%|█████▉    | 93/157 [02:39<01:47,  1.68s/it]
 60%|█████▉    | 94/157 [02:40<01:44,  1.66s/it]
 61%|██████    | 95/157 [02:42<01:43,  1.67s/it]
 61%|██████    | 96/157 [02:44<01:39,  1.62s/it]
 62%|██████▏   | 97/157 [02:45<01:38,  1.64s/it]
 62%|██████▏   | 98/157 [02:47<01:45,  1.78s/it]
 63%|██████▎   | 99/157 [02:49<01:43,  1.79s/it]
 64%|██████▎   | 100/157 [02:51<01:40,  1.76s/it]
 64%|██████▍   | 101/157 [02:52<01:37,  1.74s/it]
 65%|██████▍   | 102/157 [02:54<01:33,  1.70s/it]
 66%|██████▌   | 103/157 [02:56<01:31,  1.69s/it]
 66%|██████▌   | 104/157 [02:57<01:29,  1.68s/it]
 67%|██████▋   | 105/157 [02:59<01:27,  1.68s/it]
 68%|██████▊   | 106/157 [03:01<01:24,  1.66s/it]
 68%|██████▊   | 107/157 [03:02<01:22,  1.65s/it]
 69%|██████▉   | 108/157 [03:04<01:21,  1.66s/it]
 69%|██████▉   | 109/157 [03:06<01:18,  1.64s/it]
 70%|███████   | 110/157 [03:07<01:19,  1.68s/it]
 71%|███████   | 111/157 [03:09<01:17,  1.68s/it]
 71%|███████▏  | 112/157 [03:11<01:15,  1.69s/it]
 72%|███████▏  | 113/157 [03:12<01:14,  1.69s/it]
 73%|███████▎  | 114/157 [03:14<01:11,  1.66s/it]
 73%|███████▎  | 115/157 [03:16<01:09,  1.65s/it]
 74%|███████▍  | 116/157 [03:17<01:08,  1.66s/it]
 75%|███████▍  | 117/157 [03:19<01:06,  1.65s/it]
 75%|███████▌  | 118/157 [03:21<01:04,  1.66s/it]
 76%|███████▌  | 119/157 [03:22<01:02,  1.64s/it]
 76%|███████▋  | 120/157 [03:24<01:00,  1.63s/it]
 77%|███████▋  | 121/157 [03:25<00:58,  1.62s/it]
 78%|███████▊  | 122/157 [03:27<00:56,  1.62s/it]
 78%|███████▊  | 123/157 [03:29<00:55,  1.64s/it]
 79%|███████▉  | 124/157 [03:31<00:54,  1.66s/it]
 80%|███████▉  | 125/157 [03:32<00:53,  1.67s/it]
 80%|████████  | 126/157 [03:34<00:52,  1.70s/it]
 81%|████████  | 127/157 [03:36<00:50,  1.68s/it]
 82%|████████▏ | 128/157 [03:37<00:48,  1.66s/it]
 82%|████████▏ | 129/157 [03:39<00:46,  1.67s/it]
 83%|████████▎ | 130/157 [03:41<00:45,  1.68s/it]
 83%|████████▎ | 131/157 [03:42<00:43,  1.66s/it]
 84%|████████▍ | 132/157 [03:44<00:41,  1.66s/it]
 85%|████████▍ | 133/157 [03:45<00:39,  1.65s/it]
 85%|████████▌ | 134/157 [03:47<00:38,  1.66s/it]
 86%|████████▌ | 135/157 [03:49<00:36,  1.65s/it]
 87%|████████▋ | 136/157 [03:51<00:34,  1.66s/it]
 87%|████████▋ | 137/157 [03:52<00:33,  1.67s/it]
 88%|████████▊ | 138/157 [03:54<00:31,  1.68s/it]
 89%|████████▊ | 139/157 [03:56<00:29,  1.66s/it]
 89%|████████▉ | 140/157 [03:57<00:28,  1.66s/it]
 90%|████████▉ | 141/157 [03:59<00:26,  1.68s/it]
 90%|█████████ | 142/157 [04:00<00:24,  1.62s/it]
 91%|█████████ | 143/157 [04:02<00:22,  1.62s/it]
 92%|█████████▏| 144/157 [04:04<00:21,  1.64s/it]
 92%|█████████▏| 145/157 [04:05<00:19,  1.66s/it]
 93%|█████████▎| 146/157 [04:07<00:18,  1.65s/it]
 94%|█████████▎| 147/157 [04:09<00:16,  1.66s/it]
 94%|█████████▍| 148/157 [04:10<00:14,  1.64s/it]
 95%|█████████▍| 149/157 [04:12<00:12,  1.60s/it]
 96%|█████████▌| 150/157 [04:13<00:11,  1.61s/it]
 96%|█████████▌| 151/157 [04:15<00:09,  1.60s/it]
 97%|█████████▋| 152/157 [04:17<00:08,  1.63s/it]
 97%|█████████▋| 153/157 [04:18<00:06,  1.65s/it]
 98%|█████████▊| 154/157 [04:20<00:04,  1.64s/it]
 99%|█████████▊| 155/157 [04:22<00:03,  1.62s/it]
 99%|█████████▉| 156/157 [04:23<00:01,  1.64s/it]
100%|██████████| 157/157 [04:24<00:00,  1.33s/it]
100%|██████████| 157/157 [04:24<00:00,  1.68s/it]
computing/reading reference batch statistics...
computing sample batch activations...

  0%|          | 0/313 [00:00<?, ?it/s]
  0%|          | 1/313 [00:01<07:26,  1.43s/it]
  1%|          | 2/313 [00:03<07:52,  1.52s/it]
  1%|          | 3/313 [00:04<08:14,  1.60s/it]
  1%|▏         | 4/313 [00:06<08:14,  1.60s/it]
  2%|▏         | 5/313 [00:07<08:12,  1.60s/it]
  2%|▏         | 6/313 [00:09<08:31,  1.66s/it]
  2%|▏         | 7/313 [00:11<08:24,  1.65s/it]
  3%|▎         | 8/313 [00:12<08:18,  1.64s/it]
  3%|▎         | 9/313 [00:14<08:32,  1.69s/it]
  3%|▎         | 10/313 [00:16<08:23,  1.66s/it]
  4%|▎         | 11/313 [00:18<08:23,  1.67s/it]
  4%|▍         | 12/313 [00:19<08:18,  1.65s/it]
  4%|▍         | 13/313 [00:21<08:19,  1.67s/it]
  4%|▍         | 14/313 [00:23<08:19,  1.67s/it]
  5%|▍         | 15/313 [00:24<08:13,  1.65s/it]
  5%|▌         | 16/313 [00:26<08:23,  1.69s/it]
  5%|▌         | 17/313 [00:28<08:14,  1.67s/it]
  6%|▌         | 18/313 [00:29<08:13,  1.67s/it]
  6%|▌         | 19/313 [00:31<07:58,  1.63s/it]
  6%|▋         | 20/313 [00:32<07:54,  1.62s/it]
  7%|▋         | 21/313 [00:34<07:57,  1.64s/it]
  7%|▋         | 22/313 [00:36<07:55,  1.63s/it]
  7%|▋         | 23/313 [00:37<07:50,  1.62s/it]
  8%|▊         | 24/313 [00:39<07:55,  1.64s/it]
  8%|▊         | 25/313 [00:41<07:49,  1.63s/it]
  8%|▊         | 26/313 [00:42<07:52,  1.65s/it]
  9%|▊         | 27/313 [00:44<07:48,  1.64s/it]
  9%|▉         | 28/313 [00:45<07:50,  1.65s/it]
  9%|▉         | 29/313 [00:47<07:38,  1.61s/it]
 10%|▉         | 30/313 [00:49<07:42,  1.64s/it]
 10%|▉         | 31/313 [00:51<08:02,  1.71s/it]
 10%|█         | 32/313 [00:52<08:00,  1.71s/it]
 11%|█         | 33/313 [00:54<08:00,  1.72s/it]
 11%|█         | 34/313 [00:56<08:02,  1.73s/it]
 11%|█         | 35/313 [00:57<07:52,  1.70s/it]
 12%|█▏        | 36/313 [00:59<07:51,  1.70s/it]
 12%|█▏        | 37/313 [01:01<07:49,  1.70s/it]
 12%|█▏        | 38/313 [01:03<07:47,  1.70s/it]
 12%|█▏        | 39/313 [01:04<07:45,  1.70s/it]
 13%|█▎        | 40/313 [01:06<07:43,  1.70s/it]
 13%|█▎        | 41/313 [01:08<07:40,  1.69s/it]
 13%|█▎        | 42/313 [01:09<07:33,  1.67s/it]
 14%|█▎        | 43/313 [01:11<07:39,  1.70s/it]
 14%|█▍        | 44/313 [01:13<07:30,  1.68s/it]
 14%|█▍        | 45/313 [01:14<07:23,  1.66s/it]
 15%|█▍        | 46/313 [01:16<07:23,  1.66s/it]
 15%|█▌        | 47/313 [01:18<07:18,  1.65s/it]
 15%|█▌        | 48/313 [01:19<07:14,  1.64s/it]
 16%|█▌        | 49/313 [01:21<07:15,  1.65s/it]
 16%|█▌        | 50/313 [01:22<07:10,  1.64s/it]
 16%|█▋        | 51/313 [01:24<07:05,  1.63s/it]
 17%|█▋        | 52/313 [01:26<07:02,  1.62s/it]
 17%|█▋        | 53/313 [01:27<07:00,  1.62s/it]
 17%|█▋        | 54/313 [01:29<07:04,  1.64s/it]
 18%|█▊        | 55/313 [01:31<07:05,  1.65s/it]
 18%|█▊        | 56/313 [01:32<07:02,  1.64s/it]
 18%|█▊        | 57/313 [01:34<06:59,  1.64s/it]
 19%|█▊        | 58/313 [01:36<07:07,  1.67s/it]
 19%|█▉        | 59/313 [01:37<07:08,  1.69s/it]
 19%|█▉        | 60/313 [01:39<07:07,  1.69s/it]
 19%|█▉        | 61/313 [01:41<07:05,  1.69s/it]
 20%|█▉        | 62/313 [01:42<06:58,  1.67s/it]
 20%|██        | 63/313 [01:44<06:57,  1.67s/it]
 20%|██        | 64/313 [01:46<06:51,  1.65s/it]
 21%|██        | 65/313 [01:47<06:46,  1.64s/it]
 21%|██        | 66/313 [01:49<06:44,  1.64s/it]
 21%|██▏       | 67/313 [01:50<06:43,  1.64s/it]
 22%|██▏       | 68/313 [01:52<06:39,  1.63s/it]
 22%|██▏       | 69/313 [01:54<06:36,  1.63s/it]
 22%|██▏       | 70/313 [01:55<06:34,  1.62s/it]
 23%|██▎       | 71/313 [01:57<06:35,  1.63s/it]
 23%|██▎       | 72/313 [01:59<06:33,  1.63s/it]
 23%|██▎       | 73/313 [02:00<06:35,  1.65s/it]
 24%|██▎       | 74/313 [02:02<06:31,  1.64s/it]
 24%|██▍       | 75/313 [02:04<06:32,  1.65s/it]
 24%|██▍       | 76/313 [02:05<06:28,  1.64s/it]
 25%|██▍       | 77/313 [02:07<06:30,  1.65s/it]
 25%|██▍       | 78/313 [02:09<06:31,  1.66s/it]
 25%|██▌       | 79/313 [02:10<06:28,  1.66s/it]
 26%|██▌       | 80/313 [02:12<06:27,  1.66s/it]
 26%|██▌       | 81/313 [02:14<06:29,  1.68s/it]
 26%|██▌       | 82/313 [02:15<06:34,  1.71s/it]
 27%|██▋       | 83/313 [02:17<06:26,  1.68s/it]
 27%|██▋       | 84/313 [02:19<06:25,  1.68s/it]
 27%|██▋       | 85/313 [02:20<06:18,  1.66s/it]
 27%|██▋       | 86/313 [02:22<06:18,  1.67s/it]
 28%|██▊       | 87/313 [02:24<06:12,  1.65s/it]
 28%|██▊       | 88/313 [02:25<06:14,  1.66s/it]
 28%|██▊       | 89/313 [02:27<06:08,  1.65s/it]
 29%|██▉       | 90/313 [02:29<06:09,  1.66s/it]
 29%|██▉       | 91/313 [02:30<06:05,  1.65s/it]
 29%|██▉       | 92/313 [02:32<06:06,  1.66s/it]
 30%|██▉       | 93/313 [02:34<06:01,  1.64s/it]
 30%|███       | 94/313 [02:35<06:03,  1.66s/it]
 30%|███       | 95/313 [02:37<06:00,  1.65s/it]
 31%|███       | 96/313 [02:39<05:58,  1.65s/it]
 31%|███       | 97/313 [02:40<05:54,  1.64s/it]
 31%|███▏      | 98/313 [02:42<05:52,  1.64s/it]
 32%|███▏      | 99/313 [02:43<05:51,  1.64s/it]
 32%|███▏      | 100/313 [02:45<05:47,  1.63s/it]
 32%|███▏      | 101/313 [02:47<05:43,  1.62s/it]
 33%|███▎      | 102/313 [02:48<05:41,  1.62s/it]
 33%|███▎      | 103/313 [02:50<05:38,  1.61s/it]
 33%|███▎      | 104/313 [02:51<05:37,  1.62s/it]
 34%|███▎      | 105/313 [02:53<05:39,  1.63s/it]
 34%|███▍      | 106/313 [02:55<05:36,  1.63s/it]
 34%|███▍      | 107/313 [02:56<05:33,  1.62s/it]
 35%|███▍      | 108/313 [02:58<05:30,  1.61s/it]
 35%|███▍      | 109/313 [03:00<05:28,  1.61s/it]
 35%|███▌      | 110/313 [03:01<05:32,  1.64s/it]
 35%|███▌      | 111/313 [03:03<05:29,  1.63s/it]
 36%|███▌      | 112/313 [03:05<05:30,  1.64s/it]
 36%|███▌      | 113/313 [03:06<05:26,  1.63s/it]
 36%|███▋      | 114/313 [03:08<05:23,  1.62s/it]
 37%|███▋      | 115/313 [03:09<05:24,  1.64s/it]
 37%|███▋      | 116/313 [03:11<05:21,  1.63s/it]
 37%|███▋      | 117/313 [03:13<05:22,  1.65s/it]
 38%|███▊      | 118/313 [03:14<05:15,  1.62s/it]
 38%|███▊      | 119/313 [03:16<05:21,  1.66s/it]
 38%|███▊      | 120/313 [03:18<05:17,  1.64s/it]
 39%|███▊      | 121/313 [03:19<05:18,  1.66s/it]
 39%|███▉      | 122/313 [03:21<05:13,  1.64s/it]
 39%|███▉      | 123/313 [03:23<05:10,  1.64s/it]
 40%|███▉      | 124/313 [03:24<05:10,  1.64s/it]
 40%|███▉      | 125/313 [03:26<05:07,  1.64s/it]
 40%|████      | 126/313 [03:27<05:08,  1.65s/it]
 41%|████      | 127/313 [03:29<05:04,  1.64s/it]
 41%|████      | 128/313 [03:31<05:01,  1.63s/it]
 41%|████      | 129/313 [03:32<04:59,  1.63s/it]
 42%|████▏     | 130/313 [03:34<05:01,  1.65s/it]
 42%|████▏     | 131/313 [03:36<05:00,  1.65s/it]
 42%|████▏     | 132/313 [03:37<04:57,  1.64s/it]
 42%|████▏     | 133/313 [03:39<04:53,  1.63s/it]
 43%|████▎     | 134/313 [03:41<04:54,  1.64s/it]
 43%|████▎     | 135/313 [03:42<04:51,  1.64s/it]
 43%|████▎     | 136/313 [03:44<04:47,  1.62s/it]
 44%|████▍     | 137/313 [03:45<04:45,  1.62s/it]
 44%|████▍     | 138/313 [03:47<04:47,  1.64s/it]
 44%|████▍     | 139/313 [03:49<04:43,  1.63s/it]
 45%|████▍     | 140/313 [03:50<04:41,  1.62s/it]
 45%|████▌     | 141/313 [03:52<04:42,  1.64s/it]
 45%|████▌     | 142/313 [03:54<04:43,  1.66s/it]
 46%|████▌     | 143/313 [03:55<04:39,  1.65s/it]
 46%|████▌     | 144/313 [03:57<04:44,  1.69s/it]
 46%|████▋     | 145/313 [03:59<04:39,  1.66s/it]
 47%|████▋     | 146/313 [04:00<04:39,  1.68s/it]
 47%|████▋     | 147/313 [04:02<04:38,  1.68s/it]
 47%|████▋     | 148/313 [04:04<04:38,  1.69s/it]
 48%|████▊     | 149/313 [04:05<04:32,  1.66s/it]
 48%|████▊     | 150/313 [04:07<04:33,  1.68s/it]
 48%|████▊     | 151/313 [04:09<04:31,  1.68s/it]
 49%|████▊     | 152/313 [04:10<04:30,  1.68s/it]
 49%|████▉     | 153/313 [04:12<04:26,  1.66s/it]
 49%|████▉     | 154/313 [04:14<04:26,  1.67s/it]
 50%|████▉     | 155/313 [04:15<04:21,  1.65s/it]
 50%|████▉     | 156/313 [04:17<04:21,  1.67s/it]
 50%|█████     | 157/313 [04:19<04:20,  1.67s/it]
 50%|█████     | 158/313 [04:20<04:17,  1.66s/it]
 51%|█████     | 159/313 [04:22<04:12,  1.64s/it]
 51%|█████     | 160/313 [04:24<04:17,  1.68s/it]
 51%|█████▏    | 161/313 [04:25<04:12,  1.66s/it]
 52%|█████▏    | 162/313 [04:27<04:08,  1.65s/it]
 52%|█████▏    | 163/313 [04:29<04:08,  1.66s/it]
 52%|█████▏    | 164/313 [04:30<04:08,  1.67s/it]
 53%|█████▎    | 165/313 [04:32<04:04,  1.65s/it]
 53%|█████▎    | 166/313 [04:34<04:04,  1.66s/it]
 53%|█████▎    | 167/313 [04:35<04:00,  1.65s/it]
 54%|█████▎    | 168/313 [04:37<03:56,  1.63s/it]
 54%|█████▍    | 169/313 [04:39<03:53,  1.62s/it]
 54%|█████▍    | 170/313 [04:40<03:56,  1.65s/it]
 55%|█████▍    | 171/313 [04:42<03:55,  1.66s/it]
 55%|█████▍    | 172/313 [04:44<03:55,  1.67s/it]
 55%|█████▌    | 173/313 [04:45<03:54,  1.68s/it]
 56%|█████▌    | 174/313 [04:47<03:54,  1.69s/it]
 56%|█████▌    | 175/313 [04:49<03:49,  1.66s/it]
 56%|█████▌    | 176/313 [04:50<03:48,  1.67s/it]
 57%|█████▋    | 177/313 [04:52<03:44,  1.65s/it]
 57%|█████▋    | 178/313 [04:54<03:44,  1.67s/it]
 57%|█████▋    | 179/313 [04:55<03:41,  1.65s/it]
 58%|█████▊    | 180/313 [04:57<03:40,  1.66s/it]
 58%|█████▊    | 181/313 [04:59<03:40,  1.67s/it]
 58%|█████▊    | 182/313 [05:00<03:36,  1.65s/it]
 58%|█████▊    | 183/313 [05:02<03:36,  1.66s/it]
 59%|█████▉    | 184/313 [05:04<03:36,  1.68s/it]
 59%|█████▉    | 185/313 [05:05<03:35,  1.69s/it]
 59%|█████▉    | 186/313 [05:07<03:34,  1.69s/it]
 60%|█████▉    | 187/313 [05:09<03:33,  1.70s/it]
 60%|██████    | 188/313 [05:10<03:31,  1.69s/it]
 60%|██████    | 189/313 [05:12<03:30,  1.69s/it]
 61%|██████    | 190/313 [05:14<03:28,  1.70s/it]
 61%|██████    | 191/313 [05:16<03:30,  1.73s/it]
 61%|██████▏   | 192/313 [05:17<03:28,  1.72s/it]
 62%|██████▏   | 193/313 [05:19<03:25,  1.71s/it]
 62%|██████▏   | 194/313 [05:21<03:22,  1.70s/it]
 62%|██████▏   | 195/313 [05:22<03:18,  1.68s/it]
 63%|██████▎   | 196/313 [05:24<03:14,  1.66s/it]
 63%|██████▎   | 197/313 [05:26<03:13,  1.67s/it]
 63%|██████▎   | 198/313 [05:27<03:09,  1.65s/it]
 64%|██████▎   | 199/313 [05:29<03:09,  1.66s/it]
 64%|██████▍   | 200/313 [05:31<03:06,  1.65s/it]
 64%|██████▍   | 201/313 [05:32<03:05,  1.65s/it]
 65%|██████▍   | 202/313 [05:34<03:01,  1.64s/it]
 65%|██████▍   | 203/313 [05:35<02:59,  1.63s/it]
 65%|██████▌   | 204/313 [05:37<02:57,  1.63s/it]
 65%|██████▌   | 205/313 [05:39<02:57,  1.64s/it]
 66%|██████▌   | 206/313 [05:40<02:55,  1.64s/it]
 66%|██████▌   | 207/313 [05:42<02:54,  1.64s/it]
 66%|██████▋   | 208/313 [05:44<02:54,  1.66s/it]
 67%|██████▋   | 209/313 [05:45<02:51,  1.65s/it]
 67%|██████▋   | 210/313 [05:47<02:51,  1.66s/it]
 67%|██████▋   | 211/313 [05:49<02:53,  1.70s/it]
 68%|██████▊   | 212/313 [05:51<02:51,  1.70s/it]
 68%|██████▊   | 213/313 [05:52<02:47,  1.68s/it]
 68%|██████▊   | 214/313 [05:54<02:45,  1.68s/it]
 69%|██████▊   | 215/313 [05:55<02:42,  1.66s/it]
 69%|██████▉   | 216/313 [05:57<02:41,  1.67s/it]
 69%|██████▉   | 217/313 [05:59<02:38,  1.65s/it]
 70%|██████▉   | 218/313 [06:00<02:38,  1.67s/it]
 70%|██████▉   | 219/313 [06:02<02:35,  1.65s/it]
 70%|███████   | 220/313 [06:04<02:34,  1.66s/it]
 71%|███████   | 221/313 [06:05<02:31,  1.65s/it]
 71%|███████   | 222/313 [06:07<02:30,  1.66s/it]
 71%|███████   | 223/313 [06:09<02:30,  1.68s/it]
 72%|███████▏  | 224/313 [06:10<02:28,  1.67s/it]
 72%|███████▏  | 225/313 [06:12<02:27,  1.68s/it]
 72%|███████▏  | 226/313 [06:14<02:24,  1.66s/it]
 73%|███████▎  | 227/313 [06:15<02:21,  1.65s/it]
 73%|███████▎  | 228/313 [06:17<02:19,  1.64s/it]
 73%|███████▎  | 229/313 [06:19<02:16,  1.63s/it]
 73%|███████▎  | 230/313 [06:20<02:15,  1.64s/it]
 74%|███████▍  | 231/313 [06:22<02:15,  1.66s/it]
 74%|███████▍  | 232/313 [06:24<02:13,  1.65s/it]
 74%|███████▍  | 233/313 [06:25<02:12,  1.65s/it]
 75%|███████▍  | 234/313 [06:27<02:09,  1.64s/it]
 75%|███████▌  | 235/313 [06:29<02:09,  1.66s/it]
 75%|███████▌  | 236/313 [06:30<02:06,  1.64s/it]
 76%|███████▌  | 237/313 [06:32<02:03,  1.63s/it]
 76%|███████▌  | 238/313 [06:33<02:02,  1.63s/it]
 76%|███████▋  | 239/313 [06:35<02:01,  1.64s/it]
 77%|███████▋  | 240/313 [06:37<01:59,  1.63s/it]
 77%|███████▋  | 241/313 [06:38<01:57,  1.63s/it]
 77%|███████▋  | 242/313 [06:40<01:56,  1.64s/it]
 78%|███████▊  | 243/313 [06:42<01:54,  1.63s/it]
 78%|███████▊  | 244/313 [06:43<01:52,  1.63s/it]
 78%|███████▊  | 245/313 [06:45<01:50,  1.62s/it]
 79%|███████▊  | 246/313 [06:46<01:49,  1.63s/it]
 79%|███████▉  | 247/313 [06:48<01:47,  1.63s/it]
 79%|███████▉  | 248/313 [06:50<01:45,  1.62s/it]
 80%|███████▉  | 249/313 [06:51<01:43,  1.61s/it]
 80%|███████▉  | 250/313 [06:53<01:41,  1.60s/it]
 80%|████████  | 251/313 [06:54<01:38,  1.59s/it]
 81%|████████  | 252/313 [06:56<01:36,  1.58s/it]
 81%|████████  | 253/313 [06:58<01:35,  1.58s/it]
 81%|████████  | 254/313 [06:59<01:34,  1.59s/it]
 81%|████████▏ | 255/313 [07:01<01:32,  1.59s/it]
 82%|████████▏ | 256/313 [07:02<01:31,  1.60s/it]
 82%|████████▏ | 257/313 [07:04<01:29,  1.60s/it]
 82%|████████▏ | 258/313 [07:06<01:27,  1.60s/it]
 83%|████████▎ | 259/313 [07:07<01:26,  1.60s/it]
 83%|████████▎ | 260/313 [07:09<01:24,  1.59s/it]
 83%|████████▎ | 261/313 [07:10<01:22,  1.58s/it]
 84%|████████▎ | 262/313 [07:12<01:20,  1.57s/it]
 84%|████████▍ | 263/313 [07:13<01:18,  1.58s/it]
 84%|████████▍ | 264/313 [07:15<01:17,  1.59s/it]
 85%|████████▍ | 265/313 [07:17<01:16,  1.59s/it]
 85%|████████▍ | 266/313 [07:18<01:14,  1.57s/it]
 85%|████████▌ | 267/313 [07:20<01:13,  1.60s/it]
 86%|████████▌ | 268/313 [07:21<01:10,  1.58s/it]
 86%|████████▌ | 269/313 [07:23<01:09,  1.58s/it]
 86%|████████▋ | 270/313 [07:25<01:08,  1.58s/it]
 87%|████████▋ | 271/313 [07:26<01:06,  1.59s/it]
 87%|████████▋ | 272/313 [07:28<01:05,  1.59s/it]
 87%|████████▋ | 273/313 [07:29<01:02,  1.57s/it]
 88%|████████▊ | 274/313 [07:31<01:01,  1.58s/it]
 88%|████████▊ | 275/313 [07:32<01:00,  1.58s/it]
 88%|████████▊ | 276/313 [07:34<00:58,  1.59s/it]
 88%|████████▊ | 277/313 [07:36<00:56,  1.57s/it]
 89%|████████▉ | 278/313 [07:37<00:55,  1.60s/it]
 89%|████████▉ | 279/313 [07:39<00:53,  1.58s/it]
 89%|████████▉ | 280/313 [07:40<00:51,  1.58s/it]
 90%|████████▉ | 281/313 [07:42<00:49,  1.56s/it]
 90%|█████████ | 282/313 [07:43<00:48,  1.57s/it]
 90%|█████████ | 283/313 [07:45<00:46,  1.55s/it]
 91%|█████████ | 284/313 [07:47<00:45,  1.56s/it]
 91%|█████████ | 285/313 [07:48<00:44,  1.58s/it]
 91%|█████████▏| 286/313 [07:50<00:42,  1.58s/it]
 92%|█████████▏| 287/313 [07:51<00:41,  1.58s/it]
 92%|█████████▏| 288/313 [07:53<00:39,  1.59s/it]
 92%|█████████▏| 289/313 [07:55<00:38,  1.59s/it]
 93%|█████████▎| 290/313 [07:56<00:36,  1.60s/it]
 93%|█████████▎| 291/313 [07:58<00:35,  1.60s/it]
 93%|█████████▎| 292/313 [07:59<00:33,  1.61s/it]
 94%|█████████▎| 293/313 [08:01<00:31,  1.60s/it]
 94%|█████████▍| 294/313 [08:03<00:30,  1.60s/it]
 94%|█████████▍| 295/313 [08:04<00:28,  1.60s/it]
 95%|█████████▍| 296/313 [08:06<00:27,  1.60s/it]
 95%|█████████▍| 297/313 [08:07<00:25,  1.60s/it]
 95%|█████████▌| 298/313 [08:09<00:23,  1.60s/it]
 96%|█████████▌| 299/313 [08:11<00:22,  1.60s/it]
 96%|█████████▌| 300/313 [08:12<00:20,  1.60s/it]
 96%|█████████▌| 301/313 [08:14<00:19,  1.60s/it]
 96%|█████████▋| 302/313 [08:15<00:17,  1.60s/it]
 97%|█████████▋| 303/313 [08:17<00:16,  1.60s/it]
 97%|█████████▋| 304/313 [08:19<00:14,  1.62s/it]
 97%|█████████▋| 305/313 [08:20<00:12,  1.62s/it]
 98%|█████████▊| 306/313 [08:22<00:11,  1.62s/it]
 98%|█████████▊| 307/313 [08:24<00:09,  1.64s/it]
 98%|█████████▊| 308/313 [08:25<00:08,  1.65s/it]
 99%|█████████▊| 309/313 [08:27<00:06,  1.67s/it]
 99%|█████████▉| 310/313 [08:28<00:04,  1.62s/it]
 99%|█████████▉| 311/313 [08:30<00:03,  1.64s/it]
100%|█████████▉| 312/313 [08:32<00:01,  1.64s/it]
100%|██████████| 313/313 [08:33<00:00,  1.41s/it]
100%|██████████| 313/313 [08:33<00:00,  1.64s/it]
fid_custom.py:31: UserWarning: mid matrix contains NaN/Inf; returning NaN for Tr(sqrt(M))
  warnings.warn("mid matrix contains NaN/Inf; returning NaN for Tr(sqrt(M))")
computing/reading sample batch statistics...
Computing custom FID...
FID(custom): nan
Saved report to /gemini/space/zhaozy/guzhenyu/UAVFlow/UAV_Flow_base/exps/jsflow-experiment/samples/fid_custom_0.5.txt