Configuration Parsing Warning:Invalid JSON for config file config.json

Bird-Species-Classification

This project trying to show a demo of Bird Species Detection & Classification within 1400+ species of birds.

This model has been converted to run on the Axera NPU using w8a16 quantization.

This model has been optimized with the following LoRA:

Compatible with Pulsar2 version: 5.1

Convert tools links:

Do model convert from onnx to axmodel with commands like:

pulsar2 build --config ./quant/bird-l.json

For those who are interested in model conversion, you can try to export axmodel through

Pulsar2 Link, How to Convert ONNX to axmodel

Support Platform

AX650
- M4N-Dock(爱芯派Pro)
- M.2 Accelerator card
AX630C

Models	Platforms	latency	Top1 Accuracy	Top5 Accuracy	CMM size(MB)
	AX650	0.19ms
bird-s	AX630C	0.54ms	44%	66%	1.07
	AX615	0.87ms
	AX650	0.58ms
bird-m	AX630C	2.52ms	59%	79%	12.2
	AX615	4.97ms
	AX650	5.60ms
bird-l	AX630C	35.2ms	86%	95%	29.6
	AX615	64.1ms

How to use

Download all files from this repository to the device

(base) root@ax650:~/Bird-Species-Classification# tree
.
├── README.md
├── axmodel_infer.py
├── class_name.txt
├── config.json
├── model
│   ├── bird-l
│   │   ├── AX615
│   │   │   └── bird_615_npu2.axmodel
│   │   ├── AX620E
│   │   │   ├── bird_630_npu1.axmodel
│   │   │   └── bird_630_npu2.axmodel
│   │   └── AX650
│   │       └── bird_650_npu3.axmodel
│   ├── bird-m
│   │   ├── AX615
│   │   │   ├── bird_615_npu1.axmodel
│   │   │   └── bird_615_npu2.axmodel
│   │   ├── AX620E
│   │   │   ├── bird_630_npu1.axmodel
│   │   │   └── bird_630_npu2.axmodel
│   │   └── AX650
│   │       └── bird_650_npu3.axmodel
│   └── bird-s
│       ├── AX615
│       │   ├── bird_615_npu1.axmodel
│       │   └── bird_615_npu2.axmodel
│       ├── AX630C
│       │   ├── bird_630_npu1.axmodel
│       │   └── bird_630_npu2.axmodel
│       └── AX650
│           └── bird_650_npu3.axmodel
├── onnx_infer.py
├── prediction_result_top5.png
├── quant
│   ├── bird-l.json
│   ├── bird-m.json
│   ├── bird-s.json
│   └── bird.tar.gz
└── test_images
    ├── 03111_2c0dfa5a-c4a0-47f8-ac89-6a289208050f.jpg
    ├── 03332_01b365c3-a741-4f45-bac2-4345bc901ec6.jpg
    ├── 03412_0ffc115b-43b4-4474-a373-24233f391de3.jpg
    ├── 03615_0dfbf6ae-434d-4648-b5d2-08412546ea64.jpg
    ├── 04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
    ├── 04405_0c5a6785-0bc2-49d9-9702-b9e94ba9b686.jpg
    └── 04593_3d74d5a7-15b1-4bb9-af6f-1bcd78485787.jpg

15 directories, 31 files

python env requirement

pyaxengine

https://github.com/AXERA-TECH/pyaxengine

wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3rc0/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl

Inference with ONNX model

root@ebba5440b03c:/home/# python onnx_infer.py -m Birdmodel_inat_bird-l.onnx --image test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
build predictor with Birdmodel_inat_bird-l.onnx...
Loading ONNX model with providers: ['CPUExecutionProvider']

Image: test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
Top-5 Predictions:
#1: 04251_Animalia_Chordata_Aves_Passeriformes_Tityridae_Tityra_semifasciata (0.9231)
#2: 04019_Animalia_Chordata_Aves_Passeriformes_Oriolidae_Sphecotheres_vieilloti (0.0021)
#3: 03233_Animalia_Chordata_Aves_Anseriformes_Anatidae_Cairina_moschata (0.0006)
#4: 04219_Animalia_Chordata_Aves_Passeriformes_Thraupidae_Paroaria_capitata (0.0004)
#5: 03912_Animalia_Chordata_Aves_Passeriformes_Laniidae_Lanius_minor (0.0003)
Result saved to: prediction_result_top5.png

Inference with AX650 Host, such as M4N-Dock(爱芯派Pro)

root@ax650:~/bird# python3 axmodel_infer.py -m bird_650_npu3.axmodel -i test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
[INFO] Available providers:  ['AxEngineExecutionProvider']
build predictor with bird_650_npu3.axmodel...
Loading ONNX model with providers: ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 5.1-patch1 8c5871d5

Image: test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
Top-5 Predictions:
#1: 04251_Animalia_Chordata_Aves_Passeriformes_Tityridae_Tityra_semifasciata (0.9137)
#2: 04019_Animalia_Chordata_Aves_Passeriformes_Oriolidae_Sphecotheres_vieilloti (0.0022)
#3: 03233_Animalia_Chordata_Aves_Anseriformes_Anatidae_Cairina_moschata (0.0006)
#4: 03912_Animalia_Chordata_Aves_Passeriformes_Laniidae_Lanius_minor (0.0005)
#5: 04219_Animalia_Chordata_Aves_Passeriformes_Thraupidae_Paroaria_capitata (0.0004)
Result saved to: prediction_result_top5.png

output：预测结果

Eval the quantized model

You can use the quant_model_eval.py to eval the quantized model, to see the quantized model's accuracy. 'val_list_flat.txt' could be your valid dataset list, each line with content like 'img_path class'.

(base) root@ax630c:~/Bird# python3 quant_model_eval.py
[INFO] Available providers:  ['AxEngineExecutionProvider']
Ground truth loaded from ./val_list_flat.txt
Loading ONNX model with providers: ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC20E
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.7.2a
[INFO] Model type: 0 (half core)
[INFO] Compiler version: 5.1-patch1 fa983fc0
Found 17893 images, starting inference (Top-5)...
100%|█████████████████████████████████████████████████████████████████████████████████████████████| 17893/17893 [09:20<00:00, 31.90it/s]

=== 批量推理结果汇总 ===
总处理图片数: 17893
Top1正确数: 14365 | Top1准确率: 80.28%
Top5正确数: 16325 | Top5准确率: 91.24%
========================

Bird Detect & Classify End2End Demo

I've added a demo to do end2end inference with detection axmodel and recognition axmodel in 'model/bird-end2end'. Try to do bird object detect--->expand & crop--->classify with:

python3 onnx_infer_end2end.py
or
python3 axmodel_infer_end2end.py --image ./test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg

The result could be like:

root@ebba5440b03c:/home/# python onnx_infer_end2end.py --image ./test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg
Loading ONNX model with providers: ['CPUExecutionProvider']

========== Processing Single Image: ./test_images/04251_3a52191e-be71-4539-98ea-14a8f2347330.jpg ==========
✓ Detection: Found 1 bird(s)

  Bird #1:
    Box: [165, 54, 303, 316]
    Crop saved: ./output/crops/04251_3a52191e-be71-4539-98ea-14a8f2347330_bird_1.jpg
    Top-5 Predictions:
      #1: 04251_Animalia_Chordata_Aves_Passeriformes_Tityridae_Tityra_semifasciata (0.9173, 91.73%)
      #2: 04019_Animalia_Chordata_Aves_Passeriformes_Oriolidae_Sphecotheres_vieilloti (0.0014, 0.14%)
      #3: 03696_Animalia_Chordata_Aves_Passeriformes_Artamidae_Artamus_leucorynchus (0.0014, 0.14%)
      #4: 04219_Animalia_Chordata_Aves_Passeriformes_Thraupidae_Paroaria_capitata (0.0010, 0.10%)
      #5: 03561_Animalia_Chordata_Aves_Coraciiformes_Alcedinidae_Todiramphus_chloris (0.0007, 0.07%)

Downloads last month: 21