Khadas Docs

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
~~tag> NPU PPOCR KSNN VIM4 ~~

**Doc for version ddk-3.4.7.7**

====== PPOCR KSNN Demo - 10 ======

{{indexmenu_n>10}}

===== Download the model =====

Download the PPOCR model from [[gh>PaddlePaddle/PaddleOCR/blob/main/docs/ppocr/model_list.md|ppocr/model_list.md]]
 
{{:products:sbc:vim4:npu:ksnn:demo:ppocr-det-download.webp?800|}}

{{:products:sbc:vim4:npu:ksnn:demo:ppocr-rec-download.webp?800|}}

===== Convert the model =====

==== Get the conversion tool ====

Get source [[gh>khadas/vim4_npu_sdk]].

```shell
$ git lfs install
$ git lfs clone https://github.com/khadas/vim4_npu_sdk
$ cd vim4_npu_sdk
$ ls
adla-toolkit-binary  adla-toolkit-binary-3.1.7.4  convert-in-docker.sh  Dockerfile  docs  README.md
```

  * ''adla-toolkit-binary/docs'' - SDK documentations
  * ''adla-toolkit-binary/bin'' - SDK tools required for model conversion
  * ''adla-toolkit-binary/demo'' - Conversion examples
<WRAP important>
Only convert tool version branch npu-ddk-3.4.7.7 or higher supports PPOCR.
</WRAP>

==== Convert ====

First, convert model from Paddle to ONNX. Please refer the two doc. [[gh>PaddlePaddle/Paddle2ONNX/blob/develop/README_en.md|Paddle2ONNX/blob/develop/README_en.md]], [[gh>PaddlePaddle/PaddleOCR/blob/main/docs/ppocr/infer_deploy/paddle2onnx.en.md|PaddleOCR/blob/main/docs/ppocr/infer_deploy/paddle2onnx.en.md]].

Our version is ''paddlepaddle==2.6.1'' and ''paddle2onnx==1.2.11'' and our covnert command.

```bash
$ paddle2onnx --model_dir ./ch_PP-OCRv4_det_infer --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file ppocr_det.onnx
$ paddle2onnx --model_dir ./ch_PP-OCRv4_rec_infer --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file ppocr_rec.onnx
```

Pull ''ppocr_det.onnx'' model and ''ppocr_rec'' model into ''vim4_npu_sdk/adla-toolkit-binary-3.1.7.4/python'' and modify ''ksnn_args.txt''.

```txt ppocr_det
--model-name ppocr_det 
--model-type onnx 
--model ./ppocr_det.onnx 
--inputs "x" 
--input-shapes  "3,736,736" 
--dtypes "float32" 
--quantize-dtype int8 
--outdir onnx_output 
--channel-mean-value "123.675,116.28,103.53,57.375" 
--source-file ocr_det_dataset.txt 
--iterations 1 
--batch-size 1 
--kboard VIM4 
--inference-input-type "float32" 
--inference-output-type "float32" 
```

```txt ppocr_rec
--model-name ppocr_rec 
--model-type onnx 
--model ./ppocr_rec.onnx 
--inputs "x" 
--input-shapes  "3,48,320" 
--dtypes "float32" 
--quantize-dtype int16 
--outdir onnx_output 
--channel-mean-value "127.5,127.5,127.5,128" 
--source-file ocr_rec_dataset.txt 
--iterations 1 
--batch-size 1 
--kboard VIM4 
--inference-input-type "float32" 
--inference-output-type "float32" 
```

```shell
$ ./convert-in-docker.sh ksnn
```

===== Run inference on the NPU by KSNN =====

==== Install KSNN ====

Download KSNN library and demo code. [[gh>khadas/ksnn-vim4]]

```shell
$ git clone https://github.com/khadas/ksnn-vim4
```

<WRAP important>
Only KSNN demo tag ddk-3.4.7.7 or higher supports PPOCR. Only firmware newer than 241129 supports this PPOCR demo.
</WRAP>

If you use Ubuntu 24.04, demo must run in python virtual environment.

```shell
$ sudo apt update
$ sudo apt install python3-venv
$ python3 -m venv myenv
$ source myenv/bin/activate
```

```shell
$ cd ksnn-vim4/ksnn
$ sudo apt update
$ sudo apt install python3-pip
$ pip3 install ksnn_vim4-1.4.1-py3-none-any.whl
$ pip3 install shapely pyclipper Pillow
```

==== Picture input demo ====

<WRAP important>
The demo use cn rec model. If you want to use other rec model, you need to modify the rec model output and character index dictionary.

```python ppocr-picture.py
# model input and output
det_input_size = (736, 736) # (model height, model width)
rec_input_size = ( 48, 320) # (model height, model width)
rec_output_size = (40, 97) # rec output
```

```python postprocess.py
character_str = ["blank"]
with open("./data/ppocr_keys_v1.txt", "rb") as fin: # the path for character index dictionary
    lines = fin.readlines()
    for line in lines:
        line = line.decode("utf-8").strip("\n").strip("\r\n")
        character_str.append(line)
character_str.append(" ")
ignored_token = [0]
```

You can find the dictionary txt file in ''PaddleOCR/ppocr/utils''
</WRAP>

```shell
$ cd ksnn-vim4/example/ppocr
$ export QT_QPA_PLATFORM=xcb
$ python3 ppocr-picture.py --det_model ./models/VIM4/ppocr_det_int8.adla --det_library ./libs/libnn_ppocr_det.so --rec_model ./models/VIM4/ppocr_rec_int16.adla --rec_library ./libs/libnn_ppocr_rec.so --picture ./data/test.png
```

{{:products:sbc:vim4:npu:ksnn:demo:ppcor-ksnn-demo-output.webp?800|}}

=== Camera input demo ===

```shell
$ cd ksnn-vim4/example/ppocr
$ export QT_QPA_PLATFORM=xcb
$ python3 ppocr-cap.py --det_model ./models/VIM4/ppocr_det_int8.adla --det_library ./libs/libnn_ppocr_det.so --rec_model ./models/VIM4/ppocr_rec_int16.adla --rec_library ./libs/libnn_ppocr_rec.so --device 0
```

''0'' is the camera device index.
Khadas Docs

User Tools

Site Tools

Page Tools
