-
Notifications
You must be signed in to change notification settings - Fork 102
docs: add Frigate NVR for CIX guide. #1356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+257
−1
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
243 changes: 243 additions & 0 deletions
243
docs/common/orion-common/app-dev/artificial-intelligence/_frigate-cix.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,243 @@ | ||
| [Frigate](https://frigate.video/) 是一个 专门为 [Home Assistant](https://www.home-assistant.io/) 设计的 NVR 项目。 | ||
| 它能够充分调用本地硬件算力,并通过 AI 模型对监控画面进行实时的目标检测、识别与物体分类(如区分行人、车辆、宠物等)。 | ||
|
|
||
| :::tip | ||
| Home Assistant,目前全球最流行的开源智能家居控制中心。 | ||
| ::: | ||
|
|
||
| 更多相关内容参考[官方文档](https://docs.frigate.video/)或者[官方中文文档](https://docs.frigate-cn.video/)。 | ||
|
|
||
| 本文档使用的 Frigate 版本为 0.16.4,目前,官方还没有直接支持 CIX P1 芯片平台, | ||
| 让我们来一起看看如何通过添加自定义插件代码调用未支持的硬件实现 Frigate 的 AI 检测功能。 | ||
|
|
||
| ## 准备环节 | ||
|
|
||
| ### 目录结构 | ||
|
|
||
| 可以通过 ssh 远程连接或者直接外接显示器和键鼠使用 O6/O6N ,本文使用 ssh 远程连接。 | ||
|
|
||
| 创建初始目录结构。 | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| mkdir frigate && cd frigate | ||
| mkdir storage config && touch docker-compose.yml | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| ### 模型文件 | ||
|
|
||
| 这里我们使用官方模型仓库转换出来的 yolov8n.cix 作为检测模型。 | ||
|
|
||
| 如果你对转换模型感兴趣,后续想要使用自己的模型作为检测模型,你可以参考 [YOLOv8n](./Vision/yolov8n) 转换模型。 | ||
|
|
||
| 直接从魔搭社区的官方仓库下载模型文件。 | ||
|
|
||
| <NewCodeBlock tip="Device"> | ||
|
|
||
| ```bash | ||
| wget https://www.modelscope.cn/models/cix/ai_model_hub_25_Q3/resolve/master/models/ComputeVision/Object_Detection/onnx_yolov8_n/yolov8n.cix | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| ### 推理库 | ||
|
|
||
| 想要在容器中能够正常执行推理,我们还需要将 NPU 的 API 库挂载到容器中。 | ||
|
|
||
| 找到 系统中的 libnoe 库: | ||
|
|
||
| <NewCodeBlock tip="Device"> | ||
|
|
||
| ```bash | ||
| find / -name "libnoe" 2>/dev/null | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 这里我们找到的路径是:`/usr/local/lib/python3.11/dist-packages/libnoe/` | ||
|
|
||
| ### 插件代码 | ||
|
|
||
| 使用我们编写好的代码文件即可,你可以阅读其逻辑作为自主实现的参考。 | ||
|
|
||
| <NewCodeBlock tip="Device"> | ||
|
|
||
| ```bash | ||
| wget https://github.com/Ronin-1124/Frigate-NVR-for-CIX/blob/main/cix.py | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 这样我们就准备好了需要的所有材料。 | ||
|
|
||
| ## 启动 Frigate | ||
|
|
||
| 将下面的内容粘贴到 docker-compose.yml | ||
|
|
||
| ```yml | ||
| services: | ||
| frigate: | ||
| container_name: frigate | ||
| restart: unless-stopped | ||
| stop_grace_period: 30s | ||
| image: docker.cnb.cool/frigate-cn/frigate:stable | ||
| devices: | ||
| - /dev/aipu:/dev/aipu | ||
| volumes: | ||
| - /usr/local/lib/python3.11/dist-packages/libnoe/:/usr/local/lib/python3.11/dist-packages/libnoe/ | ||
| - ./cix.py:/opt/frigate/frigate/detectors/plugins/cix.py | ||
| - ./yolov8n.cix:/config/model_cache/yolov8n.cix | ||
| - ./config:/config | ||
| - ./storage:/media/frigate | ||
| - type: tmpfs # 可选:1GB内存,减少SSD/SD卡损耗 | ||
| target: /tmp/cache | ||
| tmpfs: | ||
| size: 1000000000 | ||
| ports: | ||
| - "8971:8971" | ||
| - "8555:8554" # RTSP 视频流,这里使用宿主机的8555端口而不是8554是因为后续模拟推流需要使用8554端口。如果你不需要模拟推流则用8554端口就行。不改也行。 | ||
| ``` | ||
|
|
||
| 接下来就可以运行下面的命令直接启动 Frigate 了。 | ||
|
|
||
| 注意这个命令需要在包含 docker-compose.yml 的文件夹下面运行。 | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| docker compose up -d | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 成功拉取并运行之后会在 https://your_device_ip:8971 开启前端界面服务。 | ||
|
|
||
| 查看开发版 IP 地址: | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| hostname -I | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 然后将最前面的 IP 地址替换上面的 your_device_ip,在浏览器输入该地址即可进入。 | ||
|
|
||
| 登录的账户和密码通过下面的命令查看: | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| docker logs frigate | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 日志消息可能比较长,需要找一下,如果找不到就在 VSCode 终端中用 Ctrl + F 输入 admin 等相关信息查找。 | ||
|
|
||
| 接下来你可以使用前端界面的**配置编辑器**修改相关配置。 | ||
|
|
||
|  | ||
|
|
||
| :::tip | ||
| 超过二十四小时未登录过 Frigate 会需要密码才能登录,建议通过左下角的账户及时修改密码,方便长期使用。 | ||
| ::: | ||
|
|
||
| ## 模拟推流并启用 AI 模型 | ||
|
|
||
| 如果你暂时没有 IP 摄像头(有当然最好,直接使用即可),使用 FFmpeg + MediaMTX 模拟推流是一个很好的验证方式。 | ||
|
|
||
| 使用方式也很简单,参考下面的步骤。 | ||
|
|
||
| ### MediaMTX | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| mkdir mediamtx && cd mediamtx | ||
| wget https://github.com/bluenviron/mediamtx/releases/download/v1.16.0/mediamtx_v1.16.0_linux_arm64.tar.gz | ||
| tar -xvf mediamtx_v1.16.0_linux_arm64.tar.gz | ||
| ./mediamtx | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 然后就会在当前终端开启一个 RTSP 服务器,等待接收视频流信息。 | ||
|
|
||
| 接下来新打开一个终端,使用 FFmpeg 进行推流。 | ||
|
|
||
| ### FFmpeg | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| sudo apt install ffmpeg | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 然后你需要准备一个视频,这里使用 huggingface 公开数据集 KAI-KratosAI/Street-videos 的一个视频。 | ||
|
|
||
| <NewCodeBlock tip='Device'> | ||
|
|
||
| ```bash | ||
| wget https://huggingface.co/datasets/KAI-KratosAI/Street-videos/resolve/0d33ede195de1fc061723d5dfa4623b30162af66/041896fb-20da-4625-9455-195172efc4d5.mp4_041896fb-20da-4625-9455-195172efc4d5.mp4#t=0.001 | ||
| mv 041896fb-20da-4625-9455-195172efc4d5.mp4_041896fb-20da-4625-9455-195172efc4d5.mp4 test.mp4 | ||
| ffmpeg -re -stream_loop -1 -i test.mp4 -c copy -f rtsp rtsp://localhost:8554/mystream | ||
| ``` | ||
|
|
||
| </NewCodeBlock> | ||
|
|
||
| 参数解释: | ||
|
|
||
| - `-re`:以“本地速度”读取。如果不加这个,FFmpeg 会以极快的速度把视频塞给服务器,导致画面像快进一样。 | ||
| - `-stream_loop -1`:让视频无限循环。这对测试 Frigate 非常有用,不用担心视频播完就断开。 | ||
| - `-i test.mp4`:指定你的视频源文件。 | ||
| - `-c copy`:直接复制视频和音频编码,不重新编码。这最省 CPU,直接把原始数据包封装进 RTSP。 | ||
| - `-f rtsp`:指定输出格式为 RTSP。 | ||
| - `rtsp://localhost:8554/mystream`:推流地址。mystream 是你给这个流起的 ID。 | ||
|
|
||
| ### 修改配置文件 | ||
|
|
||
| ```yaml | ||
| mqtt: | ||
| enabled: false | ||
|
|
||
| cameras: | ||
| name_of_your_camera: # <------ Name the camera | ||
| enabled: true | ||
| ffmpeg: | ||
| inputs: | ||
| - path: rtsp://192.168.2.223:8554/mystream # <----- The stream you want to use for detection | ||
Ronin-1124 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| roles: | ||
| - detect | ||
| detect: | ||
| enabled: true # <---- disable detection until you have a working camera feed | ||
| width: 1280 | ||
| height: 720 | ||
|
|
||
| detectors: | ||
| cix: | ||
| type: cix | ||
|
|
||
| model: | ||
| height: 640 | ||
| width: 640 | ||
| path: /config/model_cache/yolov8n.cix | ||
|
|
||
| detect: | ||
| enabled: true | ||
| version: 0.16-0 | ||
| ``` | ||
|
|
||
| 然后就可以在首页看到实时检测的监控画面了。 | ||
|
|
||
| 你可以点进调试界面查看实时检测效果。 | ||
|
|
||
|  | ||
|
|
||
| 现在你成功在 O6/O6N 上部署了 Frigate 。 | ||
9 changes: 9 additions & 0 deletions
9
docs/orion/o6/app-development/artificial-intelligence/frigate-cix.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| sidebar_position: 10 | ||
| --- | ||
|
|
||
| import Frigate from "../../../../common/orion-common/app-dev/artificial-intelligence/\_frigate-cix.mdx"; | ||
|
|
||
| # Frigate NVR for CIX | ||
|
|
||
| <Frigate /> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.