D-Robotics Official Website
D-Robotics Official Website
English

Product Introduction

What is D-Robotics-LLM

D-Robotics-LLM Toolchain is a specialized extension of D-Robotics' self-developed S100 toolchain product for large model applications. It enables the development, conversion, and deployment of large language models on the S100 series platforms and derivative platforms. The reference solutions developed based on this toolchain provide case support for large model requirements in robotics.

After obtaining the D-Robotics-LLM release package, you can get started by following these steps:

  1. First, refer to the Release Content section to understand the directory structure of the release package.

  2. Next, refer to the Development Process section for a brief overview of the overall usage flow of the deployment package.

  3. Then, refer to the Environment Deployment section to correctly complete the required environment installation and deployment.

  4. Finally, according to the model you are using, refer to the corresponding model section in S100/S100P平台 to complete the full process from model conversion to deployment.

For more comprehensive D-Robotics-LLM tutorials, please refer to the subsequent sections.

Release Content

S100/S100P

The structure of the D-Robotics-LLM deployment package is as follows:

. ├── arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu.tar.xz ├── oellm_build │ ├── requirements.txt │ ├── hbdk4_compiler-{version}-cp310-cp310-manylinux_2_17_x86_64.whl │ ├── hbdk4_runtime_aarch64_unknown_linux_gnu_nash-{version}-py3-none-any.whl │ └── leap_llm-{version}-py310-none-any.whl └── oellm_runtime ├── model ├── config ├── example ├── include ├── lib └── set_performance_mode.sh
Note

This is only a demonstration of the deployment package structure. Please refer to the actual release package you use for specific version numbers.

Among them:

  • arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu.tar.xz is the cross-compilation tool package required by the deployment package.

  • The oellm_build folder contains environment dependencies, compiler, and leap unified operator library dependencies.

    • The requirements.txt is the environment dependencies required for setting up the development environment.

    • The hbdk4_compiler-{version}-cp310-cp310-manylinux_2_17_x86_64.whl is the installation package for Horizon's hbdk4 compiler.

    • The hbdk4_runtime_aarch64_unknown_linux_gnu_nash-{version}-py3-none-any.whl is the installation package for the hbdk4 runtime.

    • The leap_llm-{version}-py310-none-any.whl is the installation package for leap-llm, providing model quantization and conversion functions.

  • The oellm_runtime folder contains dependencies and examples for the board-side deployment SDK.

    • The model provides download links for supported models and scripts for one-click download of all models.

    • The config contains reference configuration files for supported models and configuration files required at runtime.

    • The example contains demo source code, build scripts, binaries, and runtime configuration files.

    • The include contains key header files required for compiling edge-side deployment examples.

    • The lib contains libraries required by the edge-side deployment program.

    • The set_performance_mode.sh script is used to modify hardware register values to set the device to performance mode. Currently, only the S100P is supported.