chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,83 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _docker-images:
|
||||
|
||||
Docker Images
|
||||
=============
|
||||
We provide docker utility scripts to help developers to setup development environment.
|
||||
They are also helpful run through TVM demo and tutorials.
|
||||
We need `docker <https://docs.docker.com/engine/installation/>`_ and
|
||||
`NVIDIA Container Toolkit <https://github.com/NVIDIA/nvidia-container-toolkit>`_
|
||||
if we want to use CUDA.
|
||||
|
||||
Get a tvm source distribution or clone the GitHub repo to get the auxiliary scripts
|
||||
|
||||
.. code:: bash
|
||||
|
||||
git clone --recursive https://github.com/apache/tvm tvm
|
||||
|
||||
|
||||
We can then use the following command to launch a docker image.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
/path/to/tvm/docker/bash.sh <image-name>
|
||||
|
||||
Here the image-name can be a local docker image name, e.g. ``tvm.ci_cpu`` after you have done the local build.
|
||||
|
||||
This auxiliary script does the following things:
|
||||
|
||||
- Mount current directory to ``/workspace``
|
||||
- Switch user to be the same user that calls the ``bash.sh`` (so you can read/write host system)
|
||||
- Use the host-side network on Linux. Use the bridge network and expose port 8888 on macOS,
|
||||
because host networking driver isn't supported. (so you can use ``jupyter notebook``)
|
||||
|
||||
|
||||
Then you can start a Jupyter notebook by typing
|
||||
|
||||
.. code:: bash
|
||||
|
||||
jupyter notebook
|
||||
|
||||
You might see an error ``OSError: [Errno 99] Cannot assign requested address`` when starting
|
||||
a Jupyter notebook on macOS. You can change the binding IP address by
|
||||
|
||||
.. code:: bash
|
||||
|
||||
jupyter notebook --ip=0.0.0.0
|
||||
|
||||
Note that on macOS, because ``bash.sh`` uses the Docker bridge network, Jupyter will be reportedly running
|
||||
at an URL like ``http://{container_hostname}:8888/?token=...``. You should replace the ``container_hostname``
|
||||
with ``localhost`` when pasting it into browser.
|
||||
|
||||
|
||||
Docker Source
|
||||
-------------
|
||||
Check out `the docker source <https://github.com/apache/tvm/tree/main/docker>`_ if you are interested in
|
||||
building your own docker images.
|
||||
|
||||
|
||||
Run the following command to build the docker image.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
/path/to/tvm/docker/build.sh <image-name>
|
||||
|
||||
|
||||
You can find some un-official third party pre-built images at `<https://hub.docker.com/r/tlcpack/>`_.
|
||||
These images are used for test purposes and are NOT of the ASF release.
|
||||
@@ -0,0 +1,356 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _install-from-source:
|
||||
|
||||
Install from Source
|
||||
===================
|
||||
This page gives instructions on how to build and install the TVM package from source.
|
||||
|
||||
.. contents:: Table of Contents
|
||||
:local:
|
||||
:depth: 2
|
||||
|
||||
.. _install-dependencies:
|
||||
|
||||
Step 1. Install Dependencies
|
||||
----------------------------
|
||||
|
||||
Apache TVM requires the following dependencies:
|
||||
|
||||
- CMake (>= 3.24.0)
|
||||
- LLVM (recommended >= 15)
|
||||
- Git
|
||||
- A recent C++ compiler supporting C++ 20, at the minimum
|
||||
- GCC 10
|
||||
- Clang 10
|
||||
- Apple Clang 14
|
||||
- Visual Studio 2022
|
||||
|
||||
Optional dependencies that use newer C++20 standard library facilities, such
|
||||
as ``std::format``, may require a newer standard library (for example GCC 13
|
||||
or newer on Linux).
|
||||
- Python (>= 3.10)
|
||||
- (Optional) Conda (Strongly Recommended)
|
||||
|
||||
System Dependencies (Non-Conda)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
If you are not using Conda, TVM requires several system libraries. On Ubuntu/Debian systems, install them with:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
sudo apt update
|
||||
sudo apt install zlib1g-dev libxml2-dev
|
||||
|
||||
For other operating systems, please refer to your package manager documentation.
|
||||
|
||||
The easiest way to manage dependency is via conda, which maintains a set of toolchains
|
||||
including LLVM across platforms. To create the environment of those build dependencies,
|
||||
one may simply use:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# make sure to start with a fresh environment
|
||||
conda env remove -n tvm-build-venv
|
||||
# create the conda environment with build dependency
|
||||
conda create -n tvm-build-venv -c conda-forge \
|
||||
"llvmdev>=15" \
|
||||
"cmake>=3.24" \
|
||||
git \
|
||||
python=3.11
|
||||
# enter the build environment
|
||||
conda activate tvm-build-venv
|
||||
|
||||
.. note::
|
||||
**For Frontend Contributors (TFLite):** If you plan to run or contribute to the frontend tests (e.g., ``test_frontend_tflite.py``), you must install ``tensorflow==2.19.0``.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
pip install tensorflow==2.19.0
|
||||
|
||||
Python 3.11 is supported.
|
||||
|
||||
Step 2. Get Source from GitHub
|
||||
------------------------------
|
||||
You can also choose to clone the source repo from GitHub.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
git clone --recursive https://github.com/apache/tvm tvm
|
||||
|
||||
.. note::
|
||||
It's important to use the ``--recursive`` flag when cloning the TVM repository, which will
|
||||
automatically clone the submodules. If you forget to use this flag, you can manually clone the submodules
|
||||
by running ``git submodule update --init --recursive`` in the root directory of the TVM repository.
|
||||
|
||||
Step 3. Configure and Build
|
||||
---------------------------
|
||||
Create a build directory and run CMake to configure the build. The following example shows how to build
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cd tvm
|
||||
rm -rf build && mkdir build && cd build
|
||||
# Specify the build configuration via CMake options
|
||||
cp ../cmake/config.cmake .
|
||||
|
||||
We want to specifically tweak the following flags by appending them to the end of the configuration file:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# controls default compilation flags (Candidates: Release, Debug, RelWithDebInfo)
|
||||
echo "set(CMAKE_BUILD_TYPE RelWithDebInfo)" >> config.cmake
|
||||
|
||||
# LLVM is a must dependency for compiler end
|
||||
echo "set(USE_LLVM \"llvm-config --ignore-libllvm --link-static\")" >> config.cmake
|
||||
echo "set(HIDE_PRIVATE_SYMBOLS ON)" >> config.cmake
|
||||
|
||||
# GPU SDKs, turn on if needed
|
||||
echo "set(USE_CUDA OFF)" >> config.cmake
|
||||
echo "set(USE_METAL OFF)" >> config.cmake
|
||||
echo "set(USE_VULKAN OFF)" >> config.cmake
|
||||
echo "set(USE_OPENCL OFF)" >> config.cmake
|
||||
|
||||
# cuBLAS, cuDNN, cutlass support, turn on if needed
|
||||
echo "set(USE_CUBLAS OFF)" >> config.cmake
|
||||
echo "set(USE_CUDNN OFF)" >> config.cmake
|
||||
echo "set(USE_CUTLASS OFF)" >> config.cmake
|
||||
|
||||
|
||||
.. note::
|
||||
``HIDE_PRIVATE_SYMBOLS`` is a configuration option that enables the ``-fvisibility=hidden`` flag.
|
||||
This flag helps prevent potential symbol conflicts between TVM and PyTorch. These conflicts arise due to
|
||||
the frameworks shipping LLVMs of different versions.
|
||||
|
||||
`CMAKE_BUILD_TYPE <https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html>`_ controls default compilation flag:
|
||||
|
||||
- ``Debug`` sets ``-O0 -g``
|
||||
- ``RelWithDebInfo`` sets ``-O2 -g -DNDEBUG`` (recommended)
|
||||
- ``Release`` sets ``-O3 -DNDEBUG``
|
||||
|
||||
Once ``config.cmake`` is edited accordingly, kick off build with the commands below:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake .. && cmake --build . --parallel $(nproc)
|
||||
|
||||
.. note::
|
||||
``nproc`` may not be available on all systems, please replace it with the number of cores on your system
|
||||
|
||||
A success build should produce ``libtvm`` and ``libtvm_runtime`` under ``build/`` directory.
|
||||
|
||||
Apache TVM relies on the tvm-ffi package to support its python bindings.
|
||||
Therefore, after we finish the build, we need to install the tvm-ffi package.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd 3rdparty/tvm-ffi; pip install .; cd ..
|
||||
|
||||
|
||||
Leaving the build environment ``tvm-build-venv``, there are two ways to install the successful build into your environment:
|
||||
|
||||
- Install via environment variable
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export TVM_HOME=/path-to-tvm
|
||||
pip install --target=$TVM_HOME/python $TVM_HOME/3rdparty/tvm-ffi
|
||||
export PYTHONPATH=$TVM_HOME/python:$PYTHONPATH
|
||||
|
||||
- Install via pip local project
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
conda activate your-own-env
|
||||
conda install python # make sure python is installed
|
||||
export TVM_LIBRARY_PATH=/path-to-tvm/build
|
||||
pip install -e /path-to-tvm
|
||||
|
||||
Step 4. Validate Installation
|
||||
-----------------------------
|
||||
|
||||
Using a compiler infrastructure with multiple language bindings could be error-prone.
|
||||
Therefore, it is highly recommended to validate Apache TVM installation before use.
|
||||
|
||||
**Step 1. Locate TVM Python package.** The following command can help confirm that TVM is properly installed as a python package and provide the location of the TVM python package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
>>> python -c "import tvm; print(tvm.__file__)"
|
||||
/some-path/lib/python3.11/site-packages/tvm/__init__.py
|
||||
|
||||
**Step 2. Confirm which TVM library is used.** When maintaining multiple build or installation of TVM, it becomes important to double check if the python package is using the proper ``libtvm`` with the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
>>> python -c "import tvm; print(tvm.base._LOADED_LIBS['tvm_runtime'])"
|
||||
<CDLL '/some-path/lib/python3.11/site-packages/tvm/libtvm_runtime.dylib', handle 95ada510 at 0x1030e4e50>
|
||||
|
||||
**Step 3. Reflect TVM build option.** Sometimes when downstream application fails, it could likely be some mistakes with a wrong TVM commit, or wrong build flags. To find it out, the following commands will be helpful:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
>>> python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))"
|
||||
... # Omitted less relevant options
|
||||
GIT_COMMIT_HASH: 4f6289590252a1cf45a4dc37bce55a25043b8338
|
||||
HIDE_PRIVATE_SYMBOLS: ON
|
||||
USE_LLVM: llvm-config --link-static
|
||||
LLVM_VERSION: 15.0.7
|
||||
USE_VULKAN: OFF
|
||||
USE_CUDA: OFF
|
||||
CUDA_VERSION: NOT-FOUND
|
||||
USE_OPENCL: OFF
|
||||
USE_METAL: ON
|
||||
USE_ROCM: OFF
|
||||
|
||||
|
||||
**Step 4. Check device detection.** Sometimes it could be helpful to understand if TVM could detect your device at all with the following commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
>>> python -c "import tvm; print(tvm.metal().exist)"
|
||||
True # or False
|
||||
>>> python -c "import tvm; print(tvm.cuda().exist)"
|
||||
False # or True
|
||||
>>> python -c "import tvm; print(tvm.vulkan().exist)"
|
||||
False # or True
|
||||
|
||||
Please note that the commands above verify the presence of an actual device on the local machine for the TVM runtime (not the compiler) to execute properly. However, TVM compiler can perform compilation tasks without requiring a physical device. As long as the necessary toolchain, such as NVCC, is available, TVM supports cross-compilation even in the absence of an actual device.
|
||||
|
||||
|
||||
Step 5. Extra Python Dependencies
|
||||
---------------------------------
|
||||
Building from source does not ensure the installation of all necessary Python dependencies.
|
||||
|
||||
Python Dependencies
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
The following commands can be used to install the extra Python dependencies:
|
||||
|
||||
* Necessary dependencies:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
pip3 install numpy cython
|
||||
|
||||
* If you want to use RPC Tracker
|
||||
|
||||
.. code:: bash
|
||||
|
||||
pip3 install tornado
|
||||
|
||||
* If you want to use auto-tuning module
|
||||
|
||||
.. code:: bash
|
||||
|
||||
pip3 install tornado psutil 'xgboost>=1.1.0' cloudpickle
|
||||
|
||||
Windows-Specific Build Notes
|
||||
----------------------------
|
||||
|
||||
If you're building TVM on Windows, note these platform-specific considerations:
|
||||
|
||||
Path Conventions
|
||||
~~~~~~~~~~~~~~~~
|
||||
- Use forward slashes (``/``) in Python/CMake paths, not Windows backslashes
|
||||
- Example: ``python cmake/config.cmake`` not ``python cmake\\config.cmake``
|
||||
|
||||
Advanced Build Configuration
|
||||
----------------------------
|
||||
|
||||
Ccache
|
||||
~~~~~~
|
||||
On supported platforms, the `Ccache compiler wrapper <https://ccache.dev/>`_ may be helpful for
|
||||
reducing TVM's build time, especially when building with `cutlass <https://github.com/NVIDIA/cutlass>`_.
|
||||
There are several ways to enable CCache in TVM builds:
|
||||
|
||||
- Leave ``USE_CCACHE=AUTO`` in ``build/config.cmake``. CCache will be used if it is found.
|
||||
|
||||
- Ccache's Masquerade mode. This is typically enabled during the Ccache installation process.
|
||||
To have TVM use Ccache in masquerade, simply specify the appropriate C/C++ compiler
|
||||
paths when configuring TVM's build system. For example:
|
||||
``cmake -DCMAKE_CXX_COMPILER=/usr/lib/ccache/c++ ...``.
|
||||
|
||||
- Ccache as CMake's C++ compiler prefix. When configuring TVM's build system,
|
||||
set the CMake variable ``CMAKE_CXX_COMPILER_LAUNCHER`` to an appropriate value.
|
||||
E.g. ``cmake -DCMAKE_CXX_COMPILER_LAUNCHER=ccache ...``.
|
||||
|
||||
|
||||
Building on Windows
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
TVM support build via MSVC using CMake. You will need to obtain a visual studio compiler.
|
||||
The minimum required VS version is **Visual Studio Enterprise 2019** (NOTE: we test
|
||||
against GitHub Actions' `windows-latest Runner <https://github.com/actions/runner-images>`_, so see that page for full details.
|
||||
We recommend following :ref:`install-dependencies` to obtain necessary dependencies and
|
||||
get an activated tvm-build environment. Then you can run the following command to build
|
||||
|
||||
.. code:: bash
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
cd ..
|
||||
|
||||
The above command generates the solution file under the build directory.
|
||||
You can then run the following command to build
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cmake --build build --config Release -- /m
|
||||
|
||||
CUDA Configuration
|
||||
..................
|
||||
For CUDA support on Windows:
|
||||
|
||||
.. code-block:: batch
|
||||
|
||||
set CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
|
||||
set PATH=%CUDA_PATH%\bin;%PATH%
|
||||
cmake .. -DUSE_CUDA=ON
|
||||
|
||||
CMake & Compiler Setup
|
||||
......................
|
||||
- Specify generator: ``cmake -G "Visual Studio 16 2019" -A x64 ..``
|
||||
- Ensure Python is in PATH or specify: ``-DPython_EXECUTABLE=C:\Python311\python.exe``
|
||||
|
||||
|
||||
Building ROCm support
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Currently, ROCm is supported only on linux, so all the instructions are written with linux in mind.
|
||||
|
||||
- Set ``set(USE_ROCM ON)``, set ROCM_PATH to the correct path.
|
||||
- You need to first install HIP runtime from ROCm. Make sure the installation system has ROCm installed in it.
|
||||
- Install LLVM (>= 15), and LLD, make sure ``ld.lld`` is available via command line.
|
||||
|
||||
.. _install-from-source-cpp-tests:
|
||||
|
||||
Enable C++ Tests
|
||||
~~~~~~~~~~~~~~~~
|
||||
We use `Google Test <https://github.com/google/googletest>`_ to drive the C++
|
||||
tests in TVM. The easiest way to install GTest is from source.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
git clone https://github.com/google/googletest
|
||||
cd googletest
|
||||
mkdir build
|
||||
cd build
|
||||
cmake -DBUILD_SHARED_LIBS=ON ..
|
||||
make
|
||||
sudo make install
|
||||
|
||||
After installing GTest, the C++ tests can be built and started with ``./tests/scripts/task_cpp_unittest.sh``, or built via CMake with ``-DUSE_GTEST=ON`` and then running ``./build/cpptest``.
|
||||
@@ -0,0 +1,41 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _installation:
|
||||
|
||||
Installing TVM
|
||||
==============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
pypi
|
||||
from_source
|
||||
docker
|
||||
|
||||
For most Python users, the quickest way to get started is to
|
||||
:ref:`install the Apache TVM wheel from PyPI <install-from-pypi>`.
|
||||
|
||||
Visit the :ref:`install TVM from source <install-from-source>` page to install
|
||||
TVM from the source code. Installing from source gives you the maximum
|
||||
flexibility to configure the build effectively from the official source
|
||||
releases. If you are interested in deploying to mobile or embedded devices,
|
||||
you do not need to install the entire TVM stack on your device. Instead, you
|
||||
only need the runtime.
|
||||
|
||||
If you would like to quickly try out TVM or run some demo and tutorials, you
|
||||
can :ref:`install from Docker <docker-images>`.
|
||||
@@ -0,0 +1,51 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
.. http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
.. Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _install-from-pypi:
|
||||
|
||||
Install from PyPI
|
||||
=================
|
||||
|
||||
For most Python users, the quickest way to get started is to install the Apache
|
||||
TVM wheel from PyPI:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install apache-tvm
|
||||
|
||||
This installs the Python package, including modules such as ``tvm.tirx``, and
|
||||
is suitable for trying tutorials that do not require a custom build.
|
||||
|
||||
CUDA environments
|
||||
-----------------
|
||||
|
||||
Some CUDA workflows use NVIDIA's Python CUDA bindings for runtime compilation.
|
||||
Install the CUDA extra in the same environment as TVM when you need this path:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install "apache-tvm[cuda]"
|
||||
|
||||
This extra installs Python-side CUDA bindings only. It does not make the PyPI
|
||||
wheel a CUDA-enabled TVM build, and it does not install NVIDIA drivers or a CUDA
|
||||
toolkit. If you need CUDA support in TVM itself, build TVM from source with
|
||||
``USE_CUDA=ON``.
|
||||
|
||||
For more details on installing the TIRx compiler and optional kernel library,
|
||||
visit the :doc:`TIRx installation </tirx/install>` page. If you need to
|
||||
customize TVM's build configuration, visit the
|
||||
:ref:`install TVM from source <install-from-source>` page instead.
|
||||
Reference in New Issue
Block a user