chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,2 @@
|
||||
Training on CPUs
|
||||
=========================
|
||||
@@ -0,0 +1,151 @@
|
||||
"""
|
||||
Improve Scalability on Multi-Core CPUs
|
||||
=====================================================
|
||||
|
||||
Graph Neural Network (GNN) training suffers from low scalability on multi-core CPUs.
|
||||
Specificially, the performance often caps at 16 cores, and no improvement is observed when applying more than 16 cores [#f1]_.
|
||||
ARGO is a runtime system that offers scalable performance.
|
||||
With ARGO enabled, we are able to scale over 64 cores, allowing ARGO to speedup GNN training (in terms of epoch time) by up to 4.30x and 3.32x on a Xeon 8380H and a Xeon 6430L, respectively [#f2]_.
|
||||
This chapter focus on how to setup ARGO to unleash the potential of multi-core CPUs to speedup GNN training.
|
||||
|
||||
Installation
|
||||
`````````````````````````````
|
||||
|
||||
ARGO utilizes the scikit-optimize library for auto-tuning. Please install scikit-optimize to run ARGO:
|
||||
.. code-block:: shell
|
||||
conda install -c conda-forge "scikit-optimize>=0.9.0"
|
||||
or
|
||||
.. code-block:: shell
|
||||
pip install scikit-optimize>=0.9
|
||||
|
||||
Enabling ARGO on your own GNN program
|
||||
```````````````````````````````````````````
|
||||
In this section, we provide a step-by-step tutorial on how to enable ARGO on a DGL program.
|
||||
We use the *ogb_example.py* [#f3]_ as an example.
|
||||
.. note::
|
||||
We also provide the complete example file *ogb_example_ARGO.py* [#f4]_
|
||||
which followed the steps below to enable ARGO on *ogb_example.py*.
|
||||
|
||||
Step 1
|
||||
---------------------------
|
||||
First, include all necessary packages on top of the file. Please place your file and *argo.py* [#f5]_ in the same directory.
|
||||
|
||||
.. code-block:: python
|
||||
import os
|
||||
import torch.distributed as dist
|
||||
from torch.nn.parallel import DistributedDataParallel
|
||||
import torch.multiprocessing as mp
|
||||
from argo import ARGO
|
||||
|
||||
Step 2
|
||||
---------------------------
|
||||
Setup PyTorch Distributed Data Parallel (DDP)
|
||||
|
||||
2.1. Add the initialization function on top of the training program, and wrap the ```model``` with the DDP wrapper
|
||||
.. code-block:: python
|
||||
def train(...):
|
||||
dist.init_process_group('gloo', rank=rank, world_size=world_size) # newly added
|
||||
model = SAGE(...) # original code
|
||||
model = DistributedDataParallel(model) # newly added
|
||||
...
|
||||
|
||||
2.2. In the main program, add the following before launching the training function
|
||||
.. code-block:: python
|
||||
...
|
||||
os.environ['MASTER_ADDR'] = '127.0.0.1'
|
||||
os.environ['MASTER_PORT'] = '29501'
|
||||
mp.set_start_method('fork', force=True)
|
||||
train(args, device, data) # original code for launching the training function
|
||||
|
||||
Step 3
|
||||
---------------------------
|
||||
Enable ARGO by initializing the runtime system, and wrapping the training function
|
||||
.. code-block:: python
|
||||
runtime = ARGO(n_search = 15, epoch = args.num_epochs, batch_size = args.batch_size) # initialization
|
||||
runtime.run(train, args=(args, device, data)) # wrap the training function
|
||||
|
||||
.. note::
|
||||
ARGO takes three input parameters: number of searches *n_search*, number of epochs, and the mini-batch size.
|
||||
Increasing *n_search* potentially leads to a better configuration with less epoch time;
|
||||
however, searching itself also causes extra overhead. We recommend setting *n_search* from 15 to 45 for an optimal overall performance.
|
||||
|
||||
Step 4
|
||||
---------------------------
|
||||
Modify the input of the training function, by directly adding ARGO parameters after the original inputs.
|
||||
|
||||
This is the original function:
|
||||
.. code-block:: python
|
||||
def train(args, device, data):
|
||||
|
||||
Add the following variables: *rank, world_size, comp_core, load_core, counter, b_size, ep*
|
||||
.. code-block:: python
|
||||
def train(args, device, data, rank, world_size, comp_core, load_core, counter, b_size, ep):
|
||||
|
||||
Step 5
|
||||
---------------------------
|
||||
Modify the *dataloader* function in the training function
|
||||
.. code-block:: python
|
||||
dataloader = dgl.dataloading.DataLoader(
|
||||
g,
|
||||
train_nid,
|
||||
sampler,
|
||||
batch_size=b_size, # modified
|
||||
shuffle=True,
|
||||
drop_last=False,
|
||||
num_workers=len(load_core), # modified
|
||||
use_ddp = True) # newly added
|
||||
|
||||
Step 6
|
||||
---------------------------
|
||||
Enable core-binding by adding *enable_cpu_affinity()* before the training for-loop, and also change the number of epochs into the variable *ep*:
|
||||
.. code-block:: python
|
||||
with dataloader.enable_cpu_affinity(loader_cores=load_core, compute_cores=comp_core):
|
||||
for epoch in range(ep): # change num_epochs to ep
|
||||
|
||||
Step 7
|
||||
---------------------------
|
||||
Last step! Load the model before training and save it afterward.
|
||||
|
||||
Original Program:
|
||||
.. code-block:: python
|
||||
with dataloader.enable_cpu_affinity(loader_cores=load_core, compute_cores=comp_core):
|
||||
for epoch in range(ep):
|
||||
... # training operations
|
||||
|
||||
Modified:
|
||||
.. code-block:: python
|
||||
PATH = "model.pt"
|
||||
if counter[0] != 0:
|
||||
checkpoint = th.load(PATH)
|
||||
model.load_state_dict(checkpoint['model_state_dict'])
|
||||
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
|
||||
epoch = checkpoint['epoch']
|
||||
loss = checkpoint['loss']
|
||||
|
||||
with dataloader.enable_cpu_affinity(loader_cores=load_core, compute_cores=comp_core):
|
||||
for epoch in range(ep):
|
||||
... # training operations
|
||||
|
||||
dist.barrier()
|
||||
if rank == 0:
|
||||
th.save({'epoch': counter[0],
|
||||
'model_state_dict': model.state_dict(),
|
||||
'optimizer_state_dict': optimizer.state_dict(),
|
||||
'loss': loss,
|
||||
}, PATH)
|
||||
|
||||
Step 8
|
||||
---------------------------
|
||||
Done! You can now run your GNN program with ARGO enabled.
|
||||
.. code-block:: shell
|
||||
python <your_code>.py
|
||||
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
.. [#f1] https://github.com/dmlc/dgl/blob/master/examples/pytorch/argo/argo_scale.png
|
||||
.. [#f2] https://arxiv.org/abs/2402.03671
|
||||
.. [#f3] https://github.com/dmlc/dgl/blob/master/examples/pytorch/argo/ogb_example.py
|
||||
.. [#f4] https://github.com/dmlc/dgl/blob/master/examples/pytorch/argo/ogb_example_ARGO.py
|
||||
.. [#f5] https://github.com/dmlc/dgl/blob/master/examples/pytorch/argo/argo.py
|
||||
"""
|
||||
@@ -0,0 +1,88 @@
|
||||
"""
|
||||
CPU Best Practices
|
||||
=====================================================
|
||||
|
||||
This chapter focus on providing best practises for environment setup
|
||||
to get the best performance during training and inference on the CPU.
|
||||
|
||||
Intel
|
||||
`````````````````````````````
|
||||
|
||||
Hyper-threading
|
||||
---------------------------
|
||||
|
||||
For specific workloads as GNN’s domain, suggested default setting for having best performance
|
||||
is to turn off hyperthreading.
|
||||
Turning off the hyper threading feature can be done at BIOS [#f1]_ or operating system level [#f2]_ [#f3]_ .
|
||||
|
||||
Alternative memory allocators
|
||||
---------------------------
|
||||
|
||||
Alternative memory allocators, such as *tcmalloc*, might provide significant performance improvements by more efficient memory usage, reducing overhead on unnecessary memory allocations or deallocations. *tcmalloc* uses thread-local caches to reduce overhead on thread synchronization, locks contention by using spinlocks and per-thread arenas respectively and categorizes memory allocations by sizes to reduce overhead on memory fragmentation.
|
||||
|
||||
To take advantage of optimizations *tcmalloc* provides, install it on your system (on Ubuntu *tcmalloc* is included in libgoogle-perftools4 package) and add shared library to the LD_PRELOAD environment variable:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export LD_PRELOAD=/lib/x86_64-linux-gnu/libtcmalloc.so.4:$LD_PRELOAD
|
||||
|
||||
OpenMP settings
|
||||
---------------------------
|
||||
|
||||
As `OpenMP` is the default parallel backend, we could control performance
|
||||
including sampling and training via `dgl.utils.set_num_threads()`.
|
||||
|
||||
If number of OpenMP threads is not set and `num_workers` in dataloader is set
|
||||
to 0, the OpenMP runtime typically use the number of available CPU cores by
|
||||
default. This works well for most cases, and is also the default behavior in DGL.
|
||||
|
||||
If `num_workers` in dataloader is set to greater than 0, the number of
|
||||
OpenMP threads will be set to **1** for each worker process. This is the
|
||||
default behavior in PyTorch. In this case, we can set the number of OpenMP
|
||||
threads to the number of CPU cores in the main process.
|
||||
|
||||
Performance tuning is highly dependent on the workload and hardware
|
||||
configuration. We recommend users to try different settings and choose the
|
||||
best one for their own cases.
|
||||
|
||||
**Dataloader CPU affinity**
|
||||
|
||||
.. note::
|
||||
|
||||
This feature is available for `dgl.dataloading.DataLoader` only. Not
|
||||
available for dataloaders in `dgl.graphbolt` yet.
|
||||
|
||||
|
||||
If number of dataloader workers is more than 0, please consider using **use_cpu_affinity()** method
|
||||
of DGL Dataloader class, it will generally result in significant performance improvement for training.
|
||||
|
||||
*use_cpu_affinity* will set the proper OpenMP thread count (equal to the number of CPU cores allocated for main process),
|
||||
affinitize dataloader workers for separate CPU cores and restrict the main process to remaining cores
|
||||
|
||||
In multiple NUMA nodes setups *use_cpu_affinity* will only use cores of NUMA node 0 by default
|
||||
with an assumption, that the workload is scaling poorly across multiple NUMA nodes. If you believe
|
||||
your workload will have better performance utilizing more than one NUMA node, you can pass
|
||||
the list of cores to use for dataloading (loader_cores) and for compute (compute_cores).
|
||||
|
||||
loader_cores and compute_cores arguments (list of CPU cores) can be passed to *enable_cpu_affinity* for more
|
||||
control over which cores should be used, e.g. in case a workload scales well across multiple NUMA nodes.
|
||||
|
||||
Usage:
|
||||
.. code:: python
|
||||
|
||||
dataloader = dgl.dataloading.DataLoader(...)
|
||||
...
|
||||
with dataloader.enable_cpu_affinity():
|
||||
<training loop or inferencing>
|
||||
|
||||
**Manual control**
|
||||
|
||||
For advanced and more fine-grained control over OpenMP settings please refer to Maximize Performance of Intel® Optimization for PyTorch* on CPU [#f4]_ article
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
.. [#f1] https://www.intel.com/content/www/us/en/support/articles/000007645/boards-and-kits/desktop-boards.html
|
||||
.. [#f2] https://aws.amazon.com/blogs/compute/disabling-intel-hyper-threading-technology-on-amazon-linux/
|
||||
.. [#f3] https://aws.amazon.com/blogs/compute/disabling-intel-hyper-threading-technology-on-amazon-ec2-windows-instances/
|
||||
.. [#f4] https://software.intel.com/content/www/us/en/develop/articles/how-to-get-better-performance-on-pytorchcaffe2-with-intel-acceleration.html
|
||||
"""
|
||||
Reference in New Issue
Block a user