wehub-resource-sync
1.6 KiB
1.6 KiB
Multiple GPU Training
Requirements
pip install torchmetrics==0.11.4
How to run
Graph property prediction
Run with following (available dataset: "ogbg-molhiv", "ogbg-molpcba")
python3 multi_gpu_graph_prediction.py --dataset ogbg-molhiv
Results
* ogbg-molhiv: ~0.7965
* ogbg-molpcba: ~0.2239
Scalability
We test scalability of the code with dataset "ogbg-molhiv" in a machine of type Amazon EC2 g4dn.metal , which has 8 Nvidia T4 Tensor Core GPUs.
| GPU number | Speed Up | Batch size | Test accuracy | Average epoch Time |
|---|---|---|---|---|
| 1 | x | 32 | 0.7765 | 45.0s |
| 2 | 3.7x | 64 | 0.7761 | 12.1s |
| 4 | 5.9x | 128 | 0.7854 | 7.6s |
| 8 | 9.5x | 256 | 0.7751 | 4.7s |
Node classification
Run with following on dataset "ogbn-products"
python3 multi_gpu_node_classification.py
Results
Test Accuracy: ~0.7632
Link prediction
Run with following (available dataset: "ogbn-products", "reddit")
python3 multi_gpu_link_prediction.py --dataset ogbn-products
Results
Eval F1-score: ~0.7999 Test F1-score: ~0.6383
Notably,
- The loss function is defined by predicting whether an edge exists between two nodes or not.
- When computing the score of
(u, v), the connections between nodeuandvare removed from neighbor sampling. - The performance of the learned embeddings are measured by training a softmax regression with scikit-learn.