chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,23 @@
|
||||
.wy-table-responsive table td,
|
||||
.wy-table-responsive table th {
|
||||
white-space: normal;
|
||||
}
|
||||
|
||||
.wy-table-bordered-all,
|
||||
.rst-content table.docutils {
|
||||
border: none;
|
||||
}
|
||||
|
||||
.wy-table-bordered-all td,
|
||||
.rst-content table.docutils td {
|
||||
border: none;
|
||||
}
|
||||
|
||||
.wy-table td,
|
||||
.rst-content table.docutils td,
|
||||
.rst-content table.field-list td,
|
||||
.wy-table th,
|
||||
.rst-content table.docutils th,
|
||||
.rst-content table.field-list th {
|
||||
padding: 14px;
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 103 KiB |
@@ -0,0 +1,10 @@
|
||||
.. role:: hidden
|
||||
:class: hidden-section
|
||||
.. currentmodule:: {{ module }}
|
||||
|
||||
|
||||
{{ name | underline}}
|
||||
|
||||
.. autoclass:: {{ name }}
|
||||
:show-inheritance:
|
||||
:members: __getitem__, __len__, collate_fn, forward, reset_parameters, rel_emb, rel_project, explain_node, explain_graph, train_step, train_step_node
|
||||
@@ -0,0 +1,11 @@
|
||||
.. role:: hidden
|
||||
:class: hidden-section
|
||||
.. currentmodule:: {{ module }}
|
||||
|
||||
|
||||
{{ name | underline}}
|
||||
|
||||
.. autoclass:: {{ name }}
|
||||
:show-inheritance:
|
||||
:members:
|
||||
:member-order: groupwise
|
||||
@@ -0,0 +1,226 @@
|
||||
.. _apigraph:
|
||||
|
||||
dgl.DGLGraph
|
||||
=====================================================
|
||||
|
||||
.. currentmodule:: dgl
|
||||
.. class:: DGLGraph
|
||||
|
||||
Class for storing graph structure and node/edge feature data.
|
||||
|
||||
There are a few ways to create a DGLGraph:
|
||||
|
||||
* To create a homogeneous graph from Tensor data, use :func:`dgl.graph`.
|
||||
* To create a heterogeneous graph from Tensor data, use :func:`dgl.heterograph`.
|
||||
* To create a graph from other data sources, use ``dgl.*`` create ops. See
|
||||
:ref:`api-graph-create-ops`.
|
||||
|
||||
Read the user guide chapter :ref:`guide-graph` for an in-depth explanation about its
|
||||
usage.
|
||||
|
||||
Querying metagraph structure
|
||||
----------------------------
|
||||
|
||||
Methods for getting information about the node and edge types. They are typically useful
|
||||
when the graph is heterogeneous.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.ntypes
|
||||
DGLGraph.etypes
|
||||
DGLGraph.srctypes
|
||||
DGLGraph.dsttypes
|
||||
DGLGraph.canonical_etypes
|
||||
DGLGraph.metagraph
|
||||
DGLGraph.to_canonical_etype
|
||||
|
||||
.. _apigraph-querying-graph-structure:
|
||||
|
||||
Querying graph structure
|
||||
------------------------
|
||||
|
||||
Methods for getting information about the graph structure such as capacity, connectivity,
|
||||
neighborhood, etc.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.num_nodes
|
||||
DGLGraph.number_of_nodes
|
||||
DGLGraph.num_edges
|
||||
DGLGraph.number_of_edges
|
||||
DGLGraph.num_src_nodes
|
||||
DGLGraph.number_of_src_nodes
|
||||
DGLGraph.num_dst_nodes
|
||||
DGLGraph.number_of_dst_nodes
|
||||
DGLGraph.is_unibipartite
|
||||
DGLGraph.is_multigraph
|
||||
DGLGraph.is_homogeneous
|
||||
DGLGraph.has_nodes
|
||||
DGLGraph.has_edges_between
|
||||
DGLGraph.predecessors
|
||||
DGLGraph.successors
|
||||
DGLGraph.edge_ids
|
||||
DGLGraph.find_edges
|
||||
DGLGraph.in_edges
|
||||
DGLGraph.out_edges
|
||||
DGLGraph.in_degrees
|
||||
DGLGraph.out_degrees
|
||||
|
||||
Querying and manipulating sparse format
|
||||
---------------------------------------
|
||||
|
||||
Methods for getting or manipulating the internal storage formats of a ``DGLGraph``.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.formats
|
||||
DGLGraph.create_formats_
|
||||
|
||||
Querying and manipulating node/edge ID type
|
||||
-----------------------------------------
|
||||
|
||||
Methods for getting or manipulating the data type for storing structure-related
|
||||
data such as node and edge IDs.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.idtype
|
||||
DGLGraph.long
|
||||
DGLGraph.int
|
||||
|
||||
Using Node/edge features
|
||||
------------------------
|
||||
|
||||
Methods for getting or setting the data type for storing structure-related
|
||||
data such as node and edge IDs.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.nodes
|
||||
DGLGraph.ndata
|
||||
DGLGraph.edges
|
||||
DGLGraph.edata
|
||||
DGLGraph.node_attr_schemes
|
||||
DGLGraph.edge_attr_schemes
|
||||
DGLGraph.srcnodes
|
||||
DGLGraph.dstnodes
|
||||
DGLGraph.srcdata
|
||||
DGLGraph.dstdata
|
||||
|
||||
Transforming graph
|
||||
------------------
|
||||
|
||||
Methods for generating a new graph by transforming the current ones. Most of them
|
||||
are alias of the :ref:`api-subgraph-extraction` and :ref:`api-transform`
|
||||
under the ``dgl`` namespace.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.subgraph
|
||||
DGLGraph.edge_subgraph
|
||||
DGLGraph.node_type_subgraph
|
||||
DGLGraph.edge_type_subgraph
|
||||
DGLGraph.__getitem__
|
||||
DGLGraph.line_graph
|
||||
DGLGraph.reverse
|
||||
DGLGraph.add_self_loop
|
||||
DGLGraph.remove_self_loop
|
||||
DGLGraph.to_simple
|
||||
DGLGraph.to_cugraph
|
||||
DGLGraph.reorder_graph
|
||||
|
||||
Adjacency and incidence matrix
|
||||
---------------------------------
|
||||
|
||||
Methods for getting the adjacency and the incidence matrix of the graph.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.adj
|
||||
DGLGraph.adjacency_matrix
|
||||
DGLGraph.adj_tensors
|
||||
DGLGraph.adj_external
|
||||
DGLGraph.inc
|
||||
DGLGraph.incidence_matrix
|
||||
|
||||
Computing with DGLGraph
|
||||
-----------------------------
|
||||
|
||||
Methods for performing message passing, applying functions on node/edge features, etc.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.apply_nodes
|
||||
DGLGraph.apply_edges
|
||||
DGLGraph.send_and_recv
|
||||
DGLGraph.pull
|
||||
DGLGraph.push
|
||||
DGLGraph.update_all
|
||||
DGLGraph.multi_update_all
|
||||
DGLGraph.prop_nodes
|
||||
DGLGraph.prop_edges
|
||||
DGLGraph.filter_nodes
|
||||
DGLGraph.filter_edges
|
||||
|
||||
Querying and manipulating batch information
|
||||
----------------------------------------------
|
||||
|
||||
Methods for getting/setting the batching information if the current graph is a batched
|
||||
graph generated from :func:`dgl.batch`. They are also widely used in the
|
||||
:ref:`api-batch`.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.batch_size
|
||||
DGLGraph.batch_num_nodes
|
||||
DGLGraph.batch_num_edges
|
||||
DGLGraph.set_batch_num_nodes
|
||||
DGLGraph.set_batch_num_edges
|
||||
|
||||
|
||||
Mutating topology
|
||||
-----------------
|
||||
|
||||
Methods for mutating the graph structure *in-place*.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.add_nodes
|
||||
DGLGraph.add_edges
|
||||
DGLGraph.remove_nodes
|
||||
DGLGraph.remove_edges
|
||||
|
||||
Device Control
|
||||
--------------
|
||||
|
||||
Methods for getting or changing the device on which the graph is hosted.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.to
|
||||
DGLGraph.device
|
||||
DGLGraph.cpu
|
||||
DGLGraph.pin_memory_
|
||||
DGLGraph.unpin_memory_
|
||||
DGLGraph.is_pinned
|
||||
|
||||
Misc
|
||||
----
|
||||
|
||||
Other utility methods.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
DGLGraph.local_scope
|
||||
@@ -0,0 +1,148 @@
|
||||
.. _apidata:
|
||||
|
||||
dgl.data
|
||||
=========
|
||||
|
||||
.. currentmodule:: dgl.data
|
||||
.. automodule:: dgl.data
|
||||
|
||||
Base Class
|
||||
---------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
DGLDataset
|
||||
CSVDataset
|
||||
|
||||
Node Prediction Datasets
|
||||
---------------------------------------
|
||||
|
||||
Datasets for node classification/regression tasks
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
SSTDataset
|
||||
KarateClubDataset
|
||||
CoraGraphDataset
|
||||
CiteseerGraphDataset
|
||||
PubmedGraphDataset
|
||||
CoraFullDataset
|
||||
AIFBDataset
|
||||
MUTAGDataset
|
||||
BGSDataset
|
||||
AMDataset
|
||||
AmazonCoBuyComputerDataset
|
||||
AmazonCoBuyPhotoDataset
|
||||
CoauthorCSDataset
|
||||
CoauthorPhysicsDataset
|
||||
PPIDataset
|
||||
RedditDataset
|
||||
SBMMixtureDataset
|
||||
FraudDataset
|
||||
FraudYelpDataset
|
||||
FraudAmazonDataset
|
||||
BAShapeDataset
|
||||
BACommunityDataset
|
||||
TreeCycleDataset
|
||||
TreeGridDataset
|
||||
WikiCSDataset
|
||||
FlickrDataset
|
||||
YelpDataset
|
||||
PATTERNDataset
|
||||
CLUSTERDataset
|
||||
ChameleonDataset
|
||||
SquirrelDataset
|
||||
ActorDataset
|
||||
CornellDataset
|
||||
TexasDataset
|
||||
WisconsinDataset
|
||||
RomanEmpireDataset
|
||||
AmazonRatingsDataset
|
||||
MinesweeperDataset
|
||||
TolokersDataset
|
||||
QuestionsDataset
|
||||
MovieLensDataset
|
||||
VOCSuperpixelsDataset
|
||||
COCOSuperpixelsDataset
|
||||
|
||||
|
||||
Edge Prediction Datasets
|
||||
---------------------------------------
|
||||
|
||||
Datasets for edge classification/regression and link prediction
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
FB15k237Dataset
|
||||
FB15kDataset
|
||||
WN18Dataset
|
||||
BitcoinOTCDataset
|
||||
ICEWS18Dataset
|
||||
GDELTDataset
|
||||
|
||||
Graph Prediction Datasets
|
||||
---------------------------------------
|
||||
|
||||
Datasets for graph classification/regression tasks
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
QM7bDataset
|
||||
QM9Dataset
|
||||
QM9EdgeDataset
|
||||
MiniGCDataset
|
||||
TUDataset
|
||||
LegacyTUDataset
|
||||
GINDataset
|
||||
FakeNewsDataset
|
||||
BA2MotifDataset
|
||||
ZINCDataset
|
||||
PeptidesStructuralDataset
|
||||
PeptidesFunctionalDataset
|
||||
MNISTSuperPixelDataset
|
||||
CIFAR10SuperPixelDataset
|
||||
|
||||
Dataset adapters
|
||||
-------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
AsNodePredDataset
|
||||
AsLinkPredDataset
|
||||
AsGraphPredDataset
|
||||
|
||||
Utilities
|
||||
-----------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
utils.get_download_dir
|
||||
utils.download
|
||||
utils.check_sha1
|
||||
utils.extract_archive
|
||||
utils.split_dataset
|
||||
utils.load_labels
|
||||
utils.save_info
|
||||
utils.load_info
|
||||
utils.add_nodepred_split
|
||||
utils.mask_nodes_by_property
|
||||
utils.add_node_property_split
|
||||
utils.Subset
|
||||
@@ -0,0 +1,85 @@
|
||||
.. _api-dataloading:
|
||||
|
||||
dgl.dataloading
|
||||
=================================
|
||||
|
||||
.. currentmodule:: dgl.dataloading
|
||||
|
||||
The ``dgl.dataloading`` package provides two primitives to compose a data pipeline
|
||||
for loading from graph data. ``Sampler`` represents algorithms
|
||||
to generate subgraph samples from the original graph, and ``DataLoader``
|
||||
represents the iterable over these samples.
|
||||
|
||||
DGL provides a number of built-in samplers that subclass :class:`~dgl.dataloading.Sampler`.
|
||||
Creating new samplers follow the same paradigm. Read our user guide chapter
|
||||
:ref:`guide-minibatch` for more examples and explanations.
|
||||
|
||||
The entire package only works for PyTorch backend.
|
||||
|
||||
DataLoaders
|
||||
-----------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
DataLoader
|
||||
GraphDataLoader
|
||||
|
||||
.. _api-dataloading-neighbor-sampling:
|
||||
|
||||
Samplers
|
||||
--------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Sampler
|
||||
NeighborSampler
|
||||
LaborSampler
|
||||
MultiLayerFullNeighborSampler
|
||||
ClusterGCNSampler
|
||||
ShaDowKHopSampler
|
||||
SAINTSampler
|
||||
|
||||
Sampler Transformations
|
||||
-----------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
as_edge_prediction_sampler
|
||||
BlockSampler
|
||||
|
||||
.. _api-dataloading-negative-sampling:
|
||||
|
||||
Negative Samplers for Link Prediction
|
||||
-------------------------------------
|
||||
.. currentmodule:: dgl.dataloading.negative_sampler
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
Uniform
|
||||
PerSourceUniform
|
||||
GlobalUniform
|
||||
|
||||
Utility Class and Functions for Feature Prefetching
|
||||
---------------------------------------------------
|
||||
.. currentmodule:: dgl.dataloading.base
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
set_node_lazy_features
|
||||
set_edge_lazy_features
|
||||
set_src_lazy_features
|
||||
set_dst_lazy_features
|
||||
LazyFeature
|
||||
@@ -0,0 +1,115 @@
|
||||
.. _api-distributed:
|
||||
|
||||
dgl.distributed
|
||||
=================================
|
||||
|
||||
.. currentmodule:: dgl.distributed
|
||||
|
||||
DGL distributed module contains classes and functions to support
|
||||
distributed Graph Neural Network training and inference on a cluster of
|
||||
machines.
|
||||
|
||||
This includes a few submodules:
|
||||
|
||||
* distributed data structures including distributed graph, distributed tensor
|
||||
and distributed embeddings.
|
||||
* distributed sampling.
|
||||
* distributed workload split at runtime.
|
||||
* graph partition.
|
||||
|
||||
|
||||
Initialization
|
||||
---------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
initialize
|
||||
|
||||
Distributed Graph
|
||||
-----------------
|
||||
|
||||
.. autoclass:: DistGraph
|
||||
:members: ndata, edata, idtype, device, ntypes, etypes, number_of_nodes, number_of_edges, node_attr_schemes, edge_attr_schemes, rank, find_edges, get_partition_book, barrier, local_partition, num_nodes, num_edges, get_node_partition_policy, get_edge_partition_policy, get_etype_id, get_ntype_id, nodes, edges, out_degrees, in_degrees
|
||||
|
||||
Distributed Tensor
|
||||
------------------
|
||||
|
||||
.. autoclass:: DistTensor
|
||||
:members: part_policy, shape, dtype, name
|
||||
|
||||
Distributed Node Embedding
|
||||
---------------------
|
||||
|
||||
.. autoclass:: DistEmbedding
|
||||
|
||||
|
||||
Distributed embedding optimizer
|
||||
-------------------------
|
||||
|
||||
.. autoclass:: dgl.distributed.optim.SparseAdagrad
|
||||
:members: step, save, load
|
||||
|
||||
.. autoclass:: dgl.distributed.optim.SparseAdam
|
||||
:members: step, save, load
|
||||
|
||||
Distributed workload split
|
||||
--------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
node_split
|
||||
edge_split
|
||||
|
||||
Distributed Sampling
|
||||
--------------------
|
||||
|
||||
Distributed DataLoader
|
||||
``````````````````````
|
||||
|
||||
.. autoclass:: NodeCollator
|
||||
|
||||
.. autoclass:: EdgeCollator
|
||||
|
||||
.. autoclass:: DistDataLoader
|
||||
|
||||
.. autoclass:: DistNodeDataLoader
|
||||
|
||||
.. autoclass:: DistEdgeDataLoader
|
||||
|
||||
.. _api-distributed-sampling-ops:
|
||||
Distributed Graph Sampling Operators
|
||||
```````````````````````````````````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
sample_neighbors
|
||||
sample_etype_neighbors
|
||||
find_edges
|
||||
in_subgraph
|
||||
|
||||
Partition
|
||||
---------
|
||||
|
||||
Graph partition book
|
||||
````````````````````
|
||||
|
||||
.. autoclass:: GraphPartitionBook
|
||||
:members: shared_memory, num_partitions, metadata, nid2partid, eid2partid, partid2nids, partid2eids, nid2localnid, eid2localeid, partid, map_to_per_ntype, map_to_per_etype, map_to_homo_nid, map_to_homo_eid, canonical_etypes
|
||||
|
||||
.. autoclass:: PartitionPolicy
|
||||
:members: policy_str, part_id, partition_book, to_local, to_partid, get_part_size, get_size
|
||||
|
||||
Split and Load Partitions
|
||||
````````````````````````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
load_partition
|
||||
load_partition_feats
|
||||
load_partition_book
|
||||
partition_graph
|
||||
dgl_partition_to_graphbolt
|
||||
@@ -0,0 +1,190 @@
|
||||
.. _apifunction:
|
||||
|
||||
.. currentmodule:: dgl.function
|
||||
|
||||
dgl.function
|
||||
==================================
|
||||
|
||||
This subpackage hosts all the **built-in functions** provided by DGL. Built-in functions
|
||||
are DGL's recommended way to express different types of :ref:`guide-message-passing` computation
|
||||
(i.e., via :func:`~dgl.DGLGraph.update_all`) or computing edge-wise features from
|
||||
node-wise features (i.e., via :func:`~dgl.DGLGraph.apply_edges`). Built-in functions
|
||||
describe the node-wise and edge-wise computation in a symbolic way without any
|
||||
actual computation, so DGL can analyze and map them to efficient low-level kernels.
|
||||
Here are some examples:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import dgl.function as fn
|
||||
import torch as th
|
||||
g = ... # create a DGLGraph
|
||||
g.ndata['h'] = th.randn((g.num_nodes(), 10)) # each node has feature size 10
|
||||
g.edata['w'] = th.randn((g.num_edges(), 1)) # each edge has feature size 1
|
||||
# collect features from source nodes and aggregate them in destination nodes
|
||||
g.update_all(fn.copy_u('h', 'm'), fn.sum('m', 'h_sum'))
|
||||
# multiply source node features with edge weights and aggregate them in destination nodes
|
||||
g.update_all(fn.u_mul_e('h', 'w', 'm'), fn.max('m', 'h_max'))
|
||||
# compute edge embedding by multiplying source and destination node embeddings
|
||||
g.apply_edges(fn.u_mul_v('h', 'h', 'w_new'))
|
||||
|
||||
``fn.copy_u``, ``fn.u_mul_e``, ``fn.u_mul_v`` are built-in message functions, while ``fn.sum``
|
||||
and ``fn.max`` are built-in reduce functions. DGL's convention is to use ``u``, ``v``
|
||||
and ``e`` to represent source nodes, destination nodes, and edges, respectively.
|
||||
For example, ``copy_u`` tells DGL to copy the source node data as the messages;
|
||||
``u_mul_e`` tells DGL to multiply source node features with edge features.
|
||||
|
||||
To define a unary message function (e.g. ``copy_u``), specify one input feature name and one output
|
||||
message name. To define a binary message function (e.g. ``u_mul_e``), specify
|
||||
two input feature names and one output message name. During the computation,
|
||||
the message function will read the data under the given names, perform computation, and return
|
||||
the output using the output name. For example, the above ``fn.u_mul_e('h', 'w', 'm')`` is
|
||||
the same as the following user-defined function:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def udf_u_mul_e(edges):
|
||||
return {'m' : edges.src['h'] * edges.data['w']}
|
||||
|
||||
To define a reduce function, one input message name and one output node feature name
|
||||
need to be specified. For example, the above ``fn.max('m', 'h_max')`` is the same as the
|
||||
following user-defined function:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def udf_max(nodes):
|
||||
return {'h_max' : th.max(nodes.mailbox['m'], 1)[0]}
|
||||
|
||||
All binary message function supports **broadcasting**, a mechanism for extending element-wise
|
||||
operations to tensor inputs with different shapes. DGL generally follows the standard
|
||||
broadcasting semantic by `NumPy <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>`_
|
||||
and `PyTorch <https://pytorch.org/docs/stable/notes/broadcasting.html>`_. Below are some
|
||||
examples:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import dgl.function as fn
|
||||
import torch as th
|
||||
g = ... # create a DGLGraph
|
||||
|
||||
# case 1
|
||||
g.ndata['h'] = th.randn((g.num_nodes(), 10))
|
||||
g.edata['w'] = th.randn((g.num_edges(), 1))
|
||||
# OK, valid broadcasting between feature shapes (10,) and (1,)
|
||||
g.update_all(fn.u_mul_e('h', 'w', 'm'), fn.sum('m', 'h_new'))
|
||||
g.ndata['h_new'] # shape: (g.num_nodes(), 10)
|
||||
|
||||
# case 2
|
||||
g.ndata['h'] = th.randn((g.num_nodes(), 5, 10))
|
||||
g.edata['w'] = th.randn((g.num_edges(), 10))
|
||||
# OK, valid broadcasting between feature shapes (5, 10) and (10,)
|
||||
g.update_all(fn.u_mul_e('h', 'w', 'm'), fn.sum('m', 'h_new'))
|
||||
g.ndata['h_new'] # shape: (g.num_nodes(), 5, 10)
|
||||
|
||||
# case 3
|
||||
g.ndata['h'] = th.randn((g.num_nodes(), 5, 10))
|
||||
g.edata['w'] = th.randn((g.num_edges(), 5))
|
||||
# NOT OK, invalid broadcasting between feature shapes (5, 10) and (5,)
|
||||
# shapes are aligned from right
|
||||
g.update_all(fn.u_mul_e('h', 'w', 'm'), fn.sum('m', 'h_new'))
|
||||
|
||||
# case 3
|
||||
g.ndata['h1'] = th.randn((g.num_nodes(), 1, 10))
|
||||
g.ndata['h2'] = th.randn((g.num_nodes(), 5, 1))
|
||||
# OK, valid broadcasting between feature shapes (1, 10) and (5, 1)
|
||||
g.apply_edges(fn.u_add_v('h1', 'h2', 'x')) # apply_edges also supports broadcasting
|
||||
g.edata['x'] # shape: (g.num_edges(), 5, 10)
|
||||
|
||||
# case 4
|
||||
g.ndata['h1'] = th.randn((g.num_nodes(), 1, 10, 128))
|
||||
g.ndata['h2'] = th.randn((g.num_nodes(), 5, 1, 128))
|
||||
# OK, u_dot_v supports broadcasting but requires the last dimension to match
|
||||
g.apply_edges(fn.u_dot_v('h1', 'h2', 'x'))
|
||||
g.edata['x'] # shape: (g.num_edges(), 5, 10, 1)
|
||||
|
||||
|
||||
.. _api-built-in:
|
||||
|
||||
DGL Built-in Function
|
||||
-------------------------
|
||||
|
||||
Here is a cheatsheet of all the DGL built-in functions.
|
||||
|
||||
+-------------------------+-----------------------------------------------------------------+-----------------------+
|
||||
| Category | Functions | Memo |
|
||||
+=========================+=================================================================+=======================+
|
||||
| Unary message function | ``copy_u`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``copy_e`` | |
|
||||
+-------------------------+-----------------------------------------------------------------+-----------------------+
|
||||
| Binary message function | ``u_add_v``, ``u_sub_v``, ``u_mul_v``, ``u_div_v``, ``u_dot_v`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``u_add_e``, ``u_sub_e``, ``u_mul_e``, ``u_div_e``, ``u_dot_e`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``v_add_u``, ``v_sub_u``, ``v_mul_u``, ``v_div_u``, ``v_dot_u`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``v_add_e``, ``v_sub_e``, ``v_mul_e``, ``v_div_e``, ``v_dot_e`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``e_add_u``, ``e_sub_u``, ``e_mul_u``, ``e_div_u``, ``e_dot_u`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``e_add_v``, ``e_sub_v``, ``e_mul_v``, ``e_div_v``, ``e_dot_v`` | |
|
||||
+-------------------------+-----------------------------------------------------------------+-----------------------+
|
||||
| Reduce function | ``max`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``min`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``sum`` | |
|
||||
| +-----------------------------------------------------------------+-----------------------+
|
||||
| | ``mean`` | |
|
||||
+-------------------------+-----------------------------------------------------------------+-----------------------+
|
||||
|
||||
Message functions
|
||||
-----------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
copy_u
|
||||
copy_e
|
||||
u_add_v
|
||||
u_sub_v
|
||||
u_mul_v
|
||||
u_div_v
|
||||
u_add_e
|
||||
u_sub_e
|
||||
u_mul_e
|
||||
u_div_e
|
||||
v_add_u
|
||||
v_sub_u
|
||||
v_mul_u
|
||||
v_div_u
|
||||
v_add_e
|
||||
v_sub_e
|
||||
v_mul_e
|
||||
v_div_e
|
||||
e_add_u
|
||||
e_sub_u
|
||||
e_mul_u
|
||||
e_div_u
|
||||
e_add_v
|
||||
e_sub_v
|
||||
e_mul_v
|
||||
e_div_v
|
||||
u_dot_v
|
||||
u_dot_e
|
||||
v_dot_e
|
||||
v_dot_u
|
||||
e_dot_u
|
||||
e_dot_v
|
||||
|
||||
Reduce functions
|
||||
----------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
sum
|
||||
max
|
||||
min
|
||||
mean
|
||||
@@ -0,0 +1,26 @@
|
||||
.. _api-geometry:
|
||||
|
||||
dgl.geometry
|
||||
=================================
|
||||
|
||||
.. automodule:: dgl.geometry
|
||||
|
||||
.. _api-geometry-farthest-point-sampler:
|
||||
|
||||
Farthest Point Sampler
|
||||
-----------
|
||||
|
||||
Farthest point sampling is a greedy algorithm that samples from a point cloud
|
||||
data iteratively. It starts from a random single sample of point. In each iteration,
|
||||
it samples from the rest points that is the farthest from the set of sampled points.
|
||||
|
||||
.. autoclass:: farthest_point_sampler
|
||||
|
||||
.. _api-geometry-neighbor-matching:
|
||||
|
||||
Neighbor Matching
|
||||
-----------------------------
|
||||
|
||||
Neighbor matching is an important module in the Graclus clustering algorithm.
|
||||
|
||||
.. autoclass:: neighbor_matching
|
||||
@@ -0,0 +1,204 @@
|
||||
.. _apibackend:
|
||||
|
||||
🆕 dgl.graphbolt
|
||||
=================================
|
||||
|
||||
.. currentmodule:: dgl.graphbolt
|
||||
|
||||
**dgl.graphbolt** is a dataloading framework for GNNs that provides well-defined
|
||||
APIs for each stage of the data pipeline and multiple standard implementations.
|
||||
|
||||
Dataset
|
||||
-------
|
||||
|
||||
A dataset is a collection of graph structure data, feature data and tasks.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
Dataset
|
||||
OnDiskDataset
|
||||
BuiltinDataset
|
||||
LegacyDataset
|
||||
Task
|
||||
|
||||
Graph
|
||||
-----
|
||||
|
||||
A graph is a collection of nodes and edges. It can be a homogeneous graph or a
|
||||
heterogeneous graph.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
SamplingGraph
|
||||
FusedCSCSamplingGraph
|
||||
|
||||
|
||||
Feature and FeatureStore
|
||||
------------------------
|
||||
|
||||
A feature is a collection of data(tensor, array). A feature store is a
|
||||
collection of features.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
Feature
|
||||
FeatureStore
|
||||
BasicFeatureStore
|
||||
TorchBasedFeature
|
||||
TorchBasedFeatureStore
|
||||
DiskBasedFeature
|
||||
CPUCachedFeature
|
||||
GPUCachedFeature
|
||||
|
||||
|
||||
DataLoader
|
||||
----------
|
||||
|
||||
A dataloader is for iterating over a dataset and generate mini-batches.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
DataLoader
|
||||
|
||||
|
||||
ItemSet
|
||||
-------
|
||||
|
||||
An item set is an iterable collection of items.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
ItemSet
|
||||
HeteroItemSet
|
||||
|
||||
|
||||
ItemSampler
|
||||
-----------
|
||||
|
||||
An item sampler is for sampling items from an item set.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
ItemSampler
|
||||
DistributedItemSampler
|
||||
|
||||
|
||||
MiniBatch
|
||||
---------
|
||||
|
||||
A mini-batch is a collection of sampled subgraphs and their corresponding
|
||||
features. It is the basic unit for training a GNN model.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
MiniBatch
|
||||
MiniBatchTransformer
|
||||
|
||||
|
||||
NegativeSampler
|
||||
---------------
|
||||
|
||||
A negative sampler is for sampling negative items from mini-batches.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
NegativeSampler
|
||||
UniformNegativeSampler
|
||||
|
||||
|
||||
SubgraphSampler
|
||||
---------------
|
||||
|
||||
A subgraph sampler is for sampling subgraphs from a graph.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
SubgraphSampler
|
||||
SampledSubgraph
|
||||
NeighborSampler
|
||||
LayerNeighborSampler
|
||||
TemporalNeighborSampler
|
||||
TemporalLayerNeighborSampler
|
||||
SampledSubgraphImpl
|
||||
FusedSampledSubgraphImpl
|
||||
InSubgraphSampler
|
||||
|
||||
|
||||
FeatureFetcher
|
||||
--------------
|
||||
|
||||
A feature fetcher is for fetching features from a feature store.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
FeatureFetcher
|
||||
|
||||
|
||||
CopyTo
|
||||
------
|
||||
|
||||
This datapipe is for copying data to a device.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: graphbolt_classtemplate.rst
|
||||
|
||||
CopyTo
|
||||
|
||||
|
||||
Utilities
|
||||
---------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
|
||||
cpu_cached_feature
|
||||
gpu_cached_feature
|
||||
fused_csc_sampling_graph
|
||||
load_from_shared_memory
|
||||
from_dglgraph
|
||||
etype_str_to_tuple
|
||||
etype_tuple_to_str
|
||||
isin
|
||||
seed
|
||||
index_select
|
||||
expand_indptr
|
||||
indptr_edge_ids
|
||||
add_reverse_edges
|
||||
exclude_seed_edges
|
||||
compact_csc_format
|
||||
unique_and_compact
|
||||
unique_and_compact_csc_formats
|
||||
numpy_save_aligned
|
||||
@@ -0,0 +1,21 @@
|
||||
.. _apimultiprocessing:
|
||||
|
||||
dgl.multiprocessing
|
||||
===================
|
||||
|
||||
This is a minimal wrapper of Python's native :mod:`multiprocessing` module.
|
||||
It modifies the :class:`multiprocessing.Process` class to make forking
|
||||
work with OpenMP in the DGL core library.
|
||||
|
||||
The API usage is exactly the same as the native module, so DGL does not provide
|
||||
additional documentation.
|
||||
|
||||
In addition, if your backend is PyTorch, this module will also be compatible with
|
||||
:mod:`torch.multiprocessing` module.
|
||||
|
||||
.. currentmodule:: dgl.multiprocessing.pytorch
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
call_once_and_share
|
||||
shared_tensor
|
||||
@@ -0,0 +1,316 @@
|
||||
.. _apibackend:
|
||||
|
||||
.. currentmodule:: dgl.ops
|
||||
|
||||
dgl.ops
|
||||
==================================
|
||||
|
||||
Frame-agnostic operators for message passing on graphs.
|
||||
|
||||
GSpMM functions
|
||||
---------------
|
||||
|
||||
Generalized Sparse-Matrix Dense-Matrix Multiplication functions.
|
||||
It *fuses* two steps into one kernel.
|
||||
|
||||
1. Computes messages by add/sub/mul/div source node and edge features,
|
||||
or copy node features to edges.
|
||||
2. Aggregate the messages by sum/max/min/mean as the features on destination nodes.
|
||||
|
||||
Our implementation supports tensors on CPU/GPU in PyTorch/MXNet/Tensorflow
|
||||
as input. All operators are equipped with autograd (computing the input gradients
|
||||
given output gradient) and broadcasting (if the feature shape of operands do not
|
||||
match, we first broadcast them to the same shape, then applies the binary
|
||||
operators). Our broadcast semantics follows NumPy, please see
|
||||
https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html
|
||||
for more details.
|
||||
|
||||
What do we mean by *fuses* is that the messages are not materialized on edges,
|
||||
instead we compute the result on destination nodes directly, thus saving memory
|
||||
cost. The space complexity of GSpMM operators is :math:`O(|N|D)` where :math:`|N|`
|
||||
refers to the number of nodes in the graph, and :math:`D` refers to the feature
|
||||
size (:math:`D=\prod_{i=1}^{N}D_i` if your feature is a multi-dimensional tensor).
|
||||
|
||||
The following is an example showing how GSpMM works (we use PyTorch as the backend
|
||||
here, you can enjoy the same convenience on other frameworks by similar usage):
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> import dgl.ops as F
|
||||
>>> g = dgl.graph(([0, 0, 0, 1, 1, 2], [0, 1, 2, 1, 2, 2])) # 3 nodes, 6 edges
|
||||
>>> x = th.ones(3, 2, requires_grad=True)
|
||||
>>> x
|
||||
tensor([[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.]], requires_grad=True)
|
||||
>>> y = th.arange(1, 13).float().view(6, 2).requires_grad_()
|
||||
tensor([[ 1., 2.],
|
||||
[ 3., 4.],
|
||||
[ 5., 6.],
|
||||
[ 7., 8.],
|
||||
[ 9., 10.],
|
||||
[11., 12.]], requires_grad=True)
|
||||
>>> out_1 = F.u_mul_e_sum(g, x, y)
|
||||
>>> out_1 # (10, 12) = ((1, 1) * (3, 4)) + ((1, 1) * (7, 8))
|
||||
tensor([[ 1., 2.],
|
||||
[10., 12.],
|
||||
[25., 28.]], grad_fn=<GSpMMBackward>)
|
||||
>>> out_1.sum().backward()
|
||||
>>> x.grad
|
||||
tensor([[12., 15.],
|
||||
[18., 20.],
|
||||
[12., 13.]])
|
||||
>>> y.grad
|
||||
tensor([[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.]])
|
||||
>>> out_2 = F.copy_u_sum(g, x)
|
||||
>>> out_2
|
||||
tensor([[1., 1.],
|
||||
[2., 2.],
|
||||
[3., 3.]], grad_fn=<GSpMMBackward>)
|
||||
>>> out_3 = F.u_add_e_max(g, x, y)
|
||||
>>> out_3
|
||||
tensor([[ 2., 3.],
|
||||
[ 8., 9.],
|
||||
[12., 13.]], grad_fn=<GSpMMBackward>)
|
||||
>>> y1 = th.rand(6, 4, 2, requires_grad=True) # test broadcast
|
||||
>>> F.u_mul_e_sum(g, x, y1).shape # (2,), (4, 2) -> (4, 2)
|
||||
torch.Size([3, 4, 2])
|
||||
|
||||
For all operators, the input graph could either be a homogeneous or a bipartite
|
||||
graph.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
gspmm
|
||||
u_add_e_sum
|
||||
u_sub_e_sum
|
||||
u_mul_e_sum
|
||||
u_div_e_sum
|
||||
u_add_e_max
|
||||
u_sub_e_max
|
||||
u_mul_e_max
|
||||
u_div_e_max
|
||||
u_add_e_min
|
||||
u_sub_e_min
|
||||
u_mul_e_min
|
||||
u_div_e_min
|
||||
u_add_e_mean
|
||||
u_sub_e_mean
|
||||
u_mul_e_mean
|
||||
u_div_e_mean
|
||||
copy_u_sum
|
||||
copy_e_sum
|
||||
copy_u_max
|
||||
copy_e_max
|
||||
copy_u_min
|
||||
copy_e_min
|
||||
copy_u_mean
|
||||
copy_e_mean
|
||||
|
||||
GSDDMM functions
|
||||
----------------
|
||||
|
||||
Generalized Sampled Dense-Dense Matrix Multiplication.
|
||||
It computes edge features by add/sub/mul/div/dot features on source/destination
|
||||
nodes or edges.
|
||||
|
||||
Like GSpMM, our implementation supports tensors on CPU/GPU in
|
||||
PyTorch/MXNet/Tensorflow as input. All operators are equipped with autograd and
|
||||
broadcasting.
|
||||
|
||||
The memory cost of GSDDMM is :math:`O(|E|D)` where :math:`|E|` refers to the number
|
||||
of edges in the graph while :math:`D` refers to the feature size.
|
||||
|
||||
Note that we support ``dot`` operator, which semantically is the same as reduce
|
||||
the last dimension by sum to the result of ``mul`` operator. However, the ``dot``
|
||||
is more memory efficient because it *fuses* ``mul`` and sum reduction, which is
|
||||
critical in the cases while the feature size on last dimension is non-trivial
|
||||
(e.g. multi-head attention in Transformer-like models).
|
||||
|
||||
The following is an example showing how GSDDMM works:
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> import dgl.ops as F
|
||||
>>> g = dgl.graph(([0, 0, 0, 1, 1, 2], [0, 1, 2, 1, 2, 2])) # 3 nodes, 6 edges
|
||||
>>> x = th.ones(3, 2, requires_grad=True)
|
||||
>>> x
|
||||
tensor([[1., 1.],
|
||||
[1., 1.],
|
||||
[1., 1.]], requires_grad=True)
|
||||
>>> y = th.arange(1, 7).float().view(3, 2).requires_grad_()
|
||||
>>> y
|
||||
tensor([[1., 2.],
|
||||
[3., 4.],
|
||||
[5., 6.]], requires_grad=True)
|
||||
>>> e = th.ones(6, 1, 2, requires_grad=True) * 2
|
||||
tensor([[[2., 2.]],
|
||||
[[2., 2.]],
|
||||
[[2., 2.]],
|
||||
[[2., 2.]],
|
||||
[[2., 2.]],
|
||||
[[2., 2.]]], grad_fn=<MulBackward0>)
|
||||
>>> out1 = F.u_div_v(g, x, y)
|
||||
tensor([[1.0000, 0.5000],
|
||||
[0.3333, 0.2500],
|
||||
[0.2000, 0.1667],
|
||||
[0.3333, 0.2500],
|
||||
[0.2000, 0.1667],
|
||||
[0.2000, 0.1667]], grad_fn=<GSDDMMBackward>)
|
||||
>>> out1.sum().backward()
|
||||
>>> x.grad
|
||||
tensor([[1.5333, 0.9167],
|
||||
[0.5333, 0.4167],
|
||||
[0.2000, 0.1667]])
|
||||
>>> y.grad
|
||||
tensor([[-1.0000, -0.2500],
|
||||
[-0.2222, -0.1250],
|
||||
[-0.1200, -0.0833]])
|
||||
>>> out2 = F.e_sub_v(g, e, y)
|
||||
>>> out2
|
||||
tensor([[[ 1., 0.]],
|
||||
[[-1., -2.]],
|
||||
[[-3., -4.]],
|
||||
[[-1., -2.]],
|
||||
[[-3., -4.]],
|
||||
[[-3., -4.]]], grad_fn=<GSDDMMBackward>)
|
||||
>>> out3 = F.copy_v(g, y)
|
||||
>>> out3
|
||||
tensor([[1., 2.],
|
||||
[3., 4.],
|
||||
[5., 6.],
|
||||
[3., 4.],
|
||||
[5., 6.],
|
||||
[5., 6.]], grad_fn=<GSDDMMBackward>)
|
||||
>>> out4 = F.u_dot_v(g, x, y)
|
||||
>>> out4 # the last dimension was reduced to size 1.
|
||||
tensor([[ 3.],
|
||||
[ 7.],
|
||||
[11.],
|
||||
[ 7.],
|
||||
[11.],
|
||||
[11.]], grad_fn=<GSDDMMBackward>)
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
gsddmm
|
||||
u_add_v
|
||||
u_sub_v
|
||||
u_mul_v
|
||||
u_dot_v
|
||||
u_div_v
|
||||
u_add_e
|
||||
u_sub_e
|
||||
u_mul_e
|
||||
u_dot_e
|
||||
u_div_e
|
||||
e_add_v
|
||||
e_sub_v
|
||||
e_mul_v
|
||||
e_dot_v
|
||||
e_div_v
|
||||
v_add_u
|
||||
v_sub_u
|
||||
v_mul_u
|
||||
v_dot_u
|
||||
v_div_u
|
||||
e_add_u
|
||||
e_sub_u
|
||||
e_mul_u
|
||||
e_dot_u
|
||||
e_div_u
|
||||
v_add_e
|
||||
v_sub_e
|
||||
v_mul_e
|
||||
v_dot_e
|
||||
v_div_e
|
||||
copy_u
|
||||
copy_v
|
||||
|
||||
Like GSpMM, GSDDMM operators support both homogeneous and bipartite graph.
|
||||
|
||||
Segment Reduce Module
|
||||
---------------------
|
||||
|
||||
DGL provide operators to reduce value tensor along the first dimension by segments.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
segment_reduce
|
||||
|
||||
GatherMM and SegmentMM Module
|
||||
-----------------------------
|
||||
|
||||
SegmentMM: DGL provide operators to perform matrix multiplication according to segments.
|
||||
|
||||
GatherMM: DGL provide operators to gather data according to the given indices and perform matrix multiplication.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
gather_mm
|
||||
segment_mm
|
||||
|
||||
Supported Data types
|
||||
--------------------
|
||||
Operators defined in ``dgl.ops`` support floating point data types, i.e. the operands
|
||||
must be ``half`` (``float16``) /``float``/``double`` tensors.
|
||||
The input tensors must have the same data type (if one input tensor has type float16
|
||||
and the other input tensor has data type float32, user must convert one of them to
|
||||
align with the other one).
|
||||
|
||||
``float16`` data type support is disabled by default as it has a minimum GPU
|
||||
compute capacity requirement of ``sm_53`` (Pascal, Volta, Turing and Ampere
|
||||
architectures).
|
||||
|
||||
User can enable float16 for mixed precision training by compiling DGL from source
|
||||
(see :doc:`Mixed Precision Training </guide/mixed_precision>` tutorial for details).
|
||||
|
||||
Relation with Message Passing APIs
|
||||
----------------------------------
|
||||
|
||||
``dgl.update_all`` and ``dgl.apply_edges`` calls with built-in message/reduce functions
|
||||
would be dispatched into function calls of operators defined in ``dgl.ops``:
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> import dgl.ops as F
|
||||
>>> import dgl.function as fn
|
||||
>>> g = dgl.rand_graph(100, 1000) # create a DGLGraph with 100 nodes and 1000 edges.
|
||||
>>> x = th.rand(100, 20) # node features.
|
||||
>>> e = th.rand(1000, 20)
|
||||
>>>
|
||||
>>> # dgl.update_all + builtin functions
|
||||
>>> g.srcdata['x'] = x # srcdata is the same as ndata for graphs with one node type.
|
||||
>>> g.edata['e'] = e
|
||||
>>> g.update_all(fn.u_mul_e('x', 'e', 'm'), fn.sum('m', 'y'))
|
||||
>>> y = g.dstdata['y'] # dstdata is the same as ndata for graphs with one node type.
|
||||
>>>
|
||||
>>> # use GSpMM operators defined in dgl.ops directly
|
||||
>>> y = F.u_mul_e_sum(g, x, e)
|
||||
>>>
|
||||
>>> # dgl.apply_edges + builtin functions
|
||||
>>> g.srcdata['x'] = x
|
||||
>>> g.dstdata['y'] = y
|
||||
>>> g.apply_edges(fn.u_dot_v('x', 'y', 'z'))
|
||||
>>> z = g.edata['z']
|
||||
>>>
|
||||
>>> # use GSDDMM operators defined in dgl.ops directly
|
||||
>>> z = F.u_dot_v(g, x, y)
|
||||
|
||||
It up to user to decide whether to use message-passing APIs or GSpMM/GSDDMM operators, and both
|
||||
of them have the same efficiency. Programs written in message-passing APIs look more like DGL-style
|
||||
but in some cases calling GSpMM/GSDDMM operators is more concise.
|
||||
|
||||
Note that on PyTorch all operators defined in ``dgl.ops`` support higher-order gradients, so as
|
||||
message passing APIs because they entirely depend on these operators.
|
||||
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
.. _apioptim:
|
||||
|
||||
dgl.optim
|
||||
=========
|
||||
|
||||
.. automodule:: dgl.optim
|
||||
|
||||
Node embedding optimizer
|
||||
-------------------------
|
||||
.. currentmodule:: dgl.optim.pytorch
|
||||
|
||||
.. autoclass:: SparseAdagrad
|
||||
.. autoclass:: SparseAdam
|
||||
@@ -0,0 +1,245 @@
|
||||
.. _apidgl:
|
||||
|
||||
dgl
|
||||
=============================
|
||||
|
||||
.. currentmodule:: dgl
|
||||
.. automodule:: dgl
|
||||
|
||||
.. _api-graph-create-ops:
|
||||
|
||||
Graph Create Ops
|
||||
-------------------------
|
||||
|
||||
Operators for constructing :class:`DGLGraph` from raw data formats.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
graph
|
||||
heterograph
|
||||
from_cugraph
|
||||
from_scipy
|
||||
from_networkx
|
||||
bipartite_from_scipy
|
||||
bipartite_from_networkx
|
||||
rand_graph
|
||||
rand_bipartite
|
||||
knn_graph
|
||||
segmented_knn_graph
|
||||
radius_graph
|
||||
create_block
|
||||
block_to_graph
|
||||
merge
|
||||
|
||||
.. _api-subgraph-extraction:
|
||||
|
||||
Subgraph Extraction Ops
|
||||
-------------------------------------
|
||||
|
||||
Operators for extracting and returning subgraphs.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
node_subgraph
|
||||
edge_subgraph
|
||||
node_type_subgraph
|
||||
edge_type_subgraph
|
||||
in_subgraph
|
||||
out_subgraph
|
||||
khop_in_subgraph
|
||||
khop_out_subgraph
|
||||
|
||||
.. _api-transform:
|
||||
|
||||
Graph Transform Ops
|
||||
----------------------------------
|
||||
|
||||
Operators for generating new graphs by manipulating the structure of the existing ones.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
add_edges
|
||||
add_nodes
|
||||
add_reverse_edges
|
||||
add_self_loop
|
||||
adj_product_graph
|
||||
adj_sum_graph
|
||||
compact_graphs
|
||||
khop_adj
|
||||
khop_graph
|
||||
knn_graph
|
||||
laplacian_lambda_max
|
||||
line_graph
|
||||
metapath_reachable_graph
|
||||
metis_partition
|
||||
metis_partition_assignment
|
||||
norm_by_dst
|
||||
partition_graph_with_halo
|
||||
radius_graph
|
||||
remove_edges
|
||||
remove_nodes
|
||||
remove_self_loop
|
||||
reorder_graph
|
||||
reverse
|
||||
segmented_knn_graph
|
||||
sort_csr_by_tag
|
||||
sort_csc_by_tag
|
||||
to_bidirected
|
||||
to_bidirected_stale
|
||||
to_block
|
||||
to_cugraph
|
||||
to_double
|
||||
to_float
|
||||
to_half
|
||||
to_heterogeneous
|
||||
to_homogeneous
|
||||
to_networkx
|
||||
to_simple
|
||||
to_simple_graph
|
||||
|
||||
.. _api-positional-encoding:
|
||||
|
||||
Graph Positional Encoding Ops:
|
||||
-----------------------------------------
|
||||
|
||||
Operators for generating positional encodings of each node.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated
|
||||
|
||||
random_walk_pe
|
||||
lap_pe
|
||||
double_radius_node_labeling
|
||||
shortest_dist
|
||||
svd_pe
|
||||
|
||||
.. _api-partition:
|
||||
|
||||
Graph Partition Utilities
|
||||
-------------------------
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
metis_partition
|
||||
metis_partition_assignment
|
||||
partition_graph_with_halo
|
||||
|
||||
.. _api-batch:
|
||||
|
||||
Batching and Reading Out Ops
|
||||
-------------------------------
|
||||
|
||||
Operators for batching multiple graphs into one for batch processing and
|
||||
operators for computing graph-level representation for both single and batched graphs.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
batch
|
||||
unbatch
|
||||
slice_batch
|
||||
readout_nodes
|
||||
readout_edges
|
||||
sum_nodes
|
||||
sum_edges
|
||||
mean_nodes
|
||||
mean_edges
|
||||
max_nodes
|
||||
max_edges
|
||||
softmax_nodes
|
||||
softmax_edges
|
||||
broadcast_nodes
|
||||
broadcast_edges
|
||||
topk_nodes
|
||||
topk_edges
|
||||
|
||||
Adjacency Related Utilities
|
||||
-------------------------------
|
||||
|
||||
Utilities for computing adjacency matrix and Lapacian matrix.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
khop_adj
|
||||
laplacian_lambda_max
|
||||
|
||||
Graph Traversal & Message Propagation
|
||||
------------------------------------------
|
||||
|
||||
DGL implements graph traversal algorithms implemented as python generators,
|
||||
which returns the visited set of nodes or edges (in ID tensor) at each iteration.
|
||||
The naming convention is ``<algorithm>_[nodes|edges]_generator``.
|
||||
An example usage is as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g = ... # some DGLGraph
|
||||
for nodes in dgl.bfs_nodes_generator(g, 0):
|
||||
do_something(nodes)
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
bfs_nodes_generator
|
||||
bfs_edges_generator
|
||||
topological_nodes_generator
|
||||
dfs_edges_generator
|
||||
dfs_labeled_edges_generator
|
||||
|
||||
DGL provides APIs to perform message passing following graph traversal order. ``prop_nodes_XXX``
|
||||
calls traversal algorithm ``XXX`` and triggers :func:`~DGLGraph.pull()` on the visited node
|
||||
set at each iteration. ``prop_edges_YYY`` applies traversal algorithm ``YYY`` and triggers
|
||||
:func:`~DGLGraph.send_and_recv()` on the visited edge set at each iteration.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
prop_nodes
|
||||
prop_nodes_bfs
|
||||
prop_nodes_topo
|
||||
prop_edges
|
||||
prop_edges_dfs
|
||||
|
||||
Homophily Measures
|
||||
-------------------------
|
||||
|
||||
Utilities for measuring homophily of a graph
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
edge_homophily
|
||||
node_homophily
|
||||
linkx_homophily
|
||||
adjusted_homophily
|
||||
|
||||
Label Informativeness Measures
|
||||
-------------------------
|
||||
|
||||
Utilities for measuring label informativeness of a graph
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
edge_label_informativeness
|
||||
node_label_informativeness
|
||||
|
||||
Utilities
|
||||
-----------------------------------------------
|
||||
|
||||
Other utilities for controlling randomness, saving and loading graphs, setting and getting runtime configurations, functions that applies
|
||||
the same function to every elements in a container, etc.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
seed
|
||||
save_graphs
|
||||
load_graphs
|
||||
apply_each
|
||||
use_libxsmm
|
||||
is_libxsmm_enabled
|
||||
@@ -0,0 +1,36 @@
|
||||
.. _api-sampling:
|
||||
|
||||
dgl.sampling
|
||||
=================================
|
||||
|
||||
.. automodule:: dgl.sampling
|
||||
|
||||
Random walk
|
||||
------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
random_walk
|
||||
node2vec_random_walk
|
||||
pack_traces
|
||||
|
||||
Neighbor sampling
|
||||
---------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
sample_neighbors
|
||||
sample_labors
|
||||
sample_neighbors_biased
|
||||
select_topk
|
||||
PinSAGESampler
|
||||
|
||||
Negative sampling
|
||||
-----------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
global_uniform_negative_sampling
|
||||
@@ -0,0 +1,124 @@
|
||||
.. _apibackend:
|
||||
|
||||
dgl.sparse
|
||||
=================================
|
||||
|
||||
`dgl.sparse` is a library for sparse operators that are commonly used in GNN models.
|
||||
|
||||
Sparse matrix class
|
||||
-------------------------
|
||||
.. currentmodule:: dgl.sparse
|
||||
|
||||
.. class:: SparseMatrix
|
||||
|
||||
A SparseMatrix can be created from Coordinate format indices using the
|
||||
:func:`spmatrix` constructor:
|
||||
|
||||
>>> indices = torch.tensor([[1, 1, 2],
|
||||
>>> [2, 4, 3]])
|
||||
>>> A = dglsp.spmatrix(indices)
|
||||
SparseMatrix(indices=tensor([[1, 1, 2],
|
||||
[2, 4, 3]]),
|
||||
values=tensor([1., 1., 1.]),
|
||||
shape=(3, 5), nnz=3)
|
||||
|
||||
Creation Ops
|
||||
````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
spmatrix
|
||||
val_like
|
||||
from_coo
|
||||
from_csr
|
||||
from_csc
|
||||
diag
|
||||
identity
|
||||
|
||||
Attributes and methods
|
||||
``````````````````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
SparseMatrix.shape
|
||||
SparseMatrix.nnz
|
||||
SparseMatrix.dtype
|
||||
SparseMatrix.device
|
||||
SparseMatrix.val
|
||||
SparseMatrix.row
|
||||
SparseMatrix.col
|
||||
SparseMatrix.indices
|
||||
SparseMatrix.coo
|
||||
SparseMatrix.csr
|
||||
SparseMatrix.csc
|
||||
SparseMatrix.coalesce
|
||||
SparseMatrix.has_duplicate
|
||||
SparseMatrix.to_dense
|
||||
SparseMatrix.to
|
||||
SparseMatrix.cuda
|
||||
SparseMatrix.cpu
|
||||
SparseMatrix.float
|
||||
SparseMatrix.double
|
||||
SparseMatrix.int
|
||||
SparseMatrix.long
|
||||
SparseMatrix.transpose
|
||||
SparseMatrix.t
|
||||
SparseMatrix.T
|
||||
SparseMatrix.neg
|
||||
SparseMatrix.reduce
|
||||
SparseMatrix.sum
|
||||
SparseMatrix.smax
|
||||
SparseMatrix.smin
|
||||
SparseMatrix.smean
|
||||
SparseMatrix.softmax
|
||||
|
||||
Operators
|
||||
---------
|
||||
.. currentmodule:: dgl.sparse
|
||||
|
||||
Elementwise Operators
|
||||
````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
add
|
||||
sub
|
||||
mul
|
||||
div
|
||||
power
|
||||
|
||||
Matrix Multiplication
|
||||
````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
matmul
|
||||
spmm
|
||||
bspmm
|
||||
spspmm
|
||||
sddmm
|
||||
bsddmm
|
||||
|
||||
Non-linear activation functions
|
||||
````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
softmax
|
||||
|
||||
Broadcast operators
|
||||
````````
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
sp_broadcast_v
|
||||
sp_add_v
|
||||
sp_sub_v
|
||||
sp_mul_v
|
||||
sp_div_v
|
||||
@@ -0,0 +1,19 @@
|
||||
API Reference
|
||||
=============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
dgl
|
||||
dgl.data
|
||||
dgl.dataloading
|
||||
dgl.DGLGraph
|
||||
dgl.distributed
|
||||
dgl.function
|
||||
nn-pytorch
|
||||
nn-tensorflow
|
||||
nn-mxnet
|
||||
dgl.ops
|
||||
dgl.sampling
|
||||
udf
|
||||
transforms
|
||||
@@ -0,0 +1,128 @@
|
||||
.. _knn_benchmark:
|
||||
|
||||
Benchmark the performance of KNN algorithms
|
||||
===========================================
|
||||
|
||||
In this doc, we benchmark the performance on multiple K-Nearest Neighbor algorithms implemented by :func:`dgl.knn_graph`.
|
||||
|
||||
Given a dataset of ``N`` samples with ``D`` dimensions, the common use case of KNN algorithms in graph learning is to build a KNN graph by finding the ``K`` nearest neighbors for each of the ``N`` samples among the dataset.
|
||||
|
||||
Empirically, the three parameters, ``N``, ``D``, and ``K``, all have impact on the computation cost. To benchmark the algorithms, we pick a few represensitive datasets to cover most common scenarios:
|
||||
|
||||
* A synthetic dataset with mixed gaussian samples: ``N = 1000``, ``D = 3``.
|
||||
* A point cloud sample from ModelNet: ``N = 10000``, ``D = 3``.
|
||||
* Subsets of MNIST
|
||||
- A small subset: ``N = 1000``, ``D = 784``
|
||||
- A medium subset: ``N = 10000``, ``D = 784``
|
||||
- A large subset: ``N = 50000``, ``D = 784``
|
||||
|
||||
Some notes:
|
||||
|
||||
* ``bruteforce-sharemem`` is an optimized implementation of ``bruteforce`` on GPU.
|
||||
* ``kd-tree`` is currently only implemented on CPU.
|
||||
* ``bruteforce-blas`` conducts matrix multiplication, thus is memory inefficient.
|
||||
* ``nn-descent`` is an approximate algorithm, and we also report the recall rate of its result.
|
||||
|
||||
Results
|
||||
-------
|
||||
|
||||
In this section, we show the runtime and recall rate (where applicable) for the algorithms under various scenarios.
|
||||
|
||||
The experiments are run on an Amazon EC2 P3.2xlarge instance. This instance has 8 vCPUs with 61GB RAM, and one Tesla V100 GPU with 16GB RAM. In terms of the environment, we obtain the numbers with DGL==0.7.0(`64d0f3f <https://github.com/dmlc/dgl/commit/64d0f3f3554911ec06d015f1c9659180796adf9a>`_), PyTorch==1.8.1, CUDA==11.1 on Ubuntu 18.04.5 LTS.
|
||||
|
||||
* **Mixed Gaussian:**
|
||||
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| Model | CPU | GPU |
|
||||
| +------------------+-------------------+------------------+------------------+
|
||||
| | K = 8 | K = 64 | K = 8 | K = 64 |
|
||||
+=====================+==================+===================+==================+==================+
|
||||
| bruteforce-blas | 0.010 | 0.011 | 0.002 | 0.003 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| kd-tree | 0.004 | 0.006 | n/a | n/a |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce | 0.004 | 0.006 | 0.126 | 0.009 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce-sharemem | n/a | n/a | 0.002 | 0.003 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| nn-descent | 0.014 (R: 0.985) | 0.148 (R: 1.000) | 0.016 (R: 0.973) | 0.077 (R: 1.000) |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
|
||||
* **Point Cloud**
|
||||
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| Model | CPU | GPU |
|
||||
| +------------------+-------------------+------------------+------------------+
|
||||
| | K = 8 | K = 64 | K = 8 | K = 64 |
|
||||
+=====================+==================+===================+==================+==================+
|
||||
| bruteforce-blas | 0.359 | 0.432 | 0.010 | 0.010 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| kd-tree | 0.007 | 0.026 | n/a | n/a |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce | 0.074 | 0.167 | 0.008 | 0.039 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce-sharemem | n/a | n/a | 0.004 | 0.017 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| nn-descent | 0.161 (R: 0.977) | 1.345 (R: 0.999) | 0.086 (R: 0.966) | 0.445 (R: 0.999) |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
|
||||
* **Small MNIST**
|
||||
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| Model | CPU | GPU |
|
||||
| +------------------+-------------------+------------------+------------------+
|
||||
| | K = 8 | K = 64 | K = 8 | K = 64 |
|
||||
+=====================+==================+===================+==================+==================+
|
||||
| bruteforce-blas | 0.014 | 0.015 | 0.002 | 0.002 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| kd-tree | 0.179 | 0.182 | n/a | n/a |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce | 0.173 | 0.228 | 0.123 | 0.170 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce-sharemem | n/a | n/a | 0.045 | 0.054 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| nn-descent | 0.060 (R: 0.878) | 1.077 (R: 0.999) | 0.030 (R: 0.952) | 0.457 (R: 0.999) |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
|
||||
* **Medium MNIST**
|
||||
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| Model | CPU | GPU |
|
||||
| +------------------+-------------------+------------------+------------------+
|
||||
| | K = 8 | K = 64 | K = 8 | K = 64 |
|
||||
+=====================+==================+===================+==================+==================+
|
||||
| bruteforce-blas | 0.897 | 0.970 | 0.019 | 0.023 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| kd-tree | 18.902 | 18.928 | n/a | n/a |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce | 14.495 | 17.652 | 2.058 | 2.588 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce-sharemem | n/a | n/a | 2.257 | 2.524 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| nn-descent | 0.804 (R: 0.755) | 14.108 (R: 0.999) | 0.158 (R: 0.900) | 1.794 (R: 0.999) |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
|
||||
* **Large MNIST**
|
||||
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| Model | CPU | GPU |
|
||||
| +------------------+-------------------+------------------+------------------+
|
||||
| | K = 8 | K = 64 | K = 8 | K = 64 |
|
||||
+=====================+==================+===================+==================+==================+
|
||||
| bruteforce-blas | 21.829 | 22.135 | Out of Memory | Out of Memory |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| kd-tree | 542.688 | 573.379 | n/a | n/a |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce | 373.823 | 432.963 | 10.317 | 12.639 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| bruteforce-sharemem | n/a | n/a | 53.133 | 58.419 |
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
| nn-descent | 4.995 (R: 0.658) | 75.487 (R: 0.999) | 1.478 (R: 0.860) | 15.698 (R: 0.999)|
|
||||
+---------------------+------------------+-------------------+------------------+------------------+
|
||||
|
||||
Conclusion
|
||||
----------
|
||||
|
||||
- As long as you have enough memory, ``bruteforce-blas`` is the default algorithm to go with.
|
||||
- Specifically, when ``D`` is small and the data is on CPU, ``kd-tree`` is the best algorithm.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
.. _apinn-mxnet:
|
||||
|
||||
dgl.nn (MXNet)
|
||||
================
|
||||
|
||||
Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.mxnet.conv.GraphConv
|
||||
~dgl.nn.mxnet.conv.RelGraphConv
|
||||
~dgl.nn.mxnet.conv.TAGConv
|
||||
~dgl.nn.mxnet.conv.GATConv
|
||||
~dgl.nn.mxnet.conv.EdgeConv
|
||||
~dgl.nn.mxnet.conv.SAGEConv
|
||||
~dgl.nn.mxnet.conv.SGConv
|
||||
~dgl.nn.mxnet.conv.APPNPConv
|
||||
~dgl.nn.mxnet.conv.GINConv
|
||||
~dgl.nn.mxnet.conv.GatedGraphConv
|
||||
~dgl.nn.mxnet.conv.GMMConv
|
||||
~dgl.nn.mxnet.conv.ChebConv
|
||||
~dgl.nn.mxnet.conv.AGNNConv
|
||||
~dgl.nn.mxnet.conv.NNConv
|
||||
|
||||
Dense Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.mxnet.conv.DenseGraphConv
|
||||
~dgl.nn.mxnet.conv.DenseSAGEConv
|
||||
~dgl.nn.mxnet.conv.DenseChebConv
|
||||
|
||||
Global Pooling Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.mxnet.glob.SumPooling
|
||||
~dgl.nn.mxnet.glob.AvgPooling
|
||||
~dgl.nn.mxnet.glob.MaxPooling
|
||||
~dgl.nn.mxnet.glob.SortPooling
|
||||
~dgl.nn.mxnet.glob.GlobalAttentionPooling
|
||||
~dgl.nn.mxnet.glob.Set2Set
|
||||
|
||||
Heterogeneous Learning Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.mxnet.HeteroGraphConv
|
||||
|
||||
Utility Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.mxnet.utils.Sequential
|
||||
@@ -0,0 +1,162 @@
|
||||
.. _apinn-pytorch:
|
||||
|
||||
dgl.nn (PyTorch)
|
||||
================
|
||||
|
||||
Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.conv.GraphConv
|
||||
~dgl.nn.pytorch.conv.EdgeWeightNorm
|
||||
~dgl.nn.pytorch.conv.RelGraphConv
|
||||
~dgl.nn.pytorch.conv.TAGConv
|
||||
~dgl.nn.pytorch.conv.GATConv
|
||||
~dgl.nn.pytorch.conv.GATv2Conv
|
||||
~dgl.nn.pytorch.conv.EGATConv
|
||||
~dgl.nn.pytorch.conv.EdgeGATConv
|
||||
~dgl.nn.pytorch.conv.EdgeConv
|
||||
~dgl.nn.pytorch.conv.SAGEConv
|
||||
~dgl.nn.pytorch.conv.SGConv
|
||||
~dgl.nn.pytorch.conv.APPNPConv
|
||||
~dgl.nn.pytorch.conv.GINConv
|
||||
~dgl.nn.pytorch.conv.GINEConv
|
||||
~dgl.nn.pytorch.conv.GatedGraphConv
|
||||
~dgl.nn.pytorch.conv.GatedGCNConv
|
||||
~dgl.nn.pytorch.conv.GMMConv
|
||||
~dgl.nn.pytorch.conv.ChebConv
|
||||
~dgl.nn.pytorch.conv.AGNNConv
|
||||
~dgl.nn.pytorch.conv.NNConv
|
||||
~dgl.nn.pytorch.conv.AtomicConv
|
||||
~dgl.nn.pytorch.conv.CFConv
|
||||
~dgl.nn.pytorch.conv.DotGatConv
|
||||
~dgl.nn.pytorch.conv.TWIRLSConv
|
||||
~dgl.nn.pytorch.conv.TWIRLSUnfoldingAndAttention
|
||||
~dgl.nn.pytorch.conv.GCN2Conv
|
||||
~dgl.nn.pytorch.conv.HGTConv
|
||||
~dgl.nn.pytorch.conv.GroupRevRes
|
||||
~dgl.nn.pytorch.conv.EGNNConv
|
||||
~dgl.nn.pytorch.conv.PNAConv
|
||||
~dgl.nn.pytorch.conv.DGNConv
|
||||
|
||||
CuGraph Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.conv.CuGraphRelGraphConv
|
||||
~dgl.nn.pytorch.conv.CuGraphGATConv
|
||||
~dgl.nn.pytorch.conv.CuGraphSAGEConv
|
||||
|
||||
Dense Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.conv.DenseGraphConv
|
||||
~dgl.nn.pytorch.conv.DenseSAGEConv
|
||||
~dgl.nn.pytorch.conv.DenseChebConv
|
||||
|
||||
Global Pooling Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.glob.SumPooling
|
||||
~dgl.nn.pytorch.glob.AvgPooling
|
||||
~dgl.nn.pytorch.glob.MaxPooling
|
||||
~dgl.nn.pytorch.glob.SortPooling
|
||||
~dgl.nn.pytorch.glob.WeightAndSum
|
||||
~dgl.nn.pytorch.glob.GlobalAttentionPooling
|
||||
~dgl.nn.pytorch.glob.Set2Set
|
||||
~dgl.nn.pytorch.glob.SetTransformerEncoder
|
||||
~dgl.nn.pytorch.glob.SetTransformerDecoder
|
||||
|
||||
Score Modules for Link Prediction and Knowledge Graph Completion
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.link.EdgePredictor
|
||||
~dgl.nn.pytorch.link.TransE
|
||||
~dgl.nn.pytorch.link.TransR
|
||||
|
||||
Heterogeneous Learning Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.HeteroGraphConv
|
||||
~dgl.nn.pytorch.HeteroLinear
|
||||
~dgl.nn.pytorch.HeteroEmbedding
|
||||
~dgl.nn.pytorch.TypedLinear
|
||||
|
||||
Utility Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.utils.Sequential
|
||||
~dgl.nn.pytorch.utils.WeightBasis
|
||||
~dgl.nn.pytorch.factory.KNNGraph
|
||||
~dgl.nn.pytorch.factory.SegmentedKNNGraph
|
||||
~dgl.nn.pytorch.factory.RadiusGraph
|
||||
~dgl.nn.pytorch.utils.JumpingKnowledge
|
||||
~dgl.nn.pytorch.sparse_emb.NodeEmbedding
|
||||
~dgl.nn.pytorch.explain.GNNExplainer
|
||||
~dgl.nn.pytorch.explain.HeteroGNNExplainer
|
||||
~dgl.nn.pytorch.explain.SubgraphX
|
||||
~dgl.nn.pytorch.explain.HeteroSubgraphX
|
||||
~dgl.nn.pytorch.explain.PGExplainer
|
||||
~dgl.nn.pytorch.explain.HeteroPGExplainer
|
||||
~dgl.nn.pytorch.utils.LabelPropagation
|
||||
~dgl.nn.pytorch.utils.LaplacianPosEnc
|
||||
|
||||
Network Embedding Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.DeepWalk
|
||||
~dgl.nn.pytorch.MetaPath2Vec
|
||||
|
||||
Utility Modules for Graph Transformer
|
||||
----------------------------------------
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.pytorch.gt.DegreeEncoder
|
||||
~dgl.nn.pytorch.gt.LapPosEncoder
|
||||
~dgl.nn.pytorch.gt.PathEncoder
|
||||
~dgl.nn.pytorch.gt.SpatialEncoder
|
||||
~dgl.nn.pytorch.gt.SpatialEncoder3d
|
||||
~dgl.nn.pytorch.gt.BiasedMHA
|
||||
~dgl.nn.pytorch.gt.GraphormerLayer
|
||||
~dgl.nn.pytorch.gt.EGTLayer
|
||||
@@ -0,0 +1,45 @@
|
||||
.. _apinn-tensorflow:
|
||||
|
||||
dgl.nn (TensorFlow)
|
||||
================
|
||||
|
||||
Conv Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.tensorflow.conv.GraphConv
|
||||
~dgl.nn.tensorflow.conv.RelGraphConv
|
||||
~dgl.nn.tensorflow.conv.GATConv
|
||||
~dgl.nn.tensorflow.conv.SAGEConv
|
||||
~dgl.nn.tensorflow.conv.ChebConv
|
||||
~dgl.nn.tensorflow.conv.SGConv
|
||||
~dgl.nn.tensorflow.conv.APPNPConv
|
||||
~dgl.nn.tensorflow.conv.GINConv
|
||||
|
||||
Global Pooling Layers
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.tensorflow.glob.SumPooling
|
||||
~dgl.nn.tensorflow.glob.AvgPooling
|
||||
~dgl.nn.tensorflow.glob.MaxPooling
|
||||
~dgl.nn.tensorflow.glob.SortPooling
|
||||
~dgl.nn.tensorflow.glob.GlobalAttentionPooling
|
||||
|
||||
Heterogeneous Learning Modules
|
||||
----------------------------------------
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
~dgl.nn.tensorflow.glob.HeteroGraphConv
|
||||
@@ -0,0 +1,11 @@
|
||||
.. _apinn-functional:
|
||||
|
||||
dgl.nn.functional
|
||||
=================
|
||||
|
||||
.. automodule:: dgl.nn.functional
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
edge_softmax
|
||||
@@ -0,0 +1,37 @@
|
||||
.. _apitransform-namespace:
|
||||
|
||||
dgl.transforms
|
||||
==============
|
||||
|
||||
.. currentmodule:: dgl.transforms
|
||||
.. automodule:: dgl.transforms
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
:nosignatures:
|
||||
:template: classtemplate.rst
|
||||
|
||||
BaseTransform
|
||||
Compose
|
||||
AddSelfLoop
|
||||
RemoveSelfLoop
|
||||
AddReverse
|
||||
ToSimple
|
||||
LineGraph
|
||||
KHopGraph
|
||||
AddMetaPaths
|
||||
GCNNorm
|
||||
PPR
|
||||
HeatKernel
|
||||
GDC
|
||||
NodeShuffle
|
||||
DropNode
|
||||
DropEdge
|
||||
AddEdge
|
||||
RandomWalkPE
|
||||
LapPE
|
||||
FeatMask
|
||||
RowFeatNormalizer
|
||||
SIGNDiffusion
|
||||
ToLevi
|
||||
SVDPE
|
||||
@@ -0,0 +1,116 @@
|
||||
.. _apiudf:
|
||||
|
||||
User-defined Functions
|
||||
==================================================
|
||||
|
||||
.. currentmodule:: dgl.udf
|
||||
|
||||
User-defined functions (UDFs) allow arbitrary computation in message passing
|
||||
(see :ref:`guide-message-passing`) and edge feature update with
|
||||
:func:`~dgl.DGLGraph.apply_edges`. They bring more flexibility when :ref:`apifunction`
|
||||
cannot realize a desired computation.
|
||||
|
||||
Edge-wise User-defined Function
|
||||
-------------------------------
|
||||
|
||||
One can use an edge-wise user defined function for a message function in message passing or
|
||||
a function to apply in :func:`~dgl.DGLGraph.apply_edges`. It takes a batch of edges as input
|
||||
and returns messages (in message passing) or features (in :func:`~dgl.DGLGraph.apply_edges`)
|
||||
for each edge. The function may combine the features of the edges and their end nodes in
|
||||
computation.
|
||||
|
||||
Formally, it takes the following form
|
||||
|
||||
.. code::
|
||||
|
||||
def edge_udf(edges):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
edges : EdgeBatch
|
||||
A batch of edges.
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[str, tensor]
|
||||
The messages or edge features generated. It maps a message/feature name to the
|
||||
corresponding messages/features of all edges in the batch. The order of the
|
||||
messages/features is the same as the order of the edges in the input argument.
|
||||
"""
|
||||
|
||||
DGL generates :class:`~dgl.udf.EdgeBatch` instances internally, which expose the following
|
||||
interface for defining ``edge_udf``.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
EdgeBatch.src
|
||||
EdgeBatch.dst
|
||||
EdgeBatch.data
|
||||
EdgeBatch.edges
|
||||
EdgeBatch.batch_size
|
||||
|
||||
Node-wise User-defined Function
|
||||
-------------------------------
|
||||
|
||||
One can use a node-wise user defined function for a reduce function in message passing. It takes
|
||||
a batch of nodes as input and returns the updated features for each node. It may combine the
|
||||
current node features and the messages nodes received. Formally, it takes the following form
|
||||
|
||||
.. code::
|
||||
|
||||
def node_udf(nodes):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
nodes : NodeBatch
|
||||
A batch of nodes.
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[str, tensor]
|
||||
The updated node features. It maps a feature name to the corresponding features of
|
||||
all nodes in the batch. The order of the nodes is the same as the order of the nodes
|
||||
in the input argument.
|
||||
"""
|
||||
|
||||
DGL generates :class:`~dgl.udf.NodeBatch` instances internally, which expose the following
|
||||
interface for defining ``node_udf``.
|
||||
|
||||
.. autosummary::
|
||||
:toctree: ../../generated/
|
||||
|
||||
NodeBatch.data
|
||||
NodeBatch.mailbox
|
||||
NodeBatch.nodes
|
||||
NodeBatch.batch_size
|
||||
|
||||
Degree Bucketing for Message Passing with User Defined Functions
|
||||
----------------------------------------------------------------
|
||||
|
||||
DGL employs a degree-bucketing mechanism for message passing with UDFs. It groups nodes with
|
||||
a same in-degree and invokes message passing for each group of nodes. As a result, one shall
|
||||
not make any assumptions about the batch size of :class:`~dgl.udf.NodeBatch` instances.
|
||||
|
||||
For a batch of nodes, DGL stacks the incoming messages of each node along the second dimension,
|
||||
ordered by edge ID. An example goes as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch
|
||||
>>> import dgl.function as fn
|
||||
>>> g = dgl.graph(([1, 3, 5, 0, 4, 2, 3, 3, 4, 5], [1, 1, 0, 0, 1, 2, 2, 0, 3, 3]))
|
||||
>>> g.edata['eid'] = torch.arange(10)
|
||||
>>> def reducer(nodes):
|
||||
... print(nodes.mailbox['eid'])
|
||||
... return {'n': nodes.mailbox['eid'].sum(1)}
|
||||
>>> g.update_all(fn.copy_e('eid', 'eid'), reducer)
|
||||
tensor([[5, 6],
|
||||
[8, 9]])
|
||||
tensor([[3, 7, 2],
|
||||
[0, 1, 4]])
|
||||
|
||||
Essentially, node #2 and node #3 are grouped into one bucket with in-degree of 2, and node
|
||||
#0 and node #1 are grouped into one bucket with in-degree of 3. Within each bucket, the
|
||||
edges are ordered by the edge IDs for each node.
|
||||
@@ -0,0 +1,261 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file does only contain a selection of the most common options. For a
|
||||
# full list see the documentation:
|
||||
# http://www.sphinx-doc.org/en/master/config
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../../python"))
|
||||
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = "DGL"
|
||||
copyright = "2018, DGL Team"
|
||||
author = "DGL Team"
|
||||
|
||||
import dgl
|
||||
|
||||
version = dgl.__version__
|
||||
release = dgl.__version__
|
||||
dglbackend = os.environ.get("DGLBACKEND", "pytorch")
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#
|
||||
# needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.autosummary",
|
||||
"sphinx.ext.coverage",
|
||||
"sphinx.ext.mathjax",
|
||||
"sphinx.ext.napoleon",
|
||||
"sphinx.ext.viewcode",
|
||||
"sphinx.ext.intersphinx",
|
||||
"sphinx.ext.graphviz",
|
||||
"sphinxemoji.sphinxemoji",
|
||||
"sphinx_gallery.gen_gallery",
|
||||
"sphinx_copybutton",
|
||||
"nbsphinx",
|
||||
"nbsphinx_link",
|
||||
]
|
||||
|
||||
# Do not run notebooks on non-pytorch backends
|
||||
if dglbackend != "pytorch":
|
||||
nbsphinx_execute = "never"
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ["_templates"]
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
#
|
||||
source_suffix = [".rst", ".md"]
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = "index"
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = [
|
||||
"tutorials/**/*.ipynb",
|
||||
"tutorials/**/*.py",
|
||||
]
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = None
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#
|
||||
# html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ["_static"]
|
||||
html_css_files = ["css/custom.css"]
|
||||
|
||||
# Custom sidebar templates, must be a dictionary that maps document names
|
||||
# to template names.
|
||||
#
|
||||
# The default sidebars (for documents that don't match any pattern) are
|
||||
# defined by theme itself. Builtin themes are using these templates by
|
||||
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
|
||||
# 'searchbox.html']``.
|
||||
#
|
||||
# html_sidebars = {}
|
||||
|
||||
|
||||
# -- Options for HTMLHelp output ---------------------------------------------
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = "dgldoc"
|
||||
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#
|
||||
# 'papersize': 'letterpaper',
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#
|
||||
# 'pointsize': '10pt',
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#
|
||||
# 'preamble': '',
|
||||
# Latex figure (float) alignment
|
||||
#
|
||||
# 'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, "dgl.tex", "DGL Documentation", "DGL Team", "manual"),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for manual page output ------------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [(master_doc, "dgl", "DGL Documentation", [author], 1)]
|
||||
|
||||
|
||||
# -- Options for Texinfo output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(
|
||||
master_doc,
|
||||
"dgl",
|
||||
"DGL Documentation",
|
||||
author,
|
||||
"dgl",
|
||||
"Library for deep learning on graphs.",
|
||||
"Miscellaneous",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Epub output -------------------------------------------------
|
||||
|
||||
# Bibliographic Dublin Core info.
|
||||
epub_title = project
|
||||
|
||||
# The unique identifier of the text. This can be a ISBN number
|
||||
# or the project homepage.
|
||||
#
|
||||
# epub_identifier = ''
|
||||
|
||||
# A unique identification for the text.
|
||||
#
|
||||
# epub_uid = ''
|
||||
|
||||
# A list of files that should not be packed into the epub file.
|
||||
epub_exclude_files = ["search.html"]
|
||||
|
||||
|
||||
# -- Extension configuration -------------------------------------------------
|
||||
autosummary_generate = True
|
||||
autodoc_member_order = "alphabetical"
|
||||
# Skip the following members.
|
||||
autodoc_mock_imports = ["dgl.nn.mxnet", "dgl.nn.tensorflow"]
|
||||
|
||||
intersphinx_mapping = {
|
||||
"python": (
|
||||
"https://docs.python.org/{.major}".format(sys.version_info),
|
||||
None,
|
||||
),
|
||||
"numpy": ("http://docs.scipy.org/doc/numpy/", None),
|
||||
"scipy": ("http://docs.scipy.org/doc/scipy/reference", None),
|
||||
"matplotlib": ("http://matplotlib.org/", None),
|
||||
"networkx": ("https://networkx.github.io/documentation/stable", None),
|
||||
}
|
||||
|
||||
# sphinx gallery configurations
|
||||
from sphinx_gallery.sorting import FileNameSortKey
|
||||
|
||||
examples_dirs = [
|
||||
"../../tutorials/blitz",
|
||||
"../../tutorials/dist",
|
||||
"../../tutorials/models",
|
||||
"../../tutorials/multi",
|
||||
"../../tutorials/cpu",
|
||||
] # path to find sources
|
||||
gallery_dirs = [
|
||||
"tutorials/blitz/",
|
||||
"tutorials/dist/",
|
||||
"tutorials/models/",
|
||||
"tutorials/multi/",
|
||||
"tutorials/cpu",
|
||||
] # path to generate docs
|
||||
if dglbackend != "pytorch":
|
||||
examples_dirs = []
|
||||
gallery_dirs = []
|
||||
|
||||
reference_url = {
|
||||
"dgl": None,
|
||||
"numpy": "http://docs.scipy.org/doc/numpy/",
|
||||
"scipy": "http://docs.scipy.org/doc/scipy/reference",
|
||||
"matplotlib": "http://matplotlib.org/",
|
||||
"networkx": "https://networkx.github.io/documentation/stable",
|
||||
}
|
||||
|
||||
sphinx_gallery_conf = {
|
||||
"backreferences_dir": "generated/backreferences",
|
||||
"doc_module": ("dgl", "numpy"),
|
||||
"examples_dirs": examples_dirs,
|
||||
"gallery_dirs": gallery_dirs,
|
||||
"within_subsection_order": FileNameSortKey,
|
||||
"filename_pattern": ".py",
|
||||
"download_all_examples": False,
|
||||
}
|
||||
|
||||
# Compatibility for different backend when builds tutorials
|
||||
if dglbackend == "mxnet":
|
||||
sphinx_gallery_conf["filename_pattern"] = "/*(?<=mx)\.py"
|
||||
if dglbackend == "pytorch":
|
||||
sphinx_gallery_conf["filename_pattern"] = "/*(?<!mx)\.py"
|
||||
|
||||
# sphinx-copybutton tool
|
||||
copybutton_prompt_text = r">>> |\.\.\. "
|
||||
copybutton_prompt_is_regexp = True
|
||||
@@ -0,0 +1,307 @@
|
||||
Contribute to DGL
|
||||
=================
|
||||
|
||||
Any contribution to DGL is welcome. This guide covers everything
|
||||
about how to contribute to DGL.
|
||||
|
||||
General development process
|
||||
---------------------------
|
||||
|
||||
A non-inclusive list of types of contribution is as follows:
|
||||
|
||||
* New features and enhancements (`example <https://github.com/dmlc/dgl/pull/331>`__).
|
||||
* New NN Modules (`example <https://github.com/dmlc/dgl/pull/788>`__).
|
||||
* Bugfix (`example <https://github.com/dmlc/dgl/pull/247>`__).
|
||||
* Document improvement (`example <https://github.com/dmlc/dgl/pull/263>`__).
|
||||
* New models and examples (`example <https://github.com/dmlc/dgl/pull/279>`__).
|
||||
|
||||
For features and bugfix, we recommend first raise an `issue <https://github.com/dmlc/dgl/issues>`__
|
||||
using the corresponding issue template, so that the change could be fully discussed with
|
||||
the community before implementation. For document improvement and new models, we suggest
|
||||
post a thread in our `discussion forum <https://discuss.dgl.ai>`__.
|
||||
|
||||
Before development, please first read the following sections about coding styles and testing.
|
||||
All the changes need to be reviewed in the form of `pull request <https://github.com/dmlc/dgl/pulls>`__.
|
||||
Our `committors <https://github.com/orgs/dmlc/teams/dgl-team/members>`__
|
||||
(who have write permission on the repository) will review the codes and suggest the necessary
|
||||
changes. The PR could be merged once the reviewers approve the changes.
|
||||
|
||||
Git setup (for developers)
|
||||
--------------------------
|
||||
|
||||
First, fork the DGL github repository. Suppose the forked repo is ``https://github.com/username/dgl``.
|
||||
|
||||
Clone your forked repository locally:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone --recursive https://github.com/username/dgl.git
|
||||
|
||||
|
||||
Setup the upstream to the DGL official repository:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git remote add upstream https://github.com/dmlc/dgl.git
|
||||
|
||||
You could verify the remote setting by typing ``git remote -v``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
origin https://github.com/username/dgl.git (fetch)
|
||||
origin https://github.com/username/dgl.git (push)
|
||||
upstream https://github.com/dmlc/dgl.git (fetch)
|
||||
upstream https://github.com/dmlc/dgl.git (push)
|
||||
|
||||
During developing, we suggest work on another branch than the master.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git branch working-branch
|
||||
git checkout working-branch
|
||||
|
||||
Once the changes are done, `create a pull request <https://help.github.com/articles/creating-a-pull-request/>`__
|
||||
so we could review your codes.
|
||||
|
||||
Once the pull request is merged, update your forked repository and delete your working branch:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git checkout master
|
||||
git pull upstream master
|
||||
git push origin master # update your forked repo
|
||||
git branch -D working-branch # the local branch could be deleted
|
||||
|
||||
Coding styles
|
||||
-------------
|
||||
|
||||
For python codes, we generally follow the `PEP8 style guide <https://www.python.org/dev/peps/pep-0008/>`__.
|
||||
The python comments follow `NumPy style python docstrings <https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html>`__.
|
||||
|
||||
For C++ codes, we generally follow the `Google C++ style guide <https://google.github.io/styleguide/cppguide.html>`__.
|
||||
The C++ comments should be `Doxygen compatible <http://www.doxygen.nl/manual/docblocks.html#cppblock>`__.
|
||||
|
||||
Coding styles check is mandatory for every pull requests. To ease the development, please check it
|
||||
locally first (require cpplint and pylint to be installed first):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
bash tests/scripts/task_lint.sh
|
||||
|
||||
The python code style configure file is ``tests/lint/pylintrc``. We tweak it a little bit from
|
||||
the standard. For example, following variable names are accepted:
|
||||
|
||||
* ``i,j,k``: for loop variables
|
||||
* ``u,v``: for representing nodes
|
||||
* ``e``: for representing edges
|
||||
* ``g``: for representing graph
|
||||
* ``fn``: for representing functions
|
||||
* ``n,m``: for representing sizes
|
||||
* ``w,x,y``: for representing weight, input, output tensors
|
||||
* ``_``: for unused variables
|
||||
|
||||
Contributing New Models as Examples
|
||||
-----------------------------------
|
||||
|
||||
To contribute a new model within a specific supported tensor framework (e.g. PyTorch, or MXNet), simply
|
||||
|
||||
1. Make a directory with the name of your model (say ``awesome-gnn``) within the directory
|
||||
``examples/${DGLBACKEND}`` where ``${DGLBACKEND}`` refers to the framework name.
|
||||
|
||||
2. Populate it with your work, along with a README. Make a pull request once you are done. Your README should contain at least these:
|
||||
|
||||
* Instructions for running your program.
|
||||
|
||||
* The performance results, such as speed or accuracy or any metric, along with comparisons against some alternative implementations (if available).
|
||||
|
||||
* Your performance metric does not have to beat others' implementation; they are just a signal of your code being *likely* correct.
|
||||
|
||||
* Your speed also does not have to surpass others'.
|
||||
|
||||
* However, better numbers are always welcomed.
|
||||
|
||||
3. The committers will review it, suggesting or making changes as necessary.
|
||||
|
||||
4. Resolve the suggestions and reviews, and go back to step 3 until approved.
|
||||
|
||||
5. Merge it and enjoy your day.
|
||||
|
||||
Data hosting
|
||||
````````````
|
||||
|
||||
One often wishes to upload a dataset when contributing a new runnable model example, especially when covering
|
||||
a new field not in our existing examples.
|
||||
|
||||
Uploading data file into the Git repository directly is a **bad idea** because we do not want the cloners to
|
||||
always download the dataset no matter what. Instead, we strongly suggest the data files be hosted on a
|
||||
permanent cloud storage service (e.g. DropBox, Amazon S3, Baidu, Google Drive, etc.).
|
||||
|
||||
One can either
|
||||
|
||||
* Make your scripts automatically download your data if possible (e.g. when using Amazon S3), or
|
||||
* Clearly state the instructions of downloading your dataset (e.g. when using Baidu, where auto-downloading
|
||||
is hard).
|
||||
|
||||
If you have trouble doing so (e.g. you cannot find a permanent cloud storage), feel free to post in our
|
||||
`discussion forum <https://discuss.dgl.ai>`__.
|
||||
|
||||
Depending on the commonality of the contributed task, model, or dataset, we (the DGL team) would migrate
|
||||
your dataset to the official DGL Dataset Repository on Amazon S3. If you wish to host a particular dataset,
|
||||
you can either
|
||||
|
||||
* DIY: make changes in the ``dgl.data`` module; see our :ref:`dataset APIs <apidata>` for more details, or,
|
||||
* Post in our `discussion forum <https://discuss.dgl.ai>`__ (again).
|
||||
|
||||
Currently, all the datasets of DGL model examples are hosted on Amazon S3.
|
||||
|
||||
Contributing Core Features
|
||||
--------------------------
|
||||
|
||||
We call a feature that goes into the Python ``dgl`` package a *core feature*.
|
||||
|
||||
Since DGL supports multiple tensor frameworks, contributing a core feature is no easy job. However, we do
|
||||
**NOT** require knowledge of all tensor frameworks. Instead,
|
||||
|
||||
1. Before making a pull request, please make sure your code is covered with unit tests on **at least one**
|
||||
supported frameworks; see the `Building and Testing`_ section for details.
|
||||
2. Once you have done that, make a pull request and summarize your changes, and wait for the CI to finish.
|
||||
3. If the CI fails on a tensor platform that you are unfamiliar with (which is well often the case), please
|
||||
refer to `Supporting Multiple Platforms`_ section.
|
||||
4. The committers will review it, suggesting or making changes as necessary.
|
||||
5. Resolve the suggestions and reviews, and go back to step 3 until approved.
|
||||
6. Merge it and enjoy your day.
|
||||
|
||||
Supporting Multiple Platforms
|
||||
`````````````````````````````
|
||||
|
||||
This is the hard one, but you don't have to know PyTorch AND MXNet (maybe AND Tensorflow, AND Chainer, etc.,
|
||||
in the future) to do so. The rule of thumb in supporting Multiple Platforms is simple:
|
||||
|
||||
* In the ``dgl`` Python package, **always** avoid using framework-specific operators (*including array indexing!*)
|
||||
directly. Use the wrappers in ``dgl.backend`` or ``numpy`` arrays instead.
|
||||
* If you have trouble doing so (either because ``dgl.backend`` does not cover the necessary operator, or you don't
|
||||
have a GPU, or for whatever reason), please label your PR with the ``backend support`` tag, and one or more DGL
|
||||
team member who understand CPU AND GPU AND PyTorch AND MXNet (AND Tensorflow AND Chainer AND etc.) will
|
||||
look into it.
|
||||
|
||||
Building and Testing
|
||||
````````````````````
|
||||
|
||||
To build DGL locally, follow the steps described in :ref:`Install from source <install-from-source>`.
|
||||
However, to ease the development, we suggest NOT install DGL but directly working in the source tree.
|
||||
To achieve this, export following environment variables:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export DGL_HOME=/path/to/your/dgl/clone
|
||||
export DGL_LIBRARY_PATH=$DGL_HOME/build
|
||||
export PYTHONPATH=$PYTHONPATH:$DGL_HOME/python
|
||||
|
||||
If you are working on performance critical part, you may want to turn on Cython build:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd python
|
||||
python setup.py build_ext --inplace
|
||||
|
||||
You could test the build by running the following command and see the path of your local clone.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python -c 'import dgl; print(dgl.__path__)'
|
||||
|
||||
Unit tests
|
||||
~~~~~~~~~~
|
||||
|
||||
Currently, we use ``nose`` for unit tests. The organization goes as follows:
|
||||
|
||||
* ``backend``: Additional unified tensor interface for supported frameworks.
|
||||
The functions there are only used in unit tests, not DGL itself. Note that
|
||||
the code there are not unit tests by themselves. The additional backend can
|
||||
be imported with
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import backend
|
||||
|
||||
The additional backend contains the following files:
|
||||
|
||||
- ``backend/backend_unittest.py``: stub file for all additional tensor
|
||||
functions.
|
||||
- ``backend/${DGLBACKEND}/__init__.py``: implementations of the stubs
|
||||
for the backend ``${DGLBACKEND}``.
|
||||
- ``backend/__init__.py``: when imported, it replaces the stub implementations
|
||||
with the framework-specific code, depending on the selected backend. It
|
||||
also changes the signature of some existing backend functions to automatically
|
||||
select dtypes and contexts.
|
||||
|
||||
* ``compute``: All framework-agnostic computation-related unit tests go there.
|
||||
Anything inside should not depend on a specific tensor library. Tensor
|
||||
functions not provided in DGL unified tensor interface (i.e. ``dgl.backend``)
|
||||
should go into ``backend`` directory.
|
||||
* ``${DGLBACKEND}`` (e.g. ``pytorch`` and ``mxnet``): All framework-specific
|
||||
computation-related unit tests go there.
|
||||
* ``graph_index``: All unit tests for C++ graph structure implementation go
|
||||
there. The Python API being tested in this directory, if any, should be
|
||||
as minimal as possible (usually simple wrappers of corresponding C++
|
||||
functions).
|
||||
* ``lint``: Pylint-related files.
|
||||
* ``scripts``: Automated test scripts for CI.
|
||||
|
||||
To run unit tests, run
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sh tests/scripts/task_unit_test.sh <your-backend>
|
||||
|
||||
where ``<your-backend>`` can be any supported backends (i.e. ``pytorch`` or ``mxnet``).
|
||||
|
||||
Contributing Documentations
|
||||
---------------------------
|
||||
|
||||
If the change is about document improvement, we suggest (and strongly suggest if you change the runnable code
|
||||
there) building the document and render it locally before making a pull request.
|
||||
|
||||
Building Docs Locally
|
||||
`````````````````````
|
||||
|
||||
In general building the docs locally involves the following:
|
||||
|
||||
1. Install ``sphinx``, ``sphinx-gallery``, and ``sphinx_rtd_theme``.
|
||||
|
||||
2. You need both PyTorch and MXNet because our tutorial contains code from both frameworks. This does *not*
|
||||
require knowledge of coding with both frameworks, though.
|
||||
|
||||
3. Run the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd docs
|
||||
./clean.sh
|
||||
make html
|
||||
cd build/html
|
||||
python3 -m http.server 8080
|
||||
|
||||
4. Open ``http://localhost:8080`` and enjoy your work.
|
||||
|
||||
See `here <https://github.com/dmlc/dgl/tree/master/docs>`__ for more details.
|
||||
|
||||
Contributing Editorial Changes via GitHub Web Interface
|
||||
```````````````````````````````````````````````````````
|
||||
|
||||
If one is only changing the wording (i.e. not touching the runnable code at all), one can simply do
|
||||
without the usage of Git CLI:
|
||||
|
||||
1. Make your fork by clicking on the **Fork** button in the DGL main repository web page.
|
||||
2. Make whatever changes in the web interface *within your own fork*. You can usually tell
|
||||
if you are inside your own fork or in the main repository by checking whether you can commit
|
||||
to the ``master`` branch: if you cannot, you are in the wrong place.
|
||||
3. Once done, make a pull request (on the web interface).
|
||||
4. The committers will review it, suggesting or making changes as necessary.
|
||||
5. Resolve the suggestions and reviews, and go back to step 4 until approved.
|
||||
6. Merge it and enjoy your day.
|
||||
|
||||
Contributing Code Changes
|
||||
`````````````````````````
|
||||
|
||||
When changing code, please make sure to build it locally and see if it fails.
|
||||
@@ -0,0 +1,223 @@
|
||||
.. currentmodule:: dgl
|
||||
|
||||
DGL Foreign Function Interface (FFI)
|
||||
====================================
|
||||
|
||||
We all like Python because it is easy to manipulate. We all like C because it
|
||||
is fast, reliable and typed. To have the merits of both ends, DGL is mostly in
|
||||
python, for quick prototyping, while lowers the performance-critical part to C.
|
||||
Thus, DGL developers frequently face the scenario to write a C routine and has
|
||||
it exposed to python, via a mechanism called *Foreign Function Interface (FFI)*.
|
||||
|
||||
There are many FFI solutions out there. In DGL, we want to keep it simple,
|
||||
intuitive and efficient for critical use cases. That's why when we came across the
|
||||
FFI solution in the TVM project, we immediately fell for it. It exploits the idea of
|
||||
functional programming so that it exposes only a dozens of C APIs and new APIs
|
||||
can be built upon it.
|
||||
|
||||
We decided to borrow the idea (shamelessly). For example, to define a C
|
||||
API that is exposed to python is only a few lines of codes:
|
||||
|
||||
.. code:: c++
|
||||
|
||||
// file: calculator.cc (put it in dgl/src folder)
|
||||
#include <dgl/runtime/packed_func.h>
|
||||
#include <dgl/runtime/registry.h>
|
||||
|
||||
using namespace dgl::runtime;
|
||||
|
||||
DGL_REGISTER_GLOBAL("calculator.MyAdd")
|
||||
.set_body([] (DGLArgs args, DGLRetValue* rv) {
|
||||
int a = args[0];
|
||||
int b = args[1];
|
||||
*rv = a + b;
|
||||
});
|
||||
|
||||
Compile and build the library. On the python side, create a
|
||||
``calculator.py`` file under ``dgl/python/dgl/``
|
||||
|
||||
.. code:: python
|
||||
|
||||
# file: calculator.py
|
||||
from ._ffi.function import _init_api
|
||||
|
||||
def add(a, b):
|
||||
# MyAdd has been registered via `_ini_api` call below
|
||||
return MyAdd(a, b)
|
||||
|
||||
_init_api("dgl.calculator")
|
||||
|
||||
The trick is that the FFI system first masks the type information of the
|
||||
function arguments, so all the C function calls can go through one C API
|
||||
(``DGLFuncCall``). The type information is retrieved in the function body by
|
||||
static conversion, and we will do runtime type check to make sure that the type
|
||||
conversion is correct. The overhead of such back-and-forth is negligible as
|
||||
long as the function call is not too light (the above example is actually a bad
|
||||
one). TVM's `PackedFunc
|
||||
document <https://docs.tvm.ai/dev/runtime.html#packedfunc>`_ has more details.
|
||||
|
||||
Defining new types
|
||||
------------------
|
||||
|
||||
``DGLArgs`` and ``DGLRetValue`` only support a limited number of types:
|
||||
|
||||
* Numerical values: int, float, double, ...
|
||||
* string
|
||||
* Function (in the form of PackedFunc)
|
||||
* NDArray
|
||||
|
||||
Though limited, the above type system is very powerful because it supports
|
||||
function as a first-class citizen. For example, if you want to return multiple
|
||||
values, you can return a PackedFunc which returns each value given an integer
|
||||
index. However, in many cases, new types are still desired to ease the
|
||||
development process:
|
||||
|
||||
* The argument/return value is a composition of collections (e.g. dictionary of
|
||||
dictionary of list).
|
||||
* Sometimes we just want to have a notion of "structure" (e.g. given an apple,
|
||||
get its color by ``apple.color``).
|
||||
|
||||
To achieve this, we introduce the Object type system. For example, to define a
|
||||
new type ``Calculator``:
|
||||
|
||||
.. code:: c++
|
||||
|
||||
// file: calculator.cc
|
||||
#include <dgl/packed_func_ext.h>
|
||||
using namespace runtime;
|
||||
class CalculatorObject : public Object {
|
||||
public:
|
||||
std::string brand;
|
||||
int price;
|
||||
|
||||
void VisitAttrs(AttrVisitor *v) final {
|
||||
v->Visit("brand", &brand);
|
||||
v->Visit("price", &price);
|
||||
}
|
||||
|
||||
static constexpr const char* _type_key = "Calculator";
|
||||
DGL_DECLARE_OBJECT_TYPE_INFO(CalculatorObject, Object);
|
||||
};
|
||||
|
||||
// This is to define a reference class (the wrapper of an object shared pointer).
|
||||
// A minimal implementation is as follows, but you could define extra methods.
|
||||
class Calculator : public ObjectRef {
|
||||
public:
|
||||
const CalculatorObject* operator->() const {
|
||||
return static_cast<const CalculatorObject*>(obj_.get());
|
||||
}
|
||||
using ContainerType = CalculatorObject;
|
||||
};
|
||||
|
||||
DGL_REGISTER_GLOBAL("calculator.CreateCaculator")
|
||||
.set_body([] (DGLArgs args, DGLRetValue* rv) {
|
||||
std::string brand = args[0];
|
||||
int price = args[1];
|
||||
auto o = std::make_shared<CalculatorObject>();
|
||||
o->brand = brand;
|
||||
o->price = price;
|
||||
*rv = o;
|
||||
}
|
||||
|
||||
On the python side:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# file: calculator.py
|
||||
from dgl._ffi.object import register_object, ObjectBase
|
||||
from ._ffi.function import _init_api
|
||||
|
||||
@register_object
|
||||
class Calculator(ObjectBase):
|
||||
@staticmethod
|
||||
def create(brand, price):
|
||||
# invoke a C API, the return value is of `Calculator` type
|
||||
return CreateCalculator(brand, price)
|
||||
|
||||
_init_api("dgl.calculator")
|
||||
|
||||
We can then simply create ``Calculator`` object by:
|
||||
|
||||
.. code:: python
|
||||
|
||||
calc = Calculator.create("casio", 100)
|
||||
|
||||
What is nice about this object is that, it defines a visitor pattern that is
|
||||
essentially a reflection mechanism to get its internal attributes. For example,
|
||||
you can print the calculator's brand and by simply accessing its attributes.
|
||||
|
||||
.. code:: python
|
||||
|
||||
print(calc.brand)
|
||||
print(calc.price)
|
||||
|
||||
The reflection is indeed a little bit slow due to the string key lookup. To
|
||||
speed it up, you could define an attribute access API:
|
||||
|
||||
.. code:: c++
|
||||
|
||||
// file: calculator.cc
|
||||
DGL_REGISTER_GLOBAL("calculator.CaculatorGetBrand")
|
||||
.set_body([] (DGLArgs args, DGLRetValue* rv) {
|
||||
Calculator calc = args[0];
|
||||
*rv = calc->brand;
|
||||
}
|
||||
|
||||
Containers
|
||||
----------
|
||||
|
||||
Containers are also objects. For example, the C API below accepts a list of
|
||||
integers and return their sum:
|
||||
|
||||
.. code:: c++
|
||||
|
||||
// in file: calculator.cc
|
||||
#include <dgl/runtime/container.h>
|
||||
using namespace runtime;
|
||||
DGL_REGISTER_GLOBAL("calculator.Sum")
|
||||
.set_body([] (DGLArgs args, DGLRetValue* rv) {
|
||||
// All the DGL supported values are represented as a ValueObject, which
|
||||
// contains a data field.
|
||||
List<Value> values = args[0];
|
||||
int sum = 0;
|
||||
for (int i = 0; i < values.size(); ++i) {
|
||||
sum += static_cast<int>(values[i]->data);
|
||||
}
|
||||
}
|
||||
|
||||
Invoking this API is simple -- just pass a python list of integers. DGL FFI will
|
||||
automatically convert python list/tuple/dictionary to the corresponding object
|
||||
type.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# in file: calculator.py
|
||||
from ._ffi.function import _init_api
|
||||
|
||||
Sum([0, 1, 2, 3, 4, 5])
|
||||
|
||||
_init_api("dgl.calculator")
|
||||
|
||||
The elements in the containers can be any objects, which allows the containers
|
||||
to be composed. Below is an API that accepts a list of calculators and print
|
||||
out their price:
|
||||
|
||||
.. code:: c++
|
||||
|
||||
// in file: calculator.cc
|
||||
#include <iostream>
|
||||
#include <dgl/runtime/container.h>
|
||||
using namespace runtime;
|
||||
DGL_REGISTER_GLOBAL("calculator.PrintCalculators")
|
||||
.set_body([] (DGLArgs args, DGLRetValue* rv) {
|
||||
List<Calculator> calcs = args[0];
|
||||
for (int i = 0; i < calcs.size(); ++i) {
|
||||
std::cout << calcs[i]->price << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
Please note that containers are NOT meant for passing a large collection of
|
||||
items from/to C APIs. It will be quite slow in these cases. It is recommended
|
||||
to benchmark first. As an alternative, use NDArray for a large collection of
|
||||
numerical values and use ``dgl.batch`` to batch a lot of ``DGLGraph``'s into
|
||||
a single ``DGLGraph``.
|
||||
@@ -0,0 +1,34 @@
|
||||
Environment Variables
|
||||
=====================
|
||||
|
||||
Global Configurations
|
||||
---------------------
|
||||
* ``DGLDEFAULTDIR``:
|
||||
* Values: String (default=``"${HOME}/.dgl"``)
|
||||
* The directory to save the DGL configuration files.
|
||||
|
||||
* ``DGL_LOG_DEBUG``:
|
||||
* Values: Set to ``"1"`` to enable debug level logging for DGL
|
||||
* Enable debug level logging for DGL
|
||||
|
||||
Backend Options
|
||||
---------------
|
||||
* ``DGLBACKEND``:
|
||||
* Values: String (default='pytorch')
|
||||
* The backend deep learning framework for DGL.
|
||||
* Choices:
|
||||
* 'pytorch': use PyTorch as the backend implementation.
|
||||
* 'tensorflow': use Apache TensorFlow as the backend implementation.
|
||||
* 'mxnet': use Apache MXNet as the backend implementation.
|
||||
|
||||
Data Repository
|
||||
---------------
|
||||
* ``DGL_REPO``:
|
||||
* Values: String (default='https://data.dgl.ai/')
|
||||
* The repository url to be used for DGL datasets and pre-trained models.
|
||||
* Suggested values:
|
||||
* 'https://data.dgl.ai/': DGL repo for Global Region.
|
||||
* 'https://dgl-data.s3.cn-north-1.amazonaws.com.cn/': DGL repo for Mainland China
|
||||
* ``DGL_DOWNLOAD_DIR``:
|
||||
* Values: String (default=``"${HOME}/.dgl"``)
|
||||
* The local directory to cache the downloaded data.
|
||||
@@ -0,0 +1,4 @@
|
||||
Frequently Asked Questions (FAQ)
|
||||
================================
|
||||
|
||||
For frequently asked questions, refer to `this post <https://discuss.dgl.ai/t/frequently-asked-questions-faq/1681>`__.
|
||||
@@ -0,0 +1,48 @@
|
||||
Dataset (Temporary)
|
||||
|
||||
|
||||
.. table::
|
||||
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
| Datset Name | Usage |# of graphs|Avg. # of nodes|Avg. # of edges| Node field |Edge field |Temporal|
|
||||
+================+==========================================================+===========+===============+===============+============================================+===========+========+
|
||||
|BitcoinOTC |BitcoinOTC() | 136| 6005.00| 21209.98| |h |True |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Cora |CitationGraphDataset('cora') | 1| 2708.00| 10556.00|train_mask, val_mask, test_mask, label, feat| |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Citeseer |CitationGraphDataset('citeseer') | 1| 3327.00| 9228.00|train_mask, val_mask, test_mask, label, feat| |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|PubMed |CitationGraphDataset('pubmed') | 1| 19717.00| 88651.00|train_mask, val_mask, test_mask, label, feat| |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|QM7b |QM7b() | 7211| 15.42| 244.95| |h |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Reddit |RedditDataset() | 1| 232965.00| 114615892.00|train_mask, val_mask, test_mask, feat, label| |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|ENZYMES |TUDataset('ENZYMES') | 600| 32.63| 124.27|node_labels, node_attr | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|DD |TUDataset('DD') | 1178| 284.32| 1431.32|node_labels | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|COLLAB |TUDataset('COLLAB') | 5000| 74.49| 9830.00| | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|MUTAG |TUDataset('MUTAG') | 188| 17.93| 39.59|node_labels |edge_labels|False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|PROTEINS |TUDataset('PROTEINS') | 1113| 39.06| 145.63|node_labels, node_attr | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|PPI |PPIDataset('train')/PPIDataset('valid')/PPIDataset('test')| 20| 2245.30| 63563.70|feat | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|KarateClub |KarateClub() | 1| 34.00| 156.00|label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Amazon computer |AmazonCoBuy('computers') | 1| 13752.00| 574418.00|feat, label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Amazon photo |AmazonCoBuy('photo') | 1| 7650.00| 287326.00|feat, label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Coauthor cs |Coauthor('cs') | 1| 18333.00| 327576.00|feat, label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|Coauthor physics|Coauthor('physics') | 1| 34493.00| 991848.00|feat, label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|GDELT |GDELT('train')/GDELT('valid')/GDELT('test') | 2304| 23033.00| 811333.15| |rel_type |True |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|ICEWS18 |ICEWS18('train')/ICEWS18('valid')/ICEWS18('test') | 240| 23033.00| 192640.22| |rel_type |True |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
|CoraFull |CoraFull() | 1| 19793.00| 130622.00|feat, label | |False |
|
||||
+----------------+----------------------------------------------------------+-----------+---------------+---------------+--------------------------------------------+-----------+--------+
|
||||
@@ -0,0 +1,74 @@
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from dgl import DGLGraph
|
||||
|
||||
# from dgl.data.qm9 import QM9
|
||||
from dgl.data import CitationGraphDataset, PPIDataset, RedditDataset, TUDataset
|
||||
from dgl.data.bitcoinotc import BitcoinOTC
|
||||
from dgl.data.gdelt import GDELT
|
||||
from dgl.data.gindt import GINDataset
|
||||
from dgl.data.gnn_benchmark import AmazonCoBuy, Coauthor, CoraFull
|
||||
from dgl.data.icews18 import ICEWS18
|
||||
from dgl.data.karate import KarateClub
|
||||
from dgl.data.qm7b import QM7b
|
||||
from pytablewriter import MarkdownTableWriter, RstGridTableWriter
|
||||
|
||||
ds_list = {
|
||||
"BitcoinOTC": "BitcoinOTC()",
|
||||
"Cora": "CitationGraphDataset('cora')",
|
||||
"Citeseer": "CitationGraphDataset('citeseer')",
|
||||
"PubMed": "CitationGraphDataset('pubmed')",
|
||||
"QM7b": "QM7b()",
|
||||
"Reddit": "RedditDataset()",
|
||||
"ENZYMES": "TUDataset('ENZYMES')",
|
||||
"DD": "TUDataset('DD')",
|
||||
"COLLAB": "TUDataset('COLLAB')",
|
||||
"MUTAG": "TUDataset('MUTAG')",
|
||||
"PROTEINS": "TUDataset('PROTEINS')",
|
||||
"PPI": "PPIDataset('train')/PPIDataset('valid')/PPIDataset('test')",
|
||||
# "Cora Binary": "CitationGraphDataset('cora_binary')",
|
||||
"KarateClub": "KarateClub()",
|
||||
"Amazon computer": "AmazonCoBuy('computers')",
|
||||
"Amazon photo": "AmazonCoBuy('photo')",
|
||||
"Coauthor cs": "Coauthor('cs')",
|
||||
"Coauthor physics": "Coauthor('physics')",
|
||||
"GDELT": "GDELT('train')/GDELT('valid')/GDELT('test')",
|
||||
"ICEWS18": "ICEWS18('train')/ICEWS18('valid')/ICEWS18('test')",
|
||||
"CoraFull": "CoraFull()",
|
||||
}
|
||||
|
||||
writer = RstGridTableWriter()
|
||||
# writer = MarkdownTableWriter()
|
||||
|
||||
extract_graph = lambda g: g if isinstance(g, DGLGraph) else g[0]
|
||||
stat_list = []
|
||||
for k, v in ds_list.items():
|
||||
print(k, " ", v)
|
||||
ds = eval(v.split("/")[0])
|
||||
num_nodes = []
|
||||
num_edges = []
|
||||
for i in range(len(ds)):
|
||||
g = extract_graph(ds[i])
|
||||
num_nodes.append(g.num_nodes())
|
||||
num_edges.append(g.num_edges())
|
||||
|
||||
gg = extract_graph(ds[0])
|
||||
dd = {
|
||||
"Datset Name": k,
|
||||
"Usage": v,
|
||||
"# of graphs": len(ds),
|
||||
"Avg. # of nodes": np.mean(num_nodes),
|
||||
"Avg. # of edges": np.mean(num_edges),
|
||||
"Node field": ", ".join(list(gg.ndata.keys())),
|
||||
"Edge field": ", ".join(list(gg.edata.keys())),
|
||||
# "Graph field": ', '.join(ds[0][0].gdata.keys()) if hasattr(ds[0][0], "gdata") else "",
|
||||
"Temporal": hasattr(ds, "is_temporal"),
|
||||
}
|
||||
stat_list.append(dd)
|
||||
|
||||
print(dd.keys())
|
||||
df = pd.DataFrame(stat_list)
|
||||
df = df.reindex(columns=dd.keys())
|
||||
writer.from_dataframe(df)
|
||||
|
||||
writer.write_table()
|
||||
@@ -0,0 +1,91 @@
|
||||
Prepare Data
|
||||
============
|
||||
|
||||
In this section, we will prepare the data for the Graphormer model introduced before. We can use any dataset containing :class:`~dgl.DGLGraph` objects and standard PyTorch dataloader to feed the data to the model. The key is to define a collate function to group features of multiple graphs into batches. We show an example of the collate function as follows:
|
||||
|
||||
|
||||
.. code:: python
|
||||
|
||||
def collate(graphs):
|
||||
# compute shortest path features, can be done in advance
|
||||
for g in graphs:
|
||||
spd, path = dgl.shortest_dist(g, root=None, return_paths=True)
|
||||
g.ndata["spd"] = spd
|
||||
g.ndata["path"] = path
|
||||
|
||||
num_graphs = len(graphs)
|
||||
num_nodes = [g.num_nodes() for g in graphs]
|
||||
max_num_nodes = max(num_nodes)
|
||||
|
||||
attn_mask = th.zeros(num_graphs, max_num_nodes, max_num_nodes)
|
||||
node_feat = []
|
||||
in_degree, out_degree = [], []
|
||||
path_data = []
|
||||
# Since shortest_dist returns -1 for unreachable node pairs and padded
|
||||
# nodes are unreachable to others, distance relevant to padded nodes
|
||||
# use -1 padding as well.
|
||||
dist = -th.ones(
|
||||
(num_graphs, max_num_nodes, max_num_nodes), dtype=th.long
|
||||
)
|
||||
|
||||
for i in range(num_graphs):
|
||||
# A binary mask where invalid positions are indicated by True.
|
||||
# Avoid the case where all positions are invalid.
|
||||
attn_mask[i, :, num_nodes[i] + 1 :] = 1
|
||||
|
||||
# +1 to distinguish padded non-existing nodes from real nodes
|
||||
node_feat.append(graphs[i].ndata["feat"] + 1)
|
||||
|
||||
# 0 for padding
|
||||
in_degree.append(
|
||||
th.clamp(graphs[i].in_degrees() + 1, min=0, max=512)
|
||||
)
|
||||
out_degree.append(
|
||||
th.clamp(graphs[i].out_degrees() + 1, min=0, max=512)
|
||||
)
|
||||
|
||||
# Path padding to make all paths to the same length "max_len".
|
||||
path = graphs[i].ndata["path"]
|
||||
path_len = path.size(dim=2)
|
||||
# shape of shortest_path: [n, n, max_len]
|
||||
max_len = 5
|
||||
if path_len >= max_len:
|
||||
shortest_path = path[:, :, :max_len]
|
||||
else:
|
||||
p1d = (0, max_len - path_len)
|
||||
# Use the same -1 padding as shortest_dist for
|
||||
# invalid edge IDs.
|
||||
shortest_path = th.nn.functional.pad(path, p1d, "constant", -1)
|
||||
pad_num_nodes = max_num_nodes - num_nodes[i]
|
||||
p3d = (0, 0, 0, pad_num_nodes, 0, pad_num_nodes)
|
||||
shortest_path = th.nn.functional.pad(shortest_path, p3d, "constant", -1)
|
||||
# +1 to distinguish padded non-existing edges from real edges
|
||||
edata = graphs[i].edata["feat"] + 1
|
||||
|
||||
# shortest_dist pads non-existing edges (at the end of shortest
|
||||
# paths) with edge IDs -1, and th.zeros(1, edata.shape[1]) stands
|
||||
# for all padded edge features.
|
||||
edata = th.cat(
|
||||
(edata, th.zeros(1, edata.shape[1]).to(edata.device)), dim=0
|
||||
)
|
||||
path_data.append(edata[shortest_path])
|
||||
|
||||
dist[i, : num_nodes[i], : num_nodes[i]] = graphs[i].ndata["spd"]
|
||||
|
||||
# node feat padding
|
||||
node_feat = th.nn.utils.rnn.pad_sequence(node_feat, batch_first=True)
|
||||
|
||||
# degree padding
|
||||
in_degree = th.nn.utils.rnn.pad_sequence(in_degree, batch_first=True)
|
||||
out_degree = th.nn.utils.rnn.pad_sequence(out_degree, batch_first=True)
|
||||
|
||||
return (
|
||||
node_feat,
|
||||
in_degree,
|
||||
out_degree,
|
||||
attn_mask,
|
||||
th.stack(path_data),
|
||||
dist,
|
||||
)
|
||||
|
||||
In this example, we also omit details like the addition of a virtual node. For more details, please refer to the `Graphormer example <https://github.com/dmlc/dgl/tree/master/examples/core/Graphormer>`_.
|
||||
@@ -0,0 +1,12 @@
|
||||
🆕 Tutorial: Graph Transformer
|
||||
==========
|
||||
|
||||
This tutorial introduces the **graph transformer** (:mod:`~dgl.nn.gt`) module,
|
||||
which is a set of utility modules for building and training graph transformer models.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
model
|
||||
data
|
||||
@@ -0,0 +1,94 @@
|
||||
Build Model
|
||||
===========
|
||||
|
||||
**GraphTransformer** is a graph neural network that uses multi-head self-attention (sparse or dense) to encode the graph structure and node features. It is a generalization of the `Transformer <https://arxiv.org/abs/1706.03762>`_ architecture to arbitrary graphs.
|
||||
|
||||
In this tutorial, we will show how to build a graph transformer model with DGL using the `Graphormer <https://arxiv.org/abs/2106.05234>`_ model as an example.
|
||||
|
||||
Graphormer is a Transformer model designed for graph-structured data, which encodes the structural information of a graph into the standard Transformer. Specifically, Graphormer utilizes degree encoding to measure the importance of nodes, spatial and path Encoding to measure the relation between node pairs. The degree encoding and the node features serve as input to Graphormer, while the spatial and path encoding act as bias terms in the self-attention module.
|
||||
|
||||
Degree Encoding
|
||||
-------------------
|
||||
The degree encoder is a learnable embedding layer that encodes the degree of each node into a vector. It takes as input the batched input and output degrees of graph nodes, and outputs the degree embeddings of the nodes.
|
||||
|
||||
.. code:: python
|
||||
|
||||
degree_encoder = dgl.nn.DegreeEncoder(
|
||||
max_degree=8, # the maximum degree to cut off
|
||||
embedding_dim=512 # the dimension of the degree embedding
|
||||
)
|
||||
|
||||
Path Encoding
|
||||
-------------
|
||||
The path encoder encodes the edge features on the shortest path between two nodes to get attention bias for the self-attention module. It takes as input the batched edge features in shape and outputs the attention bias based on path encoding.
|
||||
|
||||
.. code:: python
|
||||
|
||||
path_encoder = PathEncoder(
|
||||
max_len=5, # the maximum length of the shortest path
|
||||
feat_dim=512, # the dimension of the edge feature
|
||||
num_heads=8, # the number of attention heads
|
||||
)
|
||||
|
||||
Spatial Encoding
|
||||
----------------
|
||||
The spatial encoder encodes the shortest distance between two nodes to get attention bias for the self-attention module. It takes as input the shortest distance between two nodes and outputs the attention bias based on spatial encoding.
|
||||
|
||||
.. code:: python
|
||||
|
||||
spatial_encoder = SpatialEncoder(
|
||||
max_dist=5, # the maximum distance between two nodes
|
||||
num_heads=8, # the number of attention heads
|
||||
)
|
||||
|
||||
|
||||
Graphormer Layer
|
||||
----------------
|
||||
The Graphormer layer is like a Transformer encoder layer with the Multi-head Attention part replaced with :class:`~dgl.nn.BiasedMHA`. It takes in not only the input node features, but also the attention bias computed computed above, and outputs the updated node features.
|
||||
|
||||
We can stack multiple Graphormer layers as a list just like implementing a Transformer encoder in PyTorch.
|
||||
|
||||
.. code:: python
|
||||
|
||||
layers = th.nn.ModuleList([
|
||||
GraphormerLayer(
|
||||
feat_size=512, # the dimension of the input node features
|
||||
hidden_size=1024, # the dimension of the hidden layer
|
||||
num_heads=8, # the number of attention heads
|
||||
dropout=0.1, # the dropout rate
|
||||
activation=th.nn.ReLU(), # the activation function
|
||||
norm_first=False, # whether to put the normalization before attention and feedforward
|
||||
)
|
||||
for _ in range(6)
|
||||
])
|
||||
|
||||
Model Forward
|
||||
-------------
|
||||
Grouping the modules above defines the primary components of the Graphormer model. We then can define the forward process as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_feat, in_degree, out_degree, attn_mask, path_data, dist = \
|
||||
next(iter(dataloader)) # we will use the first batch as an example
|
||||
num_graphs, max_num_nodes, _ = node_feat.shape
|
||||
deg_emb = degree_encoder(th.stack((in_degree, out_degree)))
|
||||
|
||||
# node feature + degree encoding as input
|
||||
node_feat = node_feat + deg_emb
|
||||
|
||||
# spatial encoding and path encoding serve as attention bias
|
||||
path_encoding = path_encoder(dist, path_data)
|
||||
spatial_encoding = spatial_encoder(dist)
|
||||
attn_bias[:, 1:, 1:, :] = path_encoding + spatial_encoding
|
||||
|
||||
# graphormer layers
|
||||
for layer in layers:
|
||||
x = layer(
|
||||
x,
|
||||
attn_mask=attn_mask,
|
||||
attn_bias=attn_bias,
|
||||
)
|
||||
|
||||
For simplicity, we omit some details in the forward process. For the complete implementation, please refer to the `Graphormer example <https://github.com/dmlc/dgl/tree/master/examples/core/Graphormer>`_.
|
||||
|
||||
You can also explore other `utility modules <https://docs.dgl.ai/api/python/nn-pytorch.html#utility-modules-for-graph-transformer>`_ to customize your own graph transformer model. In the next section, we will show how to prepare the data for training.
|
||||
@@ -0,0 +1,102 @@
|
||||
.. _guide-data-pipeline-dataset:
|
||||
|
||||
4.1 DGLDataset class
|
||||
--------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline-dataset>`
|
||||
|
||||
:class:`~dgl.data.DGLDataset` is the base class for processing, loading and saving
|
||||
graph datasets defined in :ref:`apidata`. It implements the basic pipeline
|
||||
for processing graph data. The following flow chart shows how the
|
||||
pipeline works.
|
||||
|
||||
To process a graph dataset located in a remote server or local disk, one can
|
||||
define a class, say ``MyDataset``, inheriting from :class:`dgl.data.DGLDataset`. The
|
||||
template of ``MyDataset`` is as follows.
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/userguide_data_flow.png
|
||||
:align: center
|
||||
|
||||
Flow chart for graph data input pipeline defined in class DGLDataset.
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLDataset
|
||||
|
||||
class MyDataset(DGLDataset):
|
||||
""" Template for customizing graph datasets in DGL.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
url : str
|
||||
URL to download the raw dataset
|
||||
raw_dir : str
|
||||
Specifying the directory that will store the
|
||||
downloaded data or the directory that
|
||||
already stores the input data.
|
||||
Default: ~/.dgl/
|
||||
save_dir : str
|
||||
Directory to save the processed dataset.
|
||||
Default: the value of `raw_dir`
|
||||
force_reload : bool
|
||||
Whether to reload the dataset. Default: False
|
||||
verbose : bool
|
||||
Whether to print out progress information
|
||||
"""
|
||||
def __init__(self,
|
||||
url=None,
|
||||
raw_dir=None,
|
||||
save_dir=None,
|
||||
force_reload=False,
|
||||
verbose=False):
|
||||
super(MyDataset, self).__init__(name='dataset_name',
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
save_dir=save_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def download(self):
|
||||
# download raw data to local disk
|
||||
pass
|
||||
|
||||
def process(self):
|
||||
# process raw data to graphs, labels, splitting masks
|
||||
pass
|
||||
|
||||
def __getitem__(self, idx):
|
||||
# get one example by index
|
||||
pass
|
||||
|
||||
def __len__(self):
|
||||
# number of data examples
|
||||
pass
|
||||
|
||||
def save(self):
|
||||
# save processed data to directory `self.save_path`
|
||||
pass
|
||||
|
||||
def load(self):
|
||||
# load processed data from directory `self.save_path`
|
||||
pass
|
||||
|
||||
def has_cache(self):
|
||||
# check whether there are processed data in `self.save_path`
|
||||
pass
|
||||
|
||||
|
||||
:class:`~dgl.data.DGLDataset` class has abstract functions ``process()``,
|
||||
``__getitem__(idx)`` and ``__len__()`` that must be implemented in the
|
||||
subclass. DGL also recommends implementing saving and loading as well,
|
||||
since they can save significant time for processing large datasets, and
|
||||
there are several APIs making it easy (see :ref:`guide-data-pipeline-savenload`).
|
||||
|
||||
Note that the purpose of :class:`~dgl.data.DGLDataset` is to provide a standard and
|
||||
convenient way to load graph data. One can store graphs, features,
|
||||
labels, masks and basic information about the dataset, such as number of
|
||||
classes, number of labels, etc. Operations such as sampling, partition
|
||||
or feature normalization are done outside of the :class:`~dgl.data.DGLDataset`
|
||||
subclass.
|
||||
|
||||
The rest of this chapter shows the best practices to implement the
|
||||
functions in the pipeline.
|
||||
@@ -0,0 +1,58 @@
|
||||
.. _guide-data-pipeline-download:
|
||||
|
||||
4.2 Download raw data (optional)
|
||||
--------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline-download>`
|
||||
|
||||
If a dataset is already in local disk, make sure it’s in directory
|
||||
``raw_dir``. If one wants to run the code anywhere without bothering to
|
||||
download and move data to the right directory, one can do it
|
||||
automatically by implementing function ``download()``.
|
||||
|
||||
If the dataset is a zip file, make ``MyDataset`` inherit from
|
||||
:class:`dgl.data.DGLBuiltinDataset` class, which handles the zip file extraction for us. Otherwise,
|
||||
one needs to implement ``download()`` like in :class:`~dgl.data.QM7bDataset`:
|
||||
|
||||
.. code::
|
||||
|
||||
import os
|
||||
from dgl.data.utils import download
|
||||
|
||||
def download(self):
|
||||
# path to store the file
|
||||
file_path = os.path.join(self.raw_dir, self.name + '.mat')
|
||||
# download file
|
||||
download(self.url, path=file_path)
|
||||
|
||||
The above code downloads a .mat file to directory ``self.raw_dir``. If
|
||||
the file is a .gz, .tar, .tar.gz or .tgz file, use :func:`~dgl.data.utils.extract_archive`
|
||||
function to extract. The following code shows how to download a .gz file
|
||||
in :class:`~dgl.data.BitcoinOTCDataset`:
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data.utils import download, check_sha1
|
||||
|
||||
def download(self):
|
||||
# path to store the file
|
||||
# make sure to use the same suffix as the original file name's
|
||||
gz_file_path = os.path.join(self.raw_dir, self.name + '.csv.gz')
|
||||
# download file
|
||||
download(self.url, path=gz_file_path)
|
||||
# check SHA-1
|
||||
if not check_sha1(gz_file_path, self._sha1_str):
|
||||
raise UserWarning('File {} is downloaded but the content hash does not match.'
|
||||
'The repo may be outdated or download may be incomplete. '
|
||||
'Otherwise you can create an issue for it.'.format(self.name + '.csv.gz'))
|
||||
# extract file to directory `self.name` under `self.raw_dir`
|
||||
self._extract_gz(gz_file_path, self.raw_path)
|
||||
|
||||
The above code will extract the file into directory ``self.name`` under
|
||||
``self.raw_dir``. If the class inherits from :class:`dgl.data.DGLBuiltinDataset`
|
||||
to handle zip file, it will extract the file into directory ``self.name``
|
||||
as well.
|
||||
|
||||
Optionally, one can check SHA-1 string of the downloaded file as the
|
||||
example above does, in case the author changed the file in the remote
|
||||
server some day.
|
||||
@@ -0,0 +1,575 @@
|
||||
.. _guide-data-pipeline-loadcsv:
|
||||
|
||||
4.6 Loading data from CSV files
|
||||
----------------------------------------------
|
||||
|
||||
Comma Separated Value (CSV) is a widely used data storage format. DGL provides
|
||||
:class:`~dgl.data.CSVDataset` for loading and parsing graph data stored in
|
||||
CSV format.
|
||||
|
||||
To create a ``CSVDataset`` object:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
ds = dgl.data.CSVDataset('/path/to/dataset')
|
||||
|
||||
The returned ``ds`` object is a standard :class:`~dgl.data.DGLDataset`. For
|
||||
example, one can get graph samples using ``__getitem__`` as well as node/edge
|
||||
features using ``ndata``/``edata``.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# A demonstration of how to use the loaded dataset. The feature names
|
||||
# may vary depending on the CSV contents.
|
||||
g = ds[0] # get the graph
|
||||
label = g.ndata['label']
|
||||
feat = g.ndata['feat']
|
||||
|
||||
Data folder structure
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code::
|
||||
|
||||
/path/to/dataset/
|
||||
|-- meta.yaml # metadata of the dataset
|
||||
|-- edges_0.csv # edge data including src_id, dst_id, feature, label and so on
|
||||
|-- ... # you can have as many CSVs for edge data as you want
|
||||
|-- nodes_0.csv # node data including node_id, feature, label and so on
|
||||
|-- ... # you can have as many CSVs for node data as you want
|
||||
|-- graphs.csv # graph-level features
|
||||
|
||||
Node/edge/graph-level data are stored in CSV files. ``meta.yaml`` is a metadata file specifying
|
||||
where to read nodes/edges/graphs data and how to parse them to construct the dataset
|
||||
object. A minimal data folder contains one ``meta.yaml`` and two CSVs, one for node data and one
|
||||
for edge data, in which case the dataset contains only a single graph with no graph-level data.
|
||||
|
||||
Dataset of a single feature-less graph
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When the dataset contains only one graph with no node or edge features, there need only three
|
||||
files in the data folder: ``meta.yaml``, one CSV for node IDs and one CSV for edges:
|
||||
|
||||
.. code::
|
||||
|
||||
./mini_featureless_dataset/
|
||||
|-- meta.yaml
|
||||
|-- nodes.csv
|
||||
|-- edges.csv
|
||||
|
||||
``meta.yaml`` contains the following information:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
dataset_name: mini_featureless_dataset
|
||||
edge_data:
|
||||
- file_name: edges.csv
|
||||
node_data:
|
||||
- file_name: nodes.csv
|
||||
|
||||
``nodes.csv`` lists the node IDs under the ``node_id`` field:
|
||||
|
||||
.. code::
|
||||
|
||||
node_id
|
||||
0
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
|
||||
``edges.csv`` lists all the edges in two columns (``src_id`` and ``dst_id``) specifying the
|
||||
source and destination node ID of each edge:
|
||||
|
||||
.. code::
|
||||
|
||||
src_id,dst_id
|
||||
4,4
|
||||
4,1
|
||||
3,0
|
||||
4,1
|
||||
4,0
|
||||
1,2
|
||||
1,3
|
||||
3,3
|
||||
1,1
|
||||
4,1
|
||||
|
||||
After loaded, the dataset has one graph without any features:
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> dataset = dgl.data.CSVDataset('./mini_featureless_dataset')
|
||||
>>> g = dataset[0] # only one graph
|
||||
>>> print(g)
|
||||
Graph(num_nodes=5, num_edges=10,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
.. note::
|
||||
Non-integer node IDs are allowed. When constructing the graph, ``CSVDataset`` will
|
||||
map each raw ID to an integer ID starting from zero.
|
||||
If the node IDs are already distinct integers from 0 to ``num_nodes-1``, no mapping
|
||||
is applied.
|
||||
|
||||
.. note::
|
||||
Edges are always directed. To have both directions, add reversed edges in the edge
|
||||
CSV file or use :class:`~dgl.transforms.AddReverse` to transform the loaded graph.
|
||||
|
||||
|
||||
A graph without any feature is often of less interest. In the next example, we will show
|
||||
how to load and parse node or edge features.
|
||||
|
||||
Dataset of a single graph with features and labels
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When the dataset contains a single graph with node or edge features and labels, there still
|
||||
need only three files in the data folder: ``meta.yaml``, one CSV for node IDs and one CSV
|
||||
for edges:
|
||||
|
||||
.. code::
|
||||
|
||||
./mini_feature_dataset/
|
||||
|-- meta.yaml
|
||||
|-- nodes.csv
|
||||
|-- edges.csv
|
||||
|
||||
``meta.yaml``:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
dataset_name: mini_feature_dataset
|
||||
edge_data:
|
||||
- file_name: edges.csv
|
||||
node_data:
|
||||
- file_name: nodes.csv
|
||||
|
||||
``edges.csv`` with five synthetic edge data (``label``, ``train_mask``, ``val_mask``, ``test_mask``, ``feat``):
|
||||
|
||||
.. code::
|
||||
|
||||
src_id,dst_id,label,train_mask,val_mask,test_mask,feat
|
||||
4,0,2,False,True,True,"0.5477868606453535, 0.4470617033458436, 0.936706701616337"
|
||||
4,0,0,False,False,True,"0.9794634290792008, 0.23682038840665198, 0.049629338970987646"
|
||||
0,3,1,True,True,True,"0.8586722047523594, 0.5746912787380253, 0.6462162561249654"
|
||||
0,1,2,True,False,False,"0.2730008213674695, 0.5937484188166621, 0.765544096939567"
|
||||
0,2,1,True,True,True,"0.45441619816038514, 0.1681403185591509, 0.9952376085297715"
|
||||
0,0,0,False,False,False,"0.4197669213305396, 0.849983324532477, 0.16974127573016262"
|
||||
2,2,1,False,True,True,"0.5495035052928215, 0.21394654203489705, 0.7174910641836348"
|
||||
1,0,2,False,True,False,"0.008790817766266334, 0.4216530595907526, 0.529195480661293"
|
||||
3,0,0,True,True,True,"0.6598715708878852, 0.1932390907048961, 0.9774471538377553"
|
||||
4,0,1,False,False,False,"0.16846068931179736, 0.41516080644186737, 0.002158116134429955"
|
||||
|
||||
|
||||
``nodes.csv`` with five synthetic node data (``label``, ``train_mask``, ``val_mask``, ``test_mask``, ``feat``):
|
||||
|
||||
.. code::
|
||||
|
||||
node_id,label,train_mask,val_mask,test_mask,feat
|
||||
0,1,False,True,True,"0.07816474278491703, 0.9137336384979067, 0.4654086994009452"
|
||||
1,1,True,True,True,"0.05354099924658973, 0.8753101998792645, 0.33929432608774135"
|
||||
2,1,True,False,True,"0.33234211884156384, 0.9370522452510665, 0.6694943496824788"
|
||||
3,0,False,True,False,"0.9784264442230887, 0.22131880861864428, 0.3161154827254189"
|
||||
4,1,True,True,False,"0.23142237259162102, 0.8715767748481147, 0.19117861103555467"
|
||||
|
||||
After loaded, the dataset has one graph. Node/edge features are stored in ``ndata`` and ``edata``
|
||||
with the same column names. The example demonstrates how to specify a vector-shaped feature
|
||||
using comma-separated list enclosed by double quotes ``"..."``.
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> dataset = dgl.data.CSVDataset('./mini_feature_dataset')
|
||||
>>> g = dataset[0] # only one graph
|
||||
>>> print(g)
|
||||
Graph(num_nodes=5, num_edges=10,
|
||||
ndata_schemes={'label': Scheme(shape=(), dtype=torch.int64), 'train_mask': Scheme(shape=(), dtype=torch.bool), 'val_mask': Scheme(shape=(), dtype=torch.bool), 'test_mask': Scheme(shape=(), dtype=torch.bool), 'feat': Scheme(shape=(3,), dtype=torch.float64)}
|
||||
edata_schemes={'label': Scheme(shape=(), dtype=torch.int64), 'train_mask': Scheme(shape=(), dtype=torch.bool), 'val_mask': Scheme(shape=(), dtype=torch.bool), 'test_mask': Scheme(shape=(), dtype=torch.bool), 'feat': Scheme(shape=(3,), dtype=torch.float64)})
|
||||
|
||||
.. note::
|
||||
By default, ``CSVDatatset`` assumes all feature data to be numerical values (e.g., int, float, bool or
|
||||
list) and missing values are not allowed. Users could provide custom data parser for these cases.
|
||||
See `Custom Data Parser`_ for more details.
|
||||
|
||||
Dataset of a single heterogeneous graph
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
One can specify multiple node and edge CSV files (each for one type) to represent a heterogeneous graph.
|
||||
Here is an example data with two node types and two edge types:
|
||||
|
||||
.. code::
|
||||
|
||||
./mini_hetero_dataset/
|
||||
|-- meta.yaml
|
||||
|-- nodes_0.csv
|
||||
|-- nodes_1.csv
|
||||
|-- edges_0.csv
|
||||
|-- edges_1.csv
|
||||
|
||||
The ``meta.yaml`` specifies the node type name (using ``ntype``) and edge type name (using ``etype``)
|
||||
of each CSV file. The edge type name is a string triplet containing the source node type name, relation
|
||||
name and the destination node type name.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
dataset_name: mini_hetero_dataset
|
||||
edge_data:
|
||||
- file_name: edges_0.csv
|
||||
etype: [user, follow, user]
|
||||
- file_name: edges_1.csv
|
||||
etype: [user, like, item]
|
||||
node_data:
|
||||
- file_name: nodes_0.csv
|
||||
ntype: user
|
||||
- file_name: nodes_1.csv
|
||||
ntype: item
|
||||
|
||||
The node and edge CSV files follow the same format as in homogeneous graphs. Here are some synthetic
|
||||
data for demonstration purposes:
|
||||
|
||||
``edges_0.csv`` and ``edges_1.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
src_id,dst_id,label,feat
|
||||
4,4,1,"0.736833152378035,0.10522806046048205,0.9418796835016118"
|
||||
3,4,2,"0.5749339182767451,0.20181320245665535,0.490938012147181"
|
||||
1,4,2,"0.7697294432580938,0.49397782380750765,0.10864079337442234"
|
||||
0,4,0,"0.1364240150959487,0.1393107840629273,0.7901988878812207"
|
||||
2,3,1,"0.42988138237505735,0.18389137408509248,0.18431292077750894"
|
||||
0,4,2,"0.8613368738351794,0.67985810014162,0.6580438064356824"
|
||||
2,4,1,"0.6594951663841697,0.26499036865016423,0.7891429392727503"
|
||||
4,1,0,"0.36649684241348557,0.9511783938523962,0.8494919263589972"
|
||||
1,1,2,"0.698592283371875,0.038622249776255946,0.5563827995742111"
|
||||
0,4,1,"0.5227112950269823,0.3148264185956532,0.47562693094002173"
|
||||
|
||||
``nodes_0.csv`` and ``nodes_1.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
node_id,label,feat
|
||||
0,2,"0.5400687466285844,0.7588441197954202,0.4268254673041745"
|
||||
1,1,"0.08680051341900807,0.11446843700743892,0.7196969604886617"
|
||||
2,2,"0.8964389655603473,0.23368113896545695,0.8813472954005022"
|
||||
3,1,"0.5454703921677284,0.7819383771535038,0.3027939452162367"
|
||||
4,1,"0.5365210052235699,0.8975240205792763,0.7613943085507672"
|
||||
|
||||
After loaded, the dataset has one heterograph with features and labels:
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> dataset = dgl.data.CSVDataset('./mini_hetero_dataset')
|
||||
>>> g = dataset[0] # only one graph
|
||||
>>> print(g)
|
||||
Graph(num_nodes={'item': 5, 'user': 5},
|
||||
num_edges={('user', 'follow', 'user'): 10, ('user', 'like', 'item'): 10},
|
||||
metagraph=[('user', 'user', 'follow'), ('user', 'item', 'like')])
|
||||
>>> g.nodes['user'].data
|
||||
{'label': tensor([2, 1, 2, 1, 1]), 'feat': tensor([[0.5401, 0.7588, 0.4268],
|
||||
[0.0868, 0.1145, 0.7197],
|
||||
[0.8964, 0.2337, 0.8813],
|
||||
[0.5455, 0.7819, 0.3028],
|
||||
[0.5365, 0.8975, 0.7614]], dtype=torch.float64)}
|
||||
>>> g.edges['like'].data
|
||||
{'label': tensor([1, 2, 2, 0, 1, 2, 1, 0, 2, 1]), 'feat': tensor([[0.7368, 0.1052, 0.9419],
|
||||
[0.5749, 0.2018, 0.4909],
|
||||
[0.7697, 0.4940, 0.1086],
|
||||
[0.1364, 0.1393, 0.7902],
|
||||
[0.4299, 0.1839, 0.1843],
|
||||
[0.8613, 0.6799, 0.6580],
|
||||
[0.6595, 0.2650, 0.7891],
|
||||
[0.3665, 0.9512, 0.8495],
|
||||
[0.6986, 0.0386, 0.5564],
|
||||
[0.5227, 0.3148, 0.4756]], dtype=torch.float64)}
|
||||
|
||||
Dataset of multiple graphs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When there are multiple graphs, one can include an additional CSV file for storing graph-level features.
|
||||
Here is an example:
|
||||
|
||||
.. code::
|
||||
|
||||
./mini_multi_dataset/
|
||||
|-- meta.yaml
|
||||
|-- nodes.csv
|
||||
|-- edges.csv
|
||||
|-- graphs.csv
|
||||
|
||||
Accordingly, the ``meta.yaml`` should include an extra ``graph_data`` key to tell which CSV file to
|
||||
load graph-level features from.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
dataset_name: mini_multi_dataset
|
||||
edge_data:
|
||||
- file_name: edges.csv
|
||||
node_data:
|
||||
- file_name: nodes.csv
|
||||
graph_data:
|
||||
file_name: graphs.csv
|
||||
|
||||
To distinguish nodes and edges of different graphs, the ``node.csv`` and ``edge.csv`` must contain
|
||||
an extra column ``graph_id``:
|
||||
|
||||
``edges.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
graph_id,src_id,dst_id,feat
|
||||
0,0,4,"0.39534097273254654,0.9422093637539785,0.634899790318452"
|
||||
0,3,0,"0.04486384200747007,0.6453746567017163,0.8757520744192612"
|
||||
0,3,2,"0.9397636966928355,0.6526403892728874,0.8643238446466464"
|
||||
0,1,1,"0.40559906615287566,0.9848072295736628,0.493888090726854"
|
||||
0,4,1,"0.253458867276219,0.9168191778828504,0.47224962583565544"
|
||||
0,0,1,"0.3219496197945605,0.3439899477636117,0.7051530741717352"
|
||||
0,2,1,"0.692873149428549,0.4770019763881086,0.21937428942781778"
|
||||
0,4,0,"0.620118223673067,0.08691420300562658,0.86573472329756"
|
||||
0,2,1,"0.00743445923710373,0.5251800239734318,0.054016385555202384"
|
||||
0,4,1,"0.6776417760682221,0.7291568018841328,0.4523600060547709"
|
||||
1,1,3,"0.6375445528248924,0.04878384701995819,0.4081642382536248"
|
||||
1,0,4,"0.776002616178397,0.8851294998284638,0.7321742043493028"
|
||||
1,1,0,"0.0928555079874982,0.6156748364694707,0.6985674921582508"
|
||||
1,0,2,"0.31328748118329997,0.8326121496142408,0.04133991340612775"
|
||||
1,1,0,"0.36786902637778773,0.39161865931662243,0.9971749359397111"
|
||||
1,1,1,"0.4647410679872376,0.8478810655406659,0.6746269314422184"
|
||||
1,0,2,"0.8117650553546695,0.7893727601272978,0.41527155506593394"
|
||||
1,1,3,"0.40707309111756307,0.2796588354307046,0.34846782265758314"
|
||||
1,1,0,"0.18626464175355095,0.3523777809254057,0.7863421810531344"
|
||||
1,3,0,"0.28357022069634585,0.13774964202156292,0.5913335505943637"
|
||||
|
||||
``nodes.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
graph_id,node_id,feat
|
||||
0,0,"0.5725330322207948,0.8451870383322376,0.44412796119211184"
|
||||
0,1,"0.6624186423087752,0.6118386331195641,0.7352138669985214"
|
||||
0,2,"0.7583372765843964,0.15218126307872892,0.6810484348765842"
|
||||
0,3,"0.14627522432017592,0.7457985352827006,0.1037097085190507"
|
||||
0,4,"0.49037522512771525,0.8778998699783784,0.0911194482288028"
|
||||
1,0,"0.11158102039672668,0.08543289788089736,0.6901745368284345"
|
||||
1,1,"0.28367647637469273,0.07502571020414439,0.01217200152200748"
|
||||
1,2,"0.2472495901894738,0.24285506608575758,0.6494437360242048"
|
||||
1,3,"0.5614197853127827,0.059172654879085296,0.4692371689047904"
|
||||
1,4,"0.17583413999295983,0.5191278830882644,0.8453123358491914"
|
||||
|
||||
The ``graphs.csv`` contains a ``graph_id`` column and arbitrary number of feature columns.
|
||||
The example dataset here has two graphs, each with a ``feat`` and a ``label`` graph-level
|
||||
data.
|
||||
|
||||
.. code::
|
||||
|
||||
graph_id,feat,label
|
||||
0,"0.7426272601929126,0.5197462471155317,0.8149104951283953",0
|
||||
1,"0.534822233529295,0.2863627767733977,0.1154897249106891",0
|
||||
|
||||
After loaded, the dataset has multiple homographs with features and labels:
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> dataset = dgl.data.CSVDataset('./mini_multi_dataset')
|
||||
>>> print(len(dataset))
|
||||
2
|
||||
>>> graph0, data0 = dataset[0]
|
||||
>>> print(graph0)
|
||||
Graph(num_nodes=5, num_edges=10,
|
||||
ndata_schemes={'feat': Scheme(shape=(3,), dtype=torch.float64)}
|
||||
edata_schemes={'feat': Scheme(shape=(3,), dtype=torch.float64)})
|
||||
>>> print(data0)
|
||||
{'feat': tensor([0.7426, 0.5197, 0.8149], dtype=torch.float64), 'label': tensor(0)}
|
||||
>>> graph1, data1 = dataset[1]
|
||||
>>> print(graph1)
|
||||
Graph(num_nodes=5, num_edges=10,
|
||||
ndata_schemes={'feat': Scheme(shape=(3,), dtype=torch.float64)}
|
||||
edata_schemes={'feat': Scheme(shape=(3,), dtype=torch.float64)})
|
||||
>>> print(data1)
|
||||
{'feat': tensor([0.5348, 0.2864, 0.1155], dtype=torch.float64), 'label': tensor(0)}
|
||||
|
||||
If there is a single feature column in ``graphs.csv``, ``data0`` will directly be a tensor for the feature.
|
||||
|
||||
|
||||
Custom Data Parser
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
By default, ``CSVDataset`` assumes that all the stored node-/edge-/graph- level data are numerical
|
||||
values. Users can provide custom ``DataParser`` to ``CSVDataset`` to handle more complex
|
||||
data type. A ``DataParser`` needs to implement the ``__call__`` method which takes in the
|
||||
:class:`pandas.DataFrame` object created from CSV file and should return a dictionary of
|
||||
parsed feature data. The parsed feature data will be saved to the ``ndata`` and ``edata`` of
|
||||
the corresponding ``DGLGraph`` object, and thus must be tensors or numpy arrays. Below shows an example
|
||||
``DataParser`` which converts string type labels to integers:
|
||||
|
||||
Given a dataset as follows,
|
||||
|
||||
.. code::
|
||||
|
||||
./customized_parser_dataset/
|
||||
|-- meta.yaml
|
||||
|-- nodes.csv
|
||||
|-- edges.csv
|
||||
|
||||
``meta.yaml``:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
dataset_name: customized_parser_dataset
|
||||
edge_data:
|
||||
- file_name: edges.csv
|
||||
node_data:
|
||||
- file_name: nodes.csv
|
||||
|
||||
``edges.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
src_id,dst_id,label
|
||||
4,0,positive
|
||||
4,0,negative
|
||||
0,3,positive
|
||||
0,1,positive
|
||||
0,2,negative
|
||||
0,0,positive
|
||||
2,2,negative
|
||||
1,0,positive
|
||||
3,0,negative
|
||||
4,0,positive
|
||||
|
||||
``nodes.csv``:
|
||||
|
||||
.. code::
|
||||
|
||||
node_id,label
|
||||
0,positive
|
||||
1,negative
|
||||
2,positive
|
||||
3,negative
|
||||
4,positive
|
||||
|
||||
To parse the string type labels, one can define a ``DataParser`` class as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
class MyDataParser:
|
||||
def __call__(self, df: pd.DataFrame):
|
||||
parsed = {}
|
||||
for header in df:
|
||||
if 'Unnamed' in header: # Handle Unnamed column
|
||||
print("Unnamed column is found. Ignored...")
|
||||
continue
|
||||
dt = df[header].to_numpy().squeeze()
|
||||
if header == 'label':
|
||||
dt = np.array([1 if e == 'positive' else 0 for e in dt])
|
||||
parsed[header] = dt
|
||||
return parsed
|
||||
|
||||
Create a ``CSVDataset`` using the defined ``DataParser``:
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> dataset = dgl.data.CSVDataset('./customized_parser_dataset',
|
||||
... ndata_parser=MyDataParser(),
|
||||
... edata_parser=MyDataParser())
|
||||
>>> print(dataset[0].ndata['label'])
|
||||
tensor([1, 0, 1, 0, 1])
|
||||
>>> print(dataset[0].edata['label'])
|
||||
tensor([1, 0, 1, 1, 0, 1, 0, 1, 0, 1])
|
||||
|
||||
.. note::
|
||||
|
||||
To specify different ``DataParser``\s for different node/edge types, pass a dictionary to
|
||||
``ndata_parser`` and ``edata_parser``, where the key is type name (a single string for
|
||||
node type; a string triplet for edge type) and the value is the ``DataParser`` to use.
|
||||
|
||||
|
||||
Full YAML Specification
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
``CSVDataset`` allows more flexible control over the loading and parsing process. For example, one
|
||||
can change the ID column names via ``meta.yaml``. The example below lists all the supported keys.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
version: 1.0.0
|
||||
dataset_name: some_complex_data
|
||||
separator: ',' # CSV separator symbol. Default: ','
|
||||
edge_data:
|
||||
- file_name: edges_0.csv
|
||||
etype: [user, follow, user]
|
||||
src_id_field: src_id # Column name for source node IDs. Default: src_id
|
||||
dst_id_field: dst_id # Column name for destination node IDs. Default: dst_id
|
||||
- file_name: edges_1.csv
|
||||
etype: [user, like, item]
|
||||
src_id_field: src_id
|
||||
dst_id_field: dst_id
|
||||
node_data:
|
||||
- file_name: nodes_0.csv
|
||||
ntype: user
|
||||
node_id_field: node_id # Column name for node IDs. Default: node_id
|
||||
- file_name: nodes_1.csv
|
||||
ntype: item
|
||||
node_id_field: node_id # Column name for node IDs. Default: node_id
|
||||
graph_data:
|
||||
file_name: graphs.csv
|
||||
graph_id_field: graph_id # Column name for graph IDs. Default: graph_id
|
||||
|
||||
Top-level
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
At the top level, only 6 keys are available:
|
||||
|
||||
- ``version``: Optional. String.
|
||||
It specifies which version of ``meta.yaml`` is used. More feature may be added in the future.
|
||||
- ``dataset_name``: Required. String.
|
||||
It specifies the dataset name.
|
||||
- ``separator``: Optional. String.
|
||||
It specifies how to parse data in CSV files. Default: ``','``.
|
||||
- ``edge_data``: Required. List of ``EdgeData``.
|
||||
Meta data for parsing edge CSV files.
|
||||
- ``node_data``: Required. List of ``NodeData``.
|
||||
Meta data for parsing node CSV files.
|
||||
- ``graph_data``: Optional. ``GraphData``.
|
||||
Meta data for parsing the graph CSV file.
|
||||
|
||||
``EdgeData``
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are 4 keys:
|
||||
|
||||
- ``file_name``: Required. String.
|
||||
The CSV file to load data from.
|
||||
- ``etype``: Optional. List of string.
|
||||
Edge type name in string triplet: [source node type, relation type, destination node type].
|
||||
- ``src_id_field``: Optional. String.
|
||||
Which column to read for source node IDs. Default: ``src_id``.
|
||||
- ``dst_id_field``: Optional. String.
|
||||
Which column to read for destination node IDs. Default: ``dst_id``.
|
||||
|
||||
``NodeData``
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are 3 keys:
|
||||
|
||||
- ``file_name``: Required. String.
|
||||
The CSV file to load data from.
|
||||
- ``ntype``: Optional. String.
|
||||
Node type name.
|
||||
- ``node_id_field``: Optional. String.
|
||||
Which column to read for node IDs. Default: ``node_id``.
|
||||
|
||||
``GraphData``
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are 2 keys:
|
||||
|
||||
- ``file_name``: Required. String.
|
||||
The CSV file to load data from.
|
||||
- ``graph_id_field``: Optional. String.
|
||||
Which column to read for graph IDs. Default: ``graph_id``.
|
||||
@@ -0,0 +1,79 @@
|
||||
.. _guide-data-pipeline-loadogb:
|
||||
|
||||
4.5 Loading OGB datasets using ``ogb`` package
|
||||
----------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline-loadogb>`
|
||||
|
||||
`Open Graph Benchmark (OGB) <https://ogb.stanford.edu/docs/home/>`__ is
|
||||
a collection of benchmark datasets. The official OGB package
|
||||
`ogb <https://github.com/snap-stanford/ogb>`__ provides APIs for
|
||||
downloading and processing OGB datasets into :class:`dgl.data.DGLGraph` objects. The section
|
||||
introduce their basic usage here.
|
||||
|
||||
First install ogb package using pip:
|
||||
|
||||
.. code::
|
||||
|
||||
pip install ogb
|
||||
|
||||
The following code shows how to load datasets for *Graph Property
|
||||
Prediction* tasks.
|
||||
|
||||
.. code::
|
||||
|
||||
# Load Graph Property Prediction datasets in OGB
|
||||
import dgl
|
||||
import torch
|
||||
from ogb.graphproppred import DglGraphPropPredDataset
|
||||
from dgl.dataloading import GraphDataLoader
|
||||
|
||||
|
||||
def _collate_fn(batch):
|
||||
# batch is a list of tuple (graph, label)
|
||||
graphs = [e[0] for e in batch]
|
||||
g = dgl.batch(graphs)
|
||||
labels = [e[1] for e in batch]
|
||||
labels = torch.stack(labels, 0)
|
||||
return g, labels
|
||||
|
||||
# load dataset
|
||||
dataset = DglGraphPropPredDataset(name='ogbg-molhiv')
|
||||
split_idx = dataset.get_idx_split()
|
||||
# dataloader
|
||||
train_loader = GraphDataLoader(dataset[split_idx["train"]], batch_size=32, shuffle=True, collate_fn=_collate_fn)
|
||||
valid_loader = GraphDataLoader(dataset[split_idx["valid"]], batch_size=32, shuffle=False, collate_fn=_collate_fn)
|
||||
test_loader = GraphDataLoader(dataset[split_idx["test"]], batch_size=32, shuffle=False, collate_fn=_collate_fn)
|
||||
|
||||
Loading *Node Property Prediction* datasets is similar, but note that
|
||||
there is only one graph object in this kind of dataset.
|
||||
|
||||
.. code::
|
||||
|
||||
# Load Node Property Prediction datasets in OGB
|
||||
from ogb.nodeproppred import DglNodePropPredDataset
|
||||
|
||||
dataset = DglNodePropPredDataset(name='ogbn-proteins')
|
||||
split_idx = dataset.get_idx_split()
|
||||
|
||||
# there is only one graph in Node Property Prediction datasets
|
||||
g, labels = dataset[0]
|
||||
# get split labels
|
||||
train_label = dataset.labels[split_idx['train']]
|
||||
valid_label = dataset.labels[split_idx['valid']]
|
||||
test_label = dataset.labels[split_idx['test']]
|
||||
|
||||
*Link Property Prediction* datasets also contain one graph per dataset.
|
||||
|
||||
.. code::
|
||||
|
||||
# Load Link Property Prediction datasets in OGB
|
||||
from ogb.linkproppred import DglLinkPropPredDataset
|
||||
|
||||
dataset = DglLinkPropPredDataset(name='ogbl-ppa')
|
||||
split_edge = dataset.get_edge_split()
|
||||
|
||||
graph = dataset[0]
|
||||
print(split_edge['train'].keys())
|
||||
print(split_edge['valid'].keys())
|
||||
print(split_edge['test'].keys())
|
||||
@@ -0,0 +1,326 @@
|
||||
.. _guide-data-pipeline-process:
|
||||
|
||||
4.3 Process data
|
||||
----------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline-process>`
|
||||
|
||||
One can implement the data processing code in function ``process()``, and it
|
||||
assumes that the raw data is located in ``self.raw_dir`` already. There
|
||||
are typically three types of tasks in machine learning on graphs: graph
|
||||
classification, node classification, and link prediction. This section will show
|
||||
how to process datasets related to these tasks.
|
||||
|
||||
The section focuses on the standard way to process graphs, features and masks.
|
||||
It will use builtin datasets as examples and skip the implementations
|
||||
for building graphs from files, but add links to the detailed
|
||||
implementations. Please refer to :ref:`guide-graph-external` to see a
|
||||
complete guide on how to build graphs from external sources.
|
||||
|
||||
Processing Graph Classification datasets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Graph classification datasets are almost the same as most datasets in
|
||||
typical machine learning tasks, where mini-batch training is used. So one can
|
||||
process the raw data to a list of :class:`dgl.DGLGraph` objects and a list of
|
||||
label tensors. In addition, if the raw data has been split into
|
||||
several files, one can add a parameter ``split`` to load specific part of
|
||||
the data.
|
||||
|
||||
Take :class:`~dgl.data.QM7bDataset` as example:
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLDataset
|
||||
|
||||
class QM7bDataset(DGLDataset):
|
||||
_url = 'http://deepchem.io.s3-website-us-west-1.amazonaws.com/' \
|
||||
'datasets/qm7b.mat'
|
||||
_sha1_str = '4102c744bb9d6fd7b40ac67a300e49cd87e28392'
|
||||
|
||||
def __init__(self, raw_dir=None, force_reload=False, verbose=False):
|
||||
super(QM7bDataset, self).__init__(name='qm7b',
|
||||
url=self._url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
mat_path = self.raw_path + '.mat'
|
||||
# process data to a list of graphs and a list of labels
|
||||
self.graphs, self.label = self._load_graph(mat_path)
|
||||
|
||||
def __getitem__(self, idx):
|
||||
""" Get graph and label by index
|
||||
|
||||
Parameters
|
||||
----------
|
||||
idx : int
|
||||
Item index
|
||||
|
||||
Returns
|
||||
-------
|
||||
(dgl.DGLGraph, Tensor)
|
||||
"""
|
||||
return self.graphs[idx], self.label[idx]
|
||||
|
||||
def __len__(self):
|
||||
"""Number of graphs in the dataset"""
|
||||
return len(self.graphs)
|
||||
|
||||
|
||||
In ``process()``, the raw data is processed to a list of graphs and a
|
||||
list of labels. One must implement ``__getitem__(idx)`` and ``__len__()``
|
||||
for iteration. DGL recommends making ``__getitem__(idx)`` return a
|
||||
tuple ``(graph, label)`` as above. Please check the `QM7bDataset source
|
||||
code <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/qm7b.html#QM7bDataset>`__
|
||||
for details of ``self._load_graph()`` and ``__getitem__``.
|
||||
|
||||
One can also add properties to the class to indicate some useful
|
||||
information of the dataset. In :class:`~dgl.data.QM7bDataset`, one can add a property
|
||||
``num_tasks`` to indicate the total number of prediction tasks in this
|
||||
multi-task dataset:
|
||||
|
||||
.. code::
|
||||
|
||||
@property
|
||||
def num_tasks(self):
|
||||
"""Number of labels for each graph, i.e. number of prediction tasks."""
|
||||
return 14
|
||||
|
||||
After all these coding, one can finally use :class:`~dgl.data.QM7bDataset` as
|
||||
follows:
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl
|
||||
import torch
|
||||
|
||||
from dgl.dataloading import GraphDataLoader
|
||||
|
||||
# load data
|
||||
dataset = QM7bDataset()
|
||||
num_tasks = dataset.num_tasks
|
||||
|
||||
# create dataloaders
|
||||
dataloader = GraphDataLoader(dataset, batch_size=1, shuffle=True)
|
||||
|
||||
# training
|
||||
for epoch in range(100):
|
||||
for g, labels in dataloader:
|
||||
# your training code here
|
||||
pass
|
||||
|
||||
A complete guide for training graph classification models can be found
|
||||
in :ref:`guide-training-graph-classification`.
|
||||
|
||||
For more examples of graph classification datasets, please refer to DGL's builtin graph classification
|
||||
datasets:
|
||||
|
||||
* :ref:`gindataset`
|
||||
|
||||
* :ref:`minigcdataset`
|
||||
|
||||
* :ref:`qm7bdata`
|
||||
|
||||
* :ref:`tudata`
|
||||
|
||||
Processing Node Classification datasets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Different from graph classification, node classification is typically on
|
||||
a single graph. As such, splits of the dataset are on the nodes of the
|
||||
graph. DGL recommends using node masks to specify the splits. The section uses
|
||||
builtin dataset `CitationGraphDataset <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/citation_graph.html#CitationGraphDataset>`__ as an example:
|
||||
|
||||
In addition, DGL recommends re-arrange the nodes and edges so that nodes
|
||||
near to each other have IDs in a close range. The procedure could improve
|
||||
the locality to access a node's neighbors, which may benefit follow-up
|
||||
computation and analysis conducted on the graph. DGL provides an API called
|
||||
:func:`dgl.reorder_graph` for this purpose. Please refer to ``process()``
|
||||
part in below example for more details.
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLBuiltinDataset
|
||||
from dgl.data.utils import _get_dgl_url
|
||||
|
||||
class CitationGraphDataset(DGLBuiltinDataset):
|
||||
_urls = {
|
||||
'cora_v2' : 'dataset/cora_v2.zip',
|
||||
'citeseer' : 'dataset/citeseer.zip',
|
||||
'pubmed' : 'dataset/pubmed.zip',
|
||||
}
|
||||
|
||||
def __init__(self, name, raw_dir=None, force_reload=False, verbose=True):
|
||||
assert name.lower() in ['cora', 'citeseer', 'pubmed']
|
||||
if name.lower() == 'cora':
|
||||
name = 'cora_v2'
|
||||
url = _get_dgl_url(self._urls[name])
|
||||
super(CitationGraphDataset, self).__init__(name,
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
# Skip some processing code
|
||||
# === data processing skipped ===
|
||||
|
||||
# build graph
|
||||
g = dgl.graph(graph)
|
||||
# splitting masks
|
||||
g.ndata['train_mask'] = train_mask
|
||||
g.ndata['val_mask'] = val_mask
|
||||
g.ndata['test_mask'] = test_mask
|
||||
# node labels
|
||||
g.ndata['label'] = torch.tensor(labels)
|
||||
# node features
|
||||
g.ndata['feat'] = torch.tensor(_preprocess_features(features),
|
||||
dtype=F.data_type_dict['float32'])
|
||||
self._num_tasks = onehot_labels.shape[1]
|
||||
self._labels = labels
|
||||
# reorder graph to obtain better locality.
|
||||
self._g = dgl.reorder_graph(g)
|
||||
|
||||
def __getitem__(self, idx):
|
||||
assert idx == 0, "This dataset has only one graph"
|
||||
return self._g
|
||||
|
||||
def __len__(self):
|
||||
return 1
|
||||
|
||||
For brevity, this section skips some code in ``process()`` to highlight the key
|
||||
part for processing node classification dataset: splitting masks. Node
|
||||
features and node labels are stored in ``g.ndata``. For detailed
|
||||
implementation, please refer to `CitationGraphDataset source
|
||||
code <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/citation_graph.html#CitationGraphDataset>`__.
|
||||
|
||||
Note that the implementations of ``__getitem__(idx)`` and
|
||||
``__len__()`` are changed as well, since there is often only one graph
|
||||
for node classification tasks. The masks are ``bool tensors`` in PyTorch
|
||||
and TensorFlow, and ``float tensors`` in MXNet.
|
||||
|
||||
The section uses a subclass of ``CitationGraphDataset``, :class:`dgl.data.CiteseerGraphDataset`,
|
||||
to show the usage of it:
|
||||
|
||||
.. code::
|
||||
|
||||
# load data
|
||||
dataset = CiteseerGraphDataset(raw_dir='')
|
||||
graph = dataset[0]
|
||||
|
||||
# get split masks
|
||||
train_mask = graph.ndata['train_mask']
|
||||
val_mask = graph.ndata['val_mask']
|
||||
test_mask = graph.ndata['test_mask']
|
||||
|
||||
# get node features
|
||||
feats = graph.ndata['feat']
|
||||
|
||||
# get labels
|
||||
labels = graph.ndata['label']
|
||||
|
||||
A complete guide for training node classification models can be found in
|
||||
:ref:`guide-training-node-classification`.
|
||||
|
||||
For more examples of node classification datasets, please refer to DGL's
|
||||
builtin datasets:
|
||||
|
||||
* :ref:`citationdata`
|
||||
|
||||
* :ref:`corafulldata`
|
||||
|
||||
* :ref:`amazoncobuydata`
|
||||
|
||||
* :ref:`coauthordata`
|
||||
|
||||
* :ref:`karateclubdata`
|
||||
|
||||
* :ref:`ppidata`
|
||||
|
||||
* :ref:`redditdata`
|
||||
|
||||
* :ref:`sbmdata`
|
||||
|
||||
* :ref:`sstdata`
|
||||
|
||||
* :ref:`rdfdata`
|
||||
|
||||
Processing dataset for Link Prediction datasets
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The processing of link prediction datasets is similar to that for node
|
||||
classification’s, there is often one graph in the dataset.
|
||||
|
||||
The section uses builtin dataset
|
||||
`KnowledgeGraphDataset <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/knowledge_graph.html#KnowledgeGraphDataset>`__
|
||||
as an example, and still skips the detailed data processing code to
|
||||
highlight the key part for processing link prediction datasets:
|
||||
|
||||
.. code::
|
||||
|
||||
# Example for creating Link Prediction datasets
|
||||
class KnowledgeGraphDataset(DGLBuiltinDataset):
|
||||
def __init__(self, name, reverse=True, raw_dir=None, force_reload=False, verbose=True):
|
||||
self._name = name
|
||||
self.reverse = reverse
|
||||
url = _get_dgl_url('dataset/') + '{}.tgz'.format(name)
|
||||
super(KnowledgeGraphDataset, self).__init__(name,
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
# Skip some processing code
|
||||
# === data processing skipped ===
|
||||
|
||||
# splitting mask
|
||||
g.edata['train_mask'] = train_mask
|
||||
g.edata['val_mask'] = val_mask
|
||||
g.edata['test_mask'] = test_mask
|
||||
# edge type
|
||||
g.edata['etype'] = etype
|
||||
# node type
|
||||
g.ndata['ntype'] = ntype
|
||||
self._g = g
|
||||
|
||||
def __getitem__(self, idx):
|
||||
assert idx == 0, "This dataset has only one graph"
|
||||
return self._g
|
||||
|
||||
def __len__(self):
|
||||
return 1
|
||||
|
||||
As shown in the code, it adds splitting masks into ``edata`` field of the
|
||||
graph. Check `KnowledgeGraphDataset source
|
||||
code <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/knowledge_graph.html#KnowledgeGraphDataset>`__
|
||||
to see the complete code. The following code uses a subclass of ``KnowledgeGraphDataset``,
|
||||
:class:`dgl.data.FB15k237Dataset`, to show the usage of it:
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import FB15k237Dataset
|
||||
|
||||
# load data
|
||||
dataset = FB15k237Dataset()
|
||||
graph = dataset[0]
|
||||
|
||||
# get training mask
|
||||
train_mask = graph.edata['train_mask']
|
||||
train_idx = torch.nonzero(train_mask, as_tuple=False).squeeze()
|
||||
src, dst = graph.edges(train_idx)
|
||||
# get edge types in training set
|
||||
rel = graph.edata['etype'][train_idx]
|
||||
|
||||
|
||||
A complete guide for training link prediction models can be found in
|
||||
:ref:`guide-training-link-prediction`.
|
||||
|
||||
For more examples of link prediction datasets, please refer to DGL's
|
||||
builtin datasets:
|
||||
|
||||
* :ref:`kgdata`
|
||||
|
||||
* :ref:`bitcoinotcdata`
|
||||
@@ -0,0 +1,49 @@
|
||||
.. _guide-data-pipeline-savenload:
|
||||
|
||||
4.4 Save and load data
|
||||
----------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline-savenload>`
|
||||
|
||||
DGL recommends implementing saving and loading functions to cache the
|
||||
processed data in local disk. This saves a lot of data processing time
|
||||
in most cases. DGL provides four functions to make things simple:
|
||||
|
||||
- :func:`dgl.save_graphs` and :func:`dgl.load_graphs`: save/load DGLGraph objects and labels to/from local disk.
|
||||
- :func:`dgl.data.utils.save_info` and :func:`dgl.data.utils.load_info`: save/load useful information of the dataset (python ``dict`` object) to/from local disk.
|
||||
|
||||
The following example shows how to save and load a list of graphs and
|
||||
dataset information.
|
||||
|
||||
.. code::
|
||||
|
||||
import os
|
||||
from dgl import save_graphs, load_graphs
|
||||
from dgl.data.utils import makedirs, save_info, load_info
|
||||
|
||||
def save(self):
|
||||
# save graphs and labels
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
save_graphs(graph_path, self.graphs, {'labels': self.labels})
|
||||
# save other information in python dict
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
save_info(info_path, {'num_classes': self.num_classes})
|
||||
|
||||
def load(self):
|
||||
# load processed data from directory `self.save_path`
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
self.graphs, label_dict = load_graphs(graph_path)
|
||||
self.labels = label_dict['labels']
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
self.num_classes = load_info(info_path)['num_classes']
|
||||
|
||||
def has_cache(self):
|
||||
# check whether there are processed data in `self.save_path`
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
return os.path.exists(graph_path) and os.path.exists(info_path)
|
||||
|
||||
Note that there are cases not suitable to save processed data. For
|
||||
example, in the builtin dataset :class:`~dgl.data.GDELTDataset`,
|
||||
the processed data is quite large, so it’s more effective to process
|
||||
each data example in ``__getitem__(idx)``.
|
||||
@@ -0,0 +1,38 @@
|
||||
.. _guide-data-pipeline:
|
||||
|
||||
Chapter 4: Graph Data Pipeline
|
||||
==============================
|
||||
|
||||
:ref:`(中文版) <guide_cn-data-pipeline>`
|
||||
|
||||
DGL implements many commonly used graph datasets in :ref:`apidata`. They
|
||||
follow a standard pipeline defined in class :class:`dgl.data.DGLDataset`. DGL highly
|
||||
recommends processing graph data into a :class:`dgl.data.DGLDataset` subclass, as the
|
||||
pipeline provides simple and clean solution for loading, processing and
|
||||
saving graph data.
|
||||
|
||||
Roadmap
|
||||
-------
|
||||
|
||||
This chapter introduces how to create a custom DGL-Dataset.
|
||||
The following sections explain how the pipeline works, and
|
||||
shows how to implement each component of it.
|
||||
|
||||
* :ref:`guide-data-pipeline-dataset`
|
||||
* :ref:`guide-data-pipeline-download`
|
||||
* :ref:`guide-data-pipeline-process`
|
||||
* :ref:`guide-data-pipeline-savenload`
|
||||
* :ref:`guide-data-pipeline-loadogb`
|
||||
* :ref:`guide-data-pipeline-loadcsv`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
data-dataset
|
||||
data-download
|
||||
data-process
|
||||
data-savenload
|
||||
data-loadogb
|
||||
data-loadcsv
|
||||
@@ -0,0 +1,321 @@
|
||||
.. _guide-distributed-apis:
|
||||
|
||||
7.3 Programming APIs
|
||||
-----------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-distributed-apis>`
|
||||
|
||||
This section covers the core python components commonly used in a training script. DGL
|
||||
provides three distributed data structures and various APIs for initialization,
|
||||
distributed sampling and workload split.
|
||||
|
||||
* :class:`~dgl.distributed.DistGraph` for accessing structure and feature of a distributedly
|
||||
stored graph.
|
||||
* :class:`~dgl.distributed.DistTensor` for accessing node/edge feature tensor that
|
||||
is partitioned across machines.
|
||||
* :class:`~dgl.distributed.DistEmbedding` for accessing learnable node/edge embedding
|
||||
tensor that is partitioned across machines.
|
||||
|
||||
Initialization of the DGL distributed module
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
:func:`dgl.distributed.initialize` initializes the distributed module. If invoked
|
||||
by a trainer, this API creates sampler processes and builds connections with graph
|
||||
servers; if invoked by graph server, this API starts a service loop to listen to
|
||||
trainer/sampler requests. The API *must* be called before
|
||||
:func:`torch.distributed.init_process_group` and any other ``dgl.distributed`` APIs
|
||||
as shown in the order below:
|
||||
|
||||
.. code:: python
|
||||
|
||||
dgl.distributed.initialize('ip_config.txt')
|
||||
th.distributed.init_process_group(backend='gloo')
|
||||
|
||||
.. note::
|
||||
|
||||
If the training script contains user-defined functions (UDFs) that have to be invoked on
|
||||
the servers (see the section of DistTensor and DistEmbedding for more details), these UDFs have to
|
||||
be declared before :func:`~dgl.distributed.initialize`.
|
||||
|
||||
Distributed graph
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` is a Python class to access the graph
|
||||
structure and node/edge features in a cluster of machines. Each machine is
|
||||
responsible for one and only one partition. It loads the partition data (the
|
||||
graph structure and the node data and edge data in the partition) and makes it
|
||||
accessible to all trainers in the cluster. :class:`~dgl.distributed.DistGraph`
|
||||
provides a small subset of :class:`~dgl.DGLGraph` APIs for data access.
|
||||
|
||||
Distributed mode vs. standalone mode
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` can run in two modes: *distributed mode* and *standalone mode*.
|
||||
When a user executes a training script in a Python command line or Jupyter Notebook, it runs in
|
||||
a standalone mode. That is, it runs all computation in a single process and does not communicate
|
||||
with any other processes. Thus, the standalone mode requires the input graph to have only one partition.
|
||||
This mode is mainly used for development and testing (e.g., develop and run the code in Jupyter Notebook).
|
||||
When a user executes a training script with a launch script (see the section of launch script),
|
||||
:class:`~dgl.distributed.DistGraph` runs in the distributed mode. The launch tool starts servers
|
||||
(node/edge feature access and graph sampling) behind the scene and loads the partition data in
|
||||
each machine automatically. :class:`~dgl.distributed.DistGraph` connects with the servers in the cluster
|
||||
of machines and access them through the network.
|
||||
|
||||
DistGraph creation
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
In the distributed mode, the creation of :class:`~dgl.distributed.DistGraph`
|
||||
requires the graph name given during graph partitioning. The graph name
|
||||
identifies the graph loaded in the cluster.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
g = dgl.distributed.DistGraph('graph_name')
|
||||
|
||||
When running in the standalone mode, it loads the graph data in the local
|
||||
machine. Therefore, users need to provide the partition configuration file,
|
||||
which contains all information about the input graph.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
g = dgl.distributed.DistGraph('graph_name', part_config='data/graph_name.json')
|
||||
|
||||
.. note::
|
||||
|
||||
DGL only allows one single ``DistGraph`` object. The behavior
|
||||
of destroying a DistGraph and creating a new one is undefined.
|
||||
|
||||
Accessing graph structure
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` provides a set of APIs to
|
||||
access the graph structure. Currently, most APIs provide graph information,
|
||||
such as the number of nodes and edges. The main use case of DistGraph is to run
|
||||
sampling APIs to support mini-batch training (see `Distributed sampling`_).
|
||||
|
||||
.. code:: python
|
||||
|
||||
print(g.num_nodes())
|
||||
|
||||
Access node/edge data
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Like :class:`~dgl.DGLGraph`, :class:`~dgl.distributed.DistGraph` provides ``ndata`` and ``edata``
|
||||
to access data in nodes and edges.
|
||||
The difference is that ``ndata``/``edata`` in :class:`~dgl.distributed.DistGraph` returns
|
||||
:class:`~dgl.distributed.DistTensor`, instead of the tensor of the underlying framework.
|
||||
Users can also assign a new :class:`~dgl.distributed.DistTensor` to
|
||||
:class:`~dgl.distributed.DistGraph` as node data or edge data.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g.ndata['train_mask'] # <dgl.distributed.dist_graph.DistTensor at 0x7fec820937b8>
|
||||
g.ndata['train_mask'][0] # tensor([1], dtype=torch.uint8)
|
||||
|
||||
Distributed Tensor
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
As mentioned earlier, DGL shards node/edge features and stores them in a cluster of machines.
|
||||
DGL provides distributed tensors with a tensor-like interface to access the partitioned
|
||||
node/edge features in the cluster. In the distributed setting, DGL only supports dense node/edge
|
||||
features.
|
||||
|
||||
:class:`~dgl.distributed.DistTensor` manages the dense tensors partitioned and stored in
|
||||
multiple machines. Right now, a distributed tensor has to be associated with nodes or edges
|
||||
of a graph. In other words, the number of rows in a DistTensor has to be the same as the number
|
||||
of nodes or the number of edges in a graph. The following code creates a distributed tensor.
|
||||
In addition to the shape and dtype for the tensor, a user can also provide a unique tensor name.
|
||||
This name is useful if a user wants to reference a persistent distributed tensor (the one exists
|
||||
in the cluster even if the :class:`~dgl.distributed.DistTensor` object disappears).
|
||||
|
||||
.. code:: python
|
||||
|
||||
tensor = dgl.distributed.DistTensor((g.num_nodes(), 10), th.float32, name='test')
|
||||
|
||||
.. note::
|
||||
|
||||
:class:`~dgl.distributed.DistTensor` creation is a synchronized operation. All trainers
|
||||
have to invoke the creation and the creation succeeds only when all trainers call it.
|
||||
|
||||
A user can add a :class:`~dgl.distributed.DistTensor` to a :class:`~dgl.distributed.DistGraph`
|
||||
object as one of the node data or edge data.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g.ndata['feat'] = tensor
|
||||
|
||||
.. note::
|
||||
|
||||
The node data name and the tensor name do not have to be the same. The former identifies
|
||||
node data from :class:`~dgl.distributed.DistGraph` (in the trainer process) while the latter
|
||||
identifies a distributed tensor in DGL servers.
|
||||
|
||||
:class:`~dgl.distributed.DistTensor` has the same APIs as
|
||||
regular tensors to access its metadata, such as the shape and dtype. It also
|
||||
supports indexed reads and writes but does not support
|
||||
computation operators, such as sum and mean.
|
||||
|
||||
.. code:: python
|
||||
|
||||
data = g.ndata['feat'][[1, 2, 3]]
|
||||
print(data)
|
||||
g.ndata['feat'][[3, 4, 5]] = data
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
Currently, DGL does not provide protection for concurrent writes from
|
||||
multiple trainers when a machine runs multiple servers. This may result in
|
||||
data corruption. One way to avoid concurrent writes to the same row of data
|
||||
is to run one server process on a machine.
|
||||
|
||||
Distributed DistEmbedding
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL provides :class:`~dgl.distributed.DistEmbedding` to support transductive models that require
|
||||
node embeddings. Creating distributed embeddings is very similar to creating distributed tensors.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def initializer(shape, dtype):
|
||||
arr = th.zeros(shape, dtype=dtype)
|
||||
arr.uniform_(-1, 1)
|
||||
return arr
|
||||
emb = dgl.distributed.DistEmbedding(g.num_nodes(), 10, init_func=initializer)
|
||||
|
||||
Internally, distributed embeddings are built on top of distributed tensors,
|
||||
and, thus, has very similar behaviors to distributed tensors. For example, when
|
||||
embeddings are created, they are sharded and stored across all machines in the
|
||||
cluster. It can be uniquely identified by a name.
|
||||
|
||||
.. note::
|
||||
|
||||
The initializer function is invoked in the server process. Therefore, it has to be
|
||||
declared before :class:`dgl.distributed.initialize`.
|
||||
|
||||
Because the embeddings are part of the model, a user has to attach them to an
|
||||
optimizer for mini-batch training. Currently, DGL provides a sparse Adagrad
|
||||
optimizer :class:`~dgl.distributed.SparseAdagrad` (DGL will add more optimizers
|
||||
for sparse embeddings later). Users need to collect all distributed embeddings
|
||||
from a model and pass them to the sparse optimizer. If a model has both node
|
||||
embeddings and regular dense model parameters and users want to perform sparse
|
||||
updates on the embeddings, they need to create two optimizers, one for node
|
||||
embeddings and the other for dense model parameters, as shown in the code
|
||||
below:
|
||||
|
||||
.. code:: python
|
||||
|
||||
sparse_optimizer = dgl.distributed.SparseAdagrad([emb], lr=lr1)
|
||||
optimizer = th.optim.Adam(model.parameters(), lr=lr2)
|
||||
feats = emb(nids)
|
||||
loss = model(feats)
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
sparse_optimizer.step()
|
||||
|
||||
.. note::
|
||||
|
||||
:class:`~dgl.distributed.DistEmbedding` does not inherit :class:`torch.nn.Module`,
|
||||
so we recommend using it outside of your own NN module.
|
||||
|
||||
Distributed sampling
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL provides two levels of APIs for sampling nodes and edges to generate
|
||||
mini-batches (see the section of mini-batch training). The low-level APIs
|
||||
require users to write code to explicitly define how a layer of nodes are
|
||||
sampled (e.g., using :func:`dgl.sampling.sample_neighbors` ). The high-level
|
||||
sampling APIs implement a few popular sampling algorithms for node
|
||||
classification and link prediction tasks (e.g.,
|
||||
:class:`~dgl.dataloading.NodeDataLoader` and
|
||||
:class:`~dgl.dataloading.EdgeDataLoader` ).
|
||||
|
||||
The distributed sampling module follows the same design and provides two levels
|
||||
of sampling APIs. For the lower-level sampling API, it provides
|
||||
:func:`~dgl.distributed.sample_neighbors` for distributed neighborhood sampling
|
||||
on :class:`~dgl.distributed.DistGraph`. In addition, DGL provides a distributed
|
||||
DataLoader (:class:`~dgl.distributed.DistDataLoader` ) for distributed
|
||||
sampling. The distributed DataLoader has the same interface as Pytorch
|
||||
DataLoader except that users cannot specify the number of worker processes when
|
||||
creating a dataloader. The worker processes are created in
|
||||
:func:`dgl.distributed.initialize`.
|
||||
|
||||
.. note::
|
||||
|
||||
When running :func:`dgl.distributed.sample_neighbors` on
|
||||
:class:`~dgl.distributed.DistGraph`, the sampler cannot run in Pytorch
|
||||
DataLoader with multiple worker processes. The main reason is that Pytorch
|
||||
DataLoader creates new sampling worker processes in every epoch, which
|
||||
leads to creating and destroying :class:`~dgl.distributed.DistGraph`
|
||||
objects many times.
|
||||
|
||||
When using the low-level API, the sampling code is similar to single-process sampling. The only
|
||||
difference is that users need to use :func:`dgl.distributed.sample_neighbors` and
|
||||
:class:`~dgl.distributed.DistDataLoader`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def sample_blocks(seeds):
|
||||
seeds = th.LongTensor(np.asarray(seeds))
|
||||
blocks = []
|
||||
for fanout in [10, 25]:
|
||||
frontier = dgl.distributed.sample_neighbors(g, seeds, fanout, replace=True)
|
||||
block = dgl.to_block(frontier, seeds)
|
||||
seeds = block.srcdata[dgl.NID]
|
||||
blocks.insert(0, block)
|
||||
return blocks
|
||||
dataloader = dgl.distributed.DistDataLoader(dataset=train_nid,
|
||||
batch_size=batch_size,
|
||||
collate_fn=sample_blocks,
|
||||
shuffle=True)
|
||||
for batch in dataloader:
|
||||
...
|
||||
|
||||
The high-level sampling APIs (:class:`~dgl.dataloading.NodeDataLoader` and
|
||||
:class:`~dgl.dataloading.EdgeDataLoader` ) has distributed counterparts
|
||||
(:class:`~dgl.distributed.DistNodeDataLoader` and
|
||||
:class:`~dgl.distributed.DistEdgeDataLoader`). The code is exactly the same as
|
||||
single-process sampling otherwise.
|
||||
|
||||
.. code:: python
|
||||
|
||||
sampler = dgl.sampling.MultiLayerNeighborSampler([10, 25])
|
||||
dataloader = dgl.distributed.DistNodeDataLoader(g, train_nid, sampler,
|
||||
batch_size=batch_size, shuffle=True)
|
||||
for batch in dataloader:
|
||||
...
|
||||
|
||||
|
||||
Split workloads
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
To train a model, users first need to split the dataset into training,
|
||||
validation and test sets. For distributed training, this step is usually done
|
||||
before we invoke :func:`dgl.distributed.partition_graph` to partition a graph.
|
||||
We recommend to store the data split in boolean arrays as node data or edge
|
||||
data. For node classification tasks, the length of these boolean arrays is the
|
||||
number of nodes in a graph and each of their elements indicates the existence
|
||||
of a node in a training/validation/test set. Similar boolean arrays should be
|
||||
used for link prediction tasks. :func:`dgl.distributed.partition_graph` splits
|
||||
these boolean arrays (because they are stored as the node data or edge data of
|
||||
the graph) based on the graph partitioning result and store them with graph
|
||||
partitions.
|
||||
|
||||
During distributed training, users need to assign training nodes/edges to each
|
||||
trainer. Similarly, we also need to split the validation and test set in the
|
||||
same way. DGL provides :func:`~dgl.distributed.node_split` and
|
||||
:func:`~dgl.distributed.edge_split` to split the training, validation and test
|
||||
set at runtime for distributed training. The two functions take the boolean
|
||||
arrays constructed before graph partitioning as input, split them and return a
|
||||
portion for the local trainer. By default, they ensure that all portions have
|
||||
the same number of nodes/edges. This is important for synchronous SGD, which
|
||||
assumes each trainer has the same number of mini-batches.
|
||||
|
||||
The example below splits the training set and returns a subset of nodes for the
|
||||
local process.
|
||||
|
||||
.. code:: python
|
||||
|
||||
train_nids = dgl.distributed.node_split(g.ndata['train_mask'])
|
||||
@@ -0,0 +1,189 @@
|
||||
.. _guide-distributed-hetero:
|
||||
|
||||
7.5 Heterogeneous Graph Under The Hood
|
||||
--------------------------------------------
|
||||
|
||||
The chapter covers the implementation details of distributed heterogeneous
|
||||
graph. They are transparent to users in most scenarios but could be useful
|
||||
for advanced customization.
|
||||
|
||||
In DGL, a node or edge in a heterogeneous graph has a unique ID in its own node
|
||||
type or edge type. Therefore, DGL can identify a node or an edge
|
||||
with a tuple: ``(node/edge type, type-wise ID)``. We call IDs of such form as
|
||||
**heterogeneous IDs**. To patition a heterogeneous graph for distributed training,
|
||||
DGL converts it to a homogeneous graph so that we can reuse the partitioning
|
||||
algorithms designed for homogeneous graphs. Each node/edge is thus uniquely mapped
|
||||
to an integer ID in a consecutive ID range (e.g., from 0 to the total number of
|
||||
nodes of all types). We call the IDs after conversion as **homogeneous IDs**.
|
||||
|
||||
Below is an illustration of the ID conversion process. Here, the graph has two
|
||||
types of nodes (:math:`T0` and :math:`T1` ), and four types of edges
|
||||
(:math:`R0`, :math:`R1`, :math:`R2`, :math:`R3` ). There are a total of 400
|
||||
nodes in the graph and each type has 200 nodes. Nodes of :math:`T0` have IDs in
|
||||
[0,200), while nodes of :math:`T1` have IDs in [200, 400). In this example, if
|
||||
we use a tuple to identify the nodes, nodes of :math:`T0` are identified as
|
||||
(T0, type-wise ID), where type-wise ID falls in [0, 200); nodes of :math:`T1`
|
||||
are identified as (T1, type-wise ID), where type-wise ID also falls in [0,
|
||||
200).
|
||||
|
||||
.. figure:: https://data.dgl.ai/tutorial/hetero/heterograph_ids.png
|
||||
:alt: Imgur
|
||||
|
||||
ID Conversion Utilities
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
During Preprocessing
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The steps of :ref:`Parallel Processing Pipeline <guide-distributed-preprocessing>`
|
||||
all use heterogeneous IDs for their inputs and outputs. Nevertheless, some steps such as
|
||||
ParMETIS partitioning are easier to be implemented using homogeneous IDs, thus
|
||||
requiring a utility to perform ID conversion.
|
||||
The code below implements a simple ``IDConverter`` using the metadata information
|
||||
in the metadata JSON from the chunked graph data format. It starts from some
|
||||
node type :math:`A` as node type 0, then assigns all its nodes with IDs
|
||||
in range :math:`[0, |V_A|-1)`. It then moves to the next node
|
||||
type B as node type 1 and assigns all its nodes with IDs in range
|
||||
:math:`[|V_A|, |V_A|+|V_B|-1)`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
from bisect import bisect_left
|
||||
import numpy as np
|
||||
|
||||
class IDConverter:
|
||||
def __init__(self, meta):
|
||||
# meta is the JSON object loaded from metadata.json
|
||||
self.node_type = meta['node_type']
|
||||
self.edge_type = meta['edge_type']
|
||||
self.ntype2id_map = {ntype : i for i, ntype in enumerate(self.node_type)}
|
||||
self.etype2id_map = {etype : i for i, etype in enumerate(self.edge_type)}
|
||||
self.num_nodes = [sum(ns) for ns in meta['num_nodes_per_chunk']]
|
||||
self.num_edges = [sum(ns) for ns in meta['num_edges_per_chunk']]
|
||||
self.nid_offset = np.cumsum([0] + self.num_nodes)
|
||||
self.eid_offset = np.cumsum([0] + self.num_edges)
|
||||
|
||||
def ntype2id(self, ntype):
|
||||
"""From node type name to node type ID"""
|
||||
return self.ntype2id_map[ntype]
|
||||
|
||||
def etype2id(self, etype):
|
||||
"""From edge type name to edge type ID"""
|
||||
return self.etype2id_map[etype]
|
||||
|
||||
def id2ntype(self, id):
|
||||
"""From node type ID to node type name"""
|
||||
return self.node_type[id]
|
||||
|
||||
def id2etype(self, id):
|
||||
"""From edge type ID to edge type name"""
|
||||
return self.edge_type[id]
|
||||
|
||||
def nid_het2hom(self, ntype, id):
|
||||
"""From heterogeneous node ID to homogeneous node ID"""
|
||||
tid = self.ntype2id(ntype)
|
||||
if id < 0 or id >= self.num_nodes[tid]:
|
||||
raise ValueError(f'Invalid node ID of type {ntype}. Must be within range [0, {self.num_nodes[tid]})')
|
||||
return self.nid_offset[tid] + id
|
||||
|
||||
def nid_hom2het(self, id):
|
||||
"""From heterogeneous node ID to homogeneous node ID"""
|
||||
if id < 0 or id >= self.nid_offset[-1]:
|
||||
raise ValueError(f'Invalid homogeneous node ID. Must be within range [0, self.nid_offset[-1])')
|
||||
tid = bisect_left(self.nid_offset, id) - 1
|
||||
# Return a pair (node_type, type_wise_id)
|
||||
return self.id2ntype(tid), id - self.nid_offset[tid]
|
||||
|
||||
def eid_het2hom(self, etype, id):
|
||||
"""From heterogeneous edge ID to homogeneous edge ID"""
|
||||
tid = self.etype2id(etype)
|
||||
if id < 0 or id >= self.num_edges[tid]:
|
||||
raise ValueError(f'Invalid edge ID of type {etype}. Must be within range [0, {self.num_edges[tid]})')
|
||||
return self.eid_offset[tid] + id
|
||||
|
||||
def eid_hom2het(self, id):
|
||||
"""From heterogeneous edge ID to homogeneous edge ID"""
|
||||
if id < 0 or id >= self.eid_offset[-1]:
|
||||
raise ValueError(f'Invalid homogeneous edge ID. Must be within range [0, self.eid_offset[-1])')
|
||||
tid = bisect_left(self.eid_offset, id) - 1
|
||||
# Return a pair (edge_type, type_wise_id)
|
||||
return self.id2etype(tid), id - self.eid_offset[tid]
|
||||
|
||||
After Partition Loading
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
After the partitions are loaded into trainer or server processes, the loaded
|
||||
:class:`~dgl.distributed.GraphPartitionBook` provides utilities for conversion
|
||||
between homogeneous IDs and heterogeneous IDs.
|
||||
|
||||
* :func:`~dgl.distributed.GraphPartitionBook.map_to_per_ntype`: convert a homogeneous node ID to type-wise ID and node type ID.
|
||||
* :func:`~dgl.distributed.GraphPartitionBook.map_to_per_etype`: convert a homogeneous edge ID to type-wise ID and edge type ID.
|
||||
* :func:`~dgl.distributed.GraphPartitionBook.map_to_homo_nid`: convert type-wise ID and node type to a homogeneous node ID.
|
||||
* :func:`~dgl.distributed.GraphPartitionBook.map_to_homo_eid`: convert type-wise ID and edge type to a homogeneous edge ID.
|
||||
|
||||
Because all DGL's low-level :ref:`distributed graph sampling operators
|
||||
<api-distributed-sampling-ops>` use homogeneous IDs, DGL internally converts
|
||||
the heterogeneous IDs specified by users to homogeneous IDs before invoking
|
||||
sampling operators. Below shows an example of sampling a subgraph by
|
||||
:func:`~dgl.distributed.sample_neighbors` from nodes of type ``"paper"``. It
|
||||
first performs ID conversion, and after getting the sampled subgraph, converts
|
||||
the homogeneous node/edge IDs back to heterogeneous ones.
|
||||
|
||||
.. code:: python
|
||||
|
||||
gpb = g.get_partition_book()
|
||||
# We need to map the type-wise node IDs to homogeneous IDs.
|
||||
cur = gpb.map_to_homo_nid(seeds, 'paper')
|
||||
# For a heterogeneous input graph, the returned frontier is stored in
|
||||
# the homogeneous graph format.
|
||||
frontier = dgl.distributed.sample_neighbors(g, cur, fanout, replace=False)
|
||||
block = dgl.to_block(frontier, cur)
|
||||
cur = block.srcdata[dgl.NID]
|
||||
|
||||
block.edata[dgl.EID] = frontier.edata[dgl.EID]
|
||||
# Map the homogeneous edge Ids to their edge type.
|
||||
block.edata[dgl.ETYPE], block.edata[dgl.EID] = gpb.map_to_per_etype(block.edata[dgl.EID])
|
||||
# Map the homogeneous node Ids to their node types and per-type Ids.
|
||||
block.srcdata[dgl.NTYPE], block.srcdata[dgl.NID] = gpb.map_to_per_ntype(block.srcdata[dgl.NID])
|
||||
block.dstdata[dgl.NTYPE], block.dstdata[dgl.NID] = gpb.map_to_per_ntype(block.dstdata[dgl.NID])
|
||||
|
||||
Note that getting node/edge types from type IDs is simple -- just getting them
|
||||
from the ``ntypes`` attributes of a ``DistGraph``, i.e., ``g.ntypes[node_type_id]``.
|
||||
|
||||
Access distributed graph data
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The :class:`~dgl.distributed.DistGraph` class supports similar interface as
|
||||
:class:`~dgl.DGLGraph`. Below shows an example of getting the feature data of
|
||||
nodes 0, 10, 20 of type :math:`T0`. When accessing data in
|
||||
:class:`~dgl.distributed.DistGraph`, a user needs to use type-wise IDs and
|
||||
corresponding node types or edge types.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
g = dgl.distributed.DistGraph('graph_name', part_config='data/graph_name.json')
|
||||
feat = g.nodes['T0'].data['feat'][[0, 10, 20]]
|
||||
|
||||
A user can create distributed tensors and distributed embeddings for a
|
||||
particular node type or edge type. Distributed tensors and embeddings are split
|
||||
and stored in multiple machines. To create one, a user needs to specify how it
|
||||
is partitioned with :class:`~dgl.distributed.PartitionPolicy`. By default, DGL
|
||||
chooses the right partition policy based on the size of the first dimension.
|
||||
However, if multiple node types or edge types have the same number of nodes or
|
||||
edges, DGL cannot determine the partition policy automatically. A user needs to
|
||||
explicitly specify the partition policy. Below shows an example of creating a
|
||||
distributed tensor for node type :math:`T0` by using the partition policy for :math:`T0`
|
||||
and store it as node data of :math:`T0`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g.nodes['T0'].data['feat1'] = dgl.distributed.DistTensor(
|
||||
(g.num_nodes('T0'), 1), th.float32, 'feat1',
|
||||
part_policy=g.get_node_partition_policy('T0'))
|
||||
|
||||
The partition policies used for creating distributed tensors and embeddings are
|
||||
initialized when a heterogeneous graph is loaded into the graph server. A user
|
||||
cannot create a new partition policy at runtime. Therefore, a user can only
|
||||
create distributed tensors or embeddings for a node type or edge type.
|
||||
Accessing distributed tensors and embeddings also requires type-wise IDs.
|
||||
@@ -0,0 +1,258 @@
|
||||
.. _guide-distributed-partition:
|
||||
|
||||
7.4 Advanced Graph Partitioning
|
||||
---------------------------------------
|
||||
|
||||
The chapter covers some of the advanced topics for graph partitioning.
|
||||
|
||||
METIS partition algorithm
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
`METIS <http://glaros.dtc.umn.edu/gkhome/views/metis>`__ is a state-of-the-art
|
||||
graph partitioning algorithm that can generate partitions with minimal number
|
||||
of cross-partition edges, making it suitable for distributed message passing
|
||||
where the amount of network communication is proportional to the number of
|
||||
cross-partition edges. DGL has integrated METIS as the default partitioning
|
||||
algorithm in its :func:`dgl.distributed.partition_graph` API.
|
||||
|
||||
Output format
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Regardless of the partitioning algorithm in use, the partitioned results are stored
|
||||
in data files organized as follows:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
data_root_dir/
|
||||
|-- graph_name.json # partition configuration file in JSON
|
||||
|-- part0/ # data for partition 0
|
||||
| |-- node_feats.dgl # node features stored in binary format
|
||||
| |-- edge_feats.dgl # edge features stored in binary format
|
||||
| |-- graph.dgl # graph structure of this partition stored in binary format
|
||||
|
|
||||
|-- part1/ # data for partition 1
|
||||
| |-- node_feats.dgl
|
||||
| |-- edge_feats.dgl
|
||||
| |-- graph.dgl
|
||||
|
|
||||
|-- ... # data for other partitions
|
||||
|
||||
When distributed to a cluster, the metadata JSON should be copied to all the machines
|
||||
while the ``partX`` folders should be dispatched accordingly.
|
||||
|
||||
DGL provides a :func:`dgl.distributed.load_partition` function to load one partition
|
||||
for inspection.
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> # load partition 0
|
||||
>>> part_data = dgl.distributed.load_partition('data_root_dir/graph_name.json', 0)
|
||||
>>> g, nfeat, efeat, partition_book, graph_name, ntypes, etypes = part_data # unpack
|
||||
>>> print(g)
|
||||
Graph(num_nodes=966043, num_edges=34270118,
|
||||
ndata_schemes={'orig_id': Scheme(shape=(), dtype=torch.int64),
|
||||
'part_id': Scheme(shape=(), dtype=torch.int64),
|
||||
'_ID': Scheme(shape=(), dtype=torch.int64),
|
||||
'inner_node': Scheme(shape=(), dtype=torch.int32)}
|
||||
edata_schemes={'_ID': Scheme(shape=(), dtype=torch.int64),
|
||||
'inner_edge': Scheme(shape=(), dtype=torch.int8),
|
||||
'orig_id': Scheme(shape=(), dtype=torch.int64)})
|
||||
|
||||
As mentioned in the `ID mapping`_ section, each partition carries auxiliary information
|
||||
saved as ndata or edata such as original node/edge IDs, partition IDs, etc. Each partition
|
||||
not only saves nodes/edges it owns, but also includes node/edges that are adjacent to
|
||||
the partition (called **HALO** nodes/edges). The ``inner_node`` and ``inner_edge``
|
||||
indicate whether a node/edge truely belongs to the partition (value is ``True``)
|
||||
or is a HALO node/edge (value is ``False``).
|
||||
|
||||
The :func:`~dgl.distributed.load_partition` function loads all data at once. Users can
|
||||
load features or the partition book using the :func:`dgl.distributed.load_partition_feats`
|
||||
and :func:`dgl.distributed.load_partition_book` APIs respectively.
|
||||
|
||||
|
||||
Parallel METIS partitioning
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For massive graphs where parallel preprocessing is desired, DGL supports
|
||||
`ParMETIS <http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview>`__ as one
|
||||
of the choices of partitioning algorithms.
|
||||
|
||||
.. note::
|
||||
|
||||
Because ParMETIS does not support heterogeneous graph, users need to
|
||||
conduct ID conversion before and after running ParMETIS.
|
||||
Check out chapter :ref:`guide-distributed-hetero` for explanation.
|
||||
|
||||
.. note::
|
||||
|
||||
Please make sure that the input graph to ParMETIS does not have
|
||||
duplicate edges (or parallel edges) and self-loop edges.
|
||||
|
||||
ParMETIS Installation
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
ParMETIS requires METIS and GKLib. Please follow the instructions `here
|
||||
<https://github.com/KarypisLab/GKlib>`__ to compile and install GKLib. For
|
||||
compiling and install METIS, please follow the instructions below to clone
|
||||
METIS with GIT and compile it with int64 support.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/KarypisLab/METIS.git
|
||||
make config shared=1 cc=gcc prefix=~/local i64=1
|
||||
make install
|
||||
|
||||
|
||||
For now, we need to compile and install ParMETIS manually. We clone the DGL branch of ParMETIS as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone --branch dgl https://github.com/KarypisLab/ParMETIS.git
|
||||
|
||||
Then compile and install ParMETIS.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make config cc=mpicc prefix=~/local
|
||||
make install
|
||||
|
||||
Before running ParMETIS, we need to set two environment variables: ``PATH`` and ``LD_LIBRARY_PATH``.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export PATH=$PATH:$HOME/local/bin
|
||||
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/local/lib/
|
||||
|
||||
Input format
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. note::
|
||||
|
||||
As a prerequisite, read chapter :doc:`guide-distributed-hetero` to understand
|
||||
how DGL organize heterogeneous graph for distributed training.
|
||||
|
||||
The input graph for ParMETIS is stored in three files with the following names:
|
||||
``xxx_nodes.txt``, ``xxx_edges.txt`` and ``xxx_stats.txt``, where ``xxx`` is a
|
||||
graph name.
|
||||
|
||||
Each row in ``xxx_nodes.txt`` stores the information of a node. Row ID is
|
||||
also the *homogeneous* ID of a node, e.g., row 0 is for node 0; row 1 is for
|
||||
node 1, etc. Each row has the following format:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
<node_type_id> <node_weight_list> <type_wise_node_id>
|
||||
|
||||
All fields are separated by whitespace:
|
||||
|
||||
* ``<node_type_id>`` is an integer starting from 0. Each node type is mapped to
|
||||
an integer. For a homogeneous graph, its value is always 0.
|
||||
* ``<node_weight_list>`` are integers (separated by whitespace) that indicate
|
||||
the node weights used by ParMETIS to balance graph partitions. For homogeneous
|
||||
graphs, the list has only one integer while for heterogeneous graphs with
|
||||
:math:`T` node types, the list should has :math:`T` integers. If the node
|
||||
belongs to node type :math:`t`, then all the integers except the :math:`t^{th}`
|
||||
one are zero; the :math:`t^{th}` integer is the weight of that node. ParMETIS
|
||||
will try to balance the total node weight of each partition. For heterogeneous
|
||||
graph, it will try to distribute nodes of the same type to all partitions.
|
||||
The recommended node weights are 1 for balancing the number of nodes in each
|
||||
partition or node degrees for balancing the number of edges in each partition.
|
||||
* ``<type_wise_node_id>`` is an integer representing the node ID in its own type.
|
||||
|
||||
Below shows an example of a node file for a heterogeneous graph with two node
|
||||
types. Node type 0 has three nodes; node type 1 has four nodes. It uses two
|
||||
node weights to ensure that ParMETIS will generate partitions with roughly the
|
||||
same number of nodes for type 0 and the same number of nodes for type 1.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
0 1 0 0
|
||||
0 1 0 1
|
||||
0 1 0 2
|
||||
1 0 1 0
|
||||
1 0 1 1
|
||||
1 0 1 2
|
||||
1 0 1 3
|
||||
|
||||
Similarly, each row in ``xxx_edges.txt`` stores the information of an edge. Row ID is
|
||||
also the *homogeneous* ID of an edge, e.g., row 0 is for edge 0; row 1 is for
|
||||
edge 1, etc. Each row has the following format:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
<src_node_id> <dst_node_id> <type_wise_edge_id> <edge_type_id>
|
||||
|
||||
All fields are separated by whitespace:
|
||||
|
||||
* ``<src_node_id>`` is the *homogeneous* ID of the source node.
|
||||
* ``<dst_node_id>`` is the *homogeneous* ID of the destination node.
|
||||
* ``<type_wise_edge_id>`` is the edge ID for the edge type.
|
||||
* ``<edge_type_id>`` is an integer starting from 0. Each edge type is mapped to
|
||||
an integer. For a homogeneous graph, its value is always 0.
|
||||
|
||||
``xxx_stats.txt`` stores some basic statistics of the graph. It has only one line with three fields
|
||||
separated by whitespace:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
<num_nodes> <num_edges> <total_node_weights>
|
||||
|
||||
* ``num_nodes`` stores the total number of nodes regardless of node types.
|
||||
* ``num_edges`` stores the total number of edges regardless of edge types.
|
||||
* ``total_node_weights`` stores the number of node weights in the node file.
|
||||
|
||||
Run ParMETIS and output format
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
ParMETIS contains a command called ``pm_dglpart``, which loads the graph stored
|
||||
in the three files from the machine where ``pm_dglpart`` is invoked, distributes
|
||||
data to all machines in the cluster and invokes ParMETIS to partition the
|
||||
graph. When it completes, it generates three files for each partition:
|
||||
``p<part_id>-xxx_nodes.txt``, ``p<part_id>-xxx_edges.txt``,
|
||||
``p<part_id>-xxx_stats.txt``.
|
||||
|
||||
.. note::
|
||||
|
||||
ParMETIS reassigns IDs to nodes during the partitioning. After ID reassignment,
|
||||
the nodes in a partition are assigned with contiguous IDs; furthermore, the nodes of
|
||||
the same type are assigned with contiguous IDs.
|
||||
|
||||
``p<part_id>-xxx_nodes.txt`` stores the node data of the partition. Each row represents
|
||||
a node with the following fields:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
<node_id> <node_type_id> <node_weight_list> <type_wise_node_id>
|
||||
|
||||
* ``<node_id>`` is the *homogeneous* node ID after ID reassignment.
|
||||
* ``<node_type_id>`` is the node type ID.
|
||||
* ``<node_weight_list>`` is the node weight used by ParMETIS (copied from the input file).
|
||||
* ``<type_wise_node_id>`` is an integer representing the node ID in its own type.
|
||||
|
||||
``p<part_id>-xxx_edges.txt`` stores the edge data of the partition. Each row represents
|
||||
an edge with the following fields:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
<src_id> <dst_id> <orig_src_id> <orig_dst_id> <type_wise_edge_id> <edge_type_id>
|
||||
|
||||
* ``<src_id>`` is the *homogeneous* ID of the source node after ID reassignment.
|
||||
* ``<dst_id>`` is the *homogeneous* ID of the destination node after ID reassignment.
|
||||
* ``<orig_src_id>`` is the *homogeneous* ID of the source node in the input graph.
|
||||
* ``<orig_dst_id>`` is the *homogeneous* ID of the destination node in the input graph.
|
||||
* ``<type_wise_edge_id>`` is the edge ID in its own type.
|
||||
* ``<edge_type_id>`` is the edge type ID.
|
||||
|
||||
When invoking ``pm_dglpart``, the three input files: ``xxx_nodes.txt``,
|
||||
``xxx_edges.txt``, ``xxx_stats.txt`` should be located in the directory where
|
||||
``pm_dglpart`` runs. The following command run four ParMETIS processes to
|
||||
partition the graph named ``xxx`` into eight partitions (each process handles
|
||||
two partitions).
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mpirun -np 4 pm_dglpart xxx 2
|
||||
|
||||
The output files from ParMETIS then need to be converted to the
|
||||
:ref:`partition assignment format <guide-distributed-prep-partition>` to in
|
||||
order to run subsequent preprocessing steps.
|
||||
@@ -0,0 +1,507 @@
|
||||
.. _guide-distributed-preprocessing:
|
||||
|
||||
7.1 Data Preprocessing
|
||||
------------------------------------------
|
||||
|
||||
Before launching training jobs, DGL requires the input data to be partitioned
|
||||
and distributed to the target machines. In order to handle different scales
|
||||
of graphs, DGL provides 2 partitioning approaches:
|
||||
|
||||
* A partitioning API for graphs that can fit in a single machine memory.
|
||||
* A distributed partition pipeline for graphs beyond a single machine capacity.
|
||||
|
||||
7.1.1 Partitioning API
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
For relatively small graphs, DGL provides a partitioning API
|
||||
:func:`~dgl.distributed.partition_graph` that partitions
|
||||
an in-memory :class:`~dgl.DGLGraph` object. It supports
|
||||
multiple partitioning algorithms such as random partitioning and
|
||||
`Metis <http://glaros.dtc.umn.edu/gkhome/views/metis>`__.
|
||||
The benefit of Metis partitioning is that it can generate partitions with
|
||||
minimal edge cuts to reduce network communication for distributed training and
|
||||
inference. DGL uses the latest version of Metis with the options optimized for
|
||||
the real-world graphs with power-law distribution. After partitioning, the API
|
||||
constructs the partitioned results in a format that is easy to load during the
|
||||
training. For example,
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import dgl
|
||||
|
||||
g = ... # create or load a DGLGraph object
|
||||
dgl.distributed.partition_graph(g, 'mygraph', 2, 'data_root_dir')
|
||||
|
||||
will outputs the following data file.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
data_root_dir/
|
||||
|-- mygraph.json # metadata JSON. File name is the given graph name.
|
||||
|-- part0/ # data for partition 0
|
||||
| |-- node_feats.dgl # node features stored in binary format
|
||||
| |-- edge_feats.dgl # edge features stored in binary format
|
||||
| |-- graph.dgl # graph structure of this partition stored in binary format
|
||||
|
|
||||
|-- part1/ # data for partition 1
|
||||
|-- node_feats.dgl
|
||||
|-- edge_feats.dgl
|
||||
|-- graph.dgl
|
||||
|
||||
Chapter :ref:`guide-distributed-partition` covers more details about the
|
||||
partition format. To distribute the partitions to a cluster, users can either save
|
||||
the data in some shared folder accessible by all machines, or copy the metadata
|
||||
JSON as well as the corresponding partition folder ``partX`` to the X^th machine.
|
||||
|
||||
Using :func:`~dgl.distributed.partition_graph` requires an instance with large enough
|
||||
CPU RAM to hold the entire graph structure and features, which may not be viable for
|
||||
graphs with hundreds of billions of edges or large features. We describe how to use
|
||||
the *parallel data preparation pipeline* for such cases next.
|
||||
|
||||
Load balancing
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
When partitioning a graph, by default, METIS only balances the number of nodes
|
||||
in each partition. This can result in suboptimal configuration, depending on
|
||||
the task at hand. For example, in the case of semi-supervised node
|
||||
classification, a trainer performs computation on a subset of labeled nodes in
|
||||
a local partition. A partitioning that only balances nodes in a graph (both
|
||||
labeled and unlabeled), may end up with computational load imbalance. To get a
|
||||
balanced workload in each partition, the partition API allows balancing between
|
||||
partitions with respect to the number of nodes in each node type, by specifying
|
||||
``balance_ntypes`` in :func:`~dgl.distributed.partition_graph`. Users can take
|
||||
advantage of this and consider nodes in the training set, validation set and
|
||||
test set are of different node types.
|
||||
|
||||
The following example considers nodes inside the training set and outside the
|
||||
training set are two types of nodes:
|
||||
|
||||
.. code:: python
|
||||
|
||||
dgl.distributed.partition_graph(g, 'graph_name', 4, '/tmp/test', balance_ntypes=g.ndata['train_mask'])
|
||||
|
||||
In addition to balancing the node types,
|
||||
:func:`dgl.distributed.partition_graph` also allows balancing between
|
||||
in-degrees of nodes of different node types by specifying ``balance_edges``.
|
||||
This balances the number of edges incident to the nodes of different types.
|
||||
|
||||
ID mapping
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
After partitioning, :func:`~dgl.distributed.partition_graph` remap node
|
||||
and edge IDs so that nodes of the same partition are aranged together
|
||||
(in a consecutive ID range), making it easier to store partitioned node/edge
|
||||
features. The API also automatically shuffles the node/edge features
|
||||
according to the new IDs. However, some downstream tasks may want to
|
||||
recover the original node/edge IDs (such as extracting the computed node
|
||||
embeddings for later use). For such cases, pass ``return_mapping=True``
|
||||
to :func:`~dgl.distributed.partition_graph`, which makes the API returns
|
||||
the ID mappings between the remapped node/edge IDs and their origianl ones.
|
||||
For a homogeneous graph, it returns two vectors. The first vector maps every new
|
||||
node ID to its original ID; the second vector maps every new edge ID to
|
||||
its original ID. For a heterogeneous graph, it returns two dictionaries of
|
||||
vectors. The first dictionary contains the mapping for each node type; the
|
||||
second dictionary contains the mapping for each edge type.
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_map, edge_map = dgl.distributed.partition_graph(g, 'graph_name', 4, '/tmp/test',
|
||||
balance_ntypes=g.ndata['train_mask'],
|
||||
return_mapping=True)
|
||||
# Let's assume that node_emb is saved from the distributed training.
|
||||
orig_node_emb = th.zeros(node_emb.shape, dtype=node_emb.dtype)
|
||||
orig_node_emb[node_map] = node_emb
|
||||
|
||||
|
||||
Load partitioned graphs
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
DGL provides a :func:`dgl.distributed.load_partition` function to load one partition
|
||||
for inspection.
|
||||
|
||||
.. code:: python
|
||||
|
||||
>>> import dgl
|
||||
>>> # load partition 0
|
||||
>>> part_data = dgl.distributed.load_partition('data_root_dir/graph_name.json', 0)
|
||||
>>> g, nfeat, efeat, partition_book, graph_name, ntypes, etypes = part_data # unpack
|
||||
>>> print(g)
|
||||
Graph(num_nodes=966043, num_edges=34270118,
|
||||
ndata_schemes={'orig_id': Scheme(shape=(), dtype=torch.int64),
|
||||
'part_id': Scheme(shape=(), dtype=torch.int64),
|
||||
'_ID': Scheme(shape=(), dtype=torch.int64),
|
||||
'inner_node': Scheme(shape=(), dtype=torch.int32)}
|
||||
edata_schemes={'_ID': Scheme(shape=(), dtype=torch.int64),
|
||||
'inner_edge': Scheme(shape=(), dtype=torch.int8),
|
||||
'orig_id': Scheme(shape=(), dtype=torch.int64)})
|
||||
|
||||
As mentioned in the `ID mapping`_ section, each partition carries auxiliary information
|
||||
saved as ndata or edata such as original node/edge IDs, partition IDs, etc. Each partition
|
||||
not only saves nodes/edges it owns, but also includes node/edges that are adjacent to
|
||||
the partition (called **HALO** nodes/edges). The ``inner_node`` and ``inner_edge``
|
||||
indicate whether a node/edge truely belongs to the partition (value is ``True``)
|
||||
or is a HALO node/edge (value is ``False``).
|
||||
|
||||
The :func:`~dgl.distributed.load_partition` function loads all data at once. Users can
|
||||
load features or the partition book using the :func:`dgl.distributed.load_partition_feats`
|
||||
and :func:`dgl.distributed.load_partition_book` APIs respectively.
|
||||
|
||||
|
||||
7.1.2 Distributed Graph Partitioning Pipeline
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To handle massive graph data that cannot fit in the CPU RAM of a
|
||||
single machine, DGL utilizes data chunking and parallel processing to reduce
|
||||
memory footprint and running time. The figure below illustrates the
|
||||
pipeline:
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/guide_7_distdataprep.png
|
||||
|
||||
* The pipeline takes input data stored in *Chunked Graph Format* and
|
||||
produces and dispatches data partitions to the target machines.
|
||||
* **Step.1 Graph Partitioning:** It calculates the ownership of each partition
|
||||
and saves the results as a set of files called *partition assignment*.
|
||||
To speedup the step, some algorithms (e.g., ParMETIS) support parallel computing
|
||||
using multiple machines.
|
||||
* **Step.2 Data Dispatching:** Given the partition assignment, the step then
|
||||
physically partitions the graph data and dispatches them to the machines user
|
||||
specified. It also converts the graph data into formats that are suitable for
|
||||
distributed training and evaluation.
|
||||
|
||||
The whole pipeline is modularized so that each step can be invoked
|
||||
individually. For example, users can replace Step.1 with some custom graph partition
|
||||
algorithm as long as it produces partition assignment files
|
||||
correctly.
|
||||
|
||||
.. _guide-distributed-prep-chunk:
|
||||
Chunked Graph Format
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To run the pipeline, DGL requires the input graph to be stored in multiple data
|
||||
chunks. Each data chunk is the unit of data preprocessing and thus should fit
|
||||
into CPU RAM. In this section, we use the MAG240M-LSC data from `Open Graph
|
||||
Benchmark <https://ogb.stanford.edu/docs/lsc/mag240m/>`__ as an example to
|
||||
describe the overall design, followed by a formal specification and
|
||||
tips for creating data in such format.
|
||||
|
||||
Example: MAG240M-LSC
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The MAG240M-LSC graph is a heterogeneous academic graph
|
||||
extracted from the Microsoft Academic Graph (MAG), whose schema diagram is
|
||||
illustrated below:
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/guide_7_mag240m.png
|
||||
|
||||
Its raw data files are organized as follows:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
/mydata/MAG240M-LSC/
|
||||
|-- meta.pt # # A dictionary of the number of nodes for each type saved by torch.save,
|
||||
| # as well as num_classes
|
||||
|-- processed/
|
||||
|-- author___affiliated_with___institution/
|
||||
| |-- edge_index.npy # graph, 713 MB
|
||||
|
|
||||
|-- paper/
|
||||
| |-- node_feat.npy # feature, 187 GB, (numpy memmap format)
|
||||
| |-- node_label.npy # label, 974 MB
|
||||
| |-- node_year.npy # year, 974 MB
|
||||
|
|
||||
|-- paper___cites___paper/
|
||||
| |-- edge_index.npy # graph, 21 GB
|
||||
|
|
||||
|-- author___writes___paper/
|
||||
|-- edge_index.npy # graph, 6GB
|
||||
|
||||
The graph has three node types (``"paper"``, ``"author"`` and ``"institution"``),
|
||||
three edge types/relations (``"cites"``, ``"writes"`` and ``"affiliated_with"``). The
|
||||
``"paper"`` nodes have three attributes (``"feat"``, ``"label"``, ``"year"'``), while
|
||||
other types of nodes and edges are featureless. Below shows the data files when
|
||||
it is stored in DGL Chunked Graph Format:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
/mydata/MAG240M-LSC_chunked/
|
||||
|-- metadata.json # metadata json file
|
||||
|-- edges/ # stores edge ID data
|
||||
| |-- writes-part1.csv
|
||||
| |-- writes-part2.csv
|
||||
| |-- affiliated_with-part1.csv
|
||||
| |-- affiliated_with-part2.csv
|
||||
| |-- cites-part1.csv
|
||||
| |-- cites-part1.csv
|
||||
|
|
||||
|-- node_data/ # stores node feature data
|
||||
|-- paper-feat-part1.npy
|
||||
|-- paper-feat-part2.npy
|
||||
|-- paper-label-part1.npy
|
||||
|-- paper-label-part2.npy
|
||||
|-- paper-year-part1.npy
|
||||
|-- paper-year-part2.npy
|
||||
|
||||
All the data files are chunked into two parts, including the edges of each relation
|
||||
(e.g., writes, affiliates, cites) and node features. If the graph has edge features,
|
||||
they will be chunked into multiple files too. All ID data are stored in
|
||||
CSV (we will illustrate the contents soon) while node features are stored in
|
||||
numpy arrays.
|
||||
|
||||
The ``metadata.json`` stores all the metadata information such as file names
|
||||
and chunk sizes (e.g., number of nodes, number of edges).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
{
|
||||
"graph_name" : "MAG240M-LSC", # given graph name
|
||||
"node_type": ["author", "paper", "institution"],
|
||||
"num_nodes_per_chunk": [
|
||||
[61191556, 61191556], # number of author nodes per chunk
|
||||
[61191553, 61191552], # number of paper nodes per chunk
|
||||
[12861, 12860] # number of institution nodes per chunk
|
||||
],
|
||||
# The edge type name is a colon-joined string of source, edge, and destination type.
|
||||
"edge_type": [
|
||||
"author:writes:paper",
|
||||
"author:affiliated_with:institution",
|
||||
"paper:cites:paper"
|
||||
],
|
||||
"num_edges_per_chunk": [
|
||||
[193011360, 193011360], # number of author:writes:paper edges per chunk
|
||||
[22296293, 22296293], # number of author:affiliated_with:institution edges per chunk
|
||||
[648874463, 648874463] # number of paper:cites:paper edges per chunk
|
||||
],
|
||||
"edges" : {
|
||||
"author:writes:paper" : { # edge type
|
||||
"format" : {"name": "csv", "delimiter": " "},
|
||||
# The list of paths. Can be relative or absolute.
|
||||
"data" : ["edges/writes-part1.csv", "edges/writes-part2.csv"]
|
||||
},
|
||||
"author:affiliated_with:institution" : {
|
||||
"format" : {"name": "csv", "delimiter": " "},
|
||||
"data" : ["edges/affiliated_with-part1.csv", "edges/affiliated_with-part2.csv"]
|
||||
},
|
||||
"paper:cites:paper" : {
|
||||
"format" : {"name": "csv", "delimiter": " "},
|
||||
"data" : ["edges/cites-part1.csv", "edges/cites-part2.csv"]
|
||||
}
|
||||
},
|
||||
"node_data" : {
|
||||
"paper": { # node type
|
||||
"feat": { # feature key
|
||||
"format": {"name": "numpy"},
|
||||
"data": ["node_data/paper-feat-part1.npy", "node_data/paper-feat-part2.npy"]
|
||||
},
|
||||
"label": { # feature key
|
||||
"format": {"name": "numpy"},
|
||||
"data": ["node_data/paper-label-part1.npy", "node_data/paper-label-part2.npy"]
|
||||
},
|
||||
"year": { # feature key
|
||||
"format": {"name": "numpy"},
|
||||
"data": ["node_data/paper-year-part1.npy", "node_data/paper-year-part2.npy"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"edge_data" : {} # MAG240M-LSC does not have edge features
|
||||
}
|
||||
|
||||
There are three parts in ``metadata.json``:
|
||||
|
||||
* Graph schema information and chunk sizes, e.g., ``"node_type"`` , ``"num_nodes_per_chunk"``, etc.
|
||||
* Edge index data under key ``"edges"``.
|
||||
* Node/edge feature data under keys ``"node_data"`` and ``"edge_data"``.
|
||||
|
||||
The edge index files contain edges in the form of node ID pairs:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# writes-part1.csv
|
||||
0 0
|
||||
0 1
|
||||
0 20
|
||||
0 29
|
||||
0 1203
|
||||
...
|
||||
|
||||
Specification
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In general, a chunked graph data folder just needs a ``metadata.json`` and a
|
||||
bunch of data files. The folder structure in the MAG240M-LSC example is not a
|
||||
strict requirement as long as ``metadata.json`` contains valid file paths.
|
||||
|
||||
``metadata.json`` top-level keys:
|
||||
|
||||
* ``graph_name``: String. Unique name used by :class:`dgl.distributed.DistGraph`
|
||||
to load graph.
|
||||
* ``node_type``: List of string. Node type names.
|
||||
* ``num_nodes_per_chunk``: List of list of integer. For graphs with :math:`T` node
|
||||
types stored in :math:`P` chunks, the value contains :math:`T` integer lists.
|
||||
Each list contains :math:`P` integers, which specify the number of nodes
|
||||
in each chunk.
|
||||
* ``edge_type``: List of string. Edge type names in the form of
|
||||
``<source node type>:<relation>:<destination node type>``.
|
||||
* ``num_edges_per_chunk``: List of list of integer. For graphs with :math:`R` edge
|
||||
types stored in :math:`P` chunks, the value contains :math:`R` integer lists.
|
||||
Each list contains :math:`P` integers, which specify the number of edges
|
||||
in each chunk.
|
||||
* ``edges``: Dict of ``ChunkFileSpec``. Edge index files.
|
||||
Dictionary keys are edge type names in the form of
|
||||
``<source node type>:<relation>:<destination node type>``.
|
||||
* ``node_data``: Dict of ``ChunkFileSpec``. Data files that store node attributes
|
||||
could have arbitrary number of files regardless of ``num_parts``. Dictionary
|
||||
keys are node type names.
|
||||
* ``edge_data``: Dict of ``ChunkFileSpec``. Data files that store edge attributes
|
||||
could have arbitrary number of files regardless of ``num_parts``. Dictionary
|
||||
keys are edge type names in the form of
|
||||
``<source node type>:<relation>:<destination node type>``.
|
||||
|
||||
``ChunkFileSpec`` has two keys:
|
||||
|
||||
* ``format``: File format. Depending on the format ``name``, users can configure more
|
||||
details about how to parse each data file.
|
||||
- ``"csv"``: CSV file. Use the ``delimiter`` key to specify delimiter in use.
|
||||
- ``"numpy"``: NumPy array binary file created by :func:`numpy.save`.
|
||||
- ``"parquet"``: parquet table binary file created by :func:`pyarrow.parquet.write_table`.
|
||||
* ``data``: List of string. File path to each data chunk. Support absolute path.
|
||||
|
||||
Tips for making chunked graph data
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Depending on the raw data, the implementation could include:
|
||||
|
||||
* Construct graphs out of non-structured data such as texts or tabular data.
|
||||
* Augment or transform the input graph struture or features. E.g., adding reverse
|
||||
or self-loop edges, normalizing features, etc.
|
||||
* Chunk the input graph structure and features into multiple data files so that
|
||||
each one can fit in CPU RAM for subsequent preprocessing steps.
|
||||
|
||||
To avoid running into out-of-memory error, it is recommended to process graph
|
||||
structures and feature data separately. Processing one chunk at a time can also
|
||||
reduce the maximal runtime memory footprint. As an example, DGL provides a
|
||||
`tools/chunk_graph.py
|
||||
<https://github.com/dmlc/dgl/blob/master/tools/chunk_graph.py>`_ script that
|
||||
chunks an in-memory feature-less :class:`~dgl.DGLGraph` and feature tensors
|
||||
stored in :class:`numpy.memmap`.
|
||||
|
||||
|
||||
.. _guide-distributed-prep-partition:
|
||||
Step.1 Graph Partitioning
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This step reads the chunked graph data and calculates which partition each node
|
||||
should belong to. The results are saved in a set of *partition assignment files*.
|
||||
For example, to randomly partition MAG240M-LSC to two parts, run the
|
||||
``partition_algo/random_partition.py`` script in the ``tools`` folder:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python /my/repo/dgl/tools/partition_algo/random_partition.py
|
||||
--in_dir /mydata/MAG240M-LSC_chunked
|
||||
--out_dir /mydata/MAG240M-LSC_2parts
|
||||
--num_partitions 2
|
||||
|
||||
, which outputs files as follows:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
MAG240M-LSC_2parts/
|
||||
|-- paper.txt
|
||||
|-- author.txt
|
||||
|-- institution.txt
|
||||
|
||||
Each file stores the partition assignment of the corresponding node type.
|
||||
The contents are the partition ID of each node stored in lines, i.e., line i is
|
||||
the partition ID of node i.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# paper.txt
|
||||
0
|
||||
1
|
||||
1
|
||||
0
|
||||
0
|
||||
1
|
||||
0
|
||||
...
|
||||
|
||||
Despite its simplicity, random partitioning may result in frequent
|
||||
cross-machine communication. Check out chapter
|
||||
:ref:`guide-distributed-partition` for more advanced options.
|
||||
|
||||
Step.2 Data Dispatching
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
DGL provides a ``dispatch_data.py`` script to physically partition the data and
|
||||
dispatch partitions to each training machines. It will also convert the data
|
||||
once again to data objects that can be loaded by DGL training processes
|
||||
efficiently. The entire step can be further accelerated using multi-processing.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python /myrepo/dgl/tools/dispatch_data.py \
|
||||
--in-dir /mydata/MAG240M-LSC_chunked/ \
|
||||
--partitions-dir /mydata/MAG240M-LSC_2parts/ \
|
||||
--out-dir data/MAG_LSC_partitioned \
|
||||
--ip-config ip_config.txt
|
||||
|
||||
* ``--in-dir`` specifies the path to the folder of the input chunked graph data produced
|
||||
* ``--partitions-dir`` specifies the path to the partition assignment folder produced by Step.1.
|
||||
* ``--out-dir`` specifies the path to stored the data partition on each machine.
|
||||
* ``--ip-config`` specifies the IP configuration file of the cluster.
|
||||
|
||||
An example IP configuration file is as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
172.31.19.1
|
||||
172.31.23.205
|
||||
|
||||
As a counterpart of ``return_mapping=True`` in :func:`~dgl.distributed.partition_graph`, the
|
||||
:ref:`distributed partitioning pipeline <guide-distributed-preprocessing>`
|
||||
provides two arguments in ``dispatch_data.py`` to save the original node/edge IDs to disk.
|
||||
|
||||
* ``--save-orig-nids`` save original node IDs into files.
|
||||
* ``--save-orig-eids`` save original edge IDs into files.
|
||||
|
||||
Specifying the two options will create two files ``orig_nids.dgl`` and ``orig_eids.dgl``
|
||||
under each partition folder.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
data_root_dir/
|
||||
|-- graph_name.json # partition configuration file in JSON
|
||||
|-- part0/ # data for partition 0
|
||||
| |-- orig_nids.dgl # original node IDs
|
||||
| |-- orig_eids.dgl # original edge IDs
|
||||
| |-- ... # other data such as graph and node/edge feats
|
||||
|
|
||||
|-- part1/ # data for partition 1
|
||||
| |-- orig_nids.dgl
|
||||
| |-- orig_eids.dgl
|
||||
| |-- ...
|
||||
|
|
||||
|-- ... # data for other partitions
|
||||
|
||||
The two files store the original IDs as a dictionary of tensors, where keys are node/edge
|
||||
type names and values are ID tensors. Users can use the :func:`dgl.data.load_tensors`
|
||||
utility to load them:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Load the original IDs for the nodes in partition 0.
|
||||
orig_nids_0 = dgl.data.load_tensors('/path/to/data/part0/orig_nids.dgl')
|
||||
# Get the original node IDs for node type 'user'
|
||||
user_orig_nids_0 = orig_nids_0['user']
|
||||
|
||||
# Load the original IDs for the edges in partition 0.
|
||||
orig_eids_0 = dgl.data.load_tensors('/path/to/data/part0/orig_eids.dgl')
|
||||
# Get the original edge IDs for edge type 'like'
|
||||
like_orig_eids_0 = orig_nids_0['like']
|
||||
|
||||
During data dispatching, DGL assumes that the combined CPU RAM of the cluster
|
||||
is able to hold the entire graph data. Node ownership is determined by the result
|
||||
of partitioning algorithm where as for edges the owner of the destination node
|
||||
also owns the edge as well.
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
.. _guide-distributed-tools:
|
||||
|
||||
7.2 Tools for launching distributed training/inference
|
||||
------------------------------------------------------
|
||||
|
||||
DGL provides a launching script ``launch.py`` under
|
||||
`dgl/tools <https://github.com/dmlc/dgl/tree/master/tools>`__ to launch a distributed
|
||||
training job in a cluster. This script makes the following assumptions:
|
||||
|
||||
* The partitioned data and the training script have been provisioned to the cluster or
|
||||
a shared storage (e.g., NFS) accessible to all the worker machines.
|
||||
* The machine that invokes ``launch.py`` has passwordless ssh access
|
||||
to all other machines. The launching machine must be one of the worker machines.
|
||||
|
||||
Below shows an example of launching a distributed training job in a cluster.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
python3 tools/launch.py \
|
||||
--workspace /my/workspace/ \
|
||||
--num_trainers 2 \
|
||||
--num_samplers 4 \
|
||||
--num_servers 1 \
|
||||
--part_config data/mygraph.json \
|
||||
--ip_config ip_config.txt \
|
||||
"python3 my_train_script.py"
|
||||
|
||||
The argument specifies the workspace path, where to find the partition metadata JSON
|
||||
and machine IP configurations, how many trainer, sampler, and server processes to be launched
|
||||
on each machine. The last argument is the command to launch which is usually the
|
||||
model training/evaluation script.
|
||||
|
||||
Each line of ``ip_config.txt`` is the IP address of a machine in the cluster.
|
||||
Optionally, the IP address can be followed by a network port (default is ``30050``).
|
||||
A typical example is as follows:
|
||||
|
||||
.. code:: none
|
||||
|
||||
172.31.19.1
|
||||
172.31.23.205
|
||||
172.31.29.175
|
||||
172.31.16.98
|
||||
|
||||
The workspace specified in the launch script is the working directory in the
|
||||
machines, which contains the training script, the IP configuration file, the
|
||||
partition configuration file as well as the graph partitions. All paths of the
|
||||
files should be specified as relative paths to the workspace.
|
||||
|
||||
The launch script creates a specified number of training jobs
|
||||
(``--num_trainers``) on each machine. In addition, users need to specify the
|
||||
number of sampler processes for each trainer (``--num_samplers``).
|
||||
@@ -0,0 +1,123 @@
|
||||
.. _guide-distributed:
|
||||
|
||||
Chapter 7: Distributed Training
|
||||
=====================================
|
||||
|
||||
:ref:`(中文版) <guide_cn-distributed>`
|
||||
|
||||
.. note::
|
||||
|
||||
Distributed training is only available for PyTorch backend.
|
||||
|
||||
DGL adopts a fully distributed approach that distributes both data and computation
|
||||
across a collection of computation resources. In the context of this section, we
|
||||
will assume a cluster setting (i.e., a group of machines). DGL partitions a graph
|
||||
into subgraphs and each machine in a cluster is responsible for one subgraph (partition).
|
||||
DGL runs an identical training script on all machines in the cluster to parallelize
|
||||
the computation and runs servers on the same machines to serve partitioned data to the trainers.
|
||||
|
||||
For the training script, DGL provides distributed APIs that are similar to the ones for
|
||||
mini-batch training. This makes distributed training require only small code modifications
|
||||
from mini-batch training on a single machine. Below shows an example of training GraphSage
|
||||
in a distributed fashion. The notable code modifications are:
|
||||
1) initialization of DGL's distributed module, 2) create a distributed graph object, and
|
||||
3) split the training set and calculate the nodes for the local process.
|
||||
The rest of the code, including sampler creation, model definition, training loops
|
||||
are the same as :ref:`mini-batch training <guide-minibatch>`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
from dgl.dataloading import NeighborSampler
|
||||
from dgl.distributed import DistGraph, DistDataLoader, node_split
|
||||
import torch as th
|
||||
|
||||
# initialize distributed contexts
|
||||
dgl.distributed.initialize('ip_config.txt')
|
||||
th.distributed.init_process_group(backend='gloo')
|
||||
# load distributed graph
|
||||
g = DistGraph('graph_name', 'part_config.json')
|
||||
pb = g.get_partition_book()
|
||||
# get training workload, i.e., training node IDs
|
||||
train_nid = node_split(g.ndata['train_mask'], pb, force_even=True)
|
||||
|
||||
|
||||
# Create sampler
|
||||
sampler = NeighborSampler(g, [10,25],
|
||||
dgl.distributed.sample_neighbors,
|
||||
device)
|
||||
|
||||
dataloader = DistDataLoader(
|
||||
dataset=train_nid.numpy(),
|
||||
batch_size=batch_size,
|
||||
collate_fn=sampler.sample_blocks,
|
||||
shuffle=True,
|
||||
drop_last=False)
|
||||
|
||||
# Define model and optimizer
|
||||
model = SAGE(in_feats, num_hidden, n_classes, num_layers, F.relu, dropout)
|
||||
model = th.nn.parallel.DistributedDataParallel(model)
|
||||
loss_fcn = nn.CrossEntropyLoss()
|
||||
optimizer = optim.Adam(model.parameters(), lr=args.lr)
|
||||
|
||||
# training loop
|
||||
for epoch in range(args.num_epochs):
|
||||
with model.join():
|
||||
for step, blocks in enumerate(dataloader):
|
||||
batch_inputs, batch_labels = load_subtensor(g, blocks[0].srcdata[dgl.NID],
|
||||
blocks[-1].dstdata[dgl.NID])
|
||||
batch_pred = model(blocks, batch_inputs)
|
||||
loss = loss_fcn(batch_pred, batch_labels)
|
||||
optimizer.zero_grad()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
DGL implements a few distributed components to support distributed training. The figure below
|
||||
shows the components and their interactions.
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/distributed.png
|
||||
:alt: Imgur
|
||||
|
||||
Specifically, DGL's distributed training has three types of interacting processes:
|
||||
*server*, *sampler* and *trainer*.
|
||||
|
||||
* **Servers** store graph partitions which includes both structure data and node/edge
|
||||
features. They provide services such as sampling, getting or updating node/edge
|
||||
features. Note that each machine may run multiple server processes simultaneously
|
||||
to increase service throughput. One of them is *main server* in charge of data
|
||||
loading and sharing data via shared memory with *backup servers* that provide
|
||||
services.
|
||||
* **Sampler processes** interact with the servers and sample nodes and edges to
|
||||
generate mini-batches for training.
|
||||
* **Trainers** are in charge of training networks on mini-batches. They utilize
|
||||
APIs such as :class:`~dgl.distributed.DistGraph` to access partitioned graph data,
|
||||
:class:`~dgl.distributed.DistEmbedding` and :class:`~dgl.distributed.DistTensor` to access
|
||||
node/edge features/embeddings and :class:`~dgl.distributed.DistDataLoader` to interact
|
||||
with samplers to get mini-batches. Trainers communicate gradients among each other
|
||||
using PyTorch's native ``DistributedDataParallel`` paradigm.
|
||||
|
||||
Besides Python APIs, DGL also provides `tools <https://github.com/dmlc/dgl/tree/master/tools>`__
|
||||
for provisioning graph data and processes to the entire cluster.
|
||||
|
||||
Having the distributed components in mind, the rest of the section will cover
|
||||
the following distributed components:
|
||||
|
||||
* :ref:`guide-distributed-preprocessing`
|
||||
* :ref:`guide-distributed-tools`
|
||||
* :ref:`guide-distributed-apis`
|
||||
|
||||
For more advanced users who are interested in more details:
|
||||
|
||||
* :ref:`guide-distributed-partition`
|
||||
* :ref:`guide-distributed-hetero`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
distributed-preprocessing
|
||||
distributed-tools
|
||||
distributed-apis
|
||||
distributed-partition
|
||||
distributed-hetero
|
||||
@@ -0,0 +1,36 @@
|
||||
.. _guide-graph-basic:
|
||||
|
||||
1.1 Some Basic Definitions about Graphs (Graphs 101)
|
||||
----------------------------------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-basic>`
|
||||
|
||||
A graph :math:`G=(V, E)` is a structure used to represent entities and their relations. It consists of
|
||||
two sets -- the set of nodes :math:`V` (also called vertices) and the set of edges :math:`E` (also called
|
||||
arcs). An edge :math:`(u, v) \in E` connecting a pair of nodes :math:`u` and :math:`v` indicates that there is a
|
||||
relation between them. The relation can either be undirected, e.g., capturing symmetric
|
||||
relations between nodes, or directed, capturing asymmetric relations. For example, if a
|
||||
graph is used to model the friendships relations of people in a social network, then the edges
|
||||
will be undirected as friendship is mutual; however, if the graph is used to model how people
|
||||
follow each other on Twitter, then the edges are directed. Depending on the edges'
|
||||
directionality, a graph can be *directed* or *undirected*.
|
||||
|
||||
Graphs can be *weighted* or *unweighted*. In a weighted graph, each edge is associated with a
|
||||
scalar weight. For example, such weights might represent lengths or connectivity strengths.
|
||||
|
||||
Graphs can also be either *homogeneous* or *heterogeneous*. In a homogeneous graph, all
|
||||
the nodes represent instances of the same type and all the edges represent relations of the
|
||||
same type. For instance, a social network is a graph consisting of people and their
|
||||
connections, representing the same entity type.
|
||||
|
||||
In contrast, in a heterogeneous graph, the nodes and edges can be of different types. For
|
||||
instance, the graph encoding a marketplace will have buyer, seller, and product nodes that
|
||||
are connected via wants-to-buy, has-bought, is-customer-of, and is-selling edges. The
|
||||
bipartite graph is a special, commonly-used type of heterogeneous graph, where edges
|
||||
exist between nodes of two different types. For example, in a recommender system, one can
|
||||
use a bipartite graph to represent the interactions between users and items. For working
|
||||
with heterogeneous graphs in DGL, see :ref:`guide-graph-heterogeneous`.
|
||||
|
||||
Multigraphs are graphs that can have multiple (directed) edges between the same pair of nodes,
|
||||
including self loops. For instance, two authors can coauthor a paper in different years,
|
||||
resulting in edges with different features.
|
||||
@@ -0,0 +1,117 @@
|
||||
.. _guide-graph-external:
|
||||
|
||||
1.4 Creating Graphs from External Sources
|
||||
-----------------------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-external>`
|
||||
|
||||
The options to construct a :class:`~dgl.DGLGraph` from external sources include:
|
||||
|
||||
- Conversion from external python libraries for graphs and sparse matrices (NetworkX and SciPy).
|
||||
- Loading graphs from disk.
|
||||
|
||||
The section does not cover functions that generate graphs by transforming from other
|
||||
graphs. See the API reference manual for an overview of them.
|
||||
|
||||
Creating Graphs from External Libraries
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The following code snippet is an example for creating a graph from a SciPy sparse matrix and a NetworkX graph.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> import scipy.sparse as sp
|
||||
>>> spmat = sp.rand(100, 100, density=0.05) # 5% nonzero entries
|
||||
>>> dgl.from_scipy(spmat) # from SciPy
|
||||
Graph(num_nodes=100, num_edges=500,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
>>> import networkx as nx
|
||||
>>> nx_g = nx.path_graph(5) # a chain 0-1-2-3-4
|
||||
>>> dgl.from_networkx(nx_g) # from networkx
|
||||
Graph(num_nodes=5, num_edges=8,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
Note that when constructing from the `nx.path_graph(5)`, the resulting :class:`~dgl.DGLGraph` has 8
|
||||
edges instead of 4. This is because `nx.path_graph(5)` constructs an undirected NetworkX graph
|
||||
:class:`networkx.Graph` while a :class:`~dgl.DGLGraph` is always directed. In converting an undirected
|
||||
NetworkX graph into a :class:`~dgl.DGLGraph`, DGL internally converts undirected edges to two directed edges.
|
||||
Using directed NetworkX graphs :class:`networkx.DiGraph` can avoid such behavior.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> nxg = nx.DiGraph([(2, 1), (1, 2), (2, 3), (0, 0)])
|
||||
>>> dgl.from_networkx(nxg)
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
.. note::
|
||||
|
||||
DGL internally converts SciPy matrices and NetworkX graphs to tensors to construct graphs.
|
||||
Hence, these construction methods are not meant for performance critical parts.
|
||||
|
||||
See APIs: :func:`dgl.from_scipy`, :func:`dgl.from_networkx`.
|
||||
|
||||
Loading Graphs from Disk
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are many data formats for storing graphs and it isn't possible to enumerate every option.
|
||||
Thus, this section only gives some general pointers on certain common ones.
|
||||
|
||||
Comma Separated Values (CSV)
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
One very common format is CSV, which stores nodes, edges, and their features in a tabular format:
|
||||
|
||||
.. table:: nodes.csv
|
||||
|
||||
+-----------+
|
||||
|age, title |
|
||||
+===========+
|
||||
|43, 1 |
|
||||
+-----------+
|
||||
|23, 3 |
|
||||
+-----------+
|
||||
|... |
|
||||
+-----------+
|
||||
|
||||
.. table:: edges.csv
|
||||
|
||||
+-----------------+
|
||||
|src, dst, weight |
|
||||
+=================+
|
||||
|0, 1, 0.4 |
|
||||
+-----------------+
|
||||
|0, 3, 0.9 |
|
||||
+-----------------+
|
||||
|... |
|
||||
+-----------------+
|
||||
|
||||
There are known Python libraries (e.g. pandas) for loading this type of data into python
|
||||
objects (e.g., :class:`numpy.ndarray`), which can then be used to construct a DGLGraph. If the
|
||||
backend framework also provides utilities to save/load tensors from disk (e.g., :func:`torch.save`,
|
||||
:func:`torch.load`), one can follow the same principle to build a graph.
|
||||
|
||||
See also: `Tutorial for loading a Karate Club Network from edge pairs CSV <https://github.com/dglai/WWW20-Hands-on-Tutorial/blob/master/basic_tasks/1_load_data.ipynb>`_.
|
||||
|
||||
JSON/GML Format
|
||||
"""""""""""""""
|
||||
|
||||
Though not particularly fast, NetworkX provides many utilities to parse
|
||||
`a variety of data formats <https://networkx.github.io/documentation/stable/reference/readwrite/index.html>`_
|
||||
which indirectly allows DGL to create graphs from these sources.
|
||||
|
||||
DGL Binary Format
|
||||
"""""""""""""""""
|
||||
|
||||
DGL provides APIs to save and load graphs from disk stored in binary format. Apart from the
|
||||
graph structure, the APIs also handle feature data and graph-level label data. DGL also
|
||||
supports checkpointing graphs directly to S3 or HDFS. The reference manual provides more
|
||||
details about the usage.
|
||||
|
||||
See APIs: :func:`dgl.save_graphs`, :func:`dgl.load_graphs`.
|
||||
@@ -0,0 +1,65 @@
|
||||
.. _guide-graph-feature:
|
||||
|
||||
1.3 Node and Edge Features
|
||||
--------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-feature>`
|
||||
|
||||
The nodes and edges of a :class:`~dgl.DGLGraph` can have several user-defined named features for
|
||||
storing graph-specific properties of the nodes and edges. These features can be accessed
|
||||
via the :py:attr:`~dgl.DGLGraph.ndata` and :py:attr:`~dgl.DGLGraph.edata` interface. For example, the following code creates two node
|
||||
features (named ``'x'`` and ``'y'`` in line 8 and 15) and one edge feature (named ``'x'`` in line 9).
|
||||
|
||||
.. code-block:: python
|
||||
:linenos:
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> g = dgl.graph(([0, 0, 1, 5], [1, 2, 2, 0])) # 6 nodes, 4 edges
|
||||
>>> g
|
||||
Graph(num_nodes=6, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
>>> g.ndata['x'] = th.ones(g.num_nodes(), 3) # node feature of length 3
|
||||
>>> g.edata['x'] = th.ones(g.num_edges(), dtype=th.int32) # scalar integer feature
|
||||
>>> g
|
||||
Graph(num_nodes=6, num_edges=4,
|
||||
ndata_schemes={'x' : Scheme(shape=(3,), dtype=torch.float32)}
|
||||
edata_schemes={'x' : Scheme(shape=(,), dtype=torch.int32)})
|
||||
>>> # different names can have different shapes
|
||||
>>> g.ndata['y'] = th.randn(g.num_nodes(), 5)
|
||||
>>> g.ndata['x'][1] # get node 1's feature
|
||||
tensor([1., 1., 1.])
|
||||
>>> g.edata['x'][th.tensor([0, 3])] # get features of edge 0 and 3
|
||||
tensor([1, 1], dtype=torch.int32)
|
||||
|
||||
Important facts about the :py:attr:`~dgl.DGLGraph.ndata`/:py:attr:`~dgl.DGLGraph.edata` interface:
|
||||
|
||||
- Only features of numerical types (e.g., float, double, and int) are allowed. They can
|
||||
be scalars, vectors or multi-dimensional tensors.
|
||||
- Each node feature has a unique name and each edge feature has a unique name.
|
||||
The features of nodes and edges can have the same name. (e.g., 'x' in the above example).
|
||||
- A feature is created via tensor assignment, which assigns a feature to each
|
||||
node/edge in the graph. The leading dimension of that tensor must be equal to the
|
||||
number of nodes/edges in the graph. You cannot assign a feature to a subset of the
|
||||
nodes/edges in the graph.
|
||||
- Features of the same name must have the same dimensionality and data type.
|
||||
- The feature tensor is in row-major layout -- each row-slice stores the feature of one
|
||||
node or edge (e.g., see lines 16 and 18 in the above example).
|
||||
|
||||
For weighted graphs, one can store the weights as an edge feature as below.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> # edges 0->1, 0->2, 0->3, 1->3
|
||||
>>> edges = th.tensor([0, 0, 0, 1]), th.tensor([1, 2, 3, 3])
|
||||
>>> weights = th.tensor([0.1, 0.6, 0.9, 0.7]) # weight of each edge
|
||||
>>> g = dgl.graph(edges)
|
||||
>>> g.edata['w'] = weights # give it a name 'w'
|
||||
>>> g
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={'w' : Scheme(shape=(,), dtype=torch.float32)})
|
||||
|
||||
|
||||
See APIs: :py:attr:`~dgl.DGLGraph.ndata`, :py:attr:`~dgl.DGLGraph.edata`.
|
||||
@@ -0,0 +1,47 @@
|
||||
.. _guide-graph-gpu:
|
||||
|
||||
1.6 Using DGLGraph on a GPU
|
||||
---------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-gpu>`
|
||||
|
||||
One can create a :class:`~dgl.DGLGraph` on a GPU by passing two GPU tensors during construction.
|
||||
Another approach is to use the :func:`~dgl.DGLGraph.to` API to copy a :class:`~dgl.DGLGraph` to a GPU, which
|
||||
copies the graph structure as well as the feature data to the given device.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> u, v = th.tensor([0, 1, 2]), th.tensor([2, 3, 4])
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> g.ndata['x'] = th.randn(5, 3) # original feature is on CPU
|
||||
>>> g.device
|
||||
device(type='cpu')
|
||||
>>> cuda_g = g.to('cuda:0') # accepts any device objects from backend framework
|
||||
>>> cuda_g.device
|
||||
device(type='cuda', index=0)
|
||||
>>> cuda_g.ndata['x'].device # feature data is copied to GPU too
|
||||
device(type='cuda', index=0)
|
||||
|
||||
>>> # A graph constructed from GPU tensors is also on GPU
|
||||
>>> u, v = u.to('cuda:0'), v.to('cuda:0')
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> g.device
|
||||
device(type='cuda', index=0)
|
||||
|
||||
Any operations involving a GPU graph are performed on a GPU. Thus, they require all
|
||||
tensor arguments to be placed on GPU already and the results (graph or tensor) will be on
|
||||
GPU too. Furthermore, a GPU graph only accepts feature data on a GPU.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> cuda_g.in_degrees()
|
||||
tensor([0, 0, 1, 1, 1], device='cuda:0')
|
||||
>>> cuda_g.in_edges([2, 3, 4]) # ok for non-tensor type arguments
|
||||
(tensor([0, 1, 2], device='cuda:0'), tensor([2, 3, 4], device='cuda:0'))
|
||||
>>> cuda_g.in_edges(th.tensor([2, 3, 4]).to('cuda:0')) # tensor type must be on GPU
|
||||
(tensor([0, 1, 2], device='cuda:0'), tensor([2, 3, 4], device='cuda:0'))
|
||||
>>> cuda_g.ndata['h'] = th.randn(5, 4) # ERROR! feature must be on GPU too!
|
||||
DGLError: Cannot assign node feature "h" on device cpu to a graph on device
|
||||
cuda:0. Call DGLGraph.to() to copy the graph to the same device.
|
||||
@@ -0,0 +1,96 @@
|
||||
.. _guide-graph-graphs-nodes-edges:
|
||||
|
||||
1.2 Graphs, Nodes, and Edges
|
||||
----------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-graphs-nodes-edges>`
|
||||
|
||||
DGL represents each node by a unique integer, called its node ID, and each edge by a pair
|
||||
of integers corresponding to the IDs of its end nodes. DGL assigns to each edge a unique
|
||||
integer, called its **edge ID**, based on the order in which it was added to the graph. The
|
||||
numbering of node and edge IDs starts from 0. In DGL, all the edges are directed, and an
|
||||
edge :math:`(u, v)` indicates that the direction goes from node :math:`u` to node :math:`v`.
|
||||
|
||||
To specify multiple nodes, DGL uses a 1-D integer tensor (i.e., PyTorch's tensor,
|
||||
TensorFlow's Tensor, or MXNet's ndarray) of node IDs. DGL calls this format "node-tensors".
|
||||
To specify multiple edges, it uses a tuple of node-tensors :math:`(U, V)`. :math:`(U[i], V[i])`
|
||||
decides an edge from :math:`U[i]` to :math:`V[i]`.
|
||||
|
||||
One way to create a :class:`~dgl.DGLGraph` is to use the :func:`dgl.graph` method, which takes
|
||||
as input a set of edges. DGL also supports creating graphs from other data sources, see :ref:`guide-graph-external`.
|
||||
|
||||
The following code snippet uses the :func:`dgl.graph` method to create a :class:`~dgl.DGLGraph`
|
||||
corresponding to the four-node graph shown below and illustrates some of its APIs for
|
||||
querying the graph's structure.
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/user_guide_graphch_1.png
|
||||
:height: 200px
|
||||
:width: 300px
|
||||
:align: center
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
|
||||
>>> # edges 0->1, 0->2, 0->3, 1->3
|
||||
>>> u, v = th.tensor([0, 0, 0, 1]), th.tensor([1, 2, 3, 3])
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> print(g) # number of nodes are inferred from the max node IDs in the given edges
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
>>> # Node IDs
|
||||
>>> print(g.nodes())
|
||||
tensor([0, 1, 2, 3])
|
||||
>>> # Edge end nodes
|
||||
>>> print(g.edges())
|
||||
(tensor([0, 0, 0, 1]), tensor([1, 2, 3, 3]))
|
||||
>>> # Edge end nodes and edge IDs
|
||||
>>> print(g.edges(form='all'))
|
||||
(tensor([0, 0, 0, 1]), tensor([1, 2, 3, 3]), tensor([0, 1, 2, 3]))
|
||||
|
||||
>>> # If the node with the largest ID is isolated (meaning no edges),
|
||||
>>> # then one needs to explicitly set the number of nodes
|
||||
>>> g = dgl.graph((u, v), num_nodes=8)
|
||||
|
||||
For an undirected graph, one needs to create edges for both directions. :func:`dgl.to_bidirected`
|
||||
can be helpful in this case, which converts a graph into a new one with edges for both directions.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> bg = dgl.to_bidirected(g)
|
||||
>>> bg.edges()
|
||||
(tensor([0, 0, 0, 1, 1, 2, 3, 3]), tensor([1, 2, 3, 0, 3, 0, 0, 1]))
|
||||
|
||||
.. note::
|
||||
|
||||
Tensor types are generally preferred throughout DGL APIs due to their efficient internal
|
||||
storage in C and explicit data type and device context information. However, most DGL APIs
|
||||
do support python iterable (e.g., list) or numpy.ndarray as arguments for quick prototyping.
|
||||
|
||||
DGL can use either :math:`32`- or :math:`64`-bit integers to store the node and edge IDs. The data types for
|
||||
the node and edge IDs should be the same. By using :math:`64` bits, DGL can handle graphs with
|
||||
up to :math:`2^{63} - 1` nodes or edges. However, if a graph contains less than :math:`2^{31} - 1` nodes or edges,
|
||||
one should use :math:`32`-bit integers as it leads to better speed and requires less memory.
|
||||
DGL provides methods for making such conversions. See below for an example.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> edges = th.tensor([2, 5, 3]), th.tensor([3, 5, 0]) # edges 2->3, 5->5, 3->0
|
||||
>>> g64 = dgl.graph(edges) # DGL uses int64 by default
|
||||
>>> print(g64.idtype)
|
||||
torch.int64
|
||||
>>> g32 = dgl.graph(edges, idtype=th.int32) # create a int32 graph
|
||||
>>> g32.idtype
|
||||
torch.int32
|
||||
>>> g64_2 = g32.long() # convert to int64
|
||||
>>> g64_2.idtype
|
||||
torch.int64
|
||||
>>> g32_2 = g64.int() # convert to int32
|
||||
>>> g32_2.idtype
|
||||
torch.int32
|
||||
|
||||
See APIs: :func:`dgl.graph`, :func:`dgl.DGLGraph.nodes`, :func:`dgl.DGLGraph.edges`, :func:`dgl.to_bidirected`,
|
||||
:func:`dgl.DGLGraph.int`, :func:`dgl.DGLGraph.long`, and :py:attr:`dgl.DGLGraph.idtype`.
|
||||
@@ -0,0 +1,282 @@
|
||||
.. _guide-graph-heterogeneous:
|
||||
|
||||
1.5 Heterogeneous Graphs
|
||||
------------------------
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph-heterogeneous>`
|
||||
|
||||
A heterogeneous graph can have nodes and edges of different types. Nodes/Edges of
|
||||
different types have independent ID space and feature storage. For example in the figure below, the
|
||||
user and game node IDs both start from zero and they have different features.
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/user_guide_graphch_2.png
|
||||
|
||||
An example heterogeneous graph with two types of nodes (user and game) and two types of edges (follows and plays).
|
||||
|
||||
Creating a Heterogeneous Graph
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
In DGL, a heterogeneous graph (heterograph for short) is specified with a series of graphs as below, one per
|
||||
relation. Each relation is a string triplet ``(source node type, edge type, destination node type)``.
|
||||
Since relations disambiguate the edge types, DGL calls them canonical edge types.
|
||||
|
||||
The following code snippet is an example for creating a heterogeneous graph in DGL.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
|
||||
>>> # Create a heterograph with 3 node types and 3 edges types.
|
||||
>>> graph_data = {
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... }
|
||||
>>> g = dgl.heterograph(graph_data)
|
||||
>>> g.ntypes
|
||||
['disease', 'drug', 'gene']
|
||||
>>> g.etypes
|
||||
['interacts', 'interacts', 'treats']
|
||||
>>> g.canonical_etypes
|
||||
[('drug', 'interacts', 'drug'),
|
||||
('drug', 'interacts', 'gene'),
|
||||
('drug', 'treats', 'disease')]
|
||||
|
||||
Note that homogeneous and bipartite graphs are just special heterogeneous graphs with one
|
||||
relation.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # A homogeneous graph
|
||||
>>> dgl.heterograph({('node_type', 'edge_type', 'node_type'): (u, v)})
|
||||
>>> # A bipartite graph
|
||||
>>> dgl.heterograph({('source_type', 'edge_type', 'destination_type'): (u, v)})
|
||||
|
||||
The *metagraph* associated with a heterogeneous graph is the schema of the graph. It specifies
|
||||
type constraints on the sets of nodes and edges between the nodes. A node :math:`u` in a metagraph
|
||||
corresponds to a node type in the associated heterograph. An edge :math:`(u, v)` in a metagraph indicates that
|
||||
there are edges from nodes of type :math:`u` to nodes of type :math:`v` in the associated heterograph.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g
|
||||
Graph(num_nodes={'disease': 3, 'drug': 3, 'gene': 4},
|
||||
num_edges={('drug', 'interacts', 'drug'): 2,
|
||||
('drug', 'interacts', 'gene'): 2,
|
||||
('drug', 'treats', 'disease'): 1},
|
||||
metagraph=[('drug', 'drug', 'interacts'),
|
||||
('drug', 'gene', 'interacts'),
|
||||
('drug', 'disease', 'treats')])
|
||||
>>> g.metagraph().edges()
|
||||
OutMultiEdgeDataView([('drug', 'drug'), ('drug', 'gene'), ('drug', 'disease')])
|
||||
|
||||
See APIs: :func:`dgl.heterograph`, :py:attr:`~dgl.DGLGraph.ntypes`, :py:attr:`~dgl.DGLGraph.etypes`,
|
||||
:py:attr:`~dgl.DGLGraph.canonical_etypes`, :py:attr:`~dgl.DGLGraph.metagraph`.
|
||||
|
||||
Working with Multiple Types
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
When multiple node/edge types are introduced, users need to specify the particular
|
||||
node/edge type when invoking a DGLGraph API for type-specific information. In addition,
|
||||
nodes/edges of different types have separate IDs.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # Get the number of all nodes in the graph
|
||||
>>> g.num_nodes()
|
||||
10
|
||||
>>> # Get the number of drug nodes
|
||||
>>> g.num_nodes('drug')
|
||||
3
|
||||
>>> # Nodes of different types have separate IDs,
|
||||
>>> # hence not well-defined without a type specified
|
||||
>>> g.nodes()
|
||||
DGLError: Node type name must be specified if there are more than one node types.
|
||||
>>> g.nodes('drug')
|
||||
tensor([0, 1, 2])
|
||||
|
||||
To set/get features for a specific node/edge type, DGL provides two new types of syntax --
|
||||
`g.nodes['node_type'].data['feat_name']` and `g.edges['edge_type'].data['feat_name']`.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # Set/get feature 'hv' for nodes of type 'drug'
|
||||
>>> g.nodes['drug'].data['hv'] = th.ones(3, 1)
|
||||
>>> g.nodes['drug'].data['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.]])
|
||||
>>> # Set/get feature 'he' for edge of type 'treats'
|
||||
>>> g.edges['treats'].data['he'] = th.zeros(1, 1)
|
||||
>>> g.edges['treats'].data['he']
|
||||
tensor([[0.]])
|
||||
|
||||
If the graph only has one node/edge type, there is no need to specify the node/edge type.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'is similar', 'drug'): (th.tensor([0, 1]), th.tensor([2, 3]))
|
||||
... })
|
||||
>>> g.nodes()
|
||||
tensor([0, 1, 2, 3])
|
||||
>>> # To set/get feature with a single type, no need to use the new syntax
|
||||
>>> g.ndata['hv'] = th.ones(4, 1)
|
||||
|
||||
.. note::
|
||||
|
||||
When the edge type uniquely determines the types of source and destination nodes, one
|
||||
can just use one string instead of a string triplet to specify the edge type. For example, for a
|
||||
heterograph with two relations ``('user', 'plays', 'game')`` and ``('user', 'likes', 'game')``, it
|
||||
is safe to just use ``'plays'`` or ``'likes'`` to refer to the two relations.
|
||||
|
||||
Loading Heterographs from Disk
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Comma Separated Values (CSV)
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
A common way to store a heterograph is to store nodes and edges of different types in different CSV files.
|
||||
An example is as follows.
|
||||
|
||||
.. code::
|
||||
|
||||
# data folder
|
||||
data/
|
||||
|-- drug.csv # drug nodes
|
||||
|-- gene.csv # gene nodes
|
||||
|-- disease.csv # disease nodes
|
||||
|-- drug-interact-drug.csv # drug-drug interaction edges
|
||||
|-- drug-interact-gene.csv # drug-gene interaction edges
|
||||
|-- drug-treat-disease.csv # drug-treat-disease edges
|
||||
|
||||
Similar to the case of homogeneous graphs, one can use packages like Pandas to parse
|
||||
CSV files into numpy arrays or framework tensors, build a relation dictionary and
|
||||
construct a heterograph from that. The approach also applies to other popular formats like
|
||||
GML/JSON.
|
||||
|
||||
DGL Binary Format
|
||||
"""""""""""""""""
|
||||
|
||||
DGL provides :func:`dgl.save_graphs` and :func:`dgl.load_graphs` respectively for saving
|
||||
heterogeneous graphs in binary format and loading them from binary format.
|
||||
|
||||
Edge Type Subgraph
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
One can create a subgraph of a heterogeneous graph by specifying the relations to retain, with
|
||||
features copied if any.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... })
|
||||
>>> g.nodes['drug'].data['hv'] = th.ones(3, 1)
|
||||
|
||||
>>> # Retain relations ('drug', 'interacts', 'drug') and ('drug', 'treats', 'disease')
|
||||
>>> # All nodes for 'drug' and 'disease' will be retained
|
||||
>>> eg = dgl.edge_type_subgraph(g, [('drug', 'interacts', 'drug'),
|
||||
... ('drug', 'treats', 'disease')])
|
||||
>>> eg
|
||||
Graph(num_nodes={'disease': 3, 'drug': 3},
|
||||
num_edges={('drug', 'interacts', 'drug'): 2, ('drug', 'treats', 'disease'): 1},
|
||||
metagraph=[('drug', 'drug', 'interacts'), ('drug', 'disease', 'treats')])
|
||||
>>> # The associated features will be copied as well
|
||||
>>> eg.nodes['drug'].data['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.]])
|
||||
|
||||
Converting Heterogeneous Graphs to Homogeneous Graphs
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Heterographs provide a clean interface for managing nodes/edges of different types and
|
||||
their associated features. This is particularly helpful when:
|
||||
|
||||
1. The features for nodes/edges of different types have different data types or sizes.
|
||||
2. We want to apply different operations to nodes/edges of different types.
|
||||
|
||||
If the above conditions do not hold and one does not want to distinguish node/edge types in
|
||||
modeling, then DGL allows converting a heterogeneous graph to a homogeneous graph with :func:`dgl.DGLGraph.to_homogeneous` API.
|
||||
It proceeds as follows:
|
||||
|
||||
1. Relabels nodes/edges of all types using consecutive integers starting from 0
|
||||
2. Merges the features across node/edge types specified by the user.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))})
|
||||
>>> g.nodes['drug'].data['hv'] = th.zeros(3, 1)
|
||||
>>> g.nodes['disease'].data['hv'] = th.ones(3, 1)
|
||||
>>> g.edges['interacts'].data['he'] = th.zeros(2, 1)
|
||||
>>> g.edges['treats'].data['he'] = th.zeros(1, 2)
|
||||
|
||||
>>> # By default, it does not merge any features
|
||||
>>> hg = dgl.to_homogeneous(g)
|
||||
>>> 'hv' in hg.ndata
|
||||
False
|
||||
|
||||
>>> # Copy edge features
|
||||
>>> # For feature copy, it expects features to have
|
||||
>>> # the same size and dtype across node/edge types
|
||||
>>> hg = dgl.to_homogeneous(g, edata=['he'])
|
||||
DGLError: Cannot concatenate column ‘he’ with shape Scheme(shape=(2,), dtype=torch.float32) and shape Scheme(shape=(1,), dtype=torch.float32)
|
||||
|
||||
>>> # Copy node features
|
||||
>>> hg = dgl.to_homogeneous(g, ndata=['hv'])
|
||||
>>> hg.ndata['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.],
|
||||
[0.],
|
||||
[0.],
|
||||
[0.]])
|
||||
|
||||
The original node/edge types and type-specific IDs are stored in :py:attr:`~dgl.DGLGraph.ndata` and :py:attr:`~dgl.DGLGraph.edata`.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # Order of node types in the heterograph
|
||||
>>> g.ntypes
|
||||
['disease', 'drug']
|
||||
>>> # Original node types
|
||||
>>> hg.ndata[dgl.NTYPE]
|
||||
tensor([0, 0, 0, 1, 1, 1])
|
||||
>>> # Original type-specific node IDs
|
||||
>>> hg.ndata[dgl.NID]
|
||||
tensor([0, 1, 2, 0, 1, 2])
|
||||
|
||||
>>> # Order of edge types in the heterograph
|
||||
>>> g.etypes
|
||||
['interacts', 'treats']
|
||||
>>> # Original edge types
|
||||
>>> hg.edata[dgl.ETYPE]
|
||||
tensor([0, 0, 1])
|
||||
>>> # Original type-specific edge IDs
|
||||
>>> hg.edata[dgl.EID]
|
||||
tensor([0, 1, 0])
|
||||
|
||||
For modeling purposes, one may want to group some relations together and apply the same
|
||||
operation to them. To address this need, one can first take an edge type subgraph of the
|
||||
heterograph and then convert the subgraph to a homogeneous graph.
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... })
|
||||
>>> sub_g = dgl.edge_type_subgraph(g, [('drug', 'interacts', 'drug'),
|
||||
... ('drug', 'interacts', 'gene')])
|
||||
>>> h_sub_g = dgl.to_homogeneous(sub_g)
|
||||
>>> h_sub_g
|
||||
Graph(num_nodes=7, num_edges=4,
|
||||
...)
|
||||
@@ -0,0 +1,37 @@
|
||||
.. _guide-graph:
|
||||
|
||||
Chapter 1: Graph
|
||||
======================
|
||||
|
||||
:ref:`(中文版)<guide_cn-graph>`
|
||||
|
||||
Graphs express entities (nodes) along with their relations (edges), and both nodes and
|
||||
edges can be typed (e.g., ``"user"`` and ``"item"`` are two different types of nodes). DGL provides a
|
||||
graph-centric programming abstraction with its core data structure -- :class:`~dgl.DGLGraph`. :class:`~dgl.DGLGraph`
|
||||
provides its interface to handle a graph's structure, its node/edge features, and the resulting
|
||||
computations that can be performed using these components.
|
||||
|
||||
Roadmap
|
||||
-------
|
||||
|
||||
The chapter starts with a brief introduction to graph definitions in 1.1 and then introduces some core
|
||||
concepts of :class:`~dgl.DGLGraph`:
|
||||
|
||||
* :ref:`guide-graph-basic`
|
||||
* :ref:`guide-graph-graphs-nodes-edges`
|
||||
* :ref:`guide-graph-feature`
|
||||
* :ref:`guide-graph-external`
|
||||
* :ref:`guide-graph-heterogeneous`
|
||||
* :ref:`guide-graph-gpu`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
graph-basic
|
||||
graph-graphs-nodes-edges
|
||||
graph-feature
|
||||
graph-external
|
||||
graph-heterogeneous
|
||||
graph-gpu
|
||||
@@ -0,0 +1,15 @@
|
||||
User Guide
|
||||
==========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
graph
|
||||
message
|
||||
nn
|
||||
data
|
||||
training
|
||||
minibatch
|
||||
distributed
|
||||
mixed_precision
|
||||
@@ -0,0 +1,108 @@
|
||||
.. _guide-message-passing-api:
|
||||
|
||||
2.1 Built-in Functions and Message Passing APIs
|
||||
-----------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-message-passing-api>`
|
||||
|
||||
In DGL, **message function** takes a single argument ``edges``,
|
||||
which is an :class:`~dgl.udf.EdgeBatch` instance. During message passing,
|
||||
DGL generates it internally to represent a batch of edges. It has three
|
||||
members ``src``, ``dst`` and ``data`` to access features of source nodes,
|
||||
destination nodes, and edges, respectively.
|
||||
|
||||
**reduce function** takes a single argument ``nodes``, which is a
|
||||
:class:`~dgl.udf.NodeBatch` instance. During message passing,
|
||||
DGL generates it internally to represent a batch of nodes. It has member
|
||||
``mailbox`` to access the messages received for the nodes in the batch.
|
||||
Some of the most common reduce operations include ``sum``, ``max``, ``min``, etc.
|
||||
|
||||
**update function** takes a single argument ``nodes`` as described above.
|
||||
This function operates on the aggregation result from ``reduce function``, typically
|
||||
combining it with a node’s original feature at the the last step and saving the result
|
||||
as a node feature.
|
||||
|
||||
DGL has implemented commonly used message functions and reduce functions
|
||||
as **built-in** in the namespace ``dgl.function``. In general, DGL
|
||||
suggests using built-in functions **whenever possible** since they are
|
||||
heavily optimized and automatically handle dimension broadcasting.
|
||||
|
||||
If your message passing functions cannot be implemented with built-ins,
|
||||
you can implement user-defined message/reduce function (aka. **UDF**).
|
||||
|
||||
Built-in message functions can be unary or binary. DGL supports ``copy``
|
||||
for unary. For binary funcs, DGL supports ``add``, ``sub``, ``mul``, ``div``,
|
||||
``dot``. The naming convention for message built-in funcs is that ``u``
|
||||
represents ``src`` nodes, ``v`` represents ``dst`` nodes, and ``e`` represents ``edges``.
|
||||
The parameters for those functions are strings indicating the input and output field names for
|
||||
the corresponding nodes and edges. The list of supported built-in functions
|
||||
can be found in :ref:`api-built-in`. For example, to add the ``hu`` feature from src
|
||||
nodes and ``hv`` feature from dst nodes then save the result on the edge
|
||||
at ``he`` field, one can use built-in function ``dgl.function.u_add_v('hu', 'hv', 'he')``.
|
||||
This is equivalent to the Message UDF:
|
||||
|
||||
.. code::
|
||||
|
||||
def message_func(edges):
|
||||
return {'he': edges.src['hu'] + edges.dst['hv']}
|
||||
|
||||
Built-in reduce functions support operations ``sum``, ``max``, ``min``,
|
||||
and ``mean``. Reduce functions usually have two parameters, one
|
||||
for field name in ``mailbox``, one for field name in node features, both
|
||||
are strings. For example, ``dgl.function.sum('m', 'h')`` is equivalent
|
||||
to the Reduce UDF that sums up the message ``m``:
|
||||
|
||||
.. code::
|
||||
|
||||
import torch
|
||||
def reduce_func(nodes):
|
||||
return {'h': torch.sum(nodes.mailbox['m'], dim=1)}
|
||||
|
||||
For advanced usage of UDF, see :ref:`apiudf`.
|
||||
|
||||
It is also possible to invoke only edge-wise computation by :meth:`~dgl.DGLGraph.apply_edges`
|
||||
without invoking message passing. :meth:`~dgl.DGLGraph.apply_edges` takes a message function
|
||||
for parameter and by default updates the features of all edges. For example:
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
graph.apply_edges(fn.u_add_v('el', 'er', 'e'))
|
||||
|
||||
For message passing, :meth:`~dgl.DGLGraph.update_all` is a high-level
|
||||
API that merges message generation, message aggregation and node update
|
||||
in a single call, which leaves room for optimization as a whole.
|
||||
|
||||
The parameters for :meth:`~dgl.DGLGraph.update_all` are a message function, a
|
||||
reduce function and an update function. One can call update function outside of
|
||||
``update_all`` and not specify it in invoking :meth:`~dgl.DGLGraph.update_all`.
|
||||
DGL recommends this approach since the update function can usually be
|
||||
written as pure tensor operations to make the code concise. For
|
||||
example：
|
||||
|
||||
.. code::
|
||||
|
||||
def update_all_example(graph):
|
||||
# store the result in graph.ndata['ft']
|
||||
graph.update_all(fn.u_mul_e('ft', 'a', 'm'),
|
||||
fn.sum('m', 'ft'))
|
||||
# Call update function outside of update_all
|
||||
final_ft = graph.ndata['ft'] * 2
|
||||
return final_ft
|
||||
|
||||
This call will generate the messages ``m`` by multiply src node features
|
||||
``ft`` and edge features ``a``, sum up the messages ``m`` to update node
|
||||
features ``ft``, and finally multiply ``ft`` by 2 to get the result
|
||||
``final_ft``. After the call, DGL will clean the intermediate messages ``m``.
|
||||
The math formula for the above function is:
|
||||
|
||||
.. math:: {final\_ft}_i = 2 * \sum_{j\in\mathcal{N}(i)} ({ft}_j * a_{ji})
|
||||
|
||||
DGL's built-in functions support floating point data types, i.e. the feature must
|
||||
be ``half`` (``float16``) /``float``/``double`` tensors.
|
||||
``float16`` data type support is disabled by default as it has a minimum GPU
|
||||
compute capacity requirement of ``sm_53`` (Pascal, Volta, Turing and Ampere
|
||||
architectures).
|
||||
|
||||
User can enable float16 for mixed precision training by compiling DGL from source
|
||||
(see :doc:`Mixed Precision Training <mixed_precision>` tutorial for details).
|
||||
@@ -0,0 +1,63 @@
|
||||
.. _guide-message-passing-efficient:
|
||||
|
||||
2.2 Writing Efficient Message Passing Code
|
||||
------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-message-passing-efficient>`
|
||||
|
||||
DGL optimizes memory consumption and computing speed for message
|
||||
passing. A common practise to leverage those
|
||||
optimizations is to construct one's own message passing functionality as
|
||||
a combination of :meth:`~dgl.DGLGraph.update_all` calls with built-in
|
||||
functions as parameters.
|
||||
|
||||
Besides that, considering that the number of edges is much larger than the number of nodes for some graphs, avoiding unnecessary memory copy from nodes to edges is beneficial. For some cases like
|
||||
:class:`~dgl.nn.pytorch.conv.GATConv`,
|
||||
where it is necessary to save message on the edges, one needs to call
|
||||
:meth:`~dgl.DGLGraph.apply_edges` with built-in functions. Sometimes the
|
||||
messages on the edges can be high dimensional, which is memory consuming.
|
||||
DGL recommends keeping the dimension of edge features as low as possible.
|
||||
|
||||
Here’s an example on how to achieve this by splitting operations on the
|
||||
edges to nodes. The approach does the following: concatenate the ``src``
|
||||
feature and ``dst`` feature, then apply a linear layer, i.e.
|
||||
:math:`W\times (u || v)`. The ``src`` and ``dst`` feature dimension is
|
||||
high, while the linear layer output dimension is low. A straight forward
|
||||
implementation would be like:
|
||||
|
||||
.. code::
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
|
||||
linear = nn.Parameter(torch.FloatTensor(size=(node_feat_dim * 2, out_dim)))
|
||||
def concat_message_function(edges):
|
||||
return {'cat_feat': torch.cat([edges.src['feat'], edges.dst['feat']], dim=1)}
|
||||
g.apply_edges(concat_message_function)
|
||||
g.edata['out'] = g.edata['cat_feat'] @ linear
|
||||
|
||||
The suggested implementation splits the linear operation into two,
|
||||
one applies on ``src`` feature, the other applies on ``dst`` feature.
|
||||
It then adds the output of the linear operations on the edges at the final stage,
|
||||
i.e. performing :math:`W_l\times u + W_r \times v`. This is because
|
||||
:math:`W \times (u||v) = W_l \times u + W_r \times v`, where :math:`W_l`
|
||||
and :math:`W_r` are the left and the right half of the matrix :math:`W`,
|
||||
respectively:
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
|
||||
linear_src = nn.Parameter(torch.FloatTensor(size=(node_feat_dim, out_dim)))
|
||||
linear_dst = nn.Parameter(torch.FloatTensor(size=(node_feat_dim, out_dim)))
|
||||
out_src = g.ndata['feat'] @ linear_src
|
||||
out_dst = g.ndata['feat'] @ linear_dst
|
||||
g.srcdata.update({'out_src': out_src})
|
||||
g.dstdata.update({'out_dst': out_dst})
|
||||
g.apply_edges(fn.u_add_v('out_src', 'out_dst', 'out'))
|
||||
|
||||
The above two implementations are mathematically equivalent. The latter
|
||||
one is more efficient because it does not need to save feat_src and
|
||||
feat_dst on edges, which is not memory-efficient. Plus, addition could
|
||||
be optimized with DGL’s built-in function :func:`~dgl.function.u_add_v`, which further
|
||||
speeds up computation and saves memory footprint.
|
||||
@@ -0,0 +1,46 @@
|
||||
.. _guide-message-passing-heterograph:
|
||||
|
||||
2.5 Message Passing on Heterogeneous Graph
|
||||
------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-message-passing-heterograph>`
|
||||
|
||||
Heterogeneous graphs (:ref:`guide-graph-heterogeneous`), or
|
||||
heterographs for short, are graphs that contain different types of nodes
|
||||
and edges. The different types of nodes and edges tend to have different
|
||||
types of attributes that are designed to capture the characteristics of
|
||||
each node and edge type. Within the context of graph neural networks,
|
||||
depending on their complexity, certain node and edge types might need to
|
||||
be modeled with representations that have a different number of
|
||||
dimensions.
|
||||
|
||||
The message passing on heterographs can be split into two parts:
|
||||
|
||||
1. Message computation and aggregation for each relation r.
|
||||
2. Reduction that merges the aggregation results from all relations for each node type.
|
||||
|
||||
DGL’s interface to call message passing on heterographs is
|
||||
:meth:`~dgl.DGLGraph.multi_update_all`.
|
||||
:meth:`~dgl.DGLGraph.multi_update_all` takes a dictionary containing
|
||||
the parameters for :meth:`~dgl.DGLGraph.update_all` within each relation
|
||||
using relation as the key, and a string representing the cross type reducer.
|
||||
The reducer can be one of ``sum``, ``min``, ``max``, ``mean``, ``stack``.
|
||||
Here’s an example:
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
|
||||
for c_etype in G.canonical_etypes:
|
||||
srctype, etype, dsttype = c_etype
|
||||
Wh = self.weight[etype](feat_dict[srctype])
|
||||
# Save it in graph for message passing
|
||||
G.nodes[srctype].data['Wh_%s' % etype] = Wh
|
||||
# Specify per-relation message passing functions: (message_func, reduce_func).
|
||||
# Note that the results are saved to the same destination feature 'h', which
|
||||
# hints the type wise reducer for aggregation.
|
||||
funcs[etype] = (fn.copy_u('Wh_%s' % etype, 'm'), fn.mean('m', 'h'))
|
||||
# Trigger message passing of multiple types.
|
||||
G.multi_update_all(funcs, 'sum')
|
||||
# return the updated node feature dictionary
|
||||
return {ntype : G.nodes[ntype].data['h'] for ntype in G.ntypes}
|
||||
@@ -0,0 +1,20 @@
|
||||
.. _guide-message-passing-part:
|
||||
|
||||
2.3 Apply Message Passing On Part Of The Graph
|
||||
----------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-message-passing-part>`
|
||||
|
||||
If one only wants to update part of the nodes in the graph, the practice
|
||||
is to create a subgraph by providing the IDs for the nodes to
|
||||
include in the update, then call :meth:`~dgl.DGLGraph.update_all` on the
|
||||
subgraph. For example:
|
||||
|
||||
.. code::
|
||||
|
||||
nid = [0, 2, 3, 6, 7, 9]
|
||||
sg = g.subgraph(nid)
|
||||
sg.update_all(message_func, reduce_func, apply_node_func)
|
||||
|
||||
This is a common usage in mini-batch training. Check :ref:`guide-minibatch` for more detailed
|
||||
usages.
|
||||
@@ -0,0 +1,46 @@
|
||||
.. _guide-message-passing:
|
||||
|
||||
Chapter 2: Message Passing
|
||||
==========================
|
||||
|
||||
:ref:`(中文版) <guide_cn-message-passing>`
|
||||
|
||||
Message Passing Paradigm
|
||||
------------------------
|
||||
|
||||
Let :math:`x_v\in\mathbb{R}^{d_1}` be the feature for node :math:`v`,
|
||||
and :math:`w_{e}\in\mathbb{R}^{d_2}` be the feature for edge
|
||||
:math:`({u}, {v})`. The **message passing paradigm** defines the
|
||||
following node-wise and edge-wise computation at step :math:`t+1`:
|
||||
|
||||
.. math:: \text{Edge-wise: } m_{e}^{(t+1)} = \phi \left( x_v^{(t)}, x_u^{(t)}, w_{e}^{(t)} \right) , ({u}, {v},{e}) \in \mathcal{E}.
|
||||
|
||||
.. math:: \text{Node-wise: } x_v^{(t+1)} = \psi \left(x_v^{(t)}, \rho\left(\left\lbrace m_{e}^{(t+1)} : ({u}, {v},{e}) \in \mathcal{E} \right\rbrace \right) \right).
|
||||
|
||||
In the above equations, :math:`\phi` is a **message function**
|
||||
defined on each edge to generate a message by combining the edge feature
|
||||
with the features of its incident nodes; :math:`\psi` is an
|
||||
**update function** defined on each node to update the node feature
|
||||
by aggregating its incoming messages using the **reduce function**
|
||||
:math:`\rho`.
|
||||
|
||||
Roadmap
|
||||
-------
|
||||
|
||||
This chapter introduces DGL's message passing APIs, and how to efficiently use them on both nodes and edges.
|
||||
The last section of it explains how to implement message passing on heterogeneous graphs.
|
||||
|
||||
* :ref:`guide-message-passing-api`
|
||||
* :ref:`guide-message-passing-efficient`
|
||||
* :ref:`guide-message-passing-part`
|
||||
* :ref:`guide-message-passing-heterograph`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
message-api
|
||||
message-efficient
|
||||
message-part
|
||||
message-heterograph
|
||||
@@ -0,0 +1,133 @@
|
||||
.. _guide-minibatch-customizing-neighborhood-sampler:
|
||||
|
||||
6.4 Implementing Custom Graph Samplers
|
||||
----------------------------------------------
|
||||
|
||||
Implementing custom samplers involves subclassing the
|
||||
:class:`dgl.graphbolt.SubgraphSampler` base class and implementing its abstract
|
||||
:attr:`sample_subgraphs` method. The :attr:`sample_subgraphs` method should
|
||||
take in seed nodes which are the nodes to sample neighbors from:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def sample_subgraphs(self, seed_nodes):
|
||||
return input_nodes, sampled_subgraphs
|
||||
|
||||
The method should return the input node IDs list and a list of subgraphs. Each
|
||||
subgraph is a :class:`~dgl.graphbolt.SampledSubgraph` object.
|
||||
|
||||
|
||||
Any other data that are required during sampling such as the graph structure,
|
||||
fanout size, etc. should be passed to the sampler via the constructor.
|
||||
|
||||
The code below implements a classical neighbor sampler:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@functional_datapipe("customized_sample_neighbor")
|
||||
class CustomizedNeighborSampler(dgl.graphbolt.SubgraphSampler):
|
||||
def __init__(self, datapipe, graph, fanouts):
|
||||
super().__init__(datapipe)
|
||||
self.graph = graph
|
||||
self.fanouts = fanouts
|
||||
|
||||
def sample_subgraphs(self, seed_nodes):
|
||||
subgs = []
|
||||
for fanout in reversed(self.fanouts):
|
||||
# Sample a fixed number of neighbors of the current seed nodes.
|
||||
input_nodes, sg = g.sample_neighbors(seed_nodes, fanout)
|
||||
subgs.insert(0, sg)
|
||||
seed_nodes = input_nodes
|
||||
return input_nodes, subgs
|
||||
|
||||
To use this sampler with :class:`~dgl.graphbolt.DataLoader`:
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.customized_sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
for data in dataloader:
|
||||
input_features = data.node_features["feat"]
|
||||
output_labels = data.labels
|
||||
output_predictions = model(data.blocks, input_features)
|
||||
loss = compute_loss(output_labels, output_predictions)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
|
||||
Sampler for Heterogeneous Graphs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
To write a sampler for heterogeneous graphs, one needs to be aware that
|
||||
the argument `graph` is a heterogeneous graph while `seeds` could be a
|
||||
dictionary of ID tensors. Most of DGL's graph sampling operators (e.g.,
|
||||
the ``sample_neighbors`` and ``to_block`` functions in the above example) can
|
||||
work on heterogeneous graph natively, so many samplers are automatically
|
||||
ready for heterogeneous graph. For example, the above ``CustomizedNeighborSampler``
|
||||
can be used on heterogeneous graphs:
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.graphbolt as gb
|
||||
hg = gb.FusedCSCSamplingGraph()
|
||||
train_set = item_set = gb.HeteroItemSet(
|
||||
{
|
||||
"user": gb.ItemSet(
|
||||
(torch.arange(0, 5), torch.arange(5, 10)),
|
||||
names=("seeds", "labels"),
|
||||
),
|
||||
"item": gb.ItemSet(
|
||||
(torch.arange(5, 10), torch.arange(10, 15)),
|
||||
names=("seeds", "labels"),
|
||||
),
|
||||
}
|
||||
)
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.customized_sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.fetch_feature(
|
||||
feature, node_feature_keys={"user": ["feat"], "item": ["feat"]}
|
||||
)
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
for data in dataloader:
|
||||
input_features = {
|
||||
ntype: data.node_features[(ntype, "feat")]
|
||||
for ntype in data.blocks[0].srctypes
|
||||
}
|
||||
output_labels = data.labels["user"]
|
||||
output_predictions = model(data.blocks, input_features)["user"]
|
||||
loss = compute_loss(output_labels, output_predictions)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
|
||||
Exclude Edges After Sampling
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In some cases, we may want to exclude seed edges from the sampled subgraph. For
|
||||
example, in link prediction tasks, we want to exclude the edges in the
|
||||
training set from the sampled subgraph to prevent information leakage. To
|
||||
do so, we need to add an additional datapipe right after sampling as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = datapipe.customized_sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.transform(gb.exclude_seed_edges)
|
||||
|
||||
Please check the API page of :func:`~dgl.graphbolt.exclude_seed_edges` for more
|
||||
details.
|
||||
|
||||
The above API is based on :meth:`~dgl.graphbolt.SampledSubgrahp.exclude_edges`.
|
||||
If you want to exclude edges from the sampled subgraph based on some other
|
||||
criteria, you could write your own transform function. Please check the method
|
||||
for reference.
|
||||
|
||||
You could also refer to examples in
|
||||
`Link Prediction <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/link_prediction.py>`__.
|
||||
@@ -0,0 +1,324 @@
|
||||
.. _guide-minibatch-edge-classification-sampler:
|
||||
|
||||
6.2 Training GNN for Edge Classification with Neighborhood Sampling
|
||||
----------------------------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch-edge-classification-sampler>`
|
||||
|
||||
Training for edge classification/regression is somewhat similar to that
|
||||
of node classification/regression with several notable differences.
|
||||
|
||||
Define a neighborhood sampler and data loader
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can use the
|
||||
:ref:`same neighborhood samplers as node classification <guide-minibatch-node-classification-sampler>`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10])
|
||||
# Or equivalently
|
||||
datapipe = dgl.graphbolt.NeighborSampler(datapipe, g, [10, 10])
|
||||
|
||||
The code for defining a data loader is also the same as that of node
|
||||
classification. The only difference is that it iterates over the
|
||||
edges(namely, node pairs) in the training set instead of the nodes.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.graphbolt as gb
|
||||
|
||||
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
||||
g = gb.SamplingGraph()
|
||||
seeds = torch.arange(0, 1000).reshape(-1, 2)
|
||||
labels = torch.randint(0, 2, (5,))
|
||||
train_set = gb.ItemSet((seeds, labels), names=("seeds", "labels"))
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=128, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
# Or equivalently:
|
||||
# datapipe = gb.NeighborSampler(datapipe, g, [10, 10])
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
Iterating over the DataLoader will yield :class:`~dgl.graphbolt.MiniBatch`
|
||||
which contains a list of specially created graphs representing the computation
|
||||
dependencies on each layer. You can access the *message flow graphs* (MFGs) via
|
||||
`mini_batch.blocks`.
|
||||
|
||||
.. code:: python
|
||||
mini_batch = next(iter(dataloader))
|
||||
print(mini_batch.blocks)
|
||||
|
||||
.. note::
|
||||
|
||||
See the :doc:`Stochastic Training Tutorial
|
||||
<../notebooks/stochastic_training/neighbor_sampling_overview.nblink>`__
|
||||
for the concept of message flow graph.
|
||||
|
||||
If you wish to develop your own neighborhood sampler or you want a more
|
||||
detailed explanation of the concept of MFGs, please refer to
|
||||
:ref:`guide-minibatch-customizing-neighborhood-sampler`.
|
||||
|
||||
.. _guide-minibatch-edge-classification-sampler-exclude:
|
||||
|
||||
Removing edges in the minibatch from the original graph for neighbor sampling
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
When training edge classification models, sometimes you wish to remove
|
||||
the edges appearing in the training data from the computation dependency
|
||||
as if they never existed. Otherwise, the model will “know” the fact that
|
||||
an edge exists between the two nodes, and potentially use it for
|
||||
advantage.
|
||||
|
||||
Therefore in edge classification you sometimes would like to exclude the
|
||||
seed edges as well as their reverse edges from the sampled minibatch.
|
||||
You can use :func:`~dgl.graphbolt.exclude_seed_edges` alongside with
|
||||
:class:`~dgl.graphbolt.MiniBatchTransformer` to achieve this.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.graphbolt as gb
|
||||
from functools import partial
|
||||
|
||||
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
||||
g = gb.SamplingGraph()
|
||||
seeds = torch.arange(0, 1000).reshape(-1, 2)
|
||||
labels = torch.randint(0, 2, (5,))
|
||||
train_set = gb.ItemSet((seeds, labels), names=("seeds", "labels"))
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=128, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
exclude_seed_edges = partial(gb.exclude_seed_edges, include_reverse_edges=True)
|
||||
datapipe = datapipe.transform(exclude_seed_edges)
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
|
||||
Adapt your model for minibatch training
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The edge classification model usually consists of two parts:
|
||||
|
||||
- One part that obtains the representation of incident nodes.
|
||||
- The other part that computes the edge score from the incident node
|
||||
representations.
|
||||
|
||||
The former part is exactly the same as
|
||||
:ref:`that from node classification <guide-minibatch-node-classification-model>`
|
||||
and we can simply reuse it. The input is still the list of
|
||||
MFGs generated from a data loader provided by DGL, as well as the
|
||||
input features.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class StochasticTwoLayerGCN(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features):
|
||||
super().__init__()
|
||||
self.conv1 = dglnn.GraphConv(in_features, hidden_features)
|
||||
self.conv2 = dglnn.GraphConv(hidden_features, out_features)
|
||||
|
||||
def forward(self, blocks, x):
|
||||
x = F.relu(self.conv1(blocks[0], x))
|
||||
x = F.relu(self.conv2(blocks[1], x))
|
||||
return x
|
||||
|
||||
The input to the latter part is usually the output from the
|
||||
former part, as well as the subgraph(node pairs) of the original graph induced
|
||||
by the edges in the minibatch. The subgraph is yielded from the same data
|
||||
loader.
|
||||
|
||||
The following code shows an example of predicting scores on the edges by
|
||||
concatenating the incident node features and projecting it with a dense layer.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class ScorePredictor(nn.Module):
|
||||
def __init__(self, num_classes, in_features):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(2 * in_features, num_classes)
|
||||
|
||||
def forward(self, seeds, x):
|
||||
src_x = x[seeds[:, 0]]
|
||||
dst_x = x[seeds[:, 1]]
|
||||
data = torch.cat([src_x, dst_x], 1)
|
||||
return self.W(data)
|
||||
|
||||
|
||||
The entire model will take the list of MFGs and the edges generated by the data
|
||||
loader, as well as the input node features as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features, num_classes):
|
||||
super().__init__()
|
||||
self.gcn = StochasticTwoLayerGCN(
|
||||
in_features, hidden_features, out_features)
|
||||
self.predictor = ScorePredictor(num_classes, out_features)
|
||||
|
||||
def forward(self, blocks, x, seeds):
|
||||
x = self.gcn(blocks, x)
|
||||
return self.predictor(seeds, x)
|
||||
|
||||
DGL ensures that that the nodes in the edge subgraph are the same as the
|
||||
output nodes of the last MFG in the generated list of MFGs.
|
||||
|
||||
Training Loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The training loop is very similar to node classification. You can
|
||||
iterate over the dataloader and get a subgraph induced by the edges in
|
||||
the minibatch, as well as the list of MFGs necessary for computing
|
||||
their incident node representations.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import torch.nn.functional as F
|
||||
model = Model(in_features, hidden_features, out_features, num_classes)
|
||||
model = model.to(device)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for data in dataloader:
|
||||
blocks = data.blocks
|
||||
x = data.edge_features("feat")
|
||||
y_hat = model(data.blocks, x, data.compacted_seeds)
|
||||
loss = F.cross_entropy(data.labels, y_hat)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
|
||||
For heterogeneous graphs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The models computing the node representations on heterogeneous graphs
|
||||
can also be used for computing incident node representations for edge
|
||||
classification/regression.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class StochasticTwoLayerRGCN(nn.Module):
|
||||
def __init__(self, in_feat, hidden_feat, out_feat, rel_names):
|
||||
super().__init__()
|
||||
self.conv1 = dglnn.HeteroGraphConv({
|
||||
rel : dglnn.GraphConv(in_feat, hidden_feat, norm='right')
|
||||
for rel in rel_names
|
||||
})
|
||||
self.conv2 = dglnn.HeteroGraphConv({
|
||||
rel : dglnn.GraphConv(hidden_feat, out_feat, norm='right')
|
||||
for rel in rel_names
|
||||
})
|
||||
|
||||
def forward(self, blocks, x):
|
||||
x = self.conv1(blocks[0], x)
|
||||
x = self.conv2(blocks[1], x)
|
||||
return x
|
||||
|
||||
For score prediction, the only implementation difference between the
|
||||
homogeneous graph and the heterogeneous graph is that we are looping
|
||||
over the edge types.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class ScorePredictor(nn.Module):
|
||||
def __init__(self, num_classes, in_features):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(2 * in_features, num_classes)
|
||||
|
||||
def forward(self, seeds, x):
|
||||
scores = {}
|
||||
for etype in seeds.keys():
|
||||
src, dst = seeds[etype].T
|
||||
data = torch.cat([x[etype][src], x[etype][dst]], 1)
|
||||
scores[etype] = self.W(data)
|
||||
return scores
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features, num_classes,
|
||||
etypes):
|
||||
super().__init__()
|
||||
self.rgcn = StochasticTwoLayerRGCN(
|
||||
in_features, hidden_features, out_features, etypes)
|
||||
self.pred = ScorePredictor(num_classes, out_features)
|
||||
|
||||
def forward(self, seeds, blocks, x):
|
||||
x = self.rgcn(blocks, x)
|
||||
return self.pred(seeds, x)
|
||||
|
||||
Data loader definition is almost identical to that of homogeneous graph. The
|
||||
only difference is that the train_set is now an instance of
|
||||
:class:`~dgl.graphbolt.HeteroItemSet` instead of :class:`~dgl.graphbolt.ItemSet`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.graphbolt as gb
|
||||
|
||||
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
||||
g = gb.SamplingGraph()
|
||||
seeds = torch.arange(0, 1000).reshape(-1, 2)
|
||||
labels = torch.randint(0, 3, (1000,))
|
||||
seeds_labels = {
|
||||
"user:like:item": gb.ItemSet(
|
||||
(seeds, labels), names=("seeds", "labels")
|
||||
),
|
||||
"user:follow:user": gb.ItemSet(
|
||||
(seeds, labels), names=("seeds", "labels")
|
||||
),
|
||||
}
|
||||
train_set = gb.HeteroItemSet(seeds_labels)
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=128, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.fetch_feature(
|
||||
feature, node_feature_keys={"item": ["feat"], "user": ["feat"]}
|
||||
)
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
Things become a little different if you wish to exclude the reverse
|
||||
edges on heterogeneous graphs. On heterogeneous graphs, reverse edges
|
||||
usually have a different edge type from the edges themselves, in order
|
||||
to differentiate the “forward” and “backward” relationships (e.g.
|
||||
``follow`` and ``followed_by`` are reverse relations of each other,
|
||||
``like`` and ``liked_by`` are reverse relations of each other,
|
||||
etc.).
|
||||
|
||||
If each edge in a type has a reverse edge with the same ID in another
|
||||
type, you can specify the mapping between edge types and their reverse
|
||||
types. The way to exclude the edges in the minibatch as well as their
|
||||
reverse edges then goes as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
|
||||
exclude_seed_edges = partial(
|
||||
gb.exclude_seed_edges,
|
||||
include_reverse_edges=True,
|
||||
reverse_etypes_mapping={
|
||||
"user:like:item": "item:liked_by:user",
|
||||
"user:follow:user": "user:followed_by:user",
|
||||
},
|
||||
)
|
||||
datapipe = datapipe.transform(exclude_seed_edges)
|
||||
|
||||
|
||||
The training loop is again almost the same as that on homogeneous graph,
|
||||
except for the implementation of ``compute_loss`` that will take in two
|
||||
dictionaries of node types and predictions here.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import torch.nn.functional as F
|
||||
model = Model(in_features, hidden_features, out_features, num_classes, etypes)
|
||||
model = model.to(device)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for data in dataloader:
|
||||
blocks = data.blocks
|
||||
x = data.edge_features(("user:like:item", "feat"))
|
||||
y_hat = model(data.blocks, x, data.compacted_seeds)
|
||||
loss = F.cross_entropy(data.labels, y_hat)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
.. _guide-minibatch-gpu-sampling:
|
||||
|
||||
6.8 Using GPU for Neighborhood Sampling
|
||||
---------------------------------------
|
||||
|
||||
.. note::
|
||||
GraphBolt does not support GPU-based neighborhood sampling yet. So this guide is
|
||||
utilizing :class:`~dgl.dataloading.DataLoader` for illustration.
|
||||
|
||||
DGL since 0.7 has been supporting GPU-based neighborhood sampling, which has a significant
|
||||
speed advantage over CPU-based neighborhood sampling. If you estimate that your graph
|
||||
can fit onto GPU and your model does not take a lot of GPU memory, then it is best to
|
||||
put the graph onto GPU memory and use GPU-based neighbor sampling.
|
||||
|
||||
For example, `OGB Products <https://ogb.stanford.edu/docs/nodeprop/#ogbn-products>`_ has
|
||||
2.4M nodes and 61M edges. The graph takes less than 1GB since the memory consumption of
|
||||
a graph depends on the number of edges. Therefore it is entirely possible to fit the
|
||||
whole graph onto GPU.
|
||||
|
||||
|
||||
Using GPU-based neighborhood sampling in DGL data loaders
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
One can use GPU-based neighborhood sampling with DGL data loaders via:
|
||||
|
||||
* Put the graph onto GPU.
|
||||
|
||||
* Put the ``train_nid`` onto GPU.
|
||||
|
||||
* Set ``device`` argument to a GPU device.
|
||||
|
||||
* Set ``num_workers`` argument to 0, because CUDA does not allow multiple processes
|
||||
accessing the same context.
|
||||
|
||||
All the other arguments for the :class:`~dgl.dataloading.DataLoader` can be
|
||||
the same as the other user guides and tutorials.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g = g.to('cuda:0')
|
||||
train_nid = train_nid.to('cuda:0')
|
||||
dataloader = dgl.dataloading.DataLoader(
|
||||
g, # The graph must be on GPU.
|
||||
train_nid, # train_nid must be on GPU.
|
||||
sampler,
|
||||
device=torch.device('cuda:0'), # The device argument must be GPU.
|
||||
num_workers=0, # Number of workers must be 0.
|
||||
batch_size=1000,
|
||||
drop_last=False,
|
||||
shuffle=True)
|
||||
|
||||
.. note::
|
||||
|
||||
GPU-based neighbor sampling also works for custom neighborhood samplers as long as
|
||||
(1) your sampler is subclassed from :class:`~dgl.dataloading.BlockSampler`, and (2)
|
||||
your sampler entirely works on GPU.
|
||||
|
||||
|
||||
Using CUDA UVA-based neighborhood sampling in DGL data loaders
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. note::
|
||||
New feature introduced in DGL 0.8.
|
||||
|
||||
For the case where the graph is too large to fit onto the GPU memory, we introduce the
|
||||
CUDA UVA (Unified Virtual Addressing)-based sampling, in which GPUs perform the sampling
|
||||
on the graph pinned in CPU memory via zero-copy access.
|
||||
You can enable UVA-based neighborhood sampling in DGL data loaders via:
|
||||
|
||||
* Put the ``train_nid`` onto GPU.
|
||||
|
||||
* Set ``device`` argument to a GPU device.
|
||||
|
||||
* Set ``num_workers`` argument to 0, because CUDA does not allow multiple processes
|
||||
accessing the same context.
|
||||
|
||||
* Set ``use_uva=True``.
|
||||
|
||||
All the other arguments for the :class:`~dgl.dataloading.DataLoader` can be
|
||||
the same as the other user guides and tutorials.
|
||||
|
||||
.. code:: python
|
||||
|
||||
train_nid = train_nid.to('cuda:0')
|
||||
dataloader = dgl.dataloading.DataLoader(
|
||||
g,
|
||||
train_nid, # train_nid must be on GPU.
|
||||
sampler,
|
||||
device=torch.device('cuda:0'), # The device argument must be GPU.
|
||||
num_workers=0, # Number of workers must be 0.
|
||||
batch_size=1000,
|
||||
drop_last=False,
|
||||
shuffle=True,
|
||||
use_uva=True) # Set use_uva=True
|
||||
|
||||
UVA-based sampling is the recommended solution for mini-batch training on large graphs,
|
||||
especially for multi-GPU training.
|
||||
|
||||
.. note::
|
||||
|
||||
To use UVA-based sampling in multi-GPU training, you should first materialize all the
|
||||
necessary sparse formats of the graph before spawning training processes.
|
||||
Refer to our `GraphSAGE example <https://github.com/dmlc/dgl/blob/master/examples/pytorch/graphsage/multi_gpu_node_classification.py>`_ for more details.
|
||||
|
||||
|
||||
UVA and GPU support for PinSAGESampler/RandomWalkNeighborSampler
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
PinSAGESampler and RandomWalkNeighborSampler support UVA and GPU sampling.
|
||||
You can enable them via:
|
||||
|
||||
* Pin the graph (for UVA sampling) or put the graph onto GPU (for GPU sampling).
|
||||
|
||||
* Put the ``train_nid`` onto GPU.
|
||||
|
||||
.. code:: python
|
||||
|
||||
g = dgl.heterograph({
|
||||
('item', 'bought-by', 'user'): ([0, 0, 1, 1, 2, 2, 3, 3], [0, 1, 0, 1, 2, 3, 2, 3]),
|
||||
('user', 'bought', 'item'): ([0, 1, 0, 1, 2, 3, 2, 3], [0, 0, 1, 1, 2, 2, 3, 3])})
|
||||
|
||||
# UVA setup
|
||||
# g.create_formats_()
|
||||
# g.pin_memory_()
|
||||
|
||||
# GPU setup
|
||||
device = torch.device('cuda:0')
|
||||
g = g.to(device)
|
||||
|
||||
sampler1 = dgl.sampling.PinSAGESampler(g, 'item', 'user', 4, 0.5, 3, 2)
|
||||
sampler2 = dgl.sampling.RandomWalkNeighborSampler(g, 4, 0.5, 3, 2, ['bought-by', 'bought'])
|
||||
|
||||
train_nid = torch.tensor([0, 2], dtype=g.idtype, device=device)
|
||||
sampler1(train_nid)
|
||||
sampler2(train_nid)
|
||||
|
||||
|
||||
Using GPU-based neighbor sampling with DGL functions
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can build your own GPU sampling pipelines with the following functions that support
|
||||
operating on GPU:
|
||||
|
||||
* :func:`dgl.sampling.sample_neighbors`
|
||||
* :func:`dgl.sampling.random_walk`
|
||||
|
||||
Subgraph extraction ops:
|
||||
|
||||
* :func:`dgl.node_subgraph`
|
||||
* :func:`dgl.edge_subgraph`
|
||||
* :func:`dgl.in_subgraph`
|
||||
* :func:`dgl.out_subgraph`
|
||||
|
||||
Graph transform ops for subgraph construction:
|
||||
|
||||
* :func:`dgl.to_block`
|
||||
* :func:`dgl.compact_graph`
|
||||
@@ -0,0 +1,128 @@
|
||||
.. _guide-minibatch-inference:
|
||||
|
||||
6.7 Exact Offline Inference on Large Graphs
|
||||
------------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch-inference>`
|
||||
|
||||
Both subgraph sampling and neighborhood sampling are to reduce the
|
||||
memory and time consumption for training GNNs with GPUs. When performing
|
||||
inference it is usually better to truly aggregate over all neighbors
|
||||
instead to get rid of the randomness introduced by sampling. However,
|
||||
full-graph forward propagation is usually infeasible on GPU due to
|
||||
limited memory, and slow on CPU due to slow computation. This section
|
||||
introduces the methodology of full-graph forward propagation with
|
||||
limited GPU memory via minibatch and neighborhood sampling.
|
||||
|
||||
The inference algorithm is different from the training algorithm, as the
|
||||
representations of all nodes should be computed layer by layer, starting
|
||||
from the first layer. Specifically, for a particular layer, we need to
|
||||
compute the output representations of all nodes from this GNN layer in
|
||||
minibatches. The consequence is that the inference algorithm will have
|
||||
an outer loop iterating over the layers, and an inner loop iterating
|
||||
over the minibatches of nodes. In contrast, the training algorithm has
|
||||
an outer loop iterating over the minibatches of nodes, and an inner loop
|
||||
iterating over the layers for both neighborhood sampling and message
|
||||
passing.
|
||||
|
||||
The following animation shows how the computation would look like (note
|
||||
that for every layer only the first three minibatches are drawn).
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/guide_6_6_0.gif
|
||||
:alt: Imgur
|
||||
|
||||
|
||||
|
||||
Implementing Offline Inference
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Consider the two-layer GCN we have mentioned in Section 6.1
|
||||
:ref:`guide-minibatch-node-classification-model`. The way
|
||||
to implement offline inference still involves using
|
||||
:class:`~dgl.graphbolt.NeighborSampler`, but sampling for
|
||||
only one layer at a time.
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(all_nodes_set, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [-1]) # 1 layers.
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
|
||||
Note that offline inference is implemented as a method of the GNN module
|
||||
because the computation on one layer depends on how messages are aggregated
|
||||
and combined as well.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class SAGE(nn.Module):
|
||||
def __init__(self, in_size, hidden_size, out_size):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
# Three-layer GraphSAGE-mean.
|
||||
self.layers.append(dglnn.SAGEConv(in_size, hidden_size, "mean"))
|
||||
self.layers.append(dglnn.SAGEConv(hidden_size, hidden_size, "mean"))
|
||||
self.layers.append(dglnn.SAGEConv(hidden_size, out_size, "mean"))
|
||||
self.dropout = nn.Dropout(0.5)
|
||||
self.hidden_size = hidden_size
|
||||
self.out_size = out_size
|
||||
|
||||
def forward(self, blocks, x):
|
||||
hidden_x = x
|
||||
for layer_idx, (layer, block) in enumerate(zip(self.layers, blocks)):
|
||||
hidden_x = layer(block, hidden_x)
|
||||
is_last_layer = layer_idx == len(self.layers) - 1
|
||||
if not is_last_layer:
|
||||
hidden_x = F.relu(hidden_x)
|
||||
hidden_x = self.dropout(hidden_x)
|
||||
return hidden_x
|
||||
|
||||
def inference(self, graph, features, dataloader, device):
|
||||
"""
|
||||
Offline inference with this module
|
||||
"""
|
||||
feature = features.read("node", None, "feat")
|
||||
|
||||
# Compute representations layer by layer
|
||||
for layer_idx, layer in enumerate(self.layers):
|
||||
is_last_layer = layer_idx == len(self.layers) - 1
|
||||
|
||||
y = torch.empty(
|
||||
graph.total_num_nodes,
|
||||
self.out_size if is_last_layer else self.hidden_size,
|
||||
dtype=torch.float32,
|
||||
device=buffer_device,
|
||||
pin_memory=pin_memory,
|
||||
)
|
||||
feature = feature.to(device)
|
||||
|
||||
for step, data in tqdm(enumerate(dataloader)):
|
||||
x = feature[data.input_nodes]
|
||||
hidden_x = layer(data.blocks[0], x) # len(blocks) = 1
|
||||
if not is_last_layer:
|
||||
hidden_x = F.relu(hidden_x)
|
||||
hidden_x = self.dropout(hidden_x)
|
||||
# By design, our output nodes are contiguous.
|
||||
y[
|
||||
data.seeds[0] : data.seeds[-1] + 1
|
||||
] = hidden_x.to(device)
|
||||
feature = y
|
||||
|
||||
return y
|
||||
|
||||
|
||||
Note that for the purpose of computing evaluation metric on the
|
||||
validation set for model selection we usually don’t have to compute
|
||||
exact offline inference. The reason is that we need to compute the
|
||||
representation for every single node on every single layer, which is
|
||||
usually very costly especially in the semi-supervised regime with a lot
|
||||
of unlabeled data. Neighborhood sampling will work fine for model
|
||||
selection and validation.
|
||||
|
||||
One can see
|
||||
`GraphSAGE <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/node_classification.py>`__
|
||||
and
|
||||
`RGCN <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/rgcn/hetero_rgcn.py>`__
|
||||
for examples of offline inference.
|
||||
@@ -0,0 +1,268 @@
|
||||
.. _guide-minibatch-link-classification-sampler:
|
||||
|
||||
6.3 Training GNN for Link Prediction with Neighborhood Sampling
|
||||
--------------------------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch-link-classification-sampler>`
|
||||
|
||||
Define a data loader with neighbor and negative sampling
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can still use the same data loader as the one in node/edge classification.
|
||||
The only difference is that you need to add an additional stage
|
||||
`negative sampling` before neighbor sampling stage. The following data loader
|
||||
will pick 5 negative destination nodes uniformly for each source node of an
|
||||
edge.
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = datapipe.sample_uniform_negative(graph, 5)
|
||||
|
||||
The whole data loader pipeline is as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(itemset, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_uniform_negative(graph, 5)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.transform(gb.exclude_seed_edges)
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
|
||||
For the details about the builtin uniform negative sampler please see
|
||||
:class:`~dgl.graphbolt.UniformNegativeSampler`.
|
||||
|
||||
You can also give your own negative sampler function, as long as it inherits
|
||||
from :class:`~dgl.graphbolt.NegativeSampler` and overrides the
|
||||
:meth:`~dgl.graphbolt.NegativeSampler._sample_with_etype` method which takes in
|
||||
the node pairs in minibatch, and returns the negative node pairs back.
|
||||
|
||||
The following gives an example of custom negative sampler that samples
|
||||
negative destination nodes according to a probability distribution
|
||||
proportional to a power of degrees.
|
||||
|
||||
.. code:: python
|
||||
|
||||
@functional_datapipe("customized_sample_negative")
|
||||
class CustomizedNegativeSampler(dgl.graphbolt.NegativeSampler):
|
||||
def __init__(self, datapipe, k, node_degrees):
|
||||
super().__init__(datapipe, k)
|
||||
# caches the probability distribution
|
||||
self.weights = node_degrees ** 0.75
|
||||
self.k = k
|
||||
|
||||
def _sample_with_etype(self, seeds, etype=None):
|
||||
src, _ = seeds.T
|
||||
src = src.repeat_interleave(self.k)
|
||||
dst = self.weights.multinomial(len(src), replacement=True)
|
||||
return src, dst
|
||||
|
||||
datapipe = datapipe.customized_sample_negative(5, node_degrees)
|
||||
|
||||
|
||||
Define a GraphSAGE model for minibatch training
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code:: python
|
||||
|
||||
class SAGE(nn.Module):
|
||||
def __init__(self, in_size, hidden_size):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
self.layers.append(dglnn.SAGEConv(in_size, hidden_size, "mean"))
|
||||
self.layers.append(dglnn.SAGEConv(hidden_size, hidden_size, "mean"))
|
||||
self.layers.append(dglnn.SAGEConv(hidden_size, hidden_size, "mean"))
|
||||
self.hidden_size = hidden_size
|
||||
self.predictor = nn.Sequential(
|
||||
nn.Linear(hidden_size, hidden_size),
|
||||
nn.ReLU(),
|
||||
nn.Linear(hidden_size, hidden_size),
|
||||
nn.ReLU(),
|
||||
nn.Linear(hidden_size, 1),
|
||||
)
|
||||
|
||||
def forward(self, blocks, x):
|
||||
hidden_x = x
|
||||
for layer_idx, (layer, block) in enumerate(zip(self.layers, blocks)):
|
||||
hidden_x = layer(block, hidden_x)
|
||||
is_last_layer = layer_idx == len(self.layers) - 1
|
||||
if not is_last_layer:
|
||||
hidden_x = F.relu(hidden_x)
|
||||
return hidden_x
|
||||
|
||||
|
||||
When a negative sampler is provided, the data loader will generate positive and
|
||||
negative node pairs for each minibatch besides the *Message Flow Graphs* (MFGs).
|
||||
Use `compacted_seeds` and `labels` to get compact node pairs and corresponding
|
||||
labels.
|
||||
|
||||
|
||||
Training loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The training loop simply involves iterating over the data loader and
|
||||
feeding in the graphs as well as the input features to the model defined
|
||||
above.
|
||||
|
||||
.. code:: python
|
||||
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
|
||||
|
||||
for epoch in tqdm.trange(args.epochs):
|
||||
model.train()
|
||||
total_loss = 0
|
||||
start_epoch_time = time.time()
|
||||
for step, data in enumerate(dataloader):
|
||||
# Unpack MiniBatch.
|
||||
compacted_seeds = data.compacted_seeds.T
|
||||
labels = data.labels
|
||||
node_feature = data.node_features["feat"]
|
||||
# Convert sampled subgraphs to DGL blocks.
|
||||
blocks = data.blocks
|
||||
|
||||
# Get the embeddings of the input nodes.
|
||||
y = model(blocks, node_feature)
|
||||
logits = model.predictor(
|
||||
y[compacted_seeds[0]] * y[compacted_seeds[1]]
|
||||
).squeeze()
|
||||
|
||||
# Compute loss.
|
||||
loss = F.binary_cross_entropy_with_logits(logits, labels)
|
||||
optimizer.zero_grad()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
total_loss += loss.item()
|
||||
end_epoch_time = time.time()
|
||||
|
||||
|
||||
DGL provides the
|
||||
`unsupervised learning GraphSAGE <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/link_prediction.py>`__
|
||||
that shows an example of link prediction on homogeneous graphs.
|
||||
|
||||
For heterogeneous graphs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The previous model could be easily extended to heterogeneous graphs. The only
|
||||
difference is that you need to use :class:`~dgl.nn.HeteroGraphConv` to wrap
|
||||
:class:`~dgl.nn.SAGEConv` according to edge types.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class SAGE(nn.Module):
|
||||
def __init__(self, in_size, hidden_size):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
self.layers.append(dglnn.HeteroGraphConv({
|
||||
rel : dglnn.SAGEConv(in_size, hidden_size, "mean")
|
||||
for rel in rel_names
|
||||
}))
|
||||
self.layers.append(dglnn.HeteroGraphConv({
|
||||
rel : dglnn.SAGEConv(hidden_size, hidden_size, "mean")
|
||||
for rel in rel_names
|
||||
}))
|
||||
self.layers.append(dglnn.HeteroGraphConv({
|
||||
rel : dglnn.SAGEConv(hidden_size, hidden_size, "mean")
|
||||
for rel in rel_names
|
||||
}))
|
||||
self.hidden_size = hidden_size
|
||||
self.predictor = nn.Sequential(
|
||||
nn.Linear(hidden_size, hidden_size),
|
||||
nn.ReLU(),
|
||||
nn.Linear(hidden_size, hidden_size),
|
||||
nn.ReLU(),
|
||||
nn.Linear(hidden_size, 1),
|
||||
)
|
||||
|
||||
def forward(self, blocks, x):
|
||||
hidden_x = x
|
||||
for layer_idx, (layer, block) in enumerate(zip(self.layers, blocks)):
|
||||
hidden_x = layer(block, hidden_x)
|
||||
is_last_layer = layer_idx == len(self.layers) - 1
|
||||
if not is_last_layer:
|
||||
hidden_x = F.relu(hidden_x)
|
||||
return hidden_x
|
||||
|
||||
|
||||
Data loader definition is also very similar to that for homogeneous graph. The
|
||||
only difference is that you need to give edge types for feature fetching.
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(itemset, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_uniform_negative(graph, 5)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.transform(gb.exclude_seed_edges)
|
||||
datapipe = datapipe.fetch_feature(
|
||||
feature,
|
||||
node_feature_keys={"user": ["feat"], "item": ["feat"]}
|
||||
)
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
If you want to give your own negative sampling function, just inherit from the
|
||||
:class:`~dgl.graphbolt.NegativeSampler` class and override the
|
||||
:meth:`~dgl.graphbolt.NegativeSampler._sample_with_etype` method.
|
||||
|
||||
.. code:: python
|
||||
|
||||
@functional_datapipe("customized_sample_negative")
|
||||
class CustomizedNegativeSampler(dgl.graphbolt.NegativeSampler):
|
||||
def __init__(self, datapipe, k, node_degrees):
|
||||
super().__init__(datapipe, k)
|
||||
# caches the probability distribution
|
||||
self.weights = {
|
||||
etype: node_degrees[etype] ** 0.75 for etype in node_degrees
|
||||
}
|
||||
self.k = k
|
||||
|
||||
def _sample_with_etype(self, seeds, etype):
|
||||
src, _ = seeds.T
|
||||
src = src.repeat_interleave(self.k)
|
||||
dst = self.weights[etype].multinomial(len(src), replacement=True)
|
||||
return src, dst
|
||||
|
||||
datapipe = datapipe.customized_sample_negative(5, node_degrees)
|
||||
|
||||
|
||||
For heterogeneous graphs, node pairs are grouped by edge types. The training
|
||||
loop is again almost the same as that on homogeneous graph, except for computing
|
||||
loss on specific edge type.
|
||||
|
||||
.. code:: python
|
||||
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
|
||||
|
||||
category = "user"
|
||||
for epoch in tqdm.trange(args.epochs):
|
||||
model.train()
|
||||
total_loss = 0
|
||||
start_epoch_time = time.time()
|
||||
for step, data in enumerate(dataloader):
|
||||
# Unpack MiniBatch.
|
||||
compacted_seeds = data.compacted_seeds
|
||||
labels = data.labels
|
||||
node_features = {
|
||||
ntype: data.node_features[(ntype, "feat")]
|
||||
for ntype in data.blocks[0].srctypes
|
||||
}
|
||||
# Convert sampled subgraphs to DGL blocks.
|
||||
blocks = data.blocks
|
||||
# Get the embeddings of the input nodes.
|
||||
y = model(blocks, node_feature)
|
||||
logits = model.predictor(
|
||||
y[category][compacted_pairs[category][:, 0]]
|
||||
* y[category][compacted_pairs[category][:, 1]]
|
||||
).squeeze()
|
||||
|
||||
# Compute loss.
|
||||
loss = F.binary_cross_entropy_with_logits(logits, labels[category])
|
||||
optimizer.zero_grad()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
total_loss += loss.item()
|
||||
end_epoch_time = time.time()
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
.. _guide-minibatch-custom-gnn-module:
|
||||
|
||||
6.6 Implementing Custom GNN Module for Mini-batch Training
|
||||
-------------------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch-custom-gnn-module>`
|
||||
|
||||
.. note::
|
||||
|
||||
:doc:`This tutorial <tutorials/large/L4_message_passing>` has similar
|
||||
content to this section for the homogeneous graph case.
|
||||
|
||||
|
||||
If you were familiar with how to write a custom GNN module for updating
|
||||
the entire graph for homogeneous or heterogeneous graphs (see
|
||||
:ref:`guide-nn`), the code for computing on
|
||||
MFGs is similar, with the exception that the nodes are divided into
|
||||
input nodes and output nodes.
|
||||
|
||||
For example, consider the following custom graph convolution module
|
||||
code. Note that it is not necessarily among the most efficient implementations
|
||||
- they only serve for an example of how a custom GNN module could look
|
||||
like.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class CustomGraphConv(nn.Module):
|
||||
def __init__(self, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_feats * 2, out_feats)
|
||||
|
||||
def forward(self, g, h):
|
||||
with g.local_scope():
|
||||
g.ndata['h'] = h
|
||||
g.update_all(fn.copy_u('h', 'm'), fn.mean('m', 'h_neigh'))
|
||||
return self.W(torch.cat([g.ndata['h'], g.ndata['h_neigh']], 1))
|
||||
|
||||
If you have a custom message passing NN module for the full graph, and
|
||||
you would like to make it work for MFGs, you only need to rewrite the
|
||||
forward function as follows. Note that the corresponding statements from
|
||||
the full-graph implementation are commented; you can compare the
|
||||
original statements with the new statements.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class CustomGraphConv(nn.Module):
|
||||
def __init__(self, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_feats * 2, out_feats)
|
||||
|
||||
# h is now a pair of feature tensors for input and output nodes, instead of
|
||||
# a single feature tensor.
|
||||
# def forward(self, g, h):
|
||||
def forward(self, block, h):
|
||||
# with g.local_scope():
|
||||
with block.local_scope():
|
||||
# g.ndata['h'] = h
|
||||
h_src = h
|
||||
h_dst = h[:block.number_of_dst_nodes()]
|
||||
block.srcdata['h'] = h_src
|
||||
block.dstdata['h'] = h_dst
|
||||
|
||||
# g.update_all(fn.copy_u('h', 'm'), fn.mean('m', 'h_neigh'))
|
||||
block.update_all(fn.copy_u('h', 'm'), fn.mean('m', 'h_neigh'))
|
||||
|
||||
# return self.W(torch.cat([g.ndata['h'], g.ndata['h_neigh']], 1))
|
||||
return self.W(torch.cat(
|
||||
[block.dstdata['h'], block.dstdata['h_neigh']], 1))
|
||||
|
||||
In general, you need to do the following to make your NN module work for
|
||||
MFGs.
|
||||
|
||||
- Obtain the features for output nodes from the input features by
|
||||
slicing the first few rows. The number of rows can be obtained by
|
||||
:meth:`block.number_of_dst_nodes <dgl.DGLGraph.number_of_dst_nodes>`.
|
||||
- Replace
|
||||
:attr:`g.ndata <dgl.DGLGraph.ndata>` with either
|
||||
:attr:`block.srcdata <dgl.DGLGraph.srcdata>` for features on input nodes or
|
||||
:attr:`block.dstdata <dgl.DGLGraph.dstdata>` for features on output nodes, if
|
||||
the original graph has only one node type.
|
||||
- Replace
|
||||
:attr:`g.nodes <dgl.DGLGraph.nodes>` with either
|
||||
:attr:`block.srcnodes <dgl.DGLGraph.srcnodes>` for features on input nodes or
|
||||
:attr:`block.dstnodes <dgl.DGLGraph.dstnodes>` for features on output nodes,
|
||||
if the original graph has multiple node types.
|
||||
- Replace
|
||||
:meth:`g.num_nodes <dgl.DGLGraph.num_nodes>` with either
|
||||
:meth:`block.number_of_src_nodes <dgl.DGLGraph.number_of_src_nodes>` or
|
||||
:meth:`block.number_of_dst_nodes <dgl.DGLGraph.number_of_dst_nodes>` for the number of
|
||||
input nodes or output nodes respectively.
|
||||
|
||||
Heterogeneous graphs
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For heterogeneous graph the way of writing custom GNN modules is
|
||||
similar. For instance, consider the following module that work on full
|
||||
graph.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class CustomHeteroGraphConv(nn.Module):
|
||||
def __init__(self, g, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.Ws = nn.ModuleDict()
|
||||
for etype in g.canonical_etypes:
|
||||
utype, _, vtype = etype
|
||||
self.Ws[etype] = nn.Linear(in_feats[utype], out_feats[vtype])
|
||||
for ntype in g.ntypes:
|
||||
self.Vs[ntype] = nn.Linear(in_feats[ntype], out_feats[ntype])
|
||||
|
||||
def forward(self, g, h):
|
||||
with g.local_scope():
|
||||
for ntype in g.ntypes:
|
||||
g.nodes[ntype].data['h_dst'] = self.Vs[ntype](h[ntype])
|
||||
g.nodes[ntype].data['h_src'] = h[ntype]
|
||||
for etype in g.canonical_etypes:
|
||||
utype, _, vtype = etype
|
||||
g.update_all(
|
||||
fn.copy_u('h_src', 'm'), fn.mean('m', 'h_neigh'),
|
||||
etype=etype)
|
||||
g.nodes[vtype].data['h_dst'] = g.nodes[vtype].data['h_dst'] + \
|
||||
self.Ws[etype](g.nodes[vtype].data['h_neigh'])
|
||||
return {ntype: g.nodes[ntype].data['h_dst'] for ntype in g.ntypes}
|
||||
|
||||
For ``CustomHeteroGraphConv``, the principle is to replace ``g.nodes``
|
||||
with ``g.srcnodes`` or ``g.dstnodes`` depend on whether the features
|
||||
serve for input or output.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class CustomHeteroGraphConv(nn.Module):
|
||||
def __init__(self, g, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.Ws = nn.ModuleDict()
|
||||
for etype in g.canonical_etypes:
|
||||
utype, _, vtype = etype
|
||||
self.Ws[etype] = nn.Linear(in_feats[utype], out_feats[vtype])
|
||||
for ntype in g.ntypes:
|
||||
self.Vs[ntype] = nn.Linear(in_feats[ntype], out_feats[ntype])
|
||||
|
||||
def forward(self, g, h):
|
||||
with g.local_scope():
|
||||
for ntype in g.ntypes:
|
||||
h_src, h_dst = h[ntype]
|
||||
g.dstnodes[ntype].data['h_dst'] = self.Vs[ntype](h[ntype])
|
||||
g.srcnodes[ntype].data['h_src'] = h[ntype]
|
||||
for etype in g.canonical_etypes:
|
||||
utype, _, vtype = etype
|
||||
g.update_all(
|
||||
fn.copy_u('h_src', 'm'), fn.mean('m', 'h_neigh'),
|
||||
etype=etype)
|
||||
g.dstnodes[vtype].data['h_dst'] = \
|
||||
g.dstnodes[vtype].data['h_dst'] + \
|
||||
self.Ws[etype](g.dstnodes[vtype].data['h_neigh'])
|
||||
return {ntype: g.dstnodes[ntype].data['h_dst']
|
||||
for ntype in g.ntypes}
|
||||
|
||||
Writing modules that work on homogeneous graphs, bipartite graphs, and MFGs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
All message passing modules in DGL work on homogeneous graphs,
|
||||
unidirectional bipartite graphs (that have two node types and one edge
|
||||
type), and a MFG with one edge type. Essentially, the input graph and
|
||||
feature of a builtin DGL neural network module must satisfy either of
|
||||
the following cases.
|
||||
|
||||
- If the input feature is a pair of tensors, then the input graph must
|
||||
be unidirectional bipartite.
|
||||
- If the input feature is a single tensor and the input graph is a
|
||||
MFG, DGL will automatically set the feature on the output nodes as
|
||||
the first few rows of the input node features.
|
||||
- If the input feature must be a single tensor and the input graph is
|
||||
not a MFG, then the input graph must be homogeneous.
|
||||
|
||||
For example, the following is simplified from the PyTorch implementation
|
||||
of :class:`dgl.nn.pytorch.SAGEConv` (also available in MXNet and Tensorflow)
|
||||
(removing normalization and dealing with only mean aggregation etc.).
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.function as fn
|
||||
class SAGEConv(nn.Module):
|
||||
def __init__(self, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_feats * 2, out_feats)
|
||||
|
||||
def forward(self, g, h):
|
||||
if isinstance(h, tuple):
|
||||
h_src, h_dst = h
|
||||
elif g.is_block:
|
||||
h_src = h
|
||||
h_dst = h[:g.number_of_dst_nodes()]
|
||||
else:
|
||||
h_src = h_dst = h
|
||||
|
||||
g.srcdata['h'] = h_src
|
||||
g.dstdata['h'] = h_dst
|
||||
g.update_all(fn.copy_u('h', 'm'), fn.sum('m', 'h_neigh'))
|
||||
return F.relu(
|
||||
self.W(torch.cat([g.dstdata['h'], g.dstdata['h_neigh']], 1)))
|
||||
|
||||
:ref:`guide-nn` also provides a walkthrough on :class:`dgl.nn.pytorch.SAGEConv`,
|
||||
which works on unidirectional bipartite graphs, homogeneous graphs, and MFGs.
|
||||
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
.. _guide-minibatch-node-classification-sampler:
|
||||
|
||||
6.1 Training GNN for Node Classification with Neighborhood Sampling
|
||||
-----------------------------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch-node-classification-sampler>`
|
||||
|
||||
To make your model been trained stochastically, you need to do the
|
||||
followings:
|
||||
|
||||
- Define a neighborhood sampler.
|
||||
- Adapt your model for minibatch training.
|
||||
- Modify your training loop.
|
||||
|
||||
The following sub-subsections address these steps one by one.
|
||||
|
||||
Define a neighborhood sampler and data loader
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL provides several neighborhood sampler classes that generates the
|
||||
computation dependencies needed for each layer given the nodes we wish
|
||||
to compute on.
|
||||
|
||||
The simplest neighborhood sampler is :class:`~dgl.graphbolt.NeighborSampler`
|
||||
or the equivalent function-like interface :func:`~dgl.graphbolt.sample_neighbor`
|
||||
which makes the node gather messages from its neighbors.
|
||||
|
||||
To use a sampler provided by DGL, one also need to combine it with
|
||||
:class:`~dgl.graphbolt.DataLoader`, which iterates
|
||||
over a set of indices (nodes in this case) in minibatches.
|
||||
|
||||
For example, the following code creates a DataLoader that
|
||||
iterates over the training node ID set of ``ogbn-arxiv`` in batches,
|
||||
putting the list of generated MFGs onto GPU.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import dgl.graphbolt as gb
|
||||
import dgl.nn as dglnn
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
|
||||
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
||||
dataset = gb.BuiltinDataset("ogbn-arxiv").load()
|
||||
g = dataset.graph
|
||||
feature = dataset.feature
|
||||
train_set = dataset.tasks[0].train_set
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
# Or equivalently:
|
||||
# datapipe = gb.NeighborSampler(datapipe, g, [10, 10])
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
|
||||
Iterating over the DataLoader will yield :class:`~dgl.graphbolt.MiniBatch`
|
||||
which contains a list of specially created graphs representing the computation
|
||||
dependencies on each layer. In order to train with DGL, you can access the
|
||||
*message flow graphs* (MFGs) by calling `mini_batch.blocks`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
mini_batch = next(iter(dataloader))
|
||||
print(mini_batch.blocks)
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
See the `Stochastic Training Tutorial
|
||||
<../notebooks/stochastic_training/neighbor_sampling_overview.nblink>`__
|
||||
for the concept of message flow graph.
|
||||
|
||||
If you wish to develop your own neighborhood sampler or you want a more
|
||||
detailed explanation of the concept of MFGs, please refer to
|
||||
:ref:`guide-minibatch-customizing-neighborhood-sampler`.
|
||||
|
||||
|
||||
.. _guide-minibatch-node-classification-model:
|
||||
|
||||
Adapt your model for minibatch training
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If your message passing modules are all provided by DGL, the changes
|
||||
required to adapt your model to minibatch training is minimal. Take a
|
||||
multi-layer GCN as an example. If your model on full graph is
|
||||
implemented as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class TwoLayerGCN(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features):
|
||||
super().__init__()
|
||||
self.conv1 = dglnn.GraphConv(in_features, hidden_features)
|
||||
self.conv2 = dglnn.GraphConv(hidden_features, out_features)
|
||||
|
||||
def forward(self, g, x):
|
||||
x = F.relu(self.conv1(g, x))
|
||||
x = F.relu(self.conv2(g, x))
|
||||
return x
|
||||
|
||||
Then all you need is to replace ``g`` with ``blocks`` generated above.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class StochasticTwoLayerGCN(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features):
|
||||
super().__init__()
|
||||
self.conv1 = dgl.nn.GraphConv(in_features, hidden_features)
|
||||
self.conv2 = dgl.nn.GraphConv(hidden_features, out_features)
|
||||
|
||||
def forward(self, blocks, x):
|
||||
x = F.relu(self.conv1(blocks[0], x))
|
||||
x = F.relu(self.conv2(blocks[1], x))
|
||||
return x
|
||||
|
||||
The DGL ``GraphConv`` modules above accepts an element in ``blocks``
|
||||
generated by the data loader as an argument.
|
||||
|
||||
:ref:`The API reference of each NN module <apinn>` will tell you
|
||||
whether it supports accepting a MFG as an argument.
|
||||
|
||||
If you wish to use your own message passing module, please refer to
|
||||
:ref:`guide-minibatch-custom-gnn-module`.
|
||||
|
||||
Training Loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The training loop simply consists of iterating over the dataset with the
|
||||
customized batching iterator. During each iteration that yields
|
||||
:class:`~dgl.graphbolt.MiniBatch`, we:
|
||||
|
||||
1. Access the node features corresponding to the input nodes via
|
||||
``data.node_features["feat"]``. These features are already moved to the
|
||||
target device (CPU or GPU) by the data loader.
|
||||
|
||||
2. Access the node labels corresponding to the output nodes via
|
||||
``data.labels``. These labels are already moved to the target device
|
||||
(CPU or GPU) by the data loader.
|
||||
|
||||
3. Feed the list of MFGs and the input node features to the multilayer
|
||||
GNN and get the outputs.
|
||||
|
||||
4. Compute the loss and backpropagate.
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = StochasticTwoLayerGCN(in_features, hidden_features, out_features)
|
||||
model = model.to(device)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for data in dataloader:
|
||||
input_features = data.node_features["feat"]
|
||||
output_labels = data.labels
|
||||
output_predictions = model(data.blocks, input_features)
|
||||
loss = compute_loss(output_labels, output_predictions)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
|
||||
DGL provides an end-to-end stochastic training example `GraphSAGE
|
||||
implementation <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/node_classification.py>`__.
|
||||
|
||||
For heterogeneous graphs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Training a graph neural network for node classification on heterogeneous
|
||||
graph is similar.
|
||||
|
||||
For instance, we have previously seen
|
||||
:ref:`how to train a 2-layer RGCN on full graph <guide-training-rgcn-node-classification>`.
|
||||
The code for RGCN implementation on minibatch training looks very
|
||||
similar to that (with self-loops, non-linearity and basis decomposition
|
||||
removed for simplicity):
|
||||
|
||||
.. code:: python
|
||||
|
||||
class StochasticTwoLayerRGCN(nn.Module):
|
||||
def __init__(self, in_feat, hidden_feat, out_feat, rel_names):
|
||||
super().__init__()
|
||||
self.conv1 = dglnn.HeteroGraphConv({
|
||||
rel : dglnn.GraphConv(in_feat, hidden_feat, norm='right')
|
||||
for rel in rel_names
|
||||
})
|
||||
self.conv2 = dglnn.HeteroGraphConv({
|
||||
rel : dglnn.GraphConv(hidden_feat, out_feat, norm='right')
|
||||
for rel in rel_names
|
||||
})
|
||||
|
||||
def forward(self, blocks, x):
|
||||
x = self.conv1(blocks[0], x)
|
||||
x = self.conv2(blocks[1], x)
|
||||
return x
|
||||
|
||||
The samplers provided by DGL also support heterogeneous graphs.
|
||||
For example, one can still use the provided
|
||||
:class:`~dgl.graphbolt.NeighborSampler` class and
|
||||
:class:`~dgl.graphbolt.DataLoader` class for
|
||||
stochastic training. The only difference is that the itemset is now an
|
||||
instance of :class:`~dgl.graphbolt.HeteroItemSet` which is a dictionary
|
||||
of node types to node IDs.
|
||||
|
||||
.. code:: python
|
||||
|
||||
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
||||
dataset = gb.BuiltinDataset("ogbn-mag").load()
|
||||
g = dataset.graph
|
||||
feature = dataset.feature
|
||||
train_set = dataset.tasks[0].train_set
|
||||
datapipe = gb.ItemSampler(train_set, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
# Or equivalently:
|
||||
# datapipe = gb.NeighborSampler(datapipe, g, [10, 10])
|
||||
# For heterogeneous graphs, we need to specify the node feature keys
|
||||
# for each node type.
|
||||
datapipe = datapipe.fetch_feature(
|
||||
feature, node_feature_keys={"author": ["feat"], "paper": ["feat"]}
|
||||
)
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
The training loop is almost the same as that of homogeneous graphs,
|
||||
except for the implementation of ``compute_loss`` that will take in two
|
||||
dictionaries of node types and predictions here.
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = StochasticTwoLayerRGCN(in_features, hidden_features, out_features, etypes)
|
||||
model = model.to(device)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for data in dataloader:
|
||||
# For heterogeneous graphs, we need to specify the node types and
|
||||
# feature name when accessing the node features. So does the labels.
|
||||
input_features = {
|
||||
"author": data.node_features[("author", "feat")],
|
||||
"paper": data.node_features[("paper", "feat")]
|
||||
}
|
||||
output_labels = data.labels["paper"]
|
||||
output_predictions = model(data.blocks, input_features)
|
||||
loss = compute_loss(output_labels, output_predictions)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
DGL provides an end-to-end stochastic training example `RGCN
|
||||
implementation <https://github.com/dmlc/dgl/blob/master/examples/graphbolt/rgcn/hetero_rgcn.py>`__.
|
||||
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
.. _guide-minibatch-parallelism:
|
||||
|
||||
6.9 Data Loading Parallelism
|
||||
-----------------------
|
||||
|
||||
In minibatch training of GNNs, we usually need to cover several stages to
|
||||
generate a minibatch, including:
|
||||
|
||||
* Iterate over item set and generate minibatch seeds in batch size.
|
||||
* Sample negative items for each seed from graph.
|
||||
* Sample neighbors for each seed from graph.
|
||||
* Exclude seed edges from the sampled subgraphs.
|
||||
* Fetch node and edge features for the sampled subgraphs.
|
||||
* Copy the MiniBatches to the target device.
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(itemset, batch_size=1024, shuffle=True)
|
||||
datapipe = datapipe.sample_uniform_negative(g, 5)
|
||||
datapipe = datapipe.sample_neighbor(g, [10, 10]) # 2 layers.
|
||||
datapipe = datapipe.transform(gb.exclude_seed_edges)
|
||||
datapipe = datapipe.fetch_feature(feature, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
All these stages are implemented in separate
|
||||
`IterableDataPipe <https://pytorch.org/data/0.7/torchdata.datapipes.iter.html>`__
|
||||
and stacked together with `PyTorch DataLoader <https://pytorch.org/docs/stable/data
|
||||
.html#torch.utils.data.DataLoader>`__.
|
||||
This design allows us to easily customize the data loading process by
|
||||
chaining different data pipes together. For example, if we want to sample
|
||||
negative items for each seed from graph, we can simply chain the
|
||||
:class:`~dgl.graphbolt.NegativeSampler` after the :class:`~dgl.graphbolt.ItemSampler`.
|
||||
|
||||
But simply chaining data pipes together incurs performance overheads as various
|
||||
hardware resources such as CPU, GPU, PCIe, etc. are utilized by different stages.
|
||||
As a result, the data loading mechanism is optimized to minimize the overheads
|
||||
and achieve the best performance.
|
||||
|
||||
In specific, GraphBolt wraps the data pipes before ``fetch_feature`` with
|
||||
multiprocessing which enables multiple processes to run in parallel. As for
|
||||
``fetch_feature`` data pipe, we keep it running in the main process to avoid
|
||||
data movement overheads between processes.
|
||||
|
||||
What's more, in order to overlap the data movement and model computation, we
|
||||
wrap data pipes before ``copy_to`` with
|
||||
`torchdata.datapipes.iter.Perfetcher <https://pytorch.org/data/0.7/generated/
|
||||
torchdata.datapipes.iter.Prefetcher.html>`__
|
||||
which prefetches elements from previous data pipes and puts them into a buffer.
|
||||
Such prefetching is totally transparent to users and requires no extra code. It
|
||||
brings a significant performance boost to minibatch training of GNNs.
|
||||
|
||||
Please refer to the source code of :class:`~dgl.graphbolt.DataLoader`
|
||||
for more details.
|
||||
@@ -0,0 +1,174 @@
|
||||
.. _guide-minibatch-sparse:
|
||||
|
||||
6.5 Training GNN with DGL sparse
|
||||
---------------------------------
|
||||
|
||||
This tutorial demonstrates how to use dgl sparse library to sample on graph and
|
||||
train model. It trains and tests a GraphSAGE model using the sparse sample and
|
||||
compact operators to sample submatrix from the whole matrix.
|
||||
|
||||
Training GNN with DGL sparse is quite similar to
|
||||
:ref:`guide-minibatch-node-classification-sampler`. The major difference is
|
||||
the customized sampler and matrix that represents graph.
|
||||
|
||||
We have cutomized one sampler in
|
||||
:ref:`guide-minibatch-customizing-neighborhood-sampler`. In this tutorial, we
|
||||
will customize another sampler with DGL sparse library as shown below.
|
||||
|
||||
.. code:: python
|
||||
|
||||
@functional_datapipe("sample_sparse_neighbor")
|
||||
class SparseNeighborSampler(SubgraphSampler):
|
||||
def __init__(self, datapipe, matrix, fanouts):
|
||||
super().__init__(datapipe)
|
||||
self.matrix = matrix
|
||||
# Convert fanouts to a list of tensors.
|
||||
self.fanouts = []
|
||||
for fanout in fanouts:
|
||||
if not isinstance(fanout, torch.Tensor):
|
||||
fanout = torch.LongTensor([int(fanout)])
|
||||
self.fanouts.insert(0, fanout)
|
||||
|
||||
def sample_subgraphs(self, seeds):
|
||||
sampled_matrices = []
|
||||
src = seeds
|
||||
|
||||
#####################################################################
|
||||
# (HIGHLIGHT) Using the sparse sample operator to preform random
|
||||
# sampling on the neighboring nodes of the seeds nodes. The sparse
|
||||
# compact operator is then employed to compact and relabel the sampled
|
||||
# matrix, resulting in the sampled matrix and the relabel index.
|
||||
#####################################################################
|
||||
for fanout in self.fanouts:
|
||||
# Sample neighbors.
|
||||
sampled_matrix = self.matrix.sample(1, fanout, ids=src).coalesce()
|
||||
# Compact the sampled matrix.
|
||||
compacted_mat, row_ids = sampled_matrix.compact(0)
|
||||
sampled_matrices.insert(0, compacted_mat)
|
||||
src = row_ids
|
||||
|
||||
return src, sampled_matrices
|
||||
|
||||
Another major difference is the matrix that represents graph. Previously we use
|
||||
:class:`~dgl.graphbolt.FusedCSCSamplingGraph` for sampling. In this tutorial,
|
||||
we use :class:`~dgl.sparse.SparseMatrix` to represent graph.
|
||||
|
||||
.. code:: python
|
||||
|
||||
dataset = gb.BuiltinDataset("ogbn-products").load()
|
||||
g = dataset.graph
|
||||
# Create sparse.
|
||||
N = g.num_nodes
|
||||
A = dglsp.from_csc(g.csc_indptr, g.indices, shape=(N, N))
|
||||
|
||||
|
||||
The remaining code is almost same as node classification tutorial.
|
||||
|
||||
To use this sampler with :class:`~dgl.graphbolt.DataLoader`:
|
||||
|
||||
.. code:: python
|
||||
|
||||
datapipe = gb.ItemSampler(ids, batch_size=1024)
|
||||
# Customize graphbolt sampler by sparse.
|
||||
datapipe = datapipe.sample_sparse_neighbor(A, fanouts)
|
||||
# Use grapbolt to fetch features.
|
||||
datapipe = datapipe.fetch_feature(features, node_feature_keys=["feat"])
|
||||
datapipe = datapipe.copy_to(device)
|
||||
dataloader = gb.DataLoader(datapipe)
|
||||
|
||||
Model definition is shown below:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class SAGEConv(nn.Module):
|
||||
r"""GraphSAGE layer from `Inductive Representation Learning on
|
||||
Large Graphs <https://arxiv.org/pdf/1706.02216.pdf>`__
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
in_feats,
|
||||
out_feats,
|
||||
):
|
||||
super(SAGEConv, self).__init__()
|
||||
self._in_src_feats, self._in_dst_feats = in_feats, in_feats
|
||||
self._out_feats = out_feats
|
||||
|
||||
self.fc_neigh = nn.Linear(self._in_src_feats, out_feats, bias=False)
|
||||
self.fc_self = nn.Linear(self._in_dst_feats, out_feats, bias=True)
|
||||
self.reset_parameters()
|
||||
|
||||
def reset_parameters(self):
|
||||
gain = nn.init.calculate_gain("relu")
|
||||
nn.init.xavier_uniform_(self.fc_self.weight, gain=gain)
|
||||
nn.init.xavier_uniform_(self.fc_neigh.weight, gain=gain)
|
||||
|
||||
def forward(self, A, feat):
|
||||
feat_src = feat
|
||||
feat_dst = feat[: A.shape[1]]
|
||||
|
||||
# Aggregator type: mean.
|
||||
srcdata = self.fc_neigh(feat_src)
|
||||
# Divided by degree.
|
||||
D_hat = dglsp.diag(A.sum(0)) ** -1
|
||||
A_div = A @ D_hat
|
||||
# Conv neighbors.
|
||||
dstdata = A_div.T @ srcdata
|
||||
|
||||
rst = self.fc_self(feat_dst) + dstdata
|
||||
return rst
|
||||
|
||||
|
||||
class SAGE(nn.Module):
|
||||
def __init__(self, in_size, hid_size, out_size):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
# Three-layer GraphSAGE-gcn.
|
||||
self.layers.append(SAGEConv(in_size, hid_size))
|
||||
self.layers.append(SAGEConv(hid_size, hid_size))
|
||||
self.layers.append(SAGEConv(hid_size, out_size))
|
||||
self.dropout = nn.Dropout(0.5)
|
||||
self.hid_size = hid_size
|
||||
self.out_size = out_size
|
||||
|
||||
def forward(self, sampled_matrices, x):
|
||||
hidden_x = x
|
||||
for layer_idx, (layer, sampled_matrix) in enumerate(
|
||||
zip(self.layers, sampled_matrices)
|
||||
):
|
||||
hidden_x = layer(sampled_matrix, hidden_x)
|
||||
if layer_idx != len(self.layers) - 1:
|
||||
hidden_x = F.relu(hidden_x)
|
||||
hidden_x = self.dropout(hidden_x)
|
||||
return hidden_x
|
||||
|
||||
|
||||
Launch training:
|
||||
|
||||
.. code:: python
|
||||
|
||||
features = dataset.feature
|
||||
# Create GraphSAGE model.
|
||||
in_size = features.size("node", None, "feat")[0]
|
||||
num_classes = dataset.tasks[0].metadata["num_classes"]
|
||||
out_size = num_classes
|
||||
model = SAGE(in_size, 256, out_size).to(device)
|
||||
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=5e-4)
|
||||
|
||||
for epoch in range(10):
|
||||
model.train()
|
||||
total_loss = 0
|
||||
for it, data in enumerate(dataloader):
|
||||
node_feature = data.node_features["feat"].float()
|
||||
blocks = data.sampled_subgraphs
|
||||
y = data.labels
|
||||
y_hat = model(blocks, node_feature)
|
||||
loss = F.cross_entropy(y_hat, y)
|
||||
optimizer.zero_grad()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
total_loss += loss.item()
|
||||
|
||||
For more details, please refer to the `full example
|
||||
<https://github.com/dmlc/dgl/blob/master/examples/graphbolt/sparse/graphsage.py>`__.
|
||||
@@ -0,0 +1,81 @@
|
||||
.. _guide-minibatch:
|
||||
|
||||
Chapter 6: Stochastic Training on Large Graphs
|
||||
=======================================================
|
||||
|
||||
:ref:`(中文版) <guide_cn-minibatch>`
|
||||
|
||||
If we have a massive graph with, say, millions or even billions of nodes
|
||||
or edges, usually full-graph training as described in
|
||||
:ref:`guide-training`
|
||||
would not work. Consider an :math:`L`-layer graph convolutional network
|
||||
with hidden state size :math:`H` running on an :math:`N`-node graph.
|
||||
Storing the intermediate hidden states requires :math:`O(NLH)` memory,
|
||||
easily exceeding one GPU’s capacity with large :math:`N`.
|
||||
|
||||
This section provides a way to perform stochastic minibatch training,
|
||||
where we do not have to fit the feature of all the nodes into GPU.
|
||||
|
||||
Overview of Neighborhood Sampling Approaches
|
||||
--------------------------------------------
|
||||
|
||||
Neighborhood sampling methods generally work as the following. For each
|
||||
gradient descent step, we select a minibatch of nodes whose final
|
||||
representations at the :math:`L`-th layer are to be computed. We then
|
||||
take all or some of their neighbors at the :math:`L-1` layer. This
|
||||
process continues until we reach the input. This iterative process
|
||||
builds the dependency graph starting from the output and working
|
||||
backwards to the input, as the figure below shows:
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/guide_6_0_0.png
|
||||
:alt: Imgur
|
||||
|
||||
|
||||
|
||||
With this, one can save the workload and computation resources for
|
||||
training a GNN on a large graph.
|
||||
|
||||
DGL provides a few neighborhood samplers and a pipeline for training a
|
||||
GNN with neighborhood sampling, as well as ways to customize your
|
||||
sampling strategies.
|
||||
|
||||
Roadmap
|
||||
-----------
|
||||
|
||||
The chapter starts with sections for training GNNs stochastically under
|
||||
different scenarios.
|
||||
|
||||
* :ref:`guide-minibatch-node-classification-sampler`
|
||||
* :ref:`guide-minibatch-edge-classification-sampler`
|
||||
* :ref:`guide-minibatch-link-classification-sampler`
|
||||
|
||||
The remaining sections cover more advanced topics, suitable for those who
|
||||
wish to develop new sampling algorithms, new GNN modules compatible with
|
||||
mini-batch training and understand how evaluation and inference can be
|
||||
conducted in mini-batches.
|
||||
|
||||
* :ref:`guide-minibatch-customizing-neighborhood-sampler`
|
||||
* :ref:`guide-minibatch-sparse`
|
||||
* :ref:`guide-minibatch-custom-gnn-module`
|
||||
* :ref:`guide-minibatch-inference`
|
||||
|
||||
The following are performance tips for implementing and using neighborhood
|
||||
sampling:
|
||||
|
||||
* :ref:`guide-minibatch-gpu-sampling`
|
||||
* :ref:`guide-minibatch-parallelism`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
minibatch-node
|
||||
minibatch-edge
|
||||
minibatch-link
|
||||
minibatch-custom-sampler
|
||||
minibatch-sparse
|
||||
minibatch-nn
|
||||
minibatch-inference
|
||||
minibatch-gpu-sampling
|
||||
minibatch-parallelism
|
||||
@@ -0,0 +1,257 @@
|
||||
.. _guide-mixed_precision:
|
||||
|
||||
Chapter 8: Mixed Precision Training
|
||||
===================================
|
||||
DGL is compatible with the `PyTorch Automatic Mixed Precision (AMP) package
|
||||
<https://pytorch.org/docs/stable/amp.html>`_
|
||||
for mixed precision training, thus saving both training time and GPU/CPU memory
|
||||
consumption. This feature requires DGL 0.9+ and 1.1+ for CPU bloat16.
|
||||
|
||||
Message-Passing with Half Precision
|
||||
-----------------------------------
|
||||
DGL allows message-passing on ``float16 (fp16)`` / ``bfloat16 (bf16)``
|
||||
features for both UDFs (User Defined Functions) and built-in functions
|
||||
(e.g., ``dgl.function.sum``, ``dgl.function.copy_u``).
|
||||
|
||||
.. note::
|
||||
Please check bfloat16 support via ``torch.cuda.is_bf16_supported()`` before using it.
|
||||
Typically it requires CUDA >= 11.0 and GPU compute capability >= 8.0.
|
||||
|
||||
The following example shows how to use DGL's message-passing APIs on half-precision
|
||||
features:
|
||||
|
||||
>>> import torch
|
||||
>>> import dgl
|
||||
>>> import dgl.function as fn
|
||||
>>> dev = torch.device('cuda')
|
||||
>>> g = dgl.rand_graph(30, 100).to(dev) # Create a graph on GPU w/ 30 nodes and 100 edges.
|
||||
>>> g.ndata['h'] = torch.rand(30, 16).to(dev).half() # Create fp16 node features.
|
||||
>>> g.edata['w'] = torch.rand(100, 1).to(dev).half() # Create fp16 edge features.
|
||||
>>> # Use DGL's built-in functions for message passing on fp16 features.
|
||||
>>> g.update_all(fn.u_mul_e('h', 'w', 'm'), fn.sum('m', 'x'))
|
||||
>>> g.ndata['x'].dtype
|
||||
torch.float16
|
||||
>>> g.apply_edges(fn.u_dot_v('h', 'x', 'hx'))
|
||||
>>> g.edata['hx'].dtype
|
||||
torch.float16
|
||||
|
||||
>>> # Use UDFs for message passing on fp16 features.
|
||||
>>> def message(edges):
|
||||
... return {'m': edges.src['h'] * edges.data['w']}
|
||||
...
|
||||
>>> def reduce(nodes):
|
||||
... return {'y': torch.sum(nodes.mailbox['m'], 1)}
|
||||
...
|
||||
>>> def dot(edges):
|
||||
... return {'hy': (edges.src['h'] * edges.dst['y']).sum(-1, keepdims=True)}
|
||||
...
|
||||
>>> g.update_all(message, reduce)
|
||||
>>> g.ndata['y'].dtype
|
||||
torch.float16
|
||||
>>> g.apply_edges(dot)
|
||||
>>> g.edata['hy'].dtype
|
||||
torch.float16
|
||||
|
||||
End-to-End Mixed Precision Training
|
||||
-----------------------------------
|
||||
DGL relies on PyTorch's AMP package for mixed precision training,
|
||||
and the user experience is exactly
|
||||
the same as `PyTorch's <https://pytorch.org/docs/stable/notes/amp_examples.html>`_.
|
||||
|
||||
By wrapping the forward pass with ``torch.amp.autocast()``, PyTorch automatically
|
||||
selects the appropriate datatype for each op and tensor. Half precision tensors are memory
|
||||
efficient, most operators on half precision tensors are faster as they leverage GPU tensorcores
|
||||
and CPU special instructon set.
|
||||
|
||||
.. code::
|
||||
|
||||
import torch.nn.functional as F
|
||||
from torch.amp import autocast
|
||||
|
||||
def forward(device_type, g, feat, label, mask, model, amp_dtype):
|
||||
amp_enabled = amp_dtype in (torch.float16, torch.bfloat16)
|
||||
with autocast(device_type, enabled=amp_enabled, dtype=amp_dtype):
|
||||
logit = model(g, feat)
|
||||
loss = F.cross_entropy(logit[mask], label[mask])
|
||||
return loss
|
||||
|
||||
Small Gradients in ``float16`` format have underflow problems (flush to zero).
|
||||
PyTorch provides a ``GradScaler`` module to address this issue. It multiplies
|
||||
the loss by a factor and invokes backward pass on the scaled loss to prevent
|
||||
the underflow problem. It then unscales the computed gradients before the optimizer
|
||||
updates the parameters. The scale factor is determined automatically.
|
||||
Note that ``bfloat16`` doesn't require a ``GradScaler``.
|
||||
|
||||
.. code::
|
||||
|
||||
from torch.cuda.amp import GradScaler
|
||||
|
||||
scaler = GradScaler()
|
||||
|
||||
def backward(scaler, loss, optimizer):
|
||||
scaler.scale(loss).backward()
|
||||
scaler.step(optimizer)
|
||||
scaler.update()
|
||||
|
||||
The following example trains a 3-layer GAT on the Reddit dataset (w/ 114 million edges).
|
||||
Pay attention to the differences in the code when AMP is activated or not.
|
||||
|
||||
.. code::
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import dgl
|
||||
from dgl.data import RedditDataset
|
||||
from dgl.nn import GATConv
|
||||
from dgl.transforms import AddSelfLoop
|
||||
|
||||
amp_dtype = torch.bfloat16 # or torch.float16
|
||||
|
||||
class GAT(nn.Module):
|
||||
def __init__(self,
|
||||
in_feats,
|
||||
n_hidden,
|
||||
n_classes,
|
||||
heads):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
self.layers.append(GATConv(in_feats, n_hidden, heads[0], activation=F.elu))
|
||||
self.layers.append(GATConv(n_hidden * heads[0], n_hidden, heads[1], activation=F.elu))
|
||||
self.layers.append(GATConv(n_hidden * heads[1], n_classes, heads[2], activation=F.elu))
|
||||
|
||||
def forward(self, g, h):
|
||||
for l, layer in enumerate(self.layers):
|
||||
h = layer(g, h)
|
||||
if l != len(self.layers) - 1:
|
||||
h = h.flatten(1)
|
||||
else:
|
||||
h = h.mean(1)
|
||||
return h
|
||||
|
||||
# Data loading
|
||||
transform = AddSelfLoop()
|
||||
data = RedditDataset(transform)
|
||||
device_type = 'cuda' # or 'cpu'
|
||||
dev = torch.device(device_type)
|
||||
|
||||
g = data[0]
|
||||
g = g.int().to(dev)
|
||||
train_mask = g.ndata['train_mask']
|
||||
feat = g.ndata['feat']
|
||||
label = g.ndata['label']
|
||||
|
||||
in_feats = feat.shape[1]
|
||||
n_hidden = 256
|
||||
n_classes = data.num_classes
|
||||
heads = [1, 1, 1]
|
||||
model = GAT(in_feats, n_hidden, n_classes, heads)
|
||||
model = model.to(dev)
|
||||
model.train()
|
||||
|
||||
# Create optimizer
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=5e-4)
|
||||
|
||||
for epoch in range(100):
|
||||
optimizer.zero_grad()
|
||||
loss = forward(device_type, g, feat, label, train_mask, model, amp_dtype)
|
||||
|
||||
if amp_dtype == torch.float16:
|
||||
# Backprop w/ gradient scaling
|
||||
backward(scaler, loss, optimizer)
|
||||
else:
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
print('Epoch {} | Loss {}'.format(epoch, loss.item()))
|
||||
|
||||
On a NVIDIA V100 (16GB) machine, training this model without fp16 consumes
|
||||
15.2GB GPU memory; with fp16 turned on, the training consumes 12.8G
|
||||
GPU memory, the loss converges to similar values in both settings.
|
||||
If we change the number of heads to ``[2, 2, 2]``, training without fp16
|
||||
triggers GPU OOM(out-of-memory) issue while training with fp16 consumes
|
||||
15.7G GPU memory.
|
||||
|
||||
BFloat16 CPU example
|
||||
-----------------------------------
|
||||
DGL supports running training in the bfloat16 data type on the CPU.
|
||||
This data type doesn't require any CPU feature and can improve the performance of a memory-bound model.
|
||||
Starting with Intel Xeon 4th Generation, which has `AMX
|
||||
<https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/advanced-matrix-extensions/overview.html>`_ instructon set, bfloat16 should significantly improve training and inference performance without huge code changes.
|
||||
Here is an example of simple GCN bfloat16 training:
|
||||
|
||||
.. code::
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
import dgl
|
||||
from dgl.data import CiteseerGraphDataset
|
||||
from dgl.nn import GraphConv
|
||||
from dgl.transforms import AddSelfLoop
|
||||
|
||||
|
||||
class GCN(nn.Module):
|
||||
def __init__(self, in_size, hid_size, out_size):
|
||||
super().__init__()
|
||||
self.layers = nn.ModuleList()
|
||||
# two-layer GCN
|
||||
self.layers.append(
|
||||
GraphConv(in_size, hid_size, activation=F.relu)
|
||||
)
|
||||
self.layers.append(GraphConv(hid_size, out_size))
|
||||
self.dropout = nn.Dropout(0.5)
|
||||
|
||||
def forward(self, g, features):
|
||||
h = features
|
||||
for i, layer in enumerate(self.layers):
|
||||
if i != 0:
|
||||
h = self.dropout(h)
|
||||
h = layer(g, h)
|
||||
return h
|
||||
|
||||
|
||||
# Data loading
|
||||
transform = AddSelfLoop()
|
||||
data = CiteseerGraphDataset(transform=transform)
|
||||
|
||||
g = data[0]
|
||||
g = g.int()
|
||||
train_mask = g.ndata['train_mask']
|
||||
feat = g.ndata['feat']
|
||||
label = g.ndata['label']
|
||||
|
||||
in_size = feat.shape[1]
|
||||
hid_size = 16
|
||||
out_size = data.num_classes
|
||||
model = GCN(in_size, hid_size, out_size)
|
||||
|
||||
# Convert model and graph to bfloat16
|
||||
g = dgl.to_bfloat16(g)
|
||||
feat = feat.to(dtype=torch.bfloat16)
|
||||
model = model.to(dtype=torch.bfloat16)
|
||||
|
||||
model.train()
|
||||
|
||||
# Create optimizer
|
||||
optimizer = torch.optim.Adam(model.parameters(), lr=1e-2, weight_decay=5e-4)
|
||||
loss_fcn = nn.CrossEntropyLoss()
|
||||
|
||||
for epoch in range(100):
|
||||
logits = model(g, feat)
|
||||
loss = loss_fcn(logits[train_mask], label[train_mask])
|
||||
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
print('Epoch {} | Loss {}'.format(epoch, loss.item()))
|
||||
|
||||
The only difference with common training is model and graph conversion before training/inference.
|
||||
|
||||
.. code::
|
||||
g = dgl.to_bfloat16(g)
|
||||
feat = feat.to(dtype=torch.bfloat16)
|
||||
model = model.to(dtype=torch.bfloat16)
|
||||
|
||||
|
||||
DGL is still improving its half-precision support and the compute kernel's
|
||||
performance is far from optimal, please stay tuned to our future updates.
|
||||
@@ -0,0 +1,84 @@
|
||||
.. _guide-nn-construction:
|
||||
|
||||
3.1 DGL NN Module Construction Function
|
||||
---------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-nn-construction>`
|
||||
|
||||
The construction function performs the following steps:
|
||||
|
||||
1. Set options.
|
||||
2. Register learnable parameters or submodules.
|
||||
3. Reset parameters.
|
||||
|
||||
.. code::
|
||||
|
||||
import torch.nn as nn
|
||||
|
||||
from dgl.utils import expand_as_pair
|
||||
|
||||
class SAGEConv(nn.Module):
|
||||
def __init__(self,
|
||||
in_feats,
|
||||
out_feats,
|
||||
aggregator_type,
|
||||
bias=True,
|
||||
norm=None,
|
||||
activation=None):
|
||||
super(SAGEConv, self).__init__()
|
||||
|
||||
self._in_src_feats, self._in_dst_feats = expand_as_pair(in_feats)
|
||||
self._out_feats = out_feats
|
||||
self._aggre_type = aggregator_type
|
||||
self.norm = norm
|
||||
self.activation = activation
|
||||
|
||||
In construction function, one first needs to set the data dimensions. For
|
||||
general PyTorch module, the dimensions are usually input dimension,
|
||||
output dimension and hidden dimensions. For graph neural networks, the input
|
||||
dimension can be split into source node dimension and destination node
|
||||
dimension.
|
||||
|
||||
Besides data dimensions, a typical option for graph neural network is
|
||||
aggregation type (``self._aggre_type``). Aggregation type determines how
|
||||
messages on different edges are aggregated for a certain destination
|
||||
node. Commonly used aggregation types include ``mean``, ``sum``,
|
||||
``max``, ``min``. Some modules may apply more complicated aggregation
|
||||
like an ``lstm``.
|
||||
|
||||
``norm`` here is a callable function for feature normalization. In the
|
||||
SAGEConv paper, such normalization can be l2 normalization:
|
||||
:math:`h_v = h_v / \lVert h_v \rVert_2`.
|
||||
|
||||
.. code::
|
||||
|
||||
# aggregator type: mean, pool, lstm, gcn
|
||||
if aggregator_type not in ['mean', 'pool', 'lstm', 'gcn']:
|
||||
raise KeyError('Aggregator type {} not supported.'.format(aggregator_type))
|
||||
if aggregator_type == 'pool':
|
||||
self.fc_pool = nn.Linear(self._in_src_feats, self._in_src_feats)
|
||||
if aggregator_type == 'lstm':
|
||||
self.lstm = nn.LSTM(self._in_src_feats, self._in_src_feats, batch_first=True)
|
||||
if aggregator_type in ['mean', 'pool', 'lstm']:
|
||||
self.fc_self = nn.Linear(self._in_dst_feats, out_feats, bias=bias)
|
||||
self.fc_neigh = nn.Linear(self._in_src_feats, out_feats, bias=bias)
|
||||
self.reset_parameters()
|
||||
|
||||
Register parameters and submodules. In SAGEConv, submodules vary
|
||||
according to the aggregation type. Those modules are pure PyTorch nn
|
||||
modules like ``nn.Linear``, ``nn.LSTM``, etc. At the end of construction
|
||||
function, weight initialization is applied by calling
|
||||
``reset_parameters()``.
|
||||
|
||||
.. code::
|
||||
|
||||
def reset_parameters(self):
|
||||
"""Reinitialize learnable parameters."""
|
||||
gain = nn.init.calculate_gain('relu')
|
||||
if self._aggre_type == 'pool':
|
||||
nn.init.xavier_uniform_(self.fc_pool.weight, gain=gain)
|
||||
if self._aggre_type == 'lstm':
|
||||
self.lstm.reset_parameters()
|
||||
if self._aggre_type != 'gcn':
|
||||
nn.init.xavier_uniform_(self.fc_self.weight, gain=gain)
|
||||
nn.init.xavier_uniform_(self.fc_neigh.weight, gain=gain)
|
||||
@@ -0,0 +1,163 @@
|
||||
.. _guide-nn-forward:
|
||||
|
||||
3.2 DGL NN Module Forward Function
|
||||
----------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-nn-forward>`
|
||||
|
||||
In NN module, ``forward()`` function does the actual message passing and
|
||||
computation. Compared with PyTorch’s NN module which usually takes
|
||||
tensors as the parameters, DGL NN module takes an additional parameter
|
||||
:class:`dgl.DGLGraph`. The
|
||||
workload for ``forward()`` function can be split into three parts:
|
||||
|
||||
- Graph checking and graph type specification.
|
||||
|
||||
- Message passing.
|
||||
|
||||
- Feature update.
|
||||
|
||||
The rest of the section takes a deep dive into the ``forward()`` function in SAGEConv example.
|
||||
|
||||
Graph checking and graph type specification
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code::
|
||||
|
||||
def forward(self, graph, feat):
|
||||
with graph.local_scope():
|
||||
# Specify graph type then expand input feature according to graph type
|
||||
feat_src, feat_dst = expand_as_pair(feat, graph)
|
||||
|
||||
``forward()`` needs to handle many corner cases on the input that can
|
||||
lead to invalid values in computing and message passing. One typical check in conv modules
|
||||
like :class:`~dgl.nn.pytorch.conv.GraphConv` is to verify that the input graph has no 0-in-degree nodes.
|
||||
When a node has 0 in-degree, the ``mailbox`` will be empty and the reduce function will produce
|
||||
all-zero values. This may cause silent regression in model performance. However, in
|
||||
:class:`~dgl.nn.pytorch.conv.SAGEConv` module, the aggregated representation will be concatenated
|
||||
with the original node feature, the output of ``forward()`` will not be all-zero. No such check is
|
||||
needed in this case.
|
||||
|
||||
DGL NN module should be reusable across different types of graph input
|
||||
including: homogeneous graph, heterogeneous
|
||||
graph (:ref:`guide-graph-heterogeneous`), subgraph
|
||||
block (:ref:`guide-minibatch`).
|
||||
|
||||
The math formulas for SAGEConv are:
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
h_{\mathcal{N}(dst)}^{(l+1)} = \mathrm{aggregate}
|
||||
\left(\{h_{src}^{l}, \forall src \in \mathcal{N}(dst) \}\right)
|
||||
|
||||
.. math::
|
||||
|
||||
h_{dst}^{(l+1)} = \sigma \left(W \cdot \mathrm{concat}
|
||||
(h_{dst}^{l}, h_{\mathcal{N}(dst)}^{l+1}) + b \right)
|
||||
|
||||
.. math::
|
||||
|
||||
h_{dst}^{(l+1)} = \mathrm{norm}(h_{dst}^{l+1})
|
||||
|
||||
One needs to specify the source node feature ``feat_src`` and destination
|
||||
node feature ``feat_dst`` according to the graph type.
|
||||
:meth:`~dgl.utils.expand_as_pair` is a function that specifies the graph
|
||||
type and expand ``feat`` into ``feat_src`` and ``feat_dst``.
|
||||
The detail of this function is shown below.
|
||||
|
||||
.. code::
|
||||
|
||||
def expand_as_pair(input_, g=None):
|
||||
if isinstance(input_, tuple):
|
||||
# Bipartite graph case
|
||||
return input_
|
||||
elif g is not None and g.is_block:
|
||||
# Subgraph block case
|
||||
if isinstance(input_, Mapping):
|
||||
input_dst = {
|
||||
k: F.narrow_row(v, 0, g.number_of_dst_nodes(k))
|
||||
for k, v in input_.items()}
|
||||
else:
|
||||
input_dst = F.narrow_row(input_, 0, g.number_of_dst_nodes())
|
||||
return input_, input_dst
|
||||
else:
|
||||
# Homogeneous graph case
|
||||
return input_, input_
|
||||
|
||||
For homogeneous whole graph training, source nodes and destination nodes
|
||||
are the same. They are all the nodes in the graph.
|
||||
|
||||
For heterogeneous case, the graph can be split into several bipartite
|
||||
graphs, one for each relation. The relations are represented as
|
||||
``(src_type, edge_type, dst_dtype)``. When it identifies that the input feature
|
||||
``feat`` is a tuple, it will treat the graph as bipartite. The first
|
||||
element in the tuple will be the source node feature and the second
|
||||
element will be the destination node feature.
|
||||
|
||||
In mini-batch training, the computing is applied on a subgraph sampled
|
||||
based on a bunch of destination nodes. The subgraph is called as
|
||||
``block`` in DGL. In the block creation phase,
|
||||
``dst nodes`` are in the front of the node list. One can find the
|
||||
``feat_dst`` by the index ``[0:g.number_of_dst_nodes()]``.
|
||||
|
||||
After determining ``feat_src`` and ``feat_dst``, the computing for the
|
||||
above three graph types are the same.
|
||||
|
||||
Message passing and reducing
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
import torch.nn.functional as F
|
||||
from dgl.utils import check_eq_shape
|
||||
|
||||
if self._aggre_type == 'mean':
|
||||
graph.srcdata['h'] = feat_src
|
||||
graph.update_all(fn.copy_u('h', 'm'), fn.mean('m', 'neigh'))
|
||||
h_neigh = graph.dstdata['neigh']
|
||||
elif self._aggre_type == 'gcn':
|
||||
check_eq_shape(feat)
|
||||
graph.srcdata['h'] = feat_src
|
||||
graph.dstdata['h'] = feat_dst
|
||||
graph.update_all(fn.copy_u('h', 'm'), fn.sum('m', 'neigh'))
|
||||
# divide in_degrees
|
||||
degs = graph.in_degrees().to(feat_dst)
|
||||
h_neigh = (graph.dstdata['neigh'] + graph.dstdata['h']) / (degs.unsqueeze(-1) + 1)
|
||||
elif self._aggre_type == 'pool':
|
||||
graph.srcdata['h'] = F.relu(self.fc_pool(feat_src))
|
||||
graph.update_all(fn.copy_u('h', 'm'), fn.max('m', 'neigh'))
|
||||
h_neigh = graph.dstdata['neigh']
|
||||
else:
|
||||
raise KeyError('Aggregator type {} not recognized.'.format(self._aggre_type))
|
||||
|
||||
# GraphSAGE GCN does not require fc_self.
|
||||
if self._aggre_type == 'gcn':
|
||||
rst = self.fc_neigh(h_neigh)
|
||||
else:
|
||||
rst = self.fc_self(h_self) + self.fc_neigh(h_neigh)
|
||||
|
||||
The code actually does message passing and reducing computing. This part
|
||||
of code varies module by module. Note that all the message passing in
|
||||
the above code are implemented using :meth:`~dgl.DGLGraph.update_all` API and
|
||||
``built-in`` message/reduce functions to fully utilize DGL’s performance
|
||||
optimization as described in :ref:`guide-message-passing-efficient`.
|
||||
|
||||
Update feature after reducing for output
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code::
|
||||
|
||||
# activation
|
||||
if self.activation is not None:
|
||||
rst = self.activation(rst)
|
||||
# normalization
|
||||
if self.norm is not None:
|
||||
rst = self.norm(rst)
|
||||
return rst
|
||||
|
||||
The last part of ``forward()`` function is to update the feature after
|
||||
the ``reduce function``. Common update operations are applying
|
||||
activation function and normalization according to the option set in the
|
||||
object construction phase.
|
||||
@@ -0,0 +1,108 @@
|
||||
.. _guide-nn-heterograph:
|
||||
|
||||
3.3 Heterogeneous GraphConv Module
|
||||
------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-nn-heterograph>`
|
||||
|
||||
:class:`~dgl.nn.pytorch.HeteroGraphConv`
|
||||
is a module-level encapsulation to run DGL NN module on heterogeneous
|
||||
graphs. The implementation logic is the same as message passing level API
|
||||
:meth:`~dgl.DGLGraph.multi_update_all`, including:
|
||||
|
||||
- DGL NN module within each relation :math:`r`.
|
||||
- Reduction that merges the results on the same node type from multiple
|
||||
relations.
|
||||
|
||||
This can be formulated as:
|
||||
|
||||
.. math:: h_{dst}^{(l+1)} = \underset{r\in\mathcal{R}, r_{dst}=dst}{AGG} (f_r(g_r, h_{r_{src}}^l, h_{r_{dst}}^l))
|
||||
|
||||
where :math:`f_r` is the NN module for each relation :math:`r`,
|
||||
:math:`AGG` is the aggregation function.
|
||||
|
||||
HeteroGraphConv implementation logic:
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. code::
|
||||
|
||||
import torch.nn as nn
|
||||
|
||||
class HeteroGraphConv(nn.Module):
|
||||
def __init__(self, mods, aggregate='sum'):
|
||||
super(HeteroGraphConv, self).__init__()
|
||||
self.mods = nn.ModuleDict(mods)
|
||||
if isinstance(aggregate, str):
|
||||
# An internal function to get common aggregation functions
|
||||
self.agg_fn = get_aggregate_fn(aggregate)
|
||||
else:
|
||||
self.agg_fn = aggregate
|
||||
|
||||
The heterograph convolution takes a dictionary ``mods`` that maps each
|
||||
relation to an nn module and sets the function that aggregates results on
|
||||
the same node type from multiple relations.
|
||||
|
||||
.. code::
|
||||
|
||||
def forward(self, g, inputs, mod_args=None, mod_kwargs=None):
|
||||
if mod_args is None:
|
||||
mod_args = {}
|
||||
if mod_kwargs is None:
|
||||
mod_kwargs = {}
|
||||
outputs = {nty : [] for nty in g.dsttypes}
|
||||
|
||||
Besides input graph and input tensors, the ``forward()`` function takes
|
||||
two additional dictionary parameters ``mod_args`` and ``mod_kwargs``.
|
||||
These two dictionaries have the same keys as ``self.mods``. They are
|
||||
used as customized parameters when calling their corresponding NN
|
||||
modules in ``self.mods`` for different types of relations.
|
||||
|
||||
An output dictionary is created to hold output tensor for each
|
||||
destination type ``nty`` . Note that the value for each ``nty`` is a
|
||||
list, indicating a single node type may get multiple outputs if more
|
||||
than one relations have ``nty`` as the destination type. ``HeteroGraphConv``
|
||||
will perform a further aggregation on the lists.
|
||||
|
||||
.. code::
|
||||
|
||||
if g.is_block:
|
||||
src_inputs = inputs
|
||||
dst_inputs = {k: v[:g.number_of_dst_nodes(k)] for k, v in inputs.items()}
|
||||
else:
|
||||
src_inputs = dst_inputs = inputs
|
||||
|
||||
for stype, etype, dtype in g.canonical_etypes:
|
||||
rel_graph = g[stype, etype, dtype]
|
||||
if rel_graph.num_edges() == 0:
|
||||
continue
|
||||
if stype not in src_inputs or dtype not in dst_inputs:
|
||||
continue
|
||||
dstdata = self.mods[etype](
|
||||
rel_graph,
|
||||
(src_inputs[stype], dst_inputs[dtype]),
|
||||
*mod_args.get(etype, ()),
|
||||
**mod_kwargs.get(etype, {}))
|
||||
outputs[dtype].append(dstdata)
|
||||
|
||||
The input ``g`` can be a heterogeneous graph or a subgraph block from a
|
||||
heterogeneous graph. As in ordinary NN module, the ``forward()``
|
||||
function need to handle different input graph types separately.
|
||||
|
||||
Each relation is represented as a ``canonical_etype``, which is
|
||||
``(stype, etype, dtype)``. Using ``canonical_etype`` as the key, one can
|
||||
extract out a bipartite graph ``rel_graph``. For bipartite graph, the
|
||||
input feature will be organized as a tuple
|
||||
``(src_inputs[stype], dst_inputs[dtype])``. The NN module for each
|
||||
relation is called and the output is saved. To avoid unnecessary call,
|
||||
relations with no edges or no nodes with the src type will be skipped.
|
||||
|
||||
.. code::
|
||||
|
||||
rsts = {}
|
||||
for nty, alist in outputs.items():
|
||||
if len(alist) != 0:
|
||||
rsts[nty] = self.agg_fn(alist, nty)
|
||||
|
||||
Finally, the results on the same destination node type from multiple
|
||||
relations are aggregated using ``self.agg_fn`` function. Examples can
|
||||
be found in the API Doc for :class:`~dgl.nn.pytorch.HeteroGraphConv`.
|
||||
@@ -0,0 +1,39 @@
|
||||
.. _guide-nn:
|
||||
|
||||
Chapter 3: Building GNN Modules
|
||||
===============================
|
||||
|
||||
:ref:`(中文版) <guide_cn-nn>`
|
||||
|
||||
DGL NN module consists of building blocks for GNN models. An NN module inherits
|
||||
from `Pytorch’s NN Module <https://pytorch.org/docs/1.2.0/_modules/torch/nn/modules/module.html>`__, `MXNet Gluon’s NN Block <http://mxnet.incubator.apache.org/versions/1.6/api/python/docs/api/gluon/nn/index.html>`__ and `TensorFlow’s Keras
|
||||
Layer <https://www.tensorflow.org/api_docs/python/tf/keras/layers>`__, depending on the DNN framework backend in use. In a DGL NN
|
||||
module, the parameter registration in construction function and tensor
|
||||
operation in forward function are the same with the backend framework.
|
||||
In this way, DGL code can be seamlessly integrated into the backend
|
||||
framework code. The major difference lies in the message passing
|
||||
operations that are unique in DGL.
|
||||
|
||||
DGL has integrated many commonly used
|
||||
:ref:`apinn-pytorch-conv`, :ref:`apinn-pytorch-dense-conv`, :ref:`apinn-pytorch-pooling`,
|
||||
and
|
||||
:ref:`apinn-pytorch-util`. We welcome your contribution!
|
||||
|
||||
This chapter takes :class:`~dgl.nn.pytorch.conv.SAGEConv` with Pytorch backend as an example
|
||||
to introduce how to build a custom DGL NN Module.
|
||||
|
||||
Roadmap
|
||||
-------
|
||||
|
||||
* :ref:`guide-nn-construction`
|
||||
* :ref:`guide-nn-forward`
|
||||
* :ref:`guide-nn-heterograph`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
nn-construction
|
||||
nn-forward
|
||||
nn-heterograph
|
||||
@@ -0,0 +1,330 @@
|
||||
.. _guide-training-edge-classification:
|
||||
|
||||
5.2 Edge Classification/Regression
|
||||
---------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-training-edge-classification>`
|
||||
|
||||
Sometimes you wish to predict the attributes on the edges of the graph. In that
|
||||
case, you would like to have an *edge classification/regression* model.
|
||||
|
||||
Here we generate a random graph for edge prediction as a demonstration.
|
||||
|
||||
.. code:: python
|
||||
|
||||
src = np.random.randint(0, 100, 500)
|
||||
dst = np.random.randint(0, 100, 500)
|
||||
# make it symmetric
|
||||
edge_pred_graph = dgl.graph((np.concatenate([src, dst]), np.concatenate([dst, src])))
|
||||
# synthetic node and edge features, as well as edge labels
|
||||
edge_pred_graph.ndata['feature'] = torch.randn(100, 10)
|
||||
edge_pred_graph.edata['feature'] = torch.randn(1000, 10)
|
||||
edge_pred_graph.edata['label'] = torch.randn(1000)
|
||||
# synthetic train-validation-test splits
|
||||
edge_pred_graph.edata['train_mask'] = torch.zeros(1000, dtype=torch.bool).bernoulli(0.6)
|
||||
|
||||
Overview
|
||||
~~~~~~~~
|
||||
|
||||
From the previous section you have learned how to do node classification
|
||||
with a multilayer GNN. The same technique can be applied for computing a
|
||||
hidden representation of any node. The prediction on edges can then be
|
||||
derived from the representation of their incident nodes.
|
||||
|
||||
The most common case of computing the prediction on an edge is to
|
||||
express it as a parameterized function of the representation of its
|
||||
incident nodes, and optionally the features on the edge itself.
|
||||
|
||||
Model Implementation Difference from Node Classification
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Assuming that you compute the node representation with the model from
|
||||
the previous section, you only need to write another component that
|
||||
computes the edge prediction with the
|
||||
:meth:`~dgl.DGLGraph.apply_edges` method.
|
||||
|
||||
For instance, if you would like to compute a score for each edge for
|
||||
edge regression, the following code computes the dot product of incident
|
||||
node representations on each edge.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.function as fn
|
||||
class DotProductPredictor(nn.Module):
|
||||
def forward(self, graph, h):
|
||||
# h contains the node representations computed from the GNN defined
|
||||
# in the node classification section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h
|
||||
graph.apply_edges(fn.u_dot_v('h', 'h', 'score'))
|
||||
return graph.edata['score']
|
||||
|
||||
One can also write a prediction function that predicts a vector for each
|
||||
edge with an MLP. Such vector can be used in further downstream tasks,
|
||||
e.g. as logits of a categorical distribution.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class MLPPredictor(nn.Module):
|
||||
def __init__(self, in_features, out_classes):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_features * 2, out_classes)
|
||||
|
||||
def apply_edges(self, edges):
|
||||
h_u = edges.src['h']
|
||||
h_v = edges.dst['h']
|
||||
score = self.W(torch.cat([h_u, h_v], 1))
|
||||
return {'score': score}
|
||||
|
||||
def forward(self, graph, h):
|
||||
# h contains the node representations computed from the GNN defined
|
||||
# in the node classification section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h
|
||||
graph.apply_edges(self.apply_edges)
|
||||
return graph.edata['score']
|
||||
|
||||
Training loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Given the node representation computation model and an edge predictor
|
||||
model, we can easily write a full-graph training loop where we compute
|
||||
the prediction on all edges.
|
||||
|
||||
The following example takes ``SAGE`` in the previous section as the node
|
||||
representation computation model and ``DotPredictor`` as an edge
|
||||
predictor model.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features):
|
||||
super().__init__()
|
||||
self.sage = SAGE(in_features, hidden_features, out_features)
|
||||
self.pred = DotProductPredictor()
|
||||
def forward(self, g, x):
|
||||
h = self.sage(g, x)
|
||||
return self.pred(g, h)
|
||||
|
||||
In this example, we also assume that the training/validation/test edge
|
||||
sets are identified by boolean masks on edges. This example also does
|
||||
not include early stopping and model saving.
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_features = edge_pred_graph.ndata['feature']
|
||||
edge_label = edge_pred_graph.edata['label']
|
||||
train_mask = edge_pred_graph.edata['train_mask']
|
||||
model = Model(10, 20, 5)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(10):
|
||||
pred = model(edge_pred_graph, node_features)
|
||||
loss = ((pred[train_mask] - edge_label[train_mask]) ** 2).mean()
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
.. _guide-training-edge-classification-heterogeneous-graph:
|
||||
|
||||
Heterogeneous graph
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Edge classification on heterogeneous graphs is not very different from
|
||||
that on homogeneous graphs. If you wish to perform edge classification
|
||||
on one edge type, you only need to compute the node representation for
|
||||
all node types, and predict on that edge type with
|
||||
:meth:`~dgl.DGLGraph.apply_edges` method.
|
||||
|
||||
For example, to make ``DotProductPredictor`` work on one edge type of a
|
||||
heterogeneous graph, you only need to specify the edge type in
|
||||
``apply_edges`` method.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class HeteroDotProductPredictor(nn.Module):
|
||||
def forward(self, graph, h, etype):
|
||||
# h contains the node representations for each edge type computed from
|
||||
# the GNN for heterogeneous graphs defined in the node classification
|
||||
# section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h # assigns 'h' of all node types in one shot
|
||||
graph.apply_edges(fn.u_dot_v('h', 'h', 'score'), etype=etype)
|
||||
return graph.edges[etype].data['score']
|
||||
|
||||
You can similarly write a ``HeteroMLPPredictor``.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class HeteroMLPPredictor(nn.Module):
|
||||
def __init__(self, in_features, out_classes):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_features * 2, out_classes)
|
||||
|
||||
def apply_edges(self, edges):
|
||||
h_u = edges.src['h']
|
||||
h_v = edges.dst['h']
|
||||
score = self.W(torch.cat([h_u, h_v], 1))
|
||||
return {'score': score}
|
||||
|
||||
def forward(self, graph, h, etype):
|
||||
# h contains the node representations for each edge type computed from
|
||||
# the GNN for heterogeneous graphs defined in the node classification
|
||||
# section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h # assigns 'h' of all node types in one shot
|
||||
graph.apply_edges(self.apply_edges, etype=etype)
|
||||
return graph.edges[etype].data['score']
|
||||
|
||||
The end-to-end model that predicts a score for each edge on a single
|
||||
edge type will look like this:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features, rel_names):
|
||||
super().__init__()
|
||||
self.sage = RGCN(in_features, hidden_features, out_features, rel_names)
|
||||
self.pred = HeteroDotProductPredictor()
|
||||
def forward(self, g, x, etype):
|
||||
h = self.sage(g, x)
|
||||
return self.pred(g, h, etype)
|
||||
|
||||
Using the model simply involves feeding the model a dictionary of node
|
||||
types and features.
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = Model(10, 20, 5, hetero_graph.etypes)
|
||||
user_feats = hetero_graph.nodes['user'].data['feature']
|
||||
item_feats = hetero_graph.nodes['item'].data['feature']
|
||||
label = hetero_graph.edges['click'].data['label']
|
||||
train_mask = hetero_graph.edges['click'].data['train_mask']
|
||||
node_features = {'user': user_feats, 'item': item_feats}
|
||||
|
||||
Then the training loop looks almost the same as that in homogeneous
|
||||
graph. For instance, if you wish to predict the edge labels on edge type
|
||||
``click``, then you can simply do
|
||||
|
||||
.. code:: python
|
||||
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(10):
|
||||
pred = model(hetero_graph, node_features, 'click')
|
||||
loss = ((pred[train_mask] - label[train_mask]) ** 2).mean()
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
|
||||
Predicting Edge Type of an Existing Edge on a Heterogeneous Graph
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Sometimes you may want to predict which type an existing edge belongs
|
||||
to.
|
||||
|
||||
For instance, given the
|
||||
:ref:`heterogeneous graph example <guide-training-heterogeneous-graph-example>`,
|
||||
your task is given an edge connecting a user and an item, to predict whether
|
||||
the user would ``click`` or ``dislike`` an item.
|
||||
|
||||
This is a simplified version of rating prediction, which is common in
|
||||
recommendation literature.
|
||||
|
||||
You can use a heterogeneous graph convolution network to obtain the node
|
||||
representations. For instance, you can still use the
|
||||
:ref:`RGCN defined previously <guide-training-rgcn-node-classification>`
|
||||
for this purpose.
|
||||
|
||||
To predict the type of an edge, you can simply repurpose the
|
||||
``HeteroDotProductPredictor`` above so that it takes in another graph
|
||||
with only one edge type that “merges” all the edge types to be
|
||||
predicted, and emits the score of each type for every edge.
|
||||
|
||||
In the example here, you will need a graph that has two node types
|
||||
``user`` and ``item``, and one single edge type that “merges” all the
|
||||
edge types from ``user`` and ``item``, i.e. ``click`` and ``dislike``.
|
||||
This can be conveniently created using the following syntax:
|
||||
|
||||
.. code:: python
|
||||
|
||||
dec_graph = hetero_graph['user', :, 'item']
|
||||
|
||||
which returns a heterogeneous graphs with node type ``user`` and ``item``,
|
||||
as well as a single edge type combining all edge types in between, i.e.
|
||||
``click`` and ``dislike``.
|
||||
|
||||
Since the statement above also returns the original edge types as a
|
||||
feature named ``dgl.ETYPE``, we can use that as labels.
|
||||
|
||||
.. code:: python
|
||||
|
||||
edge_label = dec_graph.edata[dgl.ETYPE]
|
||||
|
||||
Given the graph above as input to the edge type predictor module, you
|
||||
can write your predictor module as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class HeteroMLPPredictor(nn.Module):
|
||||
def __init__(self, in_dims, n_classes):
|
||||
super().__init__()
|
||||
self.W = nn.Linear(in_dims * 2, n_classes)
|
||||
|
||||
def apply_edges(self, edges):
|
||||
x = torch.cat([edges.src['h'], edges.dst['h']], 1)
|
||||
y = self.W(x)
|
||||
return {'score': y}
|
||||
|
||||
def forward(self, graph, h):
|
||||
# h contains the node representations for each edge type computed from
|
||||
# the GNN for heterogeneous graphs defined in the node classification
|
||||
# section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h # assigns 'h' of all node types in one shot
|
||||
graph.apply_edges(self.apply_edges)
|
||||
return graph.edata['score']
|
||||
|
||||
The model that combines the node representation module and the edge type
|
||||
predictor module is the following:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features, rel_names):
|
||||
super().__init__()
|
||||
self.sage = RGCN(in_features, hidden_features, out_features, rel_names)
|
||||
self.pred = HeteroMLPPredictor(out_features, len(rel_names))
|
||||
def forward(self, g, x, dec_graph):
|
||||
h = self.sage(g, x)
|
||||
return self.pred(dec_graph, h)
|
||||
|
||||
The training loop then simply be the following:
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = Model(10, 20, 5, hetero_graph.etypes)
|
||||
user_feats = hetero_graph.nodes['user'].data['feature']
|
||||
item_feats = hetero_graph.nodes['item'].data['feature']
|
||||
node_features = {'user': user_feats, 'item': item_feats}
|
||||
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(10):
|
||||
logits = model(hetero_graph, node_features, dec_graph)
|
||||
loss = F.cross_entropy(logits, edge_label)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
|
||||
DGL provides `Graph Convolutional Matrix
|
||||
Completion <https://github.com/dmlc/dgl/tree/master/examples/pytorch/gcmc>`__
|
||||
as an example of rating prediction, which is formulated by predicting
|
||||
the type of an existing edge on a heterogeneous graph. The node
|
||||
representation module in the `model implementation
|
||||
file <https://github.com/dmlc/dgl/tree/master/examples/pytorch/gcmc>`__
|
||||
is called ``GCMCLayer``. The edge type predictor module is called
|
||||
``BiDecoder``. Both of them are more complicated than the setting
|
||||
described here.
|
||||
@@ -0,0 +1,80 @@
|
||||
.. _guide-training-eweight:
|
||||
|
||||
5.5 Use of Edge Weights
|
||||
----------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-training-eweight>`
|
||||
|
||||
In a weighted graph, each edge is associated with a semantically meaningful scalar weight. For
|
||||
example, the edge weights can be connectivity strengths or confidence scores. Naturally, one
|
||||
may want to utilize edge weights in model development.
|
||||
|
||||
Message Passing with Edge Weights
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Most graph neural networks (GNNs) integrate the graph topology information in forward computation
|
||||
by and only by the message passing mechanism. A message passing operation can be viewed as
|
||||
a function that takes an adjacency matrix and additional input features as input arguments. For an
|
||||
unweighted graph, the entries in the adjacency matrix can be zero or one, where a one-valued entry
|
||||
indicates an edge. If this graph is weighted, the non-zero entries can take arbitrary scalar
|
||||
values. This is equivalent to multiplying each message by its corresponding edge weight as in
|
||||
`GAT <https://arxiv.org/pdf/1710.10903.pdf>`__.
|
||||
|
||||
With DGL, one can achieve this by:
|
||||
|
||||
- Saving the edge weights as an edge feature
|
||||
- Multplying the original message by the edge feature in the message function
|
||||
|
||||
Consider the message passing example with DGL below.
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
|
||||
# Suppose graph.ndata['ft'] stores the input node features
|
||||
graph.update_all(fn.copy_u('ft', 'm'), fn.sum('m', 'ft'))
|
||||
|
||||
One can modify it for edge weight support as follows.
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl.function as fn
|
||||
|
||||
# Save edge weights as an edge feature, which is a tensor of shape (E, *)
|
||||
# E is the number of edges
|
||||
graph.edata['w'] = eweight
|
||||
|
||||
# Suppose graph.ndata['ft'] stores the input node features
|
||||
graph.update_all(fn.u_mul_e('ft', 'w', 'm'), fn.sum('m', 'ft'))
|
||||
|
||||
Using NN Modules with Edge Weights
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
One can modify an NN module for edge weight support by modifying all message passing operations
|
||||
in it. The following code snippet is an example for NN module supporting edge weights.
|
||||
|
||||
.. code::
|
||||
import dgl.function as fn
|
||||
import torch.nn as nn
|
||||
|
||||
class GNN(nn.Module):
|
||||
def __init__(self, in_feats, out_feats):
|
||||
super().__init__()
|
||||
self.linear = nn.Linear(in_feats, out_feats)
|
||||
|
||||
def forward(self, g, feat, edge_weight=None):
|
||||
with g.local_scope():
|
||||
g.ndata['ft'] = self.linear(feat)
|
||||
if edge_weight is None:
|
||||
msg_func = fn.copy_u('ft', 'm')
|
||||
else:
|
||||
g.edata['w'] = edge_weight
|
||||
msg_func = fn.u_mul_e('ft', 'w', 'm')
|
||||
g.update_all(msg_func, fn.sum('m', 'ft'))
|
||||
return g.ndata['ft']
|
||||
|
||||
DGL's built-in NN modules support edge weights if they take an optional :attr:`edge_weight`
|
||||
argument in the forward function.
|
||||
|
||||
One may need to normalize raw edge weights. In this regard, DGL provides
|
||||
:func:`~dgl.nn.pytorch.conv.EdgeWeightNorm`.
|
||||
@@ -0,0 +1,304 @@
|
||||
.. _guide-training-graph-classification:
|
||||
|
||||
5.4 Graph Classification
|
||||
----------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-training-graph-classification>`
|
||||
|
||||
Instead of a big single graph, sometimes one might have the data in the
|
||||
form of multiple graphs, for example a list of different types of
|
||||
communities of people. By characterizing the friendship among people in
|
||||
the same community by a graph, one can get a list of graphs to classify. In
|
||||
this scenario, a graph classification model could help identify the type
|
||||
of the community, i.e. to classify each graph based on the structure and
|
||||
overall information.
|
||||
|
||||
Overview
|
||||
~~~~~~~~
|
||||
|
||||
The major difference between graph classification and node
|
||||
classification or link prediction is that the prediction result
|
||||
characterizes the property of the entire input graph. One can perform the
|
||||
message passing over nodes/edges just like the previous tasks, but also
|
||||
needs to retrieve a graph-level representation.
|
||||
|
||||
The graph classification pipeline proceeds as follows:
|
||||
|
||||
.. figure:: https://data.dgl.ai/tutorial/batch/graph_classifier.png
|
||||
:alt: Graph Classification Process
|
||||
|
||||
Graph Classification Process
|
||||
|
||||
From left to right, the common practice is:
|
||||
|
||||
- Prepare a batch of graphs
|
||||
- Perform message passing on the batched graphs to update node/edge features
|
||||
- Aggregate node/edge features into graph-level representations
|
||||
- Classify graphs based on graph-level representations
|
||||
|
||||
Batch of Graphs
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
Usually a graph classification task trains on a lot of graphs, and it
|
||||
will be very inefficient to use only one graph at a time when
|
||||
training the model. Borrowing the idea of mini-batch training from
|
||||
common deep learning practice, one can build a batch of multiple graphs
|
||||
and send them together for one training iteration.
|
||||
|
||||
In DGL, one can build a single batched graph from a list of graphs. This
|
||||
batched graph can be simply used as a single large graph, with connected
|
||||
components corresponding to the original small graphs.
|
||||
|
||||
.. figure:: https://data.dgl.ai/tutorial/batch/batch.png
|
||||
:alt: Batched Graph
|
||||
|
||||
Batched Graph
|
||||
|
||||
The following example calls :func:`dgl.batch` on a list of graphs.
|
||||
A batched graph is a single graph, while it also carries information
|
||||
about the list.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import torch as th
|
||||
|
||||
g1 = dgl.graph((th.tensor([0, 1, 2]), th.tensor([1, 2, 3])))
|
||||
g2 = dgl.graph((th.tensor([0, 0, 0, 1]), th.tensor([0, 1, 2, 0])))
|
||||
|
||||
bg = dgl.batch([g1, g2])
|
||||
bg
|
||||
# Graph(num_nodes=7, num_edges=7,
|
||||
# ndata_schemes={}
|
||||
# edata_schemes={})
|
||||
bg.batch_size
|
||||
# 2
|
||||
bg.batch_num_nodes()
|
||||
# tensor([4, 3])
|
||||
bg.batch_num_edges()
|
||||
# tensor([3, 4])
|
||||
bg.edges()
|
||||
# (tensor([0, 1, 2, 4, 4, 4, 5], tensor([1, 2, 3, 4, 5, 6, 4]))
|
||||
|
||||
Please note that most dgl transformation functions will discard the batch information.
|
||||
In order to maintain such information, please use :func:`dgl.DGLGraph.set_batch_num_nodes`
|
||||
and :func:`dgl.DGLGraph.set_batch_num_edges` on the transformed graph.
|
||||
|
||||
Graph Readout
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
Every graph in the data may have its unique structure, as well as its
|
||||
node and edge features. In order to make a single prediction, one usually
|
||||
aggregates and summarizes over the possibly abundant information. This
|
||||
type of operation is named *readout*. Common readout operations include
|
||||
summation, average, maximum or minimum over all node or edge features.
|
||||
|
||||
Given a graph :math:`g`, one can define the average node feature readout as
|
||||
|
||||
.. math:: h_g = \frac{1}{|\mathcal{V}|}\sum_{v\in \mathcal{V}}h_v
|
||||
|
||||
where :math:`h_g` is the representation of :math:`g`, :math:`\mathcal{V}` is
|
||||
the set of nodes in :math:`g`, :math:`h_v` is the feature of node :math:`v`.
|
||||
|
||||
DGL provides built-in support for common readout operations. For example,
|
||||
:func:`dgl.mean_nodes` implements the above readout operation.
|
||||
|
||||
Once :math:`h_g` is available, one can pass it through an MLP layer for
|
||||
classification output.
|
||||
|
||||
Writing Neural Network Model
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The input to the model is the batched graph with node and edge features.
|
||||
|
||||
Computation on a Batched Graph
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
First, different graphs in a batch are entirely separated, i.e. no edges
|
||||
between any two graphs. With this nice property, all message passing
|
||||
functions still have the same results.
|
||||
|
||||
Second, the readout function on a batched graph will be conducted over
|
||||
each graph separately. Assuming the batch size is :math:`B` and the
|
||||
feature to be aggregated has dimension :math:`D`, the shape of the
|
||||
readout result will be :math:`(B, D)`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import torch
|
||||
|
||||
g1 = dgl.graph(([0, 1], [1, 0]))
|
||||
g1.ndata['h'] = torch.tensor([1., 2.])
|
||||
g2 = dgl.graph(([0, 1], [1, 2]))
|
||||
g2.ndata['h'] = torch.tensor([1., 2., 3.])
|
||||
|
||||
dgl.readout_nodes(g1, 'h')
|
||||
# tensor([3.]) # 1 + 2
|
||||
|
||||
bg = dgl.batch([g1, g2])
|
||||
dgl.readout_nodes(bg, 'h')
|
||||
# tensor([3., 6.]) # [1 + 2, 1 + 2 + 3]
|
||||
|
||||
Finally, each node/edge feature in a batched graph is obtained by
|
||||
concatenating the corresponding features from all graphs in order.
|
||||
|
||||
.. code:: python
|
||||
|
||||
bg.ndata['h']
|
||||
# tensor([1., 2., 1., 2., 3.])
|
||||
|
||||
Model Definition
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
Being aware of the above computation rules, one can define a model as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.nn.pytorch as dglnn
|
||||
import torch.nn as nn
|
||||
|
||||
class Classifier(nn.Module):
|
||||
def __init__(self, in_dim, hidden_dim, n_classes):
|
||||
super(Classifier, self).__init__()
|
||||
self.conv1 = dglnn.GraphConv(in_dim, hidden_dim)
|
||||
self.conv2 = dglnn.GraphConv(hidden_dim, hidden_dim)
|
||||
self.classify = nn.Linear(hidden_dim, n_classes)
|
||||
|
||||
def forward(self, g, h):
|
||||
# Apply graph convolution and activation.
|
||||
h = F.relu(self.conv1(g, h))
|
||||
h = F.relu(self.conv2(g, h))
|
||||
with g.local_scope():
|
||||
g.ndata['h'] = h
|
||||
# Calculate graph representation by average readout.
|
||||
hg = dgl.mean_nodes(g, 'h')
|
||||
return self.classify(hg)
|
||||
|
||||
Training Loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Data Loading
|
||||
^^^^^^^^^^^^
|
||||
|
||||
Once the model is defined, one can start training. Since graph
|
||||
classification deals with lots of relatively small graphs instead of a big
|
||||
single one, one can train efficiently on stochastic mini-batches
|
||||
of graphs, without the need to design sophisticated graph sampling
|
||||
algorithms.
|
||||
|
||||
Assuming that one have a graph classification dataset as introduced in
|
||||
:ref:`guide-data-pipeline`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl.data
|
||||
dataset = dgl.data.GINDataset('MUTAG', False)
|
||||
|
||||
Each item in the graph classification dataset is a pair of a graph and
|
||||
its label. One can speed up the data loading process by taking advantage
|
||||
of the GraphDataLoader to iterate over the dataset of
|
||||
graphs in mini-batches.
|
||||
|
||||
.. code:: python
|
||||
|
||||
from dgl.dataloading import GraphDataLoader
|
||||
dataloader = GraphDataLoader(
|
||||
dataset,
|
||||
batch_size=1024,
|
||||
drop_last=False,
|
||||
shuffle=True)
|
||||
|
||||
Training loop then simply involves iterating over the dataloader and
|
||||
updating the model.
|
||||
|
||||
.. code:: python
|
||||
|
||||
import torch.nn.functional as F
|
||||
|
||||
# Only an example, 7 is the input feature size
|
||||
model = Classifier(7, 20, 5)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(20):
|
||||
for batched_graph, labels in dataloader:
|
||||
feats = batched_graph.ndata['attr']
|
||||
logits = model(batched_graph, feats)
|
||||
loss = F.cross_entropy(logits, labels)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
For an end-to-end example of graph classification, see
|
||||
`DGL's GIN example <https://github.com/dmlc/dgl/tree/master/examples/pytorch/gin>`__.
|
||||
The training loop is inside the
|
||||
function ``train`` in
|
||||
`main.py <https://github.com/dmlc/dgl/blob/master/examples/pytorch/gin/main.py>`__.
|
||||
The model implementation is inside
|
||||
`gin.py <https://github.com/dmlc/dgl/blob/master/examples/pytorch/gin/gin.py>`__
|
||||
with more components such as using
|
||||
:class:`dgl.nn.pytorch.GINConv` (also available in MXNet and Tensorflow)
|
||||
as the graph convolution layer, batch normalization, etc.
|
||||
|
||||
Heterogeneous graph
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Graph classification with heterogeneous graphs is a little different
|
||||
from that with homogeneous graphs. In addition to graph convolution modules
|
||||
compatible with heterogeneous graphs, one also needs to aggregate over the nodes of
|
||||
different types in the readout function.
|
||||
|
||||
The following shows an example of summing up the average of node
|
||||
representations for each node type.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class RGCN(nn.Module):
|
||||
def __init__(self, in_feats, hid_feats, out_feats, rel_names):
|
||||
super().__init__()
|
||||
|
||||
self.conv1 = dglnn.HeteroGraphConv({
|
||||
rel: dglnn.GraphConv(in_feats, hid_feats)
|
||||
for rel in rel_names}, aggregate='sum')
|
||||
self.conv2 = dglnn.HeteroGraphConv({
|
||||
rel: dglnn.GraphConv(hid_feats, out_feats)
|
||||
for rel in rel_names}, aggregate='sum')
|
||||
|
||||
def forward(self, graph, inputs):
|
||||
# inputs is features of nodes
|
||||
h = self.conv1(graph, inputs)
|
||||
h = {k: F.relu(v) for k, v in h.items()}
|
||||
h = self.conv2(graph, h)
|
||||
return h
|
||||
|
||||
class HeteroClassifier(nn.Module):
|
||||
def __init__(self, in_dim, hidden_dim, n_classes, rel_names):
|
||||
super().__init__()
|
||||
|
||||
self.rgcn = RGCN(in_dim, hidden_dim, hidden_dim, rel_names)
|
||||
self.classify = nn.Linear(hidden_dim, n_classes)
|
||||
|
||||
def forward(self, g):
|
||||
h = g.ndata['feat']
|
||||
h = self.rgcn(g, h)
|
||||
with g.local_scope():
|
||||
g.ndata['h'] = h
|
||||
# Calculate graph representation by average readout.
|
||||
hg = 0
|
||||
for ntype in g.ntypes:
|
||||
hg = hg + dgl.mean_nodes(g, 'h', ntype=ntype)
|
||||
return self.classify(hg)
|
||||
|
||||
The rest of the code is not different from that for homogeneous graphs.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# etypes is the list of edge types as strings.
|
||||
model = HeteroClassifier(10, 20, 5, etypes)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(20):
|
||||
for batched_graph, labels in dataloader:
|
||||
logits = model(batched_graph)
|
||||
loss = F.cross_entropy(logits, labels)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
@@ -0,0 +1,216 @@
|
||||
.. _guide-training-link-prediction:
|
||||
|
||||
5.3 Link Prediction
|
||||
---------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-training-link-prediction>`
|
||||
|
||||
In some other settings you may want to predict whether an edge exists
|
||||
between two given nodes or not. Such task is called a *link prediction*
|
||||
task.
|
||||
|
||||
Overview
|
||||
~~~~~~~~
|
||||
|
||||
A GNN-based link prediction model represents the likelihood of
|
||||
connectivity between two nodes :math:`u` and :math:`v` as a function of
|
||||
:math:`\boldsymbol{h}_u^{(L)}` and :math:`\boldsymbol{h}_v^{(L)}`, their
|
||||
node representation computed from the multi-layer GNN.
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
y_{u,v} = \phi(\boldsymbol{h}_u^{(L)}, \boldsymbol{h}_v^{(L)})
|
||||
|
||||
In this section we refer to :math:`y_{u,v}` the *score* between node
|
||||
:math:`u` and node :math:`v`.
|
||||
|
||||
Training a link prediction model involves comparing the scores between
|
||||
nodes connected by an edge against the scores between an arbitrary pair
|
||||
of nodes. For example, given an edge connecting :math:`u` and :math:`v`,
|
||||
we encourage the score between node :math:`u` and :math:`v` to be higher
|
||||
than the score between node :math:`u` and a sampled node :math:`v'` from
|
||||
an arbitrary *noise* distribution :math:`v' \sim P_n(v)`. Such
|
||||
methodology is called *negative sampling*.
|
||||
|
||||
There are lots of loss functions that can achieve the behavior above if
|
||||
minimized. A non-exhaustive list include:
|
||||
|
||||
- Cross-entropy loss:
|
||||
:math:`\mathcal{L} = - \log \sigma (y_{u,v}) - \sum_{v_i \sim P_n(v), i=1,\dots,k}\log \left[ 1 - \sigma (y_{u,v_i})\right]`
|
||||
- BPR loss:
|
||||
:math:`\mathcal{L} = \sum_{v_i \sim P_n(v), i=1,\dots,k} - \log \sigma (y_{u,v} - y_{u,v_i})`
|
||||
- Margin loss:
|
||||
:math:`\mathcal{L} = \sum_{v_i \sim P_n(v), i=1,\dots,k} \max(0, M - y_{u, v} + y_{u, v_i})`,
|
||||
where :math:`M` is a constant hyperparameter.
|
||||
|
||||
You may find this idea familiar if you know what `implicit
|
||||
feedback <https://arxiv.org/ftp/arxiv/papers/1205/1205.2618.pdf>`__ or
|
||||
`noise-contrastive
|
||||
estimation <http://proceedings.mlr.press/v9/gutmann10a/gutmann10a.pdf>`__
|
||||
is.
|
||||
|
||||
The neural network model to compute the score between :math:`u` and
|
||||
:math:`v` is identical to the edge regression model described
|
||||
:ref:`above <guide-training-edge-classification>`.
|
||||
|
||||
Here is an example of using dot product to compute the scores on edges.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class DotProductPredictor(nn.Module):
|
||||
def forward(self, graph, h):
|
||||
# h contains the node representations computed from the GNN defined
|
||||
# in the node classification section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h
|
||||
graph.apply_edges(fn.u_dot_v('h', 'h', 'score'))
|
||||
return graph.edata['score']
|
||||
|
||||
Training loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Because our score prediction model operates on graphs, we need to
|
||||
express the negative examples as another graph. The graph will contain
|
||||
all negative node pairs as edges.
|
||||
|
||||
The following shows an example of expressing negative examples as a
|
||||
graph. Each edge :math:`(u,v)` gets :math:`k` negative examples
|
||||
:math:`(u,v_i)` where :math:`v_i` is sampled from a uniform
|
||||
distribution.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def construct_negative_graph(graph, k):
|
||||
src, dst = graph.edges()
|
||||
|
||||
neg_src = src.repeat_interleave(k)
|
||||
neg_dst = torch.randint(0, graph.num_nodes(), (len(src) * k,))
|
||||
return dgl.graph((neg_src, neg_dst), num_nodes=graph.num_nodes())
|
||||
|
||||
The model that predicts edge scores is the same as that of edge
|
||||
classification/regression.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features):
|
||||
super().__init__()
|
||||
self.sage = SAGE(in_features, hidden_features, out_features)
|
||||
self.pred = DotProductPredictor()
|
||||
def forward(self, g, neg_g, x):
|
||||
h = self.sage(g, x)
|
||||
return self.pred(g, h), self.pred(neg_g, h)
|
||||
|
||||
The training loop then repeatedly constructs the negative graph and
|
||||
computes loss.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def compute_loss(pos_score, neg_score):
|
||||
# Margin loss
|
||||
n_edges = pos_score.shape[0]
|
||||
return (1 - pos_score + neg_score.view(n_edges, -1)).clamp(min=0).mean()
|
||||
|
||||
node_features = graph.ndata['feat']
|
||||
n_features = node_features.shape[1]
|
||||
k = 5
|
||||
model = Model(n_features, 100, 100)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(10):
|
||||
negative_graph = construct_negative_graph(graph, k)
|
||||
pos_score, neg_score = model(graph, negative_graph, node_features)
|
||||
loss = compute_loss(pos_score, neg_score)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
|
||||
After training, the node representation can be obtained via
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_embeddings = model.sage(graph, node_features)
|
||||
|
||||
There are multiple ways of using the node embeddings. Examples include
|
||||
training downstream classifiers, or doing nearest neighbor search or
|
||||
maximum inner product search for relevant entity recommendation.
|
||||
|
||||
Heterogeneous graphs
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Link prediction on heterogeneous graphs is not very different from that
|
||||
on homogeneous graphs. The following assumes that we are predicting on
|
||||
one edge type, and it is easy to extend it to multiple edge types.
|
||||
|
||||
For example, you can reuse the ``HeteroDotProductPredictor``
|
||||
:ref:`above <guide-training-edge-classification-heterogeneous-graph>`
|
||||
for computing the scores of the edges of an edge type for link prediction.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class HeteroDotProductPredictor(nn.Module):
|
||||
def forward(self, graph, h, etype):
|
||||
# h contains the node representations for each node type computed from
|
||||
# the GNN defined in the previous section (Section 5.1).
|
||||
with graph.local_scope():
|
||||
graph.ndata['h'] = h
|
||||
graph.apply_edges(fn.u_dot_v('h', 'h', 'score'), etype=etype)
|
||||
return graph.edges[etype].data['score']
|
||||
|
||||
To perform negative sampling, one can construct a negative graph for the
|
||||
edge type you are performing link prediction on as well.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def construct_negative_graph(graph, k, etype):
|
||||
utype, _, vtype = etype
|
||||
src, dst = graph.edges(etype=etype)
|
||||
neg_src = src.repeat_interleave(k)
|
||||
neg_dst = torch.randint(0, graph.num_nodes(vtype), (len(src) * k,))
|
||||
return dgl.heterograph(
|
||||
{etype: (neg_src, neg_dst)},
|
||||
num_nodes_dict={ntype: graph.num_nodes(ntype) for ntype in graph.ntypes})
|
||||
|
||||
The model is a bit different from that in edge classification on
|
||||
heterogeneous graphs since you need to specify edge type where you
|
||||
perform link prediction.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Model(nn.Module):
|
||||
def __init__(self, in_features, hidden_features, out_features, rel_names):
|
||||
super().__init__()
|
||||
self.sage = RGCN(in_features, hidden_features, out_features, rel_names)
|
||||
self.pred = HeteroDotProductPredictor()
|
||||
def forward(self, g, neg_g, x, etype):
|
||||
h = self.sage(g, x)
|
||||
return self.pred(g, h, etype), self.pred(neg_g, h, etype)
|
||||
|
||||
The training loop is similar to that of homogeneous graphs.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def compute_loss(pos_score, neg_score):
|
||||
# Margin loss
|
||||
n_edges = pos_score.shape[0]
|
||||
return (1 - pos_score + neg_score.view(n_edges, -1)).clamp(min=0).mean()
|
||||
|
||||
k = 5
|
||||
model = Model(10, 20, 5, hetero_graph.etypes)
|
||||
user_feats = hetero_graph.nodes['user'].data['feature']
|
||||
item_feats = hetero_graph.nodes['item'].data['feature']
|
||||
node_features = {'user': user_feats, 'item': item_feats}
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
for epoch in range(10):
|
||||
negative_graph = construct_negative_graph(hetero_graph, k, ('user', 'click', 'item'))
|
||||
pos_score, neg_score = model(hetero_graph, negative_graph, node_features, ('user', 'click', 'item'))
|
||||
loss = compute_loss(pos_score, neg_score)
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
.. _guide-training-node-classification:
|
||||
|
||||
5.1 Node Classification/Regression
|
||||
--------------------------------------------------
|
||||
|
||||
:ref:`(中文版) <guide_cn-training-node-classification>`
|
||||
|
||||
One of the most popular and widely adopted tasks for graph neural
|
||||
networks is node classification, where each node in the
|
||||
training/validation/test set is assigned a ground truth category from a
|
||||
set of predefined categories. Node regression is similar, where each
|
||||
node in the training/validation/test set is assigned a ground truth
|
||||
number.
|
||||
|
||||
Overview
|
||||
~~~~~~~~
|
||||
|
||||
To classify nodes, graph neural network performs message passing
|
||||
discussed in :ref:`guide-message-passing` to utilize the node’s own
|
||||
features, but also its neighboring node and edge features. Message
|
||||
passing can be repeated multiple rounds to incorporate information from
|
||||
larger range of neighborhood.
|
||||
|
||||
Writing neural network model
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL provides a few built-in graph convolution modules that can perform
|
||||
one round of message passing. In this guide, we choose
|
||||
:class:`dgl.nn.pytorch.SAGEConv` (also available in MXNet and Tensorflow),
|
||||
the graph convolution module for GraphSAGE.
|
||||
|
||||
Usually for deep learning models on graphs we need a multi-layer graph
|
||||
neural network, where we do multiple rounds of message passing. This can
|
||||
be achieved by stacking graph convolution modules as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Contruct a two-layer GNN model
|
||||
import dgl.nn as dglnn
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
class SAGE(nn.Module):
|
||||
def __init__(self, in_feats, hid_feats, out_feats):
|
||||
super().__init__()
|
||||
self.conv1 = dglnn.SAGEConv(
|
||||
in_feats=in_feats, out_feats=hid_feats, aggregator_type='mean')
|
||||
self.conv2 = dglnn.SAGEConv(
|
||||
in_feats=hid_feats, out_feats=out_feats, aggregator_type='mean')
|
||||
|
||||
def forward(self, graph, inputs):
|
||||
# inputs are features of nodes
|
||||
h = self.conv1(graph, inputs)
|
||||
h = F.relu(h)
|
||||
h = self.conv2(graph, h)
|
||||
return h
|
||||
|
||||
Note that you can use the model above for not only node classification,
|
||||
but also obtaining hidden node representations for other downstream
|
||||
tasks such as
|
||||
:ref:`guide-training-edge-classification`,
|
||||
:ref:`guide-training-link-prediction`, or
|
||||
:ref:`guide-training-graph-classification`.
|
||||
|
||||
For a complete list of built-in graph convolution modules, please refer
|
||||
to :ref:`apinn`.
|
||||
|
||||
For more details in how DGL
|
||||
neural network modules work and how to write a custom neural network
|
||||
module with message passing please refer to the example in :ref:`guide-nn`.
|
||||
|
||||
Training loop
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Training on the full graph simply involves a forward propagation of the
|
||||
model defined above, and computing the loss by comparing the prediction
|
||||
against ground truth labels on the training nodes.
|
||||
|
||||
This section uses a DGL built-in dataset
|
||||
:class:`dgl.data.CiteseerGraphDataset` to
|
||||
show a training loop. The node features
|
||||
and labels are stored on its graph instance, and the
|
||||
training-validation-test split are also stored on the graph as boolean
|
||||
masks. This is similar to what you have seen in :ref:`guide-data-pipeline`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_features = graph.ndata['feat']
|
||||
node_labels = graph.ndata['label']
|
||||
train_mask = graph.ndata['train_mask']
|
||||
valid_mask = graph.ndata['val_mask']
|
||||
test_mask = graph.ndata['test_mask']
|
||||
n_features = node_features.shape[1]
|
||||
n_labels = int(node_labels.max().item() + 1)
|
||||
|
||||
The following is an example of evaluating your model by accuracy.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def evaluate(model, graph, features, labels, mask):
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
logits = model(graph, features)
|
||||
logits = logits[mask]
|
||||
labels = labels[mask]
|
||||
_, indices = torch.max(logits, dim=1)
|
||||
correct = torch.sum(indices == labels)
|
||||
return correct.item() * 1.0 / len(labels)
|
||||
|
||||
You can then write our training loop as follows.
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = SAGE(in_feats=n_features, hid_feats=100, out_feats=n_labels)
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for epoch in range(10):
|
||||
model.train()
|
||||
# forward propagation by using all nodes
|
||||
logits = model(graph, node_features)
|
||||
# compute loss
|
||||
loss = F.cross_entropy(logits[train_mask], node_labels[train_mask])
|
||||
# compute validation accuracy
|
||||
acc = evaluate(model, graph, node_features, node_labels, valid_mask)
|
||||
# backward propagation
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
# Save model if necessary. Omitted in this example.
|
||||
|
||||
|
||||
`GraphSAGE <https://github.com/dmlc/dgl/blob/master/examples/pytorch/graphsage/train_full.py>`__
|
||||
provides an end-to-end homogeneous graph node classification example.
|
||||
You could see the corresponding model implementation is in the
|
||||
``GraphSAGE`` class in the example with adjustable number of layers,
|
||||
dropout probabilities, and customizable aggregation functions and
|
||||
nonlinearities.
|
||||
|
||||
.. _guide-training-rgcn-node-classification:
|
||||
|
||||
Heterogeneous graph
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If your graph is heterogeneous, you may want to gather message from
|
||||
neighbors along all edge types. You can use the module
|
||||
:class:`dgl.nn.pytorch.HeteroGraphConv` (also available in MXNet and Tensorflow)
|
||||
to perform message passing
|
||||
on all edge types, then combining different graph convolution modules
|
||||
for each edge type.
|
||||
|
||||
The following code will define a heterogeneous graph convolution module
|
||||
that first performs a separate graph convolution on each edge type, then
|
||||
sums the message aggregations on each edge type as the final result for
|
||||
all node types.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Define a Heterograph Conv model
|
||||
|
||||
class RGCN(nn.Module):
|
||||
def __init__(self, in_feats, hid_feats, out_feats, rel_names):
|
||||
super().__init__()
|
||||
|
||||
self.conv1 = dglnn.HeteroGraphConv({
|
||||
rel: dglnn.GraphConv(in_feats, hid_feats)
|
||||
for rel in rel_names}, aggregate='sum')
|
||||
self.conv2 = dglnn.HeteroGraphConv({
|
||||
rel: dglnn.GraphConv(hid_feats, out_feats)
|
||||
for rel in rel_names}, aggregate='sum')
|
||||
|
||||
def forward(self, graph, inputs):
|
||||
# inputs are features of nodes
|
||||
h = self.conv1(graph, inputs)
|
||||
h = {k: F.relu(v) for k, v in h.items()}
|
||||
h = self.conv2(graph, h)
|
||||
return h
|
||||
|
||||
``dgl.nn.HeteroGraphConv`` takes in a dictionary of node types and node
|
||||
feature tensors as input, and returns another dictionary of node types
|
||||
and node features.
|
||||
|
||||
So given that we have the user and item features in the
|
||||
:ref:`heterogeneous graph example <guide-training-heterogeneous-graph-example>`.
|
||||
|
||||
.. code:: python
|
||||
|
||||
model = RGCN(n_hetero_features, 20, n_user_classes, hetero_graph.etypes)
|
||||
user_feats = hetero_graph.nodes['user'].data['feature']
|
||||
item_feats = hetero_graph.nodes['item'].data['feature']
|
||||
labels = hetero_graph.nodes['user'].data['label']
|
||||
train_mask = hetero_graph.nodes['user'].data['train_mask']
|
||||
|
||||
One can simply perform a forward propagation as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
node_features = {'user': user_feats, 'item': item_feats}
|
||||
h_dict = model(hetero_graph, {'user': user_feats, 'item': item_feats})
|
||||
h_user = h_dict['user']
|
||||
h_item = h_dict['item']
|
||||
|
||||
Training loop is the same as the one for homogeneous graph, except that
|
||||
now you have a dictionary of node representations from which you compute
|
||||
the predictions. For instance, if you are only predicting the ``user``
|
||||
nodes, you can just extract the ``user`` node embeddings from the
|
||||
returned dictionary:
|
||||
|
||||
.. code:: python
|
||||
|
||||
opt = torch.optim.Adam(model.parameters())
|
||||
|
||||
for epoch in range(5):
|
||||
model.train()
|
||||
# forward propagation by using all nodes and extracting the user embeddings
|
||||
logits = model(hetero_graph, node_features)['user']
|
||||
# compute loss
|
||||
loss = F.cross_entropy(logits[train_mask], labels[train_mask])
|
||||
# Compute validation accuracy. Omitted in this example.
|
||||
# backward propagation
|
||||
opt.zero_grad()
|
||||
loss.backward()
|
||||
opt.step()
|
||||
print(loss.item())
|
||||
|
||||
# Save model if necessary. Omitted in the example.
|
||||
|
||||
|
||||
DGL provides an end-to-end example of
|
||||
`RGCN <https://github.com/dmlc/dgl/blob/master/examples/pytorch/rgcn-hetero/entity_classify.py>`__
|
||||
for node classification. You can see the definition of heterogeneous
|
||||
graph convolution in ``RelGraphConvLayer`` in the `model implementation
|
||||
file <https://github.com/dmlc/dgl/blob/master/examples/pytorch/rgcn-hetero/model.py>`__.
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
.. _guide-training:
|
||||
|
||||
Chapter 5: Training Graph Neural Networks
|
||||
=====================================================
|
||||
|
||||
:ref:`(中文版) <guide_cn-training>`
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
This chapter discusses how to train a graph neural network for node
|
||||
classification, edge classification, link prediction, and graph
|
||||
classification for small graph(s), by message passing methods introduced
|
||||
in :ref:`guide-message-passing` and neural network modules introduced in
|
||||
:ref:`guide-nn`.
|
||||
|
||||
This chapter assumes that your graph as well as all of its node and edge
|
||||
features can fit into GPU; see :ref:`guide-minibatch` if they cannot.
|
||||
|
||||
The following text assumes that the graph(s) and node/edge features are
|
||||
already prepared. If you plan to use the dataset DGL provides or other
|
||||
compatible ``DGLDataset`` as is described in :ref:`guide-data-pipeline`, you can
|
||||
get the graph for a single-graph dataset with something like
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
|
||||
dataset = dgl.data.CiteseerGraphDataset()
|
||||
graph = dataset[0]
|
||||
|
||||
|
||||
Note: In this chapter we will use PyTorch as backend.
|
||||
|
||||
.. _guide-training-heterogeneous-graph-example:
|
||||
|
||||
Heterogeneous Graphs
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Sometimes you would like to work on heterogeneous graphs. Here we take a
|
||||
synthetic heterogeneous graph as an example for demonstrating node
|
||||
classification, edge classification, and link prediction tasks.
|
||||
|
||||
The synthetic heterogeneous graph ``hetero_graph`` has these edge types:
|
||||
|
||||
- ``('user', 'follow', 'user')``
|
||||
- ``('user', 'followed-by', 'user')``
|
||||
- ``('user', 'click', 'item')``
|
||||
- ``('item', 'clicked-by', 'user')``
|
||||
- ``('user', 'dislike', 'item')``
|
||||
- ``('item', 'disliked-by', 'user')``
|
||||
|
||||
.. code:: python
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
n_users = 1000
|
||||
n_items = 500
|
||||
n_follows = 3000
|
||||
n_clicks = 5000
|
||||
n_dislikes = 500
|
||||
n_hetero_features = 10
|
||||
n_user_classes = 5
|
||||
n_max_clicks = 10
|
||||
|
||||
follow_src = np.random.randint(0, n_users, n_follows)
|
||||
follow_dst = np.random.randint(0, n_users, n_follows)
|
||||
click_src = np.random.randint(0, n_users, n_clicks)
|
||||
click_dst = np.random.randint(0, n_items, n_clicks)
|
||||
dislike_src = np.random.randint(0, n_users, n_dislikes)
|
||||
dislike_dst = np.random.randint(0, n_items, n_dislikes)
|
||||
|
||||
hetero_graph = dgl.heterograph({
|
||||
('user', 'follow', 'user'): (follow_src, follow_dst),
|
||||
('user', 'followed-by', 'user'): (follow_dst, follow_src),
|
||||
('user', 'click', 'item'): (click_src, click_dst),
|
||||
('item', 'clicked-by', 'user'): (click_dst, click_src),
|
||||
('user', 'dislike', 'item'): (dislike_src, dislike_dst),
|
||||
('item', 'disliked-by', 'user'): (dislike_dst, dislike_src)})
|
||||
|
||||
hetero_graph.nodes['user'].data['feature'] = torch.randn(n_users, n_hetero_features)
|
||||
hetero_graph.nodes['item'].data['feature'] = torch.randn(n_items, n_hetero_features)
|
||||
hetero_graph.nodes['user'].data['label'] = torch.randint(0, n_user_classes, (n_users,))
|
||||
hetero_graph.edges['click'].data['label'] = torch.randint(1, n_max_clicks, (n_clicks,)).float()
|
||||
# randomly generate training masks on user nodes and click edges
|
||||
hetero_graph.nodes['user'].data['train_mask'] = torch.zeros(n_users, dtype=torch.bool).bernoulli(0.6)
|
||||
hetero_graph.edges['click'].data['train_mask'] = torch.zeros(n_clicks, dtype=torch.bool).bernoulli(0.6)
|
||||
|
||||
|
||||
Roadmap
|
||||
------------
|
||||
|
||||
The chapter has four sections, each for one type of graph learning tasks.
|
||||
|
||||
* :ref:`guide-training-node-classification`
|
||||
* :ref:`guide-training-edge-classification`
|
||||
* :ref:`guide-training-link-prediction`
|
||||
* :ref:`guide-training-graph-classification`
|
||||
* :ref:`guide-training-eweight`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
training-node
|
||||
training-edge
|
||||
training-link
|
||||
training-graph
|
||||
training-eweight
|
||||
@@ -0,0 +1,89 @@
|
||||
.. _guide_cn-data-pipeline-dataset:
|
||||
|
||||
4.1 DGLDataset类
|
||||
--------------------
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline-dataset>`
|
||||
|
||||
:class:`~dgl.data.DGLDataset` 是处理、导入和保存 :ref:`apidata` 中定义的图数据集的基类。
|
||||
它实现了用于处理图数据的基本模版。下面的流程图展示了这个模版的工作方式。
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/userguide_data_flow.png
|
||||
:align: center
|
||||
|
||||
在类DGLDataset中定义的图数据处理模版的流程图。
|
||||
|
||||
为了处理位于远程服务器或本地磁盘上的图数据集，下面的例子中定义了一个类，称为 ``MyDataset``,
|
||||
它继承自 :class:`dgl.data.DGLDataset`。
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLDataset
|
||||
|
||||
class MyDataset(DGLDataset):
|
||||
""" 用于在DGL中自定义图数据集的模板：
|
||||
|
||||
Parameters
|
||||
----------
|
||||
url : str
|
||||
下载原始数据集的url。
|
||||
raw_dir : str
|
||||
指定下载数据的存储目录或已下载数据的存储目录。默认: ~/.dgl/
|
||||
save_dir : str
|
||||
处理完成的数据集的保存目录。默认：raw_dir指定的值
|
||||
force_reload : bool
|
||||
是否重新导入数据集。默认：False
|
||||
verbose : bool
|
||||
是否打印进度信息。
|
||||
"""
|
||||
def __init__(self,
|
||||
url=None,
|
||||
raw_dir=None,
|
||||
save_dir=None,
|
||||
force_reload=False,
|
||||
verbose=False):
|
||||
super(MyDataset, self).__init__(name='dataset_name',
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
save_dir=save_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def download(self):
|
||||
# 将原始数据下载到本地磁盘
|
||||
pass
|
||||
|
||||
def process(self):
|
||||
# 将原始数据处理为图、标签和数据集划分的掩码
|
||||
pass
|
||||
|
||||
def __getitem__(self, idx):
|
||||
# 通过idx得到与之对应的一个样本
|
||||
pass
|
||||
|
||||
def __len__(self):
|
||||
# 数据样本的数量
|
||||
pass
|
||||
|
||||
def save(self):
|
||||
# 将处理后的数据保存至 `self.save_path`
|
||||
pass
|
||||
|
||||
def load(self):
|
||||
# 从 `self.save_path` 导入处理后的数据
|
||||
pass
|
||||
|
||||
def has_cache(self):
|
||||
# 检查在 `self.save_path` 中是否存有处理后的数据
|
||||
pass
|
||||
|
||||
:class:`~dgl.data.DGLDataset` 类有抽象函数 ``process()``，
|
||||
``__getitem__(idx)`` 和 ``__len__()``。子类必须实现这些函数。同时DGL也建议实现保存和导入函数，
|
||||
因为对于处理后的大型数据集，这么做可以节省大量的时间，
|
||||
并且有多个已有的API可以简化此操作(请参阅 :ref:`guide_cn-data-pipeline-savenload`)。
|
||||
|
||||
请注意， :class:`~dgl.data.DGLDataset` 的目的是提供一种标准且方便的方式来导入图数据。
|
||||
用户可以存储有关数据集的图、特征、标签、掩码，以及诸如类别数、标签数等基本信息。
|
||||
诸如采样、划分或特征归一化等操作建议在 :class:`~dgl.data.DGLDataset` 子类之外完成。
|
||||
|
||||
本章的后续部分展示了实现这些函数的最佳实践。
|
||||
@@ -0,0 +1,50 @@
|
||||
.. _guide_cn-data-pipeline-download:
|
||||
|
||||
4.2 下载原始数据（可选）
|
||||
--------------------------------
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline-download>`
|
||||
|
||||
如果用户的数据集已经在本地磁盘中，请确保它被存放在目录 ``raw_dir`` 中。
|
||||
如果用户想在任何地方运行代码而又不想自己下载数据并将其移动到正确的目录中，则可以通过实现函数 ``download()`` 来自动完成。
|
||||
|
||||
如果数据集是一个zip文件，可以直接继承 :class:`dgl.data.DGLBuiltinDataset` 类。后者支持解压缩zip文件。
|
||||
否则用户需要自己实现 ``download()``，具体可以参考 :class:`~dgl.data.QM7bDataset` 类：
|
||||
|
||||
.. code::
|
||||
|
||||
import os
|
||||
from dgl.data.utils import download
|
||||
|
||||
def download(self):
|
||||
# 存储文件的路径
|
||||
file_path = os.path.join(self.raw_dir, self.name + '.mat')
|
||||
# 下载文件
|
||||
download(self.url, path=file_path)
|
||||
|
||||
上面的代码将一个.mat文件下载到目录 ``self.raw_dir``。如果文件是.gz、.tar、.tar.gz或.tgz文件，请使用
|
||||
:func:`~dgl.data.utils.extract_archive` 函数进行解压缩。以下代码展示了如何在
|
||||
:class:`~dgl.data.BitcoinOTCDataset` 类中下载一个.gz文件：
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data.utils import download, check_sha1
|
||||
|
||||
def download(self):
|
||||
# 存储文件的路径，请确保使用与原始文件名相同的后缀
|
||||
gz_file_path = os.path.join(self.raw_dir, self.name + '.csv.gz')
|
||||
# 下载文件
|
||||
download(self.url, path=gz_file_path)
|
||||
# 检查 SHA-1
|
||||
if not check_sha1(gz_file_path, self._sha1_str):
|
||||
raise UserWarning('File {} is downloaded but the content hash does not match.'
|
||||
'The repo may be outdated or download may be incomplete. '
|
||||
'Otherwise you can create an issue for it.'.format(self.name + '.csv.gz'))
|
||||
# 将文件解压缩到目录self.raw_dir下的self.name目录中
|
||||
self._extract_gz(gz_file_path, self.raw_path)
|
||||
|
||||
上面的代码会将文件解压缩到 ``self.raw_dir`` 下的目录 ``self.name`` 中。
|
||||
如果该类继承自 :class:`dgl.data.DGLBuiltinDataset` 来处理zip文件，
|
||||
则它也会将文件解压缩到目录 ``self.name`` 中。
|
||||
|
||||
一个可选项是用户可以按照上面的示例检查下载后文件的SHA-1字符串，以防作者在远程服务器上更改了文件。
|
||||
@@ -0,0 +1,76 @@
|
||||
.. _guide_cn-data-pipeline-loadogb:
|
||||
|
||||
4.5 使用ogb包导入OGB数据集
|
||||
----------------------------------------------
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline-loadogb>`
|
||||
|
||||
`Open Graph Benchmark (OGB) <https://ogb.stanford.edu/docs/home/>`__ 是一个图深度学习的基准数据集。
|
||||
官方的 `ogb <https://github.com/snap-stanford/ogb>`__ 包提供了用于下载和处理OGB数据集到
|
||||
:class:`dgl.data.DGLGraph` 对象的API。本节会介绍它们的基本用法。
|
||||
|
||||
首先使用pip安装ogb包：
|
||||
|
||||
.. code::
|
||||
|
||||
pip install ogb
|
||||
|
||||
|
||||
以下代码显示了如何为 *Graph Property Prediction* 任务加载数据集。
|
||||
|
||||
.. code::
|
||||
|
||||
# 载入OGB的Graph Property Prediction数据集
|
||||
import dgl
|
||||
import torch
|
||||
from ogb.graphproppred import DglGraphPropPredDataset
|
||||
from dgl.dataloading import GraphDataLoader
|
||||
|
||||
def _collate_fn(batch):
|
||||
# 小批次是一个元组(graph, label)列表
|
||||
graphs = [e[0] for e in batch]
|
||||
g = dgl.batch(graphs)
|
||||
labels = [e[1] for e in batch]
|
||||
labels = torch.stack(labels, 0)
|
||||
return g, labels
|
||||
|
||||
# 载入数据集
|
||||
dataset = DglGraphPropPredDataset(name='ogbg-molhiv')
|
||||
split_idx = dataset.get_idx_split()
|
||||
# dataloader
|
||||
train_loader = GraphDataLoader(dataset[split_idx["train"]], batch_size=32, shuffle=True, collate_fn=_collate_fn)
|
||||
valid_loader = GraphDataLoader(dataset[split_idx["valid"]], batch_size=32, shuffle=False, collate_fn=_collate_fn)
|
||||
test_loader = GraphDataLoader(dataset[split_idx["test"]], batch_size=32, shuffle=False, collate_fn=_collate_fn)
|
||||
|
||||
加载 *Node Property Prediction* 数据集类似，但要注意的是这种数据集只有一个图对象。
|
||||
|
||||
.. code::
|
||||
|
||||
# 载入OGB的Node Property Prediction数据集
|
||||
from ogb.nodeproppred import DglNodePropPredDataset
|
||||
|
||||
dataset = DglNodePropPredDataset(name='ogbn-proteins')
|
||||
split_idx = dataset.get_idx_split()
|
||||
|
||||
# there is only one graph in Node Property Prediction datasets
|
||||
# 在Node Property Prediction数据集里只有一个图
|
||||
g, labels = dataset[0]
|
||||
# 获取划分的标签
|
||||
train_label = dataset.labels[split_idx['train']]
|
||||
valid_label = dataset.labels[split_idx['valid']]
|
||||
test_label = dataset.labels[split_idx['test']]
|
||||
|
||||
每个 *Link Property Prediction* 数据集也只包括一个图。
|
||||
|
||||
.. code::
|
||||
|
||||
# 载入OGB的Link Property Prediction数据集
|
||||
from ogb.linkproppred import DglLinkPropPredDataset
|
||||
|
||||
dataset = DglLinkPropPredDataset(name='ogbl-ppa')
|
||||
split_edge = dataset.get_edge_split()
|
||||
|
||||
graph = dataset[0]
|
||||
print(split_edge['train'].keys())
|
||||
print(split_edge['valid'].keys())
|
||||
print(split_edge['test'].keys())
|
||||
@@ -0,0 +1,300 @@
|
||||
.. _guide_cn-data-pipeline-process:
|
||||
|
||||
4.3 处理数据
|
||||
----------------
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline-process>`
|
||||
|
||||
用户可以在 ``process()`` 函数中实现数据处理。该函数假定原始数据已经位于 ``self.raw_dir`` 目录中。
|
||||
|
||||
图上的机器学习任务通常有三种类型：整图分类、节点分类和链接预测。本节将展示如何处理与这些任务相关的数据集。
|
||||
|
||||
本节重点介绍了处理图、特征和划分掩码的标准方法。用户指南将以内置数据集为例，并跳过从文件构建图的实现。
|
||||
用户可以参考 :ref:`guide_cn-graph-external` 以查看如何从外部数据源构建图的完整指南。
|
||||
|
||||
处理整图分类数据集
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
整图分类数据集与用小批次训练的典型机器学习任务中的大多数数据集类似。
|
||||
因此，需要将原始数据处理为 :class:`dgl.DGLGraph` 对象的列表和标签张量的列表。
|
||||
此外，如果原始数据已被拆分为多个文件，则可以添加参数 ``split`` 以导入数据的特定部分。
|
||||
|
||||
下面以 :class:`~dgl.data.QM7bDataset` 为例：
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLDataset
|
||||
|
||||
class QM7bDataset(DGLDataset):
|
||||
_url = 'http://deepchem.io.s3-website-us-west-1.amazonaws.com/' \
|
||||
'datasets/qm7b.mat'
|
||||
_sha1_str = '4102c744bb9d6fd7b40ac67a300e49cd87e28392'
|
||||
|
||||
def __init__(self, raw_dir=None, force_reload=False, verbose=False):
|
||||
super(QM7bDataset, self).__init__(name='qm7b',
|
||||
url=self._url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
mat_path = self.raw_path + '.mat'
|
||||
# 将数据处理为图列表和标签列表
|
||||
self.graphs, self.label = self._load_graph(mat_path)
|
||||
|
||||
def __getitem__(self, idx):
|
||||
""" 通过idx获取对应的图和标签
|
||||
|
||||
Parameters
|
||||
----------
|
||||
idx : int
|
||||
Item index
|
||||
|
||||
Returns
|
||||
-------
|
||||
(dgl.DGLGraph, Tensor)
|
||||
"""
|
||||
return self.graphs[idx], self.label[idx]
|
||||
|
||||
def __len__(self):
|
||||
"""数据集中图的数量"""
|
||||
return len(self.graphs)
|
||||
|
||||
函数 ``process()`` 将原始数据处理为图列表和标签列表。用户必须实现 ``__getitem__(idx)`` 和 ``__len__()`` 以进行迭代。
|
||||
DGL建议让 ``__getitem__(idx)`` 返回如上面代码所示的元组 ``(图，标签)``。
|
||||
用户可以参考 `QM7bDataset源代码 <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/qm7b.html#QM7bDataset>`__
|
||||
以获得 ``self._load_graph()`` 和 ``__getitem__`` 的详细信息。
|
||||
|
||||
用户还可以向类添加属性以指示一些有用的数据集信息。在 :class:`~dgl.data.QM7bDataset` 中，
|
||||
用户可以添加属性 ``num_tasks`` 来指示此多任务数据集中的预测任务总数：
|
||||
|
||||
.. code::
|
||||
|
||||
@property
|
||||
def num_tasks(self):
|
||||
"""每个图的标签数，即预测任务数。"""
|
||||
return 14
|
||||
|
||||
在编写完这些代码之后，用户可以按如下所示的方式来使用 :class:`~dgl.data.QM7bDataset`：
|
||||
|
||||
.. code::
|
||||
|
||||
import dgl
|
||||
import torch
|
||||
|
||||
from dgl.dataloading import GraphDataLoader
|
||||
|
||||
# 数据导入
|
||||
dataset = QM7bDataset()
|
||||
num_tasks = dataset.num_tasks
|
||||
|
||||
# 创建 dataloaders
|
||||
dataloader = GraphDataLoader(dataset, batch_size=1, shuffle=True)
|
||||
|
||||
# 训练
|
||||
for epoch in range(100):
|
||||
for g, labels in dataloader:
|
||||
# 用户自己的训练代码
|
||||
pass
|
||||
|
||||
训练整图分类模型的完整指南可以在 :ref:`guide_cn-training-graph-classification` 中找到。
|
||||
|
||||
有关整图分类数据集的更多示例，用户可以参考 :ref:`guide_cn-training-graph-classification`：
|
||||
|
||||
* :ref:`gindataset`
|
||||
|
||||
* :ref:`minigcdataset`
|
||||
|
||||
* :ref:`qm7bdata`
|
||||
|
||||
* :ref:`tudata`
|
||||
|
||||
处理节点分类数据集
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
与整图分类不同，节点分类通常在单个图上进行。因此数据集的划分是在图的节点集上进行。
|
||||
DGL建议使用节点掩码来指定数据集的划分。
|
||||
本节以内置数据集 `CitationGraphDataset <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/citation_graph.html#CitationGraphDataset>`__ 为例：
|
||||
|
||||
此外，DGL推荐重新排列图的节点/边，使得相邻节点/边的ID位于邻近区间内。这个过程
|
||||
可以提高节点/边的邻居的局部性，为后续在图上进行的计算与分析的性能改善提供可能。
|
||||
DGL提供了名为 :func:`dgl.reorder_graph` 的API用于此优化。更多细节，请参考
|
||||
下面例子中的 ``process()`` 的部分。
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import DGLBuiltinDataset
|
||||
from dgl.data.utils import _get_dgl_url
|
||||
|
||||
class CitationGraphDataset(DGLBuiltinDataset):
|
||||
_urls = {
|
||||
'cora_v2' : 'dataset/cora_v2.zip',
|
||||
'citeseer' : 'dataset/citeseer.zip',
|
||||
'pubmed' : 'dataset/pubmed.zip',
|
||||
}
|
||||
|
||||
def __init__(self, name, raw_dir=None, force_reload=False, verbose=True):
|
||||
assert name.lower() in ['cora', 'citeseer', 'pubmed']
|
||||
if name.lower() == 'cora':
|
||||
name = 'cora_v2'
|
||||
url = _get_dgl_url(self._urls[name])
|
||||
super(CitationGraphDataset, self).__init__(name,
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
# 跳过一些处理的代码
|
||||
# === 跳过数据处理 ===
|
||||
|
||||
# 构建图
|
||||
g = dgl.graph(graph)
|
||||
|
||||
# 划分掩码
|
||||
g.ndata['train_mask'] = train_mask
|
||||
g.ndata['val_mask'] = val_mask
|
||||
g.ndata['test_mask'] = test_mask
|
||||
|
||||
# 节点的标签
|
||||
g.ndata['label'] = torch.tensor(labels)
|
||||
|
||||
# 节点的特征
|
||||
g.ndata['feat'] = torch.tensor(_preprocess_features(features),
|
||||
dtype=F.data_type_dict['float32'])
|
||||
self._num_tasks = onehot_labels.shape[1]
|
||||
self._labels = labels
|
||||
# 重排图以获得更优的局部性
|
||||
self._g = dgl.reorder_graph(g)
|
||||
|
||||
def __getitem__(self, idx):
|
||||
assert idx == 0, "这个数据集里只有一个图"
|
||||
return self._g
|
||||
|
||||
def __len__(self):
|
||||
return 1
|
||||
|
||||
为简便起见，这里省略了 ``process()`` 中的一些代码，以突出展示用于处理节点分类数据集的关键部分：划分掩码。
|
||||
节点特征和节点的标签被存储在 ``g.ndata`` 中。详细的实现请参考
|
||||
`CitationGraphDataset源代码 <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/citation_graph.html#CitationGraphDataset>`__ 。
|
||||
|
||||
请注意，这里 ``__getitem__(idx)`` 和 ``__len__()`` 的实现也发生了变化，
|
||||
这是因为节点分类任务通常只用一个图。掩码在PyTorch和TensorFlow中是bool张量，在MXNet中是float张量。
|
||||
|
||||
下面中使用 :class:`dgl.data.CitationGraphDataset` 的子类 :class:`dgl.data.CiteseerGraphDataset`
|
||||
来演示如何使用用于节点分类的数据集：
|
||||
|
||||
.. code::
|
||||
|
||||
# 导入数据
|
||||
dataset = CiteseerGraphDataset(raw_dir='')
|
||||
graph = dataset[0]
|
||||
|
||||
# 获取划分的掩码
|
||||
train_mask = graph.ndata['train_mask']
|
||||
val_mask = graph.ndata['val_mask']
|
||||
test_mask = graph.ndata['test_mask']
|
||||
|
||||
# 获取节点特征
|
||||
feats = graph.ndata['feat']
|
||||
|
||||
# 获取标签
|
||||
labels = graph.ndata['label']
|
||||
|
||||
:ref:`guide_cn-training-node-classification` 提供了训练节点分类模型的完整指南。
|
||||
|
||||
有关节点分类数据集的更多示例，用户可以参考以下内置数据集：
|
||||
|
||||
* :ref:`citationdata`
|
||||
|
||||
* :ref:`corafulldata`
|
||||
|
||||
* :ref:`amazoncobuydata`
|
||||
|
||||
* :ref:`coauthordata`
|
||||
|
||||
* :ref:`karateclubdata`
|
||||
|
||||
* :ref:`ppidata`
|
||||
|
||||
* :ref:`redditdata`
|
||||
|
||||
* :ref:`sbmdata`
|
||||
|
||||
* :ref:`sstdata`
|
||||
|
||||
* :ref:`rdfdata`
|
||||
|
||||
处理链接预测数据集
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
链接预测数据集的处理与节点分类相似，数据集中通常只有一个图。
|
||||
|
||||
本节以内置的数据集 `KnowledgeGraphDataset <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/knowledge_graph.html#KnowledgeGraphDataset>`__
|
||||
为例，同时省略了详细的数据处理代码以突出展示处理链接预测数据集的关键部分：
|
||||
|
||||
.. code::
|
||||
|
||||
# 创建链接预测数据集示例
|
||||
class KnowledgeGraphDataset(DGLBuiltinDataset):
|
||||
def __init__(self, name, reverse=True, raw_dir=None, force_reload=False, verbose=True):
|
||||
self._name = name
|
||||
self.reverse = reverse
|
||||
url = _get_dgl_url('dataset/') + '{}.tgz'.format(name)
|
||||
super(KnowledgeGraphDataset, self).__init__(name,
|
||||
url=url,
|
||||
raw_dir=raw_dir,
|
||||
force_reload=force_reload,
|
||||
verbose=verbose)
|
||||
|
||||
def process(self):
|
||||
# 跳过一些处理的代码
|
||||
# === 跳过数据处理 ===
|
||||
|
||||
# 划分掩码
|
||||
g.edata['train_mask'] = train_mask
|
||||
g.edata['val_mask'] = val_mask
|
||||
g.edata['test_mask'] = test_mask
|
||||
|
||||
# 边类型
|
||||
g.edata['etype'] = etype
|
||||
|
||||
# 节点类型
|
||||
g.ndata['ntype'] = ntype
|
||||
self._g = g
|
||||
|
||||
def __getitem__(self, idx):
|
||||
assert idx == 0, "这个数据集只有一个图"
|
||||
return self._g
|
||||
|
||||
def __len__(self):
|
||||
return 1
|
||||
|
||||
|
||||
如代码所示，图的 ``edata`` 存储了划分掩码。在
|
||||
`KnowledgeGraphDataset 源代码 <https://docs.dgl.ai/en/0.5.x/_modules/dgl/data/knowledge_graph.html#KnowledgeGraphDataset>`__
|
||||
中可以查看完整的代码。下面使用 ``KnowledgeGraphDataset``的子类 :class:`dgl.data.FB15k237Dataset` 来做演示如何使用用于链路预测的数据集：
|
||||
|
||||
.. code::
|
||||
|
||||
from dgl.data import FB15k237Dataset
|
||||
|
||||
# 导入数据
|
||||
dataset = FB15k237Dataset()
|
||||
graph = dataset[0]
|
||||
|
||||
# 获取训练集掩码
|
||||
train_mask = graph.edata['train_mask']
|
||||
train_idx = torch.nonzero(train_mask, as_tuple=False).squeeze()
|
||||
src, dst = graph.edges(train_idx)
|
||||
|
||||
# 获取训练集中的边类型
|
||||
rel = graph.edata['etype'][train_idx]
|
||||
|
||||
有关训练链接预测模型的完整指南，请参见 :ref:`guide_cn-training-link-prediction`。
|
||||
|
||||
有关链接预测数据集的更多示例，请参考DGL的内置数据集：
|
||||
|
||||
* :ref:`kgdata`
|
||||
|
||||
* :ref:`bitcoinotcdata`
|
||||
@@ -0,0 +1,45 @@
|
||||
.. _guide_cn-data-pipeline-savenload:
|
||||
|
||||
4.4 保存和加载数据
|
||||
----------------------
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline-savenload>`
|
||||
|
||||
DGL建议用户实现保存和加载数据的函数，将处理后的数据缓存在本地磁盘中。
|
||||
这样在多数情况下可以帮用户节省大量的数据处理时间。DGL提供了4个函数让任务变得简单。
|
||||
|
||||
- :func:`dgl.save_graphs` 和 :func:`dgl.load_graphs`: 保存DGLGraph对象和标签到本地磁盘和从本地磁盘读取它们。
|
||||
- :func:`dgl.data.utils.save_info` 和 :func:`dgl.data.utils.load_info`: 将数据集的有用信息(python dict对象)保存到本地磁盘和从本地磁盘读取它们。
|
||||
|
||||
下面的示例显示了如何保存和读取图和数据集信息的列表。
|
||||
|
||||
.. code::
|
||||
|
||||
import os
|
||||
from dgl import save_graphs, load_graphs
|
||||
from dgl.data.utils import makedirs, save_info, load_info
|
||||
|
||||
def save(self):
|
||||
# 保存图和标签
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
save_graphs(graph_path, self.graphs, {'labels': self.labels})
|
||||
# 在Python字典里保存其他信息
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
save_info(info_path, {'num_classes': self.num_classes})
|
||||
|
||||
def load(self):
|
||||
# 从目录 `self.save_path` 里读取处理过的数据
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
self.graphs, label_dict = load_graphs(graph_path)
|
||||
self.labels = label_dict['labels']
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
self.num_classes = load_info(info_path)['num_classes']
|
||||
|
||||
def has_cache(self):
|
||||
# 检查在 `self.save_path` 里是否有处理过的数据文件
|
||||
graph_path = os.path.join(self.save_path, self.mode + '_dgl_graph.bin')
|
||||
info_path = os.path.join(self.save_path, self.mode + '_info.pkl')
|
||||
return os.path.exists(graph_path) and os.path.exists(info_path)
|
||||
|
||||
请注意：有些情况下不适合保存处理过的数据。例如，在内置数据集 :class:`~dgl.data.GDELTDataset` 中，
|
||||
处理过的数据比较大。所以这个时候，在 ``__getitem__(idx)`` 中处理每个数据实例是更高效的方法。
|
||||
@@ -0,0 +1,31 @@
|
||||
.. _guide_cn-data-pipeline:
|
||||
|
||||
第4章：图数据处理管道
|
||||
==============================
|
||||
|
||||
:ref:`(English Version) <guide-data-pipeline>`
|
||||
|
||||
DGL在 :ref:`apidata` 里实现了很多常用的图数据集。它们遵循了由 :class:`dgl.data.DGLDataset` 类定义的标准的数据处理管道。
|
||||
DGL推荐用户将图数据处理为 :class:`dgl.data.DGLDataset` 的子类。该类为导入、处理和保存图数据提供了简单而干净的解决方案。
|
||||
|
||||
本章路线图
|
||||
-----------
|
||||
|
||||
本章介绍了如何为用户自己的图数据创建一个DGL数据集。以下内容说明了管道的工作方式，并展示了如何实现管道的每个组件。
|
||||
|
||||
* :ref:`guide_cn-data-pipeline-dataset`
|
||||
* :ref:`guide_cn-data-pipeline-download`
|
||||
* :ref:`guide_cn-data-pipeline-process`
|
||||
* :ref:`guide_cn-data-pipeline-savenload`
|
||||
* :ref:`guide_cn-data-pipeline-loadogb`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
data-dataset
|
||||
data-download
|
||||
data-process
|
||||
data-savenload
|
||||
data-loadogb
|
||||
@@ -0,0 +1,253 @@
|
||||
.. _guide_cn-distributed-apis:
|
||||
|
||||
7.2 分布式计算的API
|
||||
--------------------
|
||||
|
||||
:ref:`(English Version) <guide-distributed-apis>`
|
||||
|
||||
本节介绍了在训练脚本中使用的分布式计算API。DGL提供了三种分布式数据结构和多种API，用于初始化、分布式采样和数据分割。
|
||||
对于分布式训练/推断，DGL提供了三种分布式数据结构：用于分布式图的 :class:`~dgl.distributed.DistGraph`、
|
||||
用于分布式张量的 :class:`~dgl.distributed.DistTensor` 和用于分布式可学习嵌入的
|
||||
:class:`~dgl.distributed.DistEmbedding`。
|
||||
|
||||
DGL分布式模块的初始化
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
:func:`~dgl.distributed.initialize` 可以用于初始化分布式模块。当训练脚本在训练器模式下运行时，
|
||||
这个API会与DGL服务器建立连接并创建采样器进程。当脚本在服务器模式下运行时，这个API将运行服务器代码，
|
||||
直到训练任务结束。必须在DGL的任何其他分布式API之前，调用此API。在使用PyTorch时，必须在
|
||||
``torch.distributed.init_process_group`` 之前调用 :func:`~dgl.distributed.initialize`。
|
||||
通常，初始化API应按以下顺序调用：
|
||||
|
||||
.. code:: python
|
||||
|
||||
dgl.distributed.initialize('ip_config.txt')
|
||||
th.distributed.init_process_group(backend='gloo')
|
||||
|
||||
**Note**: 如果训练脚本里包含需要在服务器(细节内容可以在下面的DistTensor和DistEmbedding章节里查看)上调用的用户自定义函数(UDF)，
|
||||
这些UDF必须在 :func:`~dgl.distributed.initialize` 之前被声明。
|
||||
|
||||
分布式图
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` 是一个Python类，用于访问计算机集群中的图结构和节点/边特征。每台计算机负责一个且只负责一个分区。
|
||||
它加载分区数据(包括分区中的图结构、节点数据和边数据)，并使集群中的所有训练器均可访问它们。
|
||||
:class:`~dgl.distributed.DistGraph` 提供了一小部分 :class:`~dgl.DGLGraph` 的API以方便数据访问。
|
||||
|
||||
**Note**: :class:`~dgl.distributed.DistGraph` 当前仅支持一种节点类型和一种边类型的图。
|
||||
|
||||
分布式模式与独立模式
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` 可以在两种模式下运行：分布式模式和独立模式。
|
||||
当用户在Python命令行或Jupyter Notebook中执行训练脚本时，它将以独立模式运行。也就是说，它在单个进程中运行所有计算，
|
||||
并且不与任何其他进程通信。因此，独立模式要求输入图仅具有一个分区。此模式主要用于开发和测试
|
||||
(例如，在Jupyter Notebook中开发和运行代码)。当用户使用启动脚本执行训练脚本时(请参见启动脚本部分)，
|
||||
:class:`~dgl.distributed.DistGraph` 将以分布式模式运行。启动脚本在后台启动服务器(包括访问节点/边特征和图采样)，
|
||||
并将分区数据自动加载到每台计算机中。:class:`~dgl.distributed.DistGraph` 与集群中的服务器连接并通过网络访问它们。
|
||||
|
||||
创建DistGraph
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
在分布式模式下，:class:`~dgl.distributed.DistGraph` 的创建需要(定义)在图划分期间的图名称。
|
||||
图名称标识了集群中所需加载的图。
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
g = dgl.distributed.DistGraph('graph_name')
|
||||
|
||||
在独立模式下运行时，DistGraph将图数据加载到本地计算机中。因此，用户需要提供分区配置文件，其中包含有关输入图的所有信息。
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
g = dgl.distributed.DistGraph('graph_name', part_config='data/graph_name.json')
|
||||
|
||||
**Note**: 在当前实现中，DGL仅允许创建单个DistGraph对象。销毁DistGraph并创建一个新DistGraph的行为没有被定义。
|
||||
|
||||
访问图结构
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
:class:`~dgl.distributed.DistGraph` 提供了几个API来访问图结构。当前，它们主要被用来提供图信息，例如节点和边的数量。
|
||||
主要应用场景是运行采样API以支持小批量训练(请参阅下文里分布式图采样部分)。
|
||||
|
||||
.. code:: python
|
||||
|
||||
print(g.num_nodes())
|
||||
|
||||
访问节点/边数据
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
与 :class:`~dgl.DGLGraph` 一样， :class:`~dgl.distributed.DistGraph` 也提供了
|
||||
``ndata`` 和 ``edata`` 来访问节点和边中的数据。它们的区别在于
|
||||
:class:`~dgl.distributed.DistGraph` 中的 ``ndata`` / ``edata`` 返回的是 :class:`~dgl.distributed.DistTensor`，
|
||||
而不是底层框架里的张量。用户还可以将新的 :class:`~dgl.distributed.DistTensor` 分配给
|
||||
:class:`~dgl.distributed.DistGraph` 作为节点数据或边数据。
|
||||
|
||||
.. code:: python
|
||||
|
||||
g.ndata['train_mask']
|
||||
<dgl.distributed.dist_graph.DistTensor at 0x7fec820937b8>
|
||||
g.ndata['train_mask'][0]
|
||||
tensor([1], dtype=torch.uint8)
|
||||
|
||||
分布式张量
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
如前所述，在分布式模式下，DGL会划分节点和边特征，并将它们存储在计算机集群中。
|
||||
DGL为分布式张量提供了类似于单机普通张量的接口，以访问群集中的分区节点和边特征。
|
||||
在分布式设置中，DGL仅支持密集节点和边特征，暂不支持稀疏节点和边特征。
|
||||
|
||||
:class:`~dgl.distributed.DistTensor` 管理在多个计算机中被划分和存储的密集张量。
|
||||
目前，分布式张量必须与图的节点或边相关联。换句话说，DistTensor中的行数必须与图中的节点数或边数相同。
|
||||
以下代码创建一个分布式张量。 除了张量的形状和数据类型之外，用户还可以提供唯一的张量名称。
|
||||
如果用户要引用一个固定的分布式张量(即使 :class:`~dgl.distributed.DistTensor` 对象消失，该名称仍存在于群集中)，
|
||||
则(使用这样的)名称就很有用。
|
||||
|
||||
.. code:: python
|
||||
|
||||
tensor = dgl.distributed.DistTensor((g.num_nodes(), 10), th.float32, name='test')
|
||||
|
||||
**Note**: :class:`~dgl.distributed.DistTensor` 的创建是一个同步操作。所有训练器都必须调用创建，
|
||||
并且只有当所有训练器都调用它时，此创建过程才能成功。
|
||||
|
||||
用户可以将 :class:`~dgl.distributed.DistTensor` 作为节点数据或边数据之一添加到
|
||||
:class:`~dgl.distributed.DistGraph` 对象。
|
||||
|
||||
.. code:: python
|
||||
|
||||
g.ndata['feat'] = tensor
|
||||
|
||||
**Note**: 节点数据名称和张量名称不必相同。前者在 :class:`~dgl.distributed.DistGraph` 中标识节点数据(在训练器进程中)，
|
||||
而后者则标识DGL服务器中的分布式张量。
|
||||
|
||||
:class:`~dgl.distributed.DistTensor` 提供了一些功能。它具有与常规张量相同的API，用于访问其元数据，
|
||||
例如形状和数据类型。:class:`~dgl.distributed.DistTensor` 支持索引读取和写入，
|
||||
但不支持一些计算运算符，例如求和以及求均值。
|
||||
|
||||
.. code:: python
|
||||
|
||||
data = g.ndata['feat'][[1, 2, 3]]
|
||||
print(data)
|
||||
g.ndata['feat'][[3, 4, 5]] = data
|
||||
|
||||
**Note**: 当前，当一台机器运行多个服务器时，DGL不提供对来自多个训练器的并发写入的保护。
|
||||
这可能会导致数据损坏。
|
||||
|
||||
分布式嵌入
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL提供 :class:`~dgl.distributed.DistEmbedding` 以支持需要节点嵌入的直推(transductive)模型。
|
||||
分布式嵌入的创建与分布式张量的创建非常相似。
|
||||
|
||||
.. code:: python
|
||||
|
||||
def initializer(shape, dtype):
|
||||
arr = th.zeros(shape, dtype=dtype)
|
||||
arr.uniform_(-1, 1)
|
||||
return arr
|
||||
emb = dgl.distributed.DistEmbedding(g.num_nodes(), 10, init_func=initializer)
|
||||
|
||||
在内部，分布式嵌入建立在分布式张量之上，因此，其行为与分布式张量非常相似。
|
||||
例如，创建嵌入时，DGL会将它们分片并存储在集群中的所有计算机上。(分布式嵌入)可以通过名称唯一标识。
|
||||
|
||||
**Note**: 服务器进程负责调用初始化函数。因此，必须在初始化( :class:`~dgl.distributed.initialize` )之前声明分布式嵌入。
|
||||
|
||||
因为嵌入是模型的一部分，所以用户必须将其附加到优化器上以进行小批量训练。当前，
|
||||
DGL提供了一个稀疏的Adagrad优化器 :class:`~dgl.distributed.SparseAdagrad` (DGL以后将为稀疏嵌入添加更多的优化器)。
|
||||
用户需要从模型中收集所有分布式嵌入，并将它们传递给稀疏优化器。如果模型同时具有节点嵌入和规则的密集模型参数，
|
||||
并且用户希望对嵌入执行稀疏更新，则需要创建两个优化器，一个用于节点嵌入，另一个用于密集模型参数，如以下代码所示：
|
||||
|
||||
.. code:: python
|
||||
|
||||
sparse_optimizer = dgl.distributed.SparseAdagrad([emb], lr=lr1)
|
||||
optimizer = th.optim.Adam(model.parameters(), lr=lr2)
|
||||
feats = emb(nids)
|
||||
loss = model(feats)
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
sparse_optimizer.step()
|
||||
|
||||
**Note**: :class:`~dgl.distributed.DistEmbedding` 不是PyTorch的nn模块，因此用户无法从nn模块的参数访问它。
|
||||
|
||||
分布式采样
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
DGL提供了两个级别的API，用于对节点和边进行采样以生成小批次训练数据(请参阅小批次训练的章节)。
|
||||
底层API要求用户编写代码以明确定义如何对节点层进行采样(例如，使用 :func:`dgl.sampling.sample_neighbors` )。
|
||||
高层采样API为节点分类和链接预测任务实现了一些流行的采样算法（例如
|
||||
:class:`~dgl.dataloading.pytorch.NodeDataLoader`
|
||||
和
|
||||
:class:`~dgl.dataloading.pytorch.EdgeDataLoader` )。
|
||||
|
||||
分布式采样模块遵循相同的设计，也提供两个级别的采样API。对于底层的采样API，它为
|
||||
:class:`~dgl.distributed.DistGraph` 上的分布式邻居采样提供了
|
||||
:func:`~dgl.distributed.sample_neighbors`。另外，DGL提供了用于分布式采样的分布式数据加载器(
|
||||
:class:`~dgl.distributed.DistDataLoader`)。除了用户在创建数据加载器时无法指定工作进程的数量，
|
||||
分布式数据加载器具有与PyTorch DataLoader相同的接口。其中的工作进程(worker)在 :func:`dgl.distributed.initialize` 中创建。
|
||||
|
||||
**Note**: 在 :class:`~dgl.distributed.DistGraph` 上运行 :func:`dgl.distributed.sample_neighbors` 时，
|
||||
采样器无法在具有多个工作进程的PyTorch DataLoader中运行。主要原因是PyTorch DataLoader在每个训练周期都会创建新的采样工作进程，
|
||||
从而导致多次创建和删除 :class:`~dgl.distributed.DistGraph` 对象。
|
||||
|
||||
使用底层API时，采样代码类似于单进程采样。唯一的区别是用户需要使用
|
||||
:func:`dgl.distributed.sample_neighbors`
|
||||
和
|
||||
:class:`~dgl.distributed.DistDataLoader`。
|
||||
|
||||
.. code:: python
|
||||
|
||||
def sample_blocks(seeds):
|
||||
seeds = th.LongTensor(np.asarray(seeds))
|
||||
blocks = []
|
||||
for fanout in [10, 25]:
|
||||
frontier = dgl.distributed.sample_neighbors(g, seeds, fanout, replace=True)
|
||||
block = dgl.to_block(frontier, seeds)
|
||||
seeds = block.srcdata[dgl.NID]
|
||||
blocks.insert(0, block)
|
||||
return blocks
|
||||
dataloader = dgl.distributed.DistDataLoader(dataset=train_nid,
|
||||
batch_size=batch_size,
|
||||
collate_fn=sample_blocks,
|
||||
shuffle=True)
|
||||
for batch in dataloader:
|
||||
...
|
||||
|
||||
:class:`~dgl.dataloading.pytorch.NodeDataLoader`
|
||||
和
|
||||
:class:`~dgl.dataloading.pytorch.EdgeDataLoader` 有分布式的版本
|
||||
:class:`~dgl.dataloading.pytorch.DistNodeDataLoader`
|
||||
和
|
||||
:class:`~dgl.dataloading.pytorch.DistEdgeDataLoader` 。使用
|
||||
时分布式采样代码与单进程采样几乎完全相同。
|
||||
|
||||
.. code:: python
|
||||
|
||||
sampler = dgl.sampling.MultiLayerNeighborSampler([10, 25])
|
||||
dataloader = dgl.sampling.DistNodeDataLoader(g, train_nid, sampler,
|
||||
batch_size=batch_size, shuffle=True)
|
||||
for batch in dataloader:
|
||||
...
|
||||
|
||||
|
||||
分割数据集
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
用户需要分割训练集，以便每个训练器都可以使用自己的训练集子集。同样，用户还需要以相同的方式分割验证和测试集。
|
||||
|
||||
对于分布式训练和评估，推荐的方法是使用布尔数组表示训练、验证和测试集。对于节点分类任务，
|
||||
这些布尔数组的长度是图中节点的数量，并且它们的每个元素都表示训练/验证/测试集中是否存在对应节点。
|
||||
链接预测任务也应使用类似的布尔数组。
|
||||
|
||||
DGL提供了 :func:`~dgl.distributed.node_split` 和 :func:`~dgl.distributed.edge_split`
|
||||
函数来在运行时拆分训练、验证和测试集，以进行分布式训练。这两个函数将布尔数组作为输入，对其进行拆分，并向本地训练器返回一部分。
|
||||
默认情况下，它们确保所有部分都具有相同数量的节点和边。这对于同步SGD非常重要，
|
||||
因为同步SGD会假定每个训练器具有相同数量的小批次。
|
||||
|
||||
下面的示例演示了训练集拆分，并向本地进程返回节点的子集。
|
||||
|
||||
.. code:: python
|
||||
|
||||
train_nids = dgl.distributed.node_split(g.ndata['train_mask'])
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
.. _guide_cn-distributed-preprocessing:
|
||||
|
||||
7.1 分布式训练所需的图数据预处理
|
||||
------------------------------------------
|
||||
|
||||
:ref:`(English Version) <guide-distributed-preprocessing>`
|
||||
|
||||
DGL要求预处理图数据以进行分布式训练，这包括两个步骤：1)将一张图划分为多张子图(分区)，2)为节点和边分配新的ID。
|
||||
DGL提供了一个API以执行这两个步骤。该API支持随机划分和一个基于
|
||||
`Metis <http://glaros.dtc.umn.edu/gkhome/views/metis>`__ 的划分。Metis划分的好处在于，
|
||||
它可以用最少的边分割以生成分区，从而减少了用于分布式训练和推理的网络通信。DGL使用最新版本的Metis，
|
||||
并针对真实世界中具有幂律分布的图进行了优化。在图划分后，API以易于在训练期间加载的格式构造划分结果。
|
||||
|
||||
**Note**: 图划分API当前在一台机器上运行。 因此如果一张图很大，用户将需要一台大内存的机器来对图进行划分。
|
||||
未来DGL将支持分布式图划分。
|
||||
|
||||
默认情况下，为了在分布式训练/推理期间定位节点/边，API将新ID分配给输入图的节点和边。
|
||||
分配ID后，该API会相应地打乱所有节点数据和边数据。在训练期间，用户只需使用新的节点和边的ID。
|
||||
与此同时，用户仍然可以通过 ``g.ndata['orig_id']`` 和 ``g.edata['orig_id']`` 获取原始ID。
|
||||
其中 ``g`` 是 ``DistGraph`` 对象（详细解释，请参见:ref:`guide-distributed-apis`）。
|
||||
|
||||
DGL将图划分结果存储在输出目录中的多个文件中。输出目录里始终包含一个名为xxx.json的JSON文件，其中xxx是提供给划分API的图的名称。
|
||||
JSON文件包含所有划分的配置。如果该API没有为节点和边分配新ID，它将生成两个额外的NumPy文件：`node_map.npy` 和 `edge_map.npy`。
|
||||
它们存储节点和边ID与分区ID之间的映射。对于具有十亿级数量节点和边的图，两个文件中的NumPy数组会很大，
|
||||
这是因为图中的每个节点和边都对应一个条目。在每个分区的文件夹内，有3个文件以DGL格式存储分区数据。
|
||||
`graph.dgl` 存储分区的图结构以及节点和边上的一些元数据。`node_feats.dgl` 和 `edge_feats.dgl` 存储属于该分区的节点和边的所有特征。
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
data_root_dir/
|
||||
|-- xxx.json # JSON中的分区配置文件
|
||||
|-- node_map.npy # 存储在NumPy数组中的每个节点的分区ID（可选）
|
||||
|-- edge_map.npy # 存储在NumPy数组中的每个边的分区ID（可选）
|
||||
|-- part0/ # 分区0的数据
|
||||
|-- node_feats.dgl # 以二进制格式存储的节点特征
|
||||
|-- edge_feats.dgl # 以二进制格式存储的边特征
|
||||
|-- graph.dgl # 以二进制格式存储的子图结构
|
||||
|-- part1/ # 分区1的数据
|
||||
|-- node_feats.dgl
|
||||
|-- edge_feats.dgl
|
||||
|-- graph.dgl
|
||||
|
||||
负载均衡
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
在对图进行划分时，默认情况下，Metis仅平衡每个子图中的节点数。根据当前的任务情况，这可能带来非最优的配置。
|
||||
例如，在半监督节点分类的场景里，训练器会对局部分区中带标签节点的子集进行计算。
|
||||
一个仅平衡图中节点(带标签和未带标签)的划分可能会导致计算负载不平衡。为了在每个分区中获得平衡的工作负载，
|
||||
划分API通过在 :func:`dgl.distributed.partition_graph` 中指定 ``balance_ntypes``
|
||||
在每个节点类型中的节点数上实现分区间的平衡。用户可以利用这一点将训练集、验证集和测试集中的节点看作不同类型的节点。
|
||||
|
||||
以下示例将训练集内和训练集外的节点看作两种类型的节点：
|
||||
|
||||
.. code:: python
|
||||
|
||||
dgl.distributed.partition_graph(g, 'graph_name', 4, '/tmp/test', balance_ntypes=g.ndata['train_mask'])
|
||||
|
||||
除了平衡节点的类型之外， :func:`dgl.distributed.partition_graph` 还允许通过指定
|
||||
``balance_edges`` 来平衡每个类型节点在子图中的入度。这平衡了不同类型节点的连边数量。
|
||||
|
||||
**Note**: 传给 :func:`dgl.distributed.partition_graph` 的图名称是一个重要的参数。
|
||||
:class:`dgl.distributed.DistGraph` 使用该名称来识别一个分布式的图。一个有效的图名称应该仅包含字母和下划线。
|
||||
@@ -0,0 +1,61 @@
|
||||
.. _guide_cn-distributed-tools:
|
||||
|
||||
7.3 运行分布式训练/推断所需的工具
|
||||
------------------------------------------------------
|
||||
|
||||
:ref:`(English Version) <guide-distributed-tools>`
|
||||
|
||||
DGL提供了两个脚本来帮助用户进行分布式训练：
|
||||
|
||||
* *tools/copy_files.py* 用于将图分区复制到集群，
|
||||
* *tools/launch.py* 用于在机器集群中启动分布式训练任务。
|
||||
|
||||
*copy_files.py* 将计算机(对图进行分区的计算机)中的分区数据和相关文件(例如，训练脚本)
|
||||
复制到(负责分布式训练的)机器集群上。在这些机器上，分布式训练将需要用到这些分区。该脚本包含四个参数：
|
||||
|
||||
* ``--part_config`` 指定分区配置文件，该文件包含本地计算机中分区数据的信息。
|
||||
* ``--ip_config`` 指定集群的IP配置文件。
|
||||
* ``--workspace`` 指定训练机器中存储与分布式训练有关的所有数据的目录。
|
||||
* ``--rel_data_path`` 指定工作空间目录下存储分区数据的相对路径。
|
||||
* ``--script_folder`` 指定工作空间目录下存储用户的训练脚本的相对路径。
|
||||
|
||||
**Note**: *copy_files.py* 会根据IP配置文件找到对应的计算机来存储图分区。因此，copy_files.py和launch.py应该使用相同的IP配置文件。
|
||||
|
||||
DGL提供了用于启动集群中的分布式训练任务的tools/launch.py。该脚本有以下假设：
|
||||
|
||||
* 分区数据和训练脚本都已被复制到集群或存在集群中所有计算机均可访问的全局存储空间(例如NFS)。
|
||||
* 主计算机(执行启动脚本的计算机)具有对集群内所有其他计算机的无密码ssh访问权限。
|
||||
|
||||
**Note**: 必须在集群中的一台计算机上调用启动脚本。
|
||||
|
||||
下面展示了在集群中启动分布式训练任务的示例。
|
||||
|
||||
.. code:: none
|
||||
|
||||
python3 tools/launch.py \
|
||||
--workspace ~graphsage/ \
|
||||
--num_trainers 2 \
|
||||
--num_samplers 4 \
|
||||
--num_servers 1 \
|
||||
--part_config data/ogb-product.json \
|
||||
--ip_config ip_config.txt \
|
||||
"python3 code/train_dist.py --graph-name ogb-product --ip_config ip_config.txt --num-epochs 5 --batch-size 1000 --lr 0.1 --num_workers 4"
|
||||
|
||||
配置文件 *ip_config.txt* 包含了集群中计算机的IP地址。*ip_config.txt* 的典型示例如下：
|
||||
|
||||
.. code:: none
|
||||
|
||||
172.31.19.1
|
||||
172.31.23.205
|
||||
172.31.29.175
|
||||
172.31.16.98
|
||||
|
||||
每行是一个计算机的IP地址。IP地址后面还可以有一个端口，用来指定不同训练器之间的网络通信所使用的端口。
|
||||
如果未提供具体端口，则默认值为 ``30050``。
|
||||
|
||||
启动脚本中指定的工作空间(--workspace)是计算机中的工作目录，里面保存了训练脚本、IP配置文件、分区配置文件以及图分区。
|
||||
文件的所有路径都应指定为工作空间的相对路径。
|
||||
|
||||
启动脚本会在每台计算机上创建指定数量的训练任务(``--num_trainers``)。另外，
|
||||
用户需要为每个训练器指定采样器进程的数量(``--num_samplers``)。
|
||||
采样器进程的数量必须匹配 :func:`~dgl.distributed.initialize` 中指定的工作进程的数量。
|
||||
@@ -0,0 +1,94 @@
|
||||
.. _guide_cn-distributed:
|
||||
|
||||
第7章：分布式训练
|
||||
=====================================
|
||||
|
||||
:ref:`(English Version) <guide-distributed>`
|
||||
|
||||
DGL采用完全分布式的方法，可将数据和计算同时分布在一组计算资源中。在本节中，
|
||||
我们默认使用一个集群的环境设置(即一组机器)。DGL会将一张图划分为多张子图，
|
||||
集群中的每台机器各自负责一张子图(分区)。为了并行化计算，DGL在集群所有机器上运行相同的训练脚本，
|
||||
并在同样的机器上运行服务器以将分区数据提供给训练器。
|
||||
|
||||
对于训练脚本，DGL提供了分布式的API。它们与小批次训练的API相似。用户仅需对单机小批次训练的代码稍作修改就可实现分布式训练。
|
||||
以下代码给出了一个用分布式方式训练GraphSage的示例。仅有的代码修改出现在第4-7行：1)初始化DGL的分布式模块，2)创建分布式图对象，以及
|
||||
3)拆分训练集，并计算本地进程的节点。其余代码保持不变，与 :ref:`mini_cn-batch training <guide_cn-minibatch>` 类似，
|
||||
包括：创建采样器，模型定义，模型训练的循环。
|
||||
|
||||
.. code:: python
|
||||
|
||||
import dgl
|
||||
import torch as th
|
||||
|
||||
dgl.distributed.initialize('ip_config.txt')
|
||||
th.distributed.init_process_group(backend='gloo')
|
||||
g = dgl.distributed.DistGraph('graph_name', 'part_config.json')
|
||||
pb = g.get_partition_book()
|
||||
train_nid = dgl.distributed.node_split(g.ndata['train_mask'], pb, force_even=True)
|
||||
|
||||
# 创建采样器
|
||||
sampler = NeighborSampler(g, [10,25],
|
||||
dgl.distributed.sample_neighbors,
|
||||
device)
|
||||
|
||||
dataloader = DistDataLoader(
|
||||
dataset=train_nid.numpy(),
|
||||
batch_size=batch_size,
|
||||
collate_fn=sampler.sample_blocks,
|
||||
shuffle=True,
|
||||
drop_last=False)
|
||||
|
||||
# 定义模型和优化器
|
||||
model = SAGE(in_feats, num_hidden, n_classes, num_layers, F.relu, dropout)
|
||||
model = th.nn.parallel.DistributedDataParallel(model)
|
||||
loss_fcn = nn.CrossEntropyLoss()
|
||||
optimizer = optim.Adam(model.parameters(), lr=args.lr)
|
||||
|
||||
# 模型训练的循环
|
||||
for epoch in range(args.num_epochs):
|
||||
for step, blocks in enumerate(dataloader):
|
||||
batch_inputs, batch_labels = load_subtensor(g, blocks[0].srcdata[dgl.NID],
|
||||
blocks[-1].dstdata[dgl.NID])
|
||||
batch_pred = model(blocks, batch_inputs)
|
||||
loss = loss_fcn(batch_pred, batch_labels)
|
||||
optimizer.zero_grad()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
在一个集群的机器上运行训练脚本时，DGL提供了一些工具，可将数据复制到集群的计算机上，并在所有机器上启动训练任务。
|
||||
|
||||
**Note**: 当前版本的分布式训练API仅支持PyTorch后端。
|
||||
|
||||
**Note**: 当前版本的实现仅支持具有一种节点类型和一种边类型的图。
|
||||
|
||||
DGL实现了一些分布式组件以支持分布式训练，下图显示了这些组件及它们间的相互作用。
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/distributed.png
|
||||
:alt: Imgur
|
||||
|
||||
具体来说，DGL的分布式训练具有三种类型的交互进程：
|
||||
*服务器*，
|
||||
*采样器* 和 *训练器*。
|
||||
|
||||
* *服务器进程* 在存储图分区数据(这包括图结构和节点/边特征)的每台计算机上运行。
|
||||
这些服务器一起工作以将图数据提供给训练器。请注意，一台机器可能同时运行多个服务器进程，以并行化计算和网络通信。
|
||||
* *采样器进程* 与服务器进行交互，并对节点和边采样以生成用于训练的小批次数据。
|
||||
* *训练器进程* 包含多个与服务器交互的类。它用 :class:`~dgl.distributed.DistGraph` 来获取被划分的图分区数据，
|
||||
用 :class:`~dgl.distributed.DistEmbedding` 和
|
||||
:class:`~dgl.distributed.DistTensor` 来获取节点/边特征/嵌入，用
|
||||
:class:`~dgl.distributed.dist_dataloader.DistDataLoader` 与采样器进行交互以获得小批次数据。
|
||||
|
||||
在初步了解了分布式组件后，本章的剩余部分将介绍以下分布式组件：
|
||||
|
||||
* :ref:`guide_cn-distributed-preprocessing`
|
||||
* :ref:`guide_cn-distributed-apis`
|
||||
* :ref:`guide_cn-distributed-tools`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
distributed-preprocessing
|
||||
distributed-apis
|
||||
distributed-tools
|
||||
@@ -0,0 +1,23 @@
|
||||
.. _guide_cn-graph-basic:
|
||||
|
||||
1.1 关于图的基本概念
|
||||
-----------------
|
||||
|
||||
:ref:`(English Version) <guide-graph-basic>`
|
||||
|
||||
图是用以表示实体及其关系的结构，记为 :math:`G=(V, E)` 。图由两个集合组成，一是节点的集合 :math:`V` ，一个是边的集合 :math:`E` 。
|
||||
在边集 :math:`E` 中，一条边 :math:`(u, v)` 连接一对节点 :math:`u` 和 :math:`v` ，表明两节点间存在关系。关系可以是无向的，
|
||||
如描述节点之间的对称关系；也可以是有向的，如描述非对称关系。例如，若用图对社交网络中人们的友谊关系进行建模，因为友谊是相互的，则边是无向的；
|
||||
若用图对Twitter用户的关注行为进行建模，则边是有向的。图可以是 *有向的* 或 *无向的* ，这取决于图中边的方向性。
|
||||
|
||||
图可以是 *加权的* 或 *未加权的* 。在加权图中，每条边都与一个标量权重值相关联。例如，该权重可以表示长度或连接的强度。
|
||||
|
||||
图可以是 *同构的* 或是 *异构的* 。在同构图中，所有节点表示同一类型的实体，所有边表示同一类型的关系。
|
||||
例如，社交网络的图由表示同一实体类型的人及其相互之间的社交关系组成。
|
||||
|
||||
相对地，在异构图中，节点和边的类型可以是不同的。例如，编码市场的图可以有表示"顾客"、"商家"和"商品"的节点，
|
||||
它们通过“想购买”、“已经购买”、“是顾客”和“正在销售”的边互相连接。二分图是一类特殊的、常用的异构图，
|
||||
其中的边连接两类不同类型的节点。例如，在推荐系统中，可以使用二分图表示"用户"和"物品"之间的关系。想了解更多信息，读者可参考 :ref:`guide_cn-graph-heterogeneous`。
|
||||
|
||||
在多重图中，同一对节点之间可以有多条（有向）边，包括自循环的边。例如，两名作者可以在不同年份共同署名文章，
|
||||
这就带来了具有不同特征的多条边。
|
||||
@@ -0,0 +1,109 @@
|
||||
.. _guide_cn-graph-external:
|
||||
|
||||
1.4 从外部源创建图
|
||||
---------------
|
||||
|
||||
:ref:`(English Version)<guide-graph-external>`
|
||||
|
||||
可以从外部来源构造一个 :class:`~dgl.DGLGraph` 对象，包括：
|
||||
|
||||
- 从用于图和稀疏矩阵的外部Python库（NetworkX 和 SciPy）创建而来。
|
||||
- 从磁盘加载图数据。
|
||||
|
||||
本节不涉及通过转换其他图来生成图的函数，相关概述请阅读API参考手册。
|
||||
|
||||
从外部库创建图
|
||||
^^^^^^^^^^^
|
||||
|
||||
以下代码片段为从SciPy稀疏矩阵和NetworkX图创建DGL图的示例。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> import scipy.sparse as sp
|
||||
>>> spmat = sp.rand(100, 100, density=0.05) # 5%非零项
|
||||
>>> dgl.from_scipy(spmat) # 来自SciPy
|
||||
Graph(num_nodes=100, num_edges=500,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
>>> import networkx as nx
|
||||
>>> nx_g = nx.path_graph(5) # 一条链路0-1-2-3-4
|
||||
>>> dgl.from_networkx(nx_g) # 来自NetworkX
|
||||
Graph(num_nodes=5, num_edges=8,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
注意，当使用 `nx.path_graph(5)` 进行创建时， :class:`~dgl.DGLGraph` 对象有8条边，而非4条。
|
||||
这是由于 `nx.path_graph(5)` 构建了一个无向的NetworkX图 :class:`networkx.Graph` ，而 :class:`~dgl.DGLGraph` 的边总是有向的。
|
||||
所以当将无向的NetworkX图转换为 :class:`~dgl.DGLGraph` 对象时，DGL会在内部将1条无向边转换为2条有向边。
|
||||
使用有向的NetworkX图 :class:`networkx.DiGraph` 可避免该行为。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> nxg = nx.DiGraph([(2, 1), (1, 2), (2, 3), (0, 0)])
|
||||
>>> dgl.from_networkx(nxg)
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
.. note::
|
||||
|
||||
DGL在内部将SciPy矩阵和NetworkX图转换为张量来创建图。因此，这些构建方法并不适用于重视性能的场景。
|
||||
|
||||
相关API： :func:`dgl.from_scipy`、 :func:`dgl.from_networkx`。
|
||||
|
||||
从磁盘加载图
|
||||
^^^^^^^^^^
|
||||
|
||||
有多种文件格式可储存图，所以这里难以枚举所有选项。本节仅给出一些常见格式的一般情况。
|
||||
|
||||
逗号分隔值（CSV）
|
||||
""""""""""""""
|
||||
|
||||
CSV是一种常见的格式，以表格格式储存节点、边及其特征：
|
||||
|
||||
.. table:: nodes.csv
|
||||
|
||||
+-----------+
|
||||
|age, title |
|
||||
+===========+
|
||||
|43, 1 |
|
||||
+-----------+
|
||||
|23, 3 |
|
||||
+-----------+
|
||||
|... |
|
||||
+-----------+
|
||||
|
||||
.. table:: edges.csv
|
||||
|
||||
+-----------------+
|
||||
|src, dst, weight |
|
||||
+=================+
|
||||
|0, 1, 0.4 |
|
||||
+-----------------+
|
||||
|0, 3, 0.9 |
|
||||
+-----------------+
|
||||
|... |
|
||||
+-----------------+
|
||||
|
||||
许多知名Python库(如Pandas)可以将该类型数据加载到python对象(如 :class:`numpy.ndarray`)中，
|
||||
进而使用这些对象来构建DGLGraph对象。如果后端框架也提供了从磁盘中保存或加载张量的工具(如 :func:`torch.save`, :func:`torch.load` )，
|
||||
可以遵循相同的原理来构建图。
|
||||
|
||||
另见： `从成对的边 CSV 文件中加载 Karate Club Network 的教程 <https://github.com/dglai/WWW20-Hands-on-Tutorial/blob/master/basic_tasks/1_load_data.ipynb>`_。
|
||||
|
||||
JSON/GML 格式
|
||||
""""""""""""
|
||||
|
||||
如果对速度不太关注的话，读者可以使用NetworkX提供的工具来解析 `各种数据格式 <https://networkx.github.io/documentation/stable/reference/readwrite/index.html>`_，
|
||||
DGL可以间接地从这些来源创建图。
|
||||
|
||||
DGL 二进制格式
|
||||
""""""""""""
|
||||
|
||||
DGL提供了API以从磁盘中加载或向磁盘里保存二进制格式的图。除了图结构，API也能处理特征数据和图级别的标签数据。
|
||||
DGL也支持直接从S3/HDFS中加载或向S3/HDFS保存图。参考手册提供了该用法的更多细节。
|
||||
|
||||
相关API： :func:`dgl.save_graphs`、 :func:`dgl.load_graphs`。
|
||||
@@ -0,0 +1,60 @@
|
||||
.. _guide_cn-graph-feature:
|
||||
|
||||
1.3 节点和边的特征
|
||||
---------------
|
||||
|
||||
:ref:`(English Version)<guide-graph-feature>`
|
||||
|
||||
:class:`~dgl.DGLGraph` 对象的节点和边可具有多个用户定义的、可命名的特征，以储存图的节点和边的属性。
|
||||
通过 :py:attr:`~dgl.DGLGraph.ndata` 和 :py:attr:`~dgl.DGLGraph.edata` 接口可访问这些特征。
|
||||
例如，以下代码创建了2个节点特征（分别在第8、15行命名为 ``'x'`` 、 ``'y'`` ）和1个边特征（在第9行命名为 ``'x'`` ）。
|
||||
|
||||
.. code-block:: python
|
||||
:linenos:
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> g = dgl.graph(([0, 0, 1, 5], [1, 2, 2, 0])) # 6个节点，4条边
|
||||
>>> g
|
||||
Graph(num_nodes=6, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
>>> g.ndata['x'] = th.ones(g.num_nodes(), 3) # 长度为3的节点特征
|
||||
>>> g.edata['x'] = th.ones(g.num_edges(), dtype=th.int32) # 标量整型特征
|
||||
>>> g
|
||||
Graph(num_nodes=6, num_edges=4,
|
||||
ndata_schemes={'x' : Scheme(shape=(3,), dtype=torch.float32)}
|
||||
edata_schemes={'x' : Scheme(shape=(,), dtype=torch.int32)})
|
||||
>>> # 不同名称的特征可以具有不同形状
|
||||
>>> g.ndata['y'] = th.randn(g.num_nodes(), 5)
|
||||
>>> g.ndata['x'][1] # 获取节点1的特征
|
||||
tensor([1., 1., 1.])
|
||||
>>> g.edata['x'][th.tensor([0, 3])] # 获取边0和3的特征
|
||||
tensor([1, 1], dtype=torch.int32)
|
||||
|
||||
关于 :py:attr:`~dgl.DGLGraph.ndata` 和 :py:attr:`~dgl.DGLGraph.edata` 接口的重要说明：
|
||||
|
||||
- 仅允许使用数值类型（如单精度浮点型、双精度浮点型和整型）的特征。这些特征可以是标量、向量或多维张量。
|
||||
- 每个节点特征具有唯一名称，每个边特征也具有唯一名称。节点和边的特征可以具有相同的名称（如上述示例代码中的 ``'x'`` ）。
|
||||
- 通过张量分配创建特征时，DGL会将特征赋给图中的每个节点和每条边。该张量的第一维必须与图中节点或边的数量一致。
|
||||
不能将特征赋给图中节点或边的子集。
|
||||
- 相同名称的特征必须具有相同的维度和数据类型。
|
||||
- 特征张量使用"行优先"的原则，即每个行切片储存1个节点或1条边的特征（参考上述示例代码的第16和18行）。
|
||||
|
||||
对于加权图，用户可以将权重储存为一个边特征，如下。
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> # 边 0->1, 0->2, 0->3, 1->3
|
||||
>>> edges = th.tensor([0, 0, 0, 1]), th.tensor([1, 2, 3, 3])
|
||||
>>> weights = th.tensor([0.1, 0.6, 0.9, 0.7]) # 每条边的权重
|
||||
>>> g = dgl.graph(edges)
|
||||
>>> g.edata['w'] = weights # 将其命名为 'w'
|
||||
>>> g
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={'w' : Scheme(shape=(,), dtype=torch.float32)})
|
||||
|
||||
|
||||
|
||||
相关API： :py:attr:`~dgl.DGLGraph.ndata`、 :py:attr:`~dgl.DGLGraph.edata`。
|
||||
@@ -0,0 +1,45 @@
|
||||
.. _guide_cn-graph-gpu:
|
||||
|
||||
1.6 在GPU上使用DGLGraph
|
||||
----------------------
|
||||
|
||||
:ref:`(English Version)<guide-graph-gpu>`
|
||||
|
||||
用户可以通过在构造过程中传入两个GPU张量来创建GPU上的 :class:`~dgl.DGLGraph` 。
|
||||
另一种方法是使用 :func:`~dgl.DGLGraph.to` API将 :class:`~dgl.DGLGraph` 复制到GPU，这会将图结构和特征数据都拷贝到指定的设备。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
>>> u, v = th.tensor([0, 1, 2]), th.tensor([2, 3, 4])
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> g.ndata['x'] = th.randn(5, 3) # 原始特征在CPU上
|
||||
>>> g.device
|
||||
device(type='cpu')
|
||||
>>> cuda_g = g.to('cuda:0') # 接受来自后端框架的任何设备对象
|
||||
>>> cuda_g.device
|
||||
device(type='cuda', index=0)
|
||||
>>> cuda_g.ndata['x'].device # 特征数据也拷贝到了GPU上
|
||||
device(type='cuda', index=0)
|
||||
|
||||
>>> # 由GPU张量构造的图也在GPU上
|
||||
>>> u, v = u.to('cuda:0'), v.to('cuda:0')
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> g.device
|
||||
device(type='cuda', index=0)
|
||||
|
||||
任何涉及GPU图的操作都是在GPU上运行的。因此，这要求所有张量参数都已经放在GPU上，其结果(图或张量)也将在GPU上。
|
||||
此外，GPU图只接受GPU上的特征数据。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> cuda_g.in_degrees()
|
||||
tensor([0, 0, 1, 1, 1], device='cuda:0')
|
||||
>>> cuda_g.in_edges([2, 3, 4]) # 可以接受非张量类型的参数
|
||||
(tensor([0, 1, 2], device='cuda:0'), tensor([2, 3, 4], device='cuda:0'))
|
||||
>>> cuda_g.in_edges(th.tensor([2, 3, 4]).to('cuda:0')) # 张量类型的参数必须在GPU上
|
||||
(tensor([0, 1, 2], device='cuda:0'), tensor([2, 3, 4], device='cuda:0'))
|
||||
>>> cuda_g.ndata['h'] = th.randn(5, 4) # ERROR! 特征也必须在GPU上！
|
||||
DGLError: Cannot assign node feature "h" on device cpu to a graph on device
|
||||
cuda:0. Call DGLGraph.to() to copy the graph to the same device.
|
||||
@@ -0,0 +1,88 @@
|
||||
.. _guide_cn-graph-graphs-nodes-edges:
|
||||
|
||||
1.2 图、节点和边
|
||||
--------------
|
||||
|
||||
:ref:`(English Version)<guide-graph-graphs-nodes-edges>`
|
||||
|
||||
DGL使用一个唯一的整数来表示一个节点，称为点ID；并用对应的两个端点ID表示一条边。同时，DGL也会根据边被添加的顺序，
|
||||
给每条边分配一个唯一的整数编号，称为边ID。节点和边的ID都是从0开始构建的。在DGL的图里，所有的边都是有方向的，
|
||||
即边 :math:`(u, v)` 表示它是从节点 :math:`u` 指向节点 :math:`v` 的。
|
||||
|
||||
对于多个节点，DGL使用一个一维的整型张量（如，PyTorch的Tensor类，TensorFlow的Tensor类或MXNet的ndarray类）来保存图的点ID，
|
||||
DGL称之为"节点张量"。为了指代多条边，DGL使用一个包含2个节点张量的元组 :math:`(U, V)` ，其中，用 :math:`(U[i], V[i])` 指代一条
|
||||
:math:`U[i]` 到 :math:`V[i]` 的边。
|
||||
|
||||
创建一个 :class:`~dgl.DGLGraph` 对象的一种方法是使用 :func:`dgl.graph` 函数。它接受一个边的集合作为输入。DGL也支持从其他的数据源来创建图对象。
|
||||
读者可参考 :ref:`guide_cn-graph-external`。
|
||||
|
||||
下面的代码段使用了 :func:`dgl.graph` 函数来构建一个 :class:`~dgl.DGLGraph` 对象，对应着下图所示的包含4个节点的图。
|
||||
其中一些代码演示了查询图结构的部分API的使用方法。
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/user_guide_graphch_1.png
|
||||
:height: 200px
|
||||
:width: 300px
|
||||
:align: center
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
|
||||
>>> # 边 0->1, 0->2, 0->3, 1->3
|
||||
>>> u, v = th.tensor([0, 0, 0, 1]), th.tensor([1, 2, 3, 3])
|
||||
>>> g = dgl.graph((u, v))
|
||||
>>> print(g) # 图中节点的数量是DGL通过给定的图的边列表中最大的点ID推断所得出的
|
||||
Graph(num_nodes=4, num_edges=4,
|
||||
ndata_schemes={}
|
||||
edata_schemes={})
|
||||
|
||||
>>> # 获取节点的ID
|
||||
>>> print(g.nodes())
|
||||
tensor([0, 1, 2, 3])
|
||||
>>> # 获取边的对应端点
|
||||
>>> print(g.edges())
|
||||
(tensor([0, 0, 0, 1]), tensor([1, 2, 3, 3]))
|
||||
>>> # 获取边的对应端点和边ID
|
||||
>>> print(g.edges(form='all'))
|
||||
(tensor([0, 0, 0, 1]), tensor([1, 2, 3, 3]), tensor([0, 1, 2, 3]))
|
||||
|
||||
>>> # 如果具有最大ID的节点没有边，在创建图的时候，用户需要明确地指明节点的数量。
|
||||
>>> g = dgl.graph((u, v), num_nodes=8)
|
||||
|
||||
对于无向的图，用户需要为每条边都创建两个方向的边。可以使用 :func:`dgl.to_bidirected` 函数来实现这个目的。
|
||||
如下面的代码段所示，这个函数可以把原图转换成一个包含反向边的图。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> bg = dgl.to_bidirected(g)
|
||||
>>> bg.edges()
|
||||
(tensor([0, 0, 0, 1, 1, 2, 3, 3]), tensor([1, 2, 3, 0, 3, 0, 0, 1]))
|
||||
|
||||
.. note::
|
||||
|
||||
由于Tensor类内部使用C来存储，且显性定义了数据类型以及存储的设备信息，DGL推荐使用Tensor作为DGL API的输入。
|
||||
不过大部分的DGL API也支持Python的可迭代类型(比如列表)或numpy.ndarray类型作为API的输入，方便用户快速进行开发验证。
|
||||
|
||||
DGL支持使用 :math:`32` 位或 :math:`64` 位的整数作为节点ID和边ID。节点和边ID的数据类型必须一致。如果使用 :math:`64` 位整数，
|
||||
DGL可以处理最多 :math:`2^{63} - 1` 个节点或边。不过，如果图里的节点或者边的数量小于 :math:`2^{31} - 1` ，用户最好使用 :math:`32` 位整数。
|
||||
这样不仅能提升速度，还能减少内存的使用。DGL提供了进行数据类型转换的方法，如下例所示。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> edges = th.tensor([2, 5, 3]), th.tensor([3, 5, 0]) # 边：2->3, 5->5, 3->0
|
||||
>>> g64 = dgl.graph(edges) # DGL默认使用int64
|
||||
>>> print(g64.idtype)
|
||||
torch.int64
|
||||
>>> g32 = dgl.graph(edges, idtype=th.int32) # 使用int32构建图
|
||||
>>> g32.idtype
|
||||
torch.int32
|
||||
>>> g64_2 = g32.long() # 转换成int64
|
||||
>>> g64_2.idtype
|
||||
torch.int64
|
||||
>>> g32_2 = g64.int() # 转换成int32
|
||||
>>> g32_2.idtype
|
||||
torch.int32
|
||||
|
||||
相关API：:func:`dgl.graph`、 :func:`dgl.DGLGraph.nodes`、 :func:`dgl.DGLGraph.edges`、 :func:`dgl.to_bidirected`、
|
||||
:func:`dgl.DGLGraph.int`、 :func:`dgl.DGLGraph.long` 和 :py:attr:`dgl.DGLGraph.idtype`。
|
||||
@@ -0,0 +1,264 @@
|
||||
.. _guide_cn-graph-heterogeneous:
|
||||
|
||||
1.5 异构图
|
||||
---------
|
||||
|
||||
:ref:`(English Version)<guide-graph-heterogeneous>`
|
||||
|
||||
相比同构图，异构图里可以有不同类型的节点和边。这些不同类型的节点和边具有独立的ID空间和特征。
|
||||
例如在下图中，"用户"和"游戏"节点的ID都是从0开始的，而且两种节点具有不同的特征。
|
||||
|
||||
.. figure:: https://data.dgl.ai/asset/image/user_guide_graphch_2.png
|
||||
|
||||
一个异构图示例。该图具有两种类型的节点("用户"和"游戏")和两种类型的边("关注"和"玩")。
|
||||
|
||||
创建异构图
|
||||
^^^^^^^^
|
||||
|
||||
在DGL中，一个异构图由一系列子图构成，一个子图对应一种关系。每个关系由一个字符串三元组
|
||||
定义 ``(源节点类型, 边类型, 目标节点类型)`` 。由于这里的关系定义消除了边类型的歧义，DGL称它们为规范边类型。
|
||||
|
||||
下面的代码是一个在DGL中创建异构图的示例。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> import dgl
|
||||
>>> import torch as th
|
||||
|
||||
>>> # 创建一个具有3种节点类型和3种边类型的异构图
|
||||
>>> graph_data = {
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... }
|
||||
>>> g = dgl.heterograph(graph_data)
|
||||
>>> g.ntypes
|
||||
['disease', 'drug', 'gene']
|
||||
>>> g.etypes
|
||||
['interacts', 'interacts', 'treats']
|
||||
>>> g.canonical_etypes
|
||||
[('drug', 'interacts', 'drug'),
|
||||
('drug', 'interacts', 'gene'),
|
||||
('drug', 'treats', 'disease')]
|
||||
|
||||
注意，同构图和二分图只是一种特殊的异构图，它们只包括一种关系。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # 一个同构图
|
||||
>>> dgl.heterograph({('node_type', 'edge_type', 'node_type'): (u, v)})
|
||||
>>> # 一个二分图
|
||||
>>> dgl.heterograph({('source_type', 'edge_type', 'destination_type'): (u, v)})
|
||||
|
||||
与异构图相关联的 *metagraph* 就是图的模式。它指定节点集和节点之间的边的类型约束。
|
||||
*metagraph* 中的一个节点 :math:`u` 对应于相关异构图中的一个节点类型。
|
||||
*metagraph* 中的边 :math:`(u,v)` 表示在相关异构图中存在从 :math:`u` 型节点到 :math:`v` 型节点的边。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g
|
||||
Graph(num_nodes={'disease': 3, 'drug': 3, 'gene': 4},
|
||||
num_edges={('drug', 'interacts', 'drug'): 2,
|
||||
('drug', 'interacts', 'gene'): 2,
|
||||
('drug', 'treats', 'disease'): 1},
|
||||
metagraph=[('drug', 'drug', 'interacts'),
|
||||
('drug', 'gene', 'interacts'),
|
||||
('drug', 'disease', 'treats')])
|
||||
>>> g.metagraph().edges()
|
||||
OutMultiEdgeDataView([('drug', 'drug'), ('drug', 'gene'), ('drug', 'disease')])
|
||||
|
||||
相关API: :func:`dgl.heterograph`、 :py:attr:`~dgl.DGLGraph.ntypes`、 :py:attr:`~dgl.DGLGraph.etypes`、
|
||||
:py:attr:`~dgl.DGLGraph.canonical_etypes`、 :py:attr:`~dgl.DGLGraph.metagraph`。
|
||||
|
||||
使用多种类型
|
||||
^^^^^^^^^^
|
||||
|
||||
当引入多种节点和边类型后，用户在调用DGLGraph API以获取特定类型的信息时，需要指定具体的节点和边类型。此外，不同类型的节点和边具有单独的ID。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # 获取图中所有节点的数量
|
||||
>>> g.num_nodes()
|
||||
10
|
||||
>>> # 获取drug节点的数量
|
||||
>>> g.num_nodes('drug')
|
||||
3
|
||||
>>> # 不同类型的节点有单独的ID。因此，没有指定节点类型就没有明确的返回值。
|
||||
>>> g.nodes()
|
||||
DGLError: Node type name must be specified if there are more than one node types.
|
||||
>>> g.nodes('drug')
|
||||
tensor([0, 1, 2])
|
||||
|
||||
为了设置/获取特定节点和边类型的特征，DGL提供了两种新类型的语法： `g.nodes['node_type'].data['feat_name']` 和 `g.edges['edge_type'].data['feat_name']` 。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # 设置/获取"drug"类型的节点的"hv"特征
|
||||
>>> g.nodes['drug'].data['hv'] = th.ones(3, 1)
|
||||
>>> g.nodes['drug'].data['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.]])
|
||||
>>> # 设置/获取"treats"类型的边的"he"特征
|
||||
>>> g.edges['treats'].data['he'] = th.zeros(1, 1)
|
||||
>>> g.edges['treats'].data['he']
|
||||
tensor([[0.]])
|
||||
|
||||
如果图里只有一种节点或边类型，则不需要指定节点或边的类型。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'is similar', 'drug'): (th.tensor([0, 1]), th.tensor([2, 3]))
|
||||
... })
|
||||
>>> g.nodes()
|
||||
tensor([0, 1, 2, 3])
|
||||
>>> # 设置/获取单一类型的节点或边特征，不必使用新的语法
|
||||
>>> g.ndata['hv'] = th.ones(4, 1)
|
||||
|
||||
.. note::
|
||||
|
||||
当边类型唯一地确定了源节点和目标节点的类型时，用户可以只使用一个字符串而不是字符串三元组来指定边类型。例如，
|
||||
对于具有两个关系 ``('user', 'plays', 'game')`` 和 ``('user', 'likes', 'game')`` 的异构图，
|
||||
只使用 ``'plays'`` 或 ``'like'`` 来指代这两个关系是可以的。
|
||||
|
||||
从磁盘加载异构图
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
逗号分隔值（CSV）
|
||||
""""""""""""""
|
||||
|
||||
一种存储异构图的常见方法是在不同的CSV文件中存储不同类型的节点和边。下面是一个例子。
|
||||
|
||||
.. code::
|
||||
|
||||
# 数据文件夹
|
||||
data/
|
||||
|-- drug.csv # drug节点
|
||||
|-- gene.csv # gene节点
|
||||
|-- disease.csv # disease节点
|
||||
|-- drug-interact-drug.csv # drug-drug相互作用边
|
||||
|-- drug-interact-gene.csv # drug-gene相互作用边
|
||||
|-- drug-treat-disease.csv # drug-disease治疗边
|
||||
|
||||
与同构图的情况类似，用户可以使用像Pandas这样的包先将CSV文件解析为numpy数组或框架张量，再构建一个关系字典，并用它构造一个异构图。
|
||||
这种方法也适用于其他流行的文件格式，比如GML或JSON。
|
||||
|
||||
DGL二进制格式
|
||||
"""""""""""
|
||||
|
||||
DGL提供了 :func:`dgl.save_graphs` 和 :func:`dgl.load_graphs` 函数，分别用于以二进制格式保存异构图和加载它们。
|
||||
|
||||
边类型子图
|
||||
^^^^^^^^
|
||||
|
||||
用户可以通过指定要保留的关系来创建异构图的子图，相关的特征也会被拷贝。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... })
|
||||
>>> g.nodes['drug'].data['hv'] = th.ones(3, 1)
|
||||
|
||||
>>> # 保留关系 ('drug', 'interacts', 'drug') 和 ('drug', 'treats', 'disease') 。
|
||||
>>> # 'drug' 和 'disease' 类型的节点也会被保留
|
||||
>>> eg = dgl.edge_type_subgraph(g, [('drug', 'interacts', 'drug'),
|
||||
... ('drug', 'treats', 'disease')])
|
||||
>>> eg
|
||||
Graph(num_nodes={'disease': 3, 'drug': 3},
|
||||
num_edges={('drug', 'interacts', 'drug'): 2, ('drug', 'treats', 'disease'): 1},
|
||||
metagraph=[('drug', 'drug', 'interacts'), ('drug', 'disease', 'treats')])
|
||||
>>> # 相关的特征也会被拷贝
|
||||
>>> eg.nodes['drug'].data['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.]])
|
||||
|
||||
|
||||
将异构图转化为同构图
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
异构图为管理不同类型的节点和边及其相关特征提供了一个清晰的接口。这在以下情况下尤其有用:
|
||||
|
||||
1. 不同类型的节点和边的特征具有不同的数据类型或大小。
|
||||
2. 用户希望对不同类型的节点和边应用不同的操作。
|
||||
|
||||
如果上述情况不适用，并且用户不希望在建模中区分节点和边的类型，则DGL允许使用 :func:`dgl.DGLGraph.to_homogeneous` API将异构图转换为同构图。
|
||||
具体行为如下:
|
||||
|
||||
1. 用从0开始的连续整数重新标记所有类型的节点和边。
|
||||
2. 对所有的节点和边合并用户指定的特征。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))})
|
||||
>>> g.nodes['drug'].data['hv'] = th.zeros(3, 1)
|
||||
>>> g.nodes['disease'].data['hv'] = th.ones(3, 1)
|
||||
>>> g.edges['interacts'].data['he'] = th.zeros(2, 1)
|
||||
>>> g.edges['treats'].data['he'] = th.zeros(1, 2)
|
||||
|
||||
>>> # 默认情况下不进行特征合并
|
||||
>>> hg = dgl.to_homogeneous(g)
|
||||
>>> 'hv' in hg.ndata
|
||||
False
|
||||
|
||||
>>> # 拷贝边的特征
|
||||
>>> # 对于要拷贝的特征，DGL假定不同类型的节点或边的需要合并的特征具有相同的大小和数据类型
|
||||
>>> hg = dgl.to_homogeneous(g, edata=['he'])
|
||||
DGLError: Cannot concatenate column ‘he’ with shape Scheme(shape=(2,), dtype=torch.float32) and shape Scheme(shape=(1,), dtype=torch.float32)
|
||||
|
||||
>>> # 拷贝节点特征
|
||||
>>> hg = dgl.to_homogeneous(g, ndata=['hv'])
|
||||
>>> hg.ndata['hv']
|
||||
tensor([[1.],
|
||||
[1.],
|
||||
[1.],
|
||||
[0.],
|
||||
[0.],
|
||||
[0.]])
|
||||
|
||||
原始的节点或边的类型和对应的ID被存储在 :py:attr:`~dgl.DGLGraph.ndata` 和 :py:attr:`~dgl.DGLGraph.edata` 中。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> # 异构图中节点类型的顺序
|
||||
>>> g.ntypes
|
||||
['disease', 'drug']
|
||||
>>> # 原始节点类型
|
||||
>>> hg.ndata[dgl.NTYPE]
|
||||
tensor([0, 0, 0, 1, 1, 1])
|
||||
>>> # 原始的特定类型节点ID
|
||||
>>> hg.ndata[dgl.NID]
|
||||
tensor([0, 1, 2, 0, 1, 2])
|
||||
|
||||
>>> # 异构图中边类型的顺序
|
||||
>>> g.etypes
|
||||
['interacts', 'treats']
|
||||
>>> # 原始边类型
|
||||
>>> hg.edata[dgl.ETYPE]
|
||||
tensor([0, 0, 1])
|
||||
>>> # 原始的特定类型边ID
|
||||
>>> hg.edata[dgl.EID]
|
||||
tensor([0, 1, 0])
|
||||
|
||||
出于建模的目的，用户可能需要将一些关系合并，并对它们应用相同的操作。为了实现这一目的，可以先抽取异构图的边类型子图，然后将该子图转换为同构图。
|
||||
|
||||
.. code::
|
||||
|
||||
>>> g = dgl.heterograph({
|
||||
... ('drug', 'interacts', 'drug'): (th.tensor([0, 1]), th.tensor([1, 2])),
|
||||
... ('drug', 'interacts', 'gene'): (th.tensor([0, 1]), th.tensor([2, 3])),
|
||||
... ('drug', 'treats', 'disease'): (th.tensor([1]), th.tensor([2]))
|
||||
... })
|
||||
>>> sub_g = dgl.edge_type_subgraph(g, [('drug', 'interacts', 'drug'),
|
||||
... ('drug', 'interacts', 'gene')])
|
||||
>>> h_sub_g = dgl.to_homogeneous(sub_g)
|
||||
>>> h_sub_g
|
||||
Graph(num_nodes=7, num_edges=4,
|
||||
...)
|
||||
@@ -0,0 +1,35 @@
|
||||
.. _guide_cn-graph:
|
||||
|
||||
第1章：图
|
||||
=============
|
||||
|
||||
:ref:`(English Version)<guide-graph>`
|
||||
|
||||
图表示实体(节点)和它们的关系(边)，其中节点和边可以是有类型的 (例如，``"用户"`` 和 ``"物品"`` 是两种不同类型的节点)。
|
||||
DGL通过其核心数据结构 :class:`~dgl.DGLGraph` 提供了一个以图为中心的编程抽象。 :class:`~dgl.DGLGraph` 提供了接口以处理图的结构、节点/边
|
||||
的特征，以及使用这些组件可以执行的计算。
|
||||
|
||||
|
||||
本章路线图
|
||||
--------------
|
||||
|
||||
本章首先简要介绍了图的定义（见1.1节），然后介绍了一些 :class:`~dgl.DGLGraph` 相关的核心概念：
|
||||
|
||||
* :ref:`guide_cn-graph-basic`
|
||||
* :ref:`guide_cn-graph-graphs-nodes-edges`
|
||||
* :ref:`guide_cn-graph-feature`
|
||||
* :ref:`guide_cn-graph-external`
|
||||
* :ref:`guide_cn-graph-heterogeneous`
|
||||
* :ref:`guide_cn-graph-gpu`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
:glob:
|
||||
|
||||
graph-basic
|
||||
graph-graphs-nodes-edges
|
||||
graph-feature
|
||||
graph-external
|
||||
graph-heterogeneous
|
||||
graph-gpu
|
||||
@@ -0,0 +1,140 @@
|
||||
用户指南【包含过时信息】
|
||||
===================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:titlesonly:
|
||||
|
||||
graph
|
||||
message
|
||||
nn
|
||||
data
|
||||
training
|
||||
minibatch
|
||||
distributed
|
||||
|
||||
2020年9月，DGL社区的一群热心贡献者把DGL用户指南译成了中文，方便广大中文用户群学习和使用DGL。
|
||||
|
||||
特此致谢下述贡献者：
|
||||
|
||||
.. list-table::
|
||||
:widths: 20 20 20
|
||||
:header-rows: 1
|
||||
|
||||
* - 章节
|
||||
- 个人姓名/昵称
|
||||
- 个人链接
|
||||
* - :ref:`guide_cn-graph`
|
||||
- 张怀文/Huaiwen Zhang
|
||||
- https://github.com/huaiwen
|
||||
* - :ref:`guide_cn-graph-basic`
|
||||
- 沈成 / mlsoar
|
||||
- https://github.com/mlsoar
|
||||
* - :ref:`guide_cn-graph-graphs-nodes-edges`
|
||||
- 张建 / zhjwy9343
|
||||
- https://github.com/zhjwy9343
|
||||
* - :ref:`guide_cn-graph-feature`
|
||||
- 沈成 / mlsoar
|
||||
- https://github.com/mlsoar
|
||||
* - :ref:`guide_cn-graph-external`
|
||||
- 沈成 / mlsoar
|
||||
- https://github.com/mlsoar
|
||||
* - :ref:`guide_cn-graph-heterogeneous`
|
||||
- 张怀文/Huaiwen Zhang
|
||||
- https://github.com/huaiwen
|
||||
* - :ref:`guide_cn-message-passing`,
|
||||
- 黄崟/Brook Huang
|
||||
- https://github.com/brookhuang16211
|
||||
* - :ref:`guide_cn-message-passing-api`
|
||||
- 黄崟/Brook Huang
|
||||
- https://github.com/brookhuang16211
|
||||
* - :ref:`guide_cn-message-passing-efficient`
|
||||
- 黄崟/Brook Huang
|
||||
- https://github.com/brookhuang16211
|
||||
* - :ref:`guide_cn-message-passing-part`
|
||||
- 陈知雨/Zhiyu Chen
|
||||
- https://www.zhiyuchen.com
|
||||
* - :ref:`guide_cn-message-passing-edge`
|
||||
- 陈知雨/Zhiyu Chen
|
||||
- https://www.zhiyuchen.com
|
||||
* - :ref:`guide_cn-message-passing-heterograph`
|
||||
- 陈知雨/Zhiyu Chen
|
||||
- https://www.zhiyuchen.com
|
||||
* - :ref:`guide_cn-nn`
|
||||
- 陈知雨/Zhiyu Chen
|
||||
- https://www.zhiyuchen.com
|
||||
* - :ref:`guide_cn-nn-construction`
|
||||
- 陈知雨/Zhiyu Chen
|
||||
- https://www.zhiyuchen.com
|
||||
* - :ref:`guide_cn-nn-forward`
|
||||
- 栩栩的夏天
|
||||
-
|
||||
* - :ref:`guide_cn-nn-heterograph`
|
||||
- 栩栩的夏天
|
||||
-
|
||||
* - :ref:`guide_cn-data-pipeline`
|
||||
- 吴紫薇/ Maggie Wu
|
||||
- https://github.com/hhhiddleston
|
||||
* - :ref:`guide_cn-data-pipeline-dataset`
|
||||
- 吴紫薇/ Maggie Wu
|
||||
- https://github.com/hhhiddleston
|
||||
* - :ref:`guide_cn-data-pipeline-download`
|
||||
- 吴紫薇/ Maggie Wu
|
||||
- https://github.com/hhhiddleston
|
||||
* - :ref:`guide_cn-data-pipeline-process`
|
||||
- 吴紫薇/ Maggie Wu
|
||||
- https://github.com/hhhiddleston
|
||||
* - :ref:`guide_cn-data-pipeline-savenload`
|
||||
- 王建民/DrugAI
|
||||
- https://github.com/AspirinCode
|
||||
* - :ref:`guide_cn-data-pipeline-loadogb`
|
||||
- 王建民/DrugAI
|
||||
- https://github.com/AspirinCode
|
||||
* - :ref:`guide_cn-training`
|
||||
- 王建民/DrugAI
|
||||
- https://github.com/AspirinCode
|
||||
* - :ref:`guide_cn-training-node-classification`,
|
||||
- 王建民/DrugAI
|
||||
- https://github.com/AspirinCode
|
||||
* - :ref:`guide_cn-training-edge-classification`
|
||||
- 徐东辉/DonghuiXu
|
||||
- https://github.com/rewonderful
|
||||
* - :ref:`guide_cn-training-link-prediction`
|
||||
- 徐东辉/DonghuiXu
|
||||
- https://github.com/rewonderful
|
||||
* - :ref:`guide_cn-training-graph-classification`
|
||||
- 莫佳帅子/Molasses
|
||||
- https://github.com/sleeplessai
|
||||
* - :ref:`guide_cn-minibatch`
|
||||
- 莫佳帅子/Molasses
|
||||
- https://github.com/sleeplessai
|
||||
* - :ref:`guide_cn-minibatch-node-classification-sampler`
|
||||
- 孟凡荣/kevin-meng
|
||||
- https://github.com/kevin-meng
|
||||
* - :ref:`guide_cn-minibatch-edge-classification-sampler`
|
||||
- 莫佳帅子/Molasses
|
||||
- https://github.com/sleeplessai
|
||||
* - :ref:`guide_cn-minibatch-link-classification-sampler`
|
||||
- 孟凡荣/kevin-meng
|
||||
- https://github.com/kevin-meng
|
||||
* - :ref:`guide_cn-minibatch-customizing-neighborhood-sampler`
|
||||
- 孟凡荣/kevin-meng
|
||||
- https://github.com/kevin-meng
|
||||
* - :ref:`guide_cn-minibatch-custom-gnn-module`
|
||||
- 胡骏
|
||||
- https://github.com/CrawlScript
|
||||
* - :ref:`guide_cn-minibatch-inference`
|
||||
- 胡骏
|
||||
- https://github.com/CrawlScript
|
||||
* - :ref:`guide_cn-distributed`
|
||||
- 宋怡然/Yiran Song
|
||||
- https://github.com/rr-Yiran
|
||||
* - :ref:`guide_cn-distributed-preprocessing`
|
||||
- 宋怡然/Yiran Song
|
||||
- https://github.com/rr-Yiran
|
||||
* - :ref:`guide_cn-distributed-apis`
|
||||
- 李庆标/Qingbiao Li
|
||||
- https://qingbiaoli.github.io/
|
||||
* - :ref:`guide_cn-distributed-tools`
|
||||
- 李庆标/Qingbiao Li
|
||||
- https://qingbiaoli.github.io/
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user