chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
.. role:: hidden
|
||||
:class: hidden-section
|
||||
|
||||
.. module:: fairseq.tasks
|
||||
|
||||
.. _Tasks:
|
||||
|
||||
Tasks
|
||||
=====
|
||||
|
||||
Tasks store dictionaries and provide helpers for loading/iterating over
|
||||
Datasets, initializing the Model/Criterion and calculating the loss.
|
||||
|
||||
Tasks can be selected via the ``--task`` command-line argument. Once selected, a
|
||||
task may expose additional command-line arguments for further configuration.
|
||||
|
||||
Example usage::
|
||||
|
||||
# setup the task (e.g., load dictionaries)
|
||||
task = fairseq.tasks.setup_task(args)
|
||||
|
||||
# build model and criterion
|
||||
model = task.build_model(args)
|
||||
criterion = task.build_criterion(args)
|
||||
|
||||
# load datasets
|
||||
task.load_dataset('train')
|
||||
task.load_dataset('valid')
|
||||
|
||||
# iterate over mini-batches of data
|
||||
batch_itr = task.get_batch_iterator(
|
||||
task.dataset('train'), max_tokens=4096,
|
||||
)
|
||||
for batch in batch_itr:
|
||||
# compute the loss
|
||||
loss, sample_size, logging_output = task.get_loss(
|
||||
model, criterion, batch,
|
||||
)
|
||||
loss.backward()
|
||||
|
||||
|
||||
Translation
|
||||
-----------
|
||||
|
||||
.. autoclass:: fairseq.tasks.translation.TranslationTask
|
||||
|
||||
.. _language modeling:
|
||||
|
||||
Language Modeling
|
||||
-----------------
|
||||
|
||||
.. autoclass:: fairseq.tasks.language_modeling.LanguageModelingTask
|
||||
|
||||
|
||||
Adding new tasks
|
||||
----------------
|
||||
|
||||
.. autofunction:: fairseq.tasks.register_task
|
||||
.. autoclass:: fairseq.tasks.FairseqTask
|
||||
:members:
|
||||
:undoc-members:
|
||||
Reference in New Issue
Block a user