wehub-resource-sync
e06fe8e8c6
Secret Leaks / trufflehog (push) Failing after 1s
Build documentation / build (push) Failing after 1s
Build documentation / build_other_lang (push) Failing after 0s
CodeQL Security Analysis / CodeQL Analysis (push) Failing after 0s
PR CI / pr-ci (push) Failing after 1s
Slow tests on important models (on Push - A10) / Get all modified files (push) Failing after 1s
Slow tests on important models (on Push - A10) / Model CI (push) Has been skipped
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Has been cancelled
New model PR merged notification / Notify new model (push) Has been cancelled
Update Transformers metadata / build_and_package (push) Has been cancelled
18 lines
527 B
Python
18 lines
527 B
Python
import torch.nn as nn
|
|
|
|
from transformers.models.bert.modeling_bert import BertEmbeddings, BertModel
|
|
|
|
|
|
class RobertaEmbeddings(BertEmbeddings):
|
|
def __init__(self, config):
|
|
super().__init__(config)
|
|
self.pad_token_id = config.pad_token_id
|
|
self.position_embeddings = nn.Embedding(
|
|
config.max_position_embeddings, config.hidden_size, config.pad_token_id
|
|
)
|
|
|
|
|
|
class RobertaModel(BertModel):
|
|
def __init__(self, config, add_pooling_layer=True):
|
|
super().__init__(self, config)
|