wehub/tensorflow--tensorflow
1
0
Fork 0
Code Issues Pull Requests Actions 3 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
cffconvert / validate (push) Has been skipped
Details
License Check / license-check (push) Failing after 2s
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:14:16 +08:00
commit 8a852e4b4e
36502 changed files with 9277225 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tensorflow/c/experimental/ops/gen/BUILD
+36
View File
@@ -0,0 +1,36 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load(
"//tensorflow:tensorflow.bzl",
"tf_cc_binary",
)
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
cc_library(
name = "generate_cpp_main",
srcs = ["generate_cpp_main.cc"],
deps = [
"@com_google_absl//absl/log",
"@com_google_absl//absl/log:check",
"@com_google_absl//absl/strings",
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/c/experimental/ops/gen/cpp/renderers",
"//tensorflow/core:framework",
"//tensorflow/core:framework_internal",
"//tensorflow/c/experimental/ops/gen/cpp",
"//tensorflow/c/experimental/ops/gen/model",
"//tensorflow/core:lib",
# Without this line to link in ops, the global registry will be empty:
"//tensorflow/core:ops",
],
)
tf_cc_binary(
name = "generate_cpp",
args = ["--stderrthreshold=1"], # Warnings and errors to stderr.
deps = [":generate_cpp_main"],
)
tensorflow/c/experimental/ops/gen/README.md
+116
View File
@@ -0,0 +1,116 @@
# TensorFlow Op CodeGen Machinery (Experimental)
## Usage
```
usage: generate_cpp [flags] OpName1 [OpName2 ...]
Flags:
--help=false bool Print this help message.
--category="" string Category for generated ops (e.g. 'math', 'array').
--namespace="" string Compact C++ namespace, default is 'tensorflow::ops'.
--output_dir="" string Directory into which output files will be generated.
--source_dir="" string The tensorflow root directory, e.g. 'tensorflow/' for in-source include paths. Any path underneath the tensorflow root is also accepted.
--api_dirs="" string Comma-separated list of directories containing API definitions.
```
## Design
### Generator Framework
The generator framework is a loose Model/View/Controller arrangement:
The *Model* classes live in the ***model/*** directory. They are representations
of the `OpDef` and `ApiDef` protos, normalized and resolved.
> _For example, an `OpDef` proto's `ArgDef` members contain a type string, which
> must be dereferenced to an `AttrDef` by name to determine its type. This
> `AttrDef` proto message in turn contains a type string which may need to be
> parsed as "list(type)". Other `AttrDef` messages are not types, but instead
> argument-like modifiers. In contrast, the generator model `ArgSpec` contains a
> resolved `ArgType` which provides a boolean `is_list()` method directly, and
> the model `OpSpec` provides a list of only the argument-like attributes. In
> addition to convenience, this should aid consistency between generated code in
> each target language._
The *Controller* is in the ***common/*** directory. It is the workhorse used by
the language generators; it digests the Op registry and API definitions to build
the model and provides utilities for the language generators.
The *View* and rendering classes map the language-independent Model classes
(`OpSpec`, `ArgSpec`, `AttrSpec`, etc.) to language-specific `SourceCode`. The
framework does not impose any design on the language-specific generators, but
provides some utilities, and the C++ generator is a complete example.
### C++ Generator
The `CppGenerator` class is the interface to the `cpp/` language directory.
Given a config, it can generate source code for a .cc or .h file as a string or
write it to a target file.
The `CppFileRenderer` is the main renderer used by the generator; it renders an
entire file. The `CppConfig` defines if it is operating in header or source
mode.
"Views" are stateless and intended to be low-level building blocks: a direct
language-specific representation of the model classes. For example, an `ArgView`
is initialized from an `ArgSpec` (which was created initially from an `ArgDef`
proto message). Where they may have some similar methods between the model and
view, the view methods are language-specific.
For instance, the C++ generator's `ArgView::VariableName()` method is an
language-formatted name usable as a variable representing the model `ArgSpec`
object. In contrast, the `ArgSpec::name()` method in the model refers to the
canonical name of the object in the proto.
Where views are a representation of the *input* model, in the C++ generator,
"renderers" then use these views to build the *output* `SourceCode`; Renderers
understand the language at the statement/directive level and target a functional
section of the output, such as a block comment or an entire method or file.
Other differences between views and renderers:
* Renderers are stateful, modifying a referenced SourceCode. Views are
stateless and their public methods are all const, returning strings.
* Renderers are context-dependent, e.g. a method signature will include
default values when in "declaration" mode but not "definition" mode. A view
of some argument object simply knows its default value and does not care the
context.
* In terms of dependencies, `Renderers` use `Views` and other `Renderers`.
However, `Renderers` do **not** reference the model directly (e.g.
`OpSpec`). This is because if a renderer needs to reference part of the
model, it should get a language specific representation.
### Extending to Additional Languages
The design for the C++ generator should apply to other languages, and the
underlying generator framework (the model and controller) try to be agnostic. In
fact, some elements of the C++ design could be formalized (such as the
rendering/view framework) or re-used (e.g. `cpp:Renderer` could likely be shared
with C and Java as a common C-style language renderer base class).
Abstracted and condensed from the C++ generator, the overall control flow could
be described as follows:
From main() in *generate_lang_main.cc*:
* Call `tensorflow::port::InitMain` and parse any flags
* Initialize config objects (e.g. `PathConfig`, `LangConfig` from flags)
* Initialize a new `LangGenerator` from these config objects
* Call this generator to create/write `SourceCode` to a file
In class `LangGenerator` in *lang_generator.cc*:
* Initialize a new `Controller` from the config objects
* Call this controller to build the Op models (`OpSpec`)
* Initialize a new language-specific `View` for each model object
* Create a blank `SourceCode` rendering target (for each output file)
* Initialize a new `LangFileRenderer` from this target source code, the model
`View` objects, and config objects
* Call this renderer to generate the target `SourceCode`
The dependencies are as follows:
* `lang::Generator` depends on `Controller`, `Model`, `lang::Renderers`,
`lang::Views`
* `lang::Renderer` depends on `lang::View` (and `lang::Renderer` peers)
* `lang::View` depends on the model (e.g. `OpSpec`) (and `lang::View` peers)
tensorflow/c/experimental/ops/gen/common/BUILD
+49
View File
@@ -0,0 +1,49 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load(
"//tensorflow:tensorflow.bzl",
"tf_cc_tests",
)
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
cc_library(
name = "common",
srcs = glob(
["*.cc"],
exclude = ["*_test.cc"],
),
hdrs = glob(["*.h"]),
deps = [
"//tensorflow/c/experimental/ops/gen/model",
"//tensorflow/core:framework",
"//tensorflow/core:framework_internal",
"//tensorflow/core:lib",
"//tensorflow/core:lib_internal",
"//tensorflow/core:op_gen_lib",
"//tensorflow/core:protos_all_cc",
"//tensorflow/core/platform:str_util",
"@com_google_absl//absl/log",
"@com_google_absl//absl/log:check",
"@com_google_absl//absl/strings",
],
alwayslink = 1,
)
tf_cc_tests(
name = "all_tests",
size = "small",
srcs = glob(["*_test.cc"]),
deps = [
":common",
"//tensorflow/core:core_cpu",
"//tensorflow/core:framework",
"//tensorflow/core:test",
"//tensorflow/core:test_main",
"//tensorflow/core:testlib",
"//tensorflow/core/platform:types",
],
)
tensorflow/c/experimental/ops/gen/common/case_format.cc
+107
View File
@@ -0,0 +1,107 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/case_format.h"
#include <string>
#include "absl/strings/ascii.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace {
enum CaseFormatType {
LOWER_CAMEL,
UPPER_CAMEL,
LOWER_SNAKE,
UPPER_SNAKE,
};
std::string FormatStringCase(const std::string& str, CaseFormatType to,
const char delimiter = '_') {
const bool from_snake = (str == absl::AsciiStrToUpper(str)) ||
(str == absl::AsciiStrToLower(str));
const bool toUpper = (to == UPPER_CAMEL || to == UPPER_SNAKE);
const bool toSnake = (to == LOWER_SNAKE || to == UPPER_SNAKE);
std::string result;
bool inputStart = true;
bool wordStart = true;
for (const char c : str) {
// Find a word start.
if (c == delimiter) {
// Repeated cases of wordStart means explicit delimiter usage.
if (wordStart) {
result.push_back(delimiter);
}
wordStart = true;
continue;
}
if (!from_snake && absl::ascii_isupper(c)) {
wordStart = true;
}
// add delimiter
if (wordStart && toSnake && !inputStart) {
result.push_back(delimiter);
}
// add the next letter from the input string (choosing upper/lower case)
const bool shouldCapIfSnake = toUpper;
const bool shouldCapIfCamel = wordStart && (toUpper || !inputStart);
if ((toSnake && shouldCapIfSnake) || (!toSnake && shouldCapIfCamel)) {
result += absl::ascii_toupper(c);
} else {
result += absl::ascii_tolower(c);
}
// at this point we are no longer at the start of a word:
wordStart = false;
// .. or the input:
inputStart = false;
}
if (wordStart) {
// This only happens with a trailing delimiter, which should remain.
result.push_back(delimiter);
}
return result;
}
} // namespace
//
// Public interface
//
std::string toLowerCamel(const std::string& s, const char delimiter) {
return FormatStringCase(s, LOWER_CAMEL, delimiter);
}
std::string toLowerSnake(const std::string& s, const char delimiter) {
return FormatStringCase(s, LOWER_SNAKE, delimiter);
}
std::string toUpperCamel(const std::string& s, const char delimiter) {
return FormatStringCase(s, UPPER_CAMEL, delimiter);
}
std::string toUpperSnake(const std::string& s, const char delimiter) {
return FormatStringCase(s, UPPER_SNAKE, delimiter);
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/case_format.h
+46
View File
@@ -0,0 +1,46 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CASE_FORMAT_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CASE_FORMAT_H_
#include "tensorflow/core/platform/str_util.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
// Conversion routines between upper/lower camel/snake case formats, e.g.:
// "lowerCamelCase"
// "lower_snake_case"
// "UpperCamelCase"
// "UPPER_SNAKE_CASE"
//
// The input format is automatically detected.
// The delimiter must be specified if it is other than an underscore ('_')
// for conversion either *to* or *from* snake case.
//
// Leading and trailing delimiters are supported, e.g.:
// "__OneTwo__" (in camel case) <==> "__ONE_TWO__" (in snake case)
//
// Note: performance not yet tested.
std::string toLowerCamel(const std::string& s, const char delimiter = '_');
std::string toLowerSnake(const std::string& s, const char delimiter = '_');
std::string toUpperCamel(const std::string& s, const char delimiter = '_');
std::string toUpperSnake(const std::string& s, const char delimiter = '_');
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CASE_FORMAT_H_
tensorflow/c/experimental/ops/gen/common/case_format_test.cc
+128
View File
@@ -0,0 +1,128 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/case_format.h"
#include <string>
#include "tensorflow/core/platform/test.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace {
// For each test case, we manually construct the 4 variations in string case and
// test all 16 conversions: from and to each of the 4 string case variations.
struct Variations {
std::string lower_camel;
std::string lower_snake;
std::string upper_camel;
std::string upper_snake;
};
void TestSingleVariation(const std::string& str, Variations expected,
char delimiter = '_') {
EXPECT_EQ(expected.lower_camel, toLowerCamel(str, delimiter));
EXPECT_EQ(expected.lower_snake, toLowerSnake(str, delimiter));
EXPECT_EQ(expected.upper_camel, toUpperCamel(str, delimiter));
EXPECT_EQ(expected.upper_snake, toUpperSnake(str, delimiter));
}
void TestAllVariations(Variations variations, char delimiter = '_') {
TestSingleVariation(variations.lower_camel, variations, delimiter);
TestSingleVariation(variations.lower_snake, variations, delimiter);
TestSingleVariation(variations.upper_camel, variations, delimiter);
TestSingleVariation(variations.upper_snake, variations, delimiter);
}
TEST(CppOpGenCaseFormat, test_single_word) {
TestAllVariations(Variations{
"three",
"three",
"Three",
"THREE",
});
}
TEST(CppOpGenCaseFormat, test_complex_string) {
TestAllVariations(Variations{
"threeNTest33Words",
"three_n_test33_words",
"ThreeNTest33Words",
"THREE_N_TEST33_WORDS",
});
}
TEST(CppOpGenCaseFormat, test_hyphen_delimiter) {
TestAllVariations(
Variations{
"threeNTest33Words",
"three-n-test33-words",
"ThreeNTest33Words",
"THREE-N-TEST33-WORDS",
},
'-');
}
TEST(CppOpGenCaseFormat, test_trailing_underscore) {
TestAllVariations(Variations{
"threeNTest33Words_",
"three_n_test33_words_",
"ThreeNTest33Words_",
"THREE_N_TEST33_WORDS_",
});
}
TEST(CppOpGenCaseFormat, test_double_trailing_underscores) {
TestAllVariations(Variations{
"xxY__",
"xx_y__",
"XxY__",
"XX_Y__",
});
}
TEST(CppOpGenCaseFormat, test_leading_underscore) {
TestAllVariations(Variations{
"_threeNTest33Words",
"_three_n_test33_words",
"_ThreeNTest33Words",
"_THREE_N_TEST33_WORDS",
});
}
TEST(CppOpGenCaseFormat, test_double_leading_underscores) {
TestAllVariations(Variations{
"__threeNTest33Words",
"__three_n_test33_words",
"__ThreeNTest33Words",
"__THREE_N_TEST33_WORDS",
});
}
TEST(CppOpGenCaseFormat, test_leading_and_trailing_underscores) {
TestAllVariations(Variations{
"__threeNTest33Words____",
"__three_n_test33_words____",
"__ThreeNTest33Words____",
"__THREE_N_TEST33_WORDS____",
});
}
} // namespace
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/controller.cc
+94
View File
@@ -0,0 +1,94 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/controller.h"
#include <string>
#include <vector>
#include "absl/log/check.h"
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include "xla/tsl/platform/status.h"
#include "tensorflow/core/framework/api_def.pb.h"
#include "tensorflow/core/framework/op.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/framework/op_gen_lib.h"
#include "tensorflow/core/platform/env.h"
#include "tensorflow/core/platform/logging.h"
#include "tensorflow/core/platform/path.h"
namespace tensorflow {
namespace generator {
Controller::Controller(PathConfig path_config, Env* env)
: env_(env), path_config_(path_config) {
// Load the Op and API definitions
InitializeOpApi();
// Convert the Op and API definitions to the internal data model
BuildModel();
}
Controller::~Controller() { delete api_def_map_; }
const void Controller::WriteFile(const std::string& file_path,
const SourceCode& code) const {
TF_CHECK_OK(WriteStringToFile(env_, file_path, code.Render())) << file_path;
}
const std::vector<OpSpec>& Controller::GetModelOps() const {
return operators_;
}
void Controller::InitializeOpApi() {
OpRegistry::Global()->Export(false, &op_list_);
// Load matching API defs for each Op. Paths are visited in order, allowing
// python/api_def_Xyz.pbtxt to override base/api_def_Xyz.pbtxt, for example.
api_def_map_ = new ApiDefMap(op_list_);
for (const auto& op : op_list_.op()) {
for (const auto& dir : path_config_.api_dirs) {
const std::string file_name =
absl::Substitute("api_def_$0.pbtxt", op.name());
const std::string file_path = io::JoinPath(dir, file_name);
if (env_->FileExists(file_path).ok()) {
TF_CHECK_OK(api_def_map_->LoadFile(env_, file_path)) << file_path;
} else {
// API defs are currently used for only optional pieces.
}
}
}
// Doc strings (summary, description) typically come from the API def.
api_def_map_->UpdateDocs();
}
void Controller::BuildModel() {
// Build the internal data model for the requested ops
for (const auto& op_name : path_config_.op_names) {
const OpDef* op_def = nullptr;
TF_CHECK_OK(OpRegistry::Global()->LookUpOpDef(op_name, &op_def));
CHECK(op_def != nullptr); // Crash OK
const ApiDef* api_def = api_def_map_->GetApiDef(op_name);
CHECK(api_def != nullptr); // Crash OK
operators_.push_back(OpSpec::Create(*op_def, *api_def));
}
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/controller.h
+58
View File
@@ -0,0 +1,58 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CONTROLLER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CONTROLLER_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/framework/op_gen_lib.h"
#include "tensorflow/core/platform/env.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
class Controller {
public:
explicit Controller(PathConfig path_config, Env* env = Env::Default());
virtual ~Controller();
const void WriteFile(const std::string& file_path,
const SourceCode& code) const;
const std::vector<OpSpec>& GetModelOps() const;
private:
void InitializeOpApi();
void BuildModel();
// Data model: Ops to generate
std::vector<OpSpec> operators_;
// Configuration
Env* env_;
PathConfig path_config_;
// Initialized TensorFlow Op/API definitions
OpList op_list_;
ApiDefMap* api_def_map_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_CONTROLLER_H_
tensorflow/c/experimental/ops/gen/common/path_config.cc
+69
View File
@@ -0,0 +1,69 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include <algorithm>
#include <string>
#include <vector>
#include "absl/strings/str_join.h"
#include "tensorflow/core/lib/strings/str_util.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
PathConfig::PathConfig(const std::string& output_dir,
const std::string& source_dir,
const std::string& api_dir_list,
const std::vector<std::string> op_names)
: output_path(output_dir), op_names(op_names) {
api_dirs = str_util::Split(api_dir_list, ",", str_util::SkipEmpty());
// Decompose the directory components given the output/source directories.
//
// Be flexible; accept any path string with a "tensorflow" directory name.
// (It's hard to construct a location-agnostic include path string using the
// build system, so we accept an example, such as the source build target.)
tf_root_dir = "tensorflow";
// Prefix, e.g. "third_party" given root_dir "third_party/tensorflow/...."
std::vector<std::string> source_path_components =
tensorflow::str_util::Split(source_dir, "/");
auto source_tfroot_pos = std::find(source_path_components.begin(),
source_path_components.end(), tf_root_dir);
if (source_tfroot_pos != source_path_components.end()) {
tf_prefix_dir =
absl::StrJoin(source_path_components.begin(), source_tfroot_pos, "/");
} else {
tf_prefix_dir = source_dir;
}
// TF subdir, e.g. "c/ops" given output_dir "blah/blah/tensorflow/c/ops"
std::vector<std::string> output_path_components =
tensorflow::str_util::Split(output_dir, "/");
auto output_tfroot_pos = std::find(output_path_components.begin(),
output_path_components.end(), tf_root_dir);
if (output_tfroot_pos != output_path_components.end()) {
tf_output_dir =
absl::StrJoin(output_tfroot_pos + 1, output_path_components.end(), "/");
} else {
tf_output_dir = output_dir;
}
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/path_config.h
+43
View File
@@ -0,0 +1,43 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_PATH_CONFIG_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_PATH_CONFIG_H_
#include <vector>
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
struct PathConfig {
std::string output_path;
std::vector<std::string> op_names;
std::vector<std::string> api_dirs;
std::string tf_prefix_dir;
std::string tf_root_dir;
std::string tf_output_dir;
explicit PathConfig() = default;
explicit PathConfig(const std::string& output_dir,
const std::string& source_dir,
const std::string& api_dir_list,
const std::vector<std::string> op_names);
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_PATH_CONFIG_H_
tensorflow/c/experimental/ops/gen/common/source_code.cc
+68
View File
@@ -0,0 +1,68 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include <string>
#include "absl/log/log.h"
#include "absl/strings/ascii.h"
#include "absl/strings/match.h"
#include "absl/strings/str_cat.h"
#include "absl/strings/string_view.h"
#include "absl/strings/strip.h"
#include "tensorflow/core/platform/logging.h"
#include "tensorflow/core/platform/stringpiece.h"
namespace tensorflow {
namespace generator {
std::string SourceCode::Render() const {
std::string code;
for (const Line& line : lines_) {
absl::StrAppend(&code, std::string(line.indent * spaces_per_indent_, ' '),
line.text, "\n");
}
return code;
}
void SourceCode::AddLineWithIndent(const std::string& line) {
ValidateAndAddLine(current_indent_, line);
}
void SourceCode::AddLineWithoutIndent(const std::string& line) {
ValidateAndAddLine(0, line);
}
void SourceCode::AddBlankLine() { ValidateAndAddLine(0, ""); }
void SourceCode::IncreaseIndent() { current_indent_++; }
void SourceCode::DecreaseIndent() { current_indent_--; }
void SourceCode::ValidateAndAddLine(int indent, const std::string& raw_line) {
absl::string_view line(raw_line);
bool had_trailing_newline = absl::ConsumeSuffix(&line, "\n");
if (absl::StrContains(line, '\n')) {
LOG(ERROR) << "Attempt to add multiple lines; bad formatting is likely.";
} else if (had_trailing_newline) {
LOG(WARNING) << "Superfluous trailing newline in '" << line << "'";
}
lines_.push_back(
{indent, std::string(absl::StripTrailingAsciiWhitespace(line))});
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/source_code.h
+54
View File
@@ -0,0 +1,54 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_SOURCE_CODE_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_SOURCE_CODE_H_
#include <vector>
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
class SourceCode {
public:
std::string Render() const;
void SetSpacesPerIndent(int spaces_per_indent) {
spaces_per_indent_ = spaces_per_indent;
}
void AddLineWithIndent(const std::string& line);
void AddLineWithoutIndent(const std::string& line);
void AddBlankLine();
void IncreaseIndent();
void DecreaseIndent();
private:
struct Line {
int indent;
std::string text;
};
void ValidateAndAddLine(int indent_level, const std::string& raw_line);
int spaces_per_indent_ = 2;
int current_indent_ = 0;
std::vector<Line> lines_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_SOURCE_CODE_H_
tensorflow/c/experimental/ops/gen/common/view_util.cc
+43
View File
@@ -0,0 +1,43 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/common/view_util.h"
#include <string>
#include <vector>
#include "absl/strings/str_join.h"
#include "absl/strings/substitute.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
std::string Call(const std::string& object, const std::string& method,
std::vector<std::string> arguments, const char* oper) {
return absl::Substitute("$0$1$2($3)", object, oper, method,
absl::StrJoin(arguments, ", "));
}
std::string Call(const std::string& function,
std::vector<std::string> arguments) {
return absl::Substitute("$0($1)", function, absl::StrJoin(arguments, ", "));
}
std::string Quoted(const std::string& s) {
return absl::Substitute("\"$0\"", s);
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/common/view_util.h
+34
View File
@@ -0,0 +1,34 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_VIEW_UTIL_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_VIEW_UTIL_H_
#include <vector>
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
std::string Call(const std::string& function,
std::vector<std::string> arguments);
std::string Call(const std::string& object, const std::string& method,
std::vector<std::string> arguments, const char* oper = "->");
std::string Quoted(const std::string& s);
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_COMMON_VIEW_UTIL_H_
tensorflow/c/experimental/ops/gen/cpp/BUILD
+53
View File
@@ -0,0 +1,53 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load(
"//tensorflow:tensorflow.bzl",
"tf_cc_test",
)
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:private"],
licenses = ["notice"],
)
cc_library(
name = "cpp",
srcs = glob(
["*.cc"],
exclude = ["*_test.cc"],
),
hdrs = glob(["*.h"]),
visibility = ["//tensorflow/c/experimental/ops/gen:__pkg__"],
deps = [
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/c/experimental/ops/gen/cpp/renderers",
"//tensorflow/c/experimental/ops/gen/cpp/views",
"//tensorflow/c/experimental/ops/gen/model",
"//tensorflow/core:framework",
"//tensorflow/core:lib",
"//tensorflow/core:lib_internal",
"//tensorflow/core:op_gen_lib",
"//tensorflow/core:protos_all_cc",
"@com_google_absl//absl/strings",
],
alwayslink = 1,
)
tf_cc_test(
name = "cpp_generator_test",
size = "small",
srcs = ["cpp_generator_test.cc"],
data = ["//tensorflow/c/experimental/ops/gen/cpp/golden"],
deps = [
":cpp",
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/c/experimental/ops/gen/cpp/renderers",
"//tensorflow/core:core_cpu",
"//tensorflow/core:framework",
"//tensorflow/core:lib",
"//tensorflow/core:test",
"//tensorflow/core:test_main",
"//tensorflow/core:testlib",
"@xla//xla/tsl/platform:status",
],
)
tensorflow/c/experimental/ops/gen/cpp/cpp_generator.cc
+73
View File
@@ -0,0 +1,73 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/cpp_generator.h"
#include <string>
#include <vector>
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_file_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include "tensorflow/core/platform/path.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
CppGenerator::CppGenerator(cpp::CppConfig cpp_config, PathConfig path_config)
: controller_(path_config),
cpp_config_(cpp_config),
path_config_(path_config) {}
SourceCode CppGenerator::GenerateOneFile(
cpp::RendererContext::Mode mode) const {
SourceCode generated_code;
const std::vector<OpSpec> ops(controller_.GetModelOps());
std::vector<cpp::OpView> views(ops.begin(), ops.end());
cpp::RendererContext context{mode, generated_code, cpp_config_, path_config_};
cpp::CppFileRenderer(context, views).Render();
return generated_code;
}
SourceCode CppGenerator::HeaderFileContents() const {
return GenerateOneFile(cpp::RendererContext::kHeader);
}
SourceCode CppGenerator::SourceFileContents() const {
return GenerateOneFile(cpp::RendererContext::kSource);
}
std::string CppGenerator::HeaderFileName() const {
return io::JoinPath(path_config_.output_path, cpp_config_.unit + "_ops.h");
}
std::string CppGenerator::SourceFileName() const {
return io::JoinPath(path_config_.output_path, cpp_config_.unit + "_ops.cc");
}
void CppGenerator::WriteHeaderFile() const {
controller_.WriteFile(HeaderFileName(), HeaderFileContents());
}
void CppGenerator::WriteSourceFile() const {
controller_.WriteFile(SourceFileName(), SourceFileContents());
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/cpp_generator.h
+49
View File
@@ -0,0 +1,49 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_CPP_GENERATOR_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_CPP_GENERATOR_H_
#include "tensorflow/c/experimental/ops/gen/common/controller.h"
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
class CppGenerator {
public:
explicit CppGenerator(cpp::CppConfig cpp_config, PathConfig path_config);
SourceCode HeaderFileContents() const;
SourceCode SourceFileContents() const;
std::string HeaderFileName() const;
std::string SourceFileName() const;
void WriteHeaderFile() const;
void WriteSourceFile() const;
private:
SourceCode GenerateOneFile(cpp::RendererContext::Mode mode) const;
Controller controller_;
cpp::CppConfig cpp_config_;
PathConfig path_config_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_CPP_GENERATOR_H_
tensorflow/c/experimental/ops/gen/cpp/cpp_generator_test.cc
+83
View File
@@ -0,0 +1,83 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/cpp_generator.h"
#include <algorithm>
#include <string>
#include <vector>
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include "xla/tsl/platform/status.h"
#include "tensorflow/core/platform/env.h"
#include "tensorflow/core/platform/path.h"
#include "tensorflow/core/platform/test.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace {
TEST(CppGeneratorTest, typical_usage) {
std::string category = "testing";
std::string name_space = "tensorflow::ops";
std::string output_dir = "tensorflow/c/experimental/ops/gen/cpp/golden";
std::string source_dir = "tensorflow";
std::string api_dirs = "";
std::vector<std::string> ops = {
"Neg", // Simple unary Op
"MatMul", // 2 inputs & attrs with default values
"IdentityN", // Variadic input+output
"SparseSoftmaxCrossEntropyWithLogits", // 2 outputs
"AccumulatorApplyGradient", // 0 outputs
"VarHandleOp", // type, shape, list(string) attrs
"RestoreV2", // Variadic output-only, list(type) attr
};
cpp::CppConfig cpp_config(category, name_space);
PathConfig controller_config(output_dir, source_dir, api_dirs, ops);
CppGenerator generator(cpp_config, controller_config);
Env *env = Env::Default();
std::string golden_dir = io::JoinPath(testing::TensorFlowSrcRoot(),
controller_config.tf_output_dir);
std::string generated_header = generator.HeaderFileContents().Render();
std::string generated_source = generator.SourceFileContents().Render();
std::string expected_header;
std::string header_file_name =
io::JoinPath(golden_dir, "testing_ops.h.golden");
TF_CHECK_OK(ReadFileToString(env, header_file_name, &expected_header));
std::string expected_source;
std::string source_file_name =
io::JoinPath(golden_dir, "testing_ops.cc.golden");
TF_CHECK_OK(ReadFileToString(env, source_file_name, &expected_source));
// Remove carriage returns (for Windows)
expected_header.erase(
std::remove(expected_header.begin(), expected_header.end(), '\r'),
expected_header.end());
expected_source.erase(
std::remove(expected_source.begin(), expected_source.end(), '\r'),
expected_source.end());
EXPECT_EQ(expected_header, generated_header);
EXPECT_EQ(expected_source, generated_source);
}
} // namespace
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/golden/BUILD
+10
View File
@@ -0,0 +1,10 @@
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:public"],
licenses = ["notice"],
)
filegroup(
name = "golden",
data = glob(["*.golden"]),
)
tensorflow/c/experimental/ops/gen/cpp/golden/testing_ops.cc.golden
+150
View File
@@ -0,0 +1,150 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
// This file is MACHINE GENERATED! Do not edit.
#include "tensorflow/c/experimental/ops/gen/cpp/golden/testing_ops.h"
#include "absl/status/status.h"
#include "absl/types/span.h"
#include "tensorflow/c/eager/abstract_context.h"
#include "tensorflow/c/eager/abstract_tensor_handle.h"
#include "tensorflow/core/framework/types.h" // NOLINT
#include <cstring> // NOLINT
#include "tensorflow/c/eager/abstract_operation.h"
#include "tensorflow/c/eager/tracing_utils.h"
#include "tensorflow/core/platform/errors.h" // NOLINT
using tensorflow::tracing::MaybeSetOpName;
namespace tensorflow {
namespace ops {
// Op: Neg()
// Summary:
//
// Description:
absl::Status Neg(AbstractContext* ctx, AbstractTensorHandle* const x, AbstractTensorHandle** y, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("Neg", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInput(x));
int num_retvals = 1;
return op_ptr->Execute(absl::MakeSpan(y, 1), &num_retvals);
}
// Op: MatMul()
// Summary:
//
// Description:
absl::Status MatMul(AbstractContext* ctx, AbstractTensorHandle* const a, AbstractTensorHandle* const b, AbstractTensorHandle** product, bool transpose_a, bool transpose_b, bool grad_a, bool grad_b, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("MatMul", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInput(a));
TF_RETURN_IF_ERROR(op_ptr->AddInput(b));
TF_RETURN_IF_ERROR(op_ptr->SetAttrBool("transpose_a", transpose_a));
TF_RETURN_IF_ERROR(op_ptr->SetAttrBool("transpose_b", transpose_b));
TF_RETURN_IF_ERROR(op_ptr->SetAttrBool("grad_a", grad_a));
TF_RETURN_IF_ERROR(op_ptr->SetAttrBool("grad_b", grad_b));
int num_retvals = 1;
return op_ptr->Execute(absl::MakeSpan(product, 1), &num_retvals);
}
// Op: IdentityN()
// Summary:
//
// Description:
absl::Status IdentityN(AbstractContext* ctx, absl::Span<AbstractTensorHandle* const> input, absl::Span<AbstractTensorHandle*> output, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("IdentityN", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInputList(input));
int num_retvals = output.size();
return op_ptr->Execute(output, &num_retvals);
}
// Op: SparseSoftmaxCrossEntropyWithLogits()
// Summary:
//
// Description:
absl::Status SparseSoftmaxCrossEntropyWithLogits(AbstractContext* ctx, AbstractTensorHandle* const features, AbstractTensorHandle* const labels, AbstractTensorHandle** loss, AbstractTensorHandle** backprop, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("SparseSoftmaxCrossEntropyWithLogits", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInput(features));
TF_RETURN_IF_ERROR(op_ptr->AddInput(labels));
int num_retvals = 2;
AbstractTensorHandle* temp_outputs[2] = {nullptr};
absl::Status status = op_ptr->Execute(temp_outputs, &num_retvals);
if (status.ok()) {
*loss = temp_outputs[0];
*backprop = temp_outputs[1];
}
return status;
}
// Op: AccumulatorApplyGradient()
// Summary:
//
// Description:
absl::Status AccumulatorApplyGradient(AbstractContext* ctx, AbstractTensorHandle** handle, AbstractTensorHandle* const local_step, AbstractTensorHandle* const gradient, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("AccumulatorApplyGradient", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInput(handle));
TF_RETURN_IF_ERROR(op_ptr->AddInput(local_step));
TF_RETURN_IF_ERROR(op_ptr->AddInput(gradient));
int num_retvals = 0;
std::vector<AbstractTensorHandle*> dummy_outputs;
return op_ptr->Execute(absl::MakeSpan(dummy_outputs), &num_retvals);
}
// Op: VarHandleOp()
// Summary:
//
// Description:
absl::Status VarHandleOp(AbstractContext* ctx, AbstractTensorHandle** resource, DataType dtype, const PartialTensorShape shape, const char* container, const char* shared_name, const char* debug_name, absl::Span<std::string const> allowed_devices, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("VarHandleOp", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->SetAttrString("container", container, strlen(container)));
TF_RETURN_IF_ERROR(op_ptr->SetAttrString("shared_name", shared_name, strlen(shared_name)));
TF_RETURN_IF_ERROR(op_ptr->SetAttrString("debug_name", debug_name, strlen(debug_name)));
TF_RETURN_IF_ERROR(op_ptr->SetAttrType("dtype", dtype));
TF_RETURN_IF_ERROR(op_ptr->SetAttrShape("shape", shape));
TF_RETURN_IF_ERROR(op_ptr->SetAttrStringList("allowed_devices", allowed_devices));
int num_retvals = 1;
return op_ptr->Execute(absl::MakeSpan(resource, 1), &num_retvals);
}
// Op: RestoreV2()
// Summary:
//
// Description:
absl::Status RestoreV2(AbstractContext* ctx, AbstractTensorHandle* const prefix, AbstractTensorHandle* const tensor_names, AbstractTensorHandle* const shape_and_slices, absl::Span<AbstractTensorHandle*> tensors, absl::Span<DataType> dtypes, const char* name, const char* raw_device_name) {
AbstractOperationPtr op_ptr(ctx->CreateOperation());
TF_RETURN_IF_ERROR(op_ptr->Reset("RestoreV2", raw_device_name));
TF_RETURN_IF_ERROR(MaybeSetOpName(op_ptr.get(), name));
TF_RETURN_IF_ERROR(op_ptr->AddInput(prefix));
TF_RETURN_IF_ERROR(op_ptr->AddInput(tensor_names));
TF_RETURN_IF_ERROR(op_ptr->AddInput(shape_and_slices));
TF_RETURN_IF_ERROR(op_ptr->SetAttrTypeList("dtypes", dtypes.data(), dtypes.length()));
int num_retvals = tensors.size();
return op_ptr->Execute(tensors, &num_retvals);
}
} // namespace ops
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/golden/testing_ops.h.golden
+54
View File
@@ -0,0 +1,54 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
// This file is MACHINE GENERATED! Do not edit.
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_GOLDEN_TESTING_OPS_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_GOLDEN_TESTING_OPS_H_
#include "absl/status/status.h"
#include "absl/types/span.h"
#include "tensorflow/c/eager/abstract_context.h"
#include "tensorflow/c/eager/abstract_tensor_handle.h"
#include "tensorflow/core/framework/types.h" // NOLINT
namespace tensorflow {
namespace ops {
//
absl::Status Neg(AbstractContext* ctx, AbstractTensorHandle* const x, AbstractTensorHandle** y, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status MatMul(AbstractContext* ctx, AbstractTensorHandle* const a, AbstractTensorHandle* const b, AbstractTensorHandle** product, bool transpose_a = false, bool transpose_b = false, bool grad_a = false, bool grad_b = false, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status IdentityN(AbstractContext* ctx, absl::Span<AbstractTensorHandle* const> input, absl::Span<AbstractTensorHandle*> output, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status SparseSoftmaxCrossEntropyWithLogits(AbstractContext* ctx, AbstractTensorHandle* const features, AbstractTensorHandle* const labels, AbstractTensorHandle** loss, AbstractTensorHandle** backprop, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status AccumulatorApplyGradient(AbstractContext* ctx, AbstractTensorHandle** handle, AbstractTensorHandle* const local_step, AbstractTensorHandle* const gradient, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status VarHandleOp(AbstractContext* ctx, AbstractTensorHandle** resource, DataType dtype, const PartialTensorShape shape, const char* container = "", const char* shared_name = "", const char* debug_name = "", absl::Span<std::string const> allowed_devices = {}, const char* name = nullptr, const char* raw_device_name = nullptr);
//
absl::Status RestoreV2(AbstractContext* ctx, AbstractTensorHandle* const prefix, AbstractTensorHandle* const tensor_names, AbstractTensorHandle* const shape_and_slices, absl::Span<AbstractTensorHandle*> tensors, absl::Span<DataType> dtypes, const char* name = nullptr, const char* raw_device_name = nullptr);
} // namespace ops
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_GOLDEN_TESTING_OPS_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/BUILD
+52
View File
@@ -0,0 +1,52 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load(
"//tensorflow:tensorflow.bzl",
"tf_cc_tests",
)
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:private"],
licenses = ["notice"],
)
cc_library(
name = "renderers",
srcs = glob(
["*.cc"],
exclude = ["*_test.cc"],
),
hdrs = glob(["*.h"]),
visibility = [
"//tensorflow/c/experimental/ops/gen:__pkg__",
"//tensorflow/c/experimental/ops/gen/cpp:__pkg__",
],
deps = [
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/c/experimental/ops/gen/cpp/views",
"//tensorflow/core:framework_internal",
"//tensorflow/core:lib",
"//tensorflow/core:lib_internal",
"@com_google_absl//absl/log",
"@com_google_absl//absl/strings",
],
alwayslink = 1,
)
tf_cc_tests(
name = "renderer_tests",
size = "small",
srcs = glob(
["*_test.cc"],
),
deps = [
":renderers",
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/core:core_cpu",
"//tensorflow/core:framework",
"//tensorflow/core:test",
"//tensorflow/core:test_main",
"//tensorflow/core:testlib",
"//tensorflow/core/platform:types",
],
)
tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.cc
+34
View File
@@ -0,0 +1,34 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include <string>
#include "absl/strings/ascii.h"
#include "absl/strings/str_split.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
CppConfig::CppConfig(const std::string& category, const std::string& name_space)
: category(category),
unit(absl::AsciiStrToLower(category)),
namespaces(absl::StrSplit(name_space, "::")) {}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h
+40
View File
@@ -0,0 +1,40 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_CONFIG_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_CONFIG_H_
#include <vector>
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
struct CppConfig {
std::string category;
std::string unit;
std::vector<std::string> namespaces;
explicit CppConfig() = default;
explicit CppConfig(const std::string& category,
const std::string& name_space = "tensorflow::ops");
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_CONFIG_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_file_renderer.cc
+85
View File
@@ -0,0 +1,85 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_file_renderer.h"
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
static const char *copyright =
R"(
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
)";
static const char *machine_generated =
"// This file is MACHINE GENERATED! Do not edit.";
CppFileRenderer::CppFileRenderer(RendererContext context,
const std::vector<OpView> &ops)
: Renderer(context),
guard_(context),
name_space_(context),
includes_(context),
ops_(ops) {}
void CppFileRenderer::Render() {
CodeLines(copyright);
BlankLine();
CodeLine(machine_generated);
BlankLine();
if (context_.mode == RendererContext::kHeader) {
guard_.Open();
} else {
includes_.SelfHeader();
}
includes_.Headers(ops_);
name_space_.Open();
BlankLine();
for (const OpView &op : ops_) {
OpRenderer(context_, op).Render();
}
name_space_.Close();
if (context_.mode == RendererContext::kHeader) {
guard_.Close();
}
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_file_renderer.h
+48
View File
@@ -0,0 +1,48 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_FILE_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_FILE_RENDERER_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/guard_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/include_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/namespace_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class CppFileRenderer : public Renderer {
public:
explicit CppFileRenderer(RendererContext context,
const std::vector<OpView> &ops);
void Render();
private:
GuardRenderer guard_;
NamespaceRenderer name_space_;
IncludeRenderer includes_;
std::vector<OpView> ops_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_CPP_FILE_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/guard_renderer.cc
+53
View File
@@ -0,0 +1,53 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/guard_renderer.h"
#include <algorithm>
#include <string>
#include "tensorflow/c/experimental/ops/gen/common/case_format.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/path.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
GuardRenderer::GuardRenderer(RendererContext context) : Renderer(context) {
std::string self_path = io::JoinPath(context_.path_config.tf_root_dir,
context_.path_config.tf_output_dir,
context_.cpp_config.unit + "_ops.h");
std::string with_underscores(self_path);
std::replace(with_underscores.begin(), with_underscores.end(), '/', '_');
std::replace(with_underscores.begin(), with_underscores.end(), '.', '_');
guard_ = toUpperSnake(with_underscores) + "_";
}
void GuardRenderer::Open() {
CodeLine("#ifndef $0", guard_);
CodeLine("#define $0", guard_);
BlankLine();
}
void GuardRenderer::Close() {
BlankLine();
CodeLine("#endif // $0", guard_);
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/guard_renderer.h
+41
View File
@@ -0,0 +1,41 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_GUARD_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_GUARD_RENDERER_H_
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class GuardRenderer : public Renderer {
public:
explicit GuardRenderer(RendererContext context);
void Open();
void Close();
private:
std::string guard_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_GUARD_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/include_renderer.cc
+98
View File
@@ -0,0 +1,98 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/include_renderer.h"
#include <string>
#include <vector>
#include "absl/strings/match.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
#include "tensorflow/core/platform/path.h"
namespace tensorflow {
namespace generator {
namespace cpp {
IncludeRenderer::IncludeRenderer(RendererContext context) : Renderer(context) {}
void IncludeRenderer::SelfHeader() {
Include(SelfHeaderPath());
BlankLine();
}
std::string IncludeRenderer::SelfHeaderPath() const {
return io::JoinPath(context_.path_config.tf_root_dir,
context_.path_config.tf_output_dir,
context_.cpp_config.unit + "_ops.h");
}
void IncludeRenderer::Include(const std::string& tf_file_path) {
CodeLine("#include \"$0\"",
io::JoinPath(context_.path_config.tf_prefix_dir, tf_file_path));
}
void IncludeRenderer::Headers(const std::vector<OpView>& ops) {
Include(
"absl"
"/status/status.h");
bool needs_span = false;
if (context_.mode == RendererContext::kSource) {
needs_span = true;
} else {
for (const OpView& op : ops) {
for (const OpArgumentView& arg : op.AllArguments()) {
if (absl::StrContains(arg.Declaration(), "absl::Span")) {
needs_span = true;
break;
}
}
if (needs_span) break;
}
}
if (needs_span) {
Include(
"absl"
"/types/span.h");
}
Include("tensorflow/c/eager/abstract_context.h");
Include("tensorflow/c/eager/abstract_tensor_handle.h");
if (context_.cpp_config.unit == "resource_variable") {
Include("tensorflow/core/framework/tensor_shape.h");
}
CodeLine("#include \"$0\" // NOLINT",
io::JoinPath(context_.path_config.tf_prefix_dir,
"tensorflow/core/framework/types.h"));
if (context_.cpp_config.unit == "resource_variable") {
CodeLine("#include <string>");
}
if (context_.mode == RendererContext::kSource) {
CodeLine("#include <cstring> // NOLINT");
Include("tensorflow/c/eager/abstract_operation.h");
Include("tensorflow/c/eager/tracing_utils.h");
CodeLine("#include \"$0\" // NOLINT",
io::JoinPath(context_.path_config.tf_prefix_dir,
"tensorflow/core/platform/errors.h"));
BlankLine();
Statement("using tensorflow::tracing::MaybeSetOpName");
}
BlankLine();
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/include_renderer.h
+45
View File
@@ -0,0 +1,45 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_INCLUDE_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_INCLUDE_RENDERER_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class IncludeRenderer : public Renderer {
public:
explicit IncludeRenderer(RendererContext context);
std::string SelfHeaderPath() const;
void SelfHeader();
void Headers(const std::vector<OpView>& ops);
private:
void Include(const std::string& tf_file_path);
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_INCLUDE_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/namespace_renderer.cc
+45
View File
@@ -0,0 +1,45 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/namespace_renderer.h"
#include <string>
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
NamespaceRenderer::NamespaceRenderer(RendererContext context)
: Renderer(context) {}
void NamespaceRenderer::Open() {
for (const std::string& ns : context_.cpp_config.namespaces) {
CodeLine("namespace " + ns + " {");
}
}
void NamespaceRenderer::Close() {
for (auto it = context_.cpp_config.namespaces.rbegin();
it != context_.cpp_config.namespaces.rend(); ++it) {
CodeLine("} // namespace " + *it);
}
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/namespace_renderer.h
+40
View File
@@ -0,0 +1,40 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_NAMESPACE_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_NAMESPACE_RENDERER_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class NamespaceRenderer : public Renderer {
public:
explicit NamespaceRenderer(RendererContext context);
void Open();
void Close();
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_NAMESPACE_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/op_comment_renderer.cc
+46
View File
@@ -0,0 +1,46 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_comment_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
OpCommentRenderer::OpCommentRenderer(RendererContext context, OpView op)
: Renderer(context), op_(op) {}
void OpCommentRenderer::Render() {
if (context_.mode == RendererContext::kHeader) {
// Add a short 1-line comment to the header files.
CommentLine(op_.Summary());
return;
}
CommentLine("Op: $0()", op_.FunctionName());
CommentLine("Summary: $0", op_.Summary());
CommentLine("");
CommentLine("Description:");
for (const auto& line : op_.Description()) {
CommentLine(" $0", line);
}
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/op_comment_renderer.h
+40
View File
@@ -0,0 +1,40 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_COMMENT_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_COMMENT_RENDERER_H_
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class OpCommentRenderer : public Renderer {
public:
explicit OpCommentRenderer(RendererContext context, OpView op);
void Render();
private:
OpView op_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_COMMENT_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/op_implementation_renderer.cc
+102
View File
@@ -0,0 +1,102 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_implementation_renderer.h"
#include "tensorflow/c/experimental/ops/gen/common/view_util.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
OpImplementationRenderer::OpImplementationRenderer(RendererContext context,
OpView op)
: Renderer(context), op_(op) {}
void OpImplementationRenderer::Render() {
RenderInitialization();
if (op_.IsListOp()) {
RenderExecutionListOp();
} else if (op_.NumOutputs() == 0) {
RenderExecutionZeroOutputs();
} else if (op_.NumOutputs() == 1) {
RenderExecutionSingleOutput();
} else {
RenderExecutionMultipleOutputs();
}
}
void OpImplementationRenderer::RenderInitialization() {
// Create Op variable and initialize it
Statement("AbstractOperationPtr $0(ctx->CreateOperation())",
op_.VariableName());
TFStatement(Call(op_.VariableName(), "Reset",
{op_.OpNameString(), "raw_device_name"}));
TFStatement(Call("MaybeSetOpName", {op_.VariableName() + ".get()", "name"}));
// Set each input
for (const ArgView& ar : op_.Inputs()) {
TFStatement(Call(op_.VariableName(), ar.SetterMethod(), ar.SetterArgs()));
}
// Set each attribute
for (const AttrView& ar : op_.Attributes()) {
TFStatement(Call(op_.VariableName(), ar.SetterMethod(), ar.SetterArgs()));
}
}
void OpImplementationRenderer::RenderExecutionListOp() {
ArgView output_arg = op_.OnlyOutput();
Statement("int num_retvals = $0.size()", output_arg.VariableName());
Statement("return " + Call(op_.VariableName(), "Execute",
{output_arg.VariableName(), "&num_retvals"}));
}
void OpImplementationRenderer::RenderExecutionSingleOutput() {
ArgView output_arg = op_.OnlyOutput();
Statement("int num_retvals = 1");
Statement("return $0->Execute(absl::MakeSpan($1, 1), &num_retvals)",
op_.VariableName(), output_arg.VariableName());
}
void OpImplementationRenderer::RenderExecutionMultipleOutputs() {
Statement("int num_retvals = $0", op_.NumOutputs());
Statement("AbstractTensorHandle* temp_outputs[$0] = {nullptr}",
op_.NumOutputs());
Statement("absl::Status status = $0->Execute(temp_outputs, &num_retvals)",
op_.VariableName());
BlockOpen("if (status.ok())");
for (const ArgView& arg : op_.Outputs()) {
Statement("*$0 = temp_outputs[$1]", arg.VariableName(), arg.Position());
}
BlockClose();
Statement("return status");
}
void OpImplementationRenderer::RenderExecutionZeroOutputs() {
Statement("int num_retvals = 0");
Statement("std::vector<AbstractTensorHandle*> dummy_outputs");
Statement("return $0->Execute(absl::MakeSpan(dummy_outputs), &num_retvals)",
op_.VariableName());
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/op_implementation_renderer.h
+45
View File
@@ -0,0 +1,45 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_IMPLEMENTATION_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_IMPLEMENTATION_RENDERER_H_
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class OpImplementationRenderer : public Renderer {
public:
explicit OpImplementationRenderer(RendererContext context, OpView op);
void Render();
private:
void RenderInitialization();
void RenderExecutionListOp();
void RenderExecutionMultipleOutputs();
void RenderExecutionZeroOutputs();
void RenderExecutionSingleOutput();
OpView op_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_IMPLEMENTATION_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/op_renderer.cc
+79
View File
@@ -0,0 +1,79 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_renderer.h"
#include <iterator>
#include <string>
#include <vector>
#include "absl/strings/str_cat.h"
#include "absl/strings/str_join.h"
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_implementation_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
namespace tensorflow {
namespace generator {
namespace cpp {
std::string OpRenderer::Signature() const {
std::vector<std::string> args_with_default_val;
std::vector<std::string> args_without_default_val;
for (OpArgumentView const& argument : op_.AllArguments()) {
bool is_header = (context_.mode == RendererContext::kHeader);
std::string text = argument.Declaration(is_header);
if (is_header) {
absl::StrAppend(&text, argument.Initializer());
}
if (argument.HasDefaultValue()) {
args_with_default_val.push_back(text);
} else {
args_without_default_val.push_back(text);
}
}
std::vector<std::string> arguments;
arguments.reserve(args_without_default_val.size() +
args_with_default_val.size());
arguments.insert(arguments.end(),
std::make_move_iterator(args_without_default_val.begin()),
std::make_move_iterator(args_without_default_val.end()));
arguments.insert(arguments.end(),
std::make_move_iterator(args_with_default_val.begin()),
std::make_move_iterator(args_with_default_val.end()));
return absl::Substitute("$0 $1($2)", "absl::Status", op_.FunctionName(),
absl::StrJoin(arguments, ", "));
}
OpRenderer::OpRenderer(RendererContext context, OpView op)
: Renderer(context), op_(op), comment_(context, op) {}
void OpRenderer::Render() {
comment_.Render();
if (context_.mode == RendererContext::kHeader) {
Statement(Signature());
} else {
BlockOpen(Signature());
OpImplementationRenderer(context_, op_).Render();
BlockClose();
}
BlankLine();
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/op_renderer.h
+44
View File
@@ -0,0 +1,44 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_RENDERER_H_
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/op_comment_renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class OpRenderer : public Renderer {
public:
explicit OpRenderer(RendererContext context, OpView op);
void Render();
private:
OpView op_;
OpCommentRenderer comment_;
std::string Signature() const;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_OP_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.cc
+87
View File
@@ -0,0 +1,87 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include <string>
#include "absl/log/log.h"
#include "absl/strings/match.h"
#include "absl/strings/str_cat.h"
#include "absl/strings/string_view.h"
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/lib/strings/str_util.h"
#include "tensorflow/core/platform/logging.h"
#include "tensorflow/core/platform/stringpiece.h"
namespace tensorflow {
namespace generator {
namespace cpp {
Renderer::Renderer(RendererContext context) : context_(context) {}
Renderer& Renderer::BlankLine() {
context_.code.AddLineWithoutIndent("");
return *this;
}
Renderer& Renderer::CodeLine(const std::string& text) {
context_.code.AddLineWithoutIndent(text);
return *this;
}
Renderer& Renderer::CodeLines(const std::string& text) {
absl::string_view trimmed_text(text);
str_util::RemoveWhitespaceContext(&trimmed_text);
for (const std::string& line : str_util::Split(trimmed_text, '\n')) {
context_.code.AddLineWithoutIndent(line);
}
return *this;
}
Renderer& Renderer::Statement(const std::string& text) {
if (absl::EndsWith(text, ";")) {
LOG(WARNING) << "Superfluous terminating ';' in '" << text << "'";
context_.code.AddLineWithIndent(text);
} else {
context_.code.AddLineWithIndent(absl::StrCat(text, ";"));
}
return *this;
}
Renderer& Renderer::TFStatement(const std::string& text) {
return Statement(absl::Substitute("TF_RETURN_IF_ERROR($0)", text));
}
Renderer& Renderer::CommentLine(const std::string& text) {
context_.code.AddLineWithIndent(absl::StrCat("// ", text));
return *this;
}
Renderer& Renderer::BlockOpen(const std::string& text) {
context_.code.AddLineWithIndent(absl::StrCat(text, " {"));
context_.code.IncreaseIndent();
return *this;
}
Renderer& Renderer::BlockClose(const std::string& text) {
context_.code.DecreaseIndent();
context_.code.AddLineWithIndent(absl::StrCat("}", text));
return *this;
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h
+100
View File
@@ -0,0 +1,100 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_H_
#include "absl/strings/string_view.h"
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class Renderer {
public:
explicit Renderer(RendererContext context);
protected:
// Append a blank line.
Renderer &BlankLine();
// Append a line of source code, left-justified (not indented).
// Use for preprocessors directives ("#include"), namespaces, etc.
Renderer& CodeLine(const std::string& text);
template <typename... Args>
Renderer CodeLine(absl::string_view text, const Args &...args) {
return CodeLine(absl::Substitute(text, args...));
}
// Append a multiline string of source code, left-justified (not indented).
// Note: Trims leading/trailing whitespace including newlines, making this
// method convenient for multiline raw strings.
// Newlines ('\n') are allowed/expected.
Renderer& CodeLines(const std::string& text);
template <typename... Args>
Renderer CodeLines(absl::string_view text, const Args &...args) {
return CodeLines(absl::Substitute(text, args...));
}
// Indent and append a C++ statement.
// Note: do *not* include a trailing semicolon in the statement text.
Renderer& Statement(const std::string& text);
template <typename... Args>
Renderer Statement(absl::string_view text, const Args &...args) {
return Statement(absl::Substitute(text, args...));
}
// Indent and append a call to a TF method returning a Status to check.
// Note: do *not* include a trailing semicolon in the statement text.
Renderer& TFStatement(const std::string& text);
template <typename... Args>
Renderer TFStatement(absl::string_view text, const Args &...args) {
return TFStatement(absl::Substitute(text, args...));
}
// Indent and append a C++ single-line style comment (using '//').
Renderer& CommentLine(const std::string& text = "");
template <typename... Args>
Renderer CommentLine(absl::string_view text, const Args &...args) {
return CommentLine(absl::Substitute(text, args...));
}
// Append a line of code which starts a new block: trailing with '{') and
// indenting.
Renderer& BlockOpen(const std::string& text);
template <typename... Args>
Renderer BlockOpen(absl::string_view text, const Args &...args) {
return BlockOpen(absl::Substitute(text, args...));
}
// Append a line of code ending a block: unindenting and adding '}'.
// Note: optional trailing text is often a comment, e.g. '// namespace xyz'.
Renderer& BlockClose(const std::string& text = "");
template <typename... Args>
Renderer BlockClose(absl::string_view text, const Args &...args) {
return BlockClose(absl::Substitute(text, args...));
}
protected:
RendererContext context_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h
+39
View File
@@ -0,0 +1,39 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_CONTEXT_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_CONTEXT_H_
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
namespace tensorflow {
namespace generator {
namespace cpp {
struct RendererContext {
enum Mode { kHeader = 0, kSource };
Mode mode;
SourceCode &code;
CppConfig cpp_config;
PathConfig path_config;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_RENDERERS_RENDERER_CONTEXT_H_
tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_test.cc
+84
View File
@@ -0,0 +1,84 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer.h"
#include <string>
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/common/source_code.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/renderer_context.h"
#include "tensorflow/core/platform/test.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
namespace {
TEST(Renderer, typical_usage) {
class TestRenderer : Renderer {
public:
explicit TestRenderer(SourceCode& code)
: Renderer(
{RendererContext::kSource, code, CppConfig(), PathConfig()}) {}
void Render() {
CommentLine("File level comment.");
CodeLine("#include \"header.h\"");
BlankLine();
BlockOpen("void TestFunction()");
{
Statement("int i = 1");
BlankLine();
BlockOpen("while (i == 1)");
{
CommentLine("Do nothing, really....");
CodeLine("#if 0");
Statement("call()");
CodeLine("#endif");
BlockClose();
}
BlockClose(" // comment ending TestFunction");
}
}
};
SourceCode code;
TestRenderer(code).Render();
std::string expected = R"(// File level comment.
#include "header.h"
void TestFunction() {
int i = 1;
while (i == 1) {
// Do nothing, really....
#if 0
call();
#endif
}
} // comment ending TestFunction
)";
code.SetSpacesPerIndent(3);
EXPECT_EQ(expected, code.Render());
}
} // namespace
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/BUILD
+31
View File
@@ -0,0 +1,31 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//visibility:private"],
licenses = ["notice"],
)
cc_library(
name = "views",
srcs = glob(
["*.cc"],
exclude = ["*_test.cc"],
),
hdrs = glob(["*.h"]),
visibility = [
"//tensorflow/c/experimental/ops/gen/cpp:__pkg__",
"//tensorflow/c/experimental/ops/gen/cpp/renderers:__pkg__",
],
deps = [
"//tensorflow/c/experimental/ops/gen/common",
"//tensorflow/c/experimental/ops/gen/model",
"//tensorflow/core:lib",
"//tensorflow/core:lib_internal",
"//tensorflow/core:protos_all_cc",
"@com_google_absl//absl/log",
"@com_google_absl//absl/log:check",
"@com_google_absl//absl/strings",
],
alwayslink = 1,
)
tensorflow/c/experimental/ops/gen/cpp/views/arg_type_view.cc
+47
View File
@@ -0,0 +1,47 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_type_view.h"
#include <string>
#include "tensorflow/c/experimental/ops/gen/model/arg_type.h"
#include "tensorflow/core/framework/types.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
ArgTypeView::ArgTypeView(ArgType arg_type) : arg_type_(arg_type) {}
std::string ArgTypeView::TypeName() const {
if (arg_type_.is_read_only()) {
if (arg_type_.is_list()) {
return "absl::Span<AbstractTensorHandle* const>";
} else {
return "AbstractTensorHandle* const";
}
} else {
if (arg_type_.is_list()) {
return "absl::Span<AbstractTensorHandle*>";
} else {
return "AbstractTensorHandle**";
}
}
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/arg_type_view.h
+39
View File
@@ -0,0 +1,39 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_TYPE_VIEW_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_TYPE_VIEW_H_
#include "tensorflow/c/experimental/ops/gen/model/arg_type.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class ArgTypeView {
public:
explicit ArgTypeView(ArgType arg_type);
std::string TypeName() const;
private:
ArgType arg_type_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_TYPE_VIEW_H_
tensorflow/c/experimental/ops/gen/cpp/views/arg_view.cc
+49
View File
@@ -0,0 +1,49 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h"
#include <string>
#include <vector>
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
ArgView::ArgView(ArgSpec arg) : arg_(arg) {}
std::string ArgView::VariableName() const { return arg_.name(); }
std::string ArgView::SetterMethod() const {
if (IsList()) {
return "AddInputList";
} else {
return "AddInput";
}
}
std::vector<std::string> ArgView::SetterArgs() const {
return {VariableName()};
}
bool ArgView::IsList() const { return arg_.arg_type().is_list(); }
int ArgView::Position() const { return arg_.position(); }
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h
+47
View File
@@ -0,0 +1,47 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_VIEW_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_VIEW_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_type_view.h"
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class ArgView {
public:
explicit ArgView(ArgSpec arg);
std::string VariableName() const;
std::string SetterMethod() const;
std::vector<std::string> SetterArgs() const;
int Position() const;
bool IsList() const;
private:
ArgSpec arg_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ARG_VIEW_H_
tensorflow/c/experimental/ops/gen/cpp/views/attr_view.cc
+137
View File
@@ -0,0 +1,137 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h"
#include <string>
#include <vector>
#include "absl/log/log.h"
#include "absl/strings/str_cat.h"
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/common/case_format.h"
#include "tensorflow/c/experimental/ops/gen/common/view_util.h"
#include "tensorflow/core/framework/attr_value.pb.h"
#include "tensorflow/core/framework/types.pb.h"
namespace tensorflow {
namespace generator {
namespace cpp {
std::string AttrView::VariableName() const { return attr_.name(); }
std::string AttrView::VariableType() const {
// Completely special cases (e.g. strings are different when lists)
if (attr_.full_type() == "string") {
return "const char*";
}
if (attr_.full_type() == "list(string)") {
return "absl::Span<std::string const>";
}
// Normal path: translate base type to C++ ...
std::string c_base_type = attr_.base_type();
if (attr_.base_type() == "type") {
c_base_type = "DataType";
} else if (attr_.base_type() == "shape") {
c_base_type = "const PartialTensorShape";
}
// ... and wrap in a Span<> if it's a list.
if (attr_.is_list()) {
return absl::Substitute("absl::Span<$0>", c_base_type);
} else {
return c_base_type;
}
return attr_.full_type();
}
std::string AttrView::AttrNameString() const { return Quoted(attr_.name()); }
std::string AttrView::DefaultValue() const {
const AttrValue &attr_value = attr_.default_value();
switch (attr_value.value_case()) {
case AttrValue::VALUE_NOT_SET:
return "";
case AttrValue::kType:
return DataType_Name(attr_value.type());
case AttrValue::kS:
return "\"" + attr_value.s() + "\"";
case AttrValue::kI:
return std::to_string(attr_value.i());
case AttrValue::kF:
return std::to_string(attr_value.f());
case AttrValue::kB:
return attr_value.b() ? "true" : "false";
case AttrValue::kList:
if (attr_.full_type() == "list(string)" &&
attr_value.list().s_size() == 0) {
return "{}";
}
LOG(WARNING) << "Unimplemented: default value of list-typed attribute.";
return "/* UNIMPLEMENTED */";
case AttrValue::kShape:
case AttrValue::kTensor:
case AttrValue::kFunc:
case AttrValue::kPlaceholder:
LOG(ERROR) << "Unexpected non-primitive attribute value.";
return "/* ERROR */";
}
}
std::string AttrView::VariableStrLen() const {
return Call("strlen", {VariableName()});
}
std::string AttrView::VariableSpanData() const {
return Call(VariableName(), "data", {}, ".");
}
std::string AttrView::VariableSpanLen() const {
return Call(VariableName(), "length", {}, ".");
}
std::string AttrView::InputArg(bool with_default_value) const {
std::string default_value = DefaultValue();
if (!with_default_value || default_value.empty()) {
return absl::Substitute("$0 $1", VariableType(), attr_.name());
}
return absl::Substitute("$0 $1 = $2", VariableType(), attr_.name(),
default_value);
}
std::string AttrView::SetterMethod() const {
if (!attr_.is_list()) {
return absl::StrCat("SetAttr", toUpperCamel(attr_.full_type()));
} else {
return absl::StrCat("SetAttr", toUpperCamel(attr_.base_type()), "List");
}
}
std::vector<std::string> AttrView::SetterArgs() const {
if (attr_.full_type() == "string") {
return {AttrNameString(), VariableName(), VariableStrLen()};
} else if (attr_.full_type() == "list(string)") {
return {AttrNameString(), VariableName()}; // accepts span directly
} else if (attr_.is_list()) {
return {AttrNameString(), VariableSpanData(), VariableSpanLen()};
} else {
return {AttrNameString(), VariableName()};
}
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h
+50
View File
@@ -0,0 +1,50 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ATTR_VIEW_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ATTR_VIEW_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class AttrView {
public:
explicit AttrView(AttrSpec attr) : attr_(attr) {}
std::string VariableName() const;
std::string VariableType() const;
std::string AttrNameString() const;
std::string VariableStrLen() const;
std::string VariableSpanData() const;
std::string VariableSpanLen() const;
std::string DefaultValue() const;
std::string InputArg(bool with_default_value) const;
std::string SetterMethod() const;
std::vector<std::string> SetterArgs() const;
private:
AttrSpec attr_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_ATTR_VIEW_H_
tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.cc
+58
View File
@@ -0,0 +1,58 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h"
#include <string>
#include "absl/strings/substitute.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_type_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h"
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
namespace tensorflow {
namespace generator {
namespace cpp {
std::string OpArgumentView::Declaration(bool is_header) const {
return absl::Substitute("$0 $1", type_name_, variable_name_);
}
std::string OpArgumentView::Initializer() const {
if (default_value_.empty()) {
return "";
}
return absl::Substitute(" = $0", default_value_);
}
bool OpArgumentView::HasDefaultValue() const { return !default_value_.empty(); }
OpArgumentView::OpArgumentView(std::string type, std::string var,
std::string def)
: type_name_(type), variable_name_(var), default_value_(def) {}
OpArgumentView::OpArgumentView(ArgSpec arg)
: type_name_(ArgTypeView(arg.arg_type()).TypeName()),
variable_name_(ArgView(arg).VariableName()) {}
OpArgumentView::OpArgumentView(AttrSpec attr)
: type_name_(AttrView(attr).VariableType()),
variable_name_(AttrView(attr).VariableName()),
default_value_(AttrView(attr).DefaultValue()) {}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h
+47
View File
@@ -0,0 +1,47 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_ARGUMENT_VIEW_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_ARGUMENT_VIEW_H_
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class OpArgumentView {
public:
explicit OpArgumentView(ArgSpec arg);
explicit OpArgumentView(AttrSpec attr);
explicit OpArgumentView(std::string type, std::string var,
std::string def = "");
std::string Declaration(bool is_header = false) const;
std::string Initializer() const;
bool HasDefaultValue() const;
private:
std::string type_name_;
std::string variable_name_;
std::string default_value_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_ARGUMENT_VIEW_H_
tensorflow/c/experimental/ops/gen/cpp/views/op_view.cc
+99
View File
@@ -0,0 +1,99 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_view.h"
#include <string>
#include <vector>
#include "absl/log/check.h"
#include "tensorflow/c/experimental/ops/gen/common/view_util.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h"
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include "tensorflow/core/lib/strings/str_util.h"
#include "tensorflow/core/platform/logging.h"
namespace tensorflow {
namespace generator {
namespace cpp {
OpView::OpView(OpSpec op)
: op_(op),
input_args_(op_.Inputs().begin(), op_.Inputs().end()),
output_args_(op_.Outputs().begin(), op_.Outputs().end()),
argument_attrs_(op_.Attributes().begin(), op_.Attributes().end()) {
// Initialize function arguments
all_arguments_.push_back(OpArgumentView("AbstractContext*", "ctx"));
for (const auto& arg : op_.Inputs()) {
all_arguments_.push_back(OpArgumentView(arg));
}
for (const auto& arg : op_.Outputs()) {
all_arguments_.push_back(OpArgumentView(arg));
}
for (const auto& attr : op.Attributes()) {
all_arguments_.push_back(OpArgumentView(attr));
}
all_arguments_.push_back(OpArgumentView("const char*", "name", "nullptr"));
all_arguments_.push_back(
OpArgumentView("const char*", "raw_device_name", "nullptr"));
}
const std::vector<ArgView>& OpView::Inputs() const { return input_args_; }
const std::vector<ArgView>& OpView::Outputs() const { return output_args_; }
const std::vector<AttrView>& OpView::Attributes() const {
return argument_attrs_;
}
const std::vector<OpArgumentView>& OpView::AllArguments() const {
return all_arguments_;
}
int OpView::NumInputs() const { return input_args_.size(); }
int OpView::NumOutputs() const { return output_args_.size(); }
ArgView OpView::OnlyInput() const {
CHECK_EQ(input_args_.size(), 1); // Crash OK
return input_args_.front();
}
ArgView OpView::OnlyOutput() const {
CHECK_EQ(output_args_.size(), 1); // Crash OK
return output_args_.front();
}
std::string OpView::FunctionName() const { return op_.name(); }
std::string OpView::OpNameString() const { return Quoted(op_.name()); }
std::string OpView::VariableName() const { return "op_ptr"; }
std::vector<std::string> OpView::Description() const {
return str_util::Split(op_.description(), "\n");
}
std::string OpView::Summary() const { return op_.summary(); }
// Context
bool OpView::IsListOp() const {
return NumOutputs() == 1 && OnlyOutput().IsList();
}
} // namespace cpp
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/cpp/views/op_view.h
+63
View File
@@ -0,0 +1,63 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_VIEW_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_VIEW_H_
#include <vector>
#include "tensorflow/c/experimental/ops/gen/cpp/views/arg_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/attr_view.h"
#include "tensorflow/c/experimental/ops/gen/cpp/views/op_argument_view.h"
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
namespace cpp {
class OpView {
public:
explicit OpView(OpSpec op);
const std::vector<ArgView> &Inputs() const;
const std::vector<ArgView> &Outputs() const;
const std::vector<AttrView> &Attributes() const;
const std::vector<OpArgumentView> &AllArguments() const;
int NumInputs() const;
int NumOutputs() const;
ArgView OnlyInput() const;
ArgView OnlyOutput() const;
std::string FunctionName() const;
std::string VariableName() const;
std::string OpNameString() const;
std::string Summary() const;
std::vector<std::string> Description() const;
bool IsListOp() const;
private:
OpSpec op_;
std::vector<ArgView> input_args_;
std::vector<ArgView> output_args_;
std::vector<AttrView> argument_attrs_;
std::vector<OpArgumentView> all_arguments_;
};
} // namespace cpp
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_CPP_VIEWS_OP_VIEW_H_
tensorflow/c/experimental/ops/gen/generate_cpp_main.cc
+107
View File
@@ -0,0 +1,107 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <vector>
#include "absl/log/check.h"
#include "absl/log/log.h"
#include "absl/strings/str_cat.h"
#include "tensorflow/c/experimental/ops/gen/common/path_config.h"
#include "tensorflow/c/experimental/ops/gen/cpp/cpp_generator.h"
#include "tensorflow/c/experimental/ops/gen/cpp/renderers/cpp_config.h"
#include "tensorflow/core/platform/init_main.h"
#include "tensorflow/core/platform/logging.h"
#include "tensorflow/core/util/command_line_flags.h"
using tensorflow::string;
namespace generator = tensorflow::generator;
namespace {
class MainConfig {
public:
void InitMain(int* argc, char*** argv) {
std::vector<tensorflow::Flag> flags = Flags();
// Parse known flags
string usage = tensorflow::Flags::Usage(
absl::StrCat(*argv[0], " Op1 [Op2 ...]"), flags);
QCHECK(tensorflow::Flags::Parse(argc, *argv, flags)) << usage; // Crash OK
// Initialize any TensorFlow support, parsing boilerplate flags (e.g. logs)
tensorflow::port::InitMain(usage.c_str(), argc, argv);
// Validate flags
if (help_) {
LOG(QFATAL) << usage; // Crash OK
}
QCHECK(!source_dir_.empty()) << usage; // Crash OK
QCHECK(!output_dir_.empty()) << usage; // Crash OK
QCHECK(!category_.empty()) << usage; // Crash OK
// Remaining arguments (i.e. the positional args) are the requested Op names
op_names_.assign((*argv) + 1, (*argv) + (*argc));
}
generator::cpp::CppConfig CppConfig() {
return generator::cpp::CppConfig(category_);
}
generator::PathConfig PathConfig() {
return generator::PathConfig(output_dir_, source_dir_, api_dirs_,
op_names_);
}
private:
std::vector<tensorflow::Flag> Flags() {
return {
tensorflow::Flag("help", &help_, "Print this help message."),
tensorflow::Flag("category", &category_,
"Category for generated ops (e.g. 'math', 'array')."),
tensorflow::Flag(
"namespace", &name_space_,
"Compact C++ namespace, default is 'tensorflow::ops'."),
tensorflow::Flag(
"output_dir", &output_dir_,
"Directory into which output files will be generated."),
tensorflow::Flag(
"source_dir", &source_dir_,
"The tensorflow root directory, e.g. 'tensorflow/' for "
"in-source include paths. Any path underneath the "
"tensorflow root is also accepted."),
tensorflow::Flag(
"api_dirs", &api_dirs_,
"Comma-separated list of directories containing API definitions.")};
}
bool help_ = false;
string category_;
string name_space_;
string output_dir_;
string source_dir_;
string api_dirs_;
std::vector<string> op_names_;
};
} // namespace
int main(int argc, char* argv[]) {
MainConfig config;
config.InitMain(&argc, &argv);
generator::CppGenerator generator(config.CppConfig(), config.PathConfig());
generator.WriteHeaderFile();
generator.WriteSourceFile();
return 0;
}
tensorflow/c/experimental/ops/gen/model/BUILD
+22
View File
@@ -0,0 +1,22 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
package(
# copybara:uncomment default_applicable_licenses = ["//tensorflow:license"],
default_visibility = ["//tensorflow/c/experimental/ops/gen:__subpackages__"],
licenses = ["notice"],
)
cc_library(
name = "model",
srcs = glob(["*.cc"]),
hdrs = glob(["*.h"]),
deps = [
"//tensorflow/core:framework_internal",
"//tensorflow/core:lib",
"//tensorflow/core:lib_internal",
"//tensorflow/core:protos_all_cc",
"@com_google_absl//absl/container:flat_hash_set",
"@com_google_absl//absl/strings",
],
alwayslink = 1,
)
tensorflow/c/experimental/ops/gen/model/arg_spec.cc
+42
View File
@@ -0,0 +1,42 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/c/experimental/ops/gen/model/arg_type.h"
#include "tensorflow/core/framework/op_def.pb.h"
namespace tensorflow {
namespace generator {
ArgSpec::ArgSpec(const OpDef::ArgDef& arg_def, ArgType arg_type, int position)
: name_(arg_def.name()),
description_(arg_def.description()),
arg_type_(arg_type),
position_(position) {}
ArgSpec ArgSpec::CreateInput(const OpDef::ArgDef& arg_def, int position) {
if (arg_def.is_ref()) {
return ArgSpec(arg_def, ArgType::CreateInputRef(arg_def), position);
} else {
return ArgSpec(arg_def, ArgType::CreateInput(arg_def), position);
}
}
ArgSpec ArgSpec::CreateOutput(const OpDef::ArgDef& arg_def, int position) {
return ArgSpec(arg_def, ArgType::CreateOutput(arg_def), position);
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/model/arg_spec.h
+53
View File
@@ -0,0 +1,53 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_SPEC_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_SPEC_H_
#include "tensorflow/c/experimental/ops/gen/model/arg_type.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
// An input or output argument to an Op.
//
// Essentially, this represents an OpDef::ArgDef and its context within the Op.
class ArgSpec {
public:
ArgSpec() = default;
ArgSpec(const ArgSpec& other) = default;
static ArgSpec CreateInput(const OpDef::ArgDef& arg_def, int position);
static ArgSpec CreateOutput(const OpDef::ArgDef& arg_def, int position);
const std::string& name() const { return name_; }
const std::string& description() const { return description_; }
const ArgType arg_type() const { return arg_type_; }
const int position() const { return position_; }
private:
explicit ArgSpec(const OpDef::ArgDef& arg_def, ArgType arg_type,
int position);
std::string name_;
std::string description_;
ArgType arg_type_;
int position_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_SPEC_H_
tensorflow/c/experimental/ops/gen/model/arg_type.cc
+47
View File
@@ -0,0 +1,47 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/model/arg_type.h"
#include "tensorflow/core/framework/op_def.pb.h"
namespace tensorflow {
namespace generator {
ArgType ArgType::CreateInput(const OpDef::ArgDef& arg_def) {
return ArgType(arg_def, kInput);
}
ArgType ArgType::CreateInputRef(const OpDef::ArgDef& arg_def) {
return ArgType(arg_def, kInputRef);
}
ArgType ArgType::CreateOutput(const OpDef::ArgDef& arg_def) {
return ArgType(arg_def, kOutput);
}
ArgType::ArgType(const OpDef::ArgDef& arg_def, Kind kind)
: kind_(kind), data_type_(arg_def.type()) {
if (!arg_def.type_attr().empty()) {
type_attr_name_ = arg_def.type_attr();
}
if (!arg_def.type_list_attr().empty()) {
type_attr_name_ = arg_def.type_list_attr();
}
is_list_ =
!arg_def.type_list_attr().empty() || !arg_def.number_attr().empty();
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/model/arg_type.h
+55
View File
@@ -0,0 +1,55 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_TYPE_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_TYPE_H_
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
// Type information of an Op argument (ArgSpec)..
//
// This represents the type information with OpDef::ArgDef and any type-related
// context.
class ArgType {
public:
ArgType() = default;
ArgType(const ArgType& other) = default;
static ArgType CreateInput(const OpDef::ArgDef& arg_def);
static ArgType CreateInputRef(const OpDef::ArgDef& arg_def);
static ArgType CreateOutput(const OpDef::ArgDef& arg_def);
const tensorflow::DataType data_type() const { return data_type_; }
const std::string type_attr_name() const { return type_attr_name_; }
const bool is_read_only() const { return kind_ == kInput; }
const bool is_list() const { return is_list_; }
private:
enum Kind { kInput = 0, kInputRef, kOutput };
explicit ArgType(const OpDef::ArgDef& arg_def, Kind kind);
Kind kind_;
tensorflow::DataType data_type_;
std::string type_attr_name_;
bool is_list_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ARG_TYPE_H_
tensorflow/c/experimental/ops/gen/model/attr_spec.cc
+43
View File
@@ -0,0 +1,43 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
#include "absl/strings/match.h"
#include "tensorflow/core/framework/op_def.pb.h"
namespace tensorflow {
namespace generator {
AttrSpec AttrSpec::Create(const OpDef::AttrDef& attr_def) {
return AttrSpec(attr_def);
}
AttrSpec::AttrSpec(const OpDef::AttrDef& attr_def) {
name_ = attr_def.name();
description_ = attr_def.description();
full_type_ = attr_def.type();
default_value_ = attr_def.default_value();
if (absl::StartsWith(full_type_, "list(")) {
is_list_ = true;
// strip surrounding "list(%s)"
base_type_ = full_type_.substr(5, full_type_.length() - 6);
} else {
is_list_ = false;
base_type_ = full_type_;
}
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/model/attr_spec.h
+55
View File
@@ -0,0 +1,55 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ATTR_SPEC_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ATTR_SPEC_H_
#include "tensorflow/core/framework/attr_value.pb.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
// An attribute for an Op, such as an input/output type or for passing options.
//
// Essentially, this represents an OpDef::AttrDef and its context within the Op.
class AttrSpec {
public:
AttrSpec() = default;
AttrSpec(const AttrSpec& other) = default;
static AttrSpec Create(const OpDef::AttrDef& attr_def);
const std::string& name() const { return name_; }
const std::string& description() const { return description_; }
const std::string& full_type() const { return full_type_; }
const std::string& base_type() const { return base_type_; }
const AttrValue& default_value() const { return default_value_; }
const bool is_list() const { return is_list_; }
private:
explicit AttrSpec(const OpDef::AttrDef& attr_def);
std::string name_;
std::string description_;
std::string full_type_;
std::string base_type_;
AttrValue default_value_;
bool is_list_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_ATTR_SPEC_H_
tensorflow/c/experimental/ops/gen/model/op_spec.cc
+66
View File
@@ -0,0 +1,66 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/experimental/ops/gen/model/op_spec.h"
#include <string>
#include "absl/container/flat_hash_set.h"
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
#include "tensorflow/core/framework/api_def.pb.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
OpSpec OpSpec::Create(const OpDef& op_def, const ApiDef& api_def) {
return OpSpec(op_def, api_def);
}
OpSpec::OpSpec(const OpDef& op_def, const ApiDef& api_def)
: name_(op_def.name()),
summary_(api_def.summary()),
description_(api_def.description()) {
absl::flat_hash_set<std::string> inferred_attrs;
// Parse the arguments
for (const OpDef::ArgDef& arg_def : op_def.input_arg()) {
ArgSpec arg = ArgSpec::CreateInput(arg_def, input_args_.size());
input_args_.push_back(arg);
if (!arg_def.type_attr().empty()) {
inferred_attrs.insert(arg_def.type_attr());
if (!arg_def.number_attr().empty()) {
inferred_attrs.insert(arg_def.number_attr());
}
} else if (!arg_def.type_list_attr().empty()) {
inferred_attrs.insert(arg_def.type_list_attr());
}
}
for (const OpDef::ArgDef& arg_def : op_def.output_arg()) {
ArgSpec arg = ArgSpec::CreateOutput(arg_def, output_args_.size());
output_args_.push_back(arg);
}
// Parse the attributes.
for (const OpDef::AttrDef& attr_def : op_def.attr()) {
AttrSpec attr = AttrSpec::Create(attr_def);
// Only non-inferred args are added as arguments.
if (inferred_attrs.find(attr_def.name()) == inferred_attrs.end()) {
argument_attrs_.push_back(attr);
}
}
}
} // namespace generator
} // namespace tensorflow
tensorflow/c/experimental/ops/gen/model/op_spec.h
+60
View File
@@ -0,0 +1,60 @@
/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#ifndef TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_OP_SPEC_H_
#define TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_OP_SPEC_H_
#include <map>
#include <vector>
#include "tensorflow/c/experimental/ops/gen/model/arg_spec.h"
#include "tensorflow/c/experimental/ops/gen/model/attr_spec.h"
#include "tensorflow/core/framework/api_def.pb.h"
#include "tensorflow/core/framework/op_def.pb.h"
#include "tensorflow/core/platform/types.h"
namespace tensorflow {
namespace generator {
// An Op.
//
// Essentially, this represents an OpDef and any necessary context (e.g ApiDef).
class OpSpec {
public:
static OpSpec Create(const OpDef& op_def, const ApiDef& api_def);
const std::string& name() const { return name_; }
const std::string& summary() const { return summary_; }
const std::string& description() const { return description_; }
const std::vector<ArgSpec>& Inputs() const { return input_args_; }
const std::vector<ArgSpec>& Outputs() const { return output_args_; }
const std::vector<AttrSpec>& Attributes() const { return argument_attrs_; }
private:
explicit OpSpec(const OpDef& op_def, const ApiDef& api_def);
private:
std::string name_;
std::string summary_;
std::string description_;
std::vector<ArgSpec> input_args_;
std::vector<ArgSpec> output_args_;
std::vector<AttrSpec> argument_attrs_;
std::map<std::string, AttrSpec> type_attrs_;
};
} // namespace generator
} // namespace tensorflow
#endif // TENSORFLOW_C_EXPERIMENTAL_OPS_GEN_MODEL_OP_SPEC_H_