wehub/apache--brpc
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:29:29 +08:00
commit f52cf32e56
1718 changed files with 366293 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/en/atomic_instructions.md
+107
View File
@@ -0,0 +1,107 @@
[中文版](../cn/atomic_instructions.md)
We know that locks are extensively used in multi-threaded programming to avoid [race conditions](http://en.wikipedia.org/wiki/Race_condition) when modifying shared data. When the lock becomes a bottleneck, we try to walk around it by using atomic instructions. But it is difficult to write correct code with atomic instructions in generally and it is even hard to understand race conditions, [ABA problems](https://en.wikipedia.org/wiki/ABA_problem) and [memory fences](https://en.wikipedia.org/wiki/Memory_barrier). This article tries to introduce basics on atomic instructions(under [SMP](http://en.wikipedia.org/wiki/Symmetric_multiprocessing)). Since [Atomic instructions](http://en.cppreference.com/w/cpp/atomic/atomic) are formally introduced in C++11, we use the APIs directly.
As the name implies, atomic instructions cannot be divided into more sub-instructions. For example, `x.fetch(n)` atomically adds n to x, any internal state is not observable **to software**. Common atomic instructions are listed below:
| Atomic Instructions(type of x is std::atomic\<int\>) | Descriptions |
| ---------------------------------------- | ---------------------------------------- |
| x.load() | return the value of x. |
| x.store(n) | store n to x and return nothing. |
| x.exchange(n) | set x to n and return the value just before the modification |
| x.compare_exchange_strong(expected_ref, desired) | If x is equal to expected_ref, set x to desired and return true. Otherwise write current x to expected_ref and return false. |
| x.compare_exchange_weak(expected_ref, desired) | may have [spurious wakeup](http://en.wikipedia.org/wiki/Spurious_wakeup) comparing to compare_exchange_strong |
| x.fetch_add(n), x.fetch_sub(n) | do x += n, x-= n atomically. Return the value just before the modification. |
You can already use these instructions to count something atomically, such as counting number of operations from multiple threads. However two problems may arise:
- The operation is not as fast as expected.
- Even if multi-threaded accesses to some resources are controlled by a few atomic instructions that seem to be correct, the program still has great chance to crash.
# Cacheline
An atomic instruction is fast when there's no contentions or it's accessed only by one thread. "Contentions" happen when multiple threads access the same [cacheline](https://en.wikipedia.org/wiki/CPU_cache#Cache_entries). Modern CPU extensively use caches and divide caches into multiple levels to get high performance with a low price. The Intel E5-2620 widely used in Baidu has 32K L1 dcache and icache, 256K L2 cache and 15M L3 cache. L1 and L2 cache is owned by each core, while L3 cache is shared by all cores. Although it is very fast for one core to write data into its own L1 cache(4 cycles, ~2ns), make the data in L1 cache seen by other cores is not, because cachelines touched by the data need to be synchronized to other cores. This process is atomic and transparent to software and no instructions can be interleaved between. Applications have to wait for the completion of [cache coherence](https://en.wikipedia.org/wiki/Cache_coherence), which takes much longer time than writing local cache. It involves a complicated hardware algorithm and makes atomic instructions slow under high contentions. A single fetch_add may take more than 700ns in E5-2620 when a few threads are highly contented on the instruction. Accesses to the memory frequently shared and modified by multiple threads are not fast generally. For example, even if a critical section looks small, the spinlock protecting it may still not perform well. The cause is that the instructions used in spinlock such as exchange, fetch_add etc, need to wait for latest cachelines. It's not surprising to see that one or two instructions take several microseconds.
In order to improve performance, we need to avoid frequently synchronizing cachelines, which not only affects performance of the atomic instruction itself, but also overall performance of the program. The most effective solution is straightforward: **avoid sharing as much as possible**.
- A program relying on a global multiple-producer-multiple-consumer(MPMC) queue is hard to scale well on many CPU cores, because throughput of the queue is limited by delays of cache coherence, rather than number of cores. It would be better to use multiple SPMC or MPSC queues, or even SPSC queues instead, to avoid contentions from the very beginning.
- Another example is counters. If all threads modify a counter frequently, the performance will be poor because all cores are busy synchronizing the same cacheline. If the counter is only used for printing logs periodically or something low-priority like that, we can let each thread modify its own thread-local variables and combine all thread-local data before reading, yielding [much better performance](bvar.md).
A related programming trap is false sharing: Accesses to infrequently updated or even read-only variables are significantly slowed down because other variables in the same cacheline are frequently updated. Variables used in multi-threaded environment should be grouped by accessing frequencies or patterns, variables that are modified by that other threads frequently should be put into separated cachelines. To align a variable or struct by cacheline, `include <butil/macros.h>` and tag it with macro `BAIDU_CACHELINE_ALIGNMENT`, grep source code of brpc for examples.
# Memory fence
Just atomic counting cannot synchronize accesses to resources, simple structures like [spinlock](https://en.wikipedia.org/wiki/Spinlock) or [reference counting](https://en.wikipedia.org/wiki/Reference_counting) that seem correct may crash as well. The key is **instruction reordering**, which may change the order of read/write and cause instructions behind to be reordered to front if there are no dependencies. [Compiler](http://preshing.com/20120625/memory-ordering-at-compile-time/) and [CPU](https://en.wikipedia.org/wiki/Out-of-order_execution) both may reorder.
The motivation is natural: CPU wants to fill each cycle with instructions and execute as many as possible instructions within given time. As said in above section, an instruction for loading memory may cost hundreds of nanoseconds due to cacheline synchronization. A efficient solution for synchronizing multiple cachelines is to move them simultaneously rather than one-by-one. Thus modifications to multiple variables by a thread may be visible to another thread in a different order. On the other hand, different threads need different data, synchronizing on-demand is reasonable and may also change order between cachelines.
For example: the first variable plays the role of switch, controlling accesses to following variables. When these variables are synchronized to other CPU cores, new values may be visible in a different order, and the first variable may not be the first updated, which causes other threads to think that the following variables are still valid, which are actually not, causing undefined behavior. Check code snippet below:
```c++
// Thread 1
// bool ready was initialized to false
p.init();
ready = true;
```
```c++
// Thread2
if (ready) {
p.bar();
}
```
From a human perspective, the code is correct because thread2 only accesses `p` when `ready` is true which means p is initialized according to the logic in thread1. But the code may not run as expected on multi-core machines:
- `ready = true` in thread1 may be reordered before `p.init()` by compiler or CPU, making thread2 see uninitialized `p` when `ready` is true. The same reordering may happen in thread2 as well. Some instructions in `p.bar()` may be reordered before checking `ready`.
- Even if above reorderings do not happen, cachelines of `ready` and `p` may be synchronized independently to the CPU core that thread2 runs, making thread2 see unitialized `p` when `ready` is true.
Note: On x86/x64, `load` has acquire semantics and `store` has release semantics by default, the code above may run correctly provided that reordering by compiler is turned off.
With this simple example, you may get a glimpse of the complexity of atomic instructions. In order to solve the reordering issue, CPU and compiler offer [memory fences](http://en.wikipedia.org/wiki/Memory_barrier) to let programmers decide the visibility order between instructions. boost and C++11 conclude memory fence into following types:
| memory order | Description |
| -------------------- | ---------------------------------------- |
| memory_order_relaxed | there are no synchronization or ordering constraints imposed on other reads or writes, only this operation's atomicity is guaranteed |
| memory_order_consume | no reads or writes in the current thread dependent on the value currently loaded can be reordered before this load. |
| memory_order_acquire | no reads or writes in the current thread can be reordered before this load. |
| memory_order_release | no reads or writes in the current thread can be reordered after this store. |
| memory_order_acq_rel | No memory reads or writes in the current thread can be reordered before or after this store. |
| memory_order_seq_cst | Any operation with this memory order is both an acquire operation and a release operation, plus a single total order exists in which all threads observe all modifications in the same order. |
Above example can be modified as follows:
```c++
// Thread1
// std::atomic<bool> ready was initialized to false
p.init();
ready.store(true, std::memory_order_release);
```
```c++
// Thread2
if (ready.load(std::memory_order_acquire)) {
p.bar();
}
```
The acquire fence in thread2 matches the release fence in thread1, making thread2 see all memory operations before the release fence in thread1 when thread2 sees `ready` being set to true.
Note that memory fence does not guarantee visibility. Even if thread2 reads `ready` just after thread1 sets it to true, thread2 is not guaranteed to see the new value, because cache synchronization takes time. Memory fence guarantees ordering between visibilities: "If I see the new value of a, I should see the new value of b as well".
A related problem: How to know whether a value is updated or not? Two cases in generally:
- The value is special. In above example, `ready=true` is a special value. Once `ready` is true, `p` is ready. Reading special values or not both mean something.
- Increasing-only values. Some situations do not have special values, we can use instructions like `fetch_add` to increase variables. As long as the value range is large enough, new values are different from old ones for a long period of time, so that we can distinguish them from each other.
More examples can be found in [boost.atomic](http://www.boost.org/doc/libs/1_56_0/doc/html/atomic/usage_examples.html). Official descriptions of atomics can be found [here](http://en.cppreference.com/w/cpp/atomic/atomic).
# wait-free & lock-free
Atomic instructions provide two important properties: [wait-free](http://en.wikipedia.org/wiki/Non-blocking_algorithm#Wait-freedom) and [lock-free](http://en.wikipedia.org/wiki/Non-blocking_algorithm#Lock-freedom). Wait-free means no matter how OS schedules, all threads are doing useful jobs; lock-free, which is weaker than wait-free, means no matter how OS schedules, at least one thread is doing useful jobs. If locks are used, the thread holding the lock might be paused by OS, in which case all threads trying to hold the lock are blocked. So code using locks are neither lock-free nor wait-free. To make tasks done within given time, critical paths in real-time OS is at least lock-free. Miscellaneous online services inside Baidu also pose serious restrictions on running time. If the critical path in brpc is wait-free or lock-free, many services are benefited from better and stable QoS. Actually, both read(in the sense of even dispatching) and write in brpc are wait-free, check [IO](io.md) for details.
Note that it is common to think that wait-free or lock-free algorithms are faster, which may not be true, because:
- More complex race conditions and ABA problems must be handled in lock-free and wait-free algorithms, which means the code is often much more complicated than the one using locks. More code, more running time.
- Mutex solves contentions by backoff, which means that when contention happens, another branch is entered to avoid the contention temporarily. Threads failed to lock a mutex are put into sleep, making the thread holding the mutex complete the task or even follow-up tasks exclusively, which may increase the overall throughput.
Low performance caused by mutex is either because of too large critical sections (which limit the concurrency), or too heavy contentions (overhead of context switches becomes dominating). The real value of lock-free/wait-free algorithms is that they guarantee progress of the system, rather than absolutely high performance. Of course lock-free/wait-free algorithms perform better in some situations: if an algorithm is implemented by just one or two atomic instructions, it's probably faster than the one using mutex which needs more atomic instructions in total.
docs/en/auto_concurrency_limiter.md
+7
View File
@@ -0,0 +1,7 @@
# auto concurrency limiter
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/auto_concurrency_limiter.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/avalanche.md
+5
View File
@@ -0,0 +1,5 @@
"Avalanche" refers to the phenomenon that the vast majority of requests are timed out when accessing a service cluster and can not be recovered when the traffic decreases. Next we explain the source of this phenomenon.
When the number of request exceeds the maximum qps of service, the service will not work properly; when the traffic is back to normal(less than the service processing capacity), the backlog requests will be processed. Although most of them may be timed out due to not being processed timely, the service itself will generally return to normal. This is just like a pool has a water inlet and a water outlet, if the amount of water in is greater than that of water out, the pool will eventually be full and more water will overflow. However, if the amount of water in is less than that of water out, the pool will be eventually empty after a period of time.
docs/en/backup_request.md
+120
View File
@@ -0,0 +1,120 @@
Sometimes in order to ensure availability, we need to visit two services at the same time and get the result coming back first. There are several ways to achieve this in brpc:
# When backend servers can be hung in a naming service
Channel opens backup request. Channel sends the request to one of the servers and when the response is not returned after ChannelOptions.backup_request_ms ms, it sends to another server, taking the response that coming back first. After backup_request_ms is set up properly, in most of times only one request should be sent, causing no extra pressure to back-end services.
Read [example/backup_request_c++](https://github.com/apache/brpc/blob/master/example/backup_request_c++) as example code. In this example, client sends backup request after 2ms and server sleeps for 20ms on purpose when the number of requests is even to trigger backup request.
After running, the log in client and server is as following. "Index" is the number of request. After the server receives the first request, it will sleep for 20ms on purpose. Then the client sends the request with the same index. The final delay is not affected by the intentional sleep.
![img](../images/backup_request_1.png)
![img](../images/backup_request_2.png)
/rpcz also shows that the client triggers backup request after 2ms and sends the second request.
![img](../images/backup_request_3.png)
## Choose proper backup_request_ms
You can look the default cdf(Cumulative Distribution Function) graph of latency provided by brpc, or add it by your own. The y-axis of the cdf graph is a latency(us by default), and the x-axis is the proportion of requests whose latencies are less than the corresponding value in y-aixs. In the following graph, Choosing backup_request_ms=2ms could approximately cover 95.5% of the requests, while choosing backup_request_ms=10ms could cover 99.99% of the requests.
![img](../images/backup_request_4.png)
The way of adding it by yourself:
```c++
#include <bvar/bvar.h>
#include <butil/time.h>
...
bvar::LatencyRecorder my_func_latency("my_func");
...
butil::Timer tm;
tm.start();
my_func();
tm.stop();
my_func_latency << tm.u_elapsed(); // u represents for microsecond, and s_elapsed(), m_elapsed(), n_elapsed() correspond to second, millisecond, nanosecond.
// All work is done here. My_func_qps, my_func_latency, my_func_latency_cdf and many other counters would be shown in /vars.
```
## Rate-limited backup requests
To limit the ratio of backup requests sent, use the built-in factory function or implement the `BackupRequestPolicy` interface yourself.
Priority order: `backup_request_policy` > `backup_request_ms`.
### Using the built-in rate-limiting policy
Call `CreateRateLimitedBackupPolicy` and set the result on `ChannelOptions.backup_request_policy`:
```c++
#include "brpc/backup_request_policy.h"
#include <memory>
brpc::RateLimitedBackupPolicyOptions opts;
opts.backup_request_ms = 10; // send backup if RPC does not complete within 10ms
opts.max_backup_ratio = 0.3; // cap backup requests at 30% of total
opts.window_size_seconds = 10; // sliding window width in seconds
opts.update_interval_seconds = 5; // how often the cached ratio is refreshed
// The caller owns the returned pointer.
// The policy must outlive the channel — destroy the channel before the policy.
std::unique_ptr<brpc::BackupRequestPolicy> policy(
brpc::CreateRateLimitedBackupPolicy(opts));
brpc::ChannelOptions options;
options.backup_request_policy = policy.get(); // NOT owned by channel
channel.Init(..., &options);
// channel must be destroyed before policy goes out of scope.
```
`RateLimitedBackupPolicyOptions` fields:
| Field | Default | Description |
|-------|---------|-------------|
| `backup_request_ms` | -1 | Timeout threshold in ms. -1 means inherit from `ChannelOptions.backup_request_ms` (only works when the policy is set via `ChannelOptions.backup_request_policy`; at controller level there is no channel-level fallback, so set an explicit >= 0 value instead). Must be >= -1. |
| `max_backup_ratio` | 0.1 | Max backup ratio; range (0, 1] |
| `window_size_seconds` | 10 | Sliding window width in seconds; range [1, 3600] |
| `update_interval_seconds` | 5 | Cached-ratio refresh interval in seconds; must be >= 1 |
`CreateRateLimitedBackupPolicy` returns `NULL` if any parameter is invalid.
### Using a custom BackupRequestPolicy
For full control, implement the `BackupRequestPolicy` interface and set it on `ChannelOptions.backup_request_policy`:
```c++
#include "brpc/backup_request_policy.h"
class MyBackupPolicy : public brpc::BackupRequestPolicy {
public:
int32_t GetBackupRequestMs(const brpc::Controller*) const override {
return 10; // send backup after 10ms
}
bool DoBackup(const brpc::Controller*) const override {
return should_allow_backup(); // your logic here
}
void OnRPCEnd(const brpc::Controller*) override {
// called on every RPC completion; update stats if needed
}
};
MyBackupPolicy my_policy;
brpc::ChannelOptions options;
options.backup_request_policy = &my_policy; // NOT owned by channel; must outlive channel
channel.Init(..., &options);
```
### Implementation notes
- The ratio is computed over a sliding time window using bvar counters. The cached value is refreshed at most once per `update_interval_seconds` using a lock-free CAS election, so the overhead per RPC is very low (two atomic loads in the common path).
- Backup decisions are counted immediately at decision time (before the RPC completes) to provide faster feedback during latency spikes. Total RPCs are counted on completion. This means the ratio may transiently lag during a spike, but this is intentional — the limiter is designed for approximate, best-effort throttling, not exact enforcement.
- Each channel using rate limiting maintains two `bvar::Window` sampler tasks. Keep this in mind in deployments with a very large number of channels.
# When backend servers cannot be hung in a naming service
[Recommended] Define a SelectiveChannel that sets backup request, in which contains two sub channel. The visiting process of this SelectiveChannel is similar to the above situation. It will visit one sub channel first. If the response is not returned after channelOptions.backup_request_ms ms, then another sub channel is visited. If a sub channel corresponds to a cluster, this method does backups between two clusters. An example of SelectiveChannel can be found in [example/selective_echo_c++](https://github.com/apache/brpc/tree/master/example/selective_echo_c++). More details please refer to the above program.
[Not Recommended] Issue two asynchronous RPC calls and join them. They cancel each other in their done callback. An example is in [example/cancel_c++](https://github.com/apache/brpc/tree/master/example/cancel_c++). The problem of this method is that the program always sends two requests, doubling the pressure to back-end services. It is uneconomical in any sense and should be avoided as much as possible.
docs/en/baidu_std.md
+7
View File
@@ -0,0 +1,7 @@
# uaidu std
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/baidu_std.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/bazel_support.md
+20
View File
@@ -0,0 +1,20 @@
## bRPC as a Bazel third-party dependency
1. bRPC relies on a number of open source libraries that do not provide bazel support, so you will need to manually add some of these dependencies to your build project.
2. Move the BUILD file /example/build_with_bazel/*.BUILD and brpc_workspace.bzl to the root of your project, and add the contents of
```c++
load("@//:brpc_workspace.bzl", "brpc_workspace")
brpc_workspace();
```
to your WORKSPACE
3. link apache_brpc like:
```c++
...
deps = [
"@apache_brpc//:bthread",
"@apache_brpc//:brpc",
"@apache_brpc//:butil",
"@apache_brpc//:bvar",
]
...
```
docs/en/benchmark.md
+7
View File
@@ -0,0 +1,7 @@
# uenchmark
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/benchmark.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/benchmark_http.md
+7
View File
@@ -0,0 +1,7 @@
# uenchmark http
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/benchmark_http.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/brpc_internal.pptx
BIN
View File
Binary file not shown.
docs/en/bthread.md
+7
View File
@@ -0,0 +1,7 @@
# uthread
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/bthread.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/bthread_id.md
+7
View File
@@ -0,0 +1,7 @@
# uthread id
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/bthread_id.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/bthread_or_not.md
+7
View File
@@ -0,0 +1,7 @@
# uthread or not
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/bthread_or_not.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/bthread_tagged_task_group.md
+7
View File
@@ -0,0 +1,7 @@
# uthread tagged task group
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/bthread_tagged_task_group.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/bthread_tracer.md
+7
View File
@@ -0,0 +1,7 @@
# uthread tracer
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/bthread_tracer.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/builtin_service.md
+59
View File
@@ -0,0 +1,59 @@
[中文版](../cn/builtin_service.md)
# Builtin Services
Builtin services expose internal status of servers in different pespectives, making development and debugging over brpc more efficient. brpc serves builting services via HTTP, which can be easily accessed through curl and web browsers. Servers respond plain text or html according to `User-Agent` in the request header, or you may append `?console=1` to the uri to force the server to respond in plain text. Check the [example](http://brpc.baidu.com:8765/) running on our dev machine(only accessible from Baidu internal) for more details. If the port is forbidden from where you run curl or web browser (e.g. not all ports are accessible from a web browser inside Baidu), you can use [rpc_view](rpc_view.md) for proxying.
Following 2 screenshots show accesses to builtin services from a web browser and a terminal respectively. Note that the logo is the codename inside Baidu, and being modified to brpc in opensourced version.
**From a web browser**
![img](../images/builtin_service_more.png)
**From a terminal**
![img](../images/builtin_service_from_console.png)
# Security Mode
To avoid potential attacks and information leaks, builtin services **must** be hidden on servers that may be accessed from public, including the ones proxied by nginx or other http servers. Click [here](server.md#security-mode) for more details.
# Main services:
[/status](status.md): displays brief status of all services.
[/vars](vars.md): lists user-customizable counters on miscellaneous metrics.
[/connections](../cn/connections.md): lists all connections and their stats.
[/flags](../cn/flags.md): lists all gflags, some of them are modifiable at run-time.
[/rpcz](../cn/rpcz.md): traces all RPCs.
[cpu profiler](../cn/cpu_profiler.md): analyzes CPU hotspots.
[heap profiler](../cn/heap_profiler.md): shows how memory are allocated.
[contention profiler](../cn/contention_profiler.md): analyzes lock contentions.
# Other services
[/version](http://brpc.baidu.com:8765/version) shows version of the server. Call Server::set_version() to specify version of the server, or brpc would generate a default version like `brpc_server_<service-name1>_<service-name2> ...`
![img](../images/version_service.png)
[/health](http://brpc.baidu.com:8765/health) shows whether this server is alive or not.
![img](../images/health_service.png)
[/protobufs](http://brpc.baidu.com:8765/protobufs) shows scheme of all protobuf messages inside the server.
![img](../images/protobufs_service.png)
[/vlog](http://brpc.baidu.com:8765/vlog) shows all the [VLOG](streaming_log.md#VLOG) that can be enabled(not working with glog).
![img](../images/vlog_service.png)
/dir: browses all files on the server, convenient but too dangerous, disabled by default.
/threads: displays information of all threads of the process, hurting performance significantly when being turned on, disabled by default.
docs/en/bvar.md
+81
View File
@@ -0,0 +1,81 @@
[中文版](../cn/bvar.md)
# What is bvar?
[bvar](https://github.com/apache/brpc/tree/master/src/bvar/) is a set of counters to record and view miscellaneous statistics conveniently in multi-threaded applications. The implementation reduces cache bouncing by storing data in thread local storage(TLS), being much faster than UbMonitor(a legacy counting library inside Baidu) and even atomic operations in highly contended scenarios. brpc integrates bvar by default, namely all exposed bvars in a server are accessible through [/vars](http://brpc.baidu.com:8765/vars), and a single bvar is addressable by [/vars/VARNAME](http://brpc.baidu.com:8765/vars/rpc_socket_count). Read [vars](vars.md) to know how to query them in brpc servers. brpc extensively use bvar to expose internal status. If you are looking for an utility to collect and display metrics of your application, consider bvar in the first place. bvar definitely can't replace all counters, essentially it moves contentions occurred during write to read: which needs to combine all data written by all threads and becomes much slower than an ordinary read. If read and write on the counter are both frequent or decisions need to be made based on latest values, you should not use bvar.
To understand how bvar works, read [explaining cacheline](atomic_instructions.md#cacheline) first, in which the mentioned counter example is just bvar. When many threads are modifying a counter, each thread writes into its own area without joining the global contention and all private data are combined at read, which is much slower than an ordinary one, but OK for low-frequency logging or display. The much faster and very-little-overhead write enables users to monitor systems from all aspects without worrying about hurting performance. This is the purpose that we designed bvar.
Following graph compares overhead of bvar, atomics, static UbMonitor, dynamic UbMonitor when they're accessed by multiple threads simultaneously. We can see that overhead of bvar is not related to number of threads basically, and being constantly low (~20 nanoseconds). As a contrast, dynamic UbMonitor costs 7 microseconds on each operation when there're 24 threads, which is the overhead of using the bvar for 300 times.
![img](../images/bvar_perf.png)
# Adding new bvar
Read [Quick introduction](bvar_c++.md#quick-introduction) to know how to add bvar in C++. bvar already provides stats of many process-level and system-level variables by default, which are prefixed with `process_` and `system_`, such as:
```
process_context_switches_involuntary_second : 14
process_context_switches_voluntary_second : 15760
process_cpu_usage : 0.428
process_cpu_usage_system : 0.142
process_cpu_usage_user : 0.286
process_disk_read_bytes_second : 0
process_disk_write_bytes_second : 260902
process_faults_major : 256
process_faults_minor_second : 14
process_memory_resident : 392744960
system_core_count : 12
system_loadavg_15m : 0.040
system_loadavg_1m : 0.000
system_loadavg_5m : 0.020
```
and miscellaneous bvars used by brpc itself:
```
bthread_switch_second : 20422
bthread_timer_scheduled_second : 4
bthread_timer_triggered_second : 4
bthread_timer_usage : 2.64987e-05
bthread_worker_count : 13
bthread_worker_usage : 1.33733
bvar_collector_dump_second : 0
bvar_collector_dump_thread_usage : 0.000307385
bvar_collector_grab_second : 0
bvar_collector_grab_thread_usage : 1.9699e-05
bvar_collector_pending_samples : 0
bvar_dump_interval : 10
bvar_revision : "34975"
bvar_sampler_collector_usage : 0.00106495
iobuf_block_count : 89
iobuf_block_count_hit_tls_threshold : 0
iobuf_block_memory : 729088
iobuf_newbigview_second : 10
```
New exported files overwrite previous files, which is different from regular logs which append new data.
# Monitoring bvar
Turn on [dump feature](bvar_c++.md#export-all-variables) of bvar to export all exposed bvars to files, which are formatted just like above texts: each line is a pair of "name" and "value". Check if there're data under $PWD/monitor/ after enabling dump, for example:
```
$ ls monitor/
bvar.echo_client.data bvar.echo_server.data
$ tail -5 monitor/bvar.echo_client.data
process_swaps : 0
process_time_real : 2580.157680
process_time_system : 0.380942
process_time_user : 0.741887
process_username : "gejun"
```
The monitoring system should combine data on every single machine periodically and merge them together to provide on-demand queries. Take the "noah" system inside Baidu as an example, variables defined by bvar appear as metrics in noah, which can be checked by users to view historical curves.
![img](../images/bvar_noah2.png)
![img](../images/bvar_noah3.png)
# Export to Prometheus
To export to [Prometheus](https://prometheus.io), set the path in scraping target url to `/brpc_metrics`. For example, if brpc server is running on localhost:8080, the scraping target should be `127.0.0.1:8080/brpc_metrics`.
docs/en/bvar_c++.md
+498
View File
@@ -0,0 +1,498 @@
# Quick introduction
The basic usage of bvar is simple:
```c++
#include <bvar/bvar.h>
namespace foo {
namespace bar {
// bvar::Adder<T> used for running sum, we define a Adder for read_error as below
bvar::Adder<int> g_read_error;
// put another bvar inside window so that we can get the value over this period of time
bvar::Window<bvar::Adder<int> > g_read_error_minute("foo_bar", "read_error", &g_read_error, 60);
// ^ ^ ^
// prefix1 monitor name time window, 10 by default
// bvar::LatencyRecorder is a compound varibale, can be used for troughput、qps、avg latency, latency percentile, max latency。
bvar::LatencyRecorder g_write_latency("foo_bar", "write");
// ^ ^
// prefix1 monitor entry, LatencyRecorder includes different bvar, and expose() will add the suffix for them by default, such as write_qps, write_latency etc
// define a varible for the # of 'been-pushed task'
bvar::Adder<int> g_task_pushed("foo_bar", "task_pushed");
// put nested bvar into PerSecond so that we can get the value per second within this time window. Over here what we get is the # of tasks pushed per second
bvar::PerSecond<bvar::Adder<int> > g_task_pushed_second("foo_bar", "task_pushed_second", &g_task_pushed);
// ^ ^
// different from Window, PerSecond will be divided by the time winodw. time window is the last param, we omit here, its 10 by default
} // bar
} // foo
```
how we use the bvar
```c++
// run into read errors
foo::bar::g_read_error << 1;
// record down the latenct, which is 23ms
foo::bar::g_write_latency << 23;
// one task has been pushed
foo::bar::g_task_pushed << 1;
```
Remember Window<> and PerSecond<> are derived variables, we don't have to push value to them and they will auto-update.
Obviously, we can take bvar as local or member variables.
There are essentially 7 commonly used bvar classes, they all extend from the base class bvar:Variable.
- `bvar::Adder<T>` : counter, 0 by default, varname << N equals to varname += N。
- `bvar::Maxer<T>` : get the maximum value, std::numeric_limits::min() by default, varname << N equals to varname = max(varname, N)。
- `bvar::Miner<T>` : get the minimum value, std::numeric_limits::max() by default, varname << N equals to varname = min(varname, N)。
- `bvar::IntRecorder` : get the mean value since it was in use, notice here we don't use “over a period of time”, since this bvar always comes with Window<> to calculate the mean value within the predefined time window.
- `bvar::Window<VAR>` : get the running sum over a time window. Window derives from other existing bvar and will auto-update
- `bvar::PerSecond<VAR>` : get the value per second during a predefined amount of time. PerSecond will also auto-update and derives from other bvar
- `bvar::LatencyRecorder` : intended for recording latency and qps, when we push latency to it, for mean latency/max lantency/qps, we will get all in once.
**caveat: make sure the name of bvar is globally unique, otherwise, the expose() will fail. When the option -bvar_abort_on_same_name is true(false by default), program will abort.**
### Best Practice for Naming:
There are different bvar from different module, to avoid duplicating name, we'd better follow the rule as `module_class_indicator
- a module usually refers to the program name, can be the acronym of product line, like inf_ds, ecom_retrbs etc.
- a class usually refers to the class name/ function name, like storage_manager, file_transfer, rank_stage1.
- an indicator usually refers to qps, count, latency etc.
some legit naming examples
```
iobuf_block_count : 29 # module=iobuf class=block indicator=count
iobuf_block_memory : 237568 # module=iobuf class=block indicator=memory
process_memory_resident : 34709504 # module=process class=memory indicator=resident
process_memory_shared : 6844416 # module=process class=memory indicator=shared
rpc_channel_connection_count : 0 # module=rpc class=channel_connection indicator=count
rpc_controller_count : 1 # module=rpc class=controller indicator=count
rpc_socket_count : 6 # module=rpc class=socket indicator=count
```
bvar will normalize the variable name, no matter whether we type foo::BarNum, foo.bar.num, foo bar num , foo-bar-num, they all go to foo_bar_num in the end.
**Things about indicators:**
- use `_count` as suffix for number, like request_count, error_count
- use `_second` as suffix for number per second is clear enough, no need to use '_count_second' or '_per_second', like request_second, process_inblocks_second
- `_minute` as suffix for number per minute like request_minute, process_inblocks_minute
if we need to use a counter defined in another file, we have to declare that variable in header file
```
namespace foo {
namespace bar {
// notice g_read_error_minute and g_task_pushed_second are derived bvar, will auto update, no need to declare
extern bvar::Adder<int> g_read_error;
extern bvar::LatencyRecorder g_write_latency;
extern bvar::Adder<int> g_task_pushed;
} // bar
} // foo
```
**Don't define golabel Window<> and PerSecond<> across files. The order for the initialization of global variables in different compile units is undefined.** foo.cpp defines `Adder<int> foo_count`, then defining `PerSecond<Adder<int> > foo_qps(&foo_count);` in foo_qps.cpp is illegal
**Things about thread-safety**:
- bvar is thread-compatible. We can manipulate a bvar in different threads, such as we can expose or hide different bvar in multiple threads simultaneously, they will safely do some job on global shared variables.
- **Excpet read and write API,** any other functions of bvar are not thread-safe：u can not expose or hide a same bvar in different threads, this may cause the program crash. Generally speaking, we don't have to call any other API concurrently except read and write.
we can use butil::Timer for timer, API is as below:
```C++
#include <butil/time.h>
namespace butil {
class Timer {
public:
enum TimerType { STARTED };
Timer();
// butil::Timer tm(butil::Timer::STARTED); // tm is already started after creation.
explicit Timer(TimerType);
// Start this timer
void start();
// Stop this timer
void stop();
// Get the elapse from start() to stop().
int64_t n_elapsed() const; // in nanoseconds
int64_t u_elapsed() const; // in microseconds
int64_t m_elapsed() const; // in milliseconds
int64_t s_elapsed() const; // in seconds
};
} // namespace butil
```
# bvar::Variable:
Varibale is the base class for all bvar, it provides registering, listing and searching functions.
When user created a bvar with default params, it hasn't been registered into any global structure, it's merely a faster counter, which means we cannot use it elsewhere. The action of putting this bvar into the global registry is called `expose`, can be achieved by calling `expose()`
The name for globally exposed bvar consists of 'name' or 'name+prefix', can be searched by function with suffix `_exposed` , e.g. Variable::describe_exposed("foo") will return the description of bvar with the name 'foo'.
When there already exists the name, expose() will print FATAL log and return -1. When the option **-bvar_abort_on_same_name** is true(false by default), program will abort.
Some examples for expose() as below
```
bvar::Adder<int> count1; // create a bvar with defalut params
count1 << 10 << 20 << 30; // values add up to 60.
count1.expose("count1"); // expose the variable globally
CHECK_EQ("60", bvar::Variable::describe_exposed("count1"));
count1.expose("another_name_for_count1"); // expose the variable with another name
CHECK_EQ("", bvar::Variable::describe_exposed("count1"));
CHECK_EQ("60", bvar::Variable::describe_exposed("another_name_for_count1"));
bvar::Adder<int> count2("count2"); // exposed in constructor directly
CHECK_EQ("0", bvar::Variable::describe_exposed("count2")); // default value of Adder<int> is 0
bvar::Status<std::string> status1("count2", "hello"); // the name conflicts. if -bvar_abort_on_same_name is true,
// program aborts, otherwise a fatal log is printed.
```
To avoid duplicate name, we should have prefix for bvar, we recommend the name as `<namespace>_<module>_<name>`
For convenience, we provide expose_as() as it will accept a prefix.
```C++
// Expose this variable with a prefix.
// Example:
// namespace foo {
// namespace bar {
// class ApplePie {
// ApplePie() {
// // foo_bar_apple_pie_error
// _error.expose_as("foo_bar_apple_pie", "error");
// }
// private:
// bvar::Adder<int> _error;
// };
// } // foo
// } // bar
int expose_as(const butil::StringPiece& prefix, const butil::StringPiece& name);
```
# Export All Variables
Common needs for exporting are querying by HTTP API and writing into local file, the former is provided by brpc [/vars](https://github.com/apache/brpc/blob/master/docs/cn/vars.md) service, the latter has been implemented in bvar, and it's turned off by default. A couple of methods can activate this function：
- Using [gflags](https://github.com/apache/brpc/blob/master/docs/cn/flags.md) to parse the input params. We can add `-bvar_dump` during the starup of program or we can dynamically change the params thru brpc [/flags](https://github.com/apache/brpc/blob/master/docs/cn/flags.md) service after starup. gflags parsing is as below
```C++
#include <gflags/gflags.h>
...
int main(int argc, char* argv[]) {
if (google::SetCommandLineOption("bvar_dump", "true").empty()) {
LOG(FATAL) << "Fail to enable bvar dump";
}
...
}
```
- If u dont want to use gflags and expect them opened by default in program
```C++
#include <gflags/gflags.h>
...
int main(int argc, char* argv[]) {
if (google::SetCommandLineOption("bvar_dump", "true").empty()) {
LOG(FATAL) << "Fail to enable bvar dump";
}
...
}
```
- dump function is controlled by following gflags
| Name | Default Value | Effect |
| ------------------ | ----------------------- | ---------------------------------------- |
| bvar_dump | false | Create a background thread dumping all bvar periodically, all bvar_dump_* flags are not effective when this flag is off |
| bvar_dump_exclude | "" | Dump bvar excluded from these wildcards(separated by comma), empty means no exclusion |
| bvar_dump_file | monitor/bvar.\<app\>.data | Dump bvar into this file |
| bvar_dump_include | "" | Dump bvar matching these wildcards(separated by comma), empty means including all |
| bvar_dump_interval | 10 | Seconds between consecutive dump |
| bvar_dump_prefix | \<app\> | Every dumped name starts with this prefix |
| bvar_dump_tabs | \<check the code\> | Dump bvar into different tabs according to the filters (separated by semicolon), format: *(tab_name=wildcards) |
when the bvar_dump_file is not empty, a background thread will be started to update `bvar_dump_file` for the specified time interval called `bvar_dump_interval` , including all the bvar which is matched by `bvar_dump_include` while not matched by `bvar_dump_exclude`
such like we modify the gflags as below：
[![img](https://github.com/apache/brpc/raw/master/docs/images/bvar_dump_flags_2.png)](https://github.com/apache/brpc/blob/master/docs/images/bvar_dump_flags_2.png)
exporting file will be like：
```
$ cat bvar.echo_server.data
rpc_server_8002_builtin_service_count : 20
rpc_server_8002_connection_count : 1
rpc_server_8002_nshead_service_adaptor : brpc::policy::NovaServiceAdaptor
rpc_server_8002_service_count : 1
rpc_server_8002_start_time : 2015/07/24-21:08:03
rpc_server_8002_uptime_ms : 14740954
```
`iobuf_block_count : 8` is filtered out by bvar_dump_include, `rpc_server_8002_error : 0` is ruled out by bvar_dump_exclude.
if you didn't use brpc in your program, u also need to dynamically change gflags（normally no need), we can call google::SetCommandLineOption(), as below
```c++
#include <gflags/gflags.h>
...
if (google::SetCommandLineOption("bvar_dump_include", "*service*").empty()) {
LOG(ERROR) << "Fail to set bvar_dump_include";
return -1;
}
LOG(INFO) << "Successfully set bvar_dump_include to *service*";
```
Do not directly set FLAGS_bvar_dump_file / FLAGS_bvar_dump_include / FLAGS_bvar_dump_exclude. On the one hand, these gflags are std::string, which are not thread-safe to be overridden；On the other hand, the validator will not be triggered(call back to check the correctness), so the exporting thread will not be invoked
users can also customize dump_exposed() to export all the exposed bvar：
```c++
// Implement this class to write variables into different places.
// If dump() returns false, Variable::dump_exposed() stops and returns -1.
class Dumper {
public:
virtual bool dump(const std::string& name, const butil::StringPiece& description) = 0;
};
// Options for Variable::dump_exposed().
struct DumpOptions {
// Contructed with default options.
DumpOptions();
// If this is true, string-type values will be quoted.
bool quote_string;
// The ? in wildcards. Wildcards in URL need to use another character
// because ? is reserved.
char question_mark;
// Separator for white_wildcards and black_wildcards.
char wildcard_separator;
// Name matched by these wildcards (or exact names) are kept.
std::string white_wildcards;
// Name matched by these wildcards (or exact names) are skipped.
std::string black_wildcards;
};
class Variable {
...
...
// Find all exposed variables matching `white_wildcards' but
// `black_wildcards' and send them to `dumper'.
// Use default options when `options' is NULL.
// Return number of dumped variables, -1 on error.
static int dump_exposed(Dumper* dumper, const DumpOptions* options);
};
```
# bvar::Reducer
Reducer uses binary operators over several values to combine them into one final result, which are commutative, associative and without side effect. Only satisfying all of these thress, we can make sure the combined result is not affected by the distribution of the private variables of thread. Like substraciton does not satisfy associative nor commutative, so it cannot be taken as the operator here.
```C++
// Reduce multiple values into one with `Op': e1 Op e2 Op e3 ...
// `Op' shall satisfy:
// - associative: a Op (b Op c) == (a Op b) Op c
// - commutative: a Op b == b Op a;
// - no side effects: a Op b never changes if a and b are fixed.
// otherwise the result is undefined.
template <typename T, typename Op>
class Reducer : public Variable;
```
reducer << e1 << e2 << e3 equals to reducer = e1 op e2 op e3。
Common Redcuer subclass: bvar::Adder, bvar::Maxer, bvar::Miner
## bvar::Adder
we can tell from its name, it's intended for running sum. Opeator is `+`
```c++
bvar::Adder<int> value;
value << 1 << 2 << 3 << -4;
CHECK_EQ(2, value.get_value());
bvar::Adder<double> fp_value; // may have warning
fp_value << 1.0 << 2.0 << 3.0 << -4.0;
CHECK_DOUBLE_EQ(2.0, fp_value.get_value());
```
Adder<> can be applied to the non-primitive type, which at least overrides `T operator+(T, T)`, an existing example is `std::string`, the code below will concatenate strings：
```c++
// This is just proof-of-concept, don't use it for production code because it makes a
// bunch of temporary strings which is not efficient, use std::ostringstream instead.
bvar::Adder<std::string> concater;
std::string str1 = "world";
concater << "hello " << str1;
CHECK_EQ("hello world", concater.get_value());
```
## bvar::Maxer
is producing the maximum value, operator is `std::max` 。
```c++
bvar::Maxer<int> value;
value << 1 << 2 << 3 << -4;
CHECK_EQ(3, value.get_value());
```
Since Maxer<> use std::numeric_limits::min() as the identity, it cannot be applied to generic types unless you specialized std::numeric_limits<> (and overloaded operator<, yes, not operator>).
## bvar::Miner
producing minimum value, operator is std::min
```c++
bvar::Miner<int> value;
value << 1 << 2 << 3 << -4;
CHECK_EQ(-4, value.get_value());
```
Since Miner<> use std::numeric_limits::max() as the identity, it cannot be applied to generic types unless you specialized std::numeric_limits<> (and overloaded operator<).
# bvar::IntRecorder
used for mean value
```c++
// For calculating average of numbers.
// Example:
// IntRecorder latency;
// latency << 1 << 3 << 5;
// CHECK_EQ(3, latency.average());
class IntRecorder : public Variable;
```
# bvar::LatencyRecoder
A counter used for latency and qps. We can get latency / max_latency / qps / count as long as the latency data filled in. Time window is the last param, omit by `bvar_dump_interval`
LatencyRecoder is a compound variable, consisting of several bvar.
```c++
LatencyRecorder write_latency("table2_my_table_write"); // produces 4 variables:
// table2_my_table_write_latency
// table2_my_table_write_max_latency
// table2_my_table_write_qps
// table2_my_table_write_count
// In your write function
write_latency << the_latency_of_write;
```
# bvar::Window
Get data within a time window. Window cannot exist alone, it relies on a counter. Window will auto-update, we don't have to send data to it. For the sake of performance, the data comes from every-second sampling over the original counter, in the worst case, Window has one-second latency
```c++
// Get data within a time window.
// The time unit is 1 second fixed.
// Window relies on other bvar which should be constructed before this window and destructs after this window.
// R must:
// - have get_sampler() (not require thread-safe)
// - defined value_type and sampler_type
template <typename R>
class Window : public Variable;
```
# bvar::PerSecond
Get the mean value over the last amount of time. Its almost the same as Window, except for the value will be divided by the time window
```c++
bvar::Adder<int> sum;
// sum_per_second.get_value()is summing every-second value over the last 60 seconds, if we omit the time window, it's set to 'bvar_dump_interval' by default
bvar::PerSecond<bvar::Adder<int> > sum_per_second(&sum, 60);
```
**PerSecond does not always make sense**
There is no Maxer in the above code, since the max value over a period of time divided by the time window is meaningless.
```c++
bvar::Maxer<int> max_value;
// WRONG！max value divided by time window is pointless
bvar::PerSecond<bvar::Maxer<int> > max_value_per_second_wrong(&max_value);
// CORRECT. It's the right way to set the time window to 1s so that we can get the max value for every second
bvar::Window<bvar::Maxer<int> > max_value_per_second(&max_value, 1);
```
## Difference between Window and PerSecond
Suppose we want the memory change since last minute, if we use Window<>, the meaning for the returning value is "the memory increase over the last minute is 18M" if we use PerSecond<>, the meaning for return value will be "the average memory increase per second over the last minute is 0.3M”.
Pros of Window is preciseness, it fits in some small-number cases, like “the number of error produced over last minute“, if we use PerSecond, we might get something like "the average error rate per second over the last minute is 0.0167", which is very unclear as opposed to "one error produced over last minute". Some other non-time-related variables also fit in Window<>, such like calculating the CPU ratio over the last minute is using a Adder by summing CPU time and real time, then we use Window<> on top of the Adder to get the last-mintue CPU time and real time, dividing these two value then we get the CPU ratio for the last minute, which is not time-related. It will get wrong when use PerSeond
# bvar::Status
Record and display one value, has additional set_value() function
```c++
// Display a rarely or periodically updated value.
// Usage:
// bvar::Status<int> foo_count1(17);
// foo_count1.expose("my_value");
//
// bvar::Status<int> foo_count2;
// foo_count2.set_value(17);
//
// bvar::Status<int> foo_count3("my_value", 17);
//
// Notice that Tp needs to be std::string or acceptable by boost::atomic<Tp>.
template <typename Tp>
class Status : public Variable;
```
# bvar::PassiveStatus
Display the value when needed. In some cases, we are not able to actively set_value nor set_value in a certain time interval. We'd better print it out when needed, user can pass in the print-out callback function to achieve this.
```c++
// Display a updated-by-need value. This is done by passing in an user callback
// which is called to produce the value.
// Example:
// int print_number(void* arg) {
// ...
// return 5;
// }
//
// // number1 : 5
// bvar::PassiveStatus status1("number1", print_number, arg);
//
// // foo_number2 : 5
// bvar::PassiveStatus status2(typeid(Foo), "number2", print_number, arg);
template <typename Tp>
class PassiveStatus : public Variable;
even though it looks simple, PassiveStatus is one of the most useful bvar, since most of the statistic values have already existed, we don't have to store it again, just fetch the data according to our need. Declare a bvar which can display user process name as below：
static void get_username(std::ostream& os, void*) {
char buf[32];
if (getlogin_r(buf, sizeof(buf)) == 0) {
buf[sizeof(buf)-1] = '\0';
os << buf;
} else {
os << "unknown";
}
}
PassiveStatus<std::string> g_username("process_username", get_username, NULL);
```
# bvar::GFlag
Expose important gflags as bvar so that they're monitored (in noah).
```c++
DEFINE_int32(my_flag_that_matters, 8, "...");
// Expose the gflag as *same-named* bvar so that it's monitored (in noah).
static bvar::GFlag s_gflag_my_flag_that_matters("my_flag_that_matters");
// ^
// the gflag name
// Expose the gflag as a bvar named "foo_bar_my_flag_that_matters".
static bvar::GFlag s_gflag_my_flag_that_matters_with_prefix("foo_bar", "my_flag_that_matters");
```
# babylon counter bvar
For details on the principles and performance, see [Use concurrent counter optimize bvar](https://github.com/baidu/babylon/tree/main/example/use-counter-with-bvar).
Currently, this feature is only supported by the Bazel compilation method: `--define with_babylon_counter=true`. The required Babylon version is 1.4.4 or higher. Once enabled, you can use the higher-performance bvar implementation based on the babylon counter without modifying your code.
docs/en/case_apicontrol.md
+7
View File
@@ -0,0 +1,7 @@
# case apicontrol
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/case_apicontrol.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/case_baidu_dsp.md
+7
View File
@@ -0,0 +1,7 @@
# case uaidu dsp
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/case_baidu_dsp.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/case_elf.md
+7
View File
@@ -0,0 +1,7 @@
# case elf
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/case_elf.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/case_ubrpc.md
+7
View File
@@ -0,0 +1,7 @@
# case uurpc
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/case_ubrpc.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/circuit_breaker.md
+7
View File
@@ -0,0 +1,7 @@
# circuit ureaker
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/circuit_breaker.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/client.md
+929
View File
@@ -0,0 +1,929 @@
[中文版](../cn/client.md)
# Example
[client-side code](https://github.com/apache/brpc/blob/master/example/echo_c++/client.cpp) of echo.
# Quick facts
- Channel.Init() is not thread-safe.
- Channel.CallMethod() is thread-safe and a Channel can be used by multiple threads simultaneously.
- Channel can be put on stack.
- Channel can be destructed just after sending asynchronous request.
- No class named brpc::Client.
# Channel
Client-side of RPC sends requests. It's called [Channel](https://github.com/apache/brpc/blob/master/src/brpc/channel.h) rather than "Client" in brpc. A channel represents a communication line to one server or multiple servers, which can be used for calling services.
A Channel can be **shared by all threads** in the process. Yon don't need to create separate Channels for each thread, and you don't need to synchronize Channel.CallMethod with lock. However creation and destroying of Channel is **not** thread-safe, make sure the channel is initialized and destroyed only by one thread.
Some RPC implementations have so-called "ClientManager", including configurations and resource management at the client-side, which is not needed by brpc. "thread-num", "connection-type" such parameters are either in brpc::ChannelOptions or global gflags. Advantages of doing so:
1. Convenience. You don't have to pass a "ClientManager" when the Channel is created, and you don't have to store the "ClientManager". Otherwise code has to pass "ClientManager" layer by layer, which is troublesome. gflags makes configurations of global behaviors easier.
2. Share resources. For example, servers and channels in brpc share background workers (of bthread).
3. Better management of Lifetime. Destructing a "ClientManager" is very error-prone, which is managed by brpc right now.
Like most classes, Channel must be **Init()**-ed before usage. Parameters take default values when `options` is NULL. If you want non-default values, code as follows:
```c++
brpc::ChannelOptions options; // including default values
options.xxx = yyy;
...
channel.Init(..., &options);
```
Note that Channel neither modifies `options` nor accesses `options` after completion of Init(), thus options can be put on stack safely as in above code. Channel.options() gets options being used by the Channel.
Init() can connect one server or a cluster(multiple servers).
# Connect to a server
```c++
// Take default values when options is NULL.
int Init(EndPoint server_addr_and_port, const ChannelOptions* options);
int Init(const char* server_addr_and_port, const ChannelOptions* options);
int Init(const char* server_addr, int port, const ChannelOptions* options);
```
The server connected by these Init() has fixed address generally. The creation does not need NamingService or LoadBalancer, being relatively light-weight. The address could be a hostname, but don't frequently create Channels connecting to a hostname, which requires a DNS lookup taking at most 10 seconds. (default timeout of DNS lookup). Reuse them.
Valid "server_addr_and_port":
- 127.0.0.1:80
- www.foo.com:8765
- localhost:9000
Invalid "server_addr_and_port":
- 127.0.0.1:90000 # too large port
- 10.39.2.300:8000 # invalid IP
# Connect to a cluster
```c++
int Init(const char* naming_service_url,
const char* load_balancer_name,
const ChannelOptions* options);
```
Channels created by above Init() get server list from the NamingService specified by `naming_service_url` periodically or driven-by-events, and send request to one server chosen from the list according to the algorithm specified by `load_balancer_name` .
You **should not** create such channels ad-hocly each time before a RPC, because creation and destroying of such channels relate to many resources, say NamingService needs to be accessed once at creation otherwise server candidates are unknown. On the other hand, channels are able to be shared by multiple threads safely and has no need to be created frequently.
If `load_balancer_name` is NULL or empty, this Init() is just the one for connecting single server and `naming_service_url` should be "ip:port" or "host:port" of the server. Thus you can unify initialization of all channels with this Init(). For example, you can put values of `naming_service_url` and `load_balancer_name` in configuration file, and set `load_balancer_name` to empty for single server and a valid algorithm for a cluster.
## Naming Service
Naming service maps a name to a modifiable list of servers. It's positioned as follows at client-side:
![img](../images/ns.png)
With the help of naming service, the client remembers a name instead of every concrete server. When the servers are added or removed, only mapping in the naming service is changed, rather than telling every client that may access the cluster. This process is called "decoupling up and downstreams". Back to implementation details, the client does remember every server and will access NamingService periodically or be pushed with latest server list. The impl. has minimal impact on RPC latencies and very small pressure on the system providing naming service.
General form of `naming_service_url` is "**protocol://service_name**".
### bns://\<bns-name\>
BNS is the most common naming service inside Baidu. In "bns://rdev.matrix.all", "bns" is protocol and "rdev.matrix.all" is service-name. A related gflag is -ns_access_interval: ![img](../images/ns_access_interval.png)
If the list in BNS is non-empty, but Channel says "no servers", the status bit of the machine in BNS is probably non-zero, which means the machine is unavailable and as a correspondence not added as server candidates of the Channel. Status bits can be checked by:
`get_instance_by_service [bns_node_name] -s`
### file://\<path\>
Servers are put in the file specified by `path`. For example, in "file://conf/machine_list", "conf/machine_list" is the file:
* in which each line is address of a server.
* contents after \# are comments and ignored.
* non-comment contents after addresses are tags, which are separated from addresses by one or more spaces, same address + different tags are treated as different instances.
* brpc reloads the file when it's updated.
```
# This line is ignored
10.24.234.17:8080 tag1 # a comment
10.24.234.17:8090 tag2 # an instance different from the instance on last line
10.24.234.18:8080
10.24.234.19:8080
```
Pros: easy to modify, convenient for unittests.
Cons: need to update every client when the list changes, not suitable for online deployment.
### list://\<addr1\>,\<addr2\>...
Servers are directly written after list://, separated by comma. For example: "list://db-bce-81-3-186.db01:7000,m1-bce-44-67-72.m1:7000,cp01-rd-cos-006.cp01:7000" has 3 addresses.
Tags can be appended to addresses, separated with one or more spaces. Same address + different tags are treated as different instances.
Pros: directly configurable in CLI, convenient for unittests.
Cons: cannot be updated at runtime, not suitable for online deployment at all.
### http://\<url\>
Connect all servers under the domain, for example: http://www.baidu.com:80.
Note: although Init() for connecting single server(2 parameters) accepts hostname as well, it only connects one server under the domain.
Pros: Versatility of DNS, useable both in private or public network.
Cons: limited by transmission formats of DNS, unable to implement notification mechanisms.
### https://\<url\>
Similar with "http" prefix besides that the connections will be encrypted with SSL.
### consul://\<service-name\>
Get a list of servers with the specified service-name through consul. The default address of consul is localhost:8500, which can be modified by setting -consul\_agent\_addr in gflags. The connection timeout of consul is 200ms by default, which can be modified by -consul\_connect\_timeout\_ms.
By default, [stale](https://www.consul.io/api/index.html#consistency-modes) and passing(only servers with passing in statuses are returned) are added to [parameters of the consul request]((https://www.consul.io/api/health.html#parameters-2)), which can be modified by -consul\_url\_parameter in gflags.
Except the first request to consul, the follow-up requests use the [long polling](https://www.consul.io/api/index.html#blocking-queries) feature. That is, the consul responds only when the server list is updated or the request times out. The timeout defaults to 60 seconds, which can be modified by -consul\_blocking\_query\_wait\_secs.
If the server list returned by the consul does not follow [response format](https://www.consul.io/api/health.html#sample-response-2), or all servers in the list are filtered because the key fields such as the address and port are missing or cannot be parsed, the server list will not be updated and the consul service will be revisited after a period of time(default 500ms, can be modified by -consul\_retry\_interval\_ms）.
If consul is not accessible, the naming service can be automatically downgraded to file naming service. This feature is turned off by default and can be turned on by setting -consul\_enable\_degrade\_to\_file\_naming\_service. After downgrading, in the directory specified by -consul\_file\_naming\_service\_dir, the file whose name is the service-name will be used. This file can be generated by the consul-template, which holds the latest server list before the consul is unavailable. The consul naming service is automatically restored when consul is restored.
### nacos://\<service-name\>
NacosNamingService gets a list of servers from nacos periodically by [Open-Api](https://nacos.io/en-us/docs/open-api.html).
NacosNamingService supports [simple authentication](https://nacos.io/en-us/docs/auth.html).
`<service-name>` is a http uri query，For more detail, refer to `/nacos/v1/ns/instance/list` api document.
NOTE: `<service-name>` must be url-encoded.
```
nacos://serviceName=test&groupName=g&namespaceId=n&clusters=c&healthyOnly=true
```
The server list is cached for `cacheMillis` milliseconds as specified in the response of `/nacos/v1/ns/instance/list` api.
NOTE: The value of server weight must be an integer.
| GFlags | Description | Default value |
| ---------------------------------- | ------------------------------------ | ---------------------------- |
| nacos_address | nacos http url | "" |
| nacos_service_discovery_path | path for discovery | "/nacos/v1/ns/instance/list" |
| nacos_service_auth_path | path for login | "/nacos/v1/auth/login" |
| nacos_service_timeout_ms | timeout for connecting to nacos(ms) | 200 |
| nacos_username | url-encoded username | "" |
| nacos_password | url-encoded password | "" |
| nacos_load_balancer | load balancer for nacos clusters | "rr" |
### More naming services
User can extend to more naming services by implementing brpc::NamingService, check [this link](https://github.com/apache/brpc/blob/master/docs/cn/load_balancing.md#%E5%91%BD%E5%90%8D%E6%9C%8D%E5%8A%A1) for details.
### The tag in naming service
Every address can be attached with a tag. The common implementation is that if there're spaces after the address, the content after the spaces is the tag.
Same address with different tag are treated as different instances which are interacted with separate connections. Users can use this feature to establish connections with remote servers in a more flexible way.
If you need weighted round-robin, you should consider using [wrr algorithm](#wrr) first rather than emulate "a coarse-grained version" with tags.
### VIP related issues
VIP is often the public IP of layer-4 load balancer, which proxies traffic to RS behide. When a client connects to the VIP, a connection is established to a chosen RS. When the client connection is broken, the connection to the RS is reset as well.
If one client establishes only one connection to the VIP("single" connection type in brpc), all traffic from the client lands on one RS. If number of clients are large enough, each RS should gets many connections and roughly balanced, at least from the cluster perspective. However, if clients are not large enough or workload from clients are very different, some RS may be overloaded. Another issue is that when multiple VIP are listed together, the traffic to them may not be proportional to the number of RS behide them.
One solution to these issues is to use "pooled" connection type, so that one client may create multiple connections to one VIP (roughly the max concurrency recently) to make traffic land on different RS. If more than one VIP are present, consider using [wrr load balancing](#wrr) to assign weights to different VIP, or add different tags to VIP to form more instances.
If higher performance is demanded, or number of connections is limited (in a large cluster), consider using single connection and attach same VIP with different tags to create different connections. Comparing to pooled connections, number of connections and overhead of syscalls are often lower, but if tags are not enough, RS hotspots may still present.
### Naming Service Filter
Users can filter servers got from the NamingService before pushing to LoadBalancer.
![img](../images/ns_filter.jpg)
Interface of the filter:
```c++
// naming_service_filter.h
class NamingServiceFilter {
public:
// Return true to take this `server' as a candidate to issue RPC
// Return false to filter it out
virtual bool Accept(const ServerNode& server) const = 0;
};
// naming_service.h
struct ServerNode {
butil::EndPoint addr;
std::string tag;
};
```
The most common usage is filtering by server tags.
Customized filter is set to ChannelOptions to take effects. NULL by default means not filter.
```c++
class MyNamingServiceFilter : public brpc::NamingServiceFilter {
public:
bool Accept(const brpc::ServerNode& server) const {
return server.tag == "main";
}
};
int main() {
...
MyNamingServiceFilter my_filter;
...
brpc::ChannelOptions options;
options.ns_filter = &my_filter;
...
}
```
## Load Balancer
When there're more than one server to access, we need to divide the traffic. The process is called load balancing, which is positioned as follows at client-side.
![img](../images/lb.png)
The ideal algorithm is to make every request being processed in-time, and crash of any server makes minimal impact. However clients are not able to know delays or congestions happened at servers in realtime, and load balancing algorithms should be light-weight generally, users need to choose proper algorithms for their use cases. Algorithms provided by brpc (specified by `load_balancer_name`):
### rr
which is round robin. Always choose next server inside the list, next of the last server is the first one. No other settings. For example there're 3 servers: a,b,c, brpc will send requests to a, b, c, a, b, c, … and so on. Note that presumption of using this algorithm is the machine specs, network latencies, server loads are similar.
### wrr
which is weighted round robin. Choose the next server according to the configured weight. The chances a server is selected is consistent with its weight, and the algorithm can make each server selection scattered.
The instance tag must be an int32 number representing the weight, eg. tag="50".
### random
Randomly choose one server from the list, no other settings. Similarly with round robin, the algorithm assumes that servers to access are similar.
### wr
which is weighted random. Choose the next server according to the configured weight. The chances a server is selected is consistent with its weight.
Requirements of instance tag is the same as wrr.
### la
which is locality-aware. Perfer servers with lower latencies, until the latency is higher than others, no other settings. Check out [Locality-aware load balancing](lalb.md) for more details.
### p2c
which is power-of-two-choices with peak-EWMA latency scoring. Each selection samples two random servers and routes to the one with the lower `latency * (inflight + 1) / weight` score, where the latency is a peak-sensitive moving average: an upward spike takes effect immediately while recovery decays over `tau_ms`(default 10s). A slow or failing server is shed within one observation and selection cost is O(1) regardless of cluster size. Weight comes from the instance tag as in wrr(default 1). Optional parameters: `p2c:choices=4`(compare 4 sampled servers instead of 2, useful when many servers degrade at once), `p2c:tau_ms=5000`.
### c_murmurhash or c_md5
which is consistent hashing. Adding or removing servers does not make destinations of requests change as dramatically as in simple hashing. It's especially suitable for caching services.
Need to set Controller.set_request_code() before RPC otherwise the RPC will fail. request_code is often a 32-bit hash code of "key part" of the request, and the hashing algorithm does not need to be same with the one used by load balancer. Say `c_murmurhash` can use md5 to compute request_code of the request as well.
[src/brpc/policy/hasher.h](https://github.com/apache/brpc/blob/master/src/brpc/policy/hasher.h) includes common hash functions. If `std::string key` stands for key part of the request, controller.set_request_code(brpc::policy::MurmurHash32(key.data(), key.size())) sets request_code correctly.
Do distinguish "key" and "attributes" of the request. Don't compute request_code by full content of the request just for quick. Minor change in attributes may result in totally different hash code and change destination dramatically. Another cause is padding, for example: `struct Foo { int32_t a; int64_t b; }` has a 4-byte undefined gap between `a` and `b` on 64-bit machines, result of `hash(&foo, sizeof(foo))` is undefined. Fields need to be packed or serialized before hashing.
Check out [Consistent Hashing](consistent_hashing.md) for more details.
Other kind of lb does not need to set Controller.set_request_code(). If request code is set, it will not be used by lb. For example, lb=rr, and call Controller.set_request_code(), even if request_code is the same for every request, lb will balance the requests using the rr policy.
### Client-side throttling for recovery from cluster downtime
Cluster downtime refers to the state in which all servers in the cluster are unavailable. Due to the health check mechanism, when the cluster returns to normal, server will go online one by one. When a server is online, all traffic will be sent to it, which may cause the service to be overloaded again. If circuit breaker is enabled, server may be offline again before the other servers go online, and the cluster can never be recovered. As a solution, brpc provides a client-side throttling mechanism for recovery after cluster downtime. When no server is available in the cluster, the cluster enters recovery state. Assuming that the minimum number of servers that can serve all requests is min_working_instances, current number of servers available in the cluster is q, then in recovery state, the probability of client accepting the request is q/min_working_instances, otherwise it is discarded. If q remains unchanged for a period of time(hold_seconds), the traffic is resent to all available servers and leaves recovery state. Whether the request is rejected in recovery state is indicated by whether controller.ErrorCode() is equal to brpc::ERJECT, and the rejected request will not be retried by the framework.
This recovery mechanism requires the capabilities of downstream servers to be similar, so it is currently only valid for rr and random. The way to enable it is to add the values of min_working_instances and hold_seconds parameters after *load_balancer_name*, for example:
```c++
channel.Init("http://...", "random:min_working_instances=6 hold_seconds=10", &options);
```
## Health checking
Servers whose connections are lost are isolated temporarily to prevent them from being selected by LoadBalancer. brpc connects isolated servers periodically to test if they're healthy again. The interval is controlled by gflag -health_check_interval:
| Name | Value | Description | Defined At |
| ------------------------- | ----- | ---------------------------------------- | ----------------------- |
| health_check_interval (R) | 3 | seconds between consecutive health-checkings | src/brpc/socket_map.cpp |
Once a server is connected, it resumes as a server candidate inside LoadBalancer. If a server is removed from NamingService during health-checking, brpc removes it from health-checking as well.
# Launch RPC
Generally, we don't use Channel.CallMethod directly, instead we call XXX_Stub generated by protobuf, which feels more like a "method call". The stub has few member fields, being suitable(and recommended) to be put on stack instead of new(). Surely the stub can be saved and re-used as well. Channel.CallMethod and stub are both **thread-safe** and accessible by multiple threads simultaneously. For example:
```c++
XXX_Stub stub(&channel);
stub.some_method(controller, request, response, done);
```
Or even:
```c++
XXX_Stub(&channel).some_method(controller, request, response, done);
```
A exception is http/h2 client, which is not related to protobuf much. Call CallMethod directly to make a http call, setting all parameters to NULL except for `Controller` and `done`, check [Access http/h2](http_client.md) for details.
## Synchronous call
CallMethod blocks until response from server is received or error occurred (including timedout).
response/controller in synchronous call will not be used by brpc again after CallMethod, they can be put on stack safely. Note: if request/response has many fields and being large on size, they'd better be allocated on heap.
```c++
MyRequest request;
MyResponse response;
brpc::Controller cntl;
XXX_Stub stub(&channel);
request.set_foo(...);
cntl.set_timeout_ms(...);
stub.some_method(&cntl, &request, &response, NULL);
if (cntl.Failed()) {
// RPC failed. fields in response are undefined, don't use.
} else {
// RPC succeeded, response has what we want.
}
```
> WARNING: Do NOT use synchronous call when you are holding a pthread lock! Otherwise it is easy to cause deadlock.
>
> Solution (choose one of the two):
> 1. Replace pthread lock with bthread lock (bthread_mutex_t)
> 1. Release the lock before CallMethod
## Asynchronous call
Pass a callback `done` to CallMethod, which resumes after sending request, rather than completion of RPC. When the response from server is received or error occurred(including timedout), done->Run() is called. Post-processing code of the RPC should be put in done->Run() instead of after CallMethod.
Because end of CallMethod does not mean completion of RPC, response/controller may still be used by brpc or done->Run(). Generally they should be allocated on heap and deleted in done->Run(). If they're deleted too early, done->Run() may access invalid memory.
You can new these objects individually and create done by [NewCallback](#use-newcallback), or make response/controller be member of done and [new them together](#Inherit-google::protobuf::Closure). Former one is recommended.
Request can be destroyed immediately after asynchronous CallMethod. (SelectiveChannel is an exception, in the case of SelectiveChannel, the request object must be released after rpc finish)
Channel can be destroyed immediately after asynchronous CallMethod.
Note that "immediately" means destruction of Request/Channel can happen **after** CallMethod, not during CallMethod. Deleting a Channel just being used by another thread results in undefined behavior (crash at best).
### Use NewCallback
```c++
static void OnRPCDone(MyResponse* response, brpc::Controller* cntl) {
// unique_ptr helps us to delete response/cntl automatically. unique_ptr in gcc 3.4 is an emulated version.
std::unique_ptr<MyResponse> response_guard(response);
std::unique_ptr<brpc::Controller> cntl_guard(cntl);
if (cntl->Failed()) {
// RPC failed. fields in response are undefined, don't use.
} else {
// RPC succeeded, response has what we want. Continue the post-processing.
}
// Closure created by NewCallback deletes itself at the end of Run.
}
MyResponse* response = new MyResponse;
brpc::Controller* cntl = new brpc::Controller;
MyService_Stub stub(&channel);
MyRequest request; // you don't have to new request, even in an asynchronous call.
request.set_foo(...);
cntl->set_timeout_ms(...);
stub.some_method(cntl, &request, response, brpc::NewCallback(OnRPCDone, response, cntl));
```
Since protobuf 3 changes NewCallback to private, brpc puts NewCallback in [src/brpc/callback.h](https://github.com/apache/brpc/blob/master/src/brpc/callback.h) after r32035 (and adds more overloads). If your program has compilation issues with NewCallback, replace google::protobuf::NewCallback with brpc::NewCallback.
### Inherit google::protobuf::Closure
Drawback of using NewCallback is that you have to allocate memory on heap at least 3 times: response, controller, done. If profiler shows that the memory allocation is a hotspot, you can consider inheriting Closure by your own, and enclose response/controller as member fields. Doing so combines 3 new into one, but the code will be worse to read. Don't do this if memory allocation is not an issue.
```c++
class OnRPCDone: public google::protobuf::Closure {
public:
void Run() {
// unique_ptr helps us to delete response/cntl automatically. unique_ptr in gcc 3.4 is an emulated version.
std::unique_ptr<OnRPCDone> self_guard(this);
if (cntl->Failed()) {
// RPC failed. fields in response are undefined, don't use.
} else {
// RPC succeeded, response has what we want. Continue the post-processing.
}
}
MyResponse response;
brpc::Controller cntl;
}
OnRPCDone* done = new OnRPCDone;
MyService_Stub stub(&channel);
MyRequest request; // you don't have to new request, even in an asynchronous call.
request.set_foo(...);
done->cntl.set_timeout_ms(...);
stub.some_method(&done->cntl, &request, &done->response, done);
```
### What will happen when the callback is very complicated?
No special impact, the callback will run in separate bthread, without blocking other sessions. You can do all sorts of things in the callback.
### Does the callback run in the same thread that CallMethod runs?
The callback runs in a different bthread, even the RPC fails just after entering CallMethod. This avoids deadlock when the RPC is ongoing inside a lock(not recommended).
## Wait for completion of RPC
NOTE: [ParallelChannel](combo_channel.md#parallelchannel) is probably more convenient to launch multiple RPCs in parallel.
Following code starts 2 asynchronous RPC and waits them to complete.
```c++
const brpc::CallId cid1 = controller1->call_id();
const brpc::CallId cid2 = controller2->call_id();
...
stub.method1(controller1, request1, response1, done1);
stub.method2(controller2, request2, response2, done2);
...
brpc::Join(cid1);
brpc::Join(cid2);
```
Call `Controller.call_id()` to get an id **before launching RPC**, join the id after the RPC.
Join() blocks until completion of RPC **and end of done->Run()**, properties of Join:
- If the RPC is complete, Join() returns immediately.
- Multiple threads can Join() one id, all of them will be woken up.
- Synchronous RPC can be Join()-ed in another thread, although we rarely do this.
Join() was called JoinResponse() before, if you meet deprecated issues during compilation, rename to Join().
Calling `Join(controller->call_id())` after completion of RPC is **wrong**, do save call_id before RPC, otherwise the controller may be deleted by done at any time. The Join in following code is **wrong**.
```c++
static void on_rpc_done(Controller* controller, MyResponse* response) {
... Handle response ...
delete controller;
delete response;
}
Controller* controller1 = new Controller;
Controller* controller2 = new Controller;
MyResponse* response1 = new MyResponse;
MyResponse* response2 = new MyResponse;
...
stub.method1(controller1, &request1, response1, google::protobuf::NewCallback(on_rpc_done, controller1, response1));
stub.method2(controller2, &request2, response2, google::protobuf::NewCallback(on_rpc_done, controller2, response2));
...
brpc::Join(controller1->call_id()); // WRONG, controller1 may be deleted by on_rpc_done
brpc::Join(controller2->call_id()); // WRONG, controller2 may be deleted by on_rpc_done
```
## Semi-synchronous call
Join can be used for implementing "Semi-synchronous" call: blocks until multiple asynchronous calls to complete. Since the callsite blocks until completion of all RPC, controller/response can be put on stack safely.
```c++
brpc::Controller cntl1;
brpc::Controller cntl2;
MyResponse response1;
MyResponse response2;
...
stub1.method1(&cntl1, &request1, &response1, brpc::DoNothing());
stub2.method2(&cntl2, &request2, &response2, brpc::DoNothing());
...
brpc::Join(cntl1.call_id());
brpc::Join(cntl2.call_id());
```
brpc::DoNothing() gets a closure doing nothing, specifically for semi-synchronous calls. Its lifetime is managed by brpc.
Note that in above example, we access `controller.call_id()` after completion of RPC, which is safe right here, because DoNothing does not delete controller as in `on_rpc_done` in previous example.
## Cancel RPC
`brpc::StartCancel(call_id)` cancels corresponding RPC, call_id must be got from Controller.call_id() **before launching RPC**, race conditions may occur at any other time.
NOTE: it is `brpc::StartCancel(call_id)`, not `controller->StartCancel()`, which is forbidden and useless. The latter one is provided by protobuf by default and has serious race conditions on lifetime of controller.
As the name implies, RPC may not complete yet after calling StartCancel, you should not touch any field in Controller or delete any associated resources, they should be handled inside done->Run(). If you have to wait for completion of RPC in-place(not recommended), call Join(call_id).
Facts about StartCancel:
- call_id can be cancelled before CallMethod, the RPC will end immediately(and done will be called).
- call_id can be cancelled in another thread.
- Cancel an already-cancelled call_id has no effect. Inference: One call_id can be cancelled by multiple threads simultaneously, but only one of them takes effect.
- Cancel here is a client-only feature, **the server-side may not cancel the operation necessarily**, server cancelation is a separate feature.
## Get server-side address and port
remote_side() tells where request was sent to, the return type is [butil::EndPoint](https://github.com/apache/brpc/blob/master/src/butil/endpoint.h), which includes an ipv4 address and port. Calling this method before completion of RPC is undefined.
How to print:
```c++
LOG(INFO) << "remote_side=" << cntl->remote_side();
printf("remote_side=%s\n", butil::endpoint2str(cntl->remote_side()).c_str());
```
## Get client-side address and port
local_side() gets address and port of the client-side sending RPC after r31384
How to print:
```c++
LOG(INFO) << "local_side=" << cntl->local_side();
printf("local_side=%s\n", butil::endpoint2str(cntl->local_side()).c_str());
```
## Should brpc::Controller be reused?
Not necessary to reuse deliberately.
Controller has miscellaneous fields, some of them are buffers that can be re-used by calling Reset().
In most use cases, constructing a Controller(snippet1) and re-using a Controller(snippet2) perform similarily.
```c++
// snippet1
for (int i = 0; i < n; ++i) {
brpc::Controller controller;
...
stub.CallSomething(..., &controller);
}
// snippet2
brpc::Controller controller;
for (int i = 0; i < n; ++i) {
controller.Reset();
...
stub.CallSomething(..., &controller);
}
```
If the Controller in snippet1 is new-ed on heap, snippet1 has extra cost of "heap allcation" and may be a little slower in some cases.
# Settings
Client-side settings has 3 parts:
- brpc::ChannelOptions: defined in [src/brpc/channel.h](https://github.com/apache/brpc/blob/master/src/brpc/channel.h), for initializing Channel, becoming immutable once the initialization is done.
- brpc::Controller: defined in [src/brpc/controller.h](https://github.com/apache/brpc/blob/master/src/brpc/controller.h), for overriding fields in brpc::ChannelOptions for some RPC according to contexts.
- global gflags: for tuning global behaviors, being unchanged generally. Read comments in [/flags](flags.md) before setting.
Controller contains data and options that request may not have. server and client share the same Controller class, but they may set different fields. Read comments in Controller carefully before using.
A Controller corresponds to a RPC. A Controller can be re-used by another RPC after Reset(), but a Controller can't be used by multiple RPC simultaneously, no matter the RPCs are started from one thread or not.
Properties of Controller:
1. A Controller can only have one user. Without explicit statement, methods in Controller are **not** thread-safe by default.
2. Due to the fact that Controller is not shared generally, there's no need to manage Controller by shared_ptr. If you do, something might goes wrong.
3. Controller is constructed before RPC and destructed after RPC, some common patterns:
- Put Controller on stack before synchronous RPC, be destructed when out of scope. Note that Controller of asynchronous RPC **must not** be put on stack, otherwise the RPC may still run when the Controller is being destructed and result in undefined behavior.
- new Controller before asynchronous RPC, delete in done.
## Number of worker pthreads
There's **no** independent thread pool for client in brpc. All Channels and Servers share the same backing threads via [bthread](bthread.md). Setting number of worker pthreads in Server works for Client as well if Server is in used. Or just specify the [gflag](flags.md) [-bthread_concurrency](brpc.baidu.com:8765/flags/bthread_concurrency) to set the global number of worker pthreads.
## Timeout
**ChannelOptions.timeout_ms** is timeout in milliseconds for all RPCs via the Channel, Controller.set_timeout_ms() overrides value for one RPC. Default value is 1 second, Maximum value is 2^31 (about 24 days), -1 means wait indefinitely for response or connection error.
**ChannelOptions.connect_timeout_ms** is the timeout in milliseconds for establishing connections of RPCs over the Channel, and -1 means no deadline. This value is limited to be not greater than timeout_ms. Note that this connection timeout is different from the one in TCP, generally this one is smaller.
NOTE1: timeout_ms in brpc is **deadline**, which means once it's reached, the RPC ends without more retries. As a comparison, other implementations may have session timeouts and deadline timeouts. Do distinguish them before porting to brpc.
NOTE2: error code of RPC timeout is **ERPCTIMEDOUT (1008) **, ETIMEDOUT is connection timeout and retriable.
## Retry
ChannelOptions.max_retry is maximum retrying count for all RPC via the channel, Default value is 3, 0 means no retries. Controller.set_max_retry() overrides value for one RPC.
Controller.retried_count() returns number of retries.
Controller.has_backup_request() tells if backup_request was sent.
**Servers tried before are not retried by best efforts**
Conditions for retrying (AND relations):
- Broken connection.
- Timeout is not reached.
- Has retrying quota. Controller.set_max_retry(0) or ChannelOptions.max_retry = 0 disables retries.
- The retry makes sense. If the RPC fails due to request(EREQUEST), no retry will be done because server is very likely to reject the request again, retrying makes no sense here.
### Broken connection
If the server does not respond and connection is good, retry is not triggered. If you need to send another request after some timeout, use backup request.
How it works: If response does not return within the timeout specified by backup_request_ms, send another request, take whatever the first returned. New request will be sent to a different server that never tried before by best efforts. NOTE: If backup_request_ms is greater than timeout_ms, backup request will never be sent. backup request consumes one retry. backup request does not imply a server-side cancellation.
ChannelOptions.backup_request_ms affects all RPC via the Channel, unit is milliseconds, Default value is -1(disabled), Controller.set_backup_request_ms() overrides value for one RPC.
### Timeout is not reached
RPC will be ended soon after the timeout.
### Has retrying quota
Controller.set_max_retry(0) or ChannelOptions.max_retry = 0 disables retries.
### The retry makes sense
If the RPC fails due to request(EREQUEST), no retry will be done because server is very likely to reject the request again, retrying makes no sense here.
Users can inherit [brpc::RetryPolicy](https://github.com/apache/brpc/blob/master/src/brpc/retry_policy.h) to customize conditions of retrying. For example brpc does not retry for http/h2 related errors by default. If you want to retry for HTTP_STATUS_FORBIDDEN(403) in your app, you can do as follows:
```c++
#include <brpc/retry_policy.h>
class MyRetryPolicy : public brpc::RetryPolicy {
public:
bool DoRetry(const brpc::Controller* cntl) const {
if (cntl->ErrorCode() == brpc::EHTTP && // http/h2 error
cntl->http_response().status_code() == brpc::HTTP_STATUS_FORBIDDEN) {
return true;
}
// Leave other cases to brpc.
return brpc::DefaultRetryPolicy()->DoRetry(cntl);
}
};
...
// Assign the instance to ChannelOptions.retry_policy.
// NOTE: retry_policy must be kept valid during lifetime of Channel, and Channel does not retry_policy, so in most cases RetryPolicy should be created by singleton..
brpc::ChannelOptions options;
static MyRetryPolicy g_my_retry_policy;
options.retry_policy = &g_my_retry_policy;
...
```
Some tips：
- Get response of the RPC by cntl->response().
- RPC deadline represented by ERPCTIMEDOUT is never retried, even it's allowed by your derived RetryPolicy.
### Retrying should be conservative
Due to maintaining costs, even very large scale clusters are deployed with "just enough" instances to survive major defects, namely offline of one IDC, which is at most 1/2 of all machines. However aggressive retries may easily make pressures from all clients double or even tripple against servers, and make the whole cluster down: More and more requests stuck in buffers, because servers can't process them in-time. All requests have to wait for a very long time to be processed and finally gets timed out, as if the whole cluster is crashed. The default retrying policy is safe generally: unless the connection is broken, retries are rarely sent. However users are able to customize starting conditions for retries by inheriting RetryPolicy, which may turn retries to be "a storm". When you customized RetryPolicy, you need to carefully consider how clients and servers interact and design corresponding tests to verify that retries work as expected.
## Circuit breaker
Check out [circuit_breaker](../cn/circuit_breaker.md) for more details.
## Protocols
The default protocol used by Channel is baidu_std, which is changeable by setting ChannelOptions.protocol. The field accepts both enum and string.
Supported protocols:
- PROTOCOL_BAIDU_STD or "baidu_std", which is [the standard binary protocol inside Baidu](baidu_std.md), using single connection by default.
- PROTOCOL_HTTP or "http", which is http/1.0 or http/1.1, using pooled connection by default (Keep-Alive).
- Methods for accessing ordinary http services are listed in [Access http/h2](http_client.md).
- Methods for accessing pb services by using http:json or http:proto are listed in [http/h2 derivatives](http_derivatives.md)
- PROTOCOL_H2 or ”h2", which is http/2, using single connection by default.
- Methods for accessing ordinary h2 services are listed in [Access http/h2](http_client.md).
- Methods for accessing pb services by using h2:json or h2:proto are listed in [http/h2 derivatives](http_derivatives.md)
- "h2:grpc", which is the protocol of [gRPC](https://grpc.io) and based on h2, using single connection by default, check out [h2:grpc](http_derivatives.md#h2grpc) for details.
- PROTOCOL_THRIFT or "thrift", which is the protocol of [apache thrift](https://thrift.apache.org), using pooled connection by default, check out [Access thrift](thrift.md) for details.
- PROTOCOL_MEMCACHE or "memcache", which is binary protocol of memcached, using **single connection** by default. Check out [Access memcached](memcache_client.md) for details.
- PROTOCOL_REDIS or "redis", which is protocol of redis 1.2+ (the one supported by hiredis), using **single connection** by default. Check out [Access Redis](redis_client.md) for details.
- PROTOCOL_HULU_PBRPC or "hulu_pbrpc", which is protocol of hulu-pbrpc, using single connection by default.
- PROTOCOL_NOVA_PBRPC or "nova_pbrpc", which is protocol of Baidu ads union, using pooled connection by default.
- PROTOCOL_SOFA_PBRPC or "sofa_pbrpc", which is protocol of sofa-pbrpc, using single connection by default.
- PROTOCOL_PUBLIC_PBRPC or "public_pbrpc", which is protocol of public_pbrpc, using pooled connection by default.
- PROTOCOL_UBRPC_COMPACK or "ubrpc_compack", which is protocol of public/ubrpc, packing with compack, using pooled connection by default. check out [ubrpc (by protobuf)](ub_client.md) for details. A related protocol is PROTOCOL_UBRPC_MCPACK2 or ubrpc_mcpack2, packing with mcpack2.
- PROTOCOL_NSHEAD_CLIENT or "nshead_client", which is required by UBXXXRequest in baidu-rpc-ub, using pooled connection by default. Check out [Access UB](ub_client.md) for details.
- PROTOCOL_NSHEAD or "nshead", which is required by sending NsheadMessage, using pooled connection by default. Check out [nshead+blob](ub_client.md#nshead-blob) for details.
- PROTOCOL_NSHEAD_MCPACK or "nshead_mcpack", which is as the name implies, nshead + mcpack (parsed by protobuf via mcpack2pb), using pooled connection by default.
- PROTOCOL_ESP or "esp", for accessing services with esp protocol, using pooled connection by default.
## Connection Type
brpc supports following connection types:
- short connection: Established before each RPC, closed after completion. Since each RPC has to pay the overhead of establishing connection, this type is used for occasionally launched RPC, not frequently launched ones. No protocol use this type by default. Connections in http/1.0 are handled similarly as short connections.
- pooled connection: Pick an unused connection from a pool before each RPC, return after completion. One connection carries at most one request at the same time. One client may have multiple connections to one server. http/1.1 and the protocols using nshead use this type by default.
- single connection: all clients in one process has at most one connection to one server, one connection may carry multiple requests at the same time. The sequence of received responses does not need to be same as sending requests. This type is used by baidu_std, hulu_pbrpc, sofa_pbrpc by default.
| | short connection | pooled connection | single connection |
| ---------------------------------------- | ---------------------------------------- | --------------------------------------- | ---------------------------------------- |
| long connection | no | yes | yes |
| \#connection at server-side (from a client) | qps*latency ([little's law](https://en.wikipedia.org/wiki/Little%27s_law)) | qps*latency | 1 |
| peak qps | bad, and limited by max number of ports | medium | high |
| latency | 1.5RTT(connect) + 1RTT + processing time | 1RTT + processing time | 1RTT + processing time |
| cpu usage | high, tcp connect for each RPC | medium, every request needs a sys write | low, writes can be combined to reduce overhead. |
brpc chooses best connection type for the protocol by default, users generally have no need to change it. If you do, set ChannelOptions.connection_type to:
- CONNECTION_TYPE_SINGLE or "single" : single connection
- CONNECTION_TYPE_POOLED or "pooled": pooled connection. Max number of pooled connections from one client to one server is limited by -max_connection_pool_size. Note the number is not same as "max number of connections". New connections are always created when there's no idle ones in the pool; the returned connections are closed immediately when the pool already has max_connection_pool_size connections. Value of max_connection_pool_size should respect the concurrency, otherwise the connnections that can't be pooled are created and closed frequently which behaves similarly as short connections. If max_connection_pool_size is 0, the pool behaves just like fully short connections.
| Name | Value | Description | Defined At |
| ---------------------------- | ----- | ---------------------------------------- | ------------------- |
| max_connection_pool_size (R) | 100 | Max number of pooled connections to a single endpoint | src/brpc/socket.cpp |
- CONNECTION_TYPE_SHORT or "short" : short connection
- "" (empty string) makes brpc chooses the default one.
brpc also supports [Streaming RPC](streaming_rpc.md) which is an application-level connection for transferring streaming data.
## Close idle connections in pools
If a connection has no read or write within the seconds specified by -idle_timeout_second, it's tagged as "idle", and will be closed automatically. Default value is 10 seconds. This feature is only effective to pooled connections. If -log_idle_connection_close is true, a log is printed before closing.
| Name | Value | Description | Defined At |
| ------------------------- | ----- | ---------------------------------------- | ----------------------- |
| idle_timeout_second | 10 | Pooled connections without data transmission for so many seconds will be closed. No effect for non-positive values | src/brpc/socket_map.cpp |
| log_idle_connection_close | false | Print log when an idle connection is closed | src/brpc/socket.cpp |
## Defer connection close
Multiple channels may share a connection via referential counting. When a channel releases last reference of the connection, the connection will be closed. But in some scenarios, channels are created just before sending RPC and destroyed after completion, in which case connections are probably closed and re-open again frequently, as costly as short connections.
One solution is to cache channels commonly used by user, which avoids frequent creation and destroying of channels. However brpc does not offer an utility for doing this right now, and it's not trivial for users to implement it correctly.
Another solution is setting gflag -defer_close_second
| Name | Value | Description | Defined At |
| ------------------ | ----- | ---------------------------------------- | ----------------------- |
| defer_close_second | 0 | Defer close of connections for so many seconds even if the connection is not used by anyone. Close immediately for non-positive values | src/brpc/socket_map.cpp |
| defer_close_respect_idle | false | When defer_close_second > 0, close a connection immediately when the last reference is removed and the socket has already been idle for longer than defer_close_second | src/brpc/socket_map.cpp |
After setting, connection is not closed immediately after last referential count, instead it will be closed after so many seconds. If a channel references the connection again during the wait, the connection resumes to normal. No matter how frequent channels are created, this flag limits the frequency of closing connections. Side effect of the flag is that file descriptors are not closed immediately after destroying of channels, if the flag is wrongly set to be large, number of active file descriptors in the process may be large as well. When -defer_close_respect_idle is enabled, a connection that has already been idle for longer than defer_close_second may be closed when the last reference is removed.
## Buffer size of connections
-socket_recv_buffer_size sets receiving buffer size of all connections, -1 by default (not modified)
-socket_send_buffer_size sets sending buffer size of all connections, -1 by default (not modified)
| Name | Value | Description | Defined At |
| ----------------------- | ----- | ---------------------------------------- | ------------------- |
| socket_recv_buffer_size | -1 | Set the recv buffer size of socket if this value is positive | src/brpc/socket.cpp |
| socket_send_buffer_size | -1 | Set send buffer size of sockets if this value is positive | src/brpc/socket.cpp |
## log_id
set_log_id() sets a 64-bit integral log_id, which is sent to the server-side along with the request, and often printed in server logs to associate different services accessed in a session. String-type log-id must be converted to 64-bit integer before setting.
## Attachment
baidu_std and hulu_pbrpc supports attachments which are sent along with messages and set by users to bypass serialization of protobuf. As a client, data set in Controller::request_attachment() will be received by server and response_attachment() contains attachment sent back by the server.
Attachment is not compressed by framework.
In http/h2, attachment corresponds to [message body](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html), namely the data to post to server is stored in request_attachment().
## Turn on SSL
Update openssl to the latest version before turning on SSL, since older versions of openssl may have severe security problems and support less encryption algorithms, which is against with the purpose of using SSL.
Set ChannelOptions.mutable_ssl_options() to enable SSL. Refer to [ssl_options.h](https://github.com/apache/brpc/blob/master/src/brpc/ssl_options.h) for the detailed options. ChannelOptions.has_ssl_options() checks if ssl_options was set; ChannelOptions.ssl_options() returns const reference to the ssl_options.
```c++
// Enable client-side SSL and use default values.
options.mutable_ssl_options();
// Enable client-side SSL and customize values.
options.mutable_ssl_options()->ciphers_name = "...";
options.mutable_ssl_options()->sni_name = "...";
// Set the protocol preference of ALPN.
// (By default ALPN is disabled.)
options.mutable_ssl_options()->alpn_protocols = {"h2", "http/1.1"};
```
- Channels connecting to a single server or a cluster both support SSL (the initial implementation does not support cluster)
- After turning on SSL, all requests through this Channel will be encrypted. Users should create another Channel for non-SSL requests if needed.
- Accessibility improvements for HTTPS: Channel.Init recognizes https:// prefix and turns on SSL automatically; -http_verbose prints certificate information when SSL is on.
## Authentication
Generally there are 2 ways of authentication at the client side:
1. Request-based authentication: Each request carries authentication information. It's more flexible since the authentication information can contain fields based on this particular request. However, this leads to a performance loss due to the extra payload in each request.
2. Connection-based authentication: Once a TCP connection has been established, the client sends an authentication packet. After it has been verfied by the server, subsequent requests on this connection no longer needs authentication. Compared with the former, this method can only carry some static information such as local IP in the authentication packet. However, it has better performance especially under single connection / connection pool scenario.
It's very simple to implement the first method by just adding authentication data format into the request proto definition. Then send it as normal RPC in each request. To achieve the second one, brpc provides an interface for users to implement:
```c++
class Authenticator {
public:
virtual ~Authenticator() {}
// Implement this method to generate credential information
// into `auth_str' which will be sent to `VerifyCredential'
// at server side. This method will be called on client side.
// Returns 0 on success, error code otherwise
virtual int GenerateCredential(std::string* auth_str) const = 0;
};
```
When the user calls the RPC interface with a single connection to the same server, the framework guarantee that once the TCP connection has been established, the first request on the connection will contain the authentication string generated by `GenerateCredential`. Subsequent requests will not carried that string. The entire sending process is still highly concurrent since it won't wait for the authentication result. If the verification succeeds, all requests return without error. Otherwise, if the verification fails, generally the server will close the connection and those requests will receive the corresponding error.
Currently only those protocols support client authentication: [baidu_std](../cn/baidu_std.md) (default protocol), HTTP, hulu_pbrpc, ESP. For customized protocols, generally speaking, users could call the `Authenticator`'s interface to generate authentication string during the request packing process in order to support authentication.
## Reset
This method makes Controller back to the state as if it's just created.
Don't call Reset() during a RPC, which is undefined.
## Compression
set_request_compress_type() sets compress-type of the request, no compression by default.
NOTE: Attachment is not compressed by brpc.
Check out [compress request body](http_client.md#compress-request-body) to compress http/h2 body.
Supported compressions:
- brpc::CompressTypeSnappy : [snanpy](http://google.github.io/snappy/), compression and decompression are very fast, but compression ratio is low.
- brpc::CompressTypeGzip : [gzip](http://en.wikipedia.org/wiki/Gzip), significantly slower than snappy, with a higher compression ratio.
- brpc::CompressTypeZlib : [zlib](http://en.wikipedia.org/wiki/Zlib), 10%~20% faster than gzip but still significantly slower than snappy, with slightly better compression ratio than gzip.
Following table lists performance of different methods compressing and decompressing **data with a lot of duplications**, just for reference.
| Compress method | Compress size(B) | Compress time(us) | Decompress time(us) | Compress throughput(MB/s) | Decompress throughput(MB/s) | Compress ratio |
| --------------- | ---------------- | ----------------- | ------------------- | ------------------------- | --------------------------- | -------------- |
| Snappy | 128 | 0.753114 | 0.890815 | 162.0875 | 137.0322 | 37.50% |
| Gzip | 10.85185 | 1.849199 | 11.2488 | 66.01252 | 47.66% | |
| Zlib | 10.71955 | 1.66522 | 11.38763 | 73.30581 | 38.28% | |
| Snappy | 1024 | 1.404812 | 1.374915 | 695.1555 | 710.2713 | 8.79% |
| Gzip | 16.97748 | 3.950946 | 57.52106 | 247.1718 | 6.64% | |
| Zlib | 15.98913 | 3.06195 | 61.07665 | 318.9348 | 5.47% | |
| Snappy | 16384 | 8.822967 | 9.865008 | 1770.946 | 1583.881 | 4.96% |
| Gzip | 160.8642 | 43.85911 | 97.13162 | 356.2544 | 0.78% | |
| Zlib | 147.6828 | 29.06039 | 105.8011 | 537.6734 | 0.71% | |
| Snappy | 32768 | 16.16362 | 19.43596 | 1933.354 | 1607.844 | 4.82% |
| Gzip | 229.7803 | 82.71903 | 135.9995 | 377.7849 | 0.54% | |
| Zlib | 240.7464 | 54.44099 | 129.8046 | 574.0161 | 0.50% | |
Following table lists performance of different methods compressing and decompressing **data with very few duplications**, just for reference.
| Compress method | Compress size(B) | Compress time(us) | Decompress time(us) | Compress throughput(MB/s) | Decompress throughput(MB/s) | Compress ratio |
| --------------- | ---------------- | ----------------- | ------------------- | ------------------------- | --------------------------- | -------------- |
| Snappy | 128 | 0.866002 | 0.718052 | 140.9584 | 170.0021 | 105.47% |
| Gzip | 15.89855 | 4.936242 | 7.678077 | 24.7294 | 116.41% | |
| Zlib | 15.88757 | 4.793953 | 7.683384 | 25.46339 | 107.03% | |
| Snappy | 1024 | 2.087972 | 1.06572 | 467.7087 | 916.3403 | 100.78% |
| Gzip | 32.54279 | 12.27744 | 30.00857 | 79.5412 | 79.79% | |
| Zlib | 31.51397 | 11.2374 | 30.98824 | 86.90288 | 78.61% | |
| Snappy | 16384 | 12.598 | 6.306592 | 1240.276 | 2477.566 | 100.06% |
| Gzip | 537.1803 | 129.7558 | 29.08707 | 120.4185 | 75.32% | |
| Zlib | 519.5705 | 115.1463 | 30.07291 | 135.697 | 75.24% | |
| Snappy | 32768 | 22.68531 | 12.39793 | 1377.543 | 2520.582 | 100.03% |
| Gzip | 1403.974 | 258.9239 | 22.25825 | 120.6919 | 75.25% | |
| Zlib | 1370.201 | 230.3683 | 22.80687 | 135.6524 | 75.21% | |
# FAQ
### Q: Does brpc support unix domain socket?
No. Local TCP sockets performs just a little slower than unix domain socket since traffic over local TCP sockets bypasses network. Some scenarios where TCP sockets can't be used may require unix domain sockets. We may consider the capability in future.
### Q: Fail to connect to xx.xx.xx.xx:xxxx, Connection refused
The remote server does not serve any more (probably crashed).
### Q: often met Connection timedout to another IDC
![img](../images/connection_timedout.png)
The TCP connection is not established within connection_timeout_ms, you have to tweak options:
```c++
struct ChannelOptions {
...
// Issue error when a connection is not established after so many
// milliseconds. -1 means wait indefinitely.
// Default: 200 (milliseconds)
// Maximum: 0x7fffffff (roughly 30 days)
int32_t connect_timeout_ms;
// Max duration of RPC over this Channel. -1 means wait indefinitely.
// Overridable by Controller.set_timeout_ms().
// Default: 500 (milliseconds)
// Maximum: 0x7fffffff (roughly 30 days)
int32_t timeout_ms;
...
};
```
NOTE: Connection timeout is not RPC timeout, which is printed as "Reached timeout=...".
### Q: synchronous call is good, asynchronous call crashes
Check lifetime of Controller, Response and done. In asynchronous call, finish of CallMethod is not completion of RPC which is entering of done->Run(). So the objects should not deleted just after CallMethod, instead they should be delete in done->Run(). Generally you should allocate the objects on heap instead of putting them on stack. Check out [Asynchronous call](client.md#asynchronous-call) for details.
### Q: How to make requests be processed once and only once
This issue is not solved on RPC layer. When response returns and being successful, we know the RPC is processed at server-side. When response returns and being rejected, we know the RPC is not processed at server-side. But when response is not returned, server may or may not process the RPC. If we retry, same request may be processed twice at server-side. Generally RPC services with side effects must consider [idempotence](http://en.wikipedia.org/wiki/Idempotence) of the service, otherwise retries may make side effects be done more than once and result in unexpected behavior. Search services with only read often have no side effects (during a search), being idempotent natually. But storage services that need to write have to design versioning or serial-number mechanisms to reject side effects that already happen, to keep idempoent.
### Q: Invalid address=`bns://group.user-persona.dumi.nj03'
```
FATAL 04-07 20:00:03 7778 src/brpc/channel.cpp:123] Invalid address=`bns://group.user-persona.dumi.nj03'. You should use Init(naming_service_name, load_balancer_name, options) to access multiple servers.
```
Accessing servers under naming service needs the Init() with 3 parameters(the second param is `load_balancer_name`). The Init() here is with 2 parameters and treated by brpc as accessing single server, producing the error.
### Q: Both sides use protobuf, why can't they communicate with each other
**protocol != protobuf**. protobuf serializes one package and a message of a protocol may contain multiple packages along with extra lengths, checksums, magic numbers. The capability offered by brpc that "write code once and serve multiple protocols" is implemented by converting data from different protocols to unified API, not on protobuf layer.
### Q: Why C++ client/server may fail to talk to client/server in other languages
Check if the C++ version turns on compression (Controller::set_compress_type), Currently RPC impl. in other languages do not support compression yet.
# PS: Workflow at Client-side
![img](../images/client_side.png)
Steps:
1. Create a [bthread_id](https://github.com/apache/brpc/blob/master/src/bthread/id.h) as correlation_id of current RPC.
2. According to how the Channel is initialized, choose a server from global [SocketMap](https://github.com/apache/brpc/blob/master/src/brpc/socket_map.h) or [LoadBalancer](https://github.com/apache/brpc/blob/master/src/brpc/load_balancer.h) as destination of the request.
3. Choose a [Socket](https://github.com/apache/brpc/blob/master/src/brpc/socket.h) according to connection type (single, pooled, short)
4. If authentication is turned on and the Socket is not authenticated yet, first request enters authenticating branch, other requests block until the branch writes authenticating information into the Socket. Server-side only verifies the first request.
5. According to protocol of the Channel, choose corresponding serialization callback to serialize request into [IOBuf](https://github.com/apache/brpc/blob/master/src/butil/iobuf.h).
6. If timeout is set, setup timer. From this point on, avoid using Controller, since the timer may be triggered at anytime and calls user's callback for timeout, which may delete Controller.
7. Sending phase is completed. If error occurs at any step, Channel::HandleSendFailed is called.
8. Write IOBuf with serialized data into the Socket and add Channel::HandleSocketFailed into id_wait_list of the Socket. The callback will be called when the write is failed or connection is broken before completion of RPC.
9. In synchronous call, Join correlation_id; otherwise CallMethod() returns.
10. Send/receive messages to/from network.
11. After receiving response, get the correlation_id inside, find out associated Controller within O(1) time. The lookup does not need to lock a global hashmap, and scales well.
12. Parse response according to the protocol
13. Call Controller::OnRPCReturned, which may retry errorous RPC, or complete the RPC. Call user's done in asynchronous call. Destroy correlation_id and wakeup joining threads.
docs/en/combo_channel.md
+519
View File
@@ -0,0 +1,519 @@
[中文版](../cn/combo_channel.md)
With the growth of services, access patterns to downstream servers become increasingly complicated and often contain multiple RPCs in parallel or layered accesses. The complications could easily introduce tricky bugs around multi-threaded programming, which may even not be sensed by users and difficult to debug and reproduce. Moreover, implementations either support synchronous patterns only, or have totally different code for asynchronous patterns. Take running some code after completions of multiple asynchronous RPCs as an example, the synchronous pattern is often implemented as issuing multiple RPCs asynchronously and waiting for the completions respectively, while the asynchronous pattern is often implemented by a callback plus a referential count, which is decreased by one when one RPC completes. The callback is called when the count hits zero. Let's see drawbacks of the solution:
- The code is inconsistent between synchronous and asynchronous pattern and it's not trivial for users to move from one pattern to another. From the designing point of view, inconsistencies implies that the essence is probably not grasped yet.
- Cancellation is unlikely to be supported. It's not easy to cancel a single RPC correctly, let alone combinations of RPC. However cancellations are necessary to end pointless waiting in many scenarios.
- Not composable. It's hard to enclose the implementations above as one part of a "larger" pattern. The code can hardly be reused in a different scenario.
We need a better abstraction. If several channels are combined into a larger one with different access patterns enclosed, users would be able to do synchronous, asynchronous, cancelling operations with consistent and unified interfaces. The kind of channels are called combo channels in brpc.
# ParallelChannel
`ParallelChannel` (referred to as "pchan" sometimes) sends requests to all internal sub channels in parallel and merges the responses. Users can modify requests via `CallMapper` and merge responses with `ResponseMerger`. `ParallelChannel` looks like a `Channel`:
- Support synchronous and asynchronous accesses.
- Can be destroyed immediately after initiating an asynchronous operation.
- Support cancellation.
- Support timeout.
Check [example/parallel_echo_c++](https://github.com/apache/brpc/tree/master/example/parallel_echo_c++/) for an example.
Any subclasses of `brpc::ChannelBase` can be added into `ParallelChannel`, including `ParallelChannel` and other combo channels.
Set `ParallelChannelOptions.fail_limit` to control maximum number of failures. When number of failed responses reaches the limit, the RPC is ended immediately rather than waiting for timeout.
Set `ParallelChannelOptions.sucess_limit` to control maximum number of successful responses. When number of successful responses reaches the limit, the RPC is ended immediately.`ParallelChannelOptions.fail_limit` has a higher priority than `ParallelChannelOptions.success_limit`. Success_limit will take effect only when fail_limit is not set.
A sub channel can be added to the same `ParallelChannel` more than once, which is useful when you need to initiate multiple asynchronous RPC to the same service and wait for their completions.
Following picture shows internal structure of `ParallelChannel` (Chinese in red: can be different from request/response respectively)
![img](../images/pchan.png)
## Add sub channel
A sub channel can be added into `ParallelChannel` by following API:
```c++
int AddChannel(brpc::ChannelBase* sub_channel,
ChannelOwnership ownership,
CallMapper* call_mapper,
ResponseMerger* response_merger);
```
When `ownership` is `brpc::OWNS_CHANNEL`, `sub_channel` is destroyed when the `ParallelChannel` destructs. Although a sub channel may be added into a `ParallelChannel` multiple times, it's deleted for at most once when `ownership` in one of the additions is `brpc::OWNS_CHANNEL`.
Calling ` AddChannel` during a RPC over `ParallelChannel` is **NOT thread safe**.
## CallMapper
This class converts RPCs to `ParallelChannel` to the ones to `sub channel`. If `call_mapper` is NULL, requests to the sub channel is just the ones to `ParallelChannel`, and responses are created by calling `New()` on the responses to `ParallelChannel`. `call_mapper` is deleted when `ParallelChannel` destructs. Due to the reference counting inside, one `call_mapper` can be associated with multiple sub channels.
```c++
class CallMapper {
public:
virtual ~CallMapper();
virtual SubCall Map(int channel_index/*starting from 0*/,
int channel_count,
const google::protobuf::MethodDescriptor* method,
const google::protobuf::Message* request,
google::protobuf::Message* response) = 0;
};
```
### Map
`channel_index`: The position of the sub channel inside `ParallelChannel`, starting from zero.
`channel_count`: The sub channel count inside `ParallelChannel`.
`method/request/response`: Parameters to `ParallelChannel::CallMethod()`.
The returned `SubCall` configures the calls to the corresponding sub channel and has two special values:
- `SubCall::Bad()`: The call to ParallelChannel fails immediately and `Controller::ErrorCode()` is set to `EREQUEST`.
- `SubCall::Skip()`: Skip the call to this sub channel. If all sub channels are skipped, the call to ParallelChannel fails immediately and `Controller::ErrorCode()` is set to `ECANCELED`.
Common implementations of `Map()` are listed below:
- Broadcast the request. This is also the behavior when `call_mapper` is NULL:
```c++
class Broadcaster : public CallMapper {
public:
SubCall Map(int channel_index/*starting from 0*/,
const google::protobuf::MethodDescriptor* method,
const google::protobuf::Message* request,
google::protobuf::Message* response) {
// Use the method/request to pchan.
// response is created by `new` and the last flag tells pchan to delete response after completion of the RPC
return SubCall(method, request, response->New(), DELETE_RESPONSE);
}
};
```
- Modify some fields in the request before sending:
```c++
class ModifyRequest : public CallMapper {
public:
SubCall Map(int channel_index/*starting from 0*/,
const google::protobuf::MethodDescriptor* method,
const google::protobuf::Message* request,
google::protobuf::Message* response) {
FooRequest* copied_req = brpc::Clone<FooRequest>(request);
copied_req->set_xxx(...);
// Copy and modify the request
// The last flag tells pchan to delete the request and response after completion of the RPC
return SubCall(method, copied_req, response->New(), DELETE_REQUEST | DELETE_RESPONSE);
}
};
```
- request/response already contains sub requests/responses, use them directly.
```c++
class UseFieldAsSubRequest : public CallMapper {
public:
SubCall Map(int channel_index/*starting from 0*/,
const google::protobuf::MethodDescriptor* method,
const google::protobuf::Message* request,
google::protobuf::Message* response) {
if (channel_index >= request->sub_request_size()) {
// Not enough sub_request. The caller doesn't provide same number of requests as number of sub channels in pchan
// Return Bad() to end this RPC immediately
return SubCall::Bad();
}
// Fetch the sub request and add a new sub response.
// The last flag(0) tells pchan that there is nothing to delete.
return SubCall(sub_method, request->sub_request(channel_index), response->add_sub_response(), 0);
}
};
```
### MapController
`channel_index`: The position of the sub channel inside `ParallelChannel`, starting from zero.
`channel_count`: The sub channel count inside `ParallelChannel`.
`main_cntl`：Parameters to `ParallelChannel::CallMethod()`.
`sub_cntl`：The controller corresponding to the sub-channel's requests. Default implementation: Copy the http_request and request_attachment of `main_cntl` to the `sub_cntl`.
Note: Modifying `ClientSettings` configurations (such as timeout and retries) is ineffective because all sub controllers use the `ClientSettings` configuration of `main_cntl`.
## ResponseMerger
`response_merger` merges responses from all sub channels into one for the `ParallelChannel`. When it's NULL, `response->MergeFrom(*sub_response)` is used instead, whose behavior can be summarized as "merge repeated fields and overwrite the rest". If you need more complex behavior, implement `ResponseMerger`. Multiple `response_merger` are called one by one to merge sub responses so that you do not need to consider the race conditions between merging multiple responses simultaneously. The object is deleted when `ParallelChannel ` destructs. Due to the reference counting inside, `response_merger ` can be associated with multiple sub channels.
Possible values of `Result` are:
- MERGED: Successfully merged.
- FAIL: The `sub_response` was not merged successfully, counted as one failure. For example, there are 10 sub channels and `fail_limit` is 4, if 4 merges return FAIL, the RPC would reach fail_limit and end soon.
- FAIL_ALL: Directly fail the RPC.
## Get the controller to each sub channel
Sometimes users may need to know the details around sub calls. `Controller.sub(i)` gets the controller corresponding to a sub channel.
```c++
// Get the controllers for accessing sub channels in combo channels.
// Ordinary channel:
// sub_count() is 0 and sub() is always NULL.
// ParallelChannel/PartitionChannel:
// sub_count() is #sub-channels and sub(i) is the controller for
// accessing i-th sub channel inside ParallelChannel, if i is outside
// [0, sub_count() - 1], sub(i) is NULL.
// NOTE: You must test sub() against NULL, ALWAYS. Even if i is inside
// range, sub(i) can still be NULL:
// * the rpc call may fail and terminate before accessing the sub channel
// * the sub channel was skipped
// SelectiveChannel/DynamicPartitionChannel:
// sub_count() is always 1 and sub(0) is the controller of successful
// or last call to sub channels.
int sub_count() const;
const Controller* sub(int index) const;
```
# SelectiveChannel
[SelectiveChannel](https://github.com/apache/brpc/blob/master/src/brpc/selective_channel.h) (referred to as "schan" sometimes) accesses one of the internal sub channels with a load balancing algorithm. It's more high-level compared to ordinary channels: The requests are sent to sub channels instead of servers directly. `SelectiveChannel` is mainly for load balancing between groups of machines and shares basic properties of `Channel`:
- Support synchronous and asynchronous accesses.
- Can be destroyed immediately after initiating an asynchronous operation.
- Support cancellation.
- Support timeout.
Check [example/selective_echo_c++](https://github.com/apache/brpc/tree/master/example/selective_echo_c++/) for an example.
Any subclasses of `brpc::ChannelBase` can be added into `SelectiveChannel`, including `SelectiveChannel` and other combo channels.
Retries done by `SelectiveChannel` are independent from the ones in its sub channels. When a call to one of the sub channels fails(which may have been retried), other sub channels are retried.
Currently `SelectiveChannel` requires **the request remains valid before completion of the RPC**, while other combo or regular channels do not. If you plan to use `SelectiveChannel` asynchronously, make sure that the request is deleted inside `done`.
## Using SelectiveChannel
The initialization of `SelectiveChannel` is almost the same as regular `Channel`, except that it doesn't need a naming service in `Init`, because `SelectiveChannel` adds sub channels dynamically by `AddChannel`, while regular `Channel` adds servers in the naming service.
```c++
#include <brpc/selective_channel.h>
...
brpc::SelectiveChannel schan;
brpc::ChannelOptions schan_options;
schan_options.timeout_ms = ...;
schan_options.backup_request_ms = ...;
schan_options.max_retry = ...;
if (schan.Init(load_balancer, &schan_options) != 0) {
LOG(ERROR) << "Fail to init SelectiveChannel";
return -1;
}
```
After successful initialization, add sub channels with `AddChannel`.
```c++
// The second parameter ChannelHandle is used to delete sub channel,
// which can be NULL if this isn't necessary.
if (schan.AddChannel(sub_channel, NULL/*ChannelHandle*/) != 0) {
LOG(ERROR) << "Fail to add sub_channel";
return -1;
}
```
Note that:
- Unlike `ParallelChannel`, `SelectiveChannel::AddChannel` can be called at any time, even if a RPC over the SelectiveChannel is going on. (newly added channels take effects at the next RPC).
- `SelectiveChannel` always owns sub channels, which is different from `ParallelChannel`'s configurable ownership.
- If the second parameter to `AddChannel` is not NULL, it's filled with a value typed `brpc::SelectiveChannel::ChannelHandle`, which can be used as the parameter to `RemoveAndDestroyChannel` to remove and destroy a channel dynamically.
- `SelectiveChannel` overrides timeouts in sub channels. For example, having timeout set to 100ms for a sub channel and 500ms for `SelectiveChannel`, the actual timeout is 500ms.
`SelectiveChannel`s are accessed same as regular channels.
## Example: divide traffic to multiple naming services
Sometimes we need to divide traffic to multiple naming services, because:
- Machines for one service are listed in multiple naming services.
- Machines are split into multiple groups. Requests are sent to one of the groups and then routed to one of the machines inside the group, and traffic are divided differently between groups and machines in a group.
Above requirements can be achieved by `SelectiveChannel`.
Following code creates a `SelectiveChannel` and inserts 3 regular channels for different naming services respectively.
```c++
brpc::SelectiveChannel channel;
brpc::ChannelOptions schan_options;
schan_options.timeout_ms = FLAGS_timeout_ms;
schan_options.max_retry = FLAGS_max_retry;
if (channel.Init("c_murmurhash", &schan_options) != 0) {
LOG(ERROR) << "Fail to init SelectiveChannel";
return -1;
}
for (int i = 0; i < 3; ++i) {
brpc::Channel* sub_channel = new brpc::Channel;
if (sub_channel->Init(ns_node_name[i], "rr", NULL) != 0) {
LOG(ERROR) << "Fail to init sub channel " << i;
return -1;
}
if (channel.AddChannel(sub_channel, NULL/*handle for removal*/) != 0) {
LOG(ERROR) << "Fail to add sub_channel to channel";
return -1;
}
}
...
XXXService_Stub stub(&channel);
stub.FooMethod(&cntl, &request, &response, NULL);
...
```
# PartitionChannel
[PartitionChannel](https://github.com/apache/brpc/blob/master/src/brpc/partition_channel.h) is a specialized `ParallelChannel` to add sub channels automatically based on tags defined in the naming service. As a result, users can list all machines in one naming service and partition them by tags. Check [example/partition_echo_c++](https://github.com/apache/brpc/tree/master/example/partition_echo_c++/) for an example.
`ParititonChannel` only supports one kind to partitioning method. When multiple methods need to coexist, or one method needs to be changed to another smoothly, try `DynamicPartitionChannel`, which creates corresponding sub `PartitionChannel` for different partitioning methods, and divide traffic to partitions according to capacities of servers. Check [example/dynamic_partition_echo_c++](https://github.com/apache/brpc/tree/master/example/dynamic_partition_echo_c++/) for an example.
If partitions are listed in different naming services, users have to implement the partitioning by `ParallelChannel` and include sub channels to corresponding naming services respectively. Refer to [the previous section](#ParallelChannel) for usages of `ParellelChannel`.
## Using PartitionChannel
First of all, implement your own `PartitionParser`. In this example, the tag's format is `N/M`, where N is index of the partition and M is total number of partitions. `0/3` means that there're 3 partitions and this is the first one of them.
```c++
#include <brpc/partition_channel.h>
...
class MyPartitionParser : public brpc::PartitionParser {
public:
bool ParseFromTag(const std::string& tag, brpc::Partition* out) {
// "N/M" : #N partition of M partitions.
size_t pos = tag.find_first_of('/');
if (pos == std::string::npos) {
LOG(ERROR) << "Invalid tag=" << tag;
return false;
}
char* endptr = NULL;
out->index = strtol(tag.c_str(), &endptr, 10);
if (endptr != tag.data() + pos) {
LOG(ERROR) << "Invalid index=" << butil::StringPiece(tag.data(), pos);
return false;
}
out->num_partition_kinds = strtol(tag.c_str() + pos + 1, &endptr, 10);
if (endptr != tag.c_str() + tag.size()) {
LOG(ERROR) << "Invalid num=" << tag.data() + pos + 1;
return false;
}
return true;
}
};
```
Then initialize the `PartitionChannel`.
```c++
#include <brpc/partition_channel.h>
...
brpc::PartitionChannel channel;
brpc::PartitionChannelOptions options;
options.protocol = ...; // PartitionChannelOptions inherits ChannelOptions
options.timeout_ms = ...; // Same as above
options.fail_limit = 1; // PartitionChannel's own settting, which means the same as that of
// ParalellChannel. fail_limit=1 means the overall RPC will fail
// as long as only 1 paratition fails
if (channel.Init(num_partition_kinds, new MyPartitionParser(),
server_address, load_balancer, &options) != 0) {
LOG(ERROR) << "Fail to init PartitionChannel";
return -1;
}
// The RPC interface is the same as regular Channel
```
## Using DynamicPartitionChannel
`DynamicPartitionChannel` and `PartitionChannel` are basically same on usages. Implement `PartitionParser` first then initialize the channel, which does not need `num_partition_kinds` since `DynamicPartitionChannel` dynamically creates sub `PartitionChannel` for each partition.
Following sections demonstrate how to use `DynamicPartitionChannel` to migrate from 3 partitions to 4 partitions smoothly.
First of all, start 3 servers serving on port 8004, 8005, 8006 respectively.
```
$ ./echo_server -server_num 3
TRACE: 09-06 10:40:39: * 0 server.cpp:159] EchoServer is serving on port=8004
TRACE: 09-06 10:40:39: * 0 server.cpp:159] EchoServer is serving on port=8005
TRACE: 09-06 10:40:39: * 0 server.cpp:159] EchoServer is serving on port=8006
TRACE: 09-06 10:40:40: * 0 server.cpp:192] S[0]=0 S[1]=0 S[2]=0 [total=0]
TRACE: 09-06 10:40:41: * 0 server.cpp:192] S[0]=0 S[1]=0 S[2]=0 [total=0]
TRACE: 09-06 10:40:42: * 0 server.cpp:192] S[0]=0 S[1]=0 S[2]=0 [total=0]
```
Note that each server prints summaries on traffic received in last second, which is all 0 now.
Start a client using `DynamicPartitionChannel`, which is initialized as follows:
```c++
...
brpc::DynamicPartitionChannel channel;
brpc::PartitionChannelOptions options;
// Failure on any single partition fails the RPC immediately. You can use a more relaxed value
options.fail_limit = 1;
if (channel.Init(new MyPartitionParser(), "file://server_list", "rr", &options) != 0) {
LOG(ERROR) << "Fail to init channel";
return -1;
}
...
```
The content inside the naming service `file://server_list` is:
```
0.0.0.0:8004 0/3 # The first partition of the three
0.0.0.0:8004 1/3 # and so forth
0.0.0.0:8004 2/3
```
All 3 partitions are put on the server on port 8004, so the client begins to send requests to 8004 once started.
```
$ ./echo_client
TRACE: 09-06 10:51:10: * 0 src/brpc/policy/file_naming_service.cpp:83] Got 3 unique addresses from `server_list'
TRACE: 09-06 10:51:10: * 0 src/brpc/socket.cpp:779] Connected to 0.0.0.0:8004 via fd=3 SocketId=0 self_port=46544
TRACE: 09-06 10:51:11: * 0 client.cpp:226] Sending EchoRequest at qps=132472 latency=371
TRACE: 09-06 10:51:12: * 0 client.cpp:226] Sending EchoRequest at qps=132658 latency=370
TRACE: 09-06 10:51:13: * 0 client.cpp:226] Sending EchoRequest at qps=133208 latency=369
```
At the same time, the server on 8004 received tripled traffic due to the 3 partitions.
```
TRACE: 09-06 10:51:11: * 0 server.cpp:192] S[0]=398866 S[1]=0 S[2]=0 [total=398866]
TRACE: 09-06 10:51:12: * 0 server.cpp:192] S[0]=398117 S[1]=0 S[2]=0 [total=398117]
TRACE: 09-06 10:51:13: * 0 server.cpp:192] S[0]=398873 S[1]=0 S[2]=0 [total=398873]
```
Add new 4 partitions on the server on port 8005.
```
0.0.0.0:8004 0/3
0.0.0.0:8004 1/3
0.0.0.0:8004 2/3
0.0.0.0:8005 0/4
0.0.0.0:8005 1/4
0.0.0.0:8005 2/4
0.0.0.0:8005 3/4
```
Notice how summaries change. The client is aware of the modification to `server_list` and reloads it, but the QPS hardly changes.
```
TRACE: 09-06 10:57:10: * 0 src/brpc/policy/file_naming_service.cpp:83] Got 7 unique addresses from `server_list'
TRACE: 09-06 10:57:10: * 0 src/brpc/socket.cpp:779] Connected to 0.0.0.0:8005 via fd=7 SocketId=768 self_port=39171
TRACE: 09-06 10:57:11: * 0 client.cpp:226] Sending EchoRequest at qps=135346 latency=363
TRACE: 09-06 10:57:12: * 0 client.cpp:226] Sending EchoRequest at qps=134201 latency=366
TRACE: 09-06 10:57:13: * 0 client.cpp:226] Sending EchoRequest at qps=137627 latency=356
TRACE: 09-06 10:57:14: * 0 client.cpp:226] Sending EchoRequest at qps=136775 latency=359
TRACE: 09-06 10:57:15: * 0 client.cpp:226] Sending EchoRequest at qps=139043 latency=353
```
The server-side summary changes more obviously. The server on port 8005 has received requests and the proportion between traffic to 8004 and 8005 is roughly 3:4.
```
TRACE: 09-06 10:57:09: * 0 server.cpp:192] S[0]=398597 S[1]=0 S[2]=0 [total=398597]
TRACE: 09-06 10:57:10: * 0 server.cpp:192] S[0]=392839 S[1]=0 S[2]=0 [total=392839]
TRACE: 09-06 10:57:11: * 0 server.cpp:192] S[0]=334704 S[1]=83219 S[2]=0 [total=417923]
TRACE: 09-06 10:57:12: * 0 server.cpp:192] S[0]=206215 S[1]=273873 S[2]=0 [total=480088]
TRACE: 09-06 10:57:13: * 0 server.cpp:192] S[0]=204520 S[1]=270483 S[2]=0 [total=475003]
TRACE: 09-06 10:57:14: * 0 server.cpp:192] S[0]=207055 S[1]=273725 S[2]=0 [total=480780]
TRACE: 09-06 10:57:15: * 0 server.cpp:192] S[0]=208453 S[1]=276803 S[2]=0 [total=485256]
```
The traffic proportion between 8004 and 8005 is 3:4, considering that each RPC needs 3 calls to 8004 or 4 calls to 8005, the client issues requests to both partitioning methods in 1:1 manner, which depends on capacities calculated recursively:
- The capacity of a regular `Channel` is sum of capacities of servers that it addresses. Capacity of a server is 1 by default if the naming services does not configure weights.
- The capacity of `ParallelChannel` or `PartitionChannel` is the minimum of all its sub channel's.
- The capacity of `SelectiveChannel` is the sum of all its sub channel's.
- The capacity of `DynamicPartitionChannel` is the sum of all its sub `PartitionChannel`'s.
In this example, capacities of the 3-partition method and the 4-partition method are both 1, since the 3 partitions are all on the server on 8004 and the 4 partitions are all on the server on 8005.
Add the server on 8006 to the 4-partition method:
```
0.0.0.0:8004 0/3
0.0.0.0:8004 1/3
0.0.0.0:8004 2/3
0.0.0.0:8005 0/4
0.0.0.0:8005 1/4
0.0.0.0:8005 2/4
0.0.0.0:8005 3/4
0.0.0.0:8006 0/4
0.0.0.0:8006 1/4
0.0.0.0:8006 2/4
0.0.0.0:8006 3/4
```
The client still hardly changes.
```
TRACE: 09-06 11:11:51: * 0 src/brpc/policy/file_naming_service.cpp:83] Got 11 unique addresses from `server_list'
TRACE: 09-06 11:11:51: * 0 src/brpc/socket.cpp:779] Connected to 0.0.0.0:8006 via fd=8 SocketId=1280 self_port=40759
TRACE: 09-06 11:11:51: * 0 client.cpp:226] Sending EchoRequest at qps=131799 latency=372
TRACE: 09-06 11:11:52: * 0 client.cpp:226] Sending EchoRequest at qps=136217 latency=361
TRACE: 09-06 11:11:53: * 0 client.cpp:226] Sending EchoRequest at qps=133531 latency=368
TRACE: 09-06 11:11:54: * 0 client.cpp:226] Sending EchoRequest at qps=136072 latency=361
```
Notice the traffic on 8006 at the server side. The capacity of the 3-partition method is still 1 while capacity of the 4-partition method increases to 2 due to the addition of the server on 8006, thus the overall proportion between the methods becomes 3:8. Each partition inside the 4-partition method has 2 instances on 8005 and 8006 respectively, between which the round-robin load balancing is applied to split the traffic. As a result, the traffic proportion between the 3 servers becomes 3:4:4.
```
TRACE: 09-06 11:11:51: * 0 server.cpp:192] S[0]=199625 S[1]=263226 S[2]=0 [total=462851]
TRACE: 09-06 11:11:52: * 0 server.cpp:192] S[0]=143248 S[1]=190717 S[2]=159756 [total=493721]
TRACE: 09-06 11:11:53: * 0 server.cpp:192] S[0]=133003 S[1]=178328 S[2]=178325 [total=489656]
TRACE: 09-06 11:11:54: * 0 server.cpp:192] S[0]=135534 S[1]=180386 S[2]=180333 [total=496253]
```
See what happens if one partition in the 3-partition method is removed: (You can comment one line in file://server_list by #)
```
0.0.0.0:8004 0/3
0.0.0.0:8004 1/3
#0.0.0.0:8004 2/3
0.0.0.0:8005 0/4
0.0.0.0:8005 1/4
0.0.0.0:8005 2/4
0.0.0.0:8005 3/4
0.0.0.0:8006 0/4
0.0.0.0:8006 1/4
0.0.0.0:8006 2/4
0.0.0.0:8006 3/4
```
The client senses the change in `server_list`:
```
TRACE: 09-06 11:17:47: * 0 src/brpc/policy/file_naming_service.cpp:83] Got 10 unique addresses from `server_list'
TRACE: 09-06 11:17:47: * 0 client.cpp:226] Sending EchoRequest at qps=131653 latency=373
TRACE: 09-06 11:17:48: * 0 client.cpp:226] Sending EchoRequest at qps=120560 latency=407
TRACE: 09-06 11:17:49: * 0 client.cpp:226] Sending EchoRequest at qps=124100 latency=395
TRACE: 09-06 11:17:50: * 0 client.cpp:226] Sending EchoRequest at qps=123743 latency=397
```
The traffic on 8004 drops to zero quickly at the server side. The reason is that the 3-partition method is not complete anymore once the last 2/3 partition has been removed. The capacity becomes zero and no requests are sent to the server on 8004 anymore.
```
TRACE: 09-06 11:17:47: * 0 server.cpp:192] S[0]=130864 S[1]=174499 S[2]=174548 [total=479911]
TRACE: 09-06 11:17:48: * 0 server.cpp:192] S[0]=20063 S[1]=230027 S[2]=230098 [total=480188]
TRACE: 09-06 11:17:49: * 0 server.cpp:192] S[0]=0 S[1]=245961 S[2]=245888 [total=491849]
TRACE: 09-06 11:17:50: * 0 server.cpp:192] S[0]=0 S[1]=250198 S[2]=250150 [total=500348]
```
In real online environments, we gradually increase the number of instances on the 4-partition method and removes instances on the 3-partition method. `DynamicParititonChannel` divides the traffic based on capacities of all partitions dynamically. When capacity of the 3-partition method drops to 0, we've smoothly migrated all servers from 3 partitions to 4 partitions without changing the client-side code.
docs/en/connections.md
+7
View File
@@ -0,0 +1,7 @@
# connections
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/connections.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/consistent_hashing.md
+7
View File
@@ -0,0 +1,7 @@
# consistent hashing
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/consistent_hashing.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/contention_profiler.md
+7
View File
@@ -0,0 +1,7 @@
# contention profiler
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/contention_profiler.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/coroutine.md
+7
View File
@@ -0,0 +1,7 @@
# coroutine
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/coroutine.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/couchbase_example.md
+1209
View File
File diff suppressed because it is too large Load Diff
docs/en/cpu_profiler.md
+7
View File
@@ -0,0 +1,7 @@
# cpu profiler
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/cpu_profiler.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/dummy_server.md
+24
View File
@@ -0,0 +1,24 @@
If your program only uses client in brpc or doesn't use brpc at all, but you also want to use built-in services in brpc. The thing you should do is to start an empty server, which is called **dummy server**.
# client in brpc is used
Create a file named dummy_server.port which contains a port number(such as 8888) in the running directory of program, a dummy server would be started at this port. All of the bvar in the same process can be seen by visiting its built-in service.
![img](../images/dummy_server_1.png) ![img](../images/dummy_server_2.png)
![img](../images/dummy_server_3.png)
# brpc is not used at all
You must manually add the dummy server. First read [Getting Started](getting_started.md) to learn how to download and compile brpc, and then add the following code snippet at the program entry:
```c++
#include <brpc/server.h>
...
int main() {
...
brpc::StartDummyServerAt(8888/*port*/);
...
}
```
docs/en/endpoint.md
+7
View File
@@ -0,0 +1,7 @@
# endpoint
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/endpoint.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/error_code.md
+88
View File
@@ -0,0 +1,88 @@
[中文版](../cn/error_code.md)
brpc use [brpc::Controller](https://github.com/apache/brpc/blob/master/src/brpc/controller.h) to set and get parameters for one RPC. `Controller::ErrorCode()` and `Controller::ErrorText()` return error code and description of the RPC respectively, only accessible after completion of the RPC, otherwise the result is undefined. `ErrorText()` is defined by the base class of the Controller: `google::protobuf::RpcController`, while `ErrorCode()` is defined by `brpc::Controller`. Controller also has a method `Failed()` to tell whether RPC fails or not. Relations between the three methods:
- When `Failed()` is true, `ErrorCode()` must be non-zero and `ErrorText()` be non-empty.
- When `Failed()` is false, `ErrorCode()` is 0 and `ErrorText()` is undefined (it's empty in brpc currently, but you'd better not rely on this)
# Mark RPC as failed
Both client and server in brpc have `Controller`, which can be set with `setFailed()` to modify ErrorCode and ErrorText. Multiple calls to `Controller::SetFailed` leave the last ErrorCode and **concatenate** ErrorTexts rather than leaving the last one. The framework elaborates ErrorTexts by adding extra prefixes: number of retries at client-side and address of the server at server-side.
`Controller::SetFailed()` at client-side is usually called by the framework, such as sending failure, incomplete response, and so on. Error may be set at client-side under some situations. For example, you may set error to the RPC if an additional check before sending the request is failed.
`Controller::SetFailed()` at server-side is often called by the user in the service callback. Generally speaking when error occurs, users call `SetFailed()`, release all the resources, and return from the callback. The framework fills the error code and message into the response according to communication protocol. When the response is received, the error inside are set into the client-side Controller so that users can fetch them after end of RPC. Note that **server does not print errors to clients by default**, as frequent loggings may impact performance of the server significantly due to heavy disk IO. A client crazily producing errors could slow the entire server down and affect all other clients, which can even become an attacking method against the server. If you really want to see error messages on the server, turn on the gflag **-log_error_text** (modifiable at run-time), the server will log the ErrorText of corresponding Controller of each failed RPC.
# Error Code in brpc
All error codes in brpc are defined in [errno.proto](https://github.com/apache/brpc/blob/master/src/brpc/errno.proto), in which those begin with *SYS_* are defined by linux system and exactly same with the ones defined in `/usr/include/errno.h`. The reason that we put it in .proto is to cross language. The rest of the error codes are defined by brpc.
[berror(error_code)](https://github.com/apache/brpc/blob/master/src/butil/errno.h) gets description for the error code, and `berror()` gets description for current [system errno](http://www.cplusplus.com/reference/cerrno/errno/). Note that **ErrorText() != berror(ErorCode())** since `ErrorText()` contains more specific information. brpc includes berror by default so that you can use it in your project directly.
Following table shows common error codes and their descriptions:
| Error Code | Value | Retry | Description | Logging message |
| -------------- | ----- | ----- | ---------------------------------------- | ---------------------------------------- |
| EAGAIN | 11 | Yes | Too many requests at the same time, hardly happening as it's a soft limit. | Resource temporarily unavailable |
| ENODATA | 61 | 是 | 1. The server list returned by Naming Service is empty. 2. When Naming Service changes with all instances modified, Naming Service updates LB by first Remove all and then Add all, the LB instance list may become empty within a short period of time. | Fail to select server from xxx |
| ETIMEDOUT | 110 | Yes | Connection timeout. | Connection timed out |
| EHOSTDOWN | 112 | Yes | Possible reasons: A. The list returned by Naming Server is not empty, but LB cannot select an available server, and LB returns an EHOSTDOWN error. Specific possible reasons: a. Server is exiting (returned ELOGOFF) b. Server was blocked because of some previous failure, the specific logic of the block: 1. For single connection type, the only connection socket is blocked by SetFail, and there are many occurrences of SetFailed in the code to trigger this block. 2. For pooled/short connection type, only when the error number meets does_error_affect_main_socket (ECONNREFUSED, ENETUNREACH, EHOSTUNREACH or EINVAL) will it be blocked 3. After blocking, there is a CheckHealth thread to do health check, Just try to connect, the check interval is controlled by the health_check_interval_s of SocketOptions, and the Socket will be unblocked if it is connected successfully. B. Use the SingleServer method to initialize the Channel (without LB), and the only connection is LOGOFF or blocked (same as above) | "Fail to select server from …" "Not connected to … yet" |
| ENOSERVICE | 1001 | No | Can't locate the service, hardly happening and usually being ENOMETHOD instead | |
| ENOMETHOD | 1002 | No | Can't locate the method. | Misc forms, common ones are "Fail to find method=…" |
| EREQUEST | 1003 | No | fail to serialize the request, may be set on either client-side or server-side | Misc forms: "Missing required fields in request: …" "Fail to parse request message, …" "Bad request" |
| EAUTH | 1004 | No | Authentication failed | "Authentication failed" |
| ETOOMANYFAILS | 1005 | No | Too many sub-channel failures inside a ParallelChannel | "%d/%d channels failed, fail_limit=%d" |
| EBACKUPREQUEST | 1007 | Yes | Set when backup requests are triggered. Not returned by ErrorCode() directly, viewable from spans in /rpcz | "reached backup timeout=%dms" |
| ERPCTIMEDOUT | 1008 | No | RPC timeout. | "reached timeout=%dms" |
| EFAILEDSOCKET | 1009 | Yes | The connection is broken during RPC | "The socket was SetFailed" |
| EHTTP | 1010 | No | HTTP responses with non 2xx status code are treated as failure and set with this code. No retry by default, changeable by customizing RetryPolicy. | Bad http call |
| EOVERCROWDED | 1011 | Yes | Too many messages to buffer at the sender side. Usually caused by lots of concurrent asynchronous requests. Modifiable by `-socket_max_unwritten_bytes`, 64MB by default. | The server is overcrowded |
| EINTERNAL | 2001 | No | The default error for `Controller::SetFailed` without specifying a one. | Internal Server Error |
| ERESPONSE | 2002 | No | fail to serialize the response, may be set on either client-side or server-side | Misc forms: "Missing required fields in response: …" "Fail to parse response message, " "Bad response" |
| ELOGOFF | 2003 | Yes | Server has been stopped | "Server is going to quit" |
| ELIMIT | 2004 | Yes | Number of requests being processed concurrently exceeds `ServerOptions.max_concurrency` | "Reached server's limit=%d on concurrent requests" |
# User-defined Error Code
In C/C++, error code can be defined in macros, constants or enums:
```c++
#define ESTOP -114 // C/C++
static const int EMYERROR = 30; // C/C++
const int EMYERROR2 = -31; // C++ only
```
If you need to get the error description through `berror`, register it in the global scope of your c/cpp file by `BAIDU_REGISTER_ERRNO(error_code, description)`, for example:
```c++
BAIDU_REGISTER_ERRNO(ESTOP, "the thread is stopping")
BAIDU_REGISTER_ERRNO(EMYERROR, "my error")
```
Note that `strerror` and `strerror_r` do not recognize error codes defined by `BAIDU_REGISTER_ERRNO`. Neither does the `%m` used in `printf`. You must use `%s` paired with `berror`:
```c++
errno = ESTOP;
printf("Describe errno: %m\n"); // [Wrong] Describe errno: Unknown error -114
printf("Describe errno: %s\n", strerror_r(errno, NULL, 0)); // [Wrong] Describe errno: Unknown error -114
printf("Describe errno: %s\n", berror()); // [Correct] Describe errno: the thread is stopping
printf("Describe errno: %s\n", berror(errno)); // [Correct] Describe errno: the thread is stopping
```
When the registration of an error code is duplicated, a linking error is generated provided it's defined in C++:
```
redefinition of `class BaiduErrnoHelper<30>'
```
Or the program aborts before start:
```
Fail to define EMYERROR(30) which is already defined as `Read-only file system', abort
```
You have to make sure that different modules have same understandings on same ErrorCode. Otherwise, interactions between two modules that interpret an error code differently may be undefined. To prevent this from happening, you'd better follow these:
- Prefer system error codes which have fixed values and meanings, generally.
- Share code on error definitions between multiple modules to prevent inconsistencies after modifications.
- Use `BAIDU_REGISTER_ERRNO` to describe new error code to ensure that same error code is defined only once inside a process.
docs/en/execution_queue.md
+7
View File
@@ -0,0 +1,7 @@
# execution queue
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/execution_queue.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/flags.md
+7
View File
@@ -0,0 +1,7 @@
# flags
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/flags.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/flatmap.md
+7
View File
@@ -0,0 +1,7 @@
# flatmap
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/flatmap.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/getting_started.md
+396
View File
@@ -0,0 +1,396 @@
[中文版](../cn/getting_started.md)
# BUILD
brpc prefers static linkages of deps, so that they don't have to be installed on every machine running the app.
brpc depends on following packages:
* [gflags](https://github.com/gflags/gflags): Extensively used to define global options.
* [protobuf](https://github.com/google/protobuf): Serializations of messages, interfaces of services.
* [leveldb](https://github.com/google/leveldb): Required by [/rpcz](rpcz.md) to record RPCs for tracing.
# Supported Environment
* [Ubuntu/LinuxMint/WSL](#ubuntulinuxmintwsl)
* [Fedora/CentOS](#fedoracentos)
* [Linux with self-built deps](#linux-with-self-built-deps)
* [MacOS](#macos)
* [Docker](#docker)
## Ubuntu/LinuxMint/WSL
### Prepare deps
Install common deps, [gflags](https://github.com/gflags/gflags), [protobuf](https://github.com/google/protobuf), [leveldb](https://github.com/google/leveldb):
```shell
sudo apt-get install -y git g++ make libssl-dev libgflags-dev libprotobuf-dev libprotoc-dev protobuf-compiler libleveldb-dev
```
If you need to statically link leveldb:
```shell
sudo apt-get install -y libsnappy-dev
```
If you need to enable cpu/heap profilers in examples:
```shell
sudo apt-get install -y libgoogle-perftools-dev
```
If you need to run tests, install and compile libgtest-dev (which is not compiled yet):
```shell
sudo apt-get install -y cmake libgtest-dev && cd /usr/src/gtest && sudo cmake . && sudo make && sudo mv lib/libgtest* /usr/lib/ && cd -
```
The directory of gtest source code may be changed, try `/usr/src/googletest/googletest` if `/usr/src/gtest` is not there.
### Compile brpc with config_brpc.sh
git clone brpc, cd into the repo and run
```shell
$ sh config_brpc.sh --headers=/usr/include --libs=/usr/lib
$ make
```
To change compiler to clang, add `--cxx=clang++ --cc=clang`.
To not link debugging symbols, add `--nodebugsymbols` and compiled binaries will be much smaller.
To use brpc with glog, add `--with-glog`.
To enable [thrift support](../en/thrift.md), install thrift first and add `--with-thrift`.
**Run example**
```shell
$ cd example/echo_c++
$ make
$ ./echo_server &
$ ./echo_client
```
Examples link brpc statically, if you need to link the shared version, `make clean` and `LINK_SO=1 make`
**Run tests**
```shell
$ cd test
$ make
$ sh run_tests.sh
```
### Compile brpc with cmake
```shell
mkdir build && cd build && cmake .. && cmake --build . -j6
```
With CMake 3.13+, we can also use the following commands to build the project:
```shell
cmake -B build && cmake --build build -j6
```
To help VSCode or Emacs(LSP) to understand code correctly, add `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` to generate `compile_commands.json`
To change compiler to clang, overwrite environment variable `CC` and `CXX` to `clang` and `clang++` respectively.
To not link debugging symbols, remove `build/CMakeCache.txt` and cmake with `-DWITH_DEBUG_SYMBOLS=OFF`
To use brpc with glog, cmake with `-DWITH_GLOG=ON`.
To enable [thrift support](../en/thrift.md), install thrift first and cmake with `-DWITH_THRIFT=ON`.
**Run example with cmake**
```shell
$ cd example/echo_c++
$ cmake -B build && cmake --build build -j4
$ ./echo_server &
$ ./echo_client
```
Examples link brpc statically, if you need to link the shared version, remove `CMakeCache.txt` and cmake with `-DLINK_SO=ON`
**Run tests**
```shell
$ mkdir build && cd build && cmake -DBUILD_UNIT_TESTS=ON .. && make && make test
```
### Compile brpc with vcpkg
[vcpkg](https://github.com/microsoft/vcpkg) is a package manager that supports all platforms,
you can use vcpkg to build brpc with the following step:
```shell
$ git clone https://github.com/microsoft/vcpkg.git
$ ./bootstrap-vcpkg.bat # for powershell
$ ./bootstrap-vcpkg.sh # for bash
$ ./vcpkg install brpc
```
## Fedora/CentOS
### Prepare deps
CentOS needs to install EPEL generally otherwise many packages are not available by default.
```shell
sudo yum install epel-release
```
Install common deps:
```shell
sudo yum install git gcc-c++ make openssl-devel
```
Install [gflags](https://github.com/gflags/gflags), [protobuf](https://github.com/google/protobuf), [leveldb](https://github.com/google/leveldb):
```shell
sudo yum install gflags-devel protobuf-devel protobuf-compiler leveldb-devel
```
If you need to enable cpu/heap profilers in examples:
```shell
sudo yum install gperftools-devel
```
If you need to run tests, install and compile gtest-devel (which is not compiled yet):
```shell
sudo yum install gtest-devel
```
### Compile brpc with config_brpc.sh
git clone brpc, cd into the repo and run
```shell
$ sh config_brpc.sh --headers="/usr/include" --libs="/usr/lib64 /usr/bin"
$ make
```
To change compiler to clang, add `--cxx=clang++ --cc=clang`.
To not link debugging symbols, add `--nodebugsymbols` and compiled binaries will be much smaller.
To use brpc with glog, add `--with-glog`.
To enable [thrift support](../en/thrift.md), install thrift first and add `--with-thrift`.
**Run example**
```shell
$ cd example/echo_c++
$ make
$ ./echo_server &
$ ./echo_client
```
Examples link brpc statically, if you need to link the shared version, `make clean` and `LINK_SO=1 make`
**Run tests**
```shell
$ cd test
$ make
$ sh run_tests.sh
```
### Compile brpc with cmake
Same with [here](#compile-brpc-with-cmake)
## Linux with self-built deps
### Prepare deps
brpc builds itself to both static and shared libs by default, so it needs static and shared libs of deps to be built as well.
Take [gflags](https://github.com/gflags/gflags) as example, which does not build shared lib by default, you need to pass options to `cmake` to change the behavior:
```shell
$ cmake . -DBUILD_SHARED_LIBS=1 -DBUILD_STATIC_LIBS=1
$ make
```
### Compile brpc
Keep on with the gflags example, let `../gflags_dev` be where gflags is cloned.
git clone brpc. cd into the repo and run
```shell
$ sh config_brpc.sh --headers="../gflags_dev /usr/include" --libs="../gflags_dev /usr/lib64"
$ make
```
Here we pass multiple paths to `--headers` and `--libs` to make the script search for multiple places. You can also group all deps and brpc into one directory, then pass the directory to --headers/--libs which actually search all subdirectories recursively and will find necessary files.
To change compiler to clang, add `--cxx=clang++ --cc=clang`.
To not link debugging symbols, add `--nodebugsymbols` and compiled binaries will be much smaller.
To use brpc with glog, add `--with-glog`.
To enable [thrift support](../en/thrift.md), install thrift first and add `--with-thrift`.
```shell
$ ls my_dev
gflags_dev protobuf_dev leveldb_dev brpc_dev
$ cd brpc_dev
$ sh config_brpc.sh --headers=.. --libs=..
$ make
```
### Compile brpc with cmake
Same with [here](#compile-brpc-with-cmake)
## MacOS
Note: With same environment, the performance of the MacOS version is worse than the Linux version. If your service is performance-critical, do not use MacOS as your production environment.
### Apple Silicon
The code at master HEAD already supports M1 series chips. M2 series are not tested yet. Please feel free to report remaining warnings/errors to us by issues.
## Docker
Compile brpc with docker:
```shell
$ mkdir -p ~/brpc
$ cd ~/brpc
$ git clone https://github.com/apache/brpc.git
$ cd brpc
$ docker build -t brpc:master .
$ docker images
$ docker run -it brpc:master /bin/bash
```
### Prepare deps
Install dependencies:
```shell
brew install ./homebrew-formula/protobuf.rb
brew install openssl git gnu-getopt coreutils gflags leveldb
```
If you need to enable cpu/heap profilers in examples:
```shell
brew install gperftools
```
If you need to run tests, googletest is required. Run `brew install googletest` first to see if it works. If not (old homebrew does not have googletest), you can download and compile googletest by your own:
```shell
git clone https://github.com/google/googletest -b release-1.10.0 && cd googletest/googletest && mkdir build && cd build && cmake -DCMAKE_CXX_FLAGS="-std=c++11" .. && make
```
After the compilation, copy `include/` and `lib/` into `/usr/local/include` and `/usr/local/lib` respectively to expose gtest to all apps
### OpenSSL
openssl installed in Monterey may not be found at `/usr/local/opt/openssl`, instead it's probably put under `/opt/homebrew/Cellar`. If the compiler cannot find openssl：
* Run `brew link openssl --force` first and check if `/usr/local/opt/openssl` appears.
* If above command does not work, consider making a soft link using `sudo ln -s /opt/homebrew/Cellar/openssl@3/3.0.3 /usr/local/opt/openssl`. Note that the installed openssl in above command may be put in different places in different environments, which could be revealed by running `brew info openssl`.
### Compile brpc with config_brpc.sh
git clone brpc, cd into the repo and run
```shell
$ sh config_brpc.sh --headers=/usr/local/include --libs=/usr/local/lib --cc=clang --cxx=clang++
$ make
```
The homebrew in Monterey may install software at different directories from before. If path related errors are reported, try setting headers/libs like below:
```shell
$ sh config_brpc.sh --headers=/opt/homebrew/include --libs=/opt/homebrew/lib --cc=clang --cxx=clang++
$ make
```
To not link debugging symbols, add `--nodebugsymbols` and compiled binaries will be much smaller.
To use brpc with glog, add `--with-glog`.
To enable [thrift support](../en/thrift.md), install thrift first and add `--with-thrift`.
**Run example**
```shell
$ cd example/echo_c++
$ make
$ ./echo_server &
$ ./echo_client
```
Examples link brpc statically, if you need to link the shared version, `make clean` and `LINK_SO=1 make`
**Run tests**
```shell
$ cd test
$ make
$ sh run_tests.sh
```
### Compile brpc with cmake
Same with [here](#compile-brpc-with-cmake)
# Supported deps
## GCC: 4.8-11.2
**Prefer GCC 8.2+**
c++11 is turned on by default to remove dependencies on boost (atomic).
The over-aligned issues in GCC7 is suppressed temporarily now.
Using other versions of gcc may generate warnings, contact us to fix.
Adding `-D__const__=__unused__` to cxxflags in your makefiles is a must to avoid [errno issue in gcc4+](thread_local.md).
## Clang: 3.5-4.0
no known issues.
## glibc: 2.12-2.25
no known issues.
## protobuf: 3.0-5.29
bRPC uses some protobuf internal APIs, which may be changed upstream.
Please [submit issue](https://github.com/apache/brpc/issues) if you have any problem.
[#2406](https://github.com/apache/brpc/pull/2406) and [#2493](https://github.com/apache/brpc/pull/2493) in [version 1.8.0]((https://github.com/apache/brpc/releases/tag/1.8.0)) introduce some proto3 syntax, so currently bRPC is no longer compatible with pb 2.x version. If you want to use pb 2.x version, you can use bRPC version before 1.8.0.
## gflags: 2.0-2.2.2
[gflags patch](https://github.com/gflags/gflags/commit/408061b46974cc8377a8a794a048ecae359ad887) is required when compiled with 2.1.1.
## openssl: 0.97-1.1
required by https.
## tcmalloc: 1.7-2.5
brpc does **not** link [tcmalloc](http://goog-perftools.sourceforge.net/doc/tcmalloc.html) by default. Users link tcmalloc on-demand.
Comparing to ptmalloc embedded in glibc, tcmalloc often improves performance. However different versions of tcmalloc may behave really differently. For example, tcmalloc 2.1 may make multi-threaded examples in brpc perform significantly worse(due to a spinlock in tcmalloc) than the one using tcmalloc 1.7 and 2.5. Even different minor versions may differ. When you program behave unexpectedly, remove tcmalloc or try another version.
Code compiled with gcc 4.8.2 and linked to a tcmalloc compiled with earlier GCC may crash or deadlock before main(), E.g:
![img](../images/tcmalloc_stuck.png)
When you meet the issue, compile tcmalloc with the same GCC.
Another common issue with tcmalloc is that it does not return memory to system as early as ptmalloc. So when there's an invalid memory access, the program may not crash directly, instead it crashes at a unrelated place, or even not crash. When you program has weird memory issues, try removing tcmalloc.
If you want to use [cpu profiler](cpu_profiler.md) or [heap profiler](heap_profiler.md), do link `libtcmalloc_and_profiler.a`. These two profilers are based on tcmalloc.[contention profiler](contention_profiler.md) does not require tcmalloc.
When you remove tcmalloc, not only remove the linkage with tcmalloc but also the macro `-DBRPC_ENABLE_CPU_PROFILER`.
## glog: 3.3+
brpc implements a default [logging utility](../../src/butil/logging.h) which conflicts with glog. To replace this with glog, add `--with-glog` to config_brpc.sh or add `-DWITH_GLOG=ON` to cmake.
## valgrind: 3.8+
brpc detects valgrind automatically (and registers stacks of bthread). Older valgrind(say 3.2) is not supported.
## thrift: 0.9.3-0.11.0
## libunwind: 1.3-1.8.1
brpc does **not** link [libunwind](https://github.com/libunwind/libunwind) by default. Users link libunwind on-demand by adding `--with-bthread-tracer` to config_brpc.sh or adding `-DWITH_BTHREAD_TRACER=ON` to cmake, if building with Bazel, please add the `--define with_bthread_tracer=true` option.
It is recommended to use the latest possible version of libunwind.
no known issues.
# Track instances
We provide a program to help you to track and monitor all brpc instances. Just run [trackme_server](https://github.com/apache/brpc/tree/master/tools/trackme_server/) somewhere and launch need-to-be-tracked instances with -trackme_server=SERVER. The trackme_server will receive pings from instances periodically and print logs when it does. You can aggregate instance addresses from the log and call builtin services of the instances for further information.
docs/en/heap_profiler.md
+7
View File
@@ -0,0 +1,7 @@
# heap profiler
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/heap_profiler.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/http_client.md
+263
View File
@@ -0,0 +1,263 @@
[中文版](../cn/http_client.md)
# Example
[example/http_c++](https://github.com/apache/brpc/blob/master/example/http_c++/http_client.cpp)
# About h2
brpc names the HTTP/2 protocol to "h2", no matter encrypted or not. However HTTP/2 connections without SSL are shown on /connections with the official name "h2c", and the ones with SSL are shown as "h2".
The APIs for http and h2 in brpc are basically same. Without explicit statement, mentioned http features work for h2 as well.
# Create Channel
In order to use `brpc::Channel` to access http/h2 services, `ChannelOptions.protocol` must be set to `PROTOCOL_HTTP` or `PROTOCOL_H2`.
Once the protocol is set, the first parameter of `Channel::Init` can be any valid URL. *Note*: Only host and port inside the URL are used by Init(), other parts are discarded. Allowing full URL simply saves the user from additional parsing code.
```c++
brpc::ChannelOptions options;
options.protocol = brpc::PROTOCOL_HTTP; // or brpc::PROTOCOL_H2
if (channel.Init("www.baidu.com" /*any url*/, &options) != 0) {
LOG(ERROR) << "Fail to initialize channel";
return -1;
}
```
http/h2 channel also support BNS address or other naming services.
# GET
```c++
brpc::Controller cntl;
cntl.http_request().uri() = "www.baidu.com/index.html"; // Request URL
channel.CallMethod(NULL, &cntl, NULL, NULL, NULL/*done*/);
```
http/h2 does not relate to protobuf much, thus all parameters of `CallMethod` are NULL except `Controller` and `done`. Issue asynchronous RPC with non-NULL `done`.
`cntl.response_attachment()` is body of the http/h2 response and typed `butil::IOBuf`. `IOBuf` can be converted to `std::string` by `to_string()`, which needs to allocate memory and copy all data. If performance is important, the code should consider supporting `IOBuf` directly rather than requiring continuous memory.
# POST
The default HTTP Method is GET, which can be changed to POST or [other http methods](https://github.com/apache/brpc/blob/master/src/brpc/http_method.h). The data to POST should be put into `request_attachment()`, which is typed [butil::IOBuf](https://github.com/apache/brpc/blob/master/src/butil/iobuf.h) and able to append `std :: string` or `char *` directly.
```c++
brpc::Controller cntl;
cntl.http_request().uri() = "..."; // Request URL
cntl.http_request().set_method(brpc::HTTP_METHOD_POST);
cntl.request_attachment().append("{\"message\":\"hello world!\"}");
channel.CallMethod(NULL, &cntl, NULL, NULL, NULL/*done*/);
```
If the body needs a lot of printing to build, consider using `butil::IOBufBuilder`, which has same interfaces as `std::ostringstream`, probably simpler and more efficient than c-style printf when lots of objects need to be printed.
```c++
brpc::Controller cntl;
cntl.http_request().uri() = "..."; // Request URL
cntl.http_request().set_method(brpc::HTTP_METHOD_POST);
butil::IOBufBuilder os;
os << "A lot of printing" << printable_objects << ...;
os.move_to(cntl.request_attachment());
channel.CallMethod(NULL, &cntl, NULL, NULL, NULL/*done*/);
```
# Change HTTP version
brpc behaves as http/1.1 by default.
Comparing to http/1.1, http/1.0 lacks of long connections(KeepAlive). To communicate brpc client with some legacy http servers, the client may be configured as follows:
```c++
cntl.http_request().set_version(1, 0);
```
Setting http version does not work for h2, but the versions in h2 responses received by client and h2 requests received by server are set to (2, 0).
brpc server recognizes http versions automically and responds accordingly without users' aid.
# URL
Genaral form of an URL:
```
// URI scheme : http://en.wikipedia.org/wiki/URI_scheme
//
// foo://username:password@example.com:8042/over/there/index.dtb?type=animal&name=narwhal#nose
// \_/ \_______________/ \_________/ \__/ \___/ \_/ \______________________/ \__/
// | | | | | | | |
// | userinfo host port | | query fragment
// | \________________________________/\_____________|____|/ \__/ \__/
// scheme | | | | | |
// authority | | | | |
// path | | interpretable as keys
// | |
// \_______________________________________________|____|/ \____/ \_____/
// | | | | |
// hierarchical part | | interpretable as values
// | |
// interpretable as filename |
// |
// |
// interpretable as extension
```
As we saw in examples above, `Channel.Init()` and `cntl.http_request().uri()` both need the URL. Why does `uri()` need to be set additionally rather than using the URL to `Init()` directly?
Indeed, the settings are repeated in simple cases. But they are different in more complex scenes:
- Access multiple servers under a NamingService (for example BNS), in which case `Channel::Init` accepts a name meaningful to the NamingService(for example node names in BNS), while `uri()` is assigned with the URL.
- Access servers via http/h2 proxy, in which case `Channel::Init` takes the address of the proxy server, while `uri()` is still assigned with the URL.
## Host header
If user already sets `Host` header(case insensitive), framework makes no change.
If user does not set `Host` header and the URL has host, for example http://www.foo.com/path, the http request contains "Host: www.foo.com".
If user does not set host header and the URL does not have host either, for example "/index.html?name=value", but if the channel is initlized by a http(s) address with valid domain name. framework sets `Host` header with domain name of the target server. if this address is "http://www.foo.com", this http server should see `Host: www.foo.com`, if this address is "http://www.foo.com:8989", this http server should be see `Host: www.foo.com:8989`.
If user does not set host header and the URL does not have host as well, for example "/index.html?name=value", and the address initialized by the channel doesn't contain domain name. framework sets `Host` header with IP and port of the target server. A http server at 10.46.188.39:8989 should see `Host: 10.46.188.39:8989`.
The header is named ":authority" in h2.
# Common usages
Take http request as an example (similar with http response), common operations are listed as follows:
Access an HTTP header named `Foo`
```c++
const std::string* value = cntl->http_request().GetHeader("Foo"); // NULL when not exist
```
Set an HTTP header named `Foo`
```c++
cntl->http_request().SetHeader("Foo", "value");
```
Access a query named `Foo`
```c++
const std::string* value = cntl->http_request().uri().GetQuery("Foo"); // NULL when not exist
```
Set a query named `Foo`
```c++
cntl->http_request().uri().SetQuery("Foo", "value");
```
Set HTTP method
```c++
cntl->http_request().set_method(brpc::HTTP_METHOD_POST);
```
Set the URL
```c++
cntl->http_request().uri() = "http://www.baidu.com";
```
Set the `content-type`
```c++
cntl->http_request().set_content_type("text/plain");
```
Get HTTP body
```c++
butil::IOBuf& buf = cntl->request_attachment();
std::string str = cntl->request_attachment().to_string(); // trigger copy underlying
```
Set HTTP body
```c++
cntl->request_attachment().append("....");
butil::IOBufBuilder os; os << "....";
os.move_to(cntl->request_attachment());
```
Notes on http header:
- field_name of the header is case-insensitive according to [rfc2616](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2). brpc supports case-insensitive field names and keeps same cases at printing as users set.
- If multiple headers have same field names, according to [rfc2616](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2), values should be merged and separated by comma (,). Users figure out how to use this kind of values by their own.
- Queries are separated by "&" and key/value in a query are separated by "=". Values can be omitted. For example, `key1=value1&key2&key3=value3` is a valid query string, in which the value for `key2` is an empty string.
# Debug HTTP messages
Turn on [-http_verbose](http://brpc.baidu.com:8765/flags/http_verbose) so that the framework prints each http request and response. Note that this should only be used in tests or debuggings rather than online services.
# HTTP errors
When server returns a non-2xx HTTP status code, the HTTP RPC is considered to be failed and `cntl->ErrorCode()` at client-side is set to `EHTTP`, users can check `cntl-> http_response().status_code()` for more specific HTTP error. In addition, server can put html or json describing the error into `cntl->response_attachment()` which is sent back to the client as http body.
If client-side wants to get the real `ErrorCode` returned by the bRPC server when the HTTP RPC fails, instead of `EHTTP`, you need to set GFlag`-use_http_error_code=true`.
# Compress Request Body
`Controller::set_request_compress_type(brpc::COMPRESS_TYPE_GZIP)` makes framework try to gzip the HTTP body. "try to" means the compression may not happen, because:
* Size of body is smaller than bytes specified by -http_body_compress_threshold, which is 512 by default. The reason is that gzip is not a very fast compression algorithm, when body is small, the delay caused by compression may even larger than the latency saved by faster transportation.
# Decompress Response Body
brpc does not decompress bodies of responses automatically due to universality. The decompression code is not complicated and users can do it by themselves. The code is as follows:
```c++
#include <brpc/policy/gzip_compress.h>
...
const std::string* encoding = cntl->http_response().GetHeader("Content-Encoding");
if (encoding != NULL && *encoding == "gzip") {
butil::IOBuf uncompressed;
if (!brpc::policy::GzipDecompress(cntl->response_attachment(), &uncompressed)) {
LOG(ERROR) << "Fail to un-gzip response body";
return;
}
cntl->response_attachment().swap(uncompressed);
}
// Now cntl->response_attachment() contains the decompressed data
```
# Progressively Download
http client normally does not complete the RPC until http body has been fully downloaded. During the process http body is stored in memory. If the body is very large or infinitely large(a FLV file for live streaming), memory grows continuously until the RPC is timedout. Such http clients are not suitable for downloading very large files.
brpc client supports completing RPC before reading the full body, so that users can read http bodies progressively after RPC. Note that this feature does not mean "support for http chunked mode", actually the http implementation in brpc supports chunked mode from the very beginning. The real issue is how to let users handle very or infinitely large http bodies, which does not imply the chunked mode.
How to use:
1. Implement ProgressiveReader below:
```c++
#include <brpc/progressive_reader.h>
...
class ProgressiveReader {
public:
// Called when one part was read.
// Error returned is treated as *permanent* and the socket where the
// data was read will be closed.
// A temporary error may be handled by blocking this function, which
// may block the HTTP parsing on the socket.
virtual butil::Status OnReadOnePart(const void* data, size_t length) = 0;
// Called when there's nothing to read anymore. The `status' is a hint for
// why this method is called.
// - status.ok(): the message is complete and successfully consumed.
// - otherwise: socket was broken or OnReadOnePart() failed.
// This method will be called once and only once. No other methods will
// be called after. User can release the memory of this object inside.
virtual void OnEndOfMessage(const butil::Status& status) = 0;
};
```
`OnReadOnePart` is called each time a piece of data is read. `OnEndOfMessage` is called at the end of data or the connection is broken. Read comments carefully before implementing.
2. Set `cntl.response_will_be_read_progressively();` before RPC to make brpc end RPC just after reading all headers.
3. Call `cntl.ReadProgressiveAttachmentBy(new MyProgressiveReader);` after RPC. `MyProgressiveReader` is an instance of user-implemented `ProgressiveReader`. User may delete the object inside `OnEndOfMessage`.
# Progressively Upload
Currently the POST data should be intact before launching the http call, thus brpc http client is still not suitable for uploading very large bodies.
# Access Servers with authentications
Generate `auth_data` according to authenticating method of the server and set it into `Authorization` header. If you're using curl, add option `-H "Authorization : <auth_data>"`.
# Send https requests
https is short for "http over SSL", SSL is not exclusive for http, but effective for all protocols. The generic method for turning on client-side SSL is [here](client.md#turn-on-ssl). brpc enables SSL automatically for URIs starting with https:// to make the usage more handy.
docs/en/http_derivatives.md
+37
View File
@@ -0,0 +1,37 @@
[中文版](../cn/http_derivatives.md)
Basics for accessing and serving http/h2 in brpc are listed in [http_client](http_client.md) and [http_service](http_service.md).
Following section names are protocol names that can be directly set to ChannelOptions.protocol. The content after colon is parameters for the protocol to select derivative behaviors dynamically, but the base protocol is still http/1.x or http/2. As a result, these protocols are displayed at server-side as http or h2/h2c only.
# http:json, h2:json
Non-empty pb request is serialized to json and set to the body of the http/h2 request. The Controller.request_attachment() must be empty otherwise the RPC fails.
Non-empty pb response is converted from a json which is parsed from the body of the http/h2 response.
http/1.x behaves in this way by default, so "http" and "http:json" are just same.
# http:proto, h2:proto
Non-empty pb request is serialized (in pb's wire format) and set to the body of the http/h2 request. The Controller.request_attachment() must be empty otherwise the RPC fails.
Non-empty pb response is parsed from the body of the http/h2 response(in pb's wire format).
http/2 behaves in this way by default, so "h2" and "h2:proto" are just same.
# h2:grpc
Default protocol of [gRPC](https://github.com/grpc). The detailed format is described in [gRPC over HTTP2](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md).
Clients using brpc should be able to talk with gRPC after changing ChannelOptions.protocol to "h2:grpc".
Servers using brpc should be accessible by gRPC clients automatically without changing the code.
gRPC serializes message into pb wire format by default, so "h2:grpc" and "h2:grpc+proto" are just same.
TODO: Other configurations for gRPC
# h2:grpc+json
Comparing to h2:grpc, this protocol serializes messages into json instead of pb, which may not be supported by gRPC directly. For example, grpc-go users may reference [here](https://github.com/johanbrandhorst/grpc-json-example/blob/master/codec/json.go) to register the corresponding codec and turn on the support.
docs/en/http_service.md
+386
View File
@@ -0,0 +1,386 @@
[中文版](../cn/http_service.md)
This document talks about ordinary http/h2 services rather than protobuf services accessible via http/h2.
http/h2 services in brpc have to declare interfaces with empty request and response in a .proto file. This requirement keeps all service declarations inside proto files rather than scattering in code, configurations, and proto files.
# Example
[http_server.cpp](https://github.com/apache/brpc/blob/master/example/http_c++/http_server.cpp)
# About h2
brpc names the HTTP/2 protocol to "h2", no matter encrypted or not. However HTTP/2 connections without SSL are shown on /connections with the official name "h2c", and the ones with SSL are shown as "h2".
The APIs for http and h2 in brpc are basically same. Without explicit statement, mentioned http features work for h2 as well.
# URL types
## /ServiceName/MethodName as the prefix
Define a service named `ServiceName`(not including the package name), with a method named `MethodName` and empty request/response, the service will provide http/h2 service on `/ServiceName/MethodName` by default.
The reason that request and response can be empty is that all data are in Controller:
- Header of the http/h2 request is in Controller.http_request() and the body is in Controller.request_attachment().
- Header of the http/h2 response is in Controller.http_response() and the body is in Controller.response_attachment().
Implementation steps:
1. Add the service declaration in a proto file.
```protobuf
option cc_generic_services = true;
message HttpRequest { };
message HttpResponse { };
service HttpService {
rpc Echo(HttpRequest) returns (HttpResponse);
};
```
2. Implement the service by inheriting the base class generated in .pb.h, which is same as protobuf services.
```c++
class HttpServiceImpl : public HttpService {
public:
...
virtual void Echo(google::protobuf::RpcController* cntl_base,
const HttpRequest* /*request*/,
HttpResponse* /*response*/,
google::protobuf::Closure* done) {
brpc::ClosureGuard done_guard(done);
brpc::Controller* cntl = static_cast<brpc::Controller*>(cntl_base);
// body is plain text
cntl->http_response().set_content_type("text/plain");
// Use printed query string and body as the response.
butil::IOBufBuilder os;
os << "queries:";
for (brpc::URI::QueryIterator it = cntl->http_request().uri().QueryBegin();
it != cntl->http_request().uri().QueryEnd(); ++it) {
os << ' ' << it->first << '=' << it->second;
}
os << "\nbody: " << cntl->request_attachment() << '\n';
os.move_to(cntl->response_attachment());
}
};
```
3. After adding the implemented instance into the server, the service is accessible via following URLs (Note that the path after `/HttpService/Echo ` is filled into `cntl->http_request().unresolved_path()`, which is always normalized):
| URL | Protobuf Method | cntl->http_request().uri().path() | cntl->http_request().unresolved_path() |
| -------------------------- | ---------------- | --------------------------------- | -------------------------------------- |
| /HttpService/Echo | HttpService.Echo | "/HttpService/Echo" | "" |
| /HttpService/Echo/Foo | HttpService.Echo | "/HttpService/Echo/Foo" | "Foo" |
| /HttpService/Echo/Foo/Bar | HttpService.Echo | "/HttpService/Echo/Foo/Bar" | "Foo/Bar" |
| /HttpService//Echo///Foo// | HttpService.Echo | "/HttpService//Echo///Foo//" | "Foo" |
| /HttpService | No such method | | |
## /ServiceName as the prefix
http/h2 services for managing resources may need this kind of URL, such as `/FileService/foobar.txt` represents `./foobar.txt` and `/FileService/app/data/boot.cfg` represents `./app/data/boot.cfg`.
Implementation steps:
1. Use `FileService` as the service name and `default_method` as the method name in the proto file.
```protobuf
option cc_generic_services = true;
message HttpRequest { };
message HttpResponse { };
service FileService {
rpc default_method(HttpRequest) returns (HttpResponse);
}
```
2. Implement the service.
```c++
class FileServiceImpl: public FileService {
public:
...
virtual void default_method(google::protobuf::RpcController* cntl_base,
const HttpRequest* /*request*/,
HttpResponse* /*response*/,
google::protobuf::Closure* done) {
brpc::ClosureGuard done_guard(done);
brpc::Controller* cntl = static_cast<brpc::Controller*>(cntl_base);
cntl->response_attachment().append("Getting file: ");
cntl->response_attachment().append(cntl->http_request().unresolved_path());
}
};
```
3. After adding the implemented instance into the server, the service is accessible via following URLs (the path after `/FileService` is filled in `cntl->http_request().unresolved_path()`, which is always normalized):
| URL | Protobuf Method | cntl->http_request().uri().path() | cntl->http_request().unresolved_path() |
| ------------------------------- | -------------------------- | --------------------------------- | -------------------------------------- |
| /FileService | FileService.default_method | "/FileService" | "" |
| /FileService/123.txt | FileService.default_method | "/FileService/123.txt" | "123.txt" |
| /FileService/mydir/123.txt | FileService.default_method | "/FileService/mydir/123.txt" | "mydir/123.txt" |
| /FileService//mydir///123.txt// | FileService.default_method | "/FileService//mydir///123.txt//" | "mydir/123.txt" |
## Restful URL
brpc supports specifying a URL for each method in a service. The API is as follows:
```c++
// If `restful_mappings' is non-empty, the method in service can
// be accessed by the specified URL rather than /ServiceName/MethodName.
// Mapping rules: "PATH1 => NAME1, PATH2 => NAME2 ..."
// where `PATH' is a valid path and `NAME' is the method name.
int AddService(google::protobuf::Service* service,
ServiceOwnership ownership,
butil::StringPiece restful_mappings);
```
`QueueService` defined below contains several methods. If the service is added into the server normally, it's accessible via URLs like `/QueueService/start` and ` /QueueService/stop`.
```protobuf
service QueueService {
rpc start(HttpRequest) returns (HttpResponse);
rpc stop(HttpRequest) returns (HttpResponse);
rpc get_stats(HttpRequest) returns (HttpResponse);
rpc download_data(HttpRequest) returns (HttpResponse);
};
```
By specifying the 3rd parameter `restful_mappings` to `AddService`, the URL can be customized:
```c++
if (server.AddService(&queue_svc,
brpc::SERVER_DOESNT_OWN_SERVICE,
"/v1/queue/start => start,"
"/v1/queue/stop => stop,"
"/v1/queue/stats/* => get_stats") != 0) {
LOG(ERROR) << "Fail to add queue_svc";
return -1;
}
if (server.AddService(&queue_svc,
brpc::SERVER_DOESNT_OWN_SERVICE,
"/v1/*/start => start,"
"/v1/*/stop => stop,"
"*.data => download_data") != 0) {
LOG(ERROR) << "Fail to add queue_svc";
return -1;
}
```
There are 3 mappings separated by comma in the 3rd parameter (which is a string spanning 3 lines) to the `AddService`. Each mapping tells brpc to call the method at right side of the arrow if the left side matches the URL. The asterisk in `/v1/queue/stats/*` matches any string.
More about mapping rules:
- Multiple paths can be mapped to a same method.
- Both http/h2 and protobuf services are supported.
- Un-mapped methods are still accessible via `/ServiceName/MethodName`. Mapped methods are **not** accessible via `/ServiceName/MethodName` anymore.
- `==>` and ` ===>` are both OK, namely extra spaces at the beginning or the end, extra slashes, extra commas at the end, are all accepted.
- Pattern `PATH` and `PATH/*` can coexist.
- Support suffix matching: characters can appear after the asterisk.
- At most one asterisk is allowed in a path.
The path after asterisk can be obtained by `cntl.http_request().unresolved_path()`, which is always normalized, namely no slashes at the beginning or the end, and no repeated slashes in the middle. For example:
![img](../images/restful_1.png)
or:
![img](../images/restful_2.png)
in which unresolved_path are both `foo/bar`. The extra slashes at the left, the right, or the middle are removed.
Note that `cntl.http_request().uri().path()` is not ensured to be normalized, which is `"//v1//queue//stats//foo///bar//////"` and `"//vars///foo////bar/////"` respectively in the above example.
The built-in service page of `/status` shows customized URLs after the methods, in form of `@URL1 @URL2` ...
![img](../images/restful_3.png)
# HTTP Parameters
## HTTP headers
HTTP headers are a series of key/value pairs, some of them are defined by the HTTP specification, while others are free to use.
Query strings are also key/value pairs. Differences between HTTP headers and query strings:
* Although operations on HTTP headers are accurately defined by the http specification, but http headers cannot be modified directly from an address bar, they are often used for passing parameters of a protocol or framework.
* Query strings is part of the URL and **often** in form of `key1=value1&key2=value2&...`, which is easy to read and modify. They're often used for passing application-level parameters. However format of query strings is not defined in HTTP spec, just a convention.
```c++
// Get value for header "User-Agent" (case insensitive)
const std::string* user_agent_str = cntl->http_request().GetHeader("User-Agent");
if (user_agent_str != NULL) { // has the header
LOG(TRACE) << "User-Agent is " << *user_agent_str;
}
...
// Add a header "Accept-encoding: gzip" (case insensitive)
cntl->http_response().SetHeader("Accept-encoding", "gzip");
// Overwrite the previous header "Accept-encoding: deflate"
cntl->http_response().SetHeader("Accept-encoding", "deflate");
// Append value to the previous header so that it becomes
// "Accept-encoding: deflate,gzip" (values separated by comma)
cntl->http_response().AppendHeader("Accept-encoding", "gzip");
```
## Content-Type
`Content-type` is a frequently used header for storing type of the HTTP body, and specially processed in brpc and accessible by `cntl->http_request().content_type()` . As a correspondence, `cntl->GetHeader("Content-Type")` returns nothing.
```c++
// Get Content-Type
if (cntl->http_request().content_type() == "application/json") {
...
}
...
// Set Content-Type
cntl->http_response().set_content_type("text/html");
```
If the RPC fails (`Controller` has been `SetFailed`), the framework overwrites `Content-Type` with `text/plain` and sets the response body with `Controller::ErrorText()`.
## Status Code
Status code is a special field in HTTP response to store processing result of the http request. Possible values are defined in [http_status_code.h](https://github.com/apache/brpc/blob/master/src/brpc/http_status_code.h).
```c++
// Get Status Code
if (cntl->http_response().status_code() == brpc::HTTP_STATUS_NOT_FOUND) {
LOG(FATAL) << "FAILED: " << controller.http_response().reason_phrase();
}
...
// Set Status code
cntl->http_response().set_status_code(brpc::HTTP_STATUS_INTERNAL_SERVER_ERROR);
cntl->http_response().set_status_code(brpc::HTTP_STATUS_INTERNAL_SERVER_ERROR, "My explanation of the error...");
```
For example, following code implements redirection with status code 302:
```c++
cntl->http_response().set_status_code(brpc::HTTP_STATUS_FOUND);
cntl->http_response().SetHeader("Location", "http://bj.bs.bae.baidu.com/family/image001(4979).jpg");
```
![img](../images/302.png)
## Query String
As mentioned in above [HTTP headers](#http-headers), query strings are interpreted in common convention, whose form is `key1=value1&key2=value2&…`. Keys without values are acceptable as well and accessible by `GetQuery` which returns an empty string. Such keys are often used as boolean flags. Full API are defined in [uri.h](https://github.com/apache/brpc/blob/master/src/brpc/uri.h).
```c++
const std::string* time_value = cntl->http_request().uri().GetQuery("time");
if (time_value != NULL) { // the query string is present
LOG(TRACE) << "time = " << *time_value;
}
...
cntl->http_request().uri().SetQuery("time", "2015/1/2");
```
# Debugging
Turn on [-http_verbose](http://brpc.baidu.com:8765/flags/http_verbose) to print contents of all http requests and responses. Note that this should only be used for debugging rather than online services.
# Compress the response body
HTTP services often compress http bodies to reduce transmission latency of web pages and speed up the presentations to end users.
Call `Controller::set_response_compress_type(brpc::COMPRESS_TYPE_GZIP)` to **try to** compress the http body with gzip. "Try to" means the compression may not happen in following conditions:
* The request does not set `Accept-encoding` or the value does not contain "gzip". For example, curl does not support compression without option `--compressed`, in which case the server always returns uncompressed results.
* Body size is less than the bytes specified by -http_body_compress_threshold (512 by default). gzip is not a very fast compression algorithm. When the body is small, the delay added by compression may be larger than the time saved by network transmission. No compression when the body is relatively small is probably a better choice.
| Name | Value | Description | Defined At |
| ---------------------------- | ----- | ---------------------------------------- | ------------------------------------- |
| http_body_compress_threshold | 512 | Not compress http body when it's less than so many bytes. | src/brpc/policy/http_rpc_protocol.cpp |
# Decompress the request body
Due to generality, brpc does not decompress request bodies automatically, but users can do the job by themselves as follows:
```c++
#include <brpc/policy/gzip_compress.h>
...
const std::string* encoding = cntl->http_request().GetHeader("Content-Encoding");
if (encoding != NULL && *encoding == "gzip") {
butil::IOBuf uncompressed;
if (!brpc::policy::GzipDecompress(cntl->request_attachment(), &uncompressed)) {
LOG(ERROR) << "Fail to un-gzip request body";
return;
}
cntl->request_attachment().swap(uncompressed);
}
// cntl->request_attachment() contains the data after decompression
```
# Serve https requests
https is short for "http over SSL", SSL is not exclusive for http, but effective for all protocols. The generic method for turning on server-side SSL is [here](server.md#turn-on-ssl).
# Performance
Productions without extreme performance requirements tend to use HTTP protocol, especially mobile products. Thus we put great emphasis on implementation qualities of HTTP. To be more specific:
- Use [http parser](https://github.com/apache/brpc/blob/master/src/brpc/details/http_parser.h) of node.js to parse http messages, which is a lightweight, well-written, and extensively used implementation.
- Use [rapidjson](https://github.com/miloyip/rapidjson) to parse json, which is a json library focuses on performance.
- In the worst case, the time complexity of parsing http requests is still O(N), where N is byte size of the request. As a contrast, parsing code that requires the http request to be complete, may cost O(N^2) time in the worst case. This feature is very helpful since many HTTP requests are large.
- Processing HTTP messages from different clients is highly concurrent, even a pretty complicated http message does not block responding other clients. It's difficult to achieve this for other RPC implementations and http servers often based on [single-threaded reactor](threading_overview.md#single-threaded-reactor).
# Progressive sending
brpc server is capable of sending large or infinite sized body, in following steps:
1. Call `Controller::CreateProgressiveAttachment()` to create a body that can be written progressively. The returned `ProgressiveAttachment` object should be managed by `intrusive_ptr`
```c++
#include <brpc/progressive_attachment.h>
...
butil::intrusive_ptr<brpc::ProgressiveAttachment> pa = cntl->CreateProgressiveAttachment();
```
2. Call `ProgressiveAttachment::Write()` to send the data.
* If the write occurs before running of the server-side done, the sent data is cached until the done is called.
* If the write occurs after running of the server-side done, the sent data is written out in chunked mode immediately.
3. After usage, destruct all `butil::intrusive_ptr<brpc::ProgressiveAttachment>` to release related resources.
In addition, we can easily implement Server-Sent Events(SSE) with this feature, which enables a client to receive automatic updates from a server via a HTTP connection. SSE could be used to build real-time applications such as chatGPT. Please refer to HttpSSEServiceImpl in [http_server.cpp](https://github.com/apache/brpc/blob/master/example/http_c++/http_server.cpp) for more details.
# Progressive receiving
Currently brpc server doesn't support calling the service callback once header part in the http request is parsed. In other words, brpc server is not suitable for receiving large or infinite sized body.
# FAQ
### Q: The nginx before brpc encounters final fail
The error is caused by that brpc server closes the http connection directly without sending a response.
brpc server supports a variety of protocols on the same port. When a request is failed to be parsed in HTTP, it's hard to tell that the request is definitely in HTTP. If the request is very likely to be one, the server sends HTTP 400 errors and closes the connection. However, if the error is caused HTTP method(at the beginning) or ill-formed serialization (may be caused by bugs at the HTTP client), the server still closes the connection without sending a response, which leads to "final fail" at nginx.
Solution: When using Nginx to forward traffic, set `$HTTP_method` to allowed HTTP methods or simply specify the HTTP method in `proxy_method`.
### Q: Does brpc support http chunked mode
Yes.
### Q: Why do HTTP requests containing BASE64 encoded query string fail to parse sometimes?
According to the [HTTP specification](http://tools.ietf.org/html/rfc3986#section-2.2), following characters need to be encoded with `%`.
```
reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="
```
Base64 encoded string may end with `=` which is a reserved character (take `?wi=NDgwMDB8dGVzdA==&anothorkey=anothervalue` as an example). The strings may be parsed successfully, or may be not, depending on the implementation which should not be assumed in principle.
One solution is to remove the trailing `=` which does not affect the [Base64 decoding](http://en.wikipedia.org/wiki/Base64#Padding). Another method is to [percent-encode](https://en.wikipedia.org/wiki/Percent-encoding) the URI, and do percent-decoding before Base64 decoding.
docs/en/io.md
+51
View File
@@ -0,0 +1,51 @@
[中文版](../cn/io.md)
There are three mechanisms to operate IO in general:
- Blocking IO: once an IO operation is issued, the calling thread is blocked until the IO ends, which is a kind of synchronous IO, such as default actions of posix [read](http://linux.die.net/man/2/read) and [write](http://linux.die.net/man/2/write).
- Non-blocking IO: If there is nothing to read or too much to write, APIs that would block return immediately with an error code. Non-blocking IO is often used with IO multiplexing([poll](http://linux.die.net/man/2/poll), [select](http://linux.die.net/man/2/select), [epoll](http://linux.die.net/man/4/epoll) in Linux or [kqueue](https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2) in BSD).
- Asynchronous IO: Start a read or write operation with a callback, which will be called when the IO is done, such as [OVERLAPPED](https://msdn.microsoft.com/en-us/library/windows/desktop/ms684342(v=vs.85).aspx) + [IOCP](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx) in Windows. Native AIO in Linux only supports files.
Non-blocking IO is usually used for increasing IO concurrency in Linux. When the IO concurrency is low, non-blocking IO is not necessarily more efficient than blocking IO, which is handled completely by the kernel. System calls like read/write are highly optimized and probably more efficient. But when IO concurrency increases, the drawback of blocking-one-thread in blocking IO arises: the kernel keeps switching between threads to do effective jobs, and a CPU core may only do a little bit of work before being replaced by another thread, causing CPU cache not fully utilized. In addition a large number of threads decrease performance of code dependent on thread-local variables, such as tcmalloc. Once malloc slows down, the overall performance of the program decreases as well. As a contrast, non-blocking IO is typically composed with a relatively small number of event dispatching threads and worker threads(running user code), which are reused by different tasks (in another word, part of scheduling work is moved to userland). Event dispatchers and workers run on different CPU cores simultaneously without frequent switches in the kernel. There's no need to have many threads, so the use of thread-local variables is also more adequate. All these factors make non-blocking IO faster than blocking IO. But non-blocking IO also has its own problems, one of which is more system calls, such as [epoll_ctl](http://man7.org/linux/man-pages/man2/epoll_ctl.2.html). Since epoll is implemented as a red-black tree, epoll_ctl is not a very fast operation, especially in multi-threaded environments. Implementations heavily dependent on epoll_ctl is often confronted with multi-core scalability issues. Non-blocking IO also has to solve a lot of multi-threaded problems, producing more complex code than blocking IO.
# Receiving messages
A message is a bounded binary data read from a connection, which may be a request from upstream clients or a response from downstream servers. brpc uses one or several [EventDispatcher](https://github.com/apache/brpc/blob/master/src/brpc/event_dispatcher.cpp)(referred to as EDISP) to wait for events from file descriptors. Unlike the common "IO threads", EDISP is not responsible for reading or writing. The problem of IO threads is that one thread can only read one fd at a given time, other reads may be delayed when many fds in one IO thread are busy. Multi-tenancy, complicated load balancing and [Streaming RPC](streaming_rpc.md) worsen the problem. Under high workloads, regular long delays on one fd may slow down all fds in the IO thread, causing more long tails.
Because of a [bug](https://web.archive.org/web/20150423184820/https://patchwork.kernel.org/patch/1970231/) of epoll (at the time of developing brpc) and overhead of epoll_ctl, edge triggered mode is used in EDISP. After receiving an event, an atomic variable associated with the fd is added by one atomically. If the variable is zero before addition, a bthread is started to handle the data from the fd. The pthread worker in which EDISP runs is yielded to the newly created bthread to make it start reading ASAP and have a better cache locality. The bthread in which EDISP runs will be stolen to another pthread and keep running, this mechanism is work stealing used in bthreads. To understand exactly how that atomic variable works, you can read [atomic instructions](atomic_instructions.md) first, then check [Socket::StartInputEvent](https://github.com/apache/brpc/blob/master/src/brpc/socket.cpp). These methods make contentions on dispatching events of one fd [wait-free](http://en.wikipedia.org/wiki/Non-blocking_algorithm#Wait-freedom).
In current implementation, `Transport::ProcessEvent` chooses start mode based on `EventDispatcherUnsched()`: `false` uses `bthread_start_urgent`, and `true` uses `bthread_start_background`. In addition, RDMA handles `last_msg` differently between polling and event modes: when `rdma_use_polling=false`, `RdmaTransport::QueueMessage` does not process `last_msg`; in polling mode it continues to process it. And when `EventDispatcherUnsched()` returns `true`, `last_msg` is not processed directly in the current execution flow but in a new bthread. Users can control this behavior through `event_dispatcher_edisp_unsched`.
[InputMessenger](https://github.com/apache/brpc/blob/master/src/brpc/input_messenger.h) cuts messages and uses customizable callbacks to handle different format of data. `Parse` callback cuts messages from binary data and has relatively stable running time; `Process` parses messages further(such as parsing by protobuf) and calls users' callbacks, which vary in running time. If n(n > 1) messages are read from the fd, InputMessenger launches n-1 bthreads to handle first n-1 messages respectively, and processes the last message in-place. InputMessenger tries protocols one by one. Since one connections often has only one type of messages, InputMessenger remembers current protocol to avoid trying for protocols next time.
It can be seen that messages from different fds or even same fd are processed concurrently in brpc, which makes brpc good at handling large messages and reducing long tails on processing messages from different sources under high workloads.
# Sending Messages
A message is a bounded binary data written to a connection, which may be a response to upstream clients or a request to downstream servers. Multiple threads may send messages to a fd at the same time, however writing to a fd is non-atomic, so how to queue writes from different thread efficiently is a key technique. brpc uses a special wait-free MPSC list to solve the issue. All data ready to write is put into a node of a singly-linked list, whose next pointer points to a special value(`Socket::WriteRequest::UNCONNECTED`). When a thread wants to write out some data, it tries to atomically exchange the node with the list head(Socket::_write_head) first. If the head before exchange is empty, the caller gets the right to write and writes out the data in-place once. Otherwise there must be another thread writing. The caller points the next pointer to the head returned to make the linked list connected. The thread that is writing will see the new head later and write new data.
This method makes the writing contentions wait-free. Although the thread that gets the right to write is not wait-free nor lock-free in principle and may be blocked by a node that is still UNCONNECTED(the thread issuing write is swapped out by OS just after atomic exchange and before setting the next pointer, within execution time of just one instruction), the blocking rarely happens in practice. In current implementations, if the data cannot be written fully in one call, a KeepWrite bthread is created to write the remaining data. This mechanism is pretty complicated and the principle is depicted below. Read [socket.cpp](https://github.com/apache/brpc/blob/master/src/brpc/socket.cpp) for more details.
![img](../images/write.png)
Since writes in brpc always complete within short time, the calling thread can handle new tasks more quickly and background KeepWrite threads also get more tasks to write in one batch, forming pipelines and increasing the efficiency of IO at high throughputs.
# Socket
[Socket](https://github.com/apache/brpc/blob/master/src/brpc/socket.h) contains data structures related to fd and is one of the most complex structure in brpc. The unique feature of this structure is that it uses 64-bit SocketId to refer to a Socket object to facilitate usages of fd in multi-threaded environments. Commonly used methods:
- Create: create a Socket and return its SocketId.
- Address: retrieve Socket from an id, and wrap it into a unique_ptr(SocketUniquePtr) that will be automatically released. When Socket is set failed, the pointer returned is empty. As long as Address returns a non-null pointer, the contents are guaranteed to not change until the pointer is destructed. This function is wait-free.
- SetFailed: Mark a Socket as failed and Address() on corresponding SocketId will return empty pointer (until health checking resumes the socket). Sockets are recycled when no one is referencing it anymore. This function is lock-free.
We can see that, Socket is similar to [shared_ptr](http://en.cppreference.com/w/cpp/memory/shared_ptr) in the sense of referential counting and SocketId is similar to [weak_ptr](http://en.cppreference.com/w/cpp/memory/weak_ptr). The unique `SetFailed` prevents Socket from being addressed so that the reference count can hit zero finally. Simply using shared_ptr/weak_ptr cannot guarantee this. For example, when a server needs to quit when requests are still coming in frequently, the reference count of Socket may not hit zero and the server is unable to stop quickly. What' more, weak_ptr cannot be directly put into epoll data, but SocketId can. These factors lead to design of Socket which is stable and rarely changed since 2014.
Using SocketUniquePtr or SocketId depends on if a strong reference is needed. For example, Controller is used thoroughly inside RPC and has a lot of interactions with Socket, it uses SocketUniquePtr. Epoll notifies events on fds and events of a recycled socket can be ignored, so epoll uses SocketId.
As long as SocketUniquePtr is valid, the Socket enclosed will not be changed so that users have no need to care about race conditions and ABA problems, being safer to operate the shared socket. This method also circumvents implicit referential counting and makes ownership of memory more clear, producing better-quality programs. brpc uses SocketUniquePtr and SocketId a lot to simplify related issues.
In fact, Socket manages not only the native fd but also other resources, such as SubChannel in SelectiveChannel is also managed by Socket, making SelectiveChannel choose a SubChannel just like a normal channel choosing a downstream server. The faked Socket even implements health checking. Streaming RPC also uses Socket to reuse the code on wait-free write.
# The full picture
![img](../images/rpc_flow.png)
docs/en/iobuf.md
+103
View File
@@ -0,0 +1,103 @@
[中文版](../cn/iobuf.md)
brpc uses [butil::IOBuf](https://github.com/apache/brpc/blob/master/src/butil/iobuf.h) as data structure for attachment in some protocols and HTTP body. It's a non-contiguous zero-copied buffer, proved in previous projects, and good at performance. The interface of `IOBuf` is similar to `std::string`, but not the same.
If you've used the `BufHandle` in Kylin before, you should notice the convenience of `IOBuf`: the former one is badly encapsulated, leaving the internal structure directly in front of users, who must carefully handle the referential countings, very error prone and leading to bugs.
# What IOBuf can:
- Default constructor does not allocate memory.
- Copyable. Modifications to the copy doesn't affect the original one. Copy the managing structure of IOBuf only rather the payload.
- Append another IOBuf without copying payload.
- Can append string, by copying payload.
- Read from or write into file descriptors.
- Serialize to or parse from protobuf messages.
- constructible like a std::ostream using IOBufBuilder.
# What IOBuf can't:
- Used as universal string-like structure in the program. Lifetime of IOBuf should be short, to prevent the referentially counted blocks(8K each) in IOBuf lock too many memory.
# Cut
Cut 16 bytes from front-side of source_buf and append to dest_buf:
```c++
source_buf.cut(&dest_buf, 16); // cut all bytes of source_buf when its length < 16
```
Just pop 16 bytes from front-side of source_buf:
```c++
source_buf.pop_front(16); // Empty source_buf when its length < 16
```
# Append
Append another IOBuf to back-side：
```c++
buf.append(another_buf); // no data copy
```
Append std::string to back-sie
```c++
buf.append(str); // copy data of str into buf
```
# Parse
Parse a protobuf message from the IOBuf
```c++
IOBufAsZeroCopyInputStream wrapper(&iobuf);
pb_message.ParseFromZeroCopyStream(&wrapper);
```
Parse IOBuf in user-defined formats
```c++
IOBufAsZeroCopyInputStream wrapper(&iobuf);
CodedInputStream coded_stream(&wrapper);
coded_stream.ReadLittleEndian32(&value);
...
```
# Serialize
Serialize a protobuf message into the IOBuf
```c++
IOBufAsZeroCopyOutputStream wrapper(&iobuf);
pb_message.SerializeToZeroCopyStream(&wrapper);
```
Built IOBuf with printable data
```c++
IOBufBuilder os;
os << "anything can be sent to std::ostream";
os.buf(); // IOBuf
```
# Print
Directly printable to std::ostream. Note that the iobuf in following example should only contain printable characters.
```c++
std::cout << iobuf << std::endl;
// or
std::string str = iobuf.to_string(); // note: allocating memory
printf("%s\n", str.c_str());
```
# Performance
IOBuf is good at performance:
| Action | Throughput | QPS |
| ---------------------------------------- | ----------- | ------- |
| Read from file -> Cut 12+16 bytes -> Copy -> Merge into another buffer ->Write to /dev/null | 240.423MB/s | 8586535 |
| Read from file -> Cut 12+128 bytes -> Copy-> Merge into another buffer ->Write to /dev/null | 790.022MB/s | 5643014 |
| Read from file -> Cut 12+1024 bytes -> Copy-> Merge into another buffer ->Write to /dev/null | 1519.99MB/s | 1467171 |
docs/en/json2pb.md
+7
View File
@@ -0,0 +1,7 @@
# json2pb
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/json2pb.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/lalb.md
+22
View File
@@ -0,0 +1,22 @@
# Overview
Locality-aware load balancing(LALB) is an algorithm that can send requests to downstream servers with the lowest latency timely and automatically. The algorithm is originated from the DP system and is now added to brpc!
Problems that LALB can solve:
- Round-robin and random policy is not good at scheduling requests since configurations and latencies of downstream servers are different.
- Downstream servers, offline servers and other servers are deployed hybridly and it is hard to predict performance.
- Automatically schedule most of the requests to the server in the same machine. When a problem occurs, try the server acrossing the machines.
- Visit the server in the same server room first. When a problem occurs, try another server room.
**...**
# Background
The most common algorithm to redirect requests is round-robin and random. The premise of these two methods is that the downstream servers and networks are similar. But in the current online environment, especially the hybrid environment, it is difficult to achieve because:
- Each machine runs a different combination of programs, along with some offline tasks, the available resources of the machine are continuously changing.
- The configurations of machines are different.
- The latencies of network are different.
These problems have always been there, but are hidden by machine monitoring from hard-working OPs. There are also some attempts in the level of frameworks. For example, [WeightedStrategy](https://svn.baidu.com/public/trunk/ub/ub_client/ubclient_weightstrategy.h) in UB redirects requests based on cpu usage of downstream machines and obviously it cannot solve the latency-related issues, or even cpu issues: since it is implemented as regularly reloading a list of weights, one can imagine that the update frequency cannot be high. A lot of requests may timeout when the load balancer reacts. And there is a math problem here: how to change cpu usage to weight.
docs/en/load_balancing.md
+7
View File
@@ -0,0 +1,7 @@
# load ualancing
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/load_balancing.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/mbvar_c++.md
+7
View File
@@ -0,0 +1,7 @@
# muvar c++
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/mbvar_c++.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/memcache_client.md
+102
View File
@@ -0,0 +1,102 @@
[中文版](../cn/memcache_client.md)
[memcached](http://memcached.org/) is a common caching service. In order to access memcached more conveniently and make full use of bthread's capability of concurrency, brpc directly supports the memcached protocol. Check [example/memcache_c++](https://github.com/apache/brpc/tree/master/example/memcache_c++/) for an example.
**NOTE**: brpc only supports the binary protocol of memcache. There's little benefit to support the textual protocol which is replaced since memcached 1.3. If your memcached is older than 1.3, upgrade to a newer version.
Advantages compared to [libmemcached](http://libmemcached.org/libMemcached.html) (the official client):
- Thread safety. No need to set up separate clients for each thread.
- Support synchronous, asynchronous, semi-synchronous accesses etc. Support [ParallelChannel etc](combo_channel.md) to define access patterns declaratively.
- Support various [connection types](client.md#connection-type). Support timeout, backup request, cancellation, tracing, built-in services, and other benefits offered by brpc.
- Have the concept of requests and responses while libmemcached don't. Users have to do extra bookkeepings to associate received messages with sent messages, which is not trivial.
The current implementation takes full advantage of the RPC concurrency mechanism and avoids copying as much as possible. A single client can easily pushes a memcached instance (version 1.4.15) on the same machine to its limit: 90,000 QPS for single connection, 330,000 QPS for multiple connections. In most cases, brpc is able to make full use of memcached's capabilities.
# Request a memcached server
Create a `Channel` for accessing memcached:
```c++
#include <brpc/memcache.h>
#include <brpc/channel.h>
brpc::ChannelOptions options;
options.protocol = brpc::PROTOCOL_MEMCACHE;
if (channel.Init("0.0.0.0:11211", &options) != 0) { // 11211 is the default port for memcached
LOG(FATAL) << "Fail to init channel to memcached";
return -1;
}
...
```
Following example tries to set data to memcached:
```c++
// Set key="hello" value="world" flags=0xdeadbeef, expire in 10s, and ignore cas
brpc::MemcacheRequest request;
brpc::MemcacheResponse response;
brpc::Controller cntl;
if (!request.Set("hello", "world", 0xdeadbeef/*flags*/, 10/*expiring seconds*/, 0/*ignore cas*/)) {
LOG(FATAL) << "Fail to SET request";
return -1;
}
channel.CallMethod(NULL, &cntl, &request, &response, NULL/*done*/);
if (cntl.Failed()) {
LOG(FATAL) << "Fail to access memcached, " << cntl.ErrorText();
return -1;
}
if (!response.PopSet(NULL)) {
LOG(FATAL) << "Fail to SET memcached, " << response.LastError();
return -1;
}
...
```
Notes on above code:
- The class of the request must be `MemcacheRequest`, response must be `MemcacheResponse`, otherwise `CallMethod` fails. `stub` is not necessary, just call `channel.CallMethod` with `method` to NULL.
- Call `request.XXX()` to add an operation, where `XXX` is `Set` in this example. Multiple operations inside a request are sent to a memcached server together (often referred to as "pipeline mode").
- call `response.PopXXX()` to pop result of an operation from the response, where `XXX` is `Set` in this example. true is returned on success, and false otherwise in which case use `response.LastError()` to get the error message. `XXX` must match the corresponding operation in the request, otherwise the pop is rejected. In above example, a `PopGet` would fail with the error message of "not a GET response".
- Results of `Pop` are independent from the RPC result. Even if "a value cannot be put into the memcached", the RPC may still be successful. RPC failure means things like broken connection, timeout etc. If the business logic requires the memcache operations to be successful, you should test successfulness of both RPC and `PopXXX`.
Supported operations currently:
```c++
bool Set(const Slice& key, const Slice& value, uint32_t flags, uint32_t exptime, uint64_t cas_value);
bool Add(const Slice& key, const Slice& value, uint32_t flags, uint32_t exptime, uint64_t cas_value);
bool Replace(const Slice& key, const Slice& value, uint32_t flags, uint32_t exptime, uint64_t cas_value);
bool Append(const Slice& key, const Slice& value, uint32_t flags, uint32_t exptime, uint64_t cas_value);
bool Prepend(const Slice& key, const Slice& value, uint32_t flags, uint32_t exptime, uint64_t cas_value);
bool Delete(const Slice& key);
bool Flush(uint32_t timeout);
bool Increment(const Slice& key, uint64_t delta, uint64_t initial_value, uint32_t exptime);
bool Decrement(const Slice& key, uint64_t delta, uint64_t initial_value, uint32_t exptime);
bool Touch(const Slice& key, uint32_t exptime);
bool Version();
```
Corresponding operations in replies:
```c++
// Call LastError() of the response to check the error text when any following operation fails.
bool PopGet(IOBuf* value, uint32_t* flags, uint64_t* cas_value);
bool PopGet(std::string* value, uint32_t* flags, uint64_t* cas_value);
bool PopSet(uint64_t* cas_value);
bool PopAdd(uint64_t* cas_value);
bool PopReplace(uint64_t* cas_value);
bool PopAppend(uint64_t* cas_value);
bool PopPrepend(uint64_t* cas_value);
bool PopDelete();
bool PopFlush();
bool PopIncrement(uint64_t* new_value, uint64_t* cas_value);
bool PopDecrement(uint64_t* new_value, uint64_t* cas_value);
bool PopTouch();
bool PopVersion(std::string* version);
```
# Request a memcached cluster
Create a `Channel` using the `c_md5` as the load balancing algorithm to access a memcached cluster mounted under a naming service. Note that each `MemcacheRequest` should contain only one operation or all operations have the same key. Under current implementation, multiple operations inside a single request are always sent to a same server. If the keys are located on different servers, the result must be wrong. In which case, you have to divide the request into multilple ones with one operation each.
Another choice is to use the common [twemproxy](https://github.com/twitter/twemproxy) solution, which makes clients access the cluster just like accessing a single server, although the solution needs to deploy proxies and adds more latency.
docs/en/memory_management.md
+7
View File
@@ -0,0 +1,7 @@
# memory management
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/memory_management.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/new_protocol.md
+153
View File
@@ -0,0 +1,153 @@
# Multi-protocol support in the server side
brpc server supports all protocols in the same port, and it makes deployment and maintenance more convenient in most of the time. Since the format of different protocols is very different, it is hard to support all protocols in the same port unambiguously. In consider of decoupling and extensibility, it is also hard to build a multiplexer for all protocols. Thus our way is to classify all protocols into three categories and try one by one:
- First-class protocol: Special characters are marked in front of the protocol data, for example, the data of protocol [baidu_std](baidu_std.md) and hulu_pbrpc begins with 'PRPC' and 'HULU' respectively. Parser just check first four characters to know whether the protocol is matched. This class of protocol is checked first and all these protocols can share one TCP connection.
- Second-class protocol: Some complex protocols without special marked characters can only be detected after several input data are parsed. Currently only HTTP is classified into this category.
- Third-class protocol: Special characters are in the middle of the protocol data, such as the magic number of nshead protocol is the 25th-28th characters. It is complex to handle this case because without reading first 28 bytes, we cannot determine whether the protocol is nshead. If it is tried before http, http messages less than 28 bytes may not be parsed, since the parser consider it as an incomplete nshead message.
Considering that there will be only one protocol in most connections, we record the result of last selection so that it will be tried first when further data comes. It reduces the overhead of matching protocols to nearly zero for long connections. Although the process of matching protocols will be run every time for short connections, the bottleneck of short connections is not in here and this method is still fast enough. If there are lots of new protocols added into brpc in the future, we may consider some heuristic methods to match protocols.
# Multi-protocol support in the client side
Unlike the server side that protocols must be dynamically determined based on the data on the connection, the client side as the originator, naturally know their own protocol format. As long as the protocol data is sent through connection pool or short connection, which means it has exclusive usage of that connection, then the protocol can have any complex (or bad) format. Since the client will record the protocol when sending the data, it will use that recorded protocol to parse the data without any matching overhead when responses come back. There is no magic number in some protocols like memcache, redis, it is hard to distinguish them in the server side, but it has no problem in the client side.
# Support new protocols
brpc is designed to add new protocols at any time, just proceed as follows:
> The protocol that begins with nshead has unified support, just read [this](nshead_service.md).
## add ProtocolType
Add new protocol type in ProtocolType in [options.proto](https://github.com/apache/brpc/blob/master/src/brpc/options.proto). If you need to add new protocol, please contact us to add it for you to make sure there is no conflict with protocols of others.
Currently we support in ProtocolType(at the middle of 2018):
```c++
enum ProtocolType {
PROTOCOL_UNKNOWN = 0;
PROTOCOL_BAIDU_STD = 1;
PROTOCOL_STREAMING_RPC = 2;
PROTOCOL_HULU_PBRPC = 3;
PROTOCOL_SOFA_PBRPC = 4;
PROTOCOL_RTMP = 5;
PROTOCOL_HTTP = 6;
PROTOCOL_PUBLIC_PBRPC = 7;
PROTOCOL_NOVA_PBRPC = 8;
PROTOCOL_NSHEAD_CLIENT = 9; // implemented in brpc-ub
PROTOCOL_NSHEAD = 10;
PROTOCOL_HADOOP_RPC = 11;
PROTOCOL_HADOOP_SERVER_RPC = 12;
PROTOCOL_MONGO = 13; // server side only
PROTOCOL_UBRPC_COMPACK = 14;
PROTOCOL_DIDX_CLIENT = 15; // Client side only
PROTOCOL_REDIS = 16; // Client side only
PROTOCOL_MEMCACHE = 17; // Client side only
PROTOCOL_ITP = 18;
PROTOCOL_NSHEAD_MCPACK = 19;
PROTOCOL_DISP_IDL = 20; // Client side only
PROTOCOL_ERSDA_CLIENT = 21; // Client side only
PROTOCOL_UBRPC_MCPACK2 = 22; // Client side only
// Reserve special protocol for cds-agent, which depends on FIFO right now
PROTOCOL_CDS_AGENT = 23; // Client side only
PROTOCOL_ESP = 24; // Client side only
PROTOCOL_THRIFT = 25; // Server side only
}
```
## Implement Callbacks
All callbacks are defined in struct Protocol, which is defined in [protocol.h](https://github.com/apache/brpc/blob/master/src/brpc/protocol.h). Among all these callbacks, `parse` is a callback that must be implemented. Besides, `process_request` must be implemented in the server side and `serialize_request`, `pack_request`, `process_response` must be implemented in the client side.
It is difficult to implement callbacks of the protocol. These codes are not like the codes that ordinary users use which has good prompts and protections. You have to figure it out how to handle similar code in other protocols and implement your own protocol, then send it to us to do code review.
### parse
```c++
typedef ParseResult (*Parse)(butil::IOBuf* source, Socket *socket, bool read_eof, const void *arg);
```
This function is used to cut messages from source. Client side and server side must share the same parse function. The returned message will be passed to `process_request`(server side) or `process_response`(client side).
Argument: source is the binary content from remote side, socket is the corresponding connection, read_eof is true iff the connection is closed by remote, arg is a pointer to the corresponding server in server client and NULL in client side.
ParseResult could be an error or a cut message, its possible value contains:
- PARSE_ERROR_TRY_OTHERS: current protocol is not matched, the framework would try next protocol. The data in source cannot be comsumed.
- PARSE_ERROR_NOT_ENOUGH_DATA: the input data hasn't violated the current protocol yet, but the whole message cannot be detected as well. When there is new data from connection, new data will be appended to source and parse function is called again. If we can determine that data fits current protocol, the content of source can also be transferred to the internal state of protocol. For example, if source doesn't contain a whole http message, it will be consumed by http parser to avoid repeated parsing.
- PARSE_ERROR_TOO_BIG_DATA: message size is too big, the connection will be closed to protect server.
- PARSE_ERROR_NO_RESOURCE: internal error, such as resource allocation failure. Connections will be closed.
- PARSE_ERROR_ABSOLUTELY_WRONG: it is supposed to be some protocols(magic number is matched), but the format is not as expected. Connection will be closed.
### serialize_request
```c++
typedef bool (*SerializeRequest)(butil::IOBuf* request_buf,
Controller* cntl,
const google::protobuf::Message* request);
```
This function is used to serialize request into request_buf that client must implement. It happens before a RPC call and will only be called once. Necessary information needed by some protocols(such as http) is contained in cntl. Return true if succeed, otherwise false.
### pack_request
```c++
typedef int (*PackRequest)(butil::IOBuf* msg,
uint64_t correlation_id,
const google::protobuf::MethodDescriptor* method,
Controller* controller,
const butil::IOBuf& request_buf,
const Authenticator* auth);
```
This function is used to pack request_buf into msg, which is called every time before sending messages to server(including retrying). When auth is not NULL, authentication information is also needed to be packed. Return 0 if succeed, otherwise -1.
### process_request
```c++
typedef void (*ProcessRequest)(InputMessageBase* msg_base);
```
This function is used to parse request messages in the server side that server must implement. It and parse() may run in different threads. Multiple process_request may run simultaneously. After the processing is done, msg_base->Destroy() must be called. In order to prevent forgetting calling Destroy, consider using DestroyingPtr<>.
### process_response
```c++
typedef void (*ProcessResponse)(InputMessageBase* msg);
```
This function is used to parse message response in client side that client must implement. It and parse() may run in different threads. Multiple process_request may run simultaneously. After the processing is done, msg_base->Destroy() must be called. In order to prevent forgetting calling Destroy, consider using DestroyingPtr<>.
### verify
```c++
typedef bool (*Verify)(const InputMessageBase* msg);
```
This function is used to authenticate connections, it is called when the first message is received. It is must be implemented by servers that need authentication, otherwise the function pointer can be NULL. Return true if succeed, otherwise false.
### parse_server_address
```c++
typedef bool (*ParseServerAddress)(butil::EndPoint* out, const char* server_addr_and_port);
```
This function converts server_addr_and_port(an argument of Channel.Init) to butil::EndPoint, which is optional. Some protocols may differ in the expression and understanding of server addresses.
### get_method_name
```c++
typedef const std::string& (*GetMethodName)(const google::protobuf::MethodDescriptor* method,
const Controller*);
```
This function is used to customize method name, which is optional.
### supported_connection_type
Used to mark the supported connection method. If all connection methods are supported, this value should set to CONNECTION_TYPE_ALL. If connection pools and short connections are supported, this value should set to CONNECTION_TYPE_POOLED_AND_SHORT.
### name
The name of the protocol, which appears in the various configurations and displays, should be as short as possible and must be a string constant.
## Register to global
RegisterProtocol should be called to [register implemented protocol](https://github.com/apache/brpc/blob/master/src/brpc/global.cpp) to brpc, just like:
```c++
Protocol http_protocol = { ParseHttpMessage,
SerializeHttpRequest, PackHttpRequest,
ProcessHttpRequest, ProcessHttpResponse,
VerifyHttpRequest, ParseHttpServerAddress,
GetHttpMethodName,
CONNECTION_TYPE_POOLED_AND_SHORT,
"http" };
if (RegisterProtocol(PROTOCOL_HTTP, http_protocol) != 0) {
exit(1);
}
```
docs/en/nshead_service.md
+7
View File
@@ -0,0 +1,7 @@
# nshead service
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/nshead_service.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/overview.md
+91
View File
@@ -0,0 +1,91 @@
[中文版](../cn/overview.md)
# What is RPC?
Most machines on internet communicate with each other via [TCP/IP](https://en.wikipedia.org/wiki/Internet_protocol_suite). However, TCP/IP only guarantees reliable data transmissions. We need to abstract more to build services:
* What is the format of data transmission? Different machines and networks may have different byte-orders, directly sending in-memory data is not suitable. Fields in the data are added, modified or removed gradually, how do newer services talk with older services?
* Can TCP connection be reused for multiple requests to reduce overhead? Can multiple requests be sent through one TCP connection simultaneously?
* How to talk with a cluster with many machines?
* What should I do when the connection is broken? What if the server does not respond?
* ...
[RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) addresses the above issues by abstracting network communications as "clients accessing functions on servers": client sends a request to server, wait until server receives -> processes -> responds to the request, then do actions according to the result.
![rpc.png](../images/rpc.png)
Let's see how the issues are solved.
* RPC needs serialization which is done by [protobuf](https://github.com/google/protobuf) pretty well. Users fill requests in format of protobuf::Message, do RPC, and fetch results from responses in protobuf::Message. protobuf has good forward and backward compatibility for users to change fields and build services incrementally. For http services, [json](http://www.json.org/) is used for serialization extensively.
* Establishment and re-using of connections is transparent to users, but users can make choices like [different connection types](client.md#connection-type): short, pooled, single.
* Machines are discovered by a Naming Service, which can be implemented by [DNS](https://en.wikipedia.org/wiki/Domain_Name_System), [ZooKeeper](https://zookeeper.apache.org/) or [etcd](https://github.com/coreos/etcd). Inside Baidu, we use BNS (Baidu Naming Service). brpc provides ["list://" and "file://"](client.md#naming-service) as well. Users specify load balancing algorithms to choose one machine for each request from all machines, including: round-robin, randomized, [consistent-hashing](../cn/consistent_hashing.md)(murmurhash3 or md5) and [locality-aware](../cn/lalb.md).
* RPC retries when the connection is broken. When server does not respond within the given time, client fails with a timeout error.
# Where can I use RPC?
Almost all network communications.
RPC can't do everything surely, otherwise we don't need the layer of TCP/IP. But in most network communications, RPC meets requirements and isolates the underlying details.
Common doubts on RPC:
- My data is binary and large, using protobuf will be slow. First, this is possibly a wrong feeling and you will have to test it and prove it with [profilers](../cn/cpu_profiler.md). Second, many protocols support carrying binary data along with protobuf requests and bypass the serialization.
- I'm sending streaming data which can't be processed by RPC. Actually many protocols in RPC can handle streaming data, including [ProgressiveReader in http](http_client.md#progressively-download), streams in h2, [streaming rpc](streaming_rpc.md), and RTMP which is a specialized streaming protocol.
- I don't need replies. With some inductions, we know that in your scenario requests can be dropped at any stage because the client is always unaware of the situation. Are you really sure this is acceptable? Even if you don't need the reply, we recommend sending back small-sized replies, which are unlikely to be performance bottlenecks and will probably provide valuable clues when debugging complex bugs.
# What is ![brpc](../images/logo.png)?
An industrial-grade RPC framework used throughout [Baidu](http://ir.baidu.com/phoenix.zhtml?c=188488&p=irol-irhome), with 1,000,000+ instances(not counting clients) and thousands kinds of services, called "**baidu-rpc**" inside Baidu. Only C++ implementation is opensourced right now.
You can use it to:
* Build a server that can talk in multiple protocols (**on same port**), or access all sorts of services
* restful http/https, [h2](https://http2.github.io/http2-spec)/[gRPC](https://grpc.io). using http/h2 in brpc is much more friendly than [libcurl](https://curl.haxx.se/libcurl/). Access protobuf-based protocols with HTTP/h2+json, probably from another language.
* [redis](redis_client.md) and [memcached](memcache_client.md), thread-safe, more friendly and performant than the official clients
* [rtmp](https://github.com/apache/brpc/blob/master/src/brpc/rtmp.h)/[flv](https://en.wikipedia.org/wiki/Flash_Video)/[hls](https://en.wikipedia.org/wiki/HTTP_Live_Streaming), for building [streaming services](https://github.com/brpc/media-server).
* hadoop_rpc (may be opensourced)
* [rdma](https://en.wikipedia.org/wiki/Remote_direct_memory_access) support (will be opensourced)
* [thrift](thrift.md) support, thread-safe, more friendly and performant than the official clients.
* all sorts of protocols used in Baidu: [baidu_std](../cn/baidu_std.md), [streaming_rpc](streaming_rpc.md), hulu_pbrpc, [sofa_pbrpc](https://github.com/baidu/sofa-pbrpc), nova_pbrpc, public_pbrpc, ubrpc, and nshead-based ones.
* Build [HA](https://en.wikipedia.org/wiki/High_availability) distributed services using an industrial-grade implementation of [RAFT consensus algorithm](https://raft.github.io) which is opensourced at [braft](https://github.com/brpc/braft)
* Servers can handle requests [synchronously](server.md) or [asynchronously](server.md#asynchronous-service).
* Clients can access servers [synchronously](client.md#synchronus-call), [asynchronously](client.md#asynchronous-call), [semi-synchronously](client.md#semi-synchronous-call), or use [combo channels](combo_channel.md) to simplify sharded or parallel accesses declaratively.
* Debug services [via http](builtin_service.md), and run [cpu](../cn/cpu_profiler.md), [heap](../cn/heap_profiler.md) and [contention](../cn/contention_profiler.md) profilers.
* Get [better latency and throughput](#better-latency-and-throughput).
* [Extend brpc](new_protocol.md) with the protocols used in your organization quickly, or customize components, including [naming services](../cn/load_balancing.md#命名服务) (dns, zk, etcd), [load balancers](../cn/load_balancing.md#负载均衡) (rr, random, consistent hashing)
# Advantages of brpc
### More friendly API
Only 3 (major) user headers: [Server](https://github.com/apache/brpc/blob/master/src/brpc/server.h), [Channel](https://github.com/apache/brpc/blob/master/src/brpc/channel.h), [Controller](https://github.com/apache/brpc/blob/master/src/brpc/controller.h), corresponding to server-side, client-side and parameter-set respectively. You don't have to worry about "How to initialize XXXManager", "How to layer all these components together", "What's the relationship between XXXController and XXXContext". All you need to do is simple:
* Build service? include [brpc/server.h](https://github.com/apache/brpc/blob/master/src/brpc/server.h) and follow the comments or [examples](https://github.com/apache/brpc/blob/master/example/echo_c++/server.cpp).
* Access service? include [brpc/channel.h](https://github.com/apache/brpc/blob/master/src/brpc/channel.h) and follow the comments or [examples](https://github.com/apache/brpc/blob/master/example/echo_c++/client.cpp).
* Tweak parameters? Checkout [brpc/controller.h](https://github.com/apache/brpc/blob/master/src/brpc/controller.h). Note that the class is shared by server and channel. Methods are separated into 3 parts: client-side, server-side and both-side.
We tried to make simple things simple. Take naming service as an example. In older RPC implementations you may need to copy a pile of obscure code to make it work, however, in brpc accessing BNS is expressed as `Init("bns://node-name", ...)`, DNS is `Init("http://domain-name", ...)` and local machine list is `Init("file:///home/work/server.list", ...)`. Without any explanation, you know what it means.
### Make services more reliable
brpc is extensively used in Baidu:
* map-reduce service & table storages
* high-performance computing & model training
* all sorts of indexing & ranking servers
* ….
It's been proven.
brpc pays special attentions to development and maintenance efficency, you can [view internal status of servers](builtin_service.md) in web browser or with curl, analyze [cpu hotspots](../cn/cpu_profiler.md), [heap allocations](../cn/heap_profiler.md) and [lock contentions](../cn/contention_profiler.md) of online services, measure stats by [bvar](bvar.md) which is viewable in [/vars](vars.md).
### Better latency and throughput
Although almost all RPC implementations claim that they're "high-performant", the numbers are probably just numbers. Being really high-performant in different scenarios is difficult. To unify communication infra inside Baidu, brpc goes much deeper at performance than other implementations.
* Reading and parsing requests from different clients is fully parallelized and users don't need to distinguish between "IO-threads" and "Processing-threads". Other implementations probably have "IO-threads" and "Processing-threads" and hash file descriptors(fd) into IO-threads. When a IO-thread handles one of its fds, other fds in the thread can't be handled. If a message is large, other fds are significantly delayed. Although different IO-threads run in parallel, you won't have many IO-threads since they don't have too much to do generally except reading/parsing from fds. If you have 10 IO-threads, one fd may affect 10% of all fds, which is unacceptable to industrial online services (requiring 99.99% availability). The problem will be worse when fds are distributed unevenly across IO-threads (unfortunately common), or the service is multi-tenancy (common in cloud services). In brpc, reading from different fds is parallelized and even processing different messages from one fd is parallelized as well. Parsing a large message does not block other messages from the same fd, not to mention other fds. More details can be found [here](io.md#receiving-messages).
* Writing into one fd and multiple fds is highly concurrent. When multiple threads write into the same fd (common for multiplexed connections), the first thread directly writes in-place and other threads submit their write requests in [wait-free](https://en.wikipedia.org/wiki/Non-blocking_algorithm#Wait-freedom) manner. One fd can be written into 5,000,000 16-byte messages per second by a couple of highly-contended threads. More details can be found [here](io.md#sending-messages).
* Minimal locks. High-QPS services can utilize all CPU power on the machine. For example, [creating bthreads](../cn/memory_management.md) for processing requests, [setting up timeout](../cn/timer_keeping.md), [finding RPC contexts](../cn/bthread_id.md) according to response, [recording performance counters](bvar.md) are all highly concurrent. Users see very few contentions (via [contention profiler](../cn/contention_profiler.md)) caused by RPC framework even if the service runs at 500,000+ QPS.
* Server adjusts thread number according to load. Traditional implementations set number of threads according to latency to avoid limiting the throughput. brpc creates a new [bthread](../cn/bthread.md) for each request and ends the bthread when the request is done, which automatically adjusts thread number according to load.
Check [benchmark](../cn/benchmark.md) for a comparison between brpc and other implementations.
docs/en/parallel_http.md
+7
View File
@@ -0,0 +1,7 @@
# parallel http
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/parallel_http.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/rdma.md
+88
View File
@@ -0,0 +1,88 @@
# Build
Since RDMA requires driver and hardware support, only the build on linux is verified.
With config_brpc：
```bash
sh config_brpc.sh --with-rdma --headers="/usr/include" --libs="/usr/lib64 /usr/bin"
make
cd example/rdma_performance # example for rdma
make
```
With cmake：
```bash
mkdir bld && cd bld && cmake -DWITH_RDMA=ON ..
make
cd example/rdma_performance # example for rdma
mkdir bld && cd bld && cmake ..
make
```
With bazel:
```bash
# Server
bazel build --define=BRPC_WITH_RDMA=true example:rdma_performance_server
# Client
bazel build --define=BRPC_WITH_RDMA=true example:rdma_performance_client
```
# Basic Implementation
RDMA does not use socket API like TCP. However, the brpc::Socket class is still used. If a user sets ChannelOptions.use_rdma or ServerOptions.use_rdma to true, the Socket class created has RdmaEndpoint (see src/brpc/rdma/rdma_endpoint.cpp). When RDMA is enabled, the data which need to transmit will be posted to RDMA QP with verbs API, not written to TCP fd. For data receiving, RdmaEndpoint will get completions from RDMA CQ with verbs API (the event will be generated from a dedicated fd and be added into EventDispatcher, the handling function is RdmaEndpoint::PollCq) before parsing RPC messages with InputMessenger.
brpc uses RDMA RC mode. Every RdmaEndpoint has its own QP. Before establishing RDMA connection, a TCP connection is necessary to exchange some information such as GID and QPN. We call this procedure handshake. Since handshake needs TCP connection, the TCP fd in the corresponding Socket is still valid. The handshake procedure is completed in the AppConnect way in brpc. The TCP connection will keep in EST state but not be used for data transmission after RDMA connection is established. Once the TCP connection is closed, the corresponding RDMA connection will be set error.
The first key feature in RdmaEndpoint data transmission is zero copy. All data which need to transmit is in the Blocks of IOBuf. Thus all the Blocks need to be released after the sent CQEs are triggered. The reference of these Blocks are stored in RdmaEndpoint::_sbuf. In order to realize receiving zero copy, the receive side must post receive buffers in Blocks of IOBuf, which are stored in RdmaEndpoint::_rbuf. Note that all the Blocks posted in the receive side has a fixed size (recv_block_size). The transmit side can only send message smaller than that. Otherwise the receive side cannot receive data successfully.
The second key feature in RdmaEndpoint data transmission is sliding window flow control. The flow control is to avoid fast transmit side overwhelming slow receive side. TCP has similar mechanism in kernel TCP stack. RdmaEndpoint implements this mechanism with explicit ACKs from receive side. to reduce the overhead of ACKs, the ACK number can be piggybacked in ordinary data message as immediate data.
The third key feature in RdmaEndpoint data transmission is event suppression. The size of every message is limited to recv_block_size (default is 8KB). If every message will generate an event, the performance will be very poor, even worse than TCP (TCP has GSO/GRO). Therefore, RdmaEndpoint set solicited flag for every message according to data size, window and ACKS. The flag can control whether to generate an event in remove side or not.
All the memory used for data transmission in RDMA must be registered, which is very inefficient. Generally, a memory pool is employed to avoid frequent memory registration. In fact, brpc uses IOBuf for data transmission. In order to realize total zerocopy and compatibility with IOBuf, the memory used by IOBuf is taken over by the RDMA memory pool (see src/brpc/rdma/block_pool.cpp). Since IOBuf buffer cannot be controlled by user directly, the total memory consumption in IOBuf should be carefully managed. It is suggested that the application registers enough memory at one time according to its requirement.
The application can manage memory by itself and send data with IOBuf::append_user_data_with_meta. In this case, the application should register memory by itself with rdma::RegisterMemoryForRdma (see src/brpc/rdma/rdma_helper.h). Note that RegisterMemoryForRdma returns the lkey for registered memory. Please provide this lkey with data together when calling append_user_data_with_meta.
RDMA is hardware-related. It has some different concepts such as device, port, GID, LID, MaxSge and so on. These parameters can be read from NICs at initialization, and brpc will make the default choice (see src/brpc/rdma/rdma_helper.cpp). Sometimes the default choice is not the expectation, then it can be changed in the flag way.
`event_dispatcher_edisp_unsched` is a global flag and affects EventDispatcher scheduling in both normal mode (TCP) and RDMA mode.
`true` means EventDispatcher is unschedulable, and `false` means schedulable (default).
Runtime behavior is determined directly from user-provided values.
Recommended usage:
1. Keep the default `false` when unsched is not needed.
2. Set `-event_dispatcher_edisp_unsched=true` when unsched behavior is required.
Examples:
1. `-event_dispatcher_edisp_unsched=false`: both TCP and RDMA are schedulable.
2. `-event_dispatcher_edisp_unsched=true`: both TCP and RDMA are unschedulable.
# Parameters
Configurable parameters:
* rdma_trace_verbose: to print RDMA connection information in log，default is false.
* rdma_recv_zerocopy: enable zero copy in receive side，default is true.
* rdma_zerocopy_min_size: the min message size for receive zero copy (in Byte)，default is 512.
* rdma_recv_block_type: the block type used for receiving, can be default(8KB)/large(64KB)/huge(2MB)，default is default.
* rdma_prepared_qp_size: the size of QPs created at the beginning of the application，default is 128.
* rdma_prepared_qp_cnt: the number of QPs created at the beginning of the application，default is 1024.
* rdma_max_sge: the max length of sglist, default is 0, which is the max length allowed by the device.
* rdma_sq_size: the size of SQ，default is 128.
* rdma_rq_size: the size of RQ，default is 128.
* rdma_cqe_poll_once: the number of CQE pooled from CQ once，default is 32.
* rdma_gid_index: the index of local GID table used，default is -1，which is the maximum GID index.
* rdma_port: the port number used，default is 1.
* rdma_device: the IB device name，default is empty，which is the first active device.
* rdma_memory_pool_initial_size_mb: the initial region size of RDMA memory pool (in MB)，default is 1024.
* rdma_memory_pool_increase_size_mb: the step increase region size of RDMA memory pool (in MB)，default is 1024.
* rdma_memory_pool_max_regions: the max number of regions in RDMA memory pool，default is 3.
* rdma_memory_pool_buckets: the number of buckets for avoiding mutex contention in RDMA memory pool，default is 4.
* rdma_memory_pool_tls_cache_num: the number of thread local cached blocks in RDMA memory pool，default is 128.
* rdma_use_polling: Whether to use RDMA polling mode, default is false.
* rdma_poller_num: The number of pollers in polling mode, default is 1.
* rdma_poller_yield: Whether pollers in polling mode voluntarily relinquish the CPU, default is false.
* event_dispatcher_edisp_unsched: Global switch for EventDispatcher scheduling (true means unschedulable), default is false.
* rdma_disable_bthread: Disables bthread, default is false.
docs/en/redis_client.md
+321
View File
@@ -0,0 +1,321 @@
[中文版](../cn/redis_client.md)
[redis](http://redis.io/) is one of the most popular caching service in recent years. Compared to memcached it provides users with more data structures and operations, speeding up developments. In order to access redis servers more conveniently and make full use of bthread's capability of concurrency, brpc directly supports the redis protocol. Check [example/redis_c++](https://github.com/apache/brpc/tree/master/example/redis_c++/) for an example.
Advantages compared to [hiredis](https://github.com/redis/hiredis) (the official redis client):
- Thread safety. No need to set up separate clients for each thread.
- Support synchronous, asynchronous, semi-synchronous accesses etc. Support [ParallelChannel etc](combo_channel.md) to define access patterns declaratively.
- Support various [connection types](client.md#connection-type). Support timeout, backup request, cancellation, tracing, built-in services, and other benefits offered by brpc.
- All brpc clients in a process share a single connection to one redis-server, which is more efficient when multiple threads access one redis-server simultaneously (see [performance](#Performance)). Memory is allocated in blocks regardless of complexity of the reply, and short string optimization (SSO) is implemented to further improve performance.
Similarly with http, brpc guarantees that the time complexity of parsing redis replies is O(N) in worst cases rather than O(N^2) , where N is the number of bytes of reply. This is important when the reply consists of large arrays.
Turn on [-redis_verbose](#Debug) to print contents of all redis requests and responses, which is for debugging only.
# Request a redis server
Create a `Channel` for accessing redis:
```c++
#include <brpc/redis.h>
#include <brpc/channel.h>
brpc::ChannelOptions options;
options.protocol = brpc::PROTOCOL_REDIS;
brpc::Channel redis_channel;
if (redis_channel.Init("0.0.0.0:6379", &options) != 0) { // 6379 is the default port for redis-server
LOG(ERROR) << "Fail to init channel to redis-server";
return -1;
}
...
```
Execute `SET`, then `INCR`:
```c++
std::string my_key = "my_key_1";
int my_number = 1;
...
// Execute "SET <my_key> <my_number>"
brpc::RedisRequest set_request;
brpc::RedisResponse response;
brpc::Controller cntl;
set_request.AddCommand("SET %s %d", my_key.c_str(), my_number);
redis_channel.CallMethod(NULL, &cntl, &set_request, &response, NULL/*done*/);
if (cntl.Failed()) {
LOG(ERROR) << "Fail to access redis-server";
return -1;
}
// Get a reply by calling response.reply(i)
if (response.reply(0).is_error()) {
LOG(ERROR) << "Fail to set";
return -1;
}
// A reply is printable in multiple ways
LOG(INFO) << response.reply(0).c_str() // OK
<< response.reply(0) // OK
<< response; // OK
...
// Execute "INCR <my_key>"
brpc::RedisRequest incr_request;
incr_request.AddCommand("INCR %s", my_key.c_str());
response.Clear();
cntl.Reset();
redis_channel.CallMethod(NULL, &cntl, &incr_request, &response, NULL/*done*/);
if (cntl.Failed()) {
LOG(ERROR) << "Fail to access redis-server";
return -1;
}
if (response.reply(0).is_error()) {
LOG(ERROR) << "Fail to incr";
return -1;
}
// A reply is printable in multiple ways
LOG(INFO) << response.reply(0).integer() // 2
<< response.reply(0) // (integer) 2
<< response; // (integer) 2
```
Execute `incr` and `decr` in batch:
```c++
brpc::RedisRequest request;
brpc::RedisResponse response;
brpc::Controller cntl;
request.AddCommand("INCR counter1");
request.AddCommand("DECR counter1");
request.AddCommand("INCRBY counter1 10");
request.AddCommand("DECRBY counter1 20");
redis_channel.CallMethod(NULL, &cntl, &request, &response, NULL/*done*/);
if (cntl.Failed()) {
LOG(ERROR) << "Fail to access redis-server";
return -1;
}
CHECK_EQ(4, response.reply_size());
for (int i = 0; i < 4; ++i) {
CHECK(response.reply(i).is_integer());
CHECK_EQ(brpc::REDIS_REPLY_INTEGER, response.reply(i).type());
}
CHECK_EQ(1, response.reply(0).integer());
CHECK_EQ(0, response.reply(1).integer());
CHECK_EQ(10, response.reply(2).integer());
CHECK_EQ(-10, response.reply(3).integer());
```
# Request redis with authenticator
Create a RedisAuthenticator, and set to ChannelOptions.
```c++
brpc::ChannelOptions options;
brpc::policy::RedisAuthenticator* auth = new brpc::policy::RedisAuthenticator("my_password");
options.auth = auth;
```
# RedisRequest
A [RedisRequest](https://github.com/apache/brpc/blob/master/src/brpc/redis.h) may contain multiple commands by calling `AddCommand*`, which returns true on success and false otherwise. **The callsite backtrace is also printed on error**.
```c++
bool AddCommand(const char* fmt, ...);
bool AddCommandV(const char* fmt, va_list args);
bool AddCommandByComponents(const butil::StringPiece* components, size_t n);
```
Formatting is compatible with hiredis, namely `%b` corresponds to binary data (pointer + length), others are similar to those in `printf`. Some improvements have been made such as characters enclosed by single or double quotes are recognized as one field regardless of the spaces inside. For example, `AddCommand("Set 'a key with space' 'a value with space as well'")` sets value `a value with space as well` to key `a key with space`, while in hiredis the command must be written as `redisvCommand(..., "SET% s% s", "a key with space", "a value with space as well");`
> Note that if the `fmt` parameter of `AddCommand` and `AddCommandV` is set incorrectly, it may cause the program to crash or lead to data leakage. Please set it with caution. Should not use user input-influenced content as the `fmt` parameter!
`AddCommandByComponents` is similar to `redisCommandArgv` in hiredis. Users specify each part of the command in an array, which is immune to escaping issues often occurring in `AddCommand` and `AddCommandV`. If you encounter errors such as "Unmatched quote" or "invalid format" when using `AddCommand` and `AddCommandV`, try this method.
If `AddCommand*` fails, subsequent `AddCommand*` and `CallMethod` also fail. In general, there is no need to check return value of `AddCommand*`, since the RPC fails directly anyway.
Use `command_size()` to get number of commands added successfully.
Call `Clear()` before re-using the `RedisRequest` object.
# RedisResponse
A [RedisResponse](https://github.com/apache/brpc/blob/master/src/brpc/redis.h) may contain one or multiple [RedisReply](https://github.com/apache/brpc/blob/master/src/brpc/redis_reply.h)s. Use `reply_size()` for total number of replies and `reply(i)` for reference to the i-th reply (counting from 0). Note that in hiredis, if the request contains N commands, you have to call `redisGetReply` N times to get replies, while it's unnecessary in brpc as the `RedisResponse` already includes the N replies which are accessible by reply(i). As long as RPC is successful, `response.reply_size()` should be equal to `request.command_size()`, unless the redis-server is buggy. The precondition that redis works correctly is that replies correspond to commands one by one in the same sequence (positional correspondence).
Each `RedisReply` may be:
- REDIS_REPLY_NIL: NULL in redis, which means value does not exist. Testable by `is_nil()`.
- REDIS_REPLY_STATUS: Referred to `Simple String` in the redis document, usually used as the status of operations, such as the `OK` returned by `SET`. Testable by `is_string()` (same function for REDIS_REPLY_STRING). Use `c_str()` or `data()` to get the value.
- REDIS_REPLY_STRING: Referred to `Bulk String` in the redis document. Most return values are of this type, including those returned by `incr`. Testable by `is_string()`. Use `c_str()` or `data()` for the value.
- REDIS_REPLY_ERROR: The error message for a failed operation. Testable by `is_error()`. Use `error_message()` to get the message.
- REDIS_REPLY_INTEGER: A 64-bit signed integer. Testable by `is_integer()`. Use `integer()` to get the value.
- REDIS_REPLY_ARRAY: Array of replies. Testable by `is_array()`. Use `size()` for size of the array and `[i]` for the reference to the corresponding sub-reply.
If a response contains three replies: an integer, a string and an array with 2 items, we can use `response.reply(0).integer()`, `response.reply(1).c_str()`, and `repsonse.reply(2)[0]`, `repsonse.reply(2)[1]` to fetch values respectively. If the type is not correct, backtrace of the callsite is printed and an undefined value is returned.
Ownership of all replies belongs to `RedisResponse`. All relies are destroyed when response is destroyed.
Call `Clear()` before re-using the `RedisRespones` object.
# Request a redis cluster
Create a `Channel` using the consistent hashing as the load balancing algorithm(c_md5 or c_murmurhash) to access a redis cluster mounted under a naming service. Note that each `RedisRequest` should contain only one command or all commands have the same key. Under current implementation, multiple commands inside a single request are always sent to a same server. If the keys are located on different servers, the result must be wrong. In which case, you have to divide the request into multilple ones with one command each.
Another choice is to use the common [twemproxy](https://github.com/twitter/twemproxy) solution, which makes clients access the cluster just like accessing a single server, although the solution needs to deploy proxies and adds more latency.
For native Redis Cluster (slot based routing, MOVED/ASK redirection and topology refresh from `CLUSTER SLOTS`/`CLUSTER NODES`), use `brpc::RedisClusterChannel`:
```c++
#include <brpc/redis_cluster.h>
brpc::RedisClusterChannel channel;
brpc::RedisClusterChannelOptions options;
options.max_redirect = 5;
if (channel.Init("127.0.0.1:7000,127.0.0.1:7001", &options) != 0) {
LOG(ERROR) << "Fail to init redis cluster channel";
}
```
`RedisClusterChannel` supports synchronous/asynchronous `CallMethod`, automatic redirection retries and periodic topology refresh. Multi-key support includes `MGET/MSET/DEL/EXISTS/UNLINK/EVAL/EVALSHA`. `MULTI/EXEC` is currently not supported.
## RedisClusterChannel example
`example/redis_c++/redis_cluster_client.cpp` demonstrates:
- bootstrap from multiple seed nodes.
- MOVED/ASK auto-redirection and retry.
- topology refresh from `CLUSTER SLOTS` with `CLUSTER NODES` fallback.
- sync pipeline and async calls using one channel.
Build and run:
```bash
cd example/redis_c++
make redis_cluster_client
./redis_cluster_client \
--seeds=127.0.0.1:7000,127.0.0.1:7001 \
--max_redirect=5 \
--timeout_ms=1000
```
Frequently used options:
- `RedisClusterChannelOptions::max_redirect`: max redirects per command.
- `RedisClusterChannelOptions::refresh_interval_s`: interval of periodic topology refresh.
- `RedisClusterChannelOptions::topology_refresh_timeout_ms`: timeout for topology commands.
- `RedisClusterChannelOptions::channel_options`: normal brpc channel options for each redis node.
- `RedisClusterChannelOptions::enable_periodic_refresh`: disable this when your app controls refresh explicitly.
Notes:
- `MGET/MSET/DEL/EXISTS/UNLINK` are executed per key and merged in request order.
- `EVAL/EVALSHA` requires all declared keys to be in one slot.
- `MULTI/EXEC` returns an error reply by design.
# Debug
Turn on [-redis_verbose](http://brpc.baidu.com:8765/flags/redis_verbose) to print contents of all redis requests and responses. Note that this should only be used for debugging rather than online services.
Turn on [-redis_verbose_crlf2space](http://brpc.baidu.com:8765/flags/redis_verbose_crlf2space) to replace the `CRLF` (\r\n) with spaces in debugging logs for better readability.
| Name | Value | Description | Defined At |
| ------------------------ | ----- | ---------------------------------------- | ---------------------------------- |
| redis_verbose | false | [DEBUG] Print EVERY redis request/response | src/brpc/policy/redis_protocol.cpp |
| redis_verbose_crlf2space | false | [DEBUG] Show \r\n as a space | src/brpc/redis.cpp |
# Performance
redis version: 2.6.14
Start a client to send requests to a redis-server on the same machine using 1, 50, 200 bthreads synchronously. The latency is in microseconds.
```
$ ./client -use_bthread -thread_num 1
TRACE: 02-13 19:42:04: * 0 client.cpp:180] Accessing redis server at qps=18668 latency=50
TRACE: 02-13 19:42:05: * 0 client.cpp:180] Accessing redis server at qps=17043 latency=52
TRACE: 02-13 19:42:06: * 0 client.cpp:180] Accessing redis server at qps=16520 latency=54
$ ./client -use_bthread -thread_num 50
TRACE: 02-13 19:42:54: * 0 client.cpp:180] Accessing redis server at qps=301212 latency=164
TRACE: 02-13 19:42:55: * 0 client.cpp:180] Accessing redis server at qps=301203 latency=164
TRACE: 02-13 19:42:56: * 0 client.cpp:180] Accessing redis server at qps=302158 latency=164
$ ./client -use_bthread -thread_num 200
TRACE: 02-13 19:43:48: * 0 client.cpp:180] Accessing redis server at qps=411669 latency=483
TRACE: 02-13 19:43:49: * 0 client.cpp:180] Accessing redis server at qps=411679 latency=483
TRACE: 02-13 19:43:50: * 0 client.cpp:180] Accessing redis server at qps=412583 latency=482
```
The peak QPS at 200 threads is much higher than hiredis since brpc uses a single connection to redis-server by default and requests from multiple threads are [merged in a wait-free way](io.md#sending-messages), making the redis-server receive requests in batch and reach a much higher QPS. The lower QPS in following test that uses pooled connections is another proof.
Start a client to send requests in batch (10 commands per request) to redis-server on the same machine using 1, 50, 200 bthreads synchronously. The latency is in microseconds.
```
$ ./client -use_bthread -thread_num 1 -batch 10
TRACE: 02-13 19:46:45: * 0 client.cpp:180] Accessing redis server at qps=15880 latency=59
TRACE: 02-13 19:46:46: * 0 client.cpp:180] Accessing redis server at qps=16945 latency=57
TRACE: 02-13 19:46:47: * 0 client.cpp:180] Accessing redis server at qps=16728 latency=57
$ ./client -use_bthread -thread_num 50 -batch 10
TRACE: 02-13 19:47:14: * 0 client.cpp:180] Accessing redis server at qps=38082 latency=1307
TRACE: 02-13 19:47:15: * 0 client.cpp:180] Accessing redis server at qps=38267 latency=1304
TRACE: 02-13 19:47:16: * 0 client.cpp:180] Accessing redis server at qps=38070 latency=1305
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16878 gejun 20 0 48136 2436 1004 R 93.8 0.0 12:48.56 redis-server // thread_num=50
$ ./client -use_bthread -thread_num 200 -batch 10
TRACE: 02-13 19:49:09: * 0 client.cpp:180] Accessing redis server at qps=29053 latency=6875
TRACE: 02-13 19:49:10: * 0 client.cpp:180] Accessing redis server at qps=29163 latency=6855
TRACE: 02-13 19:49:11: * 0 client.cpp:180] Accessing redis server at qps=29271 latency=6838
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16878 gejun 20 0 48136 2508 1004 R 99.9 0.0 13:36.59 redis-server // thread_num=200
```
Note that the commands processed per second by the redis-server is the QPS times 10, which is about 400K. When thread_num equals 50 or higher, the CPU usage of the redis-server reaches limit. Note that redis-server is a [single-threaded reactor](threading_overview.md#single-threaded-reactor), utilizing one core is the maximum that it can do.
Now start a client to send requests to redis-server on the same machine using 50 bthreads synchronously through pooled connections.
```
$ ./client -use_bthread -connection_type pooled
TRACE: 02-13 18:07:40: * 0 client.cpp:180] Accessing redis server at qps=75986 latency=654
TRACE: 02-13 18:07:41: * 0 client.cpp:180] Accessing redis server at qps=75562 latency=655
TRACE: 02-13 18:07:42: * 0 client.cpp:180] Accessing redis server at qps=75238 latency=657
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16878 gejun 20 0 48136 2520 1004 R 99.9 0.0 9:52.33 redis-server
```
We can see a tremendous drop of QPS compared to the one using single connection above, and the redis-server has reached the CPU cap. The cause is that each time only one request can be read from a connection by the redis-server, which significantly increases cost of IO operations. This is also the peak performance of a hiredis client.
# Command Line Interface
[example/redis_c++/redis_cli](https://github.com/apache/brpc/blob/master/example/redis_c%2B%2B/redis_cli.cpp) is a command line tool similar to the official CLI, demostrating brpc's capability to talk with redis servers. When unexpected results are got from a redis-server using a brpc client, you can debug with this tool interactively as well.
For native Redis Cluster, you can start from [example/redis_c++/redis_cluster_client.cpp](https://github.com/apache/brpc/blob/master/example/redis_c%2B%2B/redis_cluster_client.cpp).
Like the official CLI, `redis_cli <command>` runs the command directly, and `-server` which is address of the redis-server can be specified.
```
$ ./redis_cli
__ _ __
/ /_ ____ _(_)___/ /_ __ _________ _____
/ __ \/ __ `/ / __ / / / /_____/ ___/ __ \/ ___/
/ /_/ / /_/ / / /_/ / /_/ /_____/ / / /_/ / /__
/_.___/\__,_/_/\__,_/\__,_/ /_/ / .___/\___/
/_/
This command-line tool mimics the look-n-feel of official redis-cli, as a
demostration of brpc's capability of talking to redis server. The
output and behavior is not exactly same with the official one.
redis 127.0.0.1:6379> mset key1 foo key2 bar key3 17
OK
redis 127.0.0.1:6379> mget key1 key2 key3
["foo", "bar", "17"]
redis 127.0.0.1:6379> incrby key3 10
(integer) 27
redis 127.0.0.1:6379> client setname brpc-cli
OK
redis 127.0.0.1:6379> client getname
"brpc-cli"
```
docs/en/rpc_press.md
+7
View File
@@ -0,0 +1,7 @@
# rpc press
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/rpc_press.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/rpc_replay.md
+7
View File
@@ -0,0 +1,7 @@
# rpc replay
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/rpc_replay.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/rpc_view.md
+7
View File
@@ -0,0 +1,7 @@
# rpc view
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/rpc_view.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/rpcz.md
+7
View File
@@ -0,0 +1,7 @@
# rpcz
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/rpcz.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/sanitizers.md
+7
View File
@@ -0,0 +1,7 @@
# sanitizers
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/sanitizers.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/server.md
+1113
View File
File diff suppressed because it is too large Load Diff
docs/en/server_debugging.md
+161
View File
@@ -0,0 +1,161 @@
# 1. Check the number of worker threads
Check /vars/bthread_worker_**count** and /vars/bthread_worker_**usage**, which is the number of worker threads in total and being used, respectively.
> The number of usage and count being close means that worker threads are not enough.
For example, there are 24 worker threads in the following figure, among which 23.93 worker threads are being used, indicating all the worker threads are full of jobs and not enough.
![img](../images/full_worker_usage.png)
There are 2.36 worker threads being used in the following figure. Apparently the worker threads are enough.
![img](../images/normal_worker_usage.png)
These two figures can be seen directly by putting /vars/bthread_worker_count;bthread_worker_usage?expand after service url, just like [this](http://brpc.baidu.com:8765/vars/bthread_worker_count;bthread_worker_usage?expand).
# 2. Check CPU usage
Check /vars/system_core_**count** and /vars/process_cpu_**usage**, which is the number of cpu core available and being used, respectively.
> The number of usage and count being close means that cpus are enough.
In the following figure the number of cores is 24, while the number of cores being used is 20.9, which means CPU is bottleneck.
![img](../images/high_cpu_usage.png)
The number of cores being used in the figure below is 2.06, then CPU is sufficient.
![img](../images/normal_cpu_usage.png)
# 3. Locate problems
The number of process_cpu_usage being close to bthread_worker_usage means it is a cpu-bound program and worker threads are doing calculations in most of the time.
The number of process_cpu_usage being much less than bthread_worker_usage means it is an io-bound program and worker threads are blocking in most of the time.
(1 - process_cpu_usage / bthread_worker_usage) is the time ratio that spent on blocking. For example, if process_cpu_usage = 2.4, bthread_worker_usage = 18.5, then worker threads spent 87.1% of time on blocking.
## 3.1 Locate cpu-bound problem
The possible reason may be the poor performance of single server or uneven distribution to upstreams.
### exclude the suspect of uneven distribution to upstreams
Enter qps at [vars]((http://brpc.baidu.com:8765/vars) page of different services to check whether qps is as expected, just like this:
![img](../images/bthread_creation_qps.png)
Or directly visit using curl in command line, like this:
```shell
$ curl brpc.baidu.com:8765/vars/*qps*
bthread_creation_qps : 95
rpc_server_8765_example_echo_service_echo_qps : 57
```
If the distribution of different machines is indeed uneven and difficult to solve, [Limit concurrency](server.md#user-content-limit-concurrency) can be considered to use.
### Improve performance of single server
Please use [CPU profiler](cpu_profiler.md) to analyze hot spots of the program and use data to guide optimization. Generally speaking, some big and obvious hot spots can be found in a cpu-bound program.
## 3.2 Locate io-bound problem
The possible reason:
- working threads are not enough.
- the client that visits downstream servers doesn't support bthread and the latency is too long.
- blocking that caused by internal locks, IO, etc.
If blocking is inevitable, please consider asynchronous method.
### exclude the suspect of working threads are not enough
If working threads are not enough, you can try to dynamically adjust the number of threads. Switch to the /flags page and click the (R) in the right of bthread_concurrency:
![img](../images/bthread_concurrency_1.png)
Just enter the new thread number and confirm:
![img](../images/bthread_concurrency_2.png)
Back to the /flags page, you can see that bthread_concurrency has become the new value.
![img](../images/bthread_concurrency_3.png)
However, adjusting the number of threads may not be useful. If the worker threads are largely blocked by visiting downstreams, it is useless to adjust the thread number since the real bottleneck is in the back-end and adjusting the thread number to a larger value just make the blocking time of each thread become longer.
For example, in our example, the worker threads are still full of work after the thread number is resized.
![img](../images/full_worker_usage_2.png)
### exclude the suspect of lock
If the program is blocked by some lock, it can also present features of io-bound. First use [contention profiler](contention_profiler.md) to check the contention status of locks.
### use rpcz
rpcz can help you see all the recent requests and the time(us) spent in each phase while processing them.
![img](../images/rpcz.png)
Click on a span link to see when the RPC started, the spent time in each phase and when it ended.
![img](../images/rpcz_2.png)
This is a typical example that server is blocked severely. It takes 20ms from receiving the request to starting running, indicating that the server does not have enough worker threads to get the job done in time.
For now the information of this span is less, we can add some in the program. You can use TRACEPRINTF print logs to rpcz. Printed content is embedded in the time stream of rpcz.
![img](../images/trace_printf.png)
After Re-running, you can check the span and it really contains the content we added by TRACEPRINTF.
![img](../images/rpcz_3.png)
Before running to the first TRACEPRINTF, the user callback has already run for 2051ms(suppose it meets our expectation), followed by foobar() that took 8036ms, which is expected to return very fast. The range has been further reduced.
Repeat this process until you find the function that caused the problem.
## Use bvar
TRACEPRINTF is mainly suitable for functions that called several times, so if a function is called many times, or the function itself has a small overhead, it is not appropriate to print logs to rpcz every time. You can use bvar instead.
[bvar](bvar.md) is a multi-threaded counting library, which can record the value passed from user at an extreme low cost and almost does not affect the program behavior compared to logging.
Follow the code below to monitor the runtime of foobar.
```c++
#include <butil/time.h>
#include <bvar/bvar.h>
bvar::LatencyRecorder g_foobar_latency("foobar");
...
void search() {
...
butil::Timer tm;
tm.start();
foobar();
tm.stop();
g_foobar_latency << tm.u_elapsed();
...
}
```
After rerunning the program, enter foobar in the search box of vars. The result is shown as below:
![img](../images/foobar_bvar.png)
Click on a bvar and you can see a dynamic figure. For example, after clicking on cdf:
![img](../images/foobar_latency_cdf.png)
Depending on the distribution of delays, you can infer the overall behavior of this function, how it behaves for most requests and how it behaves for long tails.
You can continue this process in the subroutine, add more bvar, compare the different distributions, and finally locate the source.
### Use brpc client only
You have to open the dummy server to provide built-in services, see [here](dummy_server.md).
docs/en/server_push.md
+28
View File
@@ -0,0 +1,28 @@
[中文版](../cn/server_push.md)
# Server push
What "server push" refers to is: server sends a message to client after occurrence of an event, rather than passively replying the client as in ordinary RPCs. Following two methods are recommended to implement server push in brpc.
## Remote event
Similar to local event, remote event is divided into two steps: registration and notification. The client sends an asynchronous RPC to the server for registration, and puts the event-handling code in the RPC callback. The RPC is also a part of the waiting for the notification, namely the server does not respond directly after receiving the request, instead it does not call done->Run() (to notify the client) until the local event triggers. As we see, the server is also asynchronous. If the connection is broken during the process, the client fails soon and may choose to retry another server or end the RPC. Server should be aware of the disconnection by using Controller.NotifyOnCancel() and delete useless done in-time.
This pattern is similar to [long polling](https://en.wikipedia.org/wiki/Push_technology#Long_polling) in some sense, sounds old but probably still be the most effective method. At first glance "server push" is server visiting client, opposite with ordinary client visiting server. But do you notice that, the response sent from server back to client is also in the opposite direction of "client visiting server"? In order to understand differences between response and push, let's analyze the process with the presumption that "client may receive messages from the server at any time".
* Client has to understand the messages from server anyway, otherwise the messaging is pointless.
* Client also needs to know how to deal with the message. If the client does not have the handling code, the message is still useless.
In other words, the client should "be prepared" to the messages from the server, and the "preparation" often relies on programming contexts that client just faces. Regarding different factors, it's more universal for client to inform server for "I am ready" (registered) first, then the server notifies the client with events later, in which case **the "push" is just a "response"**, with a very long or even infinite timeout.
In some scenarios, the registration can be ignored, such as the [push promise](https://tools.ietf.org/html/rfc7540#section-8.2) in http2, which does not require the web browser(client) to register with the server, because both client and server know that the task between them is to let the client download necessary resources ASAP. Since each resource has a unique URI, the server can directly push resources and URI to the client, which caches the resources to avoid repeated accesses. Similarly, protocols capable of "two-way communication" probably improves efficiencies of pushing in given scenarios such as multimedia streaming or fixed-format key/value pairs etc, rather than implementing universal pushing. The client knows how to handle messages that may be pushed by servers, so extra registrations are not required. The push can still be treated as a "response": to the request that client already and always promised server.
## Restful callback
The client wants a given URL to be called with necessary parameters when the event occurs. In this pattern, server can reply the client directly when it receives the registration request, because the event is not triggered by the end of the RPC. In addition, since the callback is just a URL, which can be stored in databases and message queues, this pattern is very flexible and widely used in business systems.
URL and parameters must contain enough information to make the callback know that which notification corresponds to which request, because the client may care about more than one event at the same time, or an event may be registered more than once due to network jitter or restarting machines etc. If the URL path is fixed, the client should place an unique ID in the registration request and the server should put the ID unchanged in response. Or the client generates an unique path for each registration so that each request is distinguishable from each other naturally. These methods are actually same in essense, just different positions for unique identifiers.
Callbacks should deal with [idempotence](https://en.wikipedia.org/wiki/Idempotence). The server may retry and send more than one notifications after encountering network problems. If the first notification has been successful, follow-up notifications should not have effects. The "remote event" pattern guarantees idempotency at the layer of RPC, which ensures that the done is called once and only once.
In order to avoid missing important notifications, users often need to combine RPC and message queues flexibly. RPC is much better at latency and efficiency than message queue, but due to limited memory, server needs to stop sending RPC after several failed retries and do more important things with the memory. It's better to move the notification task into a persistent message queue at the moment, which is capable of retrying during a very long time. With the idempotence in URL callback, most notifications are sent reliably and in-time.
docs/en/status.md
+37
View File
@@ -0,0 +1,37 @@
[中文版](../cn/status.md)
[/status](http://brpc.baidu.com:8765/status) shows primary statistics of services inside the server. The data sources are same with [/vars](vars.md), but stats are grouped differently.
![img](../images/status.png)
Meanings of the fields above:
- **non_service_error**: number of errors raised outside processing code of the service. When a valid service is obtained, the subsequent error is regarded as *service_error*, otherwise it is regarded as *non_service_error* (such as request parsing failed, service name does not exist, request concurrency exceeding limit, etc.). As a contrast, failing to access back-end servers during the processing is an error of the service, not a *non_service_error*. Even if the response written out successfully stands for failure, the error is counted into the service rather than *non_service_error*.
- **connection_count**: number of connections to the server from clients, not including number of outward connections which are displayed at /vars/rpc_channel_connection_count.
- **example.EchoService**: Full name of the service, including the package name defined in proto.
- **Echo (EchoRequest) returns (EchoResponse)**: Signature of the method. A service can have multiple methods. Click links on request/response to see schemes of the protobuf messages.
- **count**: Number of requests that are successfully processed.
- **error**: Number of requests that are failed to process.
- **latency**: average latency in recent *60s/60m/24h/30d* from *right to left* on html, average latency in recent 10s(by default, specified by [-bvar_dump_interval](http://brpc.baidu.com:8765/flags/bvar_dump_interval)) on plain texts.
- **latency_percentiles**: 80%, 90%, 99%, 99.9% percentiles of latency in 10 seconds(specified by[-bvar_dump_interval](http://brpc.baidu.com:8765/flags/bvar_dump_interval)). Curves with historical values are shown on html.
- **latency_cdf**: shows percentiles as [CDF](https://en.wikipedia.org/wiki/Cumulative_distribution_function), only available on html.
- **max_latency**: max latency in recent *60s/60m/24h/30d* from *right to left* on html, max latency in recent 10s(by default, specified by [-bvar_dump_interval](http://brpc.baidu.com:8765/flags/bvar_dump_interval)) on plain texts.
- **qps**: QPS(Queries Per Second) in recent *60s/60m/24h/30d* from *right to left* on html. QPS in recent 10s(by default, specified by [-bvar_dump_interval](http://brpc.baidu.com:8765/flags/bvar_dump_interval)) on plain texts.
- **processing**: (renamed to concurrency in master) Number of requests being processed by the method. If this counter can't hit zero when the traffic to the service becomes zero, the server probably has bugs, such as forgetting to call done->Run() or stuck on some processing steps.
Users may customize descriptions on /status by letting the service implement [brpc::Describable](https://github.com/apache/brpc/blob/master/src/brpc/describable.h).
```c++
class MyService : public XXXService, public brpc::Describable {
public:
...
void Describe(std::ostream& os, const brpc::DescribeOptions& options) const {
os << "my_status: blahblah";
}
};
```
For example:
![img](../images/status_2.png)
docs/en/streaming_log.md
+283
View File
@@ -0,0 +1,283 @@
# Name
streaming_log - Print log to std::ostreams
# SYNOPSIS
```c++
#include <butil/logging.h>
LOG(FATAL) << "Fatal error occurred! contexts=" << ...;
LOG(WARNING) << "Unusual thing happened ..." << ...;
LOG(TRACE) << "Something just took place..." << ...;
LOG(TRACE) << "Items:" << noflush;
LOG_IF(NOTICE, n > 10) << "This log will only be printed when n > 10";
PLOG(FATAL) << "Fail to call function setting errno";
VLOG(1) << "verbose log tier 1";
CHECK_GT(1, 2) << "1 can't be greater than 2";
LOG_EVERY_SECOND(INFO) << "High-frequent logs";
LOG_EVERY_N(ERROR, 10) << "High-frequent logs";
LOG_FIRST_N(INFO, 20) << "Logs that prints for at most 20 times";
LOG_ONCE(WARNING) << "Logs that only prints once";
```
# DESCRIPTION
Streaming log is the best choice for printing complex objects or template objects. As most objects are complicate, user needs to convert all the fields to string first in order to use `printf` with `%s`. However it's very inconvenient (can't append numbers) and needs lots of temporary memory (caused by string). The solution in C++ is to send the log as a stream to the `std::ostream` object. For example, in order to print object A, we need to implement the following interface:
```c++
std::ostream& operator<<(std::ostream& os, const A& a);
```
The signature of the function means to print object `a` to `os` and then return `os`. The return value of `os` enables us to combine binary operator `<<` (left-combine). As a result, `os << a << b << c;` means `operator<<(operator<<(operator<<(os, a), b), c);`. Apparently `operator<<` needs a returning reference to complete this process, which is also called chaining. In languages that don't support operator overloading, you will see a more tedious form, such as `os.print(a).print(b).print(c)`.
You should also use chaining in your own implementation of `operator<<`. In fact, printing a complex object is like DFS a tree: Call `operator<<` on each child node, and then each child node invokes the function on the grandchild node, and so forth. For example, object A has two member variables: B and C. Printing A becomes the process of putting B and C ostream:
```c++
struct A {
B b;
C c;
};
std::ostream& operator<<(std::ostream& os, const A& a) {
return os << "A{b=" << a.b << ", c=" << a.c << "}";
}
```
Data structure of B and C along with the print function：
```c++
struct B {
int value;
};
std::ostream& operator<<(std::ostream& os, const B& b) {
return os << "B{value=" << b.value << "}";
}
struct C {
string name;
};
std::ostream& operator<<(std::ostream& os, const C& c) {
return os << "C{name=" << c.name << "}";
}
```
Finally the result of printing object A is:
```
A{b=B{value=10}, c=C{name=tom}}
```
This way we don't need to allocate temporary memory since objects are directly passed into the ostream object. Of course, the memory management of ostream itself is another topic.
OK, now we connect the whole printing process by ostream. The most common ostream objects are `std::cout` and `std::cerr`, so objects implement the above function can be directly sent to `std::cout` and `std::cerr`. In other words, if a log stream also inherits ostream, then these objects can be written into log. Streaming log is such a log stream that inherits `std::ostream` to send the object into the log. In the current implementation, the logs are recorded in a thread-local buffer, which will be flushed into screen or ` logging::LogSink` after a complete log record. Of course, the implementation is thread safe.
## LOG
If you have ever used glog before, you should find it easy to start. The log macro is the same as glog. For example, to print a FATAL log (Note that there is no `std::endl`):
```c++
LOG(FATAL) << "Fatal error occurred! contexts=" << ...;
LOG(WARNING) << "Unusual thing happened ..." << ...;
LOG(TRACE) << "Something just took place..." << ...;
```
The log level of streaming log in accordance with glog：
| streaming log | glog | Use Cases |
| ------------- | -------------------- | ---------------------------------------- |
| FATAL | FATAL (coredump) | Fatal error. Since most fatal log inside baidu is not fatal actually, it won't trigger coredump directly as glog, unless you turn on [-crash_on_fatal_log](http://brpc.baidu.com:8765/flags/crash_on_fatal_log) |
| ERROR | ERROR | Non-fatal error. |
| WARNING | WARNING | Unusual branches |
| NOTICE | - | Generally you should not use NOTICE as it's intended for important business logs. Make sure to check with other developers. glog doesn't have NOTICE. |
| INFO, TRACE | INFO | Important side effects such as open/close some resources. |
| VLOG(n) | INFO | Detailed log that support multiple layers. |
| DEBUG | INFOVLOG(1) (NDEBUG) | Just for compatibility. Print logs only when `NDEBUG` is not defined. See DLOG/DPLOG/DVLOG for more reference. |
## PLOG
The difference of PLOG and LOG is that it will append error information at the end of log. It's kind of like `%m` in `printf`. Under POSIX environment, the error code is `errno`。
```c++
int fd = open("foo.conf", O_RDONLY); // foo.conf does not exist, errno was set to ENOENT
if (fd < 0) {
PLOG(FATAL) << "Fail to open foo.conf"; // "Fail to open foo.conf: No such file or directory"
return -1;
}
```
## noflush
If you don't want to flush the log at once, append `noflush`. It's commonly used inside a loop:
```c++
LOG(TRACE) << "Items:" << noflush;
for (iterator it = items.begin(); it != items.end(); ++it) {
LOG(TRACE) << ' ' << *it << noflush;
}
LOG(TRACE);
```
The first two LOG(TRACE) doesn't flush the log to the screen. They are recorded inside the thread-local buffer. The third LOG(TRACE) flush all logs into the screen. If there are 3 elements inside items and we don't append `noflush`, the result would be:
```
TRACE: ... Items:
TRACE: ... item1
TRACE: ... item2
TRACE: ... item3
```
After we add `noflush`:
```
TRACE: ... Items: item1 item2 item3
```
The `noflush` feature also support bthread so that we can push lots of logs from the server's bthreads without actually print them (using `noflush`), and flush the whole log at the end of RPC. Note that you should not use `noflush` when implementing an asynchronous method since it will change the underlying bthread, leaving `noflush` out of function.
## LOG_IF
`LOG_IF(log_level, condition)` prints only when condition is true. It's the same as `if (condition) { LOG() << ...; }` with shorter code：
```c++
LOG_IF(NOTICE, n > 10) << "This log will only be printed when n > 10";
```
## XXX_EVERY_SECOND
XXX represents for LOG, LOG_IF, PLOG, SYSLOG, VLOG, DLOG, and so on. These logging macros print log at most once per second. You can use these to check running status inside hotspot area. The first call to this macro prints the log immediately, and costs additional 30ns (caused by gettimeofday) compared to normal LOG.
```c++
LOG_EVERY_SECOND(INFO) << "High-frequent logs";
```
## XXX_EVERY_N
XXX represents for LOG, LOG_IF, PLOG, SYSLOG, VLOG, DLOG, and so on. These logging macros print log every N times. You can use these to check running status inside hotspot area. The first call to this macro prints the log immediately, and costs an additional atomic operation (relaxed order) compared to normal LOG. This macro is thread safe which means counting from multiple threads is also accurate while glog is not.
```c++
LOG_EVERY_N(ERROR, 10) << "High-frequent logs";
```
## XXX_FIRST_N
XXX represents for LOG, LOG_IF, PLOG, SYSLOG, VLOG, DLOG, and so on. These logging macros print log at most N times. It costs an additional atomic operation (relaxed order) compared to normal LOG before N, and zero cost after.
```c++
LOG_FIRST_N(ERROR, 20) << "Logs that prints for at most 20 times";
```
## XXX_ONCE
XX represents for LOG, LOG_IF, PLOG, SYSLOG, VLOG, DLOG, and so on. These logging macros print log at most once. It's the same as `XXX_FIRST_N(..., 1)`
```c++
LOG_ONCE(ERROR) << "Logs that only prints once";
```
## VLOG
VLOG(verbose_level) is detail log that support multiple layers. It uses 2 gflags: *--verbose* and *--verbose_module* to control the logging layer you want (Note that glog uses *--v* and *--vmodule*). The log will be printed only when `--verbose` >= `verbose_level`:
```c++
VLOG(0) << "verbose log tier 0";
VLOG(1) << "verbose log tier 1";
VLOG(2) << "verbose log tier 2";
```
When `--verbose=1`, the first 2 log will be printed while the last won't. Module means a file or file path without the extension name, and value of `--verbose_module` will overwrite `--verbose`. For example:
```bash
--verbose=1 --verbose_module="channel=2,server=3" # print VLOG of those with verbose value:
# channel.cpp <= 2
# server.cpp <= 3
# other files <= 1
--verbose=1 --verbose_module="src/brpc/channel=2,server=3"
# For files with same names, add paths
```
You can set `--verbose` and `--verbose_module` through `google::SetCommandLineOption` dynamically.
VLOG has another form VLOG2, which allows user to specify virtual path:
```c++
// public/foo/bar.cpp
VLOG2("a/b/c", 2) << "being filtered by a/b/c rather than public/foo/bar";
```
> VLOG and VLOG2 also have corresponding VLOG_IF and VLOG2_IF.
## DLOG
All log macros have debug versions, starting with D, such as DLOG, DVLOG. When NDEBUG is defined, these logs will not be printed.
**Do not put important side effects inside the log streams beginning with D.**
*No printing* means that even the parameters are not evaluated. If your parameters have side effects, they won't happend when NDEBUG is defined. For example, `DLOG(FATAL) << foo();` where foo is a function or it changes a dictionary, anyway, it's essential. However, it won't be evaluated when NDEBUG is defined.
## CHECK
Another import variation of logging is `CHECK(expression)`. When expression evaluates to false, it will print a fatal log. It's kind of like `ASSERT` in gtest, and has other form such as CHECK_EQ, CHECK_GT, and so on. When check fails, the message after will be printed.
```c++
CHECK_LT(1, 2) << "This is definitely true, this log will never be seen";
CHECK_GT(1, 2) << "1 can't be greater than 2";
```
Run the above code you should see a fatal log and the calling stack：
```
FATAL: ... Check failed: 1 > 2 (1 vs 2). 1 can't be greater than 2
#0 0x000000afaa23 butil::debug::StackTrace::StackTrace()
#1 0x000000c29fec logging::LogStream::FlushWithoutReset()
#2 0x000000c2b8e6 logging::LogStream::Flush()
#3 0x000000c2bd63 logging::DestroyLogStream()
#4 0x000000c2a52d logging::LogMessage::~LogMessage()
#5 0x000000a716b2 (anonymous namespace)::StreamingLogTest_check_Test::TestBody()
#6 0x000000d16d04 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
#7 0x000000d19e96 testing::internal::HandleExceptionsInMethodIfSupported<>()
#8 0x000000d08cd4 testing::Test::Run()
#9 0x000000d08dfe testing::TestInfo::Run()
#10 0x000000d08ec4 testing::TestCase::Run()
#11 0x000000d123c7 testing::internal::UnitTestImpl::RunAllTests()
#12 0x000000d16d94 testing::internal::HandleSehExceptionsInMethodIfSupported<>()
```
The second column of the callstack is the address of the code segment. You can use `addr2line` to check the corresponding file and line:
```
$ addr2line -e ./test_base 0x000000a716b2
/home/gejun/latest_baidu_rpc/public/common/test/test_streaming_log.cpp:223
```
You **should** use `CHECK_XX` for arithmetic condition so that you can see more detailed information when check failed.
```c++
int x = 1;
int y = 2;
CHECK_GT(x, y); // Check failed: x > y (1 vs 2).
CHECK(x > y); // Check failed: x > y.
```
Like DLOG, you should NOT include important side effects inside DCHECK.
## LogSink
The default destination of streaming log is the screen. You can change it through `logging::SetLogSink`. Users can inherit LogSink and implement their own output logic. We provide an internal LogSink as an example:
### StringSink
Inherit both LogSink and string. Store log content inside string and mainly aim for unit test. The following case shows a classic usage of StringSink:
```c++
TEST_F(StreamingLogTest, log_at) {
::logging::StringSink log_str;
::logging::LogSink* old_sink = ::logging::SetLogSink(&log_str);
LOG_AT(FATAL, "specified_file.cc", 12345) << "file/line is specified";
// the file:line part should be using the argument given by us.
ASSERT_NE(std::string::npos, log_str.find("specified_file.cc:12345"));
// restore the old sink.
::logging::SetLogSink(old_sink);
}
```
docs/en/streaming_rpc.md
+155
View File
@@ -0,0 +1,155 @@
[中文版](../cn/streaming_rpc.md)
# Overview
There are some scenarios when the client or server needs to send huge amount of data, which may grow over time or is too large to put into the RPC attachment. For example, it could be the replica or snapshot transmitting between different nodes in a distributed system. Although we could send data segmentation across multiple RPC between client and server, this will introduce the following problems:
- If these RPCs are parallel, there is no guarantee on the order of the data at the receiving side, which leads to complicate code of reassembling.
- If these RPCs are serial, we have to endure the latency of the network RTT for each RPC together with the process time, which is especially unpredictable.
In order to allow large packets to flow between client and server like a stream, we provide a new communication model: Streaming RPC. Streaming RPC enables users to establish Stream which is a user-space connection between client and service. Multiple Streams can share the same TCP connection at the same time. The basic transmission unit on Stream is message. As a result, the sender can continuously write to messages to a Stream, while the receiver can read them out in the order of sending.
Streaming RPC ensures/provides:
- The message order at the receiver is exactly the same as that of the sender
- Boundary for messages
- Full duplex
- Flow control
- Notification on timeout
- We support segment large messages automatically to avoid [Head-of-line blocking](https://en.wikipedia.org/wiki/Head-of-line_bloc
king) problem.
For examples please refer to [example/streaming_echo_c++](https://github.com/apache/brpc/tree/master/example/streaming_echo_c++/).
# Create a Stream
Currently streams are established by the client only. The new Stream objects are created in client and then are used to issue an RPC (through baidu_std protocol) to the specified service. The service could accept these streams by responding to the request without error, thus the Streams are created once the client receives the response successfully. Any error during this process fails the RPC and thus fails the Stream creation. Take the Linux environment as an example, the client creates a [socket](http://linux.die.net/man/7/socket) first (creates a Stream), and then tries to establish a connection with the remote side by [connect](http://linux.die.net/man/2/connect) (establish a Stream through RPC). Finally the stream has been created once the remote side [accept](http://linux.die.net/man/2/accept) the request.
> If the client tries to establish a stream to a server that doesn't support streaming RPC, it will always return failure.
In the code we use `StreamId` to represent a Stream, which is the key ID to pass when reading, writing, closing the Stream.
```c++
struct StreamOptions
// The max size of unconsumed data allowed at remote side.
// If |max_buf_size| <= 0, there's no limit of buf size
// default: 2097152 (2M)
int max_buf_size;
// Notify user when there's no data for at least |idle_timeout_ms|
// milliseconds since the last time that on_received_messages or on_idle_timeout
// finished.
// default: -1
long idle_timeout_ms;
// Maximum messages in batch passed to handler->on_received_messages
// default: 128
size_t messages_in_batch;
// Handle input message, if handler is NULL, the remote side is not allowed to
// write any message, who will get EBADF on writing
// default: NULL
StreamInputHandler* handler;
};
// [Called at the client side]
// Create a Stream at client-side along with the |cntl|, which will be connected
// when receiving the response with a Stream from server-side. If |options| is
// NULL, the Stream will be created with default options
// Return 0 on success, -1 otherwise
int StreamCreate(StreamId* request_stream, Controller &cntl, const StreamOptions* options);
// [Called at the client side for creating multiple streams]
// Create streams at client-side along with the |cntl|, which will be connected
// when receiving the response with streams from server-side. If |options| is
// NULL, the stream will be created with default options
// Return 0 on success, -1 otherwise
int StreamCreate(StreamIds& request_streams, int request_stream_size, Controller& cntl,
const StreamOptions* options);
```
# Accept a Stream
If a Stream is attached inside the request of an RPC, the service can accept the Stream by `StreamAccept`. On success this function fills the created Stream into `response_stream`, which can be used to send message to the client.
```c++
// [Called at the server side]
// Accept the Stream. If client didn't create a Stream with the request
// (cntl.has_remote_stream() returns false), this method would fail.
// Return 0 on success, -1 otherwise.
int StreamAccept(StreamId* response_stream, Controller &cntl, const StreamOptions* options);
// [Called at the server side for accepting multiple streams]
// Accept the streams. If client didn't create streams with the request
// (cntl.has_remote_stream() returns false), this method would fail.
// Return 0 on success, -1 otherwise.
int StreamAccept(StreamIds& response_stream, Controller& cntl, const StreamOptions* options);
```
# Read from a Stream
Upon creating/accepting a Stream, you can fill the `hander` in `StreamOptions` with your own implemented `StreamInputHandler`. Then you will be notified when the stream receives data, is closed by the other end, or reaches idle timeout.
```c++
class StreamInputHandler {
public:
// Callback when stream receives data
virtual int on_received_messages(StreamId id, butil::IOBuf *const messages[], size_t size) = 0;
// Callback when there is no data for a long time on the stream
virtual void on_idle_timeout(StreamId id) = 0;
// Callback when stream is closed by the other end
virtual void on_closed(StreamId id) = 0;
};
```
> ***The first call to `on_received_message `***
>
> On the client's side, if the creation process is synchronous, `on_received_message` will be called when the blocking RPC returns. If it's asynchronous, `on_received_message` won't be called until `done->Run()` finishes.
>
> On the server' side, `on_received_message` will be called once `done->Run()` finishes.
# Write to a Stream
```c++
// Write |message| into |stream_id|. The remote-side handler will received the
// message by the written order
// Returns 0 on success, errno otherwise
// Errno:
// - EAGAIN: |stream_id| is created with positive |max_buf_size| and buf size
// which the remote side hasn't consumed yet exceeds the number.
// - EINVAL: |stream_id| is invalid or has been closed
int StreamWrite(StreamId stream_id, const butil::IOBuf &message);
```
# Flow Control
When the amount of unacknowledged data reaches the limit, the `Write` operation at the sender will fail with EAGAIN immediately. At this moment, you should wait for the receiver to consume the data synchronously or asynchronously.
```c++
// Wait until the pending buffer size is less than |max_buf_size| or error occurs
// Returns 0 on success, errno otherwise
// Errno:
// - ETIMEDOUT: when |due_time| is not NULL and time expired this
// - EINVAL: the Stream was close during waiting
int StreamWait(StreamId stream_id, const timespec* due_time);
// Async wait
void StreamWait(StreamId stream_id, const timespec *due_time,
void (*on_writable)(StreamId stream_id, void* arg, int error_code),
void *arg);
```
# Close a Stream
```c++
// Close |stream_id|, after this function is called:
// - All the following |StreamWrite| would fail
// - |StreamWait| wakes up immediately.
// - Both sides |on_closed| would be notified after all the pending buffers have
// been received
// This function could be called multiple times without side-effects
int StreamClose(StreamId stream_id);
```
docs/en/thread_local.md
+7
View File
@@ -0,0 +1,7 @@
# thread local
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/thread_local.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/threading_overview.md
+45
View File
@@ -0,0 +1,45 @@
[中文版](../cn/threading_overview.md)
# Common threading models
## Connections own threads or processes exclusively
In this model, a thread/process handles all messages from a connection and does not quit or do other jobs before the connection is closed. When number of connections increases, resources occupied by threads/processes and costs of context switches becomes more and more overwhelming, making servers perform poorly. This situation is summarized as the [C10K](http://en.wikipedia.org/wiki/C10k_problem) problem, which was common in early web servers but rarely present today.
## Single-threaded [reactor](http://en.wikipedia.org/wiki/Reactor_pattern)
Event-loop libraries such as [libevent](http://libevent.org/), [libev](http://software.schmorp.de/pkg/libev.html) are typical examples. There's usually an event dispatcher in this model responsible for waiting on different kinds of events and calling the corresponding event handler **in-place** when an event occurs. After all handlers(that need to be called) are called, the dispatcher waits for more events again, which forms a "loop". Essentially this model multiplexes(interleaves) code written in different handlers into a system thread. One event-loop can only utilize one core, so this kind of program is either IO-bound or each handler runs within short and deterministic time(such as http servers), otherwise one callback taking long time blocks the whole program and causes high delays. In practice this kind of program is not suitable for involving many developers, because just one person adding inappropriate blocking code may significantly slow down reactivities of all other code. Since event handlers don't run simultaneously, race conditions between callbacks are relatively simple and in some scenarios locks are not needed. These programs are often scaled by deploying more processes.
How single-threaded reactors work and the problems related are demonstrated below: (The Chinese characters in red: "Uncontrollable! unless the service is specialized")
![img](../images/threading_overview_1.png)
## N:1 threading library
Also known as [Fiber](http://en.wikipedia.org/wiki/Fiber_(computer_science)). Typical examples are [GNU Pth](http://www.gnu.org/software/pth/pth-manual.html), [StateThreads](http://state-threads.sourceforge.net/index.html). This model maps N user threads into a single system thread, in which only one user thread runs at the same time and the running user thread does not switch to other user threads until a blocking primitive is called (cooperative). N:1 threading libraries are equal to single-threaded reactors on capabilities, except that callbacks are replaced by contexts (stacks, registers, signals) and running callbacks becomes jumping to contexts. Similar to event-loop libraries, a N:1 threading library cannot utilize multiple CPU cores, thus only suitable for specialized applications. However a single system thread is more friendly to CPU caches, with removal of the support for signal masks, context switches between user threads can be done very fast(100 ~ 200ns). N:1 threading libraries perform as well as event-loop libraries and are also scaled by deploying more processes in general.
## Multi-threaded reactor
[boost::asio](http://www.boost.org/doc/libs/1_56_0/doc/html/boost_asio.html) is a typical example. One or several threads run event dispatchers respectively. When an event occurs, the event handler is queued into a worker thread to run. This model is extensible from single-threaded reactor intuitively and able to make use of multiple CPU cores. Since sharing memory addresses makes interactions between threads much cheaper, the worker threads are able to balance loads between each other frequently, as a contrast multiple single-threaded reactors basically depend on the front-end servers to distribute traffic. A well-implemented multi-threaded reactor is likely to utilize CPU cores more evenly than multiple single-threaded reactors on the same machine. However, due to [cache coherence](atomic_instructions.md#cacheline), multi-threaded reactors are unlikely to achieve linear scalability on CPU cores. In particular scenarios, a badly implemented multi-threaded reactor running on 24 cores is even slower than a well-tuned single-threaded reactor. Because a multi-threaded reactor has multiple worker threads, one blocking event handler may not delay other handlers. As a result, event handlers are not required to be non-blocking unless all worker threads are blocked, in which case the overall progress is affected. In fact, most RPC frameworks are implemented in this model with event handlers that may block, such as synchronously waiting for RPCs to downstream servers.
How multi-threaded reactors work and problems related are demonstrated below:
![img](../images/threading_overview_2.png)
## M:N threading library
This model maps M user threads into N system threads. An M:N threading library is able to decide when and where to run a piece of code and when to end the execution, which is more flexible at scheduling compared to multi-threaded reactors. But full-featured M:N threading libraries are difficult to implement and remaining as active research topics. The M:N threading library that we're talking about is specialized for building online services, in which case, some of the requirements can be simplified, namely no (complete) preemptions and priorities. M:N threading libraries can be implemented either in userland or OS kernel. New programming languages prefer implementations in userland, such as GHC thread and goroutine, which is capable of adding brand-new keywords and intercepting related APIs on threading. Implementation in existing languages often have to modify the OS kernel, such as [Windows UMS](https://msdn.microsoft.com/en-us/library/windows/desktop/dd627187(v=vs.85).aspx) and google SwicthTo(which is 1:1, however M:N effects can be achieved based on it). Compared to N:1 threading libraries, usages of M:N threading libraries are more similar to system threads, which need locks or message passings to ensure thread safety.
# Issues
## Multi-core scalability
Ideally capabilities of the reactor model are maximized when all source code is programmed in event-driven manner, but in reality because of the difficulties and maintainability, users are likely to mix usages: synchronous IO is often issued in callbacks, blocking worker threads from processing other requests. A request often goes through dozens of services, making worker threads spend a lot of time waiting for responses from downstream servers. Users have to launch hundreds of threads to maintain enough throughput, which imposes intensive pressure on scheduling and lowers efficiencies of TLS related code. Tasks are often pushed into a queue protected with a global mutex and condition, which performs poorly when many threads are contending for it. A better approach is to deploy more task queues and adjust the scheduling algorithm to reduce global contentions. Namely each system thread has its own runqueue, and one or more schedulers dispatch user threads to different runqueues. Each system thread runs user threads from its own runqueue before considering other runqueues, which is more complicated but more scalable than the global mutex+condition solution. This model is also easier to support NUMA.
When an event dispatcher passes a task to a worker thread, the user code probably jumps from one CPU core to another, which may need to wait for synchronizations of relevant cachelines, which is not very fast. It would be better that the worker is able to run directly on the CPU core where the event dispatcher runs, since at most of the time, priority of running the worker ASAP is higher than getting new events from the dispatcher. Similarly, it's better to wake up the user thread blocking on RPC on the same CPU core where the response is received.
## Asynchronous programming
Flow controls in asynchronous programming are even difficult for experts. Any suspending operation such as sleeping for a while or waiting for something to finish, implies that users have to save states explicitly and restore states in callbacks. Asynchronous code is often written as state machines. A few suspensions are troublesome, but still handleable. The problem is that once the suspension occurs inside a condition, loop or sub-function, it's almost impossible to write such a state machine being understood and maintained by many people, although the scenario is quite common in distributed systems where a node often needs to interact with multiple nodes simultaneously. In addition, if the wakeup can be triggered by more than one events (such as either fd has data or timeout is reached), the suspension and resuming are prone to race conditions, which require good multi-threaded programming skills to solve. Syntactic sugars(such as lambda) just make coding less troublesome rather than reducing difficulty.
Shared pointers are common in asynchronous programming, which seems convenient, but also makes ownerships of memory elusive. If the memory is leaked, it's difficult to locate the code that forgot to release; if segment fault happens, where the double-free occurs is also unknown. Code with a lot of referential countings is hard to remain good-quality and may waste a lot of time on debugging memory related issues. If references are even counted manually, keeping quality of the code is harder and the maintainers are less willing to modify the code. [RAII](http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) cannot be used in many scenarios in asynchronous programming, sometimes resources need to be locked before a callback and unlocked inside the callback, which is very error-prone in practice.
docs/en/thrift.md
Executable
+146
View File
@@ -0,0 +1,146 @@
[中文版](../cn/thrift.md)
[thrift](https://thrift.apache.org/) is a RPC framework used widely in various environments, which was developed by Facebook and adopted by Apache later. In order to interact with thrift servers and solves issues on thread-safety, usabilities and concurrencies, brpc directly supports the thrift protocol that is used by thrift in NonBlocking mode.
Example: [example/thrift_extension_c++](https://github.com/apache/brpc/tree/master/example/thrift_extension_c++/).
Advantages compared to the official solution:
- Thread safety. No need to set up separate clients for each thread.
- Supports synchronous, asynchronous, batch synchronous, batch asynchronous, and other access methods. Combination channels such as ParallelChannel are also supported.
- Support various connection types(short, connection pool). Support timeout, backup request, cancellation, tracing, built-in services, and other benefits offered by brpc.
- Better performance.
# Compile
brpc depends on the thrift library and reuses some code generated by thrift tools. Please read official documents to find out how to write thrift files, generate code, compilations etc.
brpc does not enable thrift support or depend on the thrift lib by default. If the support is needed, compile brpc with extra --with-thrift or -DWITH_THRIFT=ON
Install thrift under Linux
Read [Official wiki](https://thrift.apache.org/docs/install/debian) to install depended libs and tools, then download thrift source code from [official site](https://thrift.apache.org/download), uncompress and compile。
```bash
wget https://downloads.apache.org/thrift/0.22.0/thrift-0.22.0.tar.gz
tar -xf thrift-0.22.0.tar.gz
cd thrift-0.22.0/
./bootstrap.sh
./configure --prefix=/usr --with-ruby=no --with-python=no --with-java=no --with-go=no --with-perl=no --with-php=no --with-csharp=no --with-erlang=no --with-lua=no --with-nodejs=no --with-rs=no --with-py3=no CXXFLAGS='-Wno-error'
make CPPFLAGS=-DFORCE_BOOST_SMART_PTR -j 4 -s
sudo make install
```
Config brpc with thrift support, then make. The compiled libbrpc.a includes extended code for thrift support and can be linked normally as in other brpc projects.
```bash
# Ubuntu
sh config_brpc.sh --headers=/usr/include --libs=/usr/lib --with-thrift
# Fedora/CentOS
sh config_brpc.sh --headers=/usr/include --libs=/usr/lib64 --with-thrift
# Or use cmake
mkdir build && cd build && cmake ../ -DWITH_THRIFT=ON
```
Read [Getting Started](getting_started.md) for more compilation options.
# Client accesses thrift server
Steps：
- Create a Channel setting protocol to brpc::PROTOCOL_THRIFT
- Create brpc::ThriftStub
- Use native request and response to start RPC directly.
Example code:
```c++
#include <brpc/channel.h>
#include <brpc/thrift_message.h>         // Defines ThriftStub
...
DEFINE_string(server, "0.0.0.0:8019", "IP Address of thrift server");
DEFINE_string(load_balancer, "", "The algorithm for load balancing");
...
brpc::ChannelOptions options;
options.protocol = brpc::PROTOCOL_THRIFT;
brpc::Channel thrift_channel;
if (thrift_channel.Init(Flags_server.c_str(), FLAGS_load_balancer.c_str(), &options) != 0) {
LOG(ERROR) << "Fail to initialize thrift channel";
return -1;
}
brpc::ThriftStub stub(&thrift_channel);
...
// example::[EchoRequest/EchoResponse] are types generated by thrift
example::EchoRequest req;
example::EchoResponse res;
req.data = "hello";
stub.CallMethod("Echo", &cntl, &req, &res, NULL);
if (cntl.Failed()) {
LOG(ERROR) << "Fail to send thrift request, " << cntl.ErrorText();
return -1;
}
```
# Server processes thrift requests
Inherit brpc::ThriftService to implement the processing code, which may call the native handler generated by thrift to re-use existing entry directly, or read the request and set the response directly just as in other protobuf services.
```c++
class EchoServiceImpl : public brpc::ThriftService {
public:
void ProcessThriftFramedRequest(brpc::Controller* cntl,
brpc::ThriftFramedMessage* req,
brpc::ThriftFramedMessage* res,
google::protobuf::Closure* done) override {
// Dispatch calls to different methods
if (cntl->thrift_method_name() == "Echo") {
return Echo(cntl, req->Cast<example::EchoRequest>(),
res->Cast<example::EchoResponse>(), done);
} else {
cntl->SetFailed(brpc::ENOMETHOD, "Fail to find method=%s",
cntl->thrift_method_name().c_str());
done->Run();
}
}
void Echo(brpc::Controller* cntl,
const example::EchoRequest* req,
example::EchoResponse* res,
google::protobuf::Closure* done) {
// This object helps you to call done->Run() in RAII style. If you need
// to process the request asynchronously, pass done_guard.release().
brpc::ClosureGuard done_guard(done);
res->data = req->data + " (processed)";
}
};
```
Set the implemented service to ServerOptions.thrift_service and start the service.
```c++
brpc::Server server;
brpc::ServerOptions options;
options.thrift_service = new EchoServiceImpl;
options.idle_timeout_sec = FLAGS_idle_timeout_s;
options.max_concurrency = FLAGS_max_concurrency;
// Start the server.
if (server.Start(FLAGS_port, &options) != 0) {
LOG(ERROR) << "Fail to start EchoServer";
return -1;
}
```
# Performance test for native thrift compare with brpc thrift implementaion
Test Env: 48 core 2.30GHz
## server side return string "hello" sent from client
Framework | Threads Num | QPS | Avg lantecy | CPU
---- | --- | --- | --- | ---
native thrift | 60 | 6.9w | 0.9ms | 2.8%
brpc thrift | 60 | 30w | 0.2ms | 18%
## server side return string "hello" * 1000
Framework | Threads Num | QPS | Avg lantecy | CPU
---- | --- | --- | --- | ---
native thrift | 60 | 5.2w | 1.1ms | 4.5%
brpc thrift | 60 | 19.5w | 0.3ms | 22%
## server side do some complicated math and return string "hello" * 1000
Framework | Threads Num | QPS | Avg lantecy | CPU
---- | --- | --- | --- | ---
native thrift | 60 | 1.7w | 3.5ms | 76%
brpc thrift | 60 | 2.1w | 2.9ms | 93%
docs/en/timeout_concurrency_limiter.md
+7
View File
@@ -0,0 +1,7 @@
# timeout concurrency limiter
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/timeout_concurrency_limiter.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/timer_keeping.md
+7
View File
@@ -0,0 +1,7 @@
# timer keeping
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/timer_keeping.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/tutorial_on_building_services.pptx
BIN
View File
Binary file not shown.
docs/en/ub_client.md
+7
View File
@@ -0,0 +1,7 @@
# uu client
This document has not yet been translated into English.
Please refer to the [Chinese version](../cn/ub_client.md) for the full content.
Contributions to translate this document are welcome. See [TRANSLATING](TRANSLATING) for guidelines.
docs/en/vars.md
+101
View File
@@ -0,0 +1,101 @@
[中文版](../cn/vars.md)
[bvar](https://github.com/apache/brpc/tree/master/src/bvar/) is a set of counters to record and view miscellaneous statistics conveniently in multi-threaded applications. The implementation reduces cache bouncing by storing data in thread local storage(TLS), being much faster than UbMonitor(a legacy counting library inside Baidu) and even atomic operations in highly contended scenarios. brpc integrates bvar by default, namely all exposed bvars in a server are accessible through [/vars](http://brpc.baidu.com:8765/vars), and a single bvar is addressable by [/vars/VARNAME](http://brpc.baidu.com:8765/vars/rpc_socket_count). Read [bvar](bvar.md) to know how to add bvars for your program. brpc extensively use bvar to expose internal status. If you are looking for an utility to collect and display metrics of your application, consider bvar in the first place. bvar definitely can't replace all counters, essentially it moves contentions occurred during write to read: which needs to combine all data written by all threads and becomes much slower than an ordinary read. If read and write on the counter are both frequent or decisions need to be made based on latest values, you should not use bvar.
## Query methods
[/vars](http://brpc.baidu.com:8765/vars) : List all exposed bvars
[/vars/NAME](http://brpc.baidu.com:8765/vars/rpc_socket_count)：List the bvar whose name is `NAME`
[/vars/NAME1,NAME2,NAME3](http://brpc.baidu.com:8765/vars/pid;process_cpu_usage;rpc_controller_count)：List bvars whose names are either `NAME1`, `NAME2` or `NAME3`.
[/vars/foo*,b$r](http://brpc.baidu.com:8765/vars/rpc_server*_count;iobuf_blo$k_*): List bvars whose names match given wildcard patterns. Note that `$` matches a single character instead of `?` which is a reserved character in URL.
Following animation shows how to find bvars with wildcard patterns. You can copy and paste the URL to others who will see same bvars that you see. (values may change)
![img](../images/vars_1.gif)
There's a search box in the upper-left corner on /vars page, in which you can type part of the names to locate bvars. Different patterns are separated by `,` `:` or space.
![img](../images/vars_2.gif)
/vars is accessible from terminal as well:
```shell
$ curl brpc.baidu.com:8765/vars/bthread*
bthread_creation_count : 125134
bthread_creation_latency : 3
bthread_creation_latency_50 : 3
bthread_creation_latency_90 : 5
bthread_creation_latency_99 : 7
bthread_creation_latency_999 : 12
bthread_creation_latency_9999 : 12
bthread_creation_latency_cdf : "click to view"
bthread_creation_latency_percentiles : "[3,5,7,12]"
bthread_creation_max_latency : 7
bthread_creation_qps : 100
bthread_group_status : "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 "
bthread_num_workers : 24
bthread_worker_usage : 1.01056
```
## View historical trends
Clicking on most of the numerical bvars shows historical trends. Each clickable bvar records values in recent *60 seconds, 60 minutes, 24 hours and 30 days*, which are *174* numbers in total. 1000 clickable bvars take roughly 1M memory.
![img](../images/vars_3.gif)
## Calculate and view percentiles
x-ile (short for x-th percentile) is the value ranked at N * x%-th position amongst a group of ordered values. E.g. If there're 1000 values inside a time window, sort them in ascending order first. The 500-th value(1000 * 50%) in the ordered list is 50-ile(a.k.a median), the 990-th(1000 * 99%) value is 99-ile, the 999-th value is 99.9-ile. Percentiles give more information on how latencies distribute than mean values, and being helpful for analyzing behavior of the system more accurately. Industrial-grade services often require SLA to be not less than 99.97% (the requirement for 2nd-level services inside Baidu, >=99.99% for 1st-level services), even if a system has good average latencies, a bad long-tail area may still break SLA. Percentiles do help analyzing the long-tail area.
Percentiles can be plotted as a CDF or percentiles-over-time curve.
**Following diagram plots percentiles as CDF**, where the X-axis is the ratio(ranked-position/total-number) and the Y-axis is the corresponding percentile. E.g. The Y value corresponding to X=50% is 50-ile. If a system requires that "99.9% requests need to be processed within Y milliseconds", you should check the Y at 99.9%.
![img](../images/vars_4.png)
Why do we call it [CDF](https://en.wikipedia.org/wiki/Cumulative_distribution_function) ? When a Y=y is chosen, the corresponding X means "percentage of values <= y". Since values are sampled randomly (and uniformly), the X can be viewed as "probability of values <= y", or P(values <= y), which is just the definition of CDF.
Derivative of the CDF is [PDF](https://en.wikipedia.org/wiki/Probability_density_function). If we divide the Y-axis of the CDF into many small-range segments, calculate the difference between X values of both ends of each segment, and use the difference as new value for X-axis, a PDF curve would be plotted, just like a normal distribution rotated 90 degrees clockwise. However density of the median is often much higher than others in a PDF and probably make long-tail area very flat and hard to read. As a result, systems prefer showing distributions in CDF rather than PDF.
Here're 2 simple rules to check if a CDF curve is good or not:
- The flatter the better. A horizontal line is an ideal CDF curve which means that there're no waitings, congestions or pauses, very unlikely in practice.
- The area between 99% and 100% should be as small as possible: right-side of 99% is the long-tail area, which has a significant impact on SLA.
A CDF with slowly ascending curve and small long-tail area is great in practice.
**Following diagram plots percentiles over time** and has four curves. The X-axis is time and Y-axis from top to bottom are 99.9% 99% 90% 50% percentiles respectively, plotted in lighter and lighter colors (from orange to yellow).
![img](../images/vars_5.png)
Hovering mouse over the curves shows corresponding values at the time. The tooltip in above diagram means "The 99% percentile of latency before 39 seconds is 330 **microseconds**". The diagram does not include the 99.99-ile curve which is usually significantly higher than others, making others hard to read. You may click bvars ended with "\_latency\_9999" to read the 99.99-ile curve separately. This diagram shows how percentiles change over time, which is helpful to analyze performance regressions of systems.
brpc calculates latency distributions of services automatically, which do not need users to add manually. The metrics are as follows:
![img](../images/vars_6.png)
`bvar::LatencyRecorder` is able to calculate latency distributions of any code, as depicted below. (checkout [bvar-c++](bvar_c++.md) for details):
```c++
#include <bvar/bvar.h>
...
bvar::LatencyRecorder g_latency_recorder("client"); // expose this recorder
...
void foo() {
...
g_latency_recorder << my_latency;
...
}
```
If the application already starts a brpc server, values like `client_latency`, `client_latency_cdf` can be viewed from `/vars` as follows. Clicking them to see (dynamically-updated) curves:
![img](../images/vars_7.png)
## Non brpc server
If your program only uses brpc client or even not use brpc, and you also want to view the curves, check [here](../cn/dummy_server.md).
docs/en/wireshark_baidu_std.md
+27
View File
@@ -0,0 +1,27 @@
[中文版](../cn/wireshark_baidu_std.md)
# Overview
`wireshark_baidu_std.lua` is a Wireshark Lua dissector written for [`baidu_std`](../cn/baidu_std.md) protocol, including the [`streaming_rpc`](streaming_rpc.md) protocol.
Example for the Echo request:
![request](../images/wireshark_baidu_std_request.png)
Example for the Echo response:
![response](../images/wireshark_baidu_std_response.png)
## How to use
1. Put [`wireshark_baidu_std.lua`](../../tools/wireshark_baidu_std.lua) under "Personal Lua Plugins"；
1. And put [`options.proto`](../../src/brpc/options.proto), [`streaming_rpc_meta.proto`](../../src/brpc/streaming_rpc_meta.proto) and [`baidu_rpc_meta.proto`](../../src/brpc/policy/baidu_rpc_meta.proto) in `protobuf` directory under "Personal configuration", create if not exist；
1. Set `Protobuf Search Paths` for Protobuf official library include directory(e.g. `/opt/homebrew/opt/protobuf/include`), see [Wireshark Protobuf](https://wiki.wireshark.org/Protobuf#protobuf-search-paths-settings) for more details.
![wireshark-protobuf-search-paths](../images/wireshark_protobuf_search_paths.png)
1. Optional, turn on `Dissect Protobuf fields as Wireshark fields` if you want to use related `baidu_std` fields for filtering:
![wireshark-protobuf-settings](../images/wireshark_protobuf_settings.png)
The "Personal Lua Plugins" and "Personal configuration" folders above can be found under the `Folders` page of `About Wireshark`, for macOS:
![About Wireshark](../images/wireshark_folders.png)