Compare commits

..

12 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] ec0e61bd40 Remove unused maxIdle field from connection pools
Run Tests / Unit Tests (push) Waiting to run
Run Tests / Etcd Integration Tests (push) Waiting to run
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 07:07:23 +00:00
copilot-swe-agent[bot] 16c31b25bc Fix race condition in connection pool lastUsed field access
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 07:06:02 +00:00
copilot-swe-agent[bot] 03223d5e23 Fix connection leaks in events/natsjs and config/source/nats
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 07:02:36 +00:00
copilot-swe-agent[bot] 25f1c6714c Implement connection pool for transport/nats
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 07:00:05 +00:00
copilot-swe-agent[bot] 8309562352 Implement NATS connection pool with configuration options
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 06:56:26 +00:00
copilot-swe-agent[bot] d3bf22f732 Initial plan 2026-02-02 19:41:26 +00:00
Copilot 87cf988e03 Fix go install @latest failures by documenting specific version (#2839)
* Initial plan

* Update documentation to use @v5.13.0 instead of @latest for go install commands

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add explanatory notes about version pinning in documentation

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add consistent explanatory notes across all documentation files

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-29 10:19:55 +00:00
Asim Aslam 109ff2169a Add blog link to homepage (#2837)
Link to /blog/ from the main navigation, replacing the badge link.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 14:08:32 +00:00
Shelley 82b7fdbec4 Add install.sh to website for curl install
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 14:02:38 +00:00
Shelley fea8c911be Fix blog post markdown rendering
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:54:31 +00:00
Shelley cf9629c41f Add blog section with first post: Introducing micro deploy
- /blog/ - Blog index page
- /blog/1 - First post announcing micro deploy
- /docs/deployment.md - Deployment guide in website docs
- Updated navigation to include Blog link
- New blog layout template

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:52:09 +00:00
Asim Aslam a5bef7af29 Add systemd-based deployment support (#2836)
* Add systemd-based deployment support

- micro init --server: Initialize server to receive deployments
  - Creates /opt/micro/{bin,data,config} directories
  - Generates systemd template unit (micro@.service)
  - Creates 'micro' system user

- micro deploy: Deploy services via SSH + systemd
  - Builds linux/amd64 binaries automatically
  - Copies via rsync/scp to server
  - Manages services via systemctl
  - Helpful error messages for common issues

- micro status --remote: Check remote service status
- micro logs --remote: Stream remote logs via journalctl
- micro stop --remote: Stop services on remote server

- Config: Added 'deploy' blocks to micro.mu for named targets

The deployment model:
- systemd is the process supervisor (battle-tested)
- SSH is the transport (standard, secure)
- No custom daemons or platforms needed

Co-authored-by: Shelley <shelley@exe.dev>

* Add deployment documentation

- docs/deployment.md: Comprehensive guide for server deployment
- README.md: Updated deployment section with full workflow

Co-authored-by: Shelley <shelley@exe.dev>

* Add deployment section to CLI documentation

Co-authored-by: Shelley <shelley@exe.dev>

* Fix systemd template escaping and rsync permission warnings

- Fix %i escaping in systemd template (was being interpreted by fmt.Sprintf)
- Handle rsync exit code 23/24 gracefully (metadata permission warnings)
- Add --omit-dir-times to rsync to avoid directory timestamp errors

Co-authored-by: Shelley <shelley@exe.dev>

* Add install script for micro CLI

Co-authored-by: Shelley <shelley@exe.dev>

* Fix non-constant format string in deploy error

Co-authored-by: Shelley <shelley@exe.dev>

---------

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:27:59 +00:00
25 changed files with 1742 additions and 63 deletions
+6 -2
View File
@@ -117,9 +117,11 @@ There's a new `genai` package for generative AI capabilities.
Install the code generator and see usage in the docs:
```bash
go install go-micro.dev/v5/cmd/protoc-gen-micro@latest
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
Docs: [`internal/website/docs/getting-started.md`](internal/website/docs/getting-started.md)
## Command Line
@@ -127,9 +129,11 @@ Docs: [`internal/website/docs/getting-started.md`](internal/website/docs/getting
Install the CLI:
```
go install go-micro.dev/v5/cmd/micro@latest
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
### Quick Start
```bash
+127 -14
View File
@@ -6,6 +6,7 @@ import (
"errors"
"strings"
"sync"
"time"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
@@ -22,10 +23,15 @@ type natsBroker struct {
connected bool
addrs []string
conn *natsp.Conn
conn *natsp.Conn // single connection (used when pool is disabled)
pool *connectionPool // connection pool (used when pooling is enabled)
opts broker.Options
nopts natsp.Options
// pool configuration
poolSize int
poolIdleTimeout time.Duration
// should we drain the connection
drain bool
closeCh chan (error)
@@ -109,6 +115,39 @@ func (n *natsBroker) Connect() error {
return nil
}
// Check if we should use connection pooling
if n.poolSize > 1 {
// Initialize connection pool
factory := func() (*natsp.Conn, error) {
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
return opts.Connect()
}
pool, err := newConnectionPool(n.poolSize, factory)
if err != nil {
return err
}
// Set idle timeout if configured
if n.poolIdleTimeout > 0 {
pool.idleTimeout = n.poolIdleTimeout
}
n.pool = pool
n.connected = true
return nil
}
// Single connection mode (original behavior)
status := natsp.CLOSED
if n.conn != nil {
status = n.conn.Status()
@@ -143,14 +182,26 @@ func (n *natsBroker) Disconnect() error {
n.Lock()
defer n.Unlock()
// drain the connection if specified
if n.drain {
n.conn.Drain()
n.closeCh <- nil
// Close connection pool if it exists
if n.pool != nil {
if err := n.pool.Close(); err != nil {
n.opts.Logger.Log(logger.ErrorLevel, "error closing connection pool:", err)
}
n.pool = nil
}
// close the client connection
n.conn.Close()
// Close single connection if it exists
if n.conn != nil {
// drain the connection if specified
if n.drain {
n.conn.Drain()
n.closeCh <- nil
}
// close the client connection
n.conn.Close()
n.conn = nil
}
// set not connected
n.connected = false
@@ -171,24 +222,42 @@ func (n *natsBroker) Publish(topic string, msg *broker.Message, opts ...broker.P
n.RLock()
defer n.RUnlock()
if n.conn == nil {
return errors.New("not connected")
}
b, err := n.opts.Codec.Marshal(msg)
if err != nil {
return err
}
// Use connection pool if enabled
if n.pool != nil {
poolConn, err := n.pool.Get()
if err != nil {
return err
}
defer n.pool.Put(poolConn)
conn := poolConn.Conn()
if conn == nil {
return errors.New("invalid connection from pool")
}
return conn.Publish(topic, b)
}
// Use single connection (original behavior)
if n.conn == nil {
return errors.New("not connected")
}
return n.conn.Publish(topic, b)
}
func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
n.RLock()
if n.conn == nil {
n.RUnlock()
hasConnection := n.conn != nil || n.pool != nil
n.RUnlock()
if !hasConnection {
return nil, errors.New("not connected")
}
n.RUnlock()
opt := broker.SubscribeOptions{
AutoAck: true,
@@ -226,6 +295,38 @@ func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...bro
var sub *natsp.Subscription
var err error
// Use connection pool if enabled
if n.pool != nil {
poolConn, err := n.pool.Get()
if err != nil {
return nil, err
}
conn := poolConn.Conn()
if conn == nil {
n.pool.Put(poolConn)
return nil, errors.New("invalid connection from pool")
}
if len(opt.Queue) > 0 {
sub, err = conn.QueueSubscribe(topic, opt.Queue, fn)
} else {
sub, err = conn.Subscribe(topic, fn)
}
if err != nil {
n.pool.Put(poolConn)
return nil, err
}
// Return connection to pool after subscription is created
// The subscription keeps the connection alive
n.pool.Put(poolConn)
return &subscriber{s: sub, opts: opt}, nil
}
// Use single connection (original behavior)
n.RLock()
if len(opt.Queue) > 0 {
sub, err = n.conn.QueueSubscribe(topic, opt.Queue, fn)
@@ -250,12 +351,24 @@ func (n *natsBroker) setOption(opts ...broker.Option) {
n.Once.Do(func() {
n.nopts = natsp.GetDefaultOptions()
n.poolSize = 1 // Default to single connection (no pooling)
n.poolIdleTimeout = 5 * time.Minute
})
if nopts, ok := n.opts.Context.Value(optionsKey{}).(natsp.Options); ok {
n.nopts = nopts
}
// Set pool size if configured
if poolSize, ok := n.opts.Context.Value(poolSizeKey{}).(int); ok && poolSize > 0 {
n.poolSize = poolSize
}
// Set pool idle timeout if configured
if idleTimeout, ok := n.opts.Context.Value(poolIdleTimeoutKey{}).(time.Duration); ok {
n.poolIdleTimeout = idleTimeout
}
// broker.Options have higher priority than nats.Options
// only if Addrs, Secure or TLSConfig were not set through a broker.Option
// we read them from nats.Option
+18
View File
@@ -1,12 +1,16 @@
package nats
import (
"time"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
)
type optionsKey struct{}
type drainConnectionKey struct{}
type poolSizeKey struct{}
type poolIdleTimeoutKey struct{}
// Options accepts nats.Options.
func Options(opts natsp.Options) broker.Option {
@@ -17,3 +21,17 @@ func Options(opts natsp.Options) broker.Option {
func DrainConnection() broker.Option {
return setBrokerOption(drainConnectionKey{}, struct{}{})
}
// PoolSize sets the size of the connection pool.
// If set to a value > 1, the broker will use a connection pool.
// Default is 1 (no pooling).
func PoolSize(size int) broker.Option {
return setBrokerOption(poolSizeKey{}, size)
}
// PoolIdleTimeout sets the timeout for idle connections in the pool.
// Connections idle for longer than this duration will be closed.
// Default is 5 minutes. Set to 0 to disable idle timeout.
func PoolIdleTimeout(timeout time.Duration) broker.Option {
return setBrokerOption(poolIdleTimeoutKey{}, timeout)
}
+188
View File
@@ -0,0 +1,188 @@
package nats
import (
"errors"
"sync"
"time"
natsp "github.com/nats-io/nats.go"
)
var (
// ErrPoolExhausted is returned when no connections are available in the pool
ErrPoolExhausted = errors.New("connection pool exhausted")
// ErrPoolClosed is returned when trying to use a closed pool
ErrPoolClosed = errors.New("connection pool is closed")
)
// connectionPool manages a pool of NATS connections
type connectionPool struct {
mu sync.RWMutex
connections chan *pooledConnection
factory func() (*natsp.Conn, error)
size int
idleTimeout time.Duration
closed bool
}
// pooledConnection wraps a NATS connection with metadata
type pooledConnection struct {
conn *natsp.Conn
createdAt time.Time
lastUsed time.Time
mu sync.Mutex
}
// newConnectionPool creates a new connection pool
func newConnectionPool(size int, factory func() (*natsp.Conn, error)) (*connectionPool, error) {
if size <= 0 {
size = 1
}
pool := &connectionPool{
connections: make(chan *pooledConnection, size),
factory: factory,
size: size,
idleTimeout: 5 * time.Minute,
closed: false,
}
return pool, nil
}
// Get retrieves a connection from the pool or creates a new one
func (p *connectionPool) Get() (*pooledConnection, error) {
p.mu.RLock()
if p.closed {
p.mu.RUnlock()
return nil, ErrPoolClosed
}
p.mu.RUnlock()
// Try to get an existing connection from the pool
select {
case conn := <-p.connections:
// Check if connection is still valid and not idle for too long
if conn.isValid() && !conn.isExpired(p.idleTimeout) {
conn.updateLastUsed()
return conn, nil
}
// Connection is invalid or expired, close it and create a new one
conn.close()
return p.createConnection()
default:
// No connection available, create a new one
return p.createConnection()
}
}
// Put returns a connection to the pool
func (p *connectionPool) Put(conn *pooledConnection) error {
p.mu.RLock()
defer p.mu.RUnlock()
if p.closed {
return conn.close()
}
// Check if connection is still valid
if !conn.isValid() {
return conn.close()
}
conn.updateLastUsed()
// Try to return connection to pool
select {
case p.connections <- conn:
return nil
default:
// Pool is full, close the connection
return conn.close()
}
}
// Close closes all connections in the pool
func (p *connectionPool) Close() error {
p.mu.Lock()
defer p.mu.Unlock()
if p.closed {
return nil
}
p.closed = true
close(p.connections)
// Close all connections in the pool
for conn := range p.connections {
conn.close()
}
return nil
}
// createConnection creates a new pooled connection
func (p *connectionPool) createConnection() (*pooledConnection, error) {
conn, err := p.factory()
if err != nil {
return nil, err
}
return &pooledConnection{
conn: conn,
createdAt: time.Now(),
lastUsed: time.Now(),
}, nil
}
// isValid checks if the underlying NATS connection is valid
func (pc *pooledConnection) isValid() bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn == nil {
return false
}
status := pc.conn.Status()
return status == natsp.CONNECTED || status == natsp.RECONNECTING
}
// isExpired checks if the connection has been idle for too long
func (pc *pooledConnection) isExpired(timeout time.Duration) bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if timeout <= 0 {
return false
}
return time.Since(pc.lastUsed) > timeout
}
// close closes the underlying NATS connection
func (pc *pooledConnection) close() error {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn != nil {
pc.conn.Close()
pc.conn = nil
}
return nil
}
// Conn returns the underlying NATS connection
func (pc *pooledConnection) Conn() *natsp.Conn {
pc.mu.Lock()
defer pc.mu.Unlock()
return pc.conn
}
// updateLastUsed updates the last used timestamp in a thread-safe manner
func (pc *pooledConnection) updateLastUsed() {
pc.mu.Lock()
defer pc.mu.Unlock()
pc.lastUsed = time.Now()
}
+204
View File
@@ -0,0 +1,204 @@
package nats
import (
"sync"
"testing"
"time"
natsp "github.com/nats-io/nats.go"
)
func TestConnectionPool_GetPut(t *testing.T) {
// Mock factory that creates connections
connCount := 0
factory := func() (*natsp.Conn, error) {
connCount++
// Return a mock connection (we can't create real NATS connections in tests without a server)
// This test is more about the pool logic
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Get a connection (should create one)
conn1, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
if conn1 == nil {
t.Fatal("Expected connection, got nil")
}
// Put it back
if err := pool.Put(conn1); err != nil {
t.Fatalf("Failed to put connection: %v", err)
}
// Get it again (should reuse the same one)
conn2, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
// Since we can't compare actual connections easily, just verify we got one
if conn2 == nil {
t.Fatal("Expected connection, got nil")
}
}
func TestConnectionPool_Concurrent(t *testing.T) {
connCount := 0
mu := sync.Mutex{}
factory := func() (*natsp.Conn, error) {
mu.Lock()
connCount++
mu.Unlock()
return nil, nil
}
pool, err := newConnectionPool(5, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Simulate concurrent access
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
conn, err := pool.Get()
if err != nil {
t.Errorf("Failed to get connection: %v", err)
return
}
// Simulate some work
time.Sleep(10 * time.Millisecond)
if err := pool.Put(conn); err != nil {
t.Errorf("Failed to put connection: %v", err)
}
}()
}
wg.Wait()
// We should have created some connections
mu.Lock()
if connCount == 0 {
t.Error("Expected at least one connection to be created")
}
mu.Unlock()
}
func TestConnectionPool_Close(t *testing.T) {
factory := func() (*natsp.Conn, error) {
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
// Get a connection
conn, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
// Close the pool
if err := pool.Close(); err != nil {
t.Fatalf("Failed to close pool: %v", err)
}
// Put connection back to closed pool should not panic
// The connection will be closed instead of returned to pool
_ = pool.Put(conn)
// Try to get from closed pool
_, err = pool.Get()
if err != ErrPoolClosed {
t.Errorf("Expected ErrPoolClosed, got: %v", err)
}
}
func TestPooledConnection_IsValid(t *testing.T) {
pc := &pooledConnection{
conn: nil, // nil connection should be invalid
createdAt: time.Now(),
lastUsed: time.Now(),
}
if pc.isValid() {
t.Error("Expected nil connection to be invalid")
}
}
func TestPooledConnection_IsExpired(t *testing.T) {
pc := &pooledConnection{
conn: nil,
createdAt: time.Now(),
lastUsed: time.Now().Add(-10 * time.Minute), // 10 minutes ago
}
// With 5 minute timeout, should be expired
if !pc.isExpired(5 * time.Minute) {
t.Error("Expected connection to be expired")
}
// With 0 timeout, should never expire
if pc.isExpired(0) {
t.Error("Expected connection not to expire with 0 timeout")
}
// With 20 minute timeout, should not be expired
if pc.isExpired(20 * time.Minute) {
t.Error("Expected connection not to be expired")
}
}
func TestNatsBroker_PoolConfiguration(t *testing.T) {
// Test that pool size is set correctly
br := NewNatsBroker(PoolSize(5))
nb, ok := br.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb.poolSize != 5 {
t.Errorf("Expected pool size 5, got %d", nb.poolSize)
}
// Test with custom idle timeout
br2 := NewNatsBroker(PoolSize(3), PoolIdleTimeout(10*time.Minute))
nb2, ok := br2.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb2.poolSize != 3 {
t.Errorf("Expected pool size 3, got %d", nb2.poolSize)
}
if nb2.poolIdleTimeout != 10*time.Minute {
t.Errorf("Expected idle timeout 10m, got %v", nb2.poolIdleTimeout)
}
}
func TestNatsBroker_DefaultSingleConnection(t *testing.T) {
// Test that default behavior is single connection (pool size 1)
br := NewNatsBroker()
nb, ok := br.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb.poolSize != 1 {
t.Errorf("Expected default pool size 1, got %d", nb.poolSize)
}
}
+3 -1
View File
@@ -13,9 +13,11 @@ write services. Surrounding this we introduce a number of tools to make it easy
Install `micro` via `go install`
```
go install go-micro.dev/v5@latest
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
Or via install script
```
+1 -1
View File
@@ -36,7 +36,7 @@ SyslogIdentifier=micro-%%i
# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=false
ProtectHome=true
ReadWritePaths=%s/data
[Install]
+12
View File
@@ -15,6 +15,7 @@ type nats struct {
url string
bucket string
key string
conn *natsgo.Conn // store connection for lifecycle management
kv natsgo.KeyValue
opts source.Options
}
@@ -128,7 +129,18 @@ func NewSource(opts ...source.Option) source.Source {
url: config.Url,
bucket: bucket,
key: key,
conn: nc, // store connection reference
kv: kv,
opts: options,
}
}
// Close implements io.Closer and closes the underlying NATS connection.
// This method is optional but recommended to prevent connection leaks.
func (n *nats) Close() error {
if n.conn != nil {
n.conn.Close()
n.conn = nil
}
return nil
}
+22 -5
View File
@@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"strings"
"time"
@@ -35,11 +36,12 @@ func NewStream(opts ...Option) (events.Stream, error) {
s := &stream{opts: options}
natsJetStreamCtx, err := connectToNatsJetStream(options)
conn, natsJetStreamCtx, err := connectToNatsJetStream(options)
if err != nil {
return nil, fmt.Errorf("error connecting to nats cluster %v: %w", options.ClusterID, err)
}
s.conn = conn
s.natsJetStreamCtx = natsJetStreamCtx
return s, nil
@@ -47,10 +49,11 @@ func NewStream(opts ...Option) (events.Stream, error) {
type stream struct {
opts Options
conn *nats.Conn // store connection for lifecycle management
natsJetStreamCtx nats.JetStreamContext
}
func connectToNatsJetStream(options Options) (nats.JetStreamContext, error) {
func connectToNatsJetStream(options Options) (*nats.Conn, nats.JetStreamContext, error) {
nopts := nats.GetDefaultOptions()
if options.TLSConfig != nil {
nopts.Secure = true
@@ -77,15 +80,16 @@ func connectToNatsJetStream(options Options) (nats.JetStreamContext, error) {
conn, err := nopts.Connect()
if err != nil {
tls := nopts.TLSConfig != nil
return nil, fmt.Errorf("error connecting to nats at %v with tls enabled (%v): %w", options.Address, tls, err)
return nil, nil, fmt.Errorf("error connecting to nats at %v with tls enabled (%v): %w", options.Address, tls, err)
}
js, err := conn.JetStream()
if err != nil {
return nil, fmt.Errorf("error while obtaining JetStream context: %w", err)
conn.Close() // Close connection if JetStream context fails
return nil, nil, fmt.Errorf("error while obtaining JetStream context: %w", err)
}
return js, nil
return conn, js, nil
}
// Publish a message to a topic.
@@ -268,3 +272,16 @@ func (s *stream) Consume(topic string, opts ...events.ConsumeOption) (<-chan eve
return channel, nil
}
// Close implements io.Closer and closes the underlying NATS connection.
// This method is optional but recommended to prevent connection leaks.
func (s *stream) Close() error {
if s.conn != nil {
s.conn.Close()
s.conn = nil
}
return nil
}
// Ensure stream implements io.Closer
var _ io.Closer = (*stream)(nil)
+2
View File
@@ -3,6 +3,8 @@ core:
url: /docs/
- title: Getting Started
url: /docs/getting-started.html
- title: Deployment
url: /docs/deployment.html
- title: Architecture
url: /docs/architecture.html
- title: Configuration
+68
View File
@@ -0,0 +1,68 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% if page.title %}{{ page.title }} | {% endif %}Go Micro Blog</title>
<meta name="description" content="{{ page.description | default: 'Go Micro Blog - News, updates, and tutorials for the Go Micro framework' }}">
<style>
:root {
--bg: #ffffff;
--border: #e5e5e5;
}
* { box-sizing: border-box; }
body { font-family: system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif; margin:0; background: var(--bg); color:#222; line-height: 1.7; }
a { color:#0366d6; text-decoration:none; }
a:hover { text-decoration:underline; }
header { display:flex; align-items:center; justify-content:space-between; padding:.75rem 1.5rem; background:#fff; border-bottom:1px solid var(--border); position:sticky; top:0; z-index:20; }
.logo-link { display:flex; align-items:center; gap:.5rem; font-weight:600; color:#222; }
.logo-link img { height:36px; }
header nav { display:flex; align-items:center; gap: 1rem; }
header nav a { font-weight:500; }
.container { max-width: 720px; margin: 0 auto; padding: 2rem 1.5rem 4rem; }
.blog-header { margin-bottom: 2rem; padding-bottom: 1rem; border-bottom: 1px solid var(--border); }
.blog-header h1 { margin: 0 0 0.5rem; font-size: 2rem; }
.blog-header .meta { color: #666; font-size: 0.9rem; }
article h1 { font-size: 2.25rem; margin: 0 0 0.5rem; line-height: 1.3; }
article .meta { color: #666; font-size: 0.9rem; margin-bottom: 2rem; }
article h2 { font-size: 1.5rem; margin-top: 2.5rem; }
article h3 { font-size: 1.25rem; margin-top: 2rem; }
article p { margin: 1rem 0; }
article ul, article ol { margin: 1rem 0; padding-left: 1.5rem; }
article li { margin: 0.5rem 0; }
pre, code { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
pre { background:#f6f8fa; border:1px solid #d0d7de; padding:1rem; border-radius:6px; overflow-x:auto; font-size: 0.9rem; }
code { background: #f6f8fa; padding: 0.15rem 0.3rem; border-radius: 3px; font-size: 0.9em; }
pre code { background: none; padding: 0; }
blockquote { margin: 1.5rem 0; padding: 0.5rem 1rem; border-left: 4px solid #0366d6; background: #f6f8fa; }
blockquote p { margin: 0; }
.post-nav { margin-top: 3rem; padding-top: 1.5rem; border-top: 1px solid var(--border); display: flex; justify-content: space-between; font-size: 0.9rem; }
footer { max-width: 720px; margin: 0 auto; padding: 2rem 1.5rem; border-top: 1px solid var(--border); font-size: 0.8rem; color: #666; }
@media (max-width: 600px) {
.container { padding: 1.5rem 1rem; }
article h1 { font-size: 1.75rem; }
header { padding: 0.5rem 1rem; }
header nav { gap: 0.5rem; font-size: 0.9rem; }
}
</style>
</head>
<body>
<header>
<a class="logo-link" href="/">
<img src="/images/logo.png" alt="Go Micro Logo">
<span style="font-size: 1.3rem;">Go Micro</span>
</a>
<nav>
<a href="/blog/">Blog</a>
<a href="/docs/">Docs</a>
<a href="https://github.com/micro/go-micro" target="_blank">GitHub</a>
</nav>
</header>
<div class="container">
{{ content }}
</div>
<footer>
© {{ site.time | date: '%Y' }} Go Micro. Apache 2.0 Licensed.
</footer>
</body>
</html>
+2 -1
View File
@@ -86,6 +86,7 @@
</a>
<button class="menu-toggle" id="menuToggle">☰ Menu</button>
<nav>
<a href="/blog/">Blog</a>
<a href="/docs/">Docs</a>
<a href="/docs/search.html">Search</a>
<a href="https://github.com/micro/go-micro" target="_blank" rel="noopener">GitHub</a>
@@ -151,7 +152,7 @@
</main>
</div>
<footer>
© {{ site.time | date: '%Y' }} Go Micro. Apache 2.0 Licensed. <a href="/docs/">Docs</a> · <a href="/docs/search.html">Search</a> · <a href="https://github.com/micro/go-micro">GitHub</a> · <a href="https://github.com/micro/go-micro/issues/new?labels=question&template=question.md" style="opacity:0.7">Support</a>
© {{ site.time | date: '%Y' }} Go Micro. Apache 2.0 Licensed. <a href="/blog/">Blog</a> · <a href="/docs/">Docs</a> · <a href="/docs/search.html">Search</a> · <a href="https://github.com/micro/go-micro">GitHub</a> · <a href="https://github.com/micro/go-micro/issues/new?labels=question&template=question.md" style="opacity:0.7">Support</a>
</footer>
<script>
(function(){
+131
View File
@@ -0,0 +1,131 @@
---
layout: blog
title: Introducing micro deploy
permalink: /blog/1
description: Deploy your Go Micro services to any Linux server with a single command
---
# Introducing micro deploy
*January 27, 2026 • By the Go Micro Team*
We're excited to announce **micro deploy** in Go Micro v5.13.0 — a simple way to deploy your services to any Linux server.
## The Problem
Go Micro has always been great for building microservices:
```bash
micro new myservice
cd myservice
micro run
```
But getting those services to production? That was on you. You'd need to figure out Docker, Kubernetes, or write your own deployment scripts.
We tried to solve this with Micro v3 — a full platform-as-a-service. But it was too much. Too complex. Nobody wanted another platform to manage.
## The Solution
The new approach is simple: **systemd + SSH**.
Every Linux server has systemd. It's battle-tested, it manages processes, it restarts them when they crash, it handles logging. Why reinvent it?
### One-Time Server Setup
```bash
ssh user@server
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
```
This creates:
- `/opt/micro/bin/` — where your binaries live
- `/opt/micro/config/` — environment files
- A systemd template for managing services
### Deploy
```bash
micro deploy user@server
```
That's it. The command:
1. Builds your services for Linux
2. Copies binaries via SSH
3. Configures systemd services
4. Verifies everything is running
### Manage
```bash
micro status --remote user@server
micro logs --remote user@server
micro logs myservice --remote user@server -f
```
## Named Deploy Targets
Add deploy targets to your `micro.mu`:
```
service users
path ./users
port 8081
service web
path ./web
port 8080
deploy prod
ssh deploy@prod.example.com
deploy staging
ssh deploy@staging.example.com
```
Then:
```bash
micro deploy prod
micro deploy staging
```
## Philosophy
- **systemd is the standard** — don't fight it, use it
- **SSH is the transport** — no custom agents or protocols
- **Errors guide you** — every failure tells you how to fix it
- **No platform** — just your server, your services
## What's Next?
This is just the beginning. We're thinking about:
- **Secrets management** — integrating with vault/sops
- **Multi-server deploys** — deploy to a fleet
- **Metrics** — Prometheus endpoints out of the box
- **Rolling updates** — zero-downtime deployments
## Try It
```bash
go install go-micro.dev/v5/cmd/micro@v5.13.0
micro new myapp
cd myapp
micro run
# When you're ready to deploy:
micro deploy user@your-server
```
See the [deployment guide](/docs/deployment.html) for full documentation.
---
*Go Micro is an open source framework for distributed systems development in Go. [Star us on GitHub](https://github.com/micro/go-micro).*
<div class="post-nav">
<div><a href="/blog/">← All Posts</a></div>
<div></div>
</div>
+19
View File
@@ -0,0 +1,19 @@
---
layout: blog
title: Blog
permalink: /blog/
---
<div class="blog-header">
<h1>Go Micro Blog</h1>
<p class="meta">News, updates, and tutorials for Go Micro</p>
</div>
<div class="posts">
<article style="margin-bottom: 2rem; padding-bottom: 1.5rem; border-bottom: 1px solid #e5e5e5;">
<h2 style="margin: 0 0 0.5rem;"><a href="/blog/1">Introducing micro deploy</a></h2>
<p class="meta" style="color: #666; font-size: 0.85rem;">January 27, 2026</p>
<p>Deploy your Go Micro services to any Linux server with a single command. No Docker, no Kubernetes, no platform — just systemd.</p>
<a href="/blog/1">Read more →</a>
</article>
</div>
+349
View File
@@ -0,0 +1,349 @@
---
layout: default
title: Deployment
---
# Deploying Go Micro Services
This guide covers deploying go-micro services to a Linux server using systemd.
## Overview
go-micro provides a simple deployment workflow:
1. **Develop locally** with `micro run`
2. **Build binaries** with `micro build`
3. **Deploy to server** with `micro deploy`
On the server, services are managed by systemd - the standard Linux process supervisor.
## Quick Start
### 1. Prepare Your Server
On your server (Ubuntu, Debian, or any systemd-based Linux):
```bash
# Install micro
curl -fsSL https://go-micro.dev/install.sh | sh
# Initialize for deployment
sudo micro init --server
```
This creates:
- `/opt/micro/bin/` - where service binaries live
- `/opt/micro/data/` - persistent data directory
- `/opt/micro/config/` - environment files
- systemd template for managing services
### 2. Deploy from Your Machine
```bash
# From your project directory
micro deploy user@your-server
```
That's it! The deploy command:
1. Builds your services for Linux
2. Copies binaries to the server
3. Configures and starts systemd services
4. Verifies everything is running
## Detailed Setup
### Server Requirements
- Linux with systemd (Ubuntu 16.04+, Debian 8+, CentOS 7+, etc.)
- SSH access
- Go installed (only if building on server)
### Server Initialization Options
```bash
# Basic setup (creates 'micro' user)
sudo micro init --server
# Custom installation path
sudo micro init --server --path /home/deploy/micro
# Run services as existing user
sudo micro init --server --user deploy
# Initialize remotely (from your laptop)
micro init --server --remote user@your-server
```
### What Gets Created
**Directories:**
```
/opt/micro/
├── bin/ # Service binaries
├── data/ # Persistent data (databases, files)
└── config/ # Environment files (*.env)
```
**Systemd Template** (`/etc/systemd/system/micro@.service`):
```ini
[Unit]
Description=Micro service: %i
After=network.target
[Service]
Type=simple
User=micro
WorkingDirectory=/opt/micro
ExecStart=/opt/micro/bin/%i
Restart=on-failure
RestartSec=5
EnvironmentFile=-/opt/micro/config/%i.env
[Install]
WantedBy=multi-user.target
```
The `%i` is replaced with the service name. So `micro@users.service` runs `/opt/micro/bin/users`.
## Deployment
### Basic Deploy
```bash
micro deploy user@server
```
### Deploy Specific Service
```bash
micro deploy user@server --service users
```
### Force Rebuild
```bash
micro deploy user@server --build
```
### Named Deploy Targets
Add to your `micro.mu`:
```
service users
path ./users
port 8081
service web
path ./web
port 8080
deploy prod
ssh deploy@prod.example.com
deploy staging
ssh deploy@staging.example.com
```
Then:
```bash
micro deploy prod # deploys to prod.example.com
micro deploy staging # deploys to staging.example.com
```
## Managing Services
### Check Status
```bash
# Local services
micro status
# Remote services
micro status --remote user@server
```
Output:
```
server.example.com
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
users ● running pid 1234
posts ● running pid 1235
web ● running pid 1236
```
### View Logs
```bash
# All services
micro logs --remote user@server
# Specific service
micro logs users --remote user@server
# Follow logs
micro logs users --remote user@server -f
```
### Stop Services
```bash
micro stop users --remote user@server
```
### Direct systemctl Access
You can also manage services directly on the server:
```bash
# Status
sudo systemctl status micro@users
# Restart
sudo systemctl restart micro@users
# Stop
sudo systemctl stop micro@users
# Logs
journalctl -u micro@users -f
```
## Environment Variables
Create environment files at `/opt/micro/config/<service>.env`:
```bash
# /opt/micro/config/users.env
DATABASE_URL=postgres://localhost/users
REDIS_URL=redis://localhost:6379
LOG_LEVEL=info
```
These are automatically loaded by systemd when the service starts.
## SSH Setup
### Key-Based Authentication
```bash
# Generate key (if you don't have one)
ssh-keygen -t ed25519
# Copy to server
ssh-copy-id user@server
```
### SSH Config
Add to `~/.ssh/config`:
```
Host prod
HostName prod.example.com
User deploy
IdentityFile ~/.ssh/deploy_key
Host staging
HostName staging.example.com
User deploy
IdentityFile ~/.ssh/deploy_key
```
Then deploy with:
```bash
micro deploy prod
```
## Troubleshooting
### "Cannot connect to server"
```
✗ Cannot connect to myserver
SSH connection failed. Check that:
• The server is reachable: ping myserver
• SSH is configured: ssh user@myserver
• Your key is added: ssh-add -l
```
**Fix:**
```bash
# Test SSH connection
ssh user@server
# Add SSH key
ssh-copy-id user@server
# Check SSH agent
eval $(ssh-agent)
ssh-add
```
### "Server not initialized"
```
✗ Server not initialized
micro is not set up on myserver.
```
**Fix:**
```bash
ssh user@server 'sudo micro init --server'
```
### "Service failed to start"
Check the logs:
```bash
micro logs myservice --remote user@server
# Or on the server:
journalctl -u micro@myservice -n 50
```
Common causes:
- Missing environment variables
- Port already in use
- Database not reachable
- Binary permissions issue
### "Permission denied"
Ensure your user can write to `/opt/micro/bin/`:
```bash
# On server
sudo chown -R deploy:deploy /opt/micro
# Or add user to micro group
sudo usermod -aG micro deploy
```
## Security Best Practices
1. **Use a dedicated deploy user** - Don't deploy as root
2. **Use SSH keys** - Disable password authentication
3. **Restrict sudo** - Only allow necessary commands
4. **Firewall** - Only expose needed ports
5. **Secrets** - Use environment files with restricted permissions (0600)
### Minimal sudo access
Add to `/etc/sudoers.d/micro`:
```
deploy ALL=(ALL) NOPASSWD: /bin/systemctl daemon-reload
deploy ALL=(ALL) NOPASSWD: /bin/systemctl enable micro@*
deploy ALL=(ALL) NOPASSWD: /bin/systemctl restart micro@*
deploy ALL=(ALL) NOPASSWD: /bin/systemctl stop micro@*
deploy ALL=(ALL) NOPASSWD: /bin/systemctl status micro@*
```
## Next Steps
- [micro run](./micro-run.md) - Local development
- [micro.mu configuration](./config.md) - Configuration file format
- [Health checks](./health.md) - Service health endpoints
+6 -2
View File
@@ -110,9 +110,11 @@ If you want to define services with protobuf you can use protoc-gen-micro (go-mi
Install the generator:
```bash
go install go-micro.dev/v5/cmd/protoc-gen-micro@latest
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
```bash
cd helloworld
mkdir proto
@@ -222,9 +224,11 @@ func main() {
Install the Micro CLI:
```
go install go-micro.dev/v5/cmd/micro@latest
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
Call a running service via RPC:
```
@@ -88,7 +88,7 @@ message Response {
```bash
# Install protoc-gen-micro
go install go-micro.dev/v5/cmd/protoc-gen-micro@latest
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.13.0
# Generate Go code
protoc --proto_path=. \
@@ -97,6 +97,8 @@ protoc --proto_path=. \
proto/helloworld.proto
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
### Server Implementation
```go
@@ -90,7 +90,7 @@ Update your proto generation:
```bash
# Install protoc-gen-micro
go install go-micro.dev/v5/cmd/protoc-gen-micro@latest
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.13.0
# Generate both gRPC and Go Micro code
protoc --proto_path=. \
@@ -100,6 +100,8 @@ protoc --proto_path=. \
proto/hello.proto
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
This generates:
- `hello.pb.go` - Protocol Buffers types
- `hello_grpc.pb.go` - gRPC client/server (keep for compatibility)
+3 -1
View File
@@ -11,9 +11,11 @@ The Micro server is an optional API and dashboard that provides a fixed entrypoi
Install the CLI which includes the server command:
```bash
go install go-micro.dev/v5/cmd/micro@latest
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
## Run
Start the server:
+1 -1
View File
@@ -251,9 +251,9 @@
<div class="links">
<a href="docs/" class="primary">Documentation</a>
<a href="blog/" class="secondary">Blog</a>
<a href="https://github.com/micro/go-micro" class="secondary">GitHub</a>
<a href="https://pkg.go.dev/go-micro.dev/v5" class="secondary">Reference</a>
<a href="badge.html" class="secondary">Get Badge</a>
</div>
<div class="showcase">
+75
View File
@@ -0,0 +1,75 @@
#!/bin/bash
# Install script for micro CLI
# Usage: curl -fsSL https://go-micro.dev/install.sh | sh
set -e
VERSION="${MICRO_VERSION:-latest}"
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
# Normalize architecture
case $ARCH in
x86_64|amd64) ARCH="amd64" ;;
aarch64|arm64) ARCH="arm64" ;;
armv7l) ARCH="arm" ;;
*) echo "Unsupported architecture: $ARCH"; exit 1 ;;
esac
# Normalize OS
case $OS in
darwin) OS="darwin" ;;
linux) OS="linux" ;;
*) echo "Unsupported OS: $OS"; exit 1 ;;
esac
# Determine install directory
if [ "$EUID" -eq 0 ] || [ "$(id -u)" -eq 0 ]; then
INSTALL_DIR="/usr/local/bin"
else
INSTALL_DIR="$HOME/.local/bin"
mkdir -p "$INSTALL_DIR"
fi
echo "Installing micro ${VERSION} for ${OS}/${ARCH}..."
# Download URL
if [ "$VERSION" = "latest" ]; then
URL="https://github.com/micro/go-micro/releases/latest/download/micro-${OS}-${ARCH}"
else
URL="https://github.com/micro/go-micro/releases/download/${VERSION}/micro-${OS}-${ARCH}"
fi
# Download
TMP_FILE=$(mktemp)
if command -v curl &> /dev/null; then
curl -fsSL "$URL" -o "$TMP_FILE"
elif command -v wget &> /dev/null; then
wget -q "$URL" -O "$TMP_FILE"
else
echo "Error: curl or wget required"
exit 1
fi
# Install
chmod +x "$TMP_FILE"
mv "$TMP_FILE" "$INSTALL_DIR/micro"
echo ""
echo "✓ Installed micro to $INSTALL_DIR/micro"
echo ""
# Verify
if command -v micro &> /dev/null; then
micro --version
else
echo "Note: Add $INSTALL_DIR to your PATH:"
echo " export PATH=\"\$PATH:$INSTALL_DIR\""
fi
echo ""
echo "Get started:"
echo " micro new myservice # Create a new service"
echo " micro run # Run locally"
echo " micro deploy # Deploy to server"
echo ""
+114 -33
View File
@@ -16,19 +16,25 @@ import (
)
type ntport struct {
addrs []string
opts transport.Options
nopts nats.Options
addrs []string
opts transport.Options
nopts nats.Options
pool *connectionPool // connection pool for clients
poolSize int
poolIdleTimeout time.Duration
mu sync.RWMutex
}
type ntportClient struct {
conn *nats.Conn
addr string
id string
local string
remote string
sub *nats.Subscription
opts transport.Options
conn *nats.Conn
pooledConn *pooledConnection // reference to pooled connection if using pool
pool *connectionPool // reference to pool to return connection on close
addr string
id string
local string
remote string
sub *nats.Subscription
opts transport.Options
}
type ntportSocket struct {
@@ -67,8 +73,20 @@ func configure(n *ntport, opts ...transport.Option) {
}
natsOptions := nats.GetDefaultOptions()
if n, ok := n.opts.Context.Value(optionsKey{}).(nats.Options); ok {
natsOptions = n
if no, ok := n.opts.Context.Value(optionsKey{}).(nats.Options); ok {
natsOptions = no
}
// Set pool size (default is 1 - no pooling)
n.poolSize = 1
if poolSize, ok := n.opts.Context.Value(poolSizeKey{}).(int); ok && poolSize > 0 {
n.poolSize = poolSize
}
// Set pool idle timeout (default is 5 minutes)
n.poolIdleTimeout = 5 * time.Minute
if idleTimeout, ok := n.opts.Context.Value(poolIdleTimeoutKey{}).(time.Duration); ok {
n.poolIdleTimeout = idleTimeout
}
// transport.Options have higher priority than nats.Options
@@ -91,6 +109,31 @@ func configure(n *ntport, opts ...transport.Option) {
n.opts.Addrs = setAddrs(n.opts.Addrs)
n.nopts = natsOptions
n.addrs = n.opts.Addrs
// Initialize connection pool if size > 1
if n.poolSize > 1 && n.pool == nil {
factory := func() (*nats.Conn, error) {
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
return opts.Connect()
}
pool, err := newConnectionPool(n.poolSize, factory)
if err == nil {
if n.poolIdleTimeout > 0 {
pool.idleTimeout = n.poolIdleTimeout
}
n.pool = pool
}
}
}
func setAddrs(addrs []string) []string {
@@ -166,6 +209,13 @@ func (n *ntportClient) Recv(m *transport.Message) error {
func (n *ntportClient) Close() error {
n.sub.Unsubscribe()
// If using a pooled connection, return it to the pool
if n.pool != nil && n.pooledConn != nil {
return n.pool.Put(n.pooledConn)
}
// Otherwise, close the connection directly
n.conn.Close()
return nil
}
@@ -344,37 +394,68 @@ func (n *ntport) Dial(addr string, dialOpts ...transport.DialOption) (transport.
o(&dopts)
}
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
opts.Timeout = dopts.Timeout
var c *nats.Conn
var pooledConn *pooledConnection
var err error
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
// Use connection pool if available
n.mu.RLock()
hasPool := n.pool != nil
n.mu.RUnlock()
c, err := opts.Connect()
if err != nil {
return nil, err
if hasPool {
pooledConn, err = n.pool.Get()
if err != nil {
return nil, err
}
c = pooledConn.Conn()
if c == nil {
n.pool.Put(pooledConn)
return nil, errors.New("invalid connection from pool")
}
} else {
// Create a new connection (original behavior)
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
opts.Timeout = dopts.Timeout
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
c, err = opts.Connect()
if err != nil {
return nil, err
}
}
id := nats.NewInbox()
sub, err := c.SubscribeSync(id)
if err != nil {
if pooledConn != nil {
n.pool.Put(pooledConn)
} else {
c.Close()
}
return nil, err
}
return &ntportClient{
conn: c,
addr: addr,
id: id,
sub: sub,
opts: n.opts,
local: id,
remote: addr,
}, nil
client := &ntportClient{
conn: c,
pooledConn: pooledConn,
pool: n.pool,
addr: addr,
id: id,
sub: sub,
opts: n.opts,
local: id,
remote: addr,
}
return client, nil
}
func (n *ntport) Listen(addr string, listenOpts ...transport.ListenOption) (transport.Listener, error) {
+27
View File
@@ -2,12 +2,15 @@ package nats
import (
"context"
"time"
"github.com/nats-io/nats.go"
"go-micro.dev/v5/transport"
)
type optionsKey struct{}
type poolSizeKey struct{}
type poolIdleTimeoutKey struct{}
// Options allow to inject a nats.Options struct for configuring
// the nats connection.
@@ -19,3 +22,27 @@ func Options(nopts nats.Options) transport.Option {
o.Context = context.WithValue(o.Context, optionsKey{}, nopts)
}
}
// PoolSize sets the size of the connection pool.
// If set to a value > 1, the transport will use a connection pool.
// Default is 1 (no pooling).
func PoolSize(size int) transport.Option {
return func(o *transport.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, poolSizeKey{}, size)
}
}
// PoolIdleTimeout sets the timeout for idle connections in the pool.
// Connections idle for longer than this duration will be closed.
// Default is 5 minutes. Set to 0 to disable idle timeout.
func PoolIdleTimeout(timeout time.Duration) transport.Option {
return func(o *transport.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, poolIdleTimeoutKey{}, timeout)
}
}
+188
View File
@@ -0,0 +1,188 @@
package nats
import (
"errors"
"sync"
"time"
natsp "github.com/nats-io/nats.go"
)
var (
// ErrPoolExhausted is returned when no connections are available in the pool
ErrPoolExhausted = errors.New("connection pool exhausted")
// ErrPoolClosed is returned when trying to use a closed pool
ErrPoolClosed = errors.New("connection pool is closed")
)
// connectionPool manages a pool of NATS connections
type connectionPool struct {
mu sync.RWMutex
connections chan *pooledConnection
factory func() (*natsp.Conn, error)
size int
idleTimeout time.Duration
closed bool
}
// pooledConnection wraps a NATS connection with metadata
type pooledConnection struct {
conn *natsp.Conn
createdAt time.Time
lastUsed time.Time
mu sync.Mutex
}
// newConnectionPool creates a new connection pool
func newConnectionPool(size int, factory func() (*natsp.Conn, error)) (*connectionPool, error) {
if size <= 0 {
size = 1
}
pool := &connectionPool{
connections: make(chan *pooledConnection, size),
factory: factory,
size: size,
idleTimeout: 5 * time.Minute,
closed: false,
}
return pool, nil
}
// Get retrieves a connection from the pool or creates a new one
func (p *connectionPool) Get() (*pooledConnection, error) {
p.mu.RLock()
if p.closed {
p.mu.RUnlock()
return nil, ErrPoolClosed
}
p.mu.RUnlock()
// Try to get an existing connection from the pool
select {
case conn := <-p.connections:
// Check if connection is still valid and not idle for too long
if conn.isValid() && !conn.isExpired(p.idleTimeout) {
conn.updateLastUsed()
return conn, nil
}
// Connection is invalid or expired, close it and create a new one
conn.close()
return p.createConnection()
default:
// No connection available, create a new one
return p.createConnection()
}
}
// Put returns a connection to the pool
func (p *connectionPool) Put(conn *pooledConnection) error {
p.mu.RLock()
defer p.mu.RUnlock()
if p.closed {
return conn.close()
}
// Check if connection is still valid
if !conn.isValid() {
return conn.close()
}
conn.updateLastUsed()
// Try to return connection to pool
select {
case p.connections <- conn:
return nil
default:
// Pool is full, close the connection
return conn.close()
}
}
// Close closes all connections in the pool
func (p *connectionPool) Close() error {
p.mu.Lock()
defer p.mu.Unlock()
if p.closed {
return nil
}
p.closed = true
close(p.connections)
// Close all connections in the pool
for conn := range p.connections {
conn.close()
}
return nil
}
// createConnection creates a new pooled connection
func (p *connectionPool) createConnection() (*pooledConnection, error) {
conn, err := p.factory()
if err != nil {
return nil, err
}
return &pooledConnection{
conn: conn,
createdAt: time.Now(),
lastUsed: time.Now(),
}, nil
}
// isValid checks if the underlying NATS connection is valid
func (pc *pooledConnection) isValid() bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn == nil {
return false
}
status := pc.conn.Status()
return status == natsp.CONNECTED || status == natsp.RECONNECTING
}
// isExpired checks if the connection has been idle for too long
func (pc *pooledConnection) isExpired(timeout time.Duration) bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if timeout <= 0 {
return false
}
return time.Since(pc.lastUsed) > timeout
}
// close closes the underlying NATS connection
func (pc *pooledConnection) close() error {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn != nil {
pc.conn.Close()
pc.conn = nil
}
return nil
}
// Conn returns the underlying NATS connection
func (pc *pooledConnection) Conn() *natsp.Conn {
pc.mu.Lock()
defer pc.mu.Unlock()
return pc.conn
}
// updateLastUsed updates the last used timestamp in a thread-safe manner
func (pc *pooledConnection) updateLastUsed() {
pc.mu.Lock()
defer pc.mu.Unlock()
pc.lastUsed = time.Now()
}
+170
View File
@@ -0,0 +1,170 @@
package nats
import (
"sync"
"testing"
"time"
natsp "github.com/nats-io/nats.go"
)
func TestTransportConnectionPool_GetPut(t *testing.T) {
// Mock factory that creates connections
connCount := 0
factory := func() (*natsp.Conn, error) {
connCount++
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Get a connection (should create one)
conn1, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
if conn1 == nil {
t.Fatal("Expected connection, got nil")
}
// Put it back
if err := pool.Put(conn1); err != nil {
t.Fatalf("Failed to put connection: %v", err)
}
// Get it again (should reuse the same one)
conn2, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
if conn2 == nil {
t.Fatal("Expected connection, got nil")
}
}
func TestTransportConnectionPool_Concurrent(t *testing.T) {
connCount := 0
mu := sync.Mutex{}
factory := func() (*natsp.Conn, error) {
mu.Lock()
connCount++
mu.Unlock()
return nil, nil
}
pool, err := newConnectionPool(5, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Simulate concurrent access
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
conn, err := pool.Get()
if err != nil {
t.Errorf("Failed to get connection: %v", err)
return
}
// Simulate some work
time.Sleep(10 * time.Millisecond)
if err := pool.Put(conn); err != nil {
t.Errorf("Failed to put connection: %v", err)
}
}()
}
wg.Wait()
// We should have created some connections
mu.Lock()
if connCount == 0 {
t.Error("Expected at least one connection to be created")
}
mu.Unlock()
}
func TestTransportConnectionPool_Close(t *testing.T) {
factory := func() (*natsp.Conn, error) {
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
// Get a connection
conn, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
// Close the pool
if err := pool.Close(); err != nil {
t.Fatalf("Failed to close pool: %v", err)
}
// Put connection back to closed pool should not panic
_ = pool.Put(conn)
// Try to get from closed pool
_, err = pool.Get()
if err != ErrPoolClosed {
t.Errorf("Expected ErrPoolClosed, got: %v", err)
}
}
func TestTransportPoolConfiguration(t *testing.T) {
// Test with pool size 5
tr := NewTransport(PoolSize(5))
nt, ok := tr.(*ntport)
if !ok {
t.Fatal("Expected transport to be of type *ntport")
}
if nt.poolSize != 5 {
t.Errorf("Expected pool size 5, got %d", nt.poolSize)
}
// Test with custom idle timeout
tr2 := NewTransport(PoolSize(3), PoolIdleTimeout(10*time.Minute))
nt2, ok := tr2.(*ntport)
if !ok {
t.Fatal("Expected transport to be of type *ntport")
}
if nt2.poolSize != 3 {
t.Errorf("Expected pool size 3, got %d", nt2.poolSize)
}
if nt2.poolIdleTimeout != 10*time.Minute {
t.Errorf("Expected idle timeout 10m, got %v", nt2.poolIdleTimeout)
}
}
func TestTransportDefaultSingleConnection(t *testing.T) {
// Test that default behavior is single connection (pool size 1)
tr := NewTransport()
nt, ok := tr.(*ntport)
if !ok {
t.Fatal("Expected transport to be of type *ntport")
}
if nt.poolSize != 1 {
t.Errorf("Expected default pool size 1, got %d", nt.poolSize)
}
// With size 1, pool should not be created
if nt.pool != nil {
t.Error("Expected no pool with size 1")
}
}