Compare commits

..

150 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 0889405d84 Add evaluation summary for reflection removal analysis
Run Tests / Unit Tests (push) Waiting to run
Run Tests / Etcd Integration Tests (push) Waiting to run
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 15:31:53 +00:00
copilot-swe-agent[bot] edd160d112 Fix performance numbers for consistency
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 15:30:30 +00:00
copilot-swe-agent[bot] 09a833b317 Add comprehensive analysis documents on reflection usage
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 15:29:13 +00:00
copilot-swe-agent[bot] 206eb51961 Initial plan 2026-02-03 15:23:49 +00:00
Copilot 75e32f4d87 Implement NATS connection pooling and fix connection leaks (#2840)
* Initial plan

* Implement NATS connection pool with configuration options

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Implement connection pool for transport/nats

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix connection leaks in events/natsjs and config/source/nats

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix race condition in connection pool lastUsed field access

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Remove unused maxIdle field from connection pools

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 11:43:45 +00:00
Copilot adc90b4d2d Apply rate limiting before singleflight to prevent goroutine blocking on etcd timeout (#2841)
* Initial plan

* Apply rate limiting before singleflight to prevent blocking

- Check rate limiting BEFORE entering singleflight
- If rate-limited AND stale cache exists, return stale cache immediately
- This prevents all goroutines from blocking when etcd is down/slow
- Maintains stampede prevention via singleflight for non-rate-limited requests
- All existing tests pass

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix variable shadowing in rate limiting check

- Rename shadowed variables to currentLastRefresh and currentMinimumRetryInterval
- Improves code clarity and prevents potential bugs
- All tests still pass

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-02-03 11:35:34 +00:00
Copilot 87cf988e03 Fix go install @latest failures by documenting specific version (#2839)
* Initial plan

* Update documentation to use @v5.13.0 instead of @latest for go install commands

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add explanatory notes about version pinning in documentation

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add consistent explanatory notes across all documentation files

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-29 10:19:55 +00:00
Asim Aslam 109ff2169a Add blog link to homepage (#2837)
Link to /blog/ from the main navigation, replacing the badge link.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 14:08:32 +00:00
Shelley 82b7fdbec4 Add install.sh to website for curl install
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 14:02:38 +00:00
Shelley fea8c911be Fix blog post markdown rendering
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:54:31 +00:00
Shelley cf9629c41f Add blog section with first post: Introducing micro deploy
- /blog/ - Blog index page
- /blog/1 - First post announcing micro deploy
- /docs/deployment.md - Deployment guide in website docs
- Updated navigation to include Blog link
- New blog layout template

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:52:09 +00:00
Asim Aslam a5bef7af29 Add systemd-based deployment support (#2836)
* Add systemd-based deployment support

- micro init --server: Initialize server to receive deployments
  - Creates /opt/micro/{bin,data,config} directories
  - Generates systemd template unit (micro@.service)
  - Creates 'micro' system user

- micro deploy: Deploy services via SSH + systemd
  - Builds linux/amd64 binaries automatically
  - Copies via rsync/scp to server
  - Manages services via systemctl
  - Helpful error messages for common issues

- micro status --remote: Check remote service status
- micro logs --remote: Stream remote logs via journalctl
- micro stop --remote: Stop services on remote server

- Config: Added 'deploy' blocks to micro.mu for named targets

The deployment model:
- systemd is the process supervisor (battle-tested)
- SSH is the transport (standard, secure)
- No custom daemons or platforms needed

Co-authored-by: Shelley <shelley@exe.dev>

* Add deployment documentation

- docs/deployment.md: Comprehensive guide for server deployment
- README.md: Updated deployment section with full workflow

Co-authored-by: Shelley <shelley@exe.dev>

* Add deployment section to CLI documentation

Co-authored-by: Shelley <shelley@exe.dev>

* Fix systemd template escaping and rsync permission warnings

- Fix %i escaping in systemd template (was being interpreted by fmt.Sprintf)
- Handle rsync exit code 23/24 gracefully (metadata permission warnings)
- Add --omit-dir-times to rsync to avoid directory timestamp errors

Co-authored-by: Shelley <shelley@exe.dev>

* Add install script for micro CLI

Co-authored-by: Shelley <shelley@exe.dev>

* Fix non-constant format string in deploy error

Co-authored-by: Shelley <shelley@exe.dev>

---------

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:27:59 +00:00
Asim Aslam 239dbfc27e fix: make build/deploy Go-native, Docker optional (#2835)
micro build:
  - Default: builds Go binaries to ./bin/
  - Cross-compile with --os and --arch
  - Docker is optional via --docker flag

micro deploy:
  - Requires --ssh user@host
  - Copies pre-built binaries (if ./bin/ exists)
  - Or syncs source and builds on remote
  - No Docker dependency

Go binaries are self-contained. No runtime needed.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:47:07 +00:00
Asim Aslam de2b3031f3 feat: add micro build and micro deploy commands (#2834)
micro build:
  - Generates Dockerfiles for services (if not present)
  - Builds container images for all services in micro.mu
  - Supports --tag, --registry, --push flags
  - --compose flag generates docker-compose.yml

micro deploy:
  - Default: deploys with docker-compose
  - --ssh user@host: deploys via SSH (rsync + build on remote)
  - --build: rebuild images before deploying

Complete workflow:
  micro run          # Develop locally
  micro build        # Build images
  micro deploy       # Deploy

Or for simple SSH deploys:
  micro deploy --ssh user@host

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:40:04 +00:00
Asim Aslam 40c3ec0e32 feat: add testing package for in-process service testing (#2833)
Provides a test harness for running micro services in isolation:

  h := testing.NewHarness(t)
  defer h.Stop()

  h.Name("users").Register(new(UsersHandler))
  h.Start()

  var rsp Response
  h.Call("UsersHandler.Create", &req, &rsp)

Features:
- Isolated registry, transport, and broker per harness
- Simple API: Name(), Register(), Start(), Stop()
- Call helpers: Call(), CallContext()
- Assertions: AssertServiceRunning(), AssertCallSucceeds(), AssertCallFails()
- Access to underlying Client(), Server(), Registry()

Note: Due to go-micro's global defaults, each harness tests one service.
For multi-service testing, use integration tests or mocks.

Also fixes README.md example showing conflicting port 8080.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:35:05 +00:00
Asim Aslam 139e70e880 feat(run): integrate HTTP gateway with micro run (#2832)
micro run now starts an HTTP gateway alongside your services:

  - Web dashboard at http://localhost:8080
  - API proxy at /api/{service}/{method}
  - Health checks at /health
  - Service listing at /services

The experience is now:
  $ micro new helloworld
  $ cd helloworld
  $ micro run

  Open http://localhost:8080 to see and call your services.

New flags:
  --address :3000    # Custom gateway port
  --no-gateway       # Disable gateway (services only)

Updated documentation to make this the central experience.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:22:45 +00:00
Asim Aslam 94e83e57f8 feat: add health check package for K8s-style probes (#2831)
Adds a new health package providing:
- /health endpoint for overall health status
- /health/live for Kubernetes liveness probes
- /health/ready for Kubernetes readiness probes
- Built-in checks: PingCheck, TCPCheck, HTTPCheck, DNSCheck
- Critical vs non-critical checks
- Concurrent check execution
- Configurable timeouts
- Service info metadata

Integrates with micro run's health check waiting when services
specify a port in their micro.mu configuration.

Example usage:

    health.Register("database", health.PingCheck(db.Ping))
    health.Register("redis", health.TCPCheck("localhost:6379", time.Second))
    health.RegisterHandlers(mux)

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:10:38 +00:00
Asim Aslam 39484560ea docs: add micro run documentation with hot reload and config file guide (#2830)
- Update main README with micro run quick start
- Expand cmd/micro/README.md with configuration options
- Add detailed guide at internal/website/docs/guides/micro-run.md

Documents:
- Hot reload with file watching
- micro.mu DSL configuration
- micro.json alternative
- Dependency ordering
- Environment management
- Graceful shutdown

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:04:15 +00:00
Asim Aslam 7690c41d5f feat(run): add hot reload, config files, and dependency ordering (#2829)
- Add micro.mu DSL and micro.json config file support
- Implement hot reload with file watching (--no-watch to disable)
- Start services in dependency order (topological sort)
- Environment management (--env flag, MICRO_ENV var)
- Health check waiting before starting dependents
- Graceful shutdown in reverse dependency order

Config file example (micro.mu):

    service users
        path ./users
        port 8081

    service posts
        path ./posts
        port 8082
        depends users

    env development
        STORE_ADDRESS file://./data

Closes #2828

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:01:47 +00:00
Asim Aslam cae6fbbe76 Framework hardening: security, reliability, and developer experience improvements (#2826)
* fix: remove deprecated rand.Seed calls

Go 1.20+ automatically seeds the global random number generator.
These calls are no-ops and generate warnings with newer Go versions.

Removed from:
- selector/strategy.go
- registry/cache/cache.go
- broker/memory.go
- broker/http.go
- cmd/cmd.go
- transport/memory.go

Co-authored-by: Shelley <shelley@exe.dev>

* fix: handle previously ignored errors

- MySQL store: properly handle prepared statement errors in initDB()
- Consul registry: handle client creation errors in Client() method

These silent failures could cause hard-to-debug issues in production.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(genai): improve provider interface with context and streaming

Breaking changes:
- Generate() and Stream() now require context.Context as first parameter
- Stream.Close() added for proper resource cleanup

Improvements:
- Proper context support for cancellation and timeouts
- Real SSE streaming for OpenAI and Gemini text generation
- Better error handling with wrapped errors and API error responses
- Thread-safe provider registry with sync.RWMutex
- New options: WithMaxTokens, WithTemperature, WithTimeout
- Stream has proper Close() method for cleanup
- Results can include Error field for per-chunk errors

Provider updates:
- OpenAI: true streaming with SSE parsing, proper HTTP client with timeout
- Gemini: true streaming with streamGenerateContent endpoint
- Default model updated to gpt-4o-mini (OpenAI) and gemini-2.0-flash (Gemini)

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): make TLS secure by default, configurable via environment

BREAKING: TLS now verifies certificates by default. Set MICRO_TLS_INSECURE=true
to restore previous behavior (NOT recommended for production).

Changes:
- Add util/tls.Config(), SecureConfig(), InsecureConfig(), ConfigFromEnv() helpers
- Update all components to use ConfigFromEnv() instead of hardcoded InsecureSkipVerify
- Set MinVersion to TLS 1.2 for all TLS configs

Affected components:
- broker/http
- broker/rabbitmq
- registry/etcd
- registry/consul
- transport/grpc

This improves security posture while allowing opt-out for development environments.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): add TLS helpers with opt-in secure mode

NOT a breaking change - keeps InsecureSkipVerify=true as default for
local development compatibility.

New util/tls helpers:
- Config() - returns config based on MICRO_TLS_SECURE env var
- SecureConfig() - certificate verification enabled
- InsecureConfig() - certificate verification disabled (dev only)

For production security, use one of:
- Set MICRO_TLS_SECURE=true with proper CA-signed certs
- Use a service mesh (Istio, Linkerd) for automatic mTLS
- Configure TLSConfig directly with your certificates

Also: Changed CLI alias from 'g' to 'gen' for clarity
- micro generate handler -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

* refactor(cli): rename generate directory to gen for consistency

Directory name now matches the command alias:
  cmd/micro/cli/gen/ -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

---------

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 10:39:25 +00:00
Asim Aslam a32cdbc117 feat(web): add SSE broadcaster for real-time event streaming (#2825)
This adds Server-Sent Events (SSE) support to the web package, enabling
real-time updates in web applications built with go-micro.

Features:
- SSEBroadcaster manages connected clients and broadcasts events
- Integrates with events.Stream for subscribing to topics
- Supports JSON events and raw HTML (for htmx/datastar)
- Keep-alive mechanism for long-lived connections
- Thread-safe client management

Usage:
  broadcaster := web.NewSSEBroadcaster(
    web.WithStream(stream),
    web.WithTopics("posts", "comments"),
  )
  broadcaster.Start()
  router.GET("/events", broadcaster.Handler())

This enables real-time features requested in #2788, including:
- Live updates without page refresh
- htmx/datastar integration
- Foundation for MCP (Model Context Protocol) support

Closes #2788

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 06:29:34 +00:00
Copilot baeb282cf1 Enhance cache rate limiting to prevent registry overload during rolling deployments (#2824)
* Initial plan

* Enhance cache rate limiting to protect against rolling deployment scenarios

Per @asim's feedback, rate limiting now applies to ALL cache refresh attempts
(not just error cases) to prevent registry overload during rolling deployments.

Changes:
- Rate limit ALL refresh attempts using lastRefreshAttempt tracking
- Always return stale cache if available (even if expired) when rate limiting
- Only return ErrNotFound when no cache exists during rate limit period
- Rate limiting happens inside singleflight to avoid race conditions
- Update test to reflect new behavior (no retry when stale cache + rate limit)
- Enhanced documentation with rolling deployment scenario examples

This addresses the scenario where all upstream services expire their cache
simultaneously during a downstream rolling deployment, which would previously
cause a stampede to the registry under high QPS.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Remove unused failedAttempts and consecutiveFailures fields

Per @asim's feedback, these fields are no longer needed since the new
rate limiting strategy uses lastRefreshAttempt to track ALL refresh
attempts, not just failed ones.

Removed:
- failedAttempts map[string]time.Time
- consecutiveFailures int
- All code that sets these fields

The rate limiting logic now only uses lastRefreshAttempt, making these
fields redundant. All tests continue to pass.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-14 10:57:14 +00:00
Copilot 2869cc16d1 Add adaptive throttling to registry cache to prevent penetration when no stale cache exists (#2823)
* Initial plan

* Add adaptive throttling to prevent cache penetration without stale cache

Implement rate limiting mechanism to protect etcd when:
- Cache is empty (no stale data available)
- Registry is failing/overloaded
- Multiple sequential requests arrive

Changes:
- Add MinimumRetryInterval option (default 5s)
- Track last failed attempt time per service
- Throttle requests within retry interval when no cache exists
- Clear throttling on successful lookup
- Add comprehensive tests validating throttling behavior

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix WaitGroup race condition in events stream tests

The WaitGroup.Add() calls were happening after goroutines started,
causing a race where Done() could be called before Add(), resulting
in "negative WaitGroup counter" panics.

Fixes:
- Move wg.Add(1) before goroutine in TestConsumeTopic
- Move wg.Add(2) before first goroutine in TestConsumeGroup

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Update registry cache documentation with adaptive throttling feature

Document the new adaptive throttling feature that prevents cache
penetration when registry is failing and no stale cache exists.

Includes:
- Feature overview
- Usage examples with configuration options
- Explanation of throttling behavior
- Example scenario demonstrating protection

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-09 12:27:55 +00:00
Copilot dbb66ec938 Fix etcd auth overload and cache stampede with KeepAlive channels (#2822)
* Initial plan

* Replace KeepAliveOnce with KeepAlive to reduce etcd auth requests

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add tests to verify cache penetration protection via singleflight

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add etcd integration tests to CI workflow

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix race conditions and improve code quality based on review feedback

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add workflow permissions to fix security scan finding

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add comprehensive documentation for performance improvements

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix memory store limit/offset bug causing events test failure

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix memory store to filter before limiting in prefix/suffix reads

The previous fix had a logic error where limit/offset were applied before
prefix/suffix filtering. This could cause incorrect results when the first
N items in the unfiltered list don't match the search criteria.

Now filters first to get all matching keys, then applies limit/offset to
the filtered results. This ensures ReadLimit(1) always returns 1 matching
record if available, regardless of map iteration order.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-09 11:12:06 +00:00
Copilot 80345fe63d docs: clarify gRPC server option ordering and service name usage (#2820)
* Initial plan

* docs: clarify gRPC server option ordering and service name usage

- Fix all examples to show Server option before Name option
- Add note explaining why option ordering matters
- Add new section on "Option Ordering Issue" in Common Errors
- Add new section on "Service Name vs Package Name" in Common Errors
- Update transport.md to show proper ordering with comment

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* docs: refine comments based on code review feedback

- Clarify that Server ordering before Name is mandatory
- Remove confusing comment about Client ordering being for consistency

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2025-11-25 09:26:59 +00:00
Copilot 00a119496e Add documentation for native gRPC compatibility vs transport (#2819)
* Initial plan

* Add documentation for native gRPC compatibility with grpc client/server packages

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix go_package path in proto example to use relative path

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2025-11-25 07:28:48 +00:00
Asim Aslam 3e25be984d x 2025-11-13 20:48:01 +00:00
Asim Aslam 4e8934c230 Get Badge 2025-11-13 20:46:20 +00:00
Asim Aslam a368be82cd add showcase 2025-11-13 20:42:45 +00:00
Asim Aslam 43386d4ec1 x 2025-11-13 20:37:54 +00:00
Asim Aslam a84f92c907 mobile hamburger menu 2025-11-13 20:34:09 +00:00
Asim Aslam 751de9e7c0 fix mobile rendering for website 2025-11-13 20:31:29 +00:00
Asim Aslam be1cf9c9d8 x 2025-11-13 20:06:28 +00:00
Asim Aslam d7699845bb Add docs link 2025-11-13 20:01:32 +00:00
Asim Aslam 520a0fa140 Update features 2025-11-13 19:59:28 +00:00
Asim Aslam 8a1591af0c x 2025-11-13 19:58:38 +00:00
Asim Aslam 1f1d9875a2 update landing 2025-11-13 19:56:01 +00:00
Asim Aslam cd4c881db5 x 2025-11-13 19:52:06 +00:00
Asim Aslam de03bbbf81 x 2025-11-13 19:49:34 +00:00
Asim Aslam ee4f656fa5 fix typo in search documentation 2025-11-13 19:26:19 +00:00
Asim Aslam 2cce3e5e1a fix links 2025-11-13 19:21:27 +00:00
Asim Aslam 0526a42efa changes to theme 2025-11-13 19:14:01 +00:00
Asim Aslam dc1e1eb45c fix config file 2025-11-13 19:04:30 +00:00
Asim Aslam 06b31f545a update the docs to easily navigate 2025-11-13 18:59:34 +00:00
Asim Aslam 8cda829320 major docs overhaul 2025-11-13 18:34:40 +00:00
Asim Aslam e755e4a823 updates for syntax highlighting 2025-11-13 18:17:45 +00:00
Asim Aslam eef06fcf01 server docs 2025-11-13 18:12:15 +00:00
Asim Aslam 9e36df224b new copilot generated documentation 2025-11-13 18:11:29 +00:00
Asim Aslam 2da1cc0edd Delete .github/PULL_REQUEST_TEMPLATE.md 2025-11-13 17:19:17 +00:00
Asim Aslam 0ef0143537 Delete .github/ISSUE_TEMPLATE/question.md 2025-11-13 17:19:04 +00:00
Asim Aslam 30dc01523a Delete .github/ISSUE_TEMPLATE/feature-request---enhancement.md 2025-11-13 17:18:54 +00:00
Asim Aslam 4a0648536f Delete .github/ISSUE_TEMPLATE/bug_report.md 2025-11-13 17:18:43 +00:00
nswdewy 01ed999b2a Change template paths from 'html' to 'web' (#2816) 2025-11-06 10:17:30 +00:00
Asim Aslam acfb4e2639 Update README.md 2025-10-29 10:46:41 +00:00
Asim Aslam cd735aba56 Update sponsorship link in README.md 2025-10-29 10:46:14 +00:00
Asim Aslam 8c995ea8ac Update sponsorship link in README 2025-10-29 10:45:28 +00:00
Asim Aslam e9ce3487b6 Update README.md 2025-10-29 10:44:16 +00:00
Asim Aslam 79f5a7a7f6 Fix README formatting and sponsor button placement 2025-10-29 10:43:48 +00:00
Asim Aslam 48c310f70b Update README.md 2025-10-29 10:43:32 +00:00
Asim Aslam 5ffb711dfe Add funding configuration for GitHub 2025-10-29 10:42:01 +00:00
Asim Aslam de5057e35d Update README to remove install script section
Removed mention of install script from README.
2025-10-22 21:19:31 +01:00
Asim Aslam 91d91eef1f Fix for https://github.com/micro/go-micro/issues/2687 2025-10-22 06:38:16 +00:00
asim 48479228b1 Fixes https://github.com/micro/go-micro/issues/2738 2025-10-22 07:18:37 +01:00
asim 5a07370970 Fix https://github.com/micro/go-micro/issues/2769 2025-10-22 07:10:41 +01:00
Asim Aslam db9b1e8d79 Update installation command to specific version 2025-10-20 11:41:26 +01:00
Asim Aslam 3e05108882 Update installation command to specific version 2025-10-20 11:41:04 +01:00
Asim Aslam 85cd80a467 Delete cmd/README.md 2025-10-14 19:34:27 +01:00
Eng Zer Jun bc40df0c9e Update github.com/imdario/mergo to dario.cat/mergo (#2810)
Mergo v1 is released with a new module path URL. No breaking changes,
only the module path update and bug fixes.

Reference: https://github.com/darccio/mergo/releases/tag/v1.0.0
Reference: https://github.com/darccio/mergo/releases/tag/v1.0.1
Reference: https://github.com/darccio/mergo/releases/tag/v1.0.2

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-10-14 11:18:28 +01:00
asim 66014eac68 . 2025-10-14 11:17:03 +01:00
asim 26b9d56165 . 2025-10-14 11:15:52 +01:00
asim 6574180b0d . 2025-10-14 11:14:36 +01:00
asim e6c9f4fa39 . 2025-10-14 11:13:53 +01:00
asim 1dde737b64 move micro cli and protoc-gen-micro to cmd/ 2025-10-14 11:13:35 +01:00
Asim Aslam 26adf49e55 Remove Command Line section from README
Removed command line section from README.
2025-10-14 10:33:20 +01:00
Eng Zer Jun 7a9c445321 Update github.com/streadway/amqp to github.com/rabbitmq/amqp091-go (#2811)
The `github.com/streadway/amqp` module is no longer actively maintained.
The new module is now maintained by the RabbitMQ core team under a
different package name.

Reference: https://github.com/rabbitmq/amqp091-go

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-10-13 21:34:07 +01:00
Asim Aslam 44cc31345d use buf pool 2025-10-07 16:38:09 +00:00
Asim Aslam aa2ed9aa91 Patch for potential deadlock 2025-10-07 16:20:45 +00:00
Asim Aslam 3d6b6521ab fix atomic int alignment bug 2025-10-07 15:57:43 +00:00
Kenley Wang 7b0fa8b6ae fix: new pool cannot customize closetimeout (#2808) 2025-10-02 14:28:06 +01:00
asim 059d317969 . 2025-09-29 10:28:59 +01:00
Asim Aslam e0af3b61a5 Change 'Toolkit' to 'Command Line' in README
Updated section headers in README for clarity.
2025-09-28 09:55:43 +01:00
Asim Aslam 5bbc24abe1 Clarify micro CLI usage and genai package details
Updated the README to clarify the usage of the micro CLI and the genai package.
2025-09-28 08:51:30 +01:00
Asim Aslam dfcaae7bf8 Update link text from 'Docs' to 'Go Doc' 2025-09-27 09:45:52 +01:00
Ak-Army 9070b3befd [feature] ability to use wildcard in topic name (factory-events.*.*) (#2804) 2025-09-09 12:51:39 +01:00
ExtinctPatronageDeodorize e5cd820c71 fix data race in memory.update (#2802)
Co-authored-by: ExtinctPatronageDeodorize <ExtinctPatronageDeodorize@users.noreply.github.com>
2025-08-29 14:14:11 +01:00
Tsln 479bd58c3f fix consul context key mismatch (#2801) 2025-08-29 07:39:20 +01:00
Byron Silvers 95540b7859 fix consul watcher address handling (#2800) 2025-08-19 09:27:34 +01:00
Asim Aslam be2559c555 Update index.html 2025-08-05 14:15:06 +01:00
Asim Aslam 6e53f541d3 Update README.md 2025-08-05 14:14:28 +01:00
Asim Aslam a6ede13f73 Update index.html 2025-08-04 11:56:59 +01:00
Asim Aslam 31d2d39f76 Update README.md 2025-07-24 21:24:04 +01:00
Roman Perekhod ad73add529 fix the concurrent map writes error (#2794) 2025-07-16 07:27:45 +01:00
asim f99a205b2b add string function 2025-07-15 20:43:58 +01:00
Asim Aslam 3cf5540b0f Update README.md 2025-07-08 19:24:19 +01:00
Asim Aslam 2a140858a8 Update README.md 2025-07-08 19:23:49 +01:00
Asim Aslam 2d19a304fc Update README.md 2025-07-08 19:23:15 +01:00
asim 7b48449a85 fix options 2025-07-07 10:23:23 +01:00
asim 680987aeeb . 2025-07-02 09:56:34 +01:00
Asim Aslam ee9f3afe37 GenAI interface (#2790)
* genai interface

* x

* x

* text to speech

* Re-add events package (#2761)

* Re-add events package

* run redis as a dep

* remove redis events

* fix: data race on event subscriber

* fix: data race in tests

* fix: store errors

* fix: lint issues

* feat: default stream

* Update file.go

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>

* .

* copilot couldn't make it compile so I did

* copilot couldn't make it compile so I did

* x

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>
2025-06-20 10:24:31 +01:00
Asim Aslam 7e1bba2baf Re-add events package (#2761)
* Re-add events package

* run redis as a dep

* remove redis events

* fix: data race on event subscriber

* fix: data race in tests

* fix: store errors

* fix: lint issues

* feat: default stream

* Update file.go

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>
2025-06-18 17:12:02 +01:00
blacksheepaul dd0944bf68 Update getting-started.md (#2787) 2025-06-12 09:56:34 +01:00
Ak-Army 88f38eaef6 Subscribe error handling (#2785)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [feature] always subscribes to all topics and if there is an error in one of them, the unsuccessful topics will be subscribed to again

---------

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
2025-06-04 08:55:17 +01:00
Asim Aslam 0e45edf439 Set service name as table name (#2780)
* add docs layout

* update all to use _layouts

* update the styling

* always use service name as table name
2025-05-21 22:43:29 +01:00
Asim Aslam 788dcd05b2 Docs (#2779)
* add docs layout

* update all to use _layouts

* update the styling
2025-05-21 13:48:03 +01:00
Asim Aslam 456cd7e092 Add links to readme contents 2025-05-21 12:06:25 +00:00
Asim Aslam 3fd52c66e7 Generated docs for the following files: 2025-05-21 12:03:24 +00:00
Asim Aslam 3c3ba55c45 Create README.md (#2778) 2025-05-21 11:22:11 +01:00
Asim Aslam 1a8074467a Update and rename docs.yml to website.yml (#2777) 2025-05-21 11:18:54 +01:00
Asim Aslam b9665e32c5 Move whole website (#2776)
* adding docs

* minor getting started typo

* move the whole website
2025-05-21 11:17:04 +01:00
Asim Aslam 11b7eb0727 Docs (#2775)
* minor getting started typo
2025-05-21 10:59:04 +01:00
Asim Aslam 5a409f2607 Create docs.yml 2025-05-21 10:49:59 +01:00
Asim Aslam 8e771c57e1 adding docs (#2774) 2025-05-21 10:47:47 +01:00
Brian Ketelsen ddc34801ee Plugins and profiles (#2764)
* feat: more plugins

* chore(ci): split out benchmarks

Attempt to resolve too many open files in ci

* chore(ci): split out benchmarks

* fix(ci): Attempt to resolve too many open files in ci

* fix: set DefaultX for cli flag and service option

* fix: restore http broker

* fix: default http broker

* feat: full nats profile

* chore: still ugly, not ready

* fix: better initialization for profiles

* fix(tests): comment out flaky listen tests

* fix: disable benchmarks on gha

* chore: cleanup, comments

* chore: add nats config source
2025-05-20 13:24:06 -04:00
Brian Ketelsen e12504ce3a feat: re-add profiles (#2772)
* feat: re-add profiles

* fix: make profile separate package

* fix: make profile separate package

* fix: profile flag

* fix: defaults
2025-05-19 13:59:28 -04:00
asim a03ab5f601 switch location of store dir to home dir 2025-05-19 09:28:13 +01:00
Asim Aslam 44ff301d2d syntactically easy way to register commands (#2770) 2025-05-18 09:45:49 +01:00
Asim Aslam 37bb1a8ab6 Revert "WIP: move file store (do not merge) (#2766)" (#2768)
This reverts commit 97275d3db9.
2025-05-16 19:06:45 +01:00
BombartSimon 1fe2638298 Implement MDNS Registry (#2767)
- Removed existing mDNS test file and replaced it with a new implementation.
- Added a new `mdns_registry.go` file that contains the MDNS registry logic.
- Updated the default registry to use the new MDNS registry.
- Refactored tests to accommodate the new MDNS registry implementation.
- Implemented service registration, deregistration, and service discovery using mDNS.
- Added encoding and decoding functions for mDNS TXT records.
- Implemented a watcher for monitoring service changes in the MDNS registry.
2025-05-16 19:03:36 +01:00
Asim Aslam 97275d3db9 WIP: move file store (do not merge) (#2766)
* move file store

* fix build
2025-05-16 15:08:13 +01:00
BombartSimon e29159e836 Rename 'Users' section to 'Adopters' in README (#2765) 2025-05-15 22:20:10 +01:00
Asim Aslam caba761c7b fix extraction field naming with standard Go types (#2763) 2025-05-15 18:59:10 +01:00
Brian Ketelsen cd2b40ca4a Add a selection of plugins to the core repo (#2755)
* WIP

* fix: default memory registry, add registrations for mdns, nats

* fix: same for broker

* fix: add more

* fix: http port

* rename redis

* chore: linting
2025-05-15 18:47:35 +01:00
Asim Aslam 7c04b7cfd6 Update README.md (#2762) 2025-05-15 16:13:27 +01:00
Asim Aslam b4a87d05f7 Update README.md (#2759) 2025-05-15 09:37:12 +01:00
asim 0e47cbf1a2 fix conflicting port range test 2025-05-14 22:11:23 +01:00
asim 12dfa797dc drop the separate go mods 2025-05-14 22:03:38 +01:00
Asim Aslam 230505bf5a Add grpc plugin (#2758)
* Add grpc plugin

* remove stale readmes
2025-05-14 21:04:42 +01:00
Asim Aslam 01b8394c81 Update README.md 2025-05-07 19:39:11 +01:00
Asim Aslam f9d08a14f3 Update README.md (#2756) 2025-05-07 19:35:20 +01:00
Asim Aslam a997f738dd Update README.md 2025-05-07 19:34:21 +01:00
Asim Aslam 46db7df218 Update README.md 2025-05-07 17:07:20 +01:00
Brian Ketelsen 17c04258a4 fix: go version breaking unit tests (#2754) 2025-05-05 21:12:34 -04:00
asim 8eb280126a add go mod 2025-05-05 22:02:32 +01:00
asim f51cd8d883 default to file store 2025-05-05 22:02:11 +01:00
asim ef4dc8b5b0 add json.NewValues to config 2025-05-05 14:39:54 +01:00
asim 23b14123ea errors for go 1.24wq 2025-05-04 21:48:02 +01:00
asim 2388f662cf break config.Get and return error with value 2025-05-04 21:31:17 +01:00
asim 60474ed38f add store encode/decode for record 2025-05-04 20:13:58 +01:00
asim 484eb3d15e public functions for store 2025-05-04 20:00:44 +01:00
asim c51095a074 store.NewRecord 2025-05-04 19:55:10 +01:00
Asim Aslam 03782bc9b3 move service to service dir to make life not suck (#2753) 2025-05-04 19:44:56 +01:00
Asim Aslam 1040b7c58e Update README.md 2025-05-01 10:27:59 +01:00
Asim Aslam bd0e9ca7cb Update README.md 2025-05-01 10:27:37 +01:00
Asim Aslam f5399d56c3 Update README.md 2025-05-01 10:25:44 +01:00
Asim Aslam 0f4c238804 protoc-gen-micro has moved to micro/micro/cmd/protoc-gen-micro (#2751) 2025-04-30 16:50:27 +01:00
Asim Aslam 66c169076c Update README.md 2025-04-25 09:43:41 +01:00
Asim Aslam 280eb5b46d Update README.md 2025-04-25 09:42:43 +01:00
Asim Aslam 4ead4ff953 Update README.md (#2750) 2025-04-25 09:41:35 +01:00
Asim Aslam 200a3cb0ce Update README.md (#2749) 2025-04-23 16:04:24 +01:00
Asim Aslam 049dea6804 Update README.md 2025-04-23 15:56:47 +01:00
309 changed files with 39940 additions and 1606 deletions
+1
View File
@@ -0,0 +1 @@
github: asim
+32 -13
View File
@@ -1,25 +1,44 @@
---
name: Bug report
about: For reporting bugs in go-micro
title: "[BUG]"
labels: ""
assignees: ""
about: Create a report to help us improve
title: '[BUG] '
labels: bug
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is.
1. What are you trying to do?
2. What did you expect to happen?
3. What happens instead?
## To Reproduce
Steps to reproduce the behavior:
1. Create service with '...'
2. Configure plugin '...'
3. Run command '...'
4. See error
## How to reproduce the bug
## Expected behavior
A clear and concise description of what you expected to happen.
If possible, please include a minimal code snippet here.
## Code sample
```go
// Minimal reproducible code
```
## Environment
- Go Micro version: [e.g. v5.3.0]
- Go version: [e.g. 1.21.0]
- OS: [e.g. Ubuntu 22.04]
- Plugins used: [e.g. consul registry, nats broker]
Go Version: please paste `go version` output here
```go
please paste `go env` output here
## Logs
```
Paste relevant logs here
```
## Additional context
Add any other context about the problem here.
## Resources
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Examples](https://github.com/micro/go-micro/tree/master/internal/website/docs/examples)
- [API Reference](https://pkg.go.dev/go-micro.dev/v5)
@@ -1,16 +0,0 @@
---
name: Feature request / Enhancement
about: If you have a need not served by go-micro
title: "[FEATURE]"
labels: ""
assignees: ""
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Additional context**
Add any other context or screenshots about the feature request here.
+30
View File
@@ -0,0 +1,30 @@
---
name: Feature request
about: Suggest an idea for this project
title: '[FEATURE] '
labels: enhancement
assignees: ''
---
## Is your feature request related to a problem?
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
## Describe the solution you'd like
A clear and concise description of what you want to happen.
## Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
## Use case
Describe how this feature would be used in practice. What problem does it solve?
## Additional context
Add any other context, code examples, or screenshots about the feature request here.
## Willing to contribute?
- [ ] I'd be willing to submit a PR for this feature
## Resources
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Plugins](https://github.com/micro/go-micro/tree/master/internal/website/docs/plugins.md)
- [Roadmap](https://github.com/micro/go-micro/blob/master/ROADMAP.md)
+26 -8
View File
@@ -1,13 +1,31 @@
---
name: Question
about: Ask a question about go-micro
title: ""
labels: ""
assignees: ""
about: Ask a question about using Go Micro
title: '[QUESTION] '
labels: question
assignees: ''
---
Before asking, please check if your question has already been answered:
## Your question
A clear and concise question about Go Micro usage.
1. Check the documentation - https://micro.mu/docs/
2. Check the examples and plugins - https://github.com/micro/examples & https://github.com/micro/go-plugins
3. Search existing issues
## What have you tried?
Describe what you've already attempted or researched.
## Code sample (if applicable)
```go
// Your code here
```
## Context
Provide any additional context that might help answer your question.
## Resources you've checked
- [ ] [Getting Started Guide](https://github.com/micro/go-micro/tree/master/internal/website/docs/getting-started.md)
- [ ] [Examples](https://github.com/micro/go-micro/tree/master/internal/website/docs/examples)
- [ ] [API Documentation](https://pkg.go.dev/go-micro.dev/v5)
- [ ] Searched existing issues
## Helpful links
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Plugins Guide](https://github.com/micro/go-micro/tree/master/internal/website/docs/plugins.md)
-11
View File
@@ -1,11 +0,0 @@
# Pull Request template
Please, go through these steps before clicking submit on this PR.
1. Make sure this PR targets the `develop` branch. We follow the git-flow branching model.
2. Give a descriptive title to your PR.
3. Provide a description of your changes.
4. Make sure you have some relevant tests.
5. Put `closes #XXXX` in your comment to auto-close the issue that your PR fixes (if applicable).
## PLEASE REMOVE THIS TEMPLATE BEFORE SUBMITTING
+43 -2
View File
@@ -18,7 +18,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.19
go-version: 1.24
check-latest: true
cache: true
- name: Get dependencies
@@ -27,7 +27,48 @@ jobs:
go get -v -t -d ./...
- name: Run tests
id: tests
run: richgo test -v -race -cover -bench=. ./...
run: richgo test -v -race -cover ./...
env:
IN_TRAVIS_CI: yes
RICHGO_FORCE_COLOR: 1
etcd-integration:
name: Etcd Integration Tests
runs-on: ubuntu-latest
permissions:
contents: read
services:
etcd:
image: quay.io/coreos/etcd:v3.5.2
env:
ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379
ETCD_ADVERTISE_CLIENT_URLS: http://0.0.0.0:2379
ports:
- 2379:2379
options: >-
--health-cmd "etcdctl endpoint health"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.24
check-latest: true
cache: true
- name: Get dependencies
run: |
go install github.com/kyoh86/richgo@latest
go get -v -t -d ./...
- name: Wait for etcd
run: |
timeout 30 bash -c 'until curl -s http://localhost:2379/health; do sleep 1; done'
- name: Run etcd integration tests
run: richgo test -v -race ./registry/etcd/...
env:
ETCD_ADDRESS: localhost:2379
IN_TRAVIS_CI: yes
RICHGO_FORCE_COLOR: 1
+51
View File
@@ -0,0 +1,51 @@
# Sample workflow for building and deploying a Jekyll site to GitHub Pages
name: Deploy Jekyll with GitHub Pages dependencies preinstalled
on:
# Runs on pushes targeting the default branch
push:
branches: ["master"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Build with Jekyll
uses: actions/jekyll-build-pages@v1
with:
source: ./internal/website
destination: ./_site
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
+207
View File
@@ -0,0 +1,207 @@
# Contributing to Go Micro
Thank you for your interest in contributing to Go Micro! This document provides guidelines and instructions for contributing.
## Code of Conduct
Be respectful, inclusive, and collaborative. We're all here to build great software together.
## Getting Started
1. Fork the repository
2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/go-micro.git`
3. Add upstream remote: `git remote add upstream https://github.com/micro/go-micro.git`
4. Create a feature branch: `git checkout -b feature/my-feature`
## Development Setup
```bash
# Install dependencies
go mod download
# Run tests
go test ./...
# Run tests with coverage
go test -race -coverprofile=coverage.out ./...
# Run linter (install golangci-lint first)
golangci-lint run
```
## Making Changes
### Code Guidelines
- Follow standard Go conventions (use `gofmt`, `golint`)
- Write clear, descriptive commit messages
- Add tests for new functionality
- Update documentation for API changes
- Keep PRs focused - one feature/fix per PR
### Commit Messages
Use conventional commits format:
```
type(scope): subject
body
footer
```
Types:
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `test`: Test additions/changes
- `refactor`: Code refactoring
- `perf`: Performance improvements
- `chore`: Maintenance tasks
Examples:
```
feat(registry): add kubernetes registry plugin
fix(broker): resolve nats connection leak
docs(examples): add streaming example
```
### Testing
- Write unit tests for all new code
- Ensure existing tests pass
- Add integration tests for plugin implementations
- Test with multiple Go versions (1.20+)
```bash
# Run specific package tests
go test ./registry/...
# Run with verbose output
go test -v ./...
# Run specific test
go test -run TestMyFunction ./pkg/...
```
### Documentation
- Update relevant markdown files in `internal/website/docs/`
- Add examples to `internal/website/docs/examples/` for new features
- Update README.md for major features
- Add godoc comments for exported functions/types
## Pull Request Process
1. **Update your branch**
```bash
git fetch upstream
git rebase upstream/master
```
2. **Run tests and linting**
```bash
go test ./...
golangci-lint run
```
3. **Push to your fork**
```bash
git push origin feature/my-feature
```
4. **Create Pull Request**
- Use a descriptive title
- Reference any related issues
- Describe what changed and why
- Add screenshots for UI changes
- Mark as draft if work in progress
5. **PR Review**
- Respond to feedback promptly
- Make requested changes
- Re-request review after updates
### PR Checklist
- [ ] Tests pass locally
- [ ] Code follows Go conventions
- [ ] Documentation updated
- [ ] Commit messages are clear
- [ ] Branch is up to date with master
- [ ] No merge conflicts
## Adding Plugins
New plugins should:
1. Live in the appropriate interface directory (e.g., `registry/myplugin/`)
2. Implement the interface completely
3. Include comprehensive tests
4. Provide usage examples
5. Document configuration options (env vars, options)
6. Add to plugin documentation
Example structure:
```
registry/myplugin/
├── myplugin.go # Main implementation
├── myplugin_test.go # Tests
├── options.go # Plugin-specific options
└── README.md # Usage and configuration
```
## Reporting Issues
Before creating an issue:
1. Search existing issues
2. Check documentation
3. Try the latest version
When reporting bugs:
- Use the bug report template
- Include minimal reproduction code
- Specify versions (Go, Go Micro, plugins)
- Provide relevant logs
## Documentation Contributions
Documentation improvements are always welcome!
- Fix typos and grammar
- Improve clarity
- Add missing examples
- Update outdated information
Documentation lives in `internal/website/docs/`. Preview locally with Jekyll:
```bash
cd internal/website
bundle install
bundle exec jekyll serve --livereload
```
## Community
- GitHub Issues: Bug reports and feature requests
- GitHub Discussions: Questions, ideas, and community chat
- Sponsorship: [GitHub Sponsors](https://github.com/sponsors/micro)
## Release Process
Maintainers handle releases:
1. Update CHANGELOG.md
2. Tag release: `git tag -a v5.x.x -m "Release v5.x.x"`
3. Push tag: `git push origin v5.x.x`
4. GitHub Actions creates release
## Questions?
- Check [documentation](internal/website/docs/)
- Browse [examples](internal/website/docs/examples/)
- Open a [question issue](.github/ISSUE_TEMPLATE/question.md)
Thank you for contributing to Go Micro! 🎉
+146 -28
View File
@@ -1,7 +1,9 @@
# Go Micro [![Go.Dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go-micro.dev/v5?tab=doc) [![Go Report Card](https://goreportcard.com/badge/github.com/go-micro/go-micro)](https://goreportcard.com/report/github.com/go-micro/go-micro)
# Go Micro [![Go.Dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go-micro.dev/v5?tab=doc) [![Go Report Card](https://goreportcard.com/badge/github.com/go-micro/go-micro)](https://goreportcard.com/report/github.com/go-micro/go-micro)
Go Micro is a framework for distributed systems development.
**[📖 Documentation](https://go-micro.dev/docs/)** | [Sponsor the project](https://github.com/sponsors/micro)
## Overview
Go Micro provides the core requirements for distributed systems development including RPC and Event driven communication.
@@ -45,15 +47,21 @@ in the plugins repo. State and persistence becomes a core requirement beyond pro
## Getting Started
To make use of Go Micro import it
To make use of Go Micro
```golang
import "go-micro.dev/v5"
```bash
go get go-micro.dev/v5@latest
```
Define a handler (protobuf is optionally supported - see [example](https://github.com/go-micro/examples/blob/main/helloworld/main.go))
Create a service and register a handler
```go
package main
import (
"go-micro.dev/v5"
)
```golang
type Request struct {
Name string `json:"name"`
}
@@ -62,47 +70,157 @@ type Response struct {
Message string `json:"message"`
}
type Helloworld struct{}
type Say struct{}
func (h *Helloworld) Greeting(ctx context.Context, req *Request, rsp *Response) error {
func (h *Say) Hello(ctx context.Context, req *Request, rsp *Response) error {
rsp.Message = "Hello " + req.Name
return nil
}
func main() {
// create the service
service := micro.New("helloworld")
// register handler
service.Handle(new(Say))
// run the service
service.Run()
}
```
Create, initialise and run the service
Set a fixed address
```golang
// create a new service
```go
service := micro.NewService(
micro.Name("helloworld"),
micro.Handle(new(Helloworld)),
)
// initialise flags
service.Init()
// start the service
service.Run()
```
Optionally set fixed address
```golang
service := micro.NewService(
// set address
micro.Address(":8080"),
)
```
Call it via curl
```
```bash
curl -XPOST \
-H 'Content-Type: application/json' \
-H 'Micro-Endpoint: Helloworld.Greeting' \
-H 'Micro-Endpoint: Say.Hello' \
-d '{"name": "alice"}' \
http://localhost:8080
```
## Experimental
There's a new `genai` package for generative AI capabilities.
## Protobuf
Install the code generator and see usage in the docs:
```bash
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
Docs: [`internal/website/docs/getting-started.md`](internal/website/docs/getting-started.md)
## Command Line
Install the CLI:
```
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
### Quick Start
```bash
micro new helloworld # Create a new service
cd helloworld
micro run # Run with API gateway
```
Then open http://localhost:8080 to see your service and call it from the browser.
### micro run
`micro run` starts your services with:
- **API Gateway** - HTTP to RPC proxy at `/api/{service}/{method}`
- **Web Dashboard** - Browse and call services at `/`
- **Health Checks** - Aggregated health at `/health`
- **Hot Reload** - Auto-rebuild on file changes
```bash
micro run # Gateway on :8080
micro run --address :3000 # Custom gateway port
micro run --no-gateway # Services only
micro run --env production # Use production environment
```
### Configuration
For multi-service projects, create a `micro.mu` file:
```
service users
path ./users
port 8081
service posts
path ./posts
port 8082
depends users
env development
DATABASE_URL sqlite://./dev.db
```
The gateway runs on :8080 by default, so services should use other ports.
### Deployment
Deploy to any Linux server with systemd:
```bash
# On your server (one-time setup)
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
# From your laptop
micro deploy user@your-server
```
The deploy command:
1. Builds binaries for Linux
2. Copies via SSH to the server
3. Sets up systemd services
4. Verifies services are healthy
Manage deployed services:
```bash
micro status --remote user@server # Check status
micro logs --remote user@server # View logs
micro logs myservice --remote user@server -f # Follow specific service
```
No Docker required. No Kubernetes. Just systemd.
See [docs/deployment.md](docs/deployment.md) for full deployment guide.
See [cmd/micro/README.md](cmd/micro/README.md) for full CLI documentation.
Docs: [`internal/website/docs`](internal/website/docs)
Package reference: https://pkg.go.dev/go-micro.dev/v5
Selected topics:
- Getting Started: [`internal/website/docs/getting-started.md`](internal/website/docs/getting-started.md)
- Plugins overview: [`internal/website/docs/plugins.md`](internal/website/docs/plugins.md)
- Learn by Example: [`internal/website/docs/examples/index.md`](internal/website/docs/examples/index.md)
- Performance Considerations: [`docs/performance.md`](docs/performance.md)
## Adopters
- [Sourse](https://sourse.eu) - Work in the field of earth observation, including embedded Kubernetes running onboard aircraft, and weve built a mission management SaaS platform using Go Micro.
+163
View File
@@ -0,0 +1,163 @@
# Go Micro Roadmap
This roadmap outlines the planned features and improvements for Go Micro. Community feedback and contributions are welcome!
## Current Focus (Q1 2026)
### Documentation & Developer Experience
- [x] Modernize documentation structure
- [x] Add learn-by-example guides
- [x] Update issue templates
- [ ] Create video tutorials
- [ ] Interactive documentation site
- [ ] Plugin discovery dashboard
### Observability
- [ ] OpenTelemetry native support
- [ ] Auto-instrumentation for handlers
- [ ] Metrics export standardization
- [ ] Distributed tracing examples
- [ ] Integration with popular observability platforms
### Developer Tools
- [ ] `micro dev` with hot reload
- [ ] Service templates (`micro new --template`)
- [ ] Better error messages with suggestions
- [ ] Debug tooling improvements
- [ ] VS Code extension for Go Micro
## Q2 2026
### Production Readiness
- [ ] Health check standardization
- [ ] Graceful shutdown improvements
- [ ] Resource cleanup best practices
- [ ] Load testing framework integration
- [ ] Performance benchmarking suite
### Cloud Native
- [ ] Kubernetes operator
- [ ] Helm charts for common setups
- [ ] Service mesh integration guides (Istio, Linkerd)
- [ ] Cloud provider quickstarts (AWS, GCP, Azure)
- [ ] Multi-cluster patterns
### Security
- [ ] mTLS by default option
- [ ] Secret management integration (Vault, AWS Secrets Manager)
- [ ] RBAC improvements
- [ ] Security audit and hardening
- [ ] CVE scanning and response process
## Q3 2026
### Plugin Ecosystem
- [ ] Plugin marketplace/registry
- [ ] Plugin quality standards
- [ ] Community plugin contributions
- [ ] Plugin compatibility matrix
- [ ] Auto-discovery of available plugins
### Streaming & Async
- [ ] Improved streaming support
- [ ] Server-sent events (SSE) support
- [ ] WebSocket plugin
- [ ] Event sourcing patterns
- [ ] CQRS examples
### Testing
- [ ] Mock generation tooling
- [ ] Integration test helpers
- [ ] Contract testing support
- [ ] Chaos engineering examples
- [ ] E2E testing framework
## Q4 2026
### Performance
- [ ] Connection pooling optimizations
- [ ] Zero-allocation paths
- [ ] gRPC performance improvements
- [ ] Caching strategies guide
- [ ] Performance profiling tools
### Developer Productivity
- [ ] Code generation improvements
- [ ] Better IDE support
- [ ] Debugging tools
- [ ] Migration automation tools
- [ ] Upgrade helpers
### Community
- [ ] Regular blog posts and case studies
- [ ] Community spotlight program
- [ ] Contribution rewards
- [ ] Monthly community calls
- [ ] Conference presence
## Long-term Vision
### Core Framework
- Maintain backward compatibility (Go Micro v5+)
- Progressive disclosure of complexity
- Best-in-class developer experience
- Production-grade reliability
- Comprehensive plugin ecosystem
### Ecosystem Goals
- 100+ production deployments documented
- 50+ community plugins
- Active contributor community
- Regular releases (monthly patches, quarterly features)
- Comprehensive benchmarks vs alternatives
### Differentiation
- **Batteries included, fully swappable** - Start simple, scale complex
- **Zero-config local development** - No infrastructure required to start
- **Plugin ecosystem in-repo** - No version compatibility hell
- **Progressive complexity** - Learn as you grow
- **Cloud-native first** - Built for Kubernetes and containers
## Contributing
We welcome contributions to any roadmap items! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
### High Priority Areas
1. Documentation improvements
2. Real-world examples
3. Plugin development
4. Performance optimizations
5. Testing infrastructure
### How to Contribute
- Pick an item from the roadmap
- Open an issue to discuss approach
- Submit a PR with implementation
- Help review others' contributions
## Feedback
Have suggestions for the roadmap?
- Open a [feature request](.github/ISSUE_TEMPLATE/feature_request.md)
- Start a discussion in GitHub Discussions
- Comment on existing roadmap issues
## Version Compatibility
We follow semantic versioning:
- Major versions (v5 → v6): Breaking changes
- Minor versions (v5.3 → v5.4): New features, backward compatible
- Patch versions (v5.3.0 → v5.3.1): Bug fixes, no API changes
## Support Timeline
- v5: Active development (current)
- v4: Security fixes only (until v6 release)
- v3: End of life
---
Last updated: November 2025
This roadmap is subject to change based on community needs and priorities. Star the repo to stay updated! ⭐
+157
View File
@@ -0,0 +1,157 @@
package jwt
import (
"sync"
"time"
jwtToken "github.com/micro/plugins/v5/auth/jwt/token"
"go-micro.dev/v5/auth"
"go-micro.dev/v5/cmd"
)
func init() {
cmd.DefaultAuths["jwt"] = NewAuth
}
// NewAuth returns a new instance of the Auth service.
func NewAuth(opts ...auth.Option) auth.Auth {
j := new(jwt)
j.Init(opts...)
return j
}
func NewRules() auth.Rules {
return new(jwtRules)
}
type jwt struct {
sync.Mutex
options auth.Options
jwt jwtToken.Provider
}
type jwtRules struct {
sync.Mutex
rules []*auth.Rule
}
func (j *jwt) String() string {
return "jwt"
}
func (j *jwt) Init(opts ...auth.Option) {
j.Lock()
defer j.Unlock()
for _, o := range opts {
o(&j.options)
}
j.jwt = jwtToken.New(
jwtToken.WithPrivateKey(j.options.PrivateKey),
jwtToken.WithPublicKey(j.options.PublicKey),
)
}
func (j *jwt) Options() auth.Options {
j.Lock()
defer j.Unlock()
return j.options
}
func (j *jwt) Generate(id string, opts ...auth.GenerateOption) (*auth.Account, error) {
options := auth.NewGenerateOptions(opts...)
account := &auth.Account{
ID: id,
Type: options.Type,
Scopes: options.Scopes,
Metadata: options.Metadata,
Issuer: j.Options().Namespace,
}
// generate a JWT secret which can be provided to the Token() method
// and exchanged for an access token
secret, err := j.jwt.Generate(account)
if err != nil {
return nil, err
}
account.Secret = secret.Token
// return the account
return account, nil
}
func (j *jwtRules) Grant(rule *auth.Rule) error {
j.Lock()
defer j.Unlock()
j.rules = append(j.rules, rule)
return nil
}
func (j *jwtRules) Revoke(rule *auth.Rule) error {
j.Lock()
defer j.Unlock()
rules := make([]*auth.Rule, 0, len(j.rules))
for _, r := range j.rules {
if r.ID != rule.ID {
rules = append(rules, r)
}
}
j.rules = rules
return nil
}
func (j *jwtRules) Verify(acc *auth.Account, res *auth.Resource, opts ...auth.VerifyOption) error {
j.Lock()
defer j.Unlock()
var options auth.VerifyOptions
for _, o := range opts {
o(&options)
}
return auth.Verify(j.rules, acc, res)
}
func (j *jwtRules) List(opts ...auth.ListOption) ([]*auth.Rule, error) {
j.Lock()
defer j.Unlock()
return j.rules, nil
}
func (j *jwt) Inspect(token string) (*auth.Account, error) {
return j.jwt.Inspect(token)
}
func (j *jwt) Token(opts ...auth.TokenOption) (*auth.Token, error) {
options := auth.NewTokenOptions(opts...)
secret := options.RefreshToken
if len(options.Secret) > 0 {
secret = options.Secret
}
account, err := j.jwt.Inspect(secret)
if err != nil {
return nil, err
}
access, err := j.jwt.Generate(account, jwtToken.WithExpiry(options.Expiry))
if err != nil {
return nil, err
}
refresh, err := j.jwt.Generate(account, jwtToken.WithExpiry(options.Expiry+time.Hour))
if err != nil {
return nil, err
}
return &auth.Token{
Created: access.Created,
Expiry: access.Expiry,
AccessToken: access.Token,
RefreshToken: refresh.Token,
}, nil
}
+109
View File
@@ -0,0 +1,109 @@
package token
import (
"encoding/base64"
"time"
"github.com/dgrijalva/jwt-go"
"go-micro.dev/v5/auth"
)
// authClaims to be encoded in the JWT.
type authClaims struct {
Type string `json:"type"`
Scopes []string `json:"scopes"`
Metadata map[string]string `json:"metadata"`
jwt.StandardClaims
}
// JWT implementation of token provider.
type JWT struct {
opts Options
}
// New returns an initialized basic provider.
func New(opts ...Option) Provider {
return &JWT{
opts: NewOptions(opts...),
}
}
// Generate a new JWT.
func (j *JWT) Generate(acc *auth.Account, opts ...GenerateOption) (*Token, error) {
// decode the private key
priv, err := base64.StdEncoding.DecodeString(j.opts.PrivateKey)
if err != nil {
return nil, err
}
// parse the private key
key, err := jwt.ParseRSAPrivateKeyFromPEM(priv)
if err != nil {
return nil, ErrEncodingToken
}
// parse the options
options := NewGenerateOptions(opts...)
// generate the JWT
expiry := time.Now().Add(options.Expiry)
t := jwt.NewWithClaims(jwt.SigningMethodRS256, authClaims{
acc.Type, acc.Scopes, acc.Metadata, jwt.StandardClaims{
Subject: acc.ID,
Issuer: acc.Issuer,
ExpiresAt: expiry.Unix(),
},
})
tok, err := t.SignedString(key)
if err != nil {
return nil, err
}
// return the token
return &Token{
Token: tok,
Expiry: expiry,
Created: time.Now(),
}, nil
}
// Inspect a JWT.
func (j *JWT) Inspect(t string) (*auth.Account, error) {
// decode the public key
pub, err := base64.StdEncoding.DecodeString(j.opts.PublicKey)
if err != nil {
return nil, err
}
// parse the public key
res, err := jwt.ParseWithClaims(t, &authClaims{}, func(token *jwt.Token) (interface{}, error) {
return jwt.ParseRSAPublicKeyFromPEM(pub)
})
if err != nil {
return nil, ErrInvalidToken
}
// validate the token
if !res.Valid {
return nil, ErrInvalidToken
}
claims, ok := res.Claims.(*authClaims)
if !ok {
return nil, ErrInvalidToken
}
// return the token
return &auth.Account{
ID: claims.Subject,
Issuer: claims.Issuer,
Type: claims.Type,
Scopes: claims.Scopes,
Metadata: claims.Metadata,
}, nil
}
// String returns JWT.
func (j *JWT) String() string {
return "jwt"
}
+85
View File
@@ -0,0 +1,85 @@
package token
import (
"os"
"testing"
"time"
"go-micro.dev/v5/auth"
)
func TestGenerate(t *testing.T) {
privKey, err := os.ReadFile("test/sample_key")
if err != nil {
t.Fatalf("Unable to read private key: %v", err)
}
j := New(
WithPrivateKey(string(privKey)),
)
_, err = j.Generate(&auth.Account{ID: "test"})
if err != nil {
t.Fatalf("Generate returned %v error, expected nil", err)
}
}
func TestInspect(t *testing.T) {
pubKey, err := os.ReadFile("test/sample_key.pub")
if err != nil {
t.Fatalf("Unable to read public key: %v", err)
}
privKey, err := os.ReadFile("test/sample_key")
if err != nil {
t.Fatalf("Unable to read private key: %v", err)
}
j := New(
WithPublicKey(string(pubKey)),
WithPrivateKey(string(privKey)),
)
t.Run("Valid token", func(t *testing.T) {
md := map[string]string{"foo": "bar"}
scopes := []string{"admin"}
subject := "test"
acc := &auth.Account{ID: subject, Scopes: scopes, Metadata: md}
tok, err := j.Generate(acc)
if err != nil {
t.Fatalf("Generate returned %v error, expected nil", err)
}
tok2, err := j.Inspect(tok.Token)
if err != nil {
t.Fatalf("Inspect returned %v error, expected nil", err)
}
if acc.ID != subject {
t.Errorf("Inspect returned %v as the token subject, expected %v", acc.ID, subject)
}
if len(tok2.Scopes) != len(scopes) {
t.Errorf("Inspect returned %v scopes, expected %v", len(tok2.Scopes), len(scopes))
}
if len(tok2.Metadata) != len(md) {
t.Errorf("Inspect returned %v as the token metadata, expected %v", tok2.Metadata, md)
}
})
t.Run("Expired token", func(t *testing.T) {
tok, err := j.Generate(&auth.Account{}, WithExpiry(-10*time.Second))
if err != nil {
t.Fatalf("Generate returned %v error, expected nil", err)
}
if _, err = j.Inspect(tok.Token); err != ErrInvalidToken {
t.Fatalf("Inspect returned %v error, expected %v", err, ErrInvalidToken)
}
})
t.Run("Invalid token", func(t *testing.T) {
_, err := j.Inspect("Invalid token")
if err != ErrInvalidToken {
t.Fatalf("Inspect returned %v error, expected %v", err, ErrInvalidToken)
}
})
}
+78
View File
@@ -0,0 +1,78 @@
package token
import (
"time"
"go-micro.dev/v5/store"
)
type Options struct {
// Store to persist the tokens
Store store.Store
// PublicKey base64 encoded, used by JWT
PublicKey string
// PrivateKey base64 encoded, used by JWT
PrivateKey string
}
type Option func(o *Options)
// WithStore sets the token providers store.
func WithStore(s store.Store) Option {
return func(o *Options) {
o.Store = s
}
}
// WithPublicKey sets the JWT public key.
func WithPublicKey(key string) Option {
return func(o *Options) {
o.PublicKey = key
}
}
// WithPrivateKey sets the JWT private key.
func WithPrivateKey(key string) Option {
return func(o *Options) {
o.PrivateKey = key
}
}
func NewOptions(opts ...Option) Options {
var options Options
for _, o := range opts {
o(&options)
}
// set default store
if options.Store == nil {
options.Store = store.DefaultStore
}
return options
}
type GenerateOptions struct {
// Expiry for the token
Expiry time.Duration
}
type GenerateOption func(o *GenerateOptions)
// WithExpiry for the generated account's token expires.
func WithExpiry(d time.Duration) GenerateOption {
return func(o *GenerateOptions) {
o.Expiry = d
}
}
// NewGenerateOptions from a slice of options.
func NewGenerateOptions(opts ...GenerateOption) GenerateOptions {
var options GenerateOptions
for _, o := range opts {
o(&options)
}
// set default Expiry of token
if options.Expiry == 0 {
options.Expiry = time.Minute * 15
}
return options
}
+1
View File
@@ -0,0 +1 @@
LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS3dJQkFBS0NBZ0VBOFNiSlA1WGJFaWRSbTViMnNOcExHbzJlV2ZVNU9KZTBpemdySHdEOEg3RjZQa1BkCi9SbDkvMXBNVjdNaU8zTEh3dGhIQzJCUllxcisxd0Zkb1pDR0JZckxhWHVYRnFLMHZ1WmhQcUUzYXpqdUlIUXUKMEJIL2xYUU1xeUVxRjVNSTJ6ZWpDNHpNenIxNU9OK2dFNEpuaXBqcC9DZGpPUEFEbUpHK0JKOXFlRS9RUGVtLwptVWRJVC9MYUY3a1F4eVlLNVZLbitOZ09Xek1sektBQXBDbjdUVEtCVWU4RlpHNldTWDdMVjBlTEdIc29pYnhsCm85akRqbFk1b0JPY3pmcWVOV0hLNUdYQjdRd3BMTmg5NDZQelpucW9hcFdVZStZL1JPaUhpekpUY3I1Wk1TTDUKd2xFcThoTmhtaG01Tk5lL08rR2dqQkROU2ZVaDA2K3E0bmdtYm1OWDVoODM4QmJqUmN5YzM2ZHd6NkpVK2R1bwpSdFFoZ2lZOTEwcFBmOWJhdVhXcXdVQ1VhNHFzSHpqS1IwTC9OMVhYQXlsQ0RqeWVnWnp6Y093MkNIOFNrZkZVCnJnTHJQYkVCOWVnY0drMzgrYnBLczNaNlJyNSt0bkQxQklQSUZHTGVJMFVPQzAreGlCdjBvenhJRE9GbldhOVUKVEdEeFV4OG9qOFZJZVJuV0RxNk1jMWlKcDhVeWNpQklUUnR3NGRabzcweG1mbmVJV3pyM0tTTmFoU29nSmRSMApsYVF6QXVQM2FpV1hJTXAyc2M4U2MrQmwrTGpYbUJveEJyYUJIaDlLa0pKRWNnQUZ3czJib2pDbEpPWXhvRi9YCmdGS1NzSW5IRHJIVk95V1BCZTNmYWRFYzc3YituYi9leE96cjFFcnhoR2c5akZtcmtPK3M0eEdodjZNQ0F3RUEKQVFLQ0FnRUFqUzc1Q2VvUlRRcUtBNzZaaFNiNGEzNVlKRENtcEpSazFsRTNKYnFzNFYxRnhXaDBjZmJYeG9VMgpSdTRRYjUrZWhsdWJGSFQ2a1BxdG9uRWhRVExjMUNmVE9WbHJOb3hocDVZM2ZyUmlQcnNnNXcwK1R3RUtrcFJUCnltanJQTXdQbGxCM2U0NmVaYmVXWGc3R3FFVmptMGcxVFRRK0tocVM4R0w3VGJlTFhRN1ZTem9ydTNCNVRKMVEKeEN6TVB0dnQ2eDYrU3JrcmhvZG1iT3VNRkpDam1TbWxmck9pZzQ4Zkc3NUpERHRObXpLWHBEUVJpYUNodFJhVQpQRHpmUTlTamhYdFFqdkZvWFFFT3BqdkZVRjR2WldNUWNQNUw1VklDM3JRSWp4MFNzQTN6S0FwakVUbjJHNjN2CktZby8zVWttbzhkUCtGRHA3NCs5a3pLNHFFaFJycEl3bEtiN0VOZWtDUXZqUFl1K3pyKzMyUXdQNTJ2L2FveWQKdjJJaUY3M2laTU1vZDhhYjJuQStyVEI2T0cvOVlSYk5kV21tay9VTi9jUHYrN214TmZ6Y1d1ZU1XcThxMXh4eAptNTNpR0NSQ29PQ1lDQk4zcUFkb1JwYW5xd3lCOUxrLzFCQjBHUld3MjgxK3VhNXNYRnZBVDBKeTVURnduMncvClU1MlJKWFlNOXVhMFBvd214b0RDUWRuNFZYVkdNZGdXaHN4aXhHRlYwOUZObWJJQWJaN0xaWGtkS1gzc1ZVbTcKWU1WYWIzVVo2bEhtdXYzT1NzcHNVUlRqN1hiRzZpaVVlaDU1aW91OENWbnRndWtFcnEzQTQwT05FVzhjNDBzOQphVTBGaSs4eWZpQTViaVZHLzF0bWlucUVERkhuQStnWk1xNEhlSkZxcWZxaEZKa1JwRGtDZ2dFQkFQeGR1NGNKCm5Da1duZDdPWFlHMVM3UDdkVWhRUzgwSDlteW9uZFc5bGFCQm84RWRPeTVTZzNOUmsxQ2pNZFZ1a3FMcjhJSnkKeStLWk15SVpvSlJvbllaMEtIUUVMR3ZLbzFOS2NLQ1FJbnYvWHVCdFJpRzBVb1pQNVkwN0RpRFBRQWpYUjlXUwpBc0EzMmQ1eEtFOC91Y3h0MjVQVzJFakNBUmtVeHQ5d0tKazN3bC9JdXVYRlExTDdDWjJsOVlFUjlHeWxUbzhNCmxXUEY3YndtUFV4UVNKaTNVS0FjTzZweTVUU1lkdWQ2aGpQeXJwSXByNU42VGpmTlRFWkVBeU9LbXVpOHVkUkoKMUg3T3RQVEhGZElKQjNrNEJnRDZtRE1HbjB2SXBLaDhZN3NtRUZBbFkvaXlCZjMvOHk5VHVMb1BycEdqR3RHbgp4Y2RpMHFud2p0SGFNbFVDZ2dFQkFQU2Z0dVFCQ2dTU2JLUSswUEFSR2VVeEQyTmlvZk1teENNTmdHUzJ5Ull3CjRGaGV4ZWkwMVJoaFk1NjE3UjduR1dzb0czd1RQa3dvRTJtbE1aQkoxeWEvUU9RRnQ3WG02OVl0RGh0T2FWbDgKL0o4dlVuSTBtWmxtT2pjTlRoYnVPZDlNSDlRdGxIRUMxMlhYdHJNb3Fsb0U2a05TT0pJalNxYm9wcDRXc1BqcApvZTZ0Nkdyd1RhOHBHeUJWWS90Mi85Ym5ORHVPVlpjODBaODdtY2gzcDNQclBqU3h5di9saGxYMFMwYUdHTkhTCk1XVjdUa25OaGo1TWlIRXFnZ1pZemtBWTkyd1JoVENnU1A2M0VNcitUWXFudXVuMXJHbndPYm95TDR2aFRpV0UKcU42UDNCTFlCZ1FpMllDTDludEJrOEl6RHZyd096dW5GVnhhZ0g5SVVoY0NnZ0VCQUwzQXlLa1BlOENWUmR6cQpzL284VkJDZmFSOFhhUGRnSGxTek1BSXZpNXEwNENqckRyMlV3MHZwTVdnM1hOZ0xUT3g5bFJpd3NrYk9SRmxHCmhhd3hRUWlBdkk0SE9WTlBTU0R1WHVNTG5USTQ0S0RFNlMrY2cxU0VMS2pWbDVqcDNFOEpkL1RJMVpLc0xBQUsKZTNHakM5UC9ZbE8xL21ndW4xNjVkWk01cFAwWHBPb2FaeFV2RHFFTktyekR0V1g0RngyOTZlUzdaSFJodFpCNwovQ2t1VUhlcmxrN2RDNnZzdWhTaTh2eTM3c0tPbmQ0K3c4cVM4czhZYVZxSDl3ZzVScUxxakp0bmJBUnc3alVDCm9KQ053M1hNdnc3clhaYzRTbnhVQUNMRGJNV2lLQy9xL1ZGWW9oTEs2WkpUVkJscWd5cjBSYzBRWmpDMlNJb0kKMjRwRWt3VUNnZ0VCQUpqb0FJVVNsVFY0WlVwaExXN3g4WkxPa01UWjBVdFFyd2NPR0hSYndPUUxGeUNGMVFWNQppejNiR2s4SmZyZHpVdk1sTmREZm9uQXVHTHhQa3VTVEUxWlg4L0xVRkJveXhyV3dvZ0cxaUtwME11QTV6em90CjROai9DbUtCQVkvWnh2anA5M2RFS21aZGxWQkdmeUFMeWpmTW5MWUovZXh5L09YSnhPUktZTUttSHg4M08zRWsKMWhvb0FwbTZabTIzMjRGME1iVU1ham5Idld2ZjhHZGJTNk5zcHd4L0dkbk1tYVMrdUJMVUhVMkNLbmc1bEIwVAp4OWJITmY0dXlPbTR0dXRmNzhCd1R5V3UreEdrVW0zZ2VZMnkvR1hqdDZyY2l1ajFGNzFDenZzcXFmZThTcDdJCnd6SHdxcTNzVHR5S2lCYTZuYUdEYWpNR1pKYSt4MVZJV204Q2dnRUJBT001ajFZR25Ba0pxR0czQWJSVDIvNUMKaVVxN0loYkswOGZsSGs5a2YwUlVjZWc0ZVlKY3dIRXJVaE4rdWQyLzE3MC81dDYra0JUdTVZOUg3bkpLREtESQpoeEg5SStyamNlVkR0RVNTRkluSXdDQ1lrOHhOUzZ0cHZMV1U5b0pibGFKMlZsalV2NGRFWGVQb0hkREh1Zk9ZClVLa0lsV2E3Uit1QzNEOHF5U1JrQnFLa3ZXZ1RxcFNmTVNkc1ZTeFIzU2Q4SVhFSHFjTDNUNEtMWGtYNEdEamYKMmZOSTFpZkx6ekhJMTN3Tk5IUTVRNU9SUC9pell2QzVzZkx4U2ZIUXJiMXJZVkpKWkI5ZjVBUjRmWFpHSVFsbApjMG8xd0JmZFlqMnZxVDlpR09IQnNSSTlSL2M2RzJQcUt3aFRpSzJVR2lmVFNEUVFuUkF6b2tpQVkrbE8vUjQ9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
+1
View File
@@ -0,0 +1 @@
LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS3dJQkFBS0NBZ0VBOFNiSlA1WGJFaWRSbTViMnNOcExHbzJlV2ZVNU9KZTBpemdySHdEOEg3RjZQa1BkCi9SbDkvMXBNVjdNaU8zTEh3dGhIQzJCUllxcisxd0Zkb1pDR0JZckxhWHVYRnFLMHZ1WmhQcUUzYXpqdUlIUXUKMEJIL2xYUU1xeUVxRjVNSTJ6ZWpDNHpNenIxNU9OK2dFNEpuaXBqcC9DZGpPUEFEbUpHK0JKOXFlRS9RUGVtLwptVWRJVC9MYUY3a1F4eVlLNVZLbitOZ09Xek1sektBQXBDbjdUVEtCVWU4RlpHNldTWDdMVjBlTEdIc29pYnhsCm85akRqbFk1b0JPY3pmcWVOV0hLNUdYQjdRd3BMTmg5NDZQelpucW9hcFdVZStZL1JPaUhpekpUY3I1Wk1TTDUKd2xFcThoTmhtaG01Tk5lL08rR2dqQkROU2ZVaDA2K3E0bmdtYm1OWDVoODM4QmJqUmN5YzM2ZHd6NkpVK2R1bwpSdFFoZ2lZOTEwcFBmOWJhdVhXcXdVQ1VhNHFzSHpqS1IwTC9OMVhYQXlsQ0RqeWVnWnp6Y093MkNIOFNrZkZVCnJnTHJQYkVCOWVnY0drMzgrYnBLczNaNlJyNSt0bkQxQklQSUZHTGVJMFVPQzAreGlCdjBvenhJRE9GbldhOVUKVEdEeFV4OG9qOFZJZVJuV0RxNk1jMWlKcDhVeWNpQklUUnR3NGRabzcweG1mbmVJV3pyM0tTTmFoU29nSmRSMApsYVF6QXVQM2FpV1hJTXAyc2M4U2MrQmwrTGpYbUJveEJyYUJIaDlLa0pKRWNnQUZ3czJib2pDbEpPWXhvRi9YCmdGS1NzSW5IRHJIVk95V1BCZTNmYWRFYzc3YituYi9leE96cjFFcnhoR2c5akZtcmtPK3M0eEdodjZNQ0F3RUEKQVFLQ0FnRUFqUzc1Q2VvUlRRcUtBNzZaaFNiNGEzNVlKRENtcEpSazFsRTNKYnFzNFYxRnhXaDBjZmJYeG9VMgpSdTRRYjUrZWhsdWJGSFQ2a1BxdG9uRWhRVExjMUNmVE9WbHJOb3hocDVZM2ZyUmlQcnNnNXcwK1R3RUtrcFJUCnltanJQTXdQbGxCM2U0NmVaYmVXWGc3R3FFVmptMGcxVFRRK0tocVM4R0w3VGJlTFhRN1ZTem9ydTNCNVRKMVEKeEN6TVB0dnQ2eDYrU3JrcmhvZG1iT3VNRkpDam1TbWxmck9pZzQ4Zkc3NUpERHRObXpLWHBEUVJpYUNodFJhVQpQRHpmUTlTamhYdFFqdkZvWFFFT3BqdkZVRjR2WldNUWNQNUw1VklDM3JRSWp4MFNzQTN6S0FwakVUbjJHNjN2CktZby8zVWttbzhkUCtGRHA3NCs5a3pLNHFFaFJycEl3bEtiN0VOZWtDUXZqUFl1K3pyKzMyUXdQNTJ2L2FveWQKdjJJaUY3M2laTU1vZDhhYjJuQStyVEI2T0cvOVlSYk5kV21tay9VTi9jUHYrN214TmZ6Y1d1ZU1XcThxMXh4eAptNTNpR0NSQ29PQ1lDQk4zcUFkb1JwYW5xd3lCOUxrLzFCQjBHUld3MjgxK3VhNXNYRnZBVDBKeTVURnduMncvClU1MlJKWFlNOXVhMFBvd214b0RDUWRuNFZYVkdNZGdXaHN4aXhHRlYwOUZObWJJQWJaN0xaWGtkS1gzc1ZVbTcKWU1WYWIzVVo2bEhtdXYzT1NzcHNVUlRqN1hiRzZpaVVlaDU1aW91OENWbnRndWtFcnEzQTQwT05FVzhjNDBzOQphVTBGaSs4eWZpQTViaVZHLzF0bWlucUVERkhuQStnWk1xNEhlSkZxcWZxaEZKa1JwRGtDZ2dFQkFQeGR1NGNKCm5Da1duZDdPWFlHMVM3UDdkVWhRUzgwSDlteW9uZFc5bGFCQm84RWRPeTVTZzNOUmsxQ2pNZFZ1a3FMcjhJSnkKeStLWk15SVpvSlJvbllaMEtIUUVMR3ZLbzFOS2NLQ1FJbnYvWHVCdFJpRzBVb1pQNVkwN0RpRFBRQWpYUjlXUwpBc0EzMmQ1eEtFOC91Y3h0MjVQVzJFakNBUmtVeHQ5d0tKazN3bC9JdXVYRlExTDdDWjJsOVlFUjlHeWxUbzhNCmxXUEY3YndtUFV4UVNKaTNVS0FjTzZweTVUU1lkdWQ2aGpQeXJwSXByNU42VGpmTlRFWkVBeU9LbXVpOHVkUkoKMUg3T3RQVEhGZElKQjNrNEJnRDZtRE1HbjB2SXBLaDhZN3NtRUZBbFkvaXlCZjMvOHk5VHVMb1BycEdqR3RHbgp4Y2RpMHFud2p0SGFNbFVDZ2dFQkFQU2Z0dVFCQ2dTU2JLUSswUEFSR2VVeEQyTmlvZk1teENNTmdHUzJ5Ull3CjRGaGV4ZWkwMVJoaFk1NjE3UjduR1dzb0czd1RQa3dvRTJtbE1aQkoxeWEvUU9RRnQ3WG02OVl0RGh0T2FWbDgKL0o4dlVuSTBtWmxtT2pjTlRoYnVPZDlNSDlRdGxIRUMxMlhYdHJNb3Fsb0U2a05TT0pJalNxYm9wcDRXc1BqcApvZTZ0Nkdyd1RhOHBHeUJWWS90Mi85Ym5ORHVPVlpjODBaODdtY2gzcDNQclBqU3h5di9saGxYMFMwYUdHTkhTCk1XVjdUa25OaGo1TWlIRXFnZ1pZemtBWTkyd1JoVENnU1A2M0VNcitUWXFudXVuMXJHbndPYm95TDR2aFRpV0UKcU42UDNCTFlCZ1FpMllDTDludEJrOEl6RHZyd096dW5GVnhhZ0g5SVVoY0NnZ0VCQUwzQXlLa1BlOENWUmR6cQpzL284VkJDZmFSOFhhUGRnSGxTek1BSXZpNXEwNENqckRyMlV3MHZwTVdnM1hOZ0xUT3g5bFJpd3NrYk9SRmxHCmhhd3hRUWlBdkk0SE9WTlBTU0R1WHVNTG5USTQ0S0RFNlMrY2cxU0VMS2pWbDVqcDNFOEpkL1RJMVpLc0xBQUsKZTNHakM5UC9ZbE8xL21ndW4xNjVkWk01cFAwWHBPb2FaeFV2RHFFTktyekR0V1g0RngyOTZlUzdaSFJodFpCNwovQ2t1VUhlcmxrN2RDNnZzdWhTaTh2eTM3c0tPbmQ0K3c4cVM4czhZYVZxSDl3ZzVScUxxakp0bmJBUnc3alVDCm9KQ053M1hNdnc3clhaYzRTbnhVQUNMRGJNV2lLQy9xL1ZGWW9oTEs2WkpUVkJscWd5cjBSYzBRWmpDMlNJb0kKMjRwRWt3VUNnZ0VCQUpqb0FJVVNsVFY0WlVwaExXN3g4WkxPa01UWjBVdFFyd2NPR0hSYndPUUxGeUNGMVFWNQppejNiR2s4SmZyZHpVdk1sTmREZm9uQXVHTHhQa3VTVEUxWlg4L0xVRkJveXhyV3dvZ0cxaUtwME11QTV6em90CjROai9DbUtCQVkvWnh2anA5M2RFS21aZGxWQkdmeUFMeWpmTW5MWUovZXh5L09YSnhPUktZTUttSHg4M08zRWsKMWhvb0FwbTZabTIzMjRGME1iVU1ham5Idld2ZjhHZGJTNk5zcHd4L0dkbk1tYVMrdUJMVUhVMkNLbmc1bEIwVAp4OWJITmY0dXlPbTR0dXRmNzhCd1R5V3UreEdrVW0zZ2VZMnkvR1hqdDZyY2l1ajFGNzFDenZzcXFmZThTcDdJCnd6SHdxcTNzVHR5S2lCYTZuYUdEYWpNR1pKYSt4MVZJV204Q2dnRUJBT001ajFZR25Ba0pxR0czQWJSVDIvNUMKaVVxN0loYkswOGZsSGs5a2YwUlVjZWc0ZVlKY3dIRXJVaE4rdWQyLzE3MC81dDYra0JUdTVZOUg3bkpLREtESQpoeEg5SStyamNlVkR0RVNTRkluSXdDQ1lrOHhOUzZ0cHZMV1U5b0pibGFKMlZsalV2NGRFWGVQb0hkREh1Zk9ZClVLa0lsV2E3Uit1QzNEOHF5U1JrQnFLa3ZXZ1RxcFNmTVNkc1ZTeFIzU2Q4SVhFSHFjTDNUNEtMWGtYNEdEamYKMmZOSTFpZkx6ekhJMTN3Tk5IUTVRNU9SUC9pell2QzVzZkx4U2ZIUXJiMXJZVkpKWkI5ZjVBUjRmWFpHSVFsbApjMG8xd0JmZFlqMnZxVDlpR09IQnNSSTlSL2M2RzJQcUt3aFRpSzJVR2lmVFNEUVFuUkF6b2tpQVkrbE8vUjQ9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
+1
View File
@@ -0,0 +1 @@
LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUE4U2JKUDVYYkVpZFJtNWIyc05wTApHbzJlV2ZVNU9KZTBpemdySHdEOEg3RjZQa1BkL1JsOS8xcE1WN01pTzNMSHd0aEhDMkJSWXFyKzF3RmRvWkNHCkJZckxhWHVYRnFLMHZ1WmhQcUUzYXpqdUlIUXUwQkgvbFhRTXF5RXFGNU1JMnplakM0ek16cjE1T04rZ0U0Sm4KaXBqcC9DZGpPUEFEbUpHK0JKOXFlRS9RUGVtL21VZElUL0xhRjdrUXh5WUs1VktuK05nT1d6TWx6S0FBcENuNwpUVEtCVWU4RlpHNldTWDdMVjBlTEdIc29pYnhsbzlqRGpsWTVvQk9jemZxZU5XSEs1R1hCN1F3cExOaDk0NlB6ClpucW9hcFdVZStZL1JPaUhpekpUY3I1Wk1TTDV3bEVxOGhOaG1obTVOTmUvTytHZ2pCRE5TZlVoMDYrcTRuZ20KYm1OWDVoODM4QmJqUmN5YzM2ZHd6NkpVK2R1b1J0UWhnaVk5MTBwUGY5YmF1WFdxd1VDVWE0cXNIempLUjBMLwpOMVhYQXlsQ0RqeWVnWnp6Y093MkNIOFNrZkZVcmdMclBiRUI5ZWdjR2szOCticEtzM1o2UnI1K3RuRDFCSVBJCkZHTGVJMFVPQzAreGlCdjBvenhJRE9GbldhOVVUR0R4VXg4b2o4VkllUm5XRHE2TWMxaUpwOFV5Y2lCSVRSdHcKNGRabzcweG1mbmVJV3pyM0tTTmFoU29nSmRSMGxhUXpBdVAzYWlXWElNcDJzYzhTYytCbCtMalhtQm94QnJhQgpIaDlLa0pKRWNnQUZ3czJib2pDbEpPWXhvRi9YZ0ZLU3NJbkhEckhWT3lXUEJlM2ZhZEVjNzdiK25iL2V4T3pyCjFFcnhoR2c5akZtcmtPK3M0eEdodjZNQ0F3RUFBUT09Ci0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo=
+33
View File
@@ -0,0 +1,33 @@
package token
import (
"errors"
"time"
"go-micro.dev/v5/auth"
)
var (
// ErrNotFound is returned when a token cannot be found.
ErrNotFound = errors.New("token not found")
// ErrEncodingToken is returned when the service encounters an error during encoding.
ErrEncodingToken = errors.New("error encoding the token")
// ErrInvalidToken is returned when the token provided is not valid.
ErrInvalidToken = errors.New("invalid token provided")
)
// Provider generates and inspects tokens.
type Provider interface {
Generate(account *auth.Account, opts ...GenerateOption) (*Token, error)
Inspect(token string) (*auth.Account, error)
String() string
}
type Token struct {
// The actual token
Token string `json:"token"`
// Time of token creation
Created time.Time `json:"created"`
// Time of token expiry
Expiry time.Time `json:"expiry"`
}
+1 -1
View File
@@ -41,7 +41,7 @@ type Subscriber interface {
var (
// DefaultBroker is the default Broker.
DefaultBroker = NewBroker()
DefaultBroker = NewHttpBroker()
)
func Init(opts ...Option) error {
+4 -7
View File
@@ -1,4 +1,3 @@
// Package broker provides a http based message broker
package broker
import (
@@ -75,14 +74,12 @@ var (
)
func init() {
rand.Seed(time.Now().Unix())
}
func newTransport(config *tls.Config) *http.Transport {
if config == nil {
config = &tls.Config{
InsecureSkipVerify: true,
}
// Use environment-based config - secure by default
config = mls.Config()
}
dialTLS := func(network string, addr string) (net.Conn, error) {
@@ -705,7 +702,7 @@ func (h *httpBroker) String() string {
return "http"
}
// NewBroker returns a new http broker.
func NewBroker(opts ...Option) Broker {
// NewHttpBroker returns a new http broker.
func NewHttpBroker(opts ...Option) Broker {
return newHttpBroker(opts...)
}
+5 -21
View File
@@ -60,7 +60,7 @@ func sub(b *testing.B, c int) {
b.StopTimer()
m := newTestRegistry()
brker := broker.NewBroker(broker.Registry(m))
brker := broker.NewHttpBroker(broker.Registry(m))
topic := uuid.New().String()
if err := brker.Init(); err != nil {
@@ -121,7 +121,7 @@ func sub(b *testing.B, c int) {
func pub(b *testing.B, c int) {
b.StopTimer()
m := newTestRegistry()
brk := broker.NewBroker(broker.Registry(m))
brk := broker.NewHttpBroker(broker.Registry(m))
topic := uuid.New().String()
if err := brk.Init(); err != nil {
@@ -190,7 +190,7 @@ func pub(b *testing.B, c int) {
func TestBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -239,7 +239,7 @@ func TestBroker(t *testing.T) {
func TestConcurrentSubBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -298,7 +298,7 @@ func TestConcurrentSubBroker(t *testing.T) {
func TestConcurrentPubBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -362,14 +362,6 @@ func BenchmarkSub32(b *testing.B) {
sub(b, 32)
}
func BenchmarkSub64(b *testing.B) {
sub(b, 64)
}
func BenchmarkSub128(b *testing.B) {
sub(b, 128)
}
func BenchmarkPub1(b *testing.B) {
pub(b, 1)
}
@@ -381,11 +373,3 @@ func BenchmarkPub8(b *testing.B) {
func BenchmarkPub32(b *testing.B) {
pub(b, 32)
}
func BenchmarkPub64(b *testing.B) {
pub(b, 64)
}
func BenchmarkPub128(b *testing.B) {
pub(b, 128)
}
-2
View File
@@ -5,7 +5,6 @@ import (
"errors"
"math/rand"
"sync"
"time"
"github.com/google/uuid"
log "go-micro.dev/v5/logger"
@@ -223,7 +222,6 @@ func (m *memorySubscriber) Unsubscribe() error {
func NewMemoryBroker(opts ...Option) Broker {
options := NewOptions(opts...)
rand.Seed(time.Now().UnixNano())
return &memoryBroker{
opts: options,
+17
View File
@@ -0,0 +1,17 @@
package nats
import (
"context"
"go-micro.dev/v5/broker"
)
// setBrokerOption returns a function to setup a context with given value.
func setBrokerOption(k, v interface{}) broker.Option {
return func(o *broker.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
+428
View File
@@ -0,0 +1,428 @@
// Package nats provides a NATS broker
package nats
import (
"context"
"errors"
"strings"
"sync"
"time"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/codec/json"
"go-micro.dev/v5/logger"
"go-micro.dev/v5/registry"
)
type natsBroker struct {
sync.Once
sync.RWMutex
// indicate if we're connected
connected bool
addrs []string
conn *natsp.Conn // single connection (used when pool is disabled)
pool *connectionPool // connection pool (used when pooling is enabled)
opts broker.Options
nopts natsp.Options
// pool configuration
poolSize int
poolIdleTimeout time.Duration
// should we drain the connection
drain bool
closeCh chan (error)
}
type subscriber struct {
s *natsp.Subscription
opts broker.SubscribeOptions
}
type publication struct {
t string
err error
m *broker.Message
}
func (p *publication) Topic() string {
return p.t
}
func (p *publication) Message() *broker.Message {
return p.m
}
func (p *publication) Ack() error {
// nats does not support acking
return nil
}
func (p *publication) Error() error {
return p.err
}
func (s *subscriber) Options() broker.SubscribeOptions {
return s.opts
}
func (s *subscriber) Topic() string {
return s.s.Subject
}
func (s *subscriber) Unsubscribe() error {
return s.s.Unsubscribe()
}
func (n *natsBroker) Address() string {
if n.conn != nil && n.conn.IsConnected() {
return n.conn.ConnectedUrl()
}
if len(n.addrs) > 0 {
return n.addrs[0]
}
return ""
}
func (n *natsBroker) setAddrs(addrs []string) []string {
//nolint:prealloc
var cAddrs []string
for _, addr := range addrs {
if len(addr) == 0 {
continue
}
if !strings.HasPrefix(addr, "nats://") {
addr = "nats://" + addr
}
cAddrs = append(cAddrs, addr)
}
if len(cAddrs) == 0 {
cAddrs = []string{natsp.DefaultURL}
}
return cAddrs
}
func (n *natsBroker) Connect() error {
n.Lock()
defer n.Unlock()
if n.connected {
return nil
}
// Check if we should use connection pooling
if n.poolSize > 1 {
// Initialize connection pool
factory := func() (*natsp.Conn, error) {
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
return opts.Connect()
}
pool, err := newConnectionPool(n.poolSize, factory)
if err != nil {
return err
}
// Set idle timeout if configured
if n.poolIdleTimeout > 0 {
pool.idleTimeout = n.poolIdleTimeout
}
n.pool = pool
n.connected = true
return nil
}
// Single connection mode (original behavior)
status := natsp.CLOSED
if n.conn != nil {
status = n.conn.Status()
}
switch status {
case natsp.CONNECTED, natsp.RECONNECTING, natsp.CONNECTING:
n.connected = true
return nil
default: // DISCONNECTED or CLOSED or DRAINING
opts := n.nopts
opts.Servers = n.addrs
opts.Secure = n.opts.Secure
opts.TLSConfig = n.opts.TLSConfig
// secure might not be set
if n.opts.TLSConfig != nil {
opts.Secure = true
}
c, err := opts.Connect()
if err != nil {
return err
}
n.conn = c
n.connected = true
return nil
}
}
func (n *natsBroker) Disconnect() error {
n.Lock()
defer n.Unlock()
// Close connection pool if it exists
if n.pool != nil {
if err := n.pool.Close(); err != nil {
n.opts.Logger.Log(logger.ErrorLevel, "error closing connection pool:", err)
}
n.pool = nil
}
// Close single connection if it exists
if n.conn != nil {
// drain the connection if specified
if n.drain {
n.conn.Drain()
n.closeCh <- nil
}
// close the client connection
n.conn.Close()
n.conn = nil
}
// set not connected
n.connected = false
return nil
}
func (n *natsBroker) Init(opts ...broker.Option) error {
n.setOption(opts...)
return nil
}
func (n *natsBroker) Options() broker.Options {
return n.opts
}
func (n *natsBroker) Publish(topic string, msg *broker.Message, opts ...broker.PublishOption) error {
n.RLock()
defer n.RUnlock()
b, err := n.opts.Codec.Marshal(msg)
if err != nil {
return err
}
// Use connection pool if enabled
if n.pool != nil {
poolConn, err := n.pool.Get()
if err != nil {
return err
}
defer n.pool.Put(poolConn)
conn := poolConn.Conn()
if conn == nil {
return errors.New("invalid connection from pool")
}
return conn.Publish(topic, b)
}
// Use single connection (original behavior)
if n.conn == nil {
return errors.New("not connected")
}
return n.conn.Publish(topic, b)
}
func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
n.RLock()
hasConnection := n.conn != nil || n.pool != nil
n.RUnlock()
if !hasConnection {
return nil, errors.New("not connected")
}
opt := broker.SubscribeOptions{
AutoAck: true,
Context: context.Background(),
}
for _, o := range opts {
o(&opt)
}
fn := func(msg *natsp.Msg) {
var m broker.Message
pub := &publication{t: msg.Subject}
eh := n.opts.ErrorHandler
err := n.opts.Codec.Unmarshal(msg.Data, &m)
pub.err = err
pub.m = &m
if err != nil {
m.Body = msg.Data
n.opts.Logger.Log(logger.ErrorLevel, err)
if eh != nil {
eh(pub)
}
return
}
if err := handler(pub); err != nil {
pub.err = err
n.opts.Logger.Log(logger.ErrorLevel, err)
if eh != nil {
eh(pub)
}
}
}
var sub *natsp.Subscription
var err error
// Use connection pool if enabled
if n.pool != nil {
poolConn, err := n.pool.Get()
if err != nil {
return nil, err
}
conn := poolConn.Conn()
if conn == nil {
n.pool.Put(poolConn)
return nil, errors.New("invalid connection from pool")
}
if len(opt.Queue) > 0 {
sub, err = conn.QueueSubscribe(topic, opt.Queue, fn)
} else {
sub, err = conn.Subscribe(topic, fn)
}
if err != nil {
n.pool.Put(poolConn)
return nil, err
}
// Return connection to pool after subscription is created
// The subscription keeps the connection alive
n.pool.Put(poolConn)
return &subscriber{s: sub, opts: opt}, nil
}
// Use single connection (original behavior)
n.RLock()
if len(opt.Queue) > 0 {
sub, err = n.conn.QueueSubscribe(topic, opt.Queue, fn)
} else {
sub, err = n.conn.Subscribe(topic, fn)
}
n.RUnlock()
if err != nil {
return nil, err
}
return &subscriber{s: sub, opts: opt}, nil
}
func (n *natsBroker) String() string {
return "nats"
}
func (n *natsBroker) setOption(opts ...broker.Option) {
for _, o := range opts {
o(&n.opts)
}
n.Once.Do(func() {
n.nopts = natsp.GetDefaultOptions()
n.poolSize = 1 // Default to single connection (no pooling)
n.poolIdleTimeout = 5 * time.Minute
})
if nopts, ok := n.opts.Context.Value(optionsKey{}).(natsp.Options); ok {
n.nopts = nopts
}
// Set pool size if configured
if poolSize, ok := n.opts.Context.Value(poolSizeKey{}).(int); ok && poolSize > 0 {
n.poolSize = poolSize
}
// Set pool idle timeout if configured
if idleTimeout, ok := n.opts.Context.Value(poolIdleTimeoutKey{}).(time.Duration); ok {
n.poolIdleTimeout = idleTimeout
}
// broker.Options have higher priority than nats.Options
// only if Addrs, Secure or TLSConfig were not set through a broker.Option
// we read them from nats.Option
if len(n.opts.Addrs) == 0 {
n.opts.Addrs = n.nopts.Servers
}
if !n.opts.Secure {
n.opts.Secure = n.nopts.Secure
}
if n.opts.TLSConfig == nil {
n.opts.TLSConfig = n.nopts.TLSConfig
}
n.addrs = n.setAddrs(n.opts.Addrs)
if n.opts.Context.Value(drainConnectionKey{}) != nil {
n.drain = true
n.closeCh = make(chan error)
n.nopts.ClosedCB = n.onClose
n.nopts.AsyncErrorCB = n.onAsyncError
n.nopts.DisconnectedErrCB = n.onDisconnectedError
}
}
func (n *natsBroker) onClose(conn *natsp.Conn) {
n.closeCh <- nil
}
func (n *natsBroker) onAsyncError(conn *natsp.Conn, sub *natsp.Subscription, err error) {
// There are kinds of different async error nats might callback, but we are interested
// in ErrDrainTimeout only here.
if err == natsp.ErrDrainTimeout {
n.closeCh <- err
}
}
func (n *natsBroker) onDisconnectedError(conn *natsp.Conn, err error) {
n.closeCh <- err
}
func NewNatsBroker(opts ...broker.Option) broker.Broker {
options := broker.Options{
// Default codec
Codec: json.Marshaler{},
Context: context.Background(),
Registry: registry.DefaultRegistry,
Logger: logger.DefaultLogger,
}
n := &natsBroker{
opts: options,
}
n.setOption(opts...)
return n
}
+96
View File
@@ -0,0 +1,96 @@
package nats
import (
"fmt"
"testing"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
)
var addrTestCases = []struct {
name string
description string
addrs map[string]string // expected address : set address
}{
{
"brokerOpts",
"set broker addresses through a broker.Option in constructor",
map[string]string{
"nats://192.168.10.1:5222": "192.168.10.1:5222",
"nats://10.20.10.0:4222": "10.20.10.0:4222"},
},
{
"brokerInit",
"set broker addresses through a broker.Option in broker.Init()",
map[string]string{
"nats://192.168.10.1:5222": "192.168.10.1:5222",
"nats://10.20.10.0:4222": "10.20.10.0:4222"},
},
{
"natsOpts",
"set broker addresses through the nats.Option in constructor",
map[string]string{
"nats://192.168.10.1:5222": "192.168.10.1:5222",
"nats://10.20.10.0:4222": "10.20.10.0:4222"},
},
{
"default",
"check if default Address is set correctly",
map[string]string{
"nats://127.0.0.1:4222": "",
},
},
}
// TestInitAddrs tests issue #100. Ensures that if the addrs is set by an option in init it will be used.
func TestInitAddrs(t *testing.T) {
for _, tc := range addrTestCases {
t.Run(fmt.Sprintf("%s: %s", tc.name, tc.description), func(t *testing.T) {
var br broker.Broker
var addrs []string
for _, addr := range tc.addrs {
addrs = append(addrs, addr)
}
switch tc.name {
case "brokerOpts":
// we know that there are just two addrs in the dict
br = NewNatsBroker(broker.Addrs(addrs[0], addrs[1]))
br.Init()
case "brokerInit":
br = NewNatsBroker()
// we know that there are just two addrs in the dict
br.Init(broker.Addrs(addrs[0], addrs[1]))
case "natsOpts":
nopts := natsp.GetDefaultOptions()
nopts.Servers = addrs
br = NewNatsBroker(Options(nopts))
br.Init()
case "default":
br = NewNatsBroker()
br.Init()
}
natsBroker, ok := br.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of types *natsBroker")
}
// check if the same amount of addrs we set has actually been set, default
// have only 1 address nats://127.0.0.1:4222 (current nats code) or
// nats://localhost:4222 (older code version)
if len(natsBroker.addrs) != len(tc.addrs) && tc.name != "default" {
t.Errorf("Expected Addr count = %d, Actual Addr count = %d",
len(natsBroker.addrs), len(tc.addrs))
}
for _, addr := range natsBroker.addrs {
_, ok := tc.addrs[addr]
if !ok {
t.Errorf("Expected '%s' has not been set", addr)
}
}
})
}
}
+37
View File
@@ -0,0 +1,37 @@
package nats
import (
"time"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
)
type optionsKey struct{}
type drainConnectionKey struct{}
type poolSizeKey struct{}
type poolIdleTimeoutKey struct{}
// Options accepts nats.Options.
func Options(opts natsp.Options) broker.Option {
return setBrokerOption(optionsKey{}, opts)
}
// DrainConnection will drain subscription on close.
func DrainConnection() broker.Option {
return setBrokerOption(drainConnectionKey{}, struct{}{})
}
// PoolSize sets the size of the connection pool.
// If set to a value > 1, the broker will use a connection pool.
// Default is 1 (no pooling).
func PoolSize(size int) broker.Option {
return setBrokerOption(poolSizeKey{}, size)
}
// PoolIdleTimeout sets the timeout for idle connections in the pool.
// Connections idle for longer than this duration will be closed.
// Default is 5 minutes. Set to 0 to disable idle timeout.
func PoolIdleTimeout(timeout time.Duration) broker.Option {
return setBrokerOption(poolIdleTimeoutKey{}, timeout)
}
+188
View File
@@ -0,0 +1,188 @@
package nats
import (
"errors"
"sync"
"time"
natsp "github.com/nats-io/nats.go"
)
var (
// ErrPoolExhausted is returned when no connections are available in the pool
ErrPoolExhausted = errors.New("connection pool exhausted")
// ErrPoolClosed is returned when trying to use a closed pool
ErrPoolClosed = errors.New("connection pool is closed")
)
// connectionPool manages a pool of NATS connections
type connectionPool struct {
mu sync.RWMutex
connections chan *pooledConnection
factory func() (*natsp.Conn, error)
size int
idleTimeout time.Duration
closed bool
}
// pooledConnection wraps a NATS connection with metadata
type pooledConnection struct {
conn *natsp.Conn
createdAt time.Time
lastUsed time.Time
mu sync.Mutex
}
// newConnectionPool creates a new connection pool
func newConnectionPool(size int, factory func() (*natsp.Conn, error)) (*connectionPool, error) {
if size <= 0 {
size = 1
}
pool := &connectionPool{
connections: make(chan *pooledConnection, size),
factory: factory,
size: size,
idleTimeout: 5 * time.Minute,
closed: false,
}
return pool, nil
}
// Get retrieves a connection from the pool or creates a new one
func (p *connectionPool) Get() (*pooledConnection, error) {
p.mu.RLock()
if p.closed {
p.mu.RUnlock()
return nil, ErrPoolClosed
}
p.mu.RUnlock()
// Try to get an existing connection from the pool
select {
case conn := <-p.connections:
// Check if connection is still valid and not idle for too long
if conn.isValid() && !conn.isExpired(p.idleTimeout) {
conn.updateLastUsed()
return conn, nil
}
// Connection is invalid or expired, close it and create a new one
conn.close()
return p.createConnection()
default:
// No connection available, create a new one
return p.createConnection()
}
}
// Put returns a connection to the pool
func (p *connectionPool) Put(conn *pooledConnection) error {
p.mu.RLock()
defer p.mu.RUnlock()
if p.closed {
return conn.close()
}
// Check if connection is still valid
if !conn.isValid() {
return conn.close()
}
conn.updateLastUsed()
// Try to return connection to pool
select {
case p.connections <- conn:
return nil
default:
// Pool is full, close the connection
return conn.close()
}
}
// Close closes all connections in the pool
func (p *connectionPool) Close() error {
p.mu.Lock()
defer p.mu.Unlock()
if p.closed {
return nil
}
p.closed = true
close(p.connections)
// Close all connections in the pool
for conn := range p.connections {
conn.close()
}
return nil
}
// createConnection creates a new pooled connection
func (p *connectionPool) createConnection() (*pooledConnection, error) {
conn, err := p.factory()
if err != nil {
return nil, err
}
return &pooledConnection{
conn: conn,
createdAt: time.Now(),
lastUsed: time.Now(),
}, nil
}
// isValid checks if the underlying NATS connection is valid
func (pc *pooledConnection) isValid() bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn == nil {
return false
}
status := pc.conn.Status()
return status == natsp.CONNECTED || status == natsp.RECONNECTING
}
// isExpired checks if the connection has been idle for too long
func (pc *pooledConnection) isExpired(timeout time.Duration) bool {
pc.mu.Lock()
defer pc.mu.Unlock()
if timeout <= 0 {
return false
}
return time.Since(pc.lastUsed) > timeout
}
// close closes the underlying NATS connection
func (pc *pooledConnection) close() error {
pc.mu.Lock()
defer pc.mu.Unlock()
if pc.conn != nil {
pc.conn.Close()
pc.conn = nil
}
return nil
}
// Conn returns the underlying NATS connection
func (pc *pooledConnection) Conn() *natsp.Conn {
pc.mu.Lock()
defer pc.mu.Unlock()
return pc.conn
}
// updateLastUsed updates the last used timestamp in a thread-safe manner
func (pc *pooledConnection) updateLastUsed() {
pc.mu.Lock()
defer pc.mu.Unlock()
pc.lastUsed = time.Now()
}
+204
View File
@@ -0,0 +1,204 @@
package nats
import (
"sync"
"testing"
"time"
natsp "github.com/nats-io/nats.go"
)
func TestConnectionPool_GetPut(t *testing.T) {
// Mock factory that creates connections
connCount := 0
factory := func() (*natsp.Conn, error) {
connCount++
// Return a mock connection (we can't create real NATS connections in tests without a server)
// This test is more about the pool logic
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Get a connection (should create one)
conn1, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
if conn1 == nil {
t.Fatal("Expected connection, got nil")
}
// Put it back
if err := pool.Put(conn1); err != nil {
t.Fatalf("Failed to put connection: %v", err)
}
// Get it again (should reuse the same one)
conn2, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
// Since we can't compare actual connections easily, just verify we got one
if conn2 == nil {
t.Fatal("Expected connection, got nil")
}
}
func TestConnectionPool_Concurrent(t *testing.T) {
connCount := 0
mu := sync.Mutex{}
factory := func() (*natsp.Conn, error) {
mu.Lock()
connCount++
mu.Unlock()
return nil, nil
}
pool, err := newConnectionPool(5, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
defer pool.Close()
// Simulate concurrent access
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
conn, err := pool.Get()
if err != nil {
t.Errorf("Failed to get connection: %v", err)
return
}
// Simulate some work
time.Sleep(10 * time.Millisecond)
if err := pool.Put(conn); err != nil {
t.Errorf("Failed to put connection: %v", err)
}
}()
}
wg.Wait()
// We should have created some connections
mu.Lock()
if connCount == 0 {
t.Error("Expected at least one connection to be created")
}
mu.Unlock()
}
func TestConnectionPool_Close(t *testing.T) {
factory := func() (*natsp.Conn, error) {
return nil, nil
}
pool, err := newConnectionPool(3, factory)
if err != nil {
t.Fatalf("Failed to create pool: %v", err)
}
// Get a connection
conn, err := pool.Get()
if err != nil {
t.Fatalf("Failed to get connection: %v", err)
}
// Close the pool
if err := pool.Close(); err != nil {
t.Fatalf("Failed to close pool: %v", err)
}
// Put connection back to closed pool should not panic
// The connection will be closed instead of returned to pool
_ = pool.Put(conn)
// Try to get from closed pool
_, err = pool.Get()
if err != ErrPoolClosed {
t.Errorf("Expected ErrPoolClosed, got: %v", err)
}
}
func TestPooledConnection_IsValid(t *testing.T) {
pc := &pooledConnection{
conn: nil, // nil connection should be invalid
createdAt: time.Now(),
lastUsed: time.Now(),
}
if pc.isValid() {
t.Error("Expected nil connection to be invalid")
}
}
func TestPooledConnection_IsExpired(t *testing.T) {
pc := &pooledConnection{
conn: nil,
createdAt: time.Now(),
lastUsed: time.Now().Add(-10 * time.Minute), // 10 minutes ago
}
// With 5 minute timeout, should be expired
if !pc.isExpired(5 * time.Minute) {
t.Error("Expected connection to be expired")
}
// With 0 timeout, should never expire
if pc.isExpired(0) {
t.Error("Expected connection not to expire with 0 timeout")
}
// With 20 minute timeout, should not be expired
if pc.isExpired(20 * time.Minute) {
t.Error("Expected connection not to be expired")
}
}
func TestNatsBroker_PoolConfiguration(t *testing.T) {
// Test that pool size is set correctly
br := NewNatsBroker(PoolSize(5))
nb, ok := br.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb.poolSize != 5 {
t.Errorf("Expected pool size 5, got %d", nb.poolSize)
}
// Test with custom idle timeout
br2 := NewNatsBroker(PoolSize(3), PoolIdleTimeout(10*time.Minute))
nb2, ok := br2.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb2.poolSize != 3 {
t.Errorf("Expected pool size 3, got %d", nb2.poolSize)
}
if nb2.poolIdleTimeout != 10*time.Minute {
t.Errorf("Expected idle timeout 10m, got %v", nb2.poolIdleTimeout)
}
}
func TestNatsBroker_DefaultSingleConnection(t *testing.T) {
// Test that default behavior is single connection (pool size 1)
br := NewNatsBroker()
nb, ok := br.(*natsBroker)
if !ok {
t.Fatal("Expected broker to be of type *natsBroker")
}
if nb.poolSize != 1 {
t.Errorf("Expected default pool size 1, got %d", nb.poolSize)
}
}
+12
View File
@@ -0,0 +1,12 @@
package rabbitmq
type ExternalAuthentication struct {
}
func (auth *ExternalAuthentication) Mechanism() string {
return "EXTERNAL"
}
func (auth *ExternalAuthentication) Response() string {
return ""
}
+178
View File
@@ -0,0 +1,178 @@
package rabbitmq
//
// All credit to Mondo
//
import (
"errors"
"sync"
"github.com/google/uuid"
amqp "github.com/rabbitmq/amqp091-go"
)
type rabbitMQChannel struct {
uuid string
connection *amqp.Connection
channel *amqp.Channel
confirmPublish chan amqp.Confirmation
mtx sync.Mutex
}
func newRabbitChannel(conn *amqp.Connection, prefetchCount int, prefetchGlobal bool, confirmPublish bool) (*rabbitMQChannel, error) {
id, err := uuid.NewRandom()
if err != nil {
return nil, err
}
rabbitCh := &rabbitMQChannel{
uuid: id.String(),
connection: conn,
}
if err := rabbitCh.Connect(prefetchCount, prefetchGlobal, confirmPublish); err != nil {
return nil, err
}
return rabbitCh, nil
}
func (r *rabbitMQChannel) Connect(prefetchCount int, prefetchGlobal bool, confirmPublish bool) error {
var err error
r.channel, err = r.connection.Channel()
if err != nil {
return err
}
err = r.channel.Qos(prefetchCount, 0, prefetchGlobal)
if err != nil {
return err
}
if confirmPublish {
r.confirmPublish = r.channel.NotifyPublish(make(chan amqp.Confirmation, 1))
err = r.channel.Confirm(false)
if err != nil {
return err
}
}
return nil
}
func (r *rabbitMQChannel) Close() error {
if r.channel == nil {
return errors.New("Channel is nil")
}
return r.channel.Close()
}
func (r *rabbitMQChannel) Publish(exchange, key string, message amqp.Publishing) error {
if r.channel == nil {
return errors.New("Channel is nil")
}
if r.confirmPublish != nil {
r.mtx.Lock()
defer r.mtx.Unlock()
}
err := r.channel.Publish(exchange, key, false, false, message)
if err != nil {
return err
}
if r.confirmPublish != nil {
confirmation, ok := <-r.confirmPublish
if !ok {
return errors.New("Channel closed before could receive confirmation of publish")
}
if !confirmation.Ack {
return errors.New("Could not publish message, received nack from broker on confirmation")
}
}
return nil
}
func (r *rabbitMQChannel) DeclareExchange(ex Exchange) error {
return r.channel.ExchangeDeclare(
ex.Name, // name
string(ex.Type), // kind
ex.Durable, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
)
}
func (r *rabbitMQChannel) DeclareDurableExchange(ex Exchange) error {
return r.channel.ExchangeDeclare(
ex.Name, // name
string(ex.Type), // kind
true, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
)
}
func (r *rabbitMQChannel) DeclareQueue(queue string, args amqp.Table) error {
_, err := r.channel.QueueDeclare(
queue, // name
false, // durable
true, // autoDelete
false, // exclusive
false, // noWait
args, // args
)
return err
}
func (r *rabbitMQChannel) DeclareDurableQueue(queue string, args amqp.Table) error {
_, err := r.channel.QueueDeclare(
queue, // name
true, // durable
false, // autoDelete
false, // exclusive
false, // noWait
args, // args
)
return err
}
func (r *rabbitMQChannel) DeclareReplyQueue(queue string) error {
_, err := r.channel.QueueDeclare(
queue, // name
false, // durable
true, // autoDelete
true, // exclusive
false, // noWait
nil, // args
)
return err
}
func (r *rabbitMQChannel) ConsumeQueue(queue string, autoAck bool) (<-chan amqp.Delivery, error) {
return r.channel.Consume(
queue, // queue
r.uuid, // consumer
autoAck, // autoAck
false, // exclusive
false, // nolocal
false, // nowait
nil, // args
)
}
func (r *rabbitMQChannel) BindQueue(queue, key, exchange string, args amqp.Table) error {
return r.channel.QueueBind(
queue, // name
key, // key
exchange, // exchange
false, // noWait
args, // args
)
}
+299
View File
@@ -0,0 +1,299 @@
package rabbitmq
//
// All credit to Mondo
//
import (
"regexp"
"strings"
"sync"
"time"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/logger"
mtls "go-micro.dev/v5/util/tls"
)
type MQExchangeType string
const (
ExchangeTypeFanout MQExchangeType = "fanout"
ExchangeTypeTopic = "topic"
ExchangeTypeDirect = "direct"
)
var (
DefaultExchange = Exchange{
Name: "micro",
Type: ExchangeTypeTopic,
}
DefaultRabbitURL = "amqp://guest:guest@127.0.0.1:5672"
DefaultPrefetchCount = 0
DefaultPrefetchGlobal = false
DefaultRequeueOnError = false
DefaultConfirmPublish = false
DefaultWithoutExchange = false
// The amqp library does not seem to set these when using amqp.DialConfig
// (even though it says so in the comments) so we set them manually to make
// sure to not brake any existing functionality.
defaultHeartbeat = 10 * time.Second
defaultLocale = "en_US"
defaultAmqpConfig = amqp.Config{
Heartbeat: defaultHeartbeat,
Locale: defaultLocale,
}
dial = amqp.Dial
dialTLS = amqp.DialTLS
dialConfig = amqp.DialConfig
)
type rabbitMQConn struct {
Connection *amqp.Connection
Channel *rabbitMQChannel
ExchangeChannel *rabbitMQChannel
exchange Exchange
withoutExchange bool
url string
prefetchCount int
prefetchGlobal bool
confirmPublish bool
sync.Mutex
connected bool
close chan bool
waitConnection chan struct{}
logger logger.Logger
}
// Exchange is the rabbitmq exchange.
type Exchange struct {
// Name of the exchange
Name string
// Type of the exchange
Type MQExchangeType
// Whether its persistent
Durable bool
}
func newRabbitMQConn(ex Exchange, urls []string, prefetchCount int, prefetchGlobal bool, confirmPublish bool, withoutExchange bool, logger logger.Logger) *rabbitMQConn {
var url string
if len(urls) > 0 && regexp.MustCompile("^amqp(s)?://.*").MatchString(urls[0]) {
url = urls[0]
} else {
url = DefaultRabbitURL
}
ret := &rabbitMQConn{
exchange: ex,
url: url,
withoutExchange: withoutExchange,
prefetchCount: prefetchCount,
prefetchGlobal: prefetchGlobal,
confirmPublish: confirmPublish,
close: make(chan bool),
waitConnection: make(chan struct{}),
logger: logger,
}
// its bad case of nil == waitConnection, so close it at start
close(ret.waitConnection)
return ret
}
func (r *rabbitMQConn) connect(secure bool, config *amqp.Config) error {
// try connect
if err := r.tryConnect(secure, config); err != nil {
return err
}
// connected
r.Lock()
r.connected = true
r.Unlock()
// create reconnect loop
go r.reconnect(secure, config)
return nil
}
func (r *rabbitMQConn) reconnect(secure bool, config *amqp.Config) {
// skip first connect
var connect bool
for {
if connect {
// try reconnect
if err := r.tryConnect(secure, config); err != nil {
time.Sleep(1 * time.Second)
continue
}
// connected
r.Lock()
r.connected = true
r.Unlock()
// unblock resubscribe cycle - close channel
//at this point channel is created and unclosed - close it without any additional checks
close(r.waitConnection)
}
connect = true
notifyClose := make(chan *amqp.Error)
r.Connection.NotifyClose(notifyClose)
chanNotifyClose := make(chan *amqp.Error)
var channel *amqp.Channel
if !r.withoutExchange {
channel = r.ExchangeChannel.channel
} else {
channel = r.Channel.channel
}
channel.NotifyClose(chanNotifyClose)
// To avoid deadlocks it is necessary to consume the messages from all channels.
for notifyClose != nil || chanNotifyClose != nil {
// block until closed
select {
case err := <-chanNotifyClose:
r.logger.Log(logger.ErrorLevel, err)
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
chanNotifyClose = nil
case err := <-notifyClose:
r.logger.Log(logger.ErrorLevel, err)
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
notifyClose = nil
case <-r.close:
return
}
}
}
}
func (r *rabbitMQConn) Connect(secure bool, config *amqp.Config) error {
r.Lock()
// already connected
if r.connected {
r.Unlock()
return nil
}
// check it was closed
select {
case <-r.close:
r.close = make(chan bool)
default:
// no op
// new conn
}
r.Unlock()
return r.connect(secure, config)
}
func (r *rabbitMQConn) Close() error {
r.Lock()
defer r.Unlock()
select {
case <-r.close:
return nil
default:
close(r.close)
r.connected = false
}
return r.Connection.Close()
}
func (r *rabbitMQConn) tryConnect(secure bool, config *amqp.Config) error {
var err error
if config == nil {
config = &defaultAmqpConfig
}
url := r.url
if secure || config.TLSClientConfig != nil || strings.HasPrefix(r.url, "amqps://") {
if config.TLSClientConfig == nil {
// Use environment-based config - secure by default
config.TLSClientConfig = mtls.Config()
}
url = strings.Replace(r.url, "amqp://", "amqps://", 1)
}
r.Connection, err = dialConfig(url, *config)
if err != nil {
return err
}
if r.Channel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish); err != nil {
return err
}
if !r.withoutExchange {
if r.exchange.Durable {
r.Channel.DeclareDurableExchange(r.exchange)
} else {
r.Channel.DeclareExchange(r.exchange)
}
r.ExchangeChannel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish)
}
return err
}
func (r *rabbitMQConn) Consume(queue, key string, headers amqp.Table, qArgs amqp.Table, autoAck, durableQueue bool) (*rabbitMQChannel, <-chan amqp.Delivery, error) {
consumerChannel, err := newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish)
if err != nil {
return nil, nil, err
}
if durableQueue {
err = consumerChannel.DeclareDurableQueue(queue, qArgs)
} else {
err = consumerChannel.DeclareQueue(queue, qArgs)
}
if err != nil {
return nil, nil, err
}
deliveries, err := consumerChannel.ConsumeQueue(queue, autoAck)
if err != nil {
return nil, nil, err
}
if !r.withoutExchange {
err = consumerChannel.BindQueue(queue, key, r.exchange.Name, headers)
if err != nil {
return nil, nil, err
}
}
return consumerChannel, deliveries, nil
}
func (r *rabbitMQConn) Publish(exchange, key string, msg amqp.Publishing) error {
if r.withoutExchange {
return r.Channel.Publish("", key, msg)
}
return r.ExchangeChannel.Publish(exchange, key, msg)
}
+111
View File
@@ -0,0 +1,111 @@
package rabbitmq
import (
"crypto/tls"
"errors"
"testing"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/logger"
)
func TestNewRabbitMQConnURL(t *testing.T) {
testcases := []struct {
title string
urls []string
want string
}{
{"Multiple URLs", []string{"amqp://example.com/one", "amqp://example.com/two"}, "amqp://example.com/one"},
{"Insecure URL", []string{"amqp://example.com"}, "amqp://example.com"},
{"Secure URL", []string{"amqps://example.com"}, "amqps://example.com"},
{"Invalid URL", []string{"http://example.com"}, DefaultRabbitURL},
{"No URLs", []string{}, DefaultRabbitURL},
}
for _, test := range testcases {
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, 0, false, false, false, logger.DefaultLogger)
if have, want := conn.url, test.want; have != want {
t.Errorf("%s: invalid url, want %q, have %q", test.title, want, have)
}
}
}
func TestTryToConnectTLS(t *testing.T) {
var (
dialCount, dialTLSCount int
err = errors.New("stop connect here")
)
dialConfig = func(_ string, c amqp.Config) (*amqp.Connection, error) {
if c.TLSClientConfig != nil {
dialTLSCount++
return nil, err
}
dialCount++
return nil, err
}
testcases := []struct {
title string
url string
secure bool
amqpConfig *amqp.Config
wantTLS bool
}{
{"unsecure url, secure false, no tls config", "amqp://example.com", false, nil, false},
{"secure url, secure false, no tls config", "amqps://example.com", false, nil, true},
{"unsecure url, secure true, no tls config", "amqp://example.com", true, nil, true},
{"unsecure url, secure false, tls config", "amqp://example.com", false, &amqp.Config{TLSClientConfig: &tls.Config{}}, true},
}
for _, test := range testcases {
dialCount, dialTLSCount = 0, 0
conn := newRabbitMQConn(Exchange{Name: "exchange"}, []string{test.url}, 0, false, false, false, logger.DefaultLogger)
conn.tryConnect(test.secure, test.amqpConfig)
have := dialCount
if test.wantTLS {
have = dialTLSCount
}
if have != 1 {
t.Errorf("%s: used wrong dialer, Dial called %d times, DialTLS called %d times", test.title, dialCount, dialTLSCount)
}
}
}
func TestNewRabbitMQPrefetchConfirmPublish(t *testing.T) {
testcases := []struct {
title string
urls []string
prefetchCount int
prefetchGlobal bool
confirmPublish bool
}{
{"Multiple URLs", []string{"amqp://example.com/one", "amqp://example.com/two"}, 1, true, true},
{"Insecure URL", []string{"amqp://example.com"}, 1, true, true},
{"Secure URL", []string{"amqps://example.com"}, 1, true, true},
{"Invalid URL", []string{"http://example.com"}, 1, true, true},
{"No URLs", []string{}, 1, true, true},
}
for _, test := range testcases {
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, test.prefetchCount, test.prefetchGlobal, test.confirmPublish, false, logger.DefaultLogger)
if have, want := conn.prefetchCount, test.prefetchCount; have != want {
t.Errorf("%s: invalid prefetch count, want %d, have %d", test.title, want, have)
}
if have, want := conn.prefetchGlobal, test.prefetchGlobal; have != want {
t.Errorf("%s: invalid prefetch global setting, want %t, have %t", test.title, want, have)
}
if have, want := conn.confirmPublish, test.confirmPublish; have != want {
t.Errorf("%s: invalid confirm setting, want %t, have %t", test.title, want, have)
}
}
}
+48
View File
@@ -0,0 +1,48 @@
package rabbitmq
import (
"context"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/server"
)
// setSubscribeOption returns a function to setup a context with given value.
func setSubscribeOption(k, v interface{}) broker.SubscribeOption {
return func(o *broker.SubscribeOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// setBrokerOption returns a function to setup a context with given value.
func setBrokerOption(k, v interface{}) broker.Option {
return func(o *broker.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// setBrokerOption returns a function to setup a context with given value.
func setServerSubscriberOption(k, v interface{}) server.SubscriberOption {
return func(o *server.SubscriberOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// setPublishOption returns a function to setup a context with given value.
func setPublishOption(k, v interface{}) broker.PublishOption {
return func(o *broker.PublishOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
+189
View File
@@ -0,0 +1,189 @@
package rabbitmq
import (
"context"
"time"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/client"
"go-micro.dev/v5/server"
)
type durableQueueKey struct{}
type headersKey struct{}
type queueArgumentsKey struct{}
type prefetchCountKey struct{}
type prefetchGlobalKey struct{}
type confirmPublishKey struct{}
type exchangeKey struct{}
type exchangeTypeKey struct{}
type withoutExchangeKey struct{}
type requeueOnErrorKey struct{}
type deliveryMode struct{}
type priorityKey struct{}
type contentType struct{}
type contentEncoding struct{}
type correlationID struct{}
type replyTo struct{}
type expiration struct{}
type messageID struct{}
type timestamp struct{}
type typeMsg struct{}
type userID struct{}
type appID struct{}
type externalAuth struct{}
type durableExchange struct{}
// ServerDurableQueue provide durable queue option for micro.RegisterSubscriber
func ServerDurableQueue() server.SubscriberOption {
return setServerSubscriberOption(durableQueueKey{}, true)
}
// ServerAckOnSuccess export AckOnSuccess server.SubscriberOption
func ServerAckOnSuccess() server.SubscriberOption {
return setServerSubscriberOption(ackSuccessKey{}, true)
}
// DurableQueue creates a durable queue when subscribing.
func DurableQueue() broker.SubscribeOption {
return setSubscribeOption(durableQueueKey{}, true)
}
// DurableExchange is an option to set the Exchange to be durable.
func DurableExchange() broker.Option {
return setBrokerOption(durableExchange{}, true)
}
// Headers adds headers used by the headers exchange.
func Headers(h map[string]interface{}) broker.SubscribeOption {
return setSubscribeOption(headersKey{}, h)
}
// QueueArguments sets arguments for queue creation.
func QueueArguments(h map[string]interface{}) broker.SubscribeOption {
return setSubscribeOption(queueArgumentsKey{}, h)
}
func RequeueOnError() broker.SubscribeOption {
return setSubscribeOption(requeueOnErrorKey{}, true)
}
// ExchangeName is an option to set the ExchangeName.
func ExchangeName(e string) broker.Option {
return setBrokerOption(exchangeKey{}, e)
}
// WithoutExchange is an option to use the rabbitmq default exchange.
// means it would not create any custom exchange.
func WithoutExchange() broker.Option {
return setBrokerOption(withoutExchangeKey{}, true)
}
// ExchangeType is an option to set the rabbitmq exchange type.
func ExchangeType(t MQExchangeType) broker.Option {
return setBrokerOption(exchangeTypeKey{}, t)
}
// PrefetchCount ...
func PrefetchCount(c int) broker.Option {
return setBrokerOption(prefetchCountKey{}, c)
}
// PrefetchGlobal creates a durable queue when subscribing.
func PrefetchGlobal() broker.Option {
return setBrokerOption(prefetchGlobalKey{}, true)
}
// ConfirmPublish ensures all published messages are confirmed by waiting for an ack/nack from the broker.
func ConfirmPublish() broker.Option {
return setBrokerOption(confirmPublishKey{}, true)
}
// DeliveryMode sets a delivery mode for publishing.
func DeliveryMode(value uint8) broker.PublishOption {
return setPublishOption(deliveryMode{}, value)
}
// Priority sets a priority level for publishing.
func Priority(value uint8) broker.PublishOption {
return setPublishOption(priorityKey{}, value)
}
// ContentType sets a property MIME content type for publishing.
func ContentType(value string) broker.PublishOption {
return setPublishOption(contentType{}, value)
}
// ContentEncoding sets a property MIME content encoding for publishing.
func ContentEncoding(value string) broker.PublishOption {
return setPublishOption(contentEncoding{}, value)
}
// CorrelationID sets a property correlation ID for publishing.
func CorrelationID(value string) broker.PublishOption {
return setPublishOption(correlationID{}, value)
}
// ReplyTo sets a property address to to reply to (ex: RPC) for publishing.
func ReplyTo(value string) broker.PublishOption {
return setPublishOption(replyTo{}, value)
}
// Expiration sets a property message expiration spec for publishing.
func Expiration(value string) broker.PublishOption {
return setPublishOption(expiration{}, value)
}
// MessageId sets a property message identifier for publishing.
func MessageId(value string) broker.PublishOption {
return setPublishOption(messageID{}, value)
}
// Timestamp sets a property message timestamp for publishing.
func Timestamp(value time.Time) broker.PublishOption {
return setPublishOption(timestamp{}, value)
}
// TypeMsg sets a property message type name for publishing.
func TypeMsg(value string) broker.PublishOption {
return setPublishOption(typeMsg{}, value)
}
// UserID sets a property user id for publishing.
func UserID(value string) broker.PublishOption {
return setPublishOption(userID{}, value)
}
// AppID sets a property application id for publishing.
func AppID(value string) broker.PublishOption {
return setPublishOption(appID{}, value)
}
func ExternalAuth() broker.Option {
return setBrokerOption(externalAuth{}, ExternalAuthentication{})
}
type subscribeContextKey struct{}
// SubscribeContext set the context for broker.SubscribeOption.
func SubscribeContext(ctx context.Context) broker.SubscribeOption {
return setSubscribeOption(subscribeContextKey{}, ctx)
}
type ackSuccessKey struct{}
// AckOnSuccess will automatically acknowledge messages when no error is returned.
func AckOnSuccess() broker.SubscribeOption {
return setSubscribeOption(ackSuccessKey{}, true)
}
// PublishDeliveryMode client.PublishOption for setting message "delivery mode"
// mode , Transient (0 or 1) or Persistent (2)
func PublishDeliveryMode(mode uint8) client.PublishOption {
return func(o *client.PublishOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, deliveryMode{}, mode)
}
}
+443
View File
@@ -0,0 +1,443 @@
// Package rabbitmq provides a RabbitMQ broker
package rabbitmq
import (
"context"
"errors"
"fmt"
"net/url"
"sync"
"time"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/logger"
)
type rbroker struct {
conn *rabbitMQConn
addrs []string
opts broker.Options
prefetchCount int
prefetchGlobal bool
mtx sync.Mutex
wg sync.WaitGroup
}
type subscriber struct {
mtx sync.Mutex
unsub chan bool
opts broker.SubscribeOptions
topic string
ch *rabbitMQChannel
durableQueue bool
queueArgs map[string]interface{}
r *rbroker
fn func(msg amqp.Delivery)
headers map[string]interface{}
wg sync.WaitGroup
}
type publication struct {
d amqp.Delivery
m *broker.Message
t string
err error
}
func (p *publication) Ack() error {
return p.d.Ack(false)
}
func (p *publication) Error() error {
return p.err
}
func (p *publication) Topic() string {
return p.t
}
func (p *publication) Message() *broker.Message {
return p.m
}
func (s *subscriber) Options() broker.SubscribeOptions {
return s.opts
}
func (s *subscriber) Topic() string {
return s.topic
}
func (s *subscriber) Unsubscribe() error {
s.unsub <- true
// Need to wait on subscriber to exit if autoack is disabled
// since closing the channel will prevent the ack/nack from
// being sent upon handler completion.
if !s.opts.AutoAck {
s.wg.Wait()
}
s.mtx.Lock()
defer s.mtx.Unlock()
if s.ch != nil {
return s.ch.Close()
}
return nil
}
func (s *subscriber) resubscribe() {
s.wg.Add(1)
defer s.wg.Done()
minResubscribeDelay := 100 * time.Millisecond
maxResubscribeDelay := 30 * time.Second
expFactor := time.Duration(2)
reSubscribeDelay := minResubscribeDelay
// loop until unsubscribe
for {
select {
// unsubscribe case
case <-s.unsub:
return
// check shutdown case
case <-s.r.conn.close:
// yep, its shutdown case
return
// wait until we reconect to rabbit
case <-s.r.conn.waitConnection:
// When the connection is disconnected, the waitConnection will be re-assigned, so '<-s.r.conn.waitConnection' maybe blocked.
// Here, it returns once a second, and then the latest waitconnection will be used
case <-time.After(time.Second):
continue
}
// it may crash (panic) in case of Consume without connection, so recheck it
s.r.mtx.Lock()
if !s.r.conn.connected {
s.r.mtx.Unlock()
continue
}
ch, sub, err := s.r.conn.Consume(
s.opts.Queue,
s.topic,
s.headers,
s.queueArgs,
s.opts.AutoAck,
s.durableQueue,
)
s.r.mtx.Unlock()
switch err {
case nil:
reSubscribeDelay = minResubscribeDelay
s.mtx.Lock()
s.ch = ch
s.mtx.Unlock()
default:
if reSubscribeDelay > maxResubscribeDelay {
reSubscribeDelay = maxResubscribeDelay
}
time.Sleep(reSubscribeDelay)
reSubscribeDelay *= expFactor
continue
}
SubLoop:
for {
select {
case <-s.unsub:
return
case d, ok := <-sub:
if !ok {
break SubLoop
}
s.r.wg.Add(1)
s.fn(d)
s.r.wg.Done()
}
}
}
}
func (r *rbroker) Publish(topic string, msg *broker.Message, opts ...broker.PublishOption) error {
m := amqp.Publishing{
Body: msg.Body,
Headers: amqp.Table{},
}
options := broker.PublishOptions{}
for _, o := range opts {
o(&options)
}
if options.Context != nil {
if value, ok := options.Context.Value(deliveryMode{}).(uint8); ok {
m.DeliveryMode = value
}
if value, ok := options.Context.Value(priorityKey{}).(uint8); ok {
m.Priority = value
}
if value, ok := options.Context.Value(contentType{}).(string); ok {
m.Headers["Content-Type"] = value
m.ContentType = value
}
if value, ok := options.Context.Value(contentEncoding{}).(string); ok {
m.ContentEncoding = value
}
if value, ok := options.Context.Value(correlationID{}).(string); ok {
m.CorrelationId = value
}
if value, ok := options.Context.Value(replyTo{}).(string); ok {
m.ReplyTo = value
}
if value, ok := options.Context.Value(expiration{}).(string); ok {
m.Expiration = value
}
if value, ok := options.Context.Value(messageID{}).(string); ok {
m.MessageId = value
}
if value, ok := options.Context.Value(timestamp{}).(time.Time); ok {
m.Timestamp = value
}
if value, ok := options.Context.Value(typeMsg{}).(string); ok {
m.Type = value
}
if value, ok := options.Context.Value(userID{}).(string); ok {
m.UserId = value
}
if value, ok := options.Context.Value(appID{}).(string); ok {
m.AppId = value
}
}
for k, v := range msg.Header {
m.Headers[k] = v
}
if r.getWithoutExchange() {
m.Headers["Micro-Topic"] = topic
}
if r.conn == nil {
return errors.New("connection is nil")
}
return r.conn.Publish(r.conn.exchange.Name, topic, m)
}
func (r *rbroker) Subscribe(topic string, handler broker.Handler, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
var ackSuccess bool
if r.conn == nil {
return nil, errors.New("not connected")
}
opt := broker.SubscribeOptions{
AutoAck: true,
}
for _, o := range opts {
o(&opt)
}
// Make sure context is setup
if opt.Context == nil {
opt.Context = context.Background()
}
ctx := opt.Context
if subscribeContext, ok := ctx.Value(subscribeContextKey{}).(context.Context); ok && subscribeContext != nil {
ctx = subscribeContext
}
var requeueOnError bool
requeueOnError, _ = ctx.Value(requeueOnErrorKey{}).(bool)
var durableQueue bool
durableQueue, _ = ctx.Value(durableQueueKey{}).(bool)
var qArgs map[string]interface{}
if qa, ok := ctx.Value(queueArgumentsKey{}).(map[string]interface{}); ok {
qArgs = qa
}
var headers map[string]interface{}
if h, ok := ctx.Value(headersKey{}).(map[string]interface{}); ok {
headers = h
}
if bval, ok := ctx.Value(ackSuccessKey{}).(bool); ok && bval {
opt.AutoAck = false
ackSuccess = true
}
fn := func(msg amqp.Delivery) {
header := make(map[string]string)
for k, v := range msg.Headers {
header[k] = fmt.Sprintf("%v", v)
}
// Get rid of dependence on 'Micro-Topic'
msgTopic := header["Micro-Topic"]
if msgTopic == "" {
header["Micro-Topic"] = msg.RoutingKey
}
m := &broker.Message{
Header: header,
Body: msg.Body,
}
p := &publication{d: msg, m: m, t: msg.RoutingKey}
p.err = handler(p)
if p.err == nil && ackSuccess && !opt.AutoAck {
msg.Ack(false)
} else if p.err != nil && !opt.AutoAck {
msg.Nack(false, requeueOnError)
}
}
sret := &subscriber{topic: topic, opts: opt, unsub: make(chan bool), r: r,
durableQueue: durableQueue, fn: fn, headers: headers, queueArgs: qArgs,
wg: sync.WaitGroup{}}
go sret.resubscribe()
return sret, nil
}
func (r *rbroker) Options() broker.Options {
return r.opts
}
func (r *rbroker) String() string {
return "rabbitmq"
}
func (r *rbroker) Address() string {
if len(r.addrs) > 0 {
u, err := url.Parse(r.addrs[0])
if err != nil {
return ""
}
return u.Redacted()
}
return ""
}
func (r *rbroker) Init(opts ...broker.Option) error {
for _, o := range opts {
o(&r.opts)
}
r.addrs = r.opts.Addrs
return nil
}
func (r *rbroker) Connect() error {
if r.conn == nil {
r.conn = newRabbitMQConn(
r.getExchange(),
r.opts.Addrs,
r.getPrefetchCount(),
r.getPrefetchGlobal(),
r.getConfirmPublish(),
r.getWithoutExchange(),
r.opts.Logger,
)
}
conf := defaultAmqpConfig
if auth, ok := r.opts.Context.Value(externalAuth{}).(ExternalAuthentication); ok {
conf.SASL = []amqp.Authentication{&auth}
}
conf.TLSClientConfig = r.opts.TLSConfig
return r.conn.Connect(r.opts.Secure, &conf)
}
func (r *rbroker) Disconnect() error {
if r.conn == nil {
return errors.New("connection is nil")
}
ret := r.conn.Close()
r.wg.Wait() // wait all goroutines
return ret
}
func NewBroker(opts ...broker.Option) broker.Broker {
options := broker.Options{
Context: context.Background(),
Logger: logger.DefaultLogger,
}
for _, o := range opts {
o(&options)
}
return &rbroker{
addrs: options.Addrs,
opts: options,
}
}
func (r *rbroker) getExchange() Exchange {
ex := DefaultExchange
if e, ok := r.opts.Context.Value(exchangeKey{}).(string); ok {
ex.Name = e
}
if t, ok := r.opts.Context.Value(exchangeTypeKey{}).(MQExchangeType); ok {
ex.Type = t
}
if d, ok := r.opts.Context.Value(durableExchange{}).(bool); ok {
ex.Durable = d
}
return ex
}
func (r *rbroker) getPrefetchCount() int {
if e, ok := r.opts.Context.Value(prefetchCountKey{}).(int); ok {
return e
}
return DefaultPrefetchCount
}
func (r *rbroker) getPrefetchGlobal() bool {
if e, ok := r.opts.Context.Value(prefetchGlobalKey{}).(bool); ok {
return e
}
return DefaultPrefetchGlobal
}
func (r *rbroker) getConfirmPublish() bool {
if e, ok := r.opts.Context.Value(confirmPublishKey{}).(bool); ok {
return e
}
return DefaultConfirmPublish
}
func (r *rbroker) getWithoutExchange() bool {
if e, ok := r.opts.Context.Value(withoutExchangeKey{}).(bool); ok {
return e
}
return DefaultWithoutExchange
}
+305
View File
@@ -0,0 +1,305 @@
package rabbitmq_test
import (
"context"
"encoding/json"
"os"
"testing"
"time"
"go-micro.dev/v5/logger"
micro "go-micro.dev/v5"
broker "go-micro.dev/v5/broker"
rabbitmq "go-micro.dev/v5/broker/rabbitmq"
server "go-micro.dev/v5/server"
)
type Example struct{}
func init() {
rabbitmq.DefaultRabbitURL = "amqp://rabbitmq:rabbitmq@127.0.0.1:5672"
}
type TestEvent struct {
Name string `json:"name"`
Age int `json:"age"`
Time time.Time `json:"time"`
}
func (e *Example) Handler(ctx context.Context, r interface{}) error {
return nil
}
func TestDurable(t *testing.T) {
if tr := os.Getenv("TRAVIS"); len(tr) > 0 {
t.Skip()
}
brkrSub := broker.NewSubscribeOptions(
broker.Queue("queue.default"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
b := rabbitmq.NewBroker()
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
h := &Example{}
// Register a subscriber
micro.RegisterSubscriber(
"topic",
service.Server(),
h.Handler,
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("queue.default"),
)
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestWithoutExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.WithoutExchange())
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("direct.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"direct.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("direct.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("direct.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestFanoutExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.ExchangeType(rabbitmq.ExchangeTypeFanout), rabbitmq.ExchangeName("fanout.test"))
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("fanout.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"fanout.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("fanout.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("fanout.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestDirectExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.ExchangeType(rabbitmq.ExchangeTypeDirect), rabbitmq.ExchangeName("direct.test"))
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("direct.exchange.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"direct.exchange.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("direct.exchange.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("direct.exchange.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestTopicExchange(t *testing.T) {
b := rabbitmq.NewBroker()
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("topic.exchange.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"my-test-topic",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("topic.exchange.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("my-test-topic", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
+48
View File
@@ -0,0 +1,48 @@
package redis
import (
"context"
rclient "github.com/go-redis/redis/v8"
"go-micro.dev/v5/cache"
)
type redisOptionsContextKey struct{}
// WithRedisOptions sets advanced options for redis.
func WithRedisOptions(options rclient.UniversalOptions) cache.Option {
return func(o *cache.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, redisOptionsContextKey{}, options)
}
}
func newUniversalClient(options cache.Options) rclient.UniversalClient {
if options.Context == nil {
options.Context = context.Background()
}
opts, ok := options.Context.Value(redisOptionsContextKey{}).(rclient.UniversalOptions)
if !ok {
addr := "redis://127.0.0.1:6379"
if len(options.Address) > 0 {
addr = options.Address
}
redisOptions, err := rclient.ParseURL(addr)
if err != nil {
redisOptions = &rclient.Options{Addr: addr}
}
return rclient.NewClient(redisOptions)
}
if len(opts.Addrs) == 0 && len(options.Address) > 0 {
opts.Addrs = []string{options.Address}
}
return rclient.NewUniversalClient(&opts)
}
+139
View File
@@ -0,0 +1,139 @@
package redis
import (
"context"
"reflect"
"testing"
rclient "github.com/go-redis/redis/v8"
"go-micro.dev/v5/cache"
)
func Test_newUniversalClient(t *testing.T) {
type fields struct {
options cache.Options
}
type wantValues struct {
username string
password string
address string
}
tests := []struct {
name string
fields fields
want wantValues
}{
{name: "No Url", fields: fields{options: cache.Options{}},
want: wantValues{
username: "",
password: "",
address: "127.0.0.1:6379",
}},
{name: "legacy Url", fields: fields{options: cache.Options{Address: "127.0.0.1:6379"}},
want: wantValues{
username: "",
password: "",
address: "127.0.0.1:6379",
}},
{name: "New Url", fields: fields{options: cache.Options{Address: "redis://127.0.0.1:6379"}},
want: wantValues{
username: "",
password: "",
address: "127.0.0.1:6379",
}},
{name: "Url with Pwd", fields: fields{options: cache.Options{Address: "redis://:password@redis:6379"}},
want: wantValues{
username: "",
password: "password",
address: "redis:6379",
}},
{name: "Url with username and Pwd", fields: fields{
options: cache.Options{Address: "redis://username:password@redis:6379"}},
want: wantValues{
username: "username",
password: "password",
address: "redis:6379",
}},
{name: "Sentinel Failover client", fields: fields{
options: cache.Options{
Context: context.WithValue(
context.TODO(), redisOptionsContextKey{},
rclient.UniversalOptions{MasterName: "master-name"}),
}},
want: wantValues{
username: "",
password: "",
address: "FailoverClient", // <- Placeholder set by NewFailoverClient
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
univClient := newUniversalClient(tt.fields.options)
client, ok := univClient.(*rclient.Client)
if !ok {
t.Errorf("newUniversalClient() expect a *redis.Client")
return
}
if client.Options().Addr != tt.want.address {
t.Errorf("newUniversalClient() Address = %v, want address %v", client.Options().Addr, tt.want.address)
}
if client.Options().Password != tt.want.password {
t.Errorf("newUniversalClient() password = %v, want password %v", client.Options().Password, tt.want.password)
}
if client.Options().Username != tt.want.username {
t.Errorf("newUniversalClient() username = %v, want username %v", client.Options().Username, tt.want.username)
}
})
}
}
func Test_newUniversalClientCluster(t *testing.T) {
type fields struct {
options cache.Options
}
type wantValues struct {
username string
password string
addrs []string
}
tests := []struct {
name string
fields fields
want wantValues
}{
{name: "Addrs in redis options", fields: fields{
options: cache.Options{
Address: "127.0.0.1:6379", // <- ignored
Context: context.WithValue(
context.TODO(), redisOptionsContextKey{},
rclient.UniversalOptions{Addrs: []string{"127.0.0.1:6381", "127.0.0.1:6382"}}),
}},
want: wantValues{
username: "",
password: "",
addrs: []string{"127.0.0.1:6381", "127.0.0.1:6382"},
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
univClient := newUniversalClient(tt.fields.options)
client, ok := univClient.(*rclient.ClusterClient)
if !ok {
t.Errorf("newUniversalClient() expect a *redis.ClusterClient")
return
}
if !reflect.DeepEqual(client.Options().Addrs, tt.want.addrs) {
t.Errorf("newUniversalClient() Addrs = %v, want addrs %v", client.Options().Addrs, tt.want.addrs)
}
if client.Options().Password != tt.want.password {
t.Errorf("newUniversalClient() password = %v, want password %v", client.Options().Password, tt.want.password)
}
if client.Options().Username != tt.want.username {
t.Errorf("newUniversalClient() username = %v, want username %v", client.Options().Username, tt.want.username)
}
})
}
}
+57
View File
@@ -0,0 +1,57 @@
package redis
import (
"context"
"time"
rclient "github.com/go-redis/redis/v8"
"go-micro.dev/v5/cache"
)
// NewRedisCache returns a new redis cache.
func NewRedisCache(opts ...cache.Option) cache.Cache {
options := cache.NewOptions(opts...)
return &redisCache{
opts: options,
client: newUniversalClient(options),
}
}
type redisCache struct {
opts cache.Options
client rclient.UniversalClient
}
func (c *redisCache) Get(ctx context.Context, key string) (interface{}, time.Time, error) {
val, err := c.client.Get(ctx, key).Bytes()
if err != nil && err == rclient.Nil {
return nil, time.Time{}, cache.ErrKeyNotFound
} else if err != nil {
return nil, time.Time{}, err
}
dur, err := c.client.TTL(ctx, key).Result()
if err != nil {
return nil, time.Time{}, err
}
if dur == -1 {
return val, time.Unix(1<<63-1, 0), nil
}
if dur == -2 {
return val, time.Time{}, cache.ErrItemExpired
}
return val, time.Now().Add(dur), nil
}
func (c *redisCache) Put(ctx context.Context, key string, val interface{}, dur time.Duration) error {
return c.client.Set(ctx, key, val, dur).Err()
}
func (c *redisCache) Delete(ctx context.Context, key string) error {
return c.client.Del(ctx, key).Err()
}
func (m *redisCache) String() string {
return "redis"
}
+88
View File
@@ -0,0 +1,88 @@
package redis
import (
"context"
"os"
"testing"
"time"
"go-micro.dev/v5/cache"
)
var (
ctx = context.TODO()
key string = "redistestkey"
val interface{} = "hello go-micro"
addr = cache.WithAddress("redis://127.0.0.1:6379")
)
// TestMemCache tests the in-memory cache implementation.
func TestCache(t *testing.T) {
if len(os.Getenv("LOCAL")) == 0 {
t.Skip()
}
t.Run("CacheGetMiss", func(t *testing.T) {
if _, _, err := NewRedisCache(addr).Get(ctx, key); err == nil {
t.Error("expected to get no value from cache")
}
})
t.Run("CacheGetHit", func(t *testing.T) {
c := NewRedisCache(addr)
if err := c.Put(ctx, key, val, 0); err != nil {
t.Error(err)
}
if a, _, err := c.Get(ctx, key); err != nil {
t.Errorf("Expected a value, got err: %s", err)
} else if string(a.([]byte)) != val {
t.Errorf("Expected '%v', got '%v'", val, a)
}
})
t.Run("CacheGetExpired", func(t *testing.T) {
c := NewRedisCache(addr)
d := 20 * time.Millisecond
if err := c.Put(ctx, key, val, d); err != nil {
t.Error(err)
}
<-time.After(25 * time.Millisecond)
if _, _, err := c.Get(ctx, key); err == nil {
t.Error("expected to get no value from cache")
}
})
t.Run("CacheGetValid", func(t *testing.T) {
c := NewRedisCache(addr)
e := 25 * time.Millisecond
if err := c.Put(ctx, key, val, e); err != nil {
t.Error(err)
}
<-time.After(20 * time.Millisecond)
if _, _, err := c.Get(ctx, key); err != nil {
t.Errorf("expected a value, got err: %s", err)
}
})
t.Run("CacheDeleteHit", func(t *testing.T) {
c := NewRedisCache(addr)
if err := c.Put(ctx, key, val, 0); err != nil {
t.Error(err)
}
if err := c.Delete(ctx, key); err != nil {
t.Errorf("Expected to delete an item, got err: %s", err)
}
if _, _, err := c.Get(ctx, key); err == nil {
t.Errorf("Expected error")
}
})
}
+209
View File
@@ -0,0 +1,209 @@
package grpc
import (
b "bytes"
"encoding/json"
"fmt"
"strings"
"go-micro.dev/v5/codec"
"go-micro.dev/v5/codec/bytes"
"google.golang.org/grpc"
"google.golang.org/grpc/encoding"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/runtime/protoiface"
"google.golang.org/protobuf/runtime/protoimpl"
)
type jsonCodec struct{}
type protoCodec struct{}
type bytesCodec struct{}
type wrapCodec struct{ encoding.Codec }
var useNumber bool
var (
defaultGRPCCodecs = map[string]encoding.Codec{
"application/json": jsonCodec{},
"application/proto": protoCodec{},
"application/protobuf": protoCodec{},
"application/octet-stream": protoCodec{},
"application/grpc": protoCodec{},
"application/grpc+json": jsonCodec{},
"application/grpc+proto": protoCodec{},
"application/grpc+bytes": bytesCodec{},
}
)
// UseNumber fix unmarshal Number(8234567890123456789) to interface(8.234567890123457e+18).
func UseNumber() {
useNumber = true
}
func (w wrapCodec) String() string {
return w.Codec.Name()
}
func (w wrapCodec) Marshal(v interface{}) ([]byte, error) {
b, ok := v.(*bytes.Frame)
if ok {
return b.Data, nil
}
return w.Codec.Marshal(v)
}
func (w wrapCodec) Unmarshal(data []byte, v interface{}) error {
b, ok := v.(*bytes.Frame)
if ok {
b.Data = data
return nil
}
return w.Codec.Unmarshal(data, v)
}
func (protoCodec) Marshal(v interface{}) ([]byte, error) {
switch m := v.(type) {
case *bytes.Frame:
return m.Data, nil
case proto.Message:
return proto.Marshal(m)
case protoiface.MessageV1:
// #2333 compatible with etcd legacy proto.Message
m2 := protoimpl.X.ProtoMessageV2Of(m)
return proto.Marshal(m2)
}
return nil, fmt.Errorf("failed to marshal: %v is not type of *bytes.Frame or proto.Message", v)
}
func (protoCodec) Unmarshal(data []byte, v interface{}) error {
switch m := v.(type) {
case proto.Message:
return proto.Unmarshal(data, m)
case protoiface.MessageV1:
// #2333 compatible with etcd legacy proto.Message
m2 := protoimpl.X.ProtoMessageV2Of(m)
return proto.Unmarshal(data, m2)
}
return fmt.Errorf("failed to unmarshal: %v is not type of proto.Message", v)
}
func (protoCodec) Name() string {
return "proto"
}
func (bytesCodec) Marshal(v interface{}) ([]byte, error) {
b, ok := v.(*[]byte)
if !ok {
return nil, fmt.Errorf("failed to marshal: %v is not type of *[]byte", v)
}
return *b, nil
}
func (bytesCodec) Unmarshal(data []byte, v interface{}) error {
b, ok := v.(*[]byte)
if !ok {
return fmt.Errorf("failed to unmarshal: %v is not type of *[]byte", v)
}
*b = data
return nil
}
func (bytesCodec) Name() string {
return "bytes"
}
func (jsonCodec) Marshal(v interface{}) ([]byte, error) {
if b, ok := v.(*bytes.Frame); ok {
return b.Data, nil
}
if pb, ok := v.(proto.Message); ok {
bytes, err := protojson.Marshal(pb)
if err != nil {
return nil, err
}
return bytes, nil
}
return json.Marshal(v)
}
func (jsonCodec) Unmarshal(data []byte, v interface{}) error {
if len(data) == 0 {
return nil
}
if b, ok := v.(*bytes.Frame); ok {
b.Data = data
return nil
}
if pb, ok := v.(proto.Message); ok {
return protojson.Unmarshal(data, pb)
}
dec := json.NewDecoder(b.NewReader(data))
if useNumber {
dec.UseNumber()
}
return dec.Decode(v)
}
func (jsonCodec) Name() string {
return "json"
}
type grpcCodec struct {
// headers
id string
target string
method string
endpoint string
s grpc.ClientStream
c encoding.Codec
}
func (g *grpcCodec) ReadHeader(m *codec.Message, mt codec.MessageType) error {
md, err := g.s.Header()
if err != nil {
return err
}
if m == nil {
m = new(codec.Message)
}
if m.Header == nil {
m.Header = make(map[string]string, len(md))
}
for k, v := range md {
m.Header[k] = strings.Join(v, ",")
}
m.Id = g.id
m.Target = g.target
m.Method = g.method
m.Endpoint = g.endpoint
return nil
}
func (g *grpcCodec) ReadBody(v interface{}) error {
if f, ok := v.(*bytes.Frame); ok {
return g.s.RecvMsg(f)
}
return g.s.RecvMsg(v)
}
func (g *grpcCodec) Write(m *codec.Message, v interface{}) error {
// if we don't have a body
if v != nil {
return g.s.SendMsg(v)
}
// write the body using the framing codec
return g.s.SendMsg(&bytes.Frame{Data: m.Body})
}
func (g *grpcCodec) Close() error {
return g.s.CloseSend()
}
func (g *grpcCodec) String() string {
return g.c.Name()
}
+69
View File
@@ -0,0 +1,69 @@
package grpc
import (
"net/http"
"go-micro.dev/v5/errors"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func microError(err error) error {
// no error
switch err {
case nil:
return nil
}
if verr, ok := err.(*errors.Error); ok {
return verr
}
// grpc error
s, ok := status.FromError(err)
if !ok {
return err
}
// return first error from details
if details := s.Details(); len(details) > 0 {
return microError(details[0].(error))
}
// try to decode micro *errors.Error
if e := errors.Parse(s.Message()); e.Code > 0 {
return e // actually a micro error
}
// fallback
return errors.New("go.micro.client", s.Message(), microStatusFromGrpcCode(s.Code()))
}
func microStatusFromGrpcCode(code codes.Code) int32 {
switch code {
case codes.OK:
return http.StatusOK
case codes.InvalidArgument:
return http.StatusBadRequest
case codes.DeadlineExceeded:
return http.StatusRequestTimeout
case codes.NotFound:
return http.StatusNotFound
case codes.AlreadyExists:
return http.StatusConflict
case codes.PermissionDenied:
return http.StatusForbidden
case codes.Unauthenticated:
return http.StatusUnauthorized
case codes.FailedPrecondition:
return http.StatusPreconditionFailed
case codes.Unimplemented:
return http.StatusNotImplemented
case codes.Internal:
return http.StatusInternalServerError
case codes.Unavailable:
return http.StatusServiceUnavailable
}
return http.StatusInternalServerError
}
+709
View File
@@ -0,0 +1,709 @@
// Package grpc provides a gRPC client
package grpc
import (
"context"
"crypto/tls"
"fmt"
"net"
"reflect"
"strings"
"sync/atomic"
"time"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/client"
"go-micro.dev/v5/cmd"
raw "go-micro.dev/v5/codec/bytes"
"go-micro.dev/v5/errors"
"go-micro.dev/v5/metadata"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/selector"
pnet "go-micro.dev/v5/util/net"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/encoding"
gmetadata "google.golang.org/grpc/metadata"
)
type grpcClient struct {
opts client.Options
pool *pool
once atomic.Value
}
func init() {
cmd.DefaultClients["grpc"] = NewClient
encoding.RegisterCodec(wrapCodec{jsonCodec{}})
encoding.RegisterCodec(wrapCodec{protoCodec{}})
encoding.RegisterCodec(wrapCodec{bytesCodec{}})
}
// secure returns the dial option for whether its a secure or insecure connection.
func (g *grpcClient) secure(addr string) grpc.DialOption {
// first we check if theres'a tls config
if g.opts.Context != nil {
if v := g.opts.Context.Value(tlsAuth{}); v != nil {
tls := v.(*tls.Config)
creds := credentials.NewTLS(tls)
// return tls config if it exists
return grpc.WithTransportCredentials(creds)
}
}
// default config
tlsConfig := &tls.Config{}
defaultCreds := grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig))
// check if the address is prepended with https
if strings.HasPrefix(addr, "https://") {
return defaultCreds
}
// if no port is specified or port is 443 default to tls
_, port, err := net.SplitHostPort(addr)
// assuming with no port its going to be secured
if port == "443" {
return defaultCreds
} else if err != nil && strings.Contains(err.Error(), "missing port in address") {
return defaultCreds
}
// other fallback to insecure
return grpc.WithInsecure()
}
func (g *grpcClient) next(request client.Request, opts client.CallOptions) (selector.Next, error) {
service, address, _ := pnet.Proxy(request.Service(), opts.Address)
// return remote address
if len(address) > 0 {
return func() (*registry.Node, error) {
return &registry.Node{
Address: address[0],
}, nil
}, nil
}
// get next nodes from the selector
next, err := g.opts.Selector.Select(service, opts.SelectOptions...)
if err != nil {
if err == selector.ErrNotFound {
return nil, errors.InternalServerError("go.micro.client", "service %s: %s", service, err.Error())
}
return nil, errors.InternalServerError("go.micro.client", "error selecting %s node: %s", service, err.Error())
}
return next, nil
}
func (g *grpcClient) call(ctx context.Context, node *registry.Node, req client.Request, rsp interface{}, opts client.CallOptions) error {
var header map[string]string
address := node.Address
if md, ok := metadata.FromContext(ctx); ok {
header = make(map[string]string, len(md))
for k, v := range md {
header[strings.ToLower(k)] = v
}
} else {
header = make(map[string]string)
}
// set timeout in nanoseconds
header["timeout"] = fmt.Sprintf("%d", opts.RequestTimeout)
// set the content type for the request
header["x-content-type"] = req.ContentType()
md := gmetadata.New(header)
ctx = gmetadata.NewOutgoingContext(ctx, md)
cf, err := g.newGRPCCodec(req.ContentType())
if err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
maxRecvMsgSize := g.maxRecvMsgSizeValue()
maxSendMsgSize := g.maxSendMsgSizeValue()
var grr error
var dialCtx context.Context
var cancel context.CancelFunc
if opts.DialTimeout > 0 {
dialCtx, cancel = context.WithTimeout(ctx, opts.DialTimeout)
} else {
dialCtx, cancel = context.WithCancel(ctx)
}
defer cancel()
grpcDialOptions := []grpc.DialOption{
g.secure(address),
grpc.WithDefaultCallOptions(
grpc.MaxCallRecvMsgSize(maxRecvMsgSize),
grpc.MaxCallSendMsgSize(maxSendMsgSize),
),
}
if opts := g.getGrpcDialOptions(); opts != nil {
grpcDialOptions = append(grpcDialOptions, opts...)
}
cc, err := g.pool.getConn(dialCtx, address, grpcDialOptions...)
if err != nil {
return errors.InternalServerError("go.micro.client", fmt.Sprintf("Error sending request: %v", err))
}
defer func() {
// defer execution of release
g.pool.release(address, cc, grr)
}()
ch := make(chan error, 1)
go func() {
grpcCallOptions := []grpc.CallOption{
grpc.ForceCodec(cf),
grpc.CallContentSubtype(cf.Name())}
if opts := callOpts(opts); opts != nil {
grpcCallOptions = append(grpcCallOptions, opts...)
}
err := cc.Invoke(ctx, methodToGRPC(req.Service(), req.Endpoint()), req.Body(), rsp, grpcCallOptions...)
ch <- microError(err)
}()
select {
case err := <-ch:
grr = err
case <-ctx.Done():
grr = errors.Timeout("go.micro.client", "%v", ctx.Err())
}
return grr
}
func (g *grpcClient) stream(ctx context.Context, node *registry.Node, req client.Request, rsp interface{}, opts client.CallOptions) error {
var header map[string]string
address := node.Address
if md, ok := metadata.FromContext(ctx); ok {
header = make(map[string]string, len(md))
for k, v := range md {
header[k] = v
}
} else {
header = make(map[string]string)
}
// set timeout in nanoseconds
if opts.StreamTimeout > time.Duration(0) {
header["timeout"] = fmt.Sprintf("%d", opts.StreamTimeout)
}
// set the content type for the request
header["x-content-type"] = req.ContentType()
md := gmetadata.New(header)
// WebSocket connection adds the `Connection: Upgrade` header.
// But as per the HTTP/2 spec, the `Connection` header makes the request malformed
delete(md, "connection")
ctx = gmetadata.NewOutgoingContext(ctx, md)
cf, err := g.newGRPCCodec(req.ContentType())
if err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
var dialCtx context.Context
var cancel context.CancelFunc
if opts.DialTimeout > 0 {
dialCtx, cancel = context.WithTimeout(ctx, opts.DialTimeout)
} else {
dialCtx, cancel = context.WithCancel(ctx)
}
defer cancel()
wc := wrapCodec{cf}
grpcDialOptions := []grpc.DialOption{
g.secure(address),
}
if opts := g.getGrpcDialOptions(); opts != nil {
grpcDialOptions = append(grpcDialOptions, opts...)
}
cc, err := g.pool.getConn(dialCtx, address, grpcDialOptions...)
if err != nil {
return errors.InternalServerError("go.micro.client", fmt.Sprintf("Error sending request: %v", err))
}
desc := &grpc.StreamDesc{
StreamName: req.Service() + req.Endpoint(),
ClientStreams: true,
ServerStreams: true,
}
grpcCallOptions := []grpc.CallOption{
grpc.ForceCodec(wc),
grpc.CallContentSubtype(cf.Name()),
}
if opts := callOpts(opts); opts != nil {
grpcCallOptions = append(grpcCallOptions, opts...)
}
// create a new canceling context
newCtx, cancel := context.WithCancel(ctx)
st, err := cc.NewStream(newCtx, desc, methodToGRPC(req.Service(), req.Endpoint()), grpcCallOptions...)
if err != nil {
// we need to cleanup as we dialed and created a context
// cancel the context
cancel()
// close the connection
cc.Close()
// now return the error
return errors.InternalServerError("go.micro.client", fmt.Sprintf("Error creating stream: %v", err))
}
codec := &grpcCodec{
s: st,
c: wc,
}
// set request codec
if r, ok := req.(*grpcRequest); ok {
r.codec = codec
}
// setup the stream response
stream := &grpcStream{
context: ctx,
request: req,
response: &response{
conn: cc.ClientConn,
stream: st,
codec: cf,
gcodec: codec,
},
stream: st,
cancel: cancel,
release: func(err error) {
g.pool.release(address, cc, err)
},
}
// set the stream as the response
val := reflect.ValueOf(rsp).Elem()
val.Set(reflect.ValueOf(stream).Elem())
return nil
}
func (g *grpcClient) poolMaxStreams() int {
if g.opts.Context == nil {
return DefaultPoolMaxStreams
}
v := g.opts.Context.Value(poolMaxStreams{})
if v == nil {
return DefaultPoolMaxStreams
}
return v.(int)
}
func (g *grpcClient) poolMaxIdle() int {
if g.opts.Context == nil {
return DefaultPoolMaxIdle
}
v := g.opts.Context.Value(poolMaxIdle{})
if v == nil {
return DefaultPoolMaxIdle
}
return v.(int)
}
func (g *grpcClient) maxRecvMsgSizeValue() int {
if g.opts.Context == nil {
return DefaultMaxRecvMsgSize
}
v := g.opts.Context.Value(maxRecvMsgSizeKey{})
if v == nil {
return DefaultMaxRecvMsgSize
}
return v.(int)
}
func (g *grpcClient) maxSendMsgSizeValue() int {
if g.opts.Context == nil {
return DefaultMaxSendMsgSize
}
v := g.opts.Context.Value(maxSendMsgSizeKey{})
if v == nil {
return DefaultMaxSendMsgSize
}
return v.(int)
}
func (g *grpcClient) newGRPCCodec(contentType string) (encoding.Codec, error) {
codecs := make(map[string]encoding.Codec)
if g.opts.Context != nil {
if v := g.opts.Context.Value(codecsKey{}); v != nil {
codecs = v.(map[string]encoding.Codec)
}
}
if c, ok := codecs[contentType]; ok {
return wrapCodec{c}, nil
}
if c, ok := defaultGRPCCodecs[contentType]; ok {
return wrapCodec{c}, nil
}
return nil, fmt.Errorf("unsupported Content-Type: %s", contentType)
}
func (g *grpcClient) Init(opts ...client.Option) error {
size := g.opts.PoolSize
ttl := g.opts.PoolTTL
for _, o := range opts {
o(&g.opts)
}
// update pool configuration if the options changed
if size != g.opts.PoolSize || ttl != g.opts.PoolTTL {
g.pool.Lock()
g.pool.size = g.opts.PoolSize
g.pool.ttl = int64(g.opts.PoolTTL.Seconds())
g.pool.Unlock()
}
return nil
}
func (g *grpcClient) Options() client.Options {
return g.opts
}
func (g *grpcClient) NewMessage(topic string, msg interface{}, opts ...client.MessageOption) client.Message {
return newGRPCEvent(topic, msg, g.opts.ContentType, opts...)
}
func (g *grpcClient) NewRequest(service, method string, req interface{}, reqOpts ...client.RequestOption) client.Request {
return newGRPCRequest(service, method, req, g.opts.ContentType, reqOpts...)
}
func (g *grpcClient) Call(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
if req == nil {
return errors.InternalServerError("go.micro.client", "req is nil")
} else if rsp == nil {
return errors.InternalServerError("go.micro.client", "rsp is nil")
}
// make a copy of call opts
callOpts := g.opts.CallOptions
for _, opt := range opts {
opt(&callOpts)
}
next, err := g.next(req, callOpts)
if err != nil {
return err
}
// check if we already have a deadline
d, ok := ctx.Deadline()
if !ok {
// no deadline so we create a new one
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, callOpts.RequestTimeout)
defer cancel()
} else {
// got a deadline so no need to setup context
// but we need to set the timeout we pass along
opt := client.WithRequestTimeout(time.Until(d))
opt(&callOpts)
}
// should we noop right here?
select {
case <-ctx.Done():
return errors.New("go.micro.client", fmt.Sprintf("%v", ctx.Err()), 408)
default:
}
// make copy of call method
gcall := g.call
// wrap the call in reverse
for i := len(callOpts.CallWrappers); i > 0; i-- {
gcall = callOpts.CallWrappers[i-1](gcall)
}
// return errors.New("go.micro.client", "request timeout", 408)
call := func(i int) error {
// call backoff first. Someone may want an initial start delay
t, err := callOpts.Backoff(ctx, req, i)
if err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
// only sleep if greater than 0
if t.Seconds() > 0 {
time.Sleep(t)
}
// select next node
node, err := next()
service := req.Service()
if err != nil {
if err == selector.ErrNotFound {
return errors.InternalServerError("go.micro.client", "service %s: %s", service, err.Error())
}
return errors.InternalServerError("go.micro.client", "error selecting %s node: %s", service, err.Error())
}
// make the call
err = gcall(ctx, node, req, rsp, callOpts)
g.opts.Selector.Mark(service, node, err)
if verr, ok := err.(*errors.Error); ok {
return verr
}
return err
}
ch := make(chan error, callOpts.Retries+1)
var gerr error
for i := 0; i <= callOpts.Retries; i++ {
go func(i int) {
ch <- call(i)
}(i)
select {
case <-ctx.Done():
return errors.New("go.micro.client", fmt.Sprintf("%v", ctx.Err()), 408)
case err := <-ch:
// if the call succeeded lets bail early
if err == nil {
return nil
}
retry, rerr := callOpts.Retry(ctx, req, i, err)
if rerr != nil {
return rerr
}
if !retry {
return err
}
gerr = err
}
}
return gerr
}
func (g *grpcClient) Stream(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
// make a copy of call opts
callOpts := g.opts.CallOptions
for _, opt := range opts {
opt(&callOpts)
}
next, err := g.next(req, callOpts)
if err != nil {
return nil, err
}
// #200 - streams shouldn't have a request timeout set on the context
// should we noop right here?
select {
case <-ctx.Done():
return nil, errors.New("go.micro.client", fmt.Sprintf("%v", ctx.Err()), 408)
default:
}
// make a copy of stream
gstream := g.stream
// wrap the call in reverse
for i := len(callOpts.CallWrappers); i > 0; i-- {
gstream = callOpts.CallWrappers[i-1](gstream)
}
call := func(i int) (client.Stream, error) {
// call backoff first. Someone may want an initial start delay
t, err := callOpts.Backoff(ctx, req, i)
if err != nil {
return nil, errors.InternalServerError("go.micro.client", err.Error())
}
// only sleep if greater than 0
if t.Seconds() > 0 {
time.Sleep(t)
}
node, err := next()
service := req.Service()
if err != nil {
if err == selector.ErrNotFound {
return nil, errors.InternalServerError("go.micro.client", "service %s: %s", service, err.Error())
}
return nil, errors.InternalServerError("go.micro.client", "error selecting %s node: %s", service, err.Error())
}
// make the call
stream := &grpcStream{}
err = g.stream(ctx, node, req, stream, callOpts)
g.opts.Selector.Mark(service, node, err)
return stream, err
}
type response struct {
stream client.Stream
err error
}
ch := make(chan response, callOpts.Retries+1)
var grr error
for i := 0; i <= callOpts.Retries; i++ {
go func(i int) {
s, err := call(i)
ch <- response{s, err}
}(i)
select {
case <-ctx.Done():
return nil, errors.New("go.micro.client", fmt.Sprintf("%v", ctx.Err()), 408)
case rsp := <-ch:
// if the call succeeded lets bail early
if rsp.err == nil {
return rsp.stream, nil
}
retry, rerr := callOpts.Retry(ctx, req, i, err)
if rerr != nil {
return nil, rerr
}
if !retry {
return nil, rsp.err
}
grr = rsp.err
}
}
return nil, grr
}
func (g *grpcClient) Publish(ctx context.Context, p client.Message, opts ...client.PublishOption) error {
var options client.PublishOptions
for _, o := range opts {
o(&options)
}
md, ok := metadata.FromContext(ctx)
if !ok {
md = make(map[string]string)
}
md["Content-Type"] = p.ContentType()
md["Micro-Topic"] = p.Topic()
cf, err := g.newGRPCCodec(p.ContentType())
if err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
var body []byte
// passed in raw data
if d, ok := p.Payload().(*raw.Frame); ok {
body = d.Data
} else {
// set the body
b, err := cf.Marshal(p.Payload())
if err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
body = b
}
if !g.once.Load().(bool) {
if err = g.opts.Broker.Connect(); err != nil {
return errors.InternalServerError("go.micro.client", err.Error())
}
g.once.Store(true)
}
topic := p.Topic()
// get the exchange
if len(options.Exchange) > 0 {
topic = options.Exchange
}
return g.opts.Broker.Publish(topic, &broker.Message{
Header: md,
Body: body,
}, broker.PublishContext(options.Context))
}
func (g *grpcClient) String() string {
return "grpc"
}
func (g *grpcClient) getGrpcDialOptions() []grpc.DialOption {
if g.opts.CallOptions.Context == nil {
return nil
}
v := g.opts.CallOptions.Context.Value(grpcDialOptions{})
if v == nil {
return nil
}
opts, ok := v.([]grpc.DialOption)
if !ok {
return nil
}
return opts
}
func newClient(opts ...client.Option) client.Client {
options := client.NewOptions()
// default content type for grpc
options.ContentType = "application/grpc+proto"
for _, o := range opts {
o(&options)
}
rc := &grpcClient{
opts: options,
}
rc.once.Store(false)
rc.pool = newPool(options.PoolSize, options.PoolTTL, rc.poolMaxIdle(), rc.poolMaxStreams())
c := client.Client(rc)
// wrap in reverse
for i := len(options.Wrappers); i > 0; i-- {
c = options.Wrappers[i-1](c)
}
return c
}
func NewClient(opts ...client.Option) client.Client {
return newClient(opts...)
}
+218
View File
@@ -0,0 +1,218 @@
package grpc
import (
"context"
"sync"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/connectivity"
)
type pool struct {
size int
ttl int64
// max streams on a *poolConn
maxStreams int
// max idle conns
maxIdle int
sync.Mutex
conns map[string]*streamsPool
}
type streamsPool struct {
// head of list
head *poolConn
// busy conns list
busy *poolConn
// the size of list
count int
// idle conn
idle int
}
type poolConn struct {
// grpc conn
*grpc.ClientConn
err error
addr string
// pool and streams pool
pool *pool
sp *streamsPool
streams int
created int64
// list
pre *poolConn
next *poolConn
in bool
}
func newPool(size int, ttl time.Duration, idle int, ms int) *pool {
if ms <= 0 {
ms = 1
}
if idle < 0 {
idle = 0
}
return &pool{
size: size,
ttl: int64(ttl.Seconds()),
maxStreams: ms,
maxIdle: idle,
conns: make(map[string]*streamsPool),
}
}
func (p *pool) getConn(dialCtx context.Context, addr string, opts ...grpc.DialOption) (*poolConn, error) {
now := time.Now().Unix()
p.Lock()
sp, ok := p.conns[addr]
if !ok {
sp = &streamsPool{head: &poolConn{}, busy: &poolConn{}, count: 0, idle: 0}
p.conns[addr] = sp
}
// while we have conns check streams and then return one
// otherwise we'll create a new conn
conn := sp.head.next
for conn != nil {
// check conn state
// https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md
switch conn.GetState() {
case connectivity.Connecting:
conn = conn.next
continue
case connectivity.Shutdown:
next := conn.next
if conn.streams == 0 {
removeConn(conn)
sp.idle--
}
conn = next
continue
case connectivity.TransientFailure:
next := conn.next
if conn.streams == 0 {
removeConn(conn)
conn.ClientConn.Close()
sp.idle--
}
conn = next
continue
case connectivity.Ready:
case connectivity.Idle:
}
// a old conn
if now-conn.created > p.ttl {
next := conn.next
if conn.streams == 0 {
removeConn(conn)
conn.ClientConn.Close()
sp.idle--
}
conn = next
continue
}
// a busy conn
if conn.streams >= p.maxStreams {
next := conn.next
removeConn(conn)
addConnAfter(conn, sp.busy)
conn = next
continue
}
// a idle conn
if conn.streams == 0 {
sp.idle--
}
// a good conn
conn.streams++
p.Unlock()
return conn, nil
}
p.Unlock()
// create new conn
cc, err := grpc.DialContext(dialCtx, addr, opts...)
if err != nil {
return nil, err
}
conn = &poolConn{cc, nil, addr, p, sp, 1, time.Now().Unix(), nil, nil, false}
// add conn to streams pool
p.Lock()
if sp.count < p.size {
addConnAfter(conn, sp.head)
}
p.Unlock()
return conn, nil
}
func (p *pool) release(addr string, conn *poolConn, err error) {
p.Lock()
p, sp, created := conn.pool, conn.sp, conn.created
// try to add conn
if !conn.in && sp.count < p.size {
addConnAfter(conn, sp.head)
}
if !conn.in {
p.Unlock()
conn.ClientConn.Close()
return
}
// a busy conn
if conn.streams >= p.maxStreams {
removeConn(conn)
addConnAfter(conn, sp.head)
}
conn.streams--
// if streams == 0, we can do something
if conn.streams == 0 {
// 1. it has errored
// 2. too many idle conn or
// 3. conn is too old
now := time.Now().Unix()
if err != nil || sp.idle >= p.maxIdle || now-created > p.ttl {
removeConn(conn)
p.Unlock()
conn.ClientConn.Close()
return
}
sp.idle++
}
p.Unlock()
}
func (conn *poolConn) Close() {
conn.pool.release(conn.addr, conn, conn.err)
}
func removeConn(conn *poolConn) {
if conn.pre != nil {
conn.pre.next = conn.next
}
if conn.next != nil {
conn.next.pre = conn.pre
}
conn.pre = nil
conn.next = nil
conn.in = false
conn.sp.count--
return
}
func addConnAfter(conn *poolConn, after *poolConn) {
conn.next = after.next
conn.pre = after
if after.next != nil {
after.next.pre = conn
}
after.next = conn
conn.in = true
conn.sp.count++
return
}
+63
View File
@@ -0,0 +1,63 @@
package grpc
import (
"context"
"net"
"testing"
"time"
"google.golang.org/grpc"
pb "google.golang.org/grpc/examples/helloworld/helloworld"
)
func testPool(t *testing.T, size int, ttl time.Duration, idle int, ms int) {
// setup server
l, err := net.Listen("tcp", ":0")
if err != nil {
t.Errorf("failed to listen: %v", err)
}
defer l.Close()
s := grpc.NewServer()
pb.RegisterGreeterServer(s, &greeterServer{})
go s.Serve(l)
defer s.Stop()
// zero pool
p := newPool(size, ttl, idle, ms)
for i := 0; i < 10; i++ {
// get a conn
cc, err := p.getConn(context.TODO(), l.Addr().String(), grpc.WithInsecure())
if err != nil {
t.Fatal(err)
}
rsp := pb.HelloReply{}
err = cc.Invoke(context.TODO(), "/helloworld.Greeter/SayHello", &pb.HelloRequest{Name: "John"}, &rsp)
if err != nil {
t.Fatal(err)
}
if rsp.Message != "Hello John" {
t.Errorf("Got unexpected response %v", rsp.Message)
}
// release the conn
p.release(l.Addr().String(), cc, nil)
p.Lock()
if i := p.conns[l.Addr().String()].count; i > size {
p.Unlock()
t.Errorf("pool size %d is greater than expected %d", i, size)
}
p.Unlock()
}
}
func TestGRPCPool(t *testing.T) {
testPool(t, 0, time.Minute, 10, 2)
testPool(t, 2, time.Minute, 10, 1)
}
+112
View File
@@ -0,0 +1,112 @@
package grpc
import (
"context"
"net"
"testing"
"go-micro.dev/v5/client"
"go-micro.dev/v5/errors"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/selector"
pgrpc "google.golang.org/grpc"
pb "google.golang.org/grpc/examples/helloworld/helloworld"
)
// server is used to implement helloworld.GreeterServer.
type greeterServer struct {
pb.UnimplementedGreeterServer
}
// SayHello implements helloworld.GreeterServer.
func (g *greeterServer) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
if in.Name == "Error" {
return nil, &errors.Error{Id: "1", Code: 99, Detail: "detail"}
}
return &pb.HelloReply{Message: "Hello " + in.Name}, nil
}
func TestGRPCClient(t *testing.T) {
l, err := net.Listen("tcp", ":0")
if err != nil {
t.Fatalf("failed to listen: %v", err)
}
defer l.Close()
s := pgrpc.NewServer()
pb.RegisterGreeterServer(s, &greeterServer{})
go s.Serve(l)
defer s.Stop()
// create mock registry
r := registry.NewMemoryRegistry()
// register service
r.Register(&registry.Service{
Name: "helloworld",
Version: "test",
Nodes: []*registry.Node{
{
Id: "test-1",
Address: l.Addr().String(),
Metadata: map[string]string{
"protocol": "grpc",
},
},
},
})
// create selector
se := selector.NewSelector(
selector.Registry(r),
)
// create client
c := NewClient(
client.Registry(r),
client.Selector(se),
)
testMethods := []string{
"/helloworld.Greeter/SayHello",
"Greeter.SayHello",
}
for _, method := range testMethods {
req := c.NewRequest("helloworld", method, &pb.HelloRequest{
Name: "John",
})
rsp := pb.HelloReply{}
err = c.Call(context.TODO(), req, &rsp)
if err != nil {
t.Fatal(err)
}
if rsp.Message != "Hello John" {
t.Fatalf("Got unexpected response %v", rsp.Message)
}
}
req := c.NewRequest("helloworld", "/helloworld.Greeter/SayHello", &pb.HelloRequest{
Name: "Error",
})
rsp := pb.HelloReply{}
err = c.Call(context.TODO(), req, &rsp)
if err == nil {
t.Fatal("nil error received")
}
verr, ok := err.(*errors.Error)
if !ok {
t.Fatalf("invalid error received %#+v\n", err)
}
if verr.Code != 99 && verr.Id != "1" && verr.Detail != "detail" {
t.Fatalf("invalid error received %#+v\n", verr)
}
}
+40
View File
@@ -0,0 +1,40 @@
package grpc
import (
"go-micro.dev/v5/client"
)
type grpcEvent struct {
topic string
contentType string
payload interface{}
}
func newGRPCEvent(topic string, payload interface{}, contentType string, opts ...client.MessageOption) client.Message {
var options client.MessageOptions
for _, o := range opts {
o(&options)
}
if len(options.ContentType) > 0 {
contentType = options.ContentType
}
return &grpcEvent{
payload: payload,
topic: topic,
contentType: contentType,
}
}
func (g *grpcEvent) ContentType() string {
return g.contentType
}
func (g *grpcEvent) Topic() string {
return g.topic
}
func (g *grpcEvent) Payload() interface{} {
return g.payload
}
+143
View File
@@ -0,0 +1,143 @@
// Package grpc provides a gRPC options
package grpc
import (
"context"
"crypto/tls"
"go-micro.dev/v5/client"
"google.golang.org/grpc"
"google.golang.org/grpc/encoding"
)
var (
// DefaultPoolMaxStreams maximum streams on a connectioin
// (20).
DefaultPoolMaxStreams = 20
// DefaultPoolMaxIdle maximum idle conns of a pool
// (50).
DefaultPoolMaxIdle = 50
// DefaultMaxRecvMsgSize maximum message that client can receive
// (4 MB).
DefaultMaxRecvMsgSize = 1024 * 1024 * 4
// DefaultMaxSendMsgSize maximum message that client can send
// (4 MB).
DefaultMaxSendMsgSize = 1024 * 1024 * 4
)
type poolMaxStreams struct{}
type poolMaxIdle struct{}
type codecsKey struct{}
type tlsAuth struct{}
type maxRecvMsgSizeKey struct{}
type maxSendMsgSizeKey struct{}
type grpcDialOptions struct{}
type grpcCallOptions struct{}
// maximum streams on a connectioin.
func PoolMaxStreams(n int) client.Option {
return func(o *client.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, poolMaxStreams{}, n)
}
}
// maximum idle conns of a pool.
func PoolMaxIdle(d int) client.Option {
return func(o *client.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, poolMaxIdle{}, d)
}
}
// gRPC Codec to be used to encode/decode requests for a given content type.
func Codec(contentType string, c encoding.Codec) client.Option {
return func(o *client.Options) {
codecs := make(map[string]encoding.Codec)
if o.Context == nil {
o.Context = context.Background()
}
if v := o.Context.Value(codecsKey{}); v != nil {
codecs = v.(map[string]encoding.Codec)
}
codecs[contentType] = c
o.Context = context.WithValue(o.Context, codecsKey{}, codecs)
}
}
// AuthTLS should be used to setup a secure authentication using TLS.
func AuthTLS(t *tls.Config) client.Option {
return func(o *client.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, tlsAuth{}, t)
}
}
// MaxRecvMsgSize set the maximum size of message that client can receive.
func MaxRecvMsgSize(s int) client.Option {
return func(o *client.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, maxRecvMsgSizeKey{}, s)
}
}
// MaxSendMsgSize set the maximum size of message that client can send.
func MaxSendMsgSize(s int) client.Option {
return func(o *client.Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, maxSendMsgSizeKey{}, s)
}
}
// DialOptions to be used to configure gRPC dial options.
func DialOptions(opts ...grpc.DialOption) client.CallOption {
return func(o *client.CallOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, grpcDialOptions{}, opts)
}
}
// CallOptions to be used to configure gRPC call options.
func CallOptions(opts ...grpc.CallOption) client.CallOption {
return func(o *client.CallOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, grpcCallOptions{}, opts)
}
}
func callOpts(opts client.CallOptions) []grpc.CallOption {
if opts.Context == nil {
return nil
}
v := opts.Context.Value(grpcCallOptions{})
if v == nil {
return nil
}
options, ok := v.([]grpc.CallOption)
if !ok {
return nil
}
return options
}
+87
View File
@@ -0,0 +1,87 @@
package grpc
import (
"fmt"
"strings"
"go-micro.dev/v5/client"
"go-micro.dev/v5/codec"
)
type grpcRequest struct {
service string
method string
contentType string
request interface{}
opts client.RequestOptions
codec codec.Codec
}
// service Struct.Method /service.Struct/Method.
func methodToGRPC(service, method string) string {
// no method or already grpc method
if len(method) == 0 || method[0] == '/' {
return method
}
// assume method is Foo.Bar
mParts := strings.Split(method, ".")
if len(mParts) != 2 {
return method
}
if len(service) == 0 {
return fmt.Sprintf("/%s/%s", mParts[0], mParts[1])
}
// return /pkg.Foo/Bar
return fmt.Sprintf("/%s.%s/%s", service, mParts[0], mParts[1])
}
func newGRPCRequest(service, method string, request interface{}, contentType string, reqOpts ...client.RequestOption) client.Request {
var opts client.RequestOptions
for _, o := range reqOpts {
o(&opts)
}
// set the content-type specified
if len(opts.ContentType) > 0 {
contentType = opts.ContentType
}
return &grpcRequest{
service: service,
method: method,
request: request,
contentType: contentType,
opts: opts,
}
}
func (g *grpcRequest) ContentType() string {
return g.contentType
}
func (g *grpcRequest) Service() string {
return g.service
}
func (g *grpcRequest) Method() string {
return g.method
}
func (g *grpcRequest) Endpoint() string {
return g.method
}
func (g *grpcRequest) Codec() codec.Writer {
return g.codec
}
func (g *grpcRequest) Body() interface{} {
return g.request
}
func (g *grpcRequest) Stream() bool {
return g.opts.Stream
}
+41
View File
@@ -0,0 +1,41 @@
package grpc
import (
"testing"
)
func TestMethodToGRPC(t *testing.T) {
testData := []struct {
service string
method string
expect string
}{
{
"helloworld",
"Greeter.SayHello",
"/helloworld.Greeter/SayHello",
},
{
"helloworld",
"/helloworld.Greeter/SayHello",
"/helloworld.Greeter/SayHello",
},
{
"",
"/helloworld.Greeter/SayHello",
"/helloworld.Greeter/SayHello",
},
{
"",
"Greeter.SayHello",
"/Greeter/SayHello",
},
}
for _, d := range testData {
method := methodToGRPC(d.service, d.method)
if method != d.expect {
t.Fatalf("expected %s got %s", d.expect, method)
}
}
}
+44
View File
@@ -0,0 +1,44 @@
package grpc
import (
"strings"
"go-micro.dev/v5/codec"
"go-micro.dev/v5/codec/bytes"
"google.golang.org/grpc"
"google.golang.org/grpc/encoding"
)
type response struct {
conn *grpc.ClientConn
stream grpc.ClientStream
codec encoding.Codec
gcodec codec.Codec
}
// Read the response.
func (r *response) Codec() codec.Reader {
return r.gcodec
}
// read the header.
func (r *response) Header() map[string]string {
md, err := r.stream.Header()
if err != nil {
return map[string]string{}
}
hdr := make(map[string]string, len(md))
for k, v := range md {
hdr[k] = strings.Join(v, ",")
}
return hdr
}
// Read the undecoded response.
func (r *response) Read() ([]byte, error) {
f := &bytes.Frame{}
if err := r.gcodec.ReadBody(f); err != nil {
return nil, err
}
return f.Data, nil
}
+84
View File
@@ -0,0 +1,84 @@
package grpc
import (
"context"
"io"
"sync"
"go-micro.dev/v5/client"
"google.golang.org/grpc"
)
// Implements the streamer interface.
type grpcStream struct {
sync.RWMutex
closed bool
err error
stream grpc.ClientStream
request client.Request
response client.Response
context context.Context
cancel func()
release func(error)
}
func (g *grpcStream) Context() context.Context {
return g.context
}
func (g *grpcStream) Request() client.Request {
return g.request
}
func (g *grpcStream) Response() client.Response {
return g.response
}
func (g *grpcStream) Send(msg interface{}) error {
if err := g.stream.SendMsg(msg); err != nil {
g.setError(err)
return err
}
return nil
}
func (g *grpcStream) Recv(msg interface{}) (err error) {
if err = g.stream.RecvMsg(msg); err != nil {
if err != io.EOF {
g.setError(err)
}
return err
}
return
}
func (g *grpcStream) Error() error {
g.RLock()
defer g.RUnlock()
return g.err
}
func (g *grpcStream) setError(e error) {
g.Lock()
g.err = e
g.Unlock()
}
func (g *grpcStream) CloseSend() error {
return g.stream.CloseSend()
}
func (g *grpcStream) Close() error {
g.Lock()
defer g.Unlock()
if g.closed {
return nil
}
// cancel the context
g.cancel()
g.closed = true
// release back to pool
g.release(g.err)
return nil
}
+8 -1
View File
@@ -89,7 +89,7 @@ type CallOptions struct {
CallWrappers []CallWrapper
// ConnectionTimeout of one request to the server.
// Set this lower than the RequestTimeout to enbale retries on connection timeout.
// Set this lower than the RequestTimeout to enable retries on connection timeout.
ConnectionTimeout time.Duration
// Request/Response timeout of entire srv.Call, for single request timeout set ConnectionTimeout.
RequestTimeout time.Duration
@@ -261,6 +261,13 @@ func Retry(fn RetryFunc) Option {
}
}
// ConnectionTimeout sets the connection timeout
func ConnectionTimeout(t time.Duration) Option {
return func(o *Options) {
o.CallOptions.ConnectionTimeout = t
}
}
// RequestTimeout set the request timeout.
func RequestTimeout(d time.Duration) Option {
return func(o *Options) {
+3 -4
View File
@@ -30,13 +30,11 @@ const (
)
type rpcClient struct {
seq uint64
opts Options
once atomic.Value
pool pool.Pool
seq uint64
mu sync.RWMutex
mu sync.RWMutex
}
func newRPCClient(opt ...Option) Client {
@@ -373,6 +371,7 @@ func (r *rpcClient) Init(opts ...Option) error {
pool.Size(r.opts.PoolSize),
pool.TTL(r.opts.PoolTTL),
pool.Transport(r.opts.Transport),
pool.CloseTimeout(r.opts.PoolCloseTimeout),
)
}
+271 -63
View File
@@ -3,26 +3,42 @@ package cmd
import (
"fmt"
"math/rand"
"os"
"sort"
"strings"
"time"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/auth"
"go-micro.dev/v5/broker"
nbroker "go-micro.dev/v5/broker/nats"
rabbit "go-micro.dev/v5/broker/rabbitmq"
"go-micro.dev/v5/cache"
"go-micro.dev/v5/cache/redis"
"go-micro.dev/v5/client"
"go-micro.dev/v5/config"
"go-micro.dev/v5/debug/profile"
"go-micro.dev/v5/debug/profile/http"
"go-micro.dev/v5/debug/profile/pprof"
"go-micro.dev/v5/debug/trace"
"go-micro.dev/v5/events"
"go-micro.dev/v5/genai"
"go-micro.dev/v5/genai/gemini"
"go-micro.dev/v5/genai/openai"
"go-micro.dev/v5/logger"
mprofile "go-micro.dev/v5/profile"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/registry/consul"
"go-micro.dev/v5/registry/etcd"
"go-micro.dev/v5/registry/nats"
"go-micro.dev/v5/selector"
"go-micro.dev/v5/server"
"go-micro.dev/v5/store"
"go-micro.dev/v5/store/mysql"
natsjskv "go-micro.dev/v5/store/nats-js-kv"
postgres "go-micro.dev/v5/store/postgres"
"go-micro.dev/v5/transport"
ntransport "go-micro.dev/v5/transport/nats"
)
type Cmd interface {
@@ -132,7 +148,12 @@ var (
},
&cli.StringFlag{
Name: "profile",
Usage: "Debug profiler for cpu and memory stats",
Usage: "Plugin profile to use. (local, nats, etc)",
EnvVars: []string{"MICRO_PROFILE"},
},
&cli.StringFlag{
Name: "debug-profile",
Usage: "Debug Plugin profile to use.",
EnvVars: []string{"MICRO_DEBUG_PROFILE"},
},
&cli.StringFlag{
@@ -226,67 +247,108 @@ var (
EnvVars: []string{"MICRO_CONFIG"},
Usage: "The source of the config to be used to get configuration",
},
&cli.StringFlag{
Name: "genai",
EnvVars: []string{"MICRO_GENAI"},
Usage: "GenAI provider to use (e.g. openai, gemini, noop)",
},
&cli.StringFlag{
Name: "genai_key",
EnvVars: []string{"MICRO_GENAI_KEY"},
Usage: "GenAI API key",
},
&cli.StringFlag{
Name: "genai_model",
EnvVars: []string{"MICRO_GENAI_MODEL"},
Usage: "GenAI model to use (optional)",
},
}
DefaultBrokers = map[string]func(...broker.Option) broker.Broker{}
DefaultBrokers = map[string]func(...broker.Option) broker.Broker{
"memory": broker.NewMemoryBroker,
"http": broker.NewHttpBroker,
"nats": nbroker.NewNatsBroker,
"rabbitmq": rabbit.NewBroker,
}
DefaultClients = map[string]func(...client.Option) client.Client{}
DefaultRegistries = map[string]func(...registry.Option) registry.Registry{}
DefaultRegistries = map[string]func(...registry.Option) registry.Registry{
"consul": consul.NewConsulRegistry,
"memory": registry.NewMemoryRegistry,
"nats": nats.NewNatsRegistry,
"mdns": registry.NewMDNSRegistry,
"etcd": etcd.NewEtcdRegistry,
}
DefaultSelectors = map[string]func(...selector.Option) selector.Selector{}
DefaultServers = map[string]func(...server.Option) server.Server{}
DefaultTransports = map[string]func(...transport.Option) transport.Transport{}
DefaultTransports = map[string]func(...transport.Option) transport.Transport{
"nats": ntransport.NewTransport,
}
DefaultStores = map[string]func(...store.Option) store.Store{}
DefaultStores = map[string]func(...store.Option) store.Store{
"memory": store.NewMemoryStore,
"mysql": mysql.NewMysqlStore,
"natsjskv": natsjskv.NewStore,
"postgres": postgres.NewStore,
}
DefaultTracers = map[string]func(...trace.Option) trace.Tracer{}
DefaultAuths = map[string]func(...auth.Option) auth.Auth{}
DefaultProfiles = map[string]func(...profile.Option) profile.Profile{
DefaultDebugProfiles = map[string]func(...profile.Option) profile.Profile{
"http": http.NewProfile,
"pprof": pprof.NewProfile,
}
DefaultConfigs = map[string]func(...config.Option) (config.Config, error){}
DefaultCaches = map[string]func(...cache.Option) cache.Cache{}
DefaultCaches = map[string]func(...cache.Option) cache.Cache{
"redis": redis.NewRedisCache,
}
DefaultStreams = map[string]func(...events.Option) (events.Stream, error){}
DefaultGenAI = map[string]func(...genai.Option) genai.GenAI{
"openai": openai.New,
"gemini": gemini.New,
}
)
func init() {
rand.Seed(time.Now().Unix())
}
func newCmd(opts ...Option) Cmd {
options := Options{
Auth: &auth.DefaultAuth,
Broker: &broker.DefaultBroker,
Client: &client.DefaultClient,
Registry: &registry.DefaultRegistry,
Server: &server.DefaultServer,
Selector: &selector.DefaultSelector,
Transport: &transport.DefaultTransport,
Store: &store.DefaultStore,
Tracer: &trace.DefaultTracer,
Profile: &profile.DefaultProfile,
Config: &config.DefaultConfig,
Cache: &cache.DefaultCache,
Auth: &auth.DefaultAuth,
Broker: &broker.DefaultBroker,
Client: &client.DefaultClient,
Registry: &registry.DefaultRegistry,
Server: &server.DefaultServer,
Selector: &selector.DefaultSelector,
Transport: &transport.DefaultTransport,
Store: &store.DefaultStore,
Tracer: &trace.DefaultTracer,
DebugProfile: &profile.DefaultProfile,
Config: &config.DefaultConfig,
Cache: &cache.DefaultCache,
Stream: &events.DefaultStream,
Brokers: DefaultBrokers,
Clients: DefaultClients,
Registries: DefaultRegistries,
Selectors: DefaultSelectors,
Servers: DefaultServers,
Transports: DefaultTransports,
Stores: DefaultStores,
Tracers: DefaultTracers,
Auths: DefaultAuths,
Profiles: DefaultProfiles,
Configs: DefaultConfigs,
Caches: DefaultCaches,
Brokers: DefaultBrokers,
Clients: DefaultClients,
Registries: DefaultRegistries,
Selectors: DefaultSelectors,
Servers: DefaultServers,
Transports: DefaultTransports,
Stores: DefaultStores,
Tracers: DefaultTracers,
Auths: DefaultAuths,
DebugProfiles: DefaultDebugProfiles,
Configs: DefaultConfigs,
Caches: DefaultCaches,
}
for _, o := range opts {
@@ -325,15 +387,73 @@ func (c *cmd) Options() Options {
}
func (c *cmd) Before(ctx *cli.Context) error {
// Set GenAI provider from flags/env
setGenAIFromFlags(ctx)
// If flags are set then use them otherwise do nothing
var serverOpts []server.Option
var clientOpts []client.Option
// --- Profile Grouping Extension ---
profileName := ctx.String("profile")
if profileName == "" {
profileName = os.Getenv("MICRO_PROFILE")
}
if profileName != "" {
switch profileName {
case "local":
imported, ierr := mprofile.LocalProfile()
if ierr != nil {
return fmt.Errorf("failed to load local profile: %v", ierr)
}
*c.opts.Registry = imported.Registry
registry.DefaultRegistry = imported.Registry
*c.opts.Broker = imported.Broker
broker.DefaultBroker = imported.Broker
*c.opts.Store = imported.Store
store.DefaultStore = imported.Store
*c.opts.Transport = imported.Transport
transport.DefaultTransport = imported.Transport
case "nats":
imported, ierr := mprofile.NatsProfile()
if ierr != nil {
return fmt.Errorf("failed to load nats profile: %v", ierr)
}
// Set the registry
sopts, clopts := c.setRegistry(imported.Registry)
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
// set the store
sopts, clopts = c.setStore(imported.Store)
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
// set the transport
sopts, clopts = c.setTransport(imported.Transport)
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
// Set the broker
sopts, clopts = c.setBroker(imported.Broker)
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
// Set the stream
sopts, clopts = c.setStream(imported.Stream)
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
// Add more profiles as needed
default:
return fmt.Errorf("unsupported profile: %s", profileName)
}
}
// Set the client
if name := ctx.String("client"); len(name) > 0 {
// only change if we have the client and type differs
if cl, ok := c.opts.Clients[name]; ok && (*c.opts.Client).String() != name {
*c.opts.Client = cl()
client.DefaultClient = *c.opts.Client
}
}
@@ -342,6 +462,7 @@ func (c *cmd) Before(ctx *cli.Context) error {
// only change if we have the server and type differs
if s, ok := c.opts.Servers[name]; ok && (*c.opts.Server).String() != name {
*c.opts.Server = s()
server.DefaultServer = *c.opts.Server
}
}
@@ -349,20 +470,22 @@ func (c *cmd) Before(ctx *cli.Context) error {
if name := ctx.String("store"); len(name) > 0 {
s, ok := c.opts.Stores[name]
if !ok {
return fmt.Errorf("Unsupported store: %s", name)
return fmt.Errorf("unsupported store: %s", name)
}
*c.opts.Store = s(store.WithClient(*c.opts.Client))
store.DefaultStore = *c.opts.Store
}
// Set the tracer
if name := ctx.String("tracer"); len(name) > 0 {
r, ok := c.opts.Tracers[name]
if !ok {
return fmt.Errorf("Unsupported tracer: %s", name)
return fmt.Errorf("unsupported tracer: %s", name)
}
*c.opts.Tracer = r()
trace.DefaultTracer = *c.opts.Tracer
}
// Setup auth
@@ -385,10 +508,11 @@ func (c *cmd) Before(ctx *cli.Context) error {
if name := ctx.String("auth"); len(name) > 0 {
r, ok := c.opts.Auths[name]
if !ok {
return fmt.Errorf("Unsupported auth: %s", name)
return fmt.Errorf("unsupported auth: %s", name)
}
*c.opts.Auth = r(authOpts...)
auth.DefaultAuth = *c.opts.Auth
}
// Set the registry
@@ -398,29 +522,19 @@ func (c *cmd) Before(ctx *cli.Context) error {
return fmt.Errorf("Registry %s not found", name)
}
*c.opts.Registry = r()
serverOpts = append(serverOpts, server.Registry(*c.opts.Registry))
clientOpts = append(clientOpts, client.Registry(*c.opts.Registry))
if err := (*c.opts.Selector).Init(selector.Registry(*c.opts.Registry)); err != nil {
logger.Fatalf("Error configuring registry: %v", err)
}
clientOpts = append(clientOpts, client.Selector(*c.opts.Selector))
if err := (*c.opts.Broker).Init(broker.Registry(*c.opts.Registry)); err != nil {
logger.Fatalf("Error configuring broker: %v", err)
}
sopts, clopts := c.setRegistry(r())
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
}
// Set the profile
if name := ctx.String("profile"); len(name) > 0 {
p, ok := c.opts.Profiles[name]
// Set the debug profile
if name := ctx.String("debug-profile"); len(name) > 0 {
p, ok := c.opts.DebugProfiles[name]
if !ok {
return fmt.Errorf("Unsupported profile: %s", name)
return fmt.Errorf("unsupported profile: %s", name)
}
*c.opts.Profile = p()
*c.opts.DebugProfile = p()
profile.DefaultProfile = *c.opts.DebugProfile
}
// Set the broker
@@ -429,10 +543,9 @@ func (c *cmd) Before(ctx *cli.Context) error {
if !ok {
return fmt.Errorf("Broker %s not found", name)
}
*c.opts.Broker = b()
serverOpts = append(serverOpts, server.Broker(*c.opts.Broker))
clientOpts = append(clientOpts, client.Broker(*c.opts.Broker))
sopts, clopts := c.setBroker(b())
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
}
// Set the selector
@@ -446,6 +559,7 @@ func (c *cmd) Before(ctx *cli.Context) error {
// No server option here. Should there be?
clientOpts = append(clientOpts, client.Selector(*c.opts.Selector))
selector.DefaultSelector = *c.opts.Selector
}
// Set the transport
@@ -455,9 +569,10 @@ func (c *cmd) Before(ctx *cli.Context) error {
return fmt.Errorf("Transport %s not found", name)
}
*c.opts.Transport = t()
serverOpts = append(serverOpts, server.Transport(*c.opts.Transport))
clientOpts = append(clientOpts, client.Transport(*c.opts.Transport))
sopts, clopts := c.setTransport(t())
serverOpts = append(serverOpts, sopts...)
clientOpts = append(clientOpts, clopts...)
}
// Parse the server options
@@ -597,12 +712,71 @@ func (c *cmd) Before(ctx *cli.Context) error {
logger.Fatalf("Error configuring config: %v", err)
}
*c.opts.Config = rc
config.DefaultConfig = *c.opts.Config
}
}
return nil
}
func (c *cmd) setRegistry(r registry.Registry) ([]server.Option, []client.Option) {
var serverOpts []server.Option
var clientOpts []client.Option
*c.opts.Registry = r
serverOpts = append(serverOpts, server.Registry(*c.opts.Registry))
clientOpts = append(clientOpts, client.Registry(*c.opts.Registry))
if err := (*c.opts.Selector).Init(selector.Registry(*c.opts.Registry)); err != nil {
logger.Fatalf("Error configuring registry: %v", err)
}
clientOpts = append(clientOpts, client.Selector(*c.opts.Selector))
if err := (*c.opts.Broker).Init(broker.Registry(*c.opts.Registry)); err != nil {
logger.Fatalf("Error configuring broker: %v", err)
}
registry.DefaultRegistry = *c.opts.Registry
return serverOpts, clientOpts
}
func (c *cmd) setStream(s events.Stream) ([]server.Option, []client.Option) {
var serverOpts []server.Option
var clientOpts []client.Option
*c.opts.Stream = s
// TODO: do server and client need a Stream?
// serverOpts = append(serverOpts, server.Registry(*c.opts.Registry))
// clientOpts = append(clientOpts, client.Registry(*c.opts.Registry))
events.DefaultStream = *c.opts.Stream
return serverOpts, clientOpts
}
func (c *cmd) setBroker(b broker.Broker) ([]server.Option, []client.Option) {
var serverOpts []server.Option
var clientOpts []client.Option
*c.opts.Broker = b
serverOpts = append(serverOpts, server.Broker(*c.opts.Broker))
clientOpts = append(clientOpts, client.Broker(*c.opts.Broker))
broker.DefaultBroker = *c.opts.Broker
return serverOpts, clientOpts
}
func (c *cmd) setStore(s store.Store) ([]server.Option, []client.Option) {
var serverOpts []server.Option
var clientOpts []client.Option
*c.opts.Store = s
store.DefaultStore = *c.opts.Store
return serverOpts, clientOpts
}
func (c *cmd) setTransport(t transport.Transport) ([]server.Option, []client.Option) {
var serverOpts []server.Option
var clientOpts []client.Option
*c.opts.Transport = t
serverOpts = append(serverOpts, server.Transport(*c.opts.Transport))
clientOpts = append(clientOpts, client.Transport(*c.opts.Transport))
transport.DefaultTransport = *c.opts.Transport
return serverOpts, clientOpts
}
func (c *cmd) Init(opts ...Option) error {
for _, o := range opts {
o(&c.opts)
@@ -634,3 +808,37 @@ func Init(opts ...Option) error {
func NewCmd(opts ...Option) Cmd {
return newCmd(opts...)
}
// Register CLI commands
func Register(cmds ...*cli.Command) {
app := DefaultCmd.App()
app.Commands = append(app.Commands, cmds...)
// sort the commands so they're listed in order on the cli
// todo: move this to micro/cli so it's only run when the
// commands are printed during "help"
sort.Slice(app.Commands, func(i, j int) bool {
return app.Commands[i].Name < app.Commands[j].Name
})
}
func setGenAIFromFlags(ctx *cli.Context) {
provider := ctx.String("genai")
key := ctx.String("genai_key")
model := ctx.String("genai_model")
switch provider {
case "openai":
if key == "" {
key = os.Getenv("OPENAI_API_KEY")
}
genai.DefaultGenAI = openai.New(genai.WithAPIKey(key), genai.WithModel(model))
case "gemini":
if key == "" {
key = os.Getenv("GEMINI_API_KEY")
}
genai.DefaultGenAI = gemini.New(genai.WithAPIKey(key), genai.WithModel(model))
default:
// No GenAI provider configured - using default noop
}
}
+389
View File
@@ -0,0 +1,389 @@
# Micro
Go Micro Command Line
## Install the CLI
Install `micro` via `go install`
```
go install go-micro.dev/v5/cmd/micro@v5.10.0
```
## Create a service
Create your service (all setup is now automatic!):
```
micro new helloworld
```
This will:
- Create a new service in the `helloworld` directory
- Automatically run `go mod tidy` and `make proto` for you
- Show the updated project tree including generated files
- Warn you if `protoc` is not installed, with install instructions
## Run the service
Run your service:
```
micro run
```
This starts:
- **API Gateway** on http://localhost:8080
- **Web Dashboard** at http://localhost:8080
- **Hot Reload** watching for file changes
- **Services** in dependency order
Open http://localhost:8080 to see your services and call them from the browser.
### Output
```
┌─────────────────────────────────────────────────────────────┐
│ │
│ Micro │
│ │
│ Web: http://localhost:8080 │
│ API: http://localhost:8080/api/{service}/{method} │
│ Health: http://localhost:8080/health │
│ │
│ Services: │
│ ● helloworld │
│ │
│ Watching for changes... │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Options
```
micro run # Gateway on :8080, hot reload enabled
micro run --address :3000 # Gateway on custom port
micro run --no-gateway # Services only, no HTTP gateway
micro run --no-watch # Disable hot reload
micro run --env production # Use production environment
micro run github.com/micro/blog # Clone and run from GitHub
```
### Calling Services
Via curl:
```bash
curl -X POST http://localhost:8080/api/helloworld/Helloworld.Call -d '{"name": "World"}'
```
Or browse to http://localhost:8080 and use the web interface.
List services:
```
micro services
```
## Configuration (micro.mu)
For multi-service projects, create a `micro.mu` file to define services, dependencies, and environments:
```
service users
path ./users
port 8081
service posts
path ./posts
port 8082
depends users
service web
path ./web
port 8089
depends users posts
env development
STORE_ADDRESS file://./data
DEBUG true
env production
STORE_ADDRESS postgres://localhost/db
```
### Configuration Options
| Property | Description |
|----------|-------------|
| `path` | Directory containing the service (with main.go) |
| `port` | Port the service listens on (for health checks) |
| `depends` | Services that must start first (space-separated) |
### Environment Management
Environment variables are injected based on the `--env` flag:
```
micro run # Uses 'development' env (default)
micro run --env production # Uses 'production' env
MICRO_ENV=staging micro run # Uses 'staging' env
```
### JSON Alternative
You can also use `micro.json` if you prefer:
```json
{
"services": {
"users": { "path": "./users", "port": 8081 },
"posts": { "path": "./posts", "port": 8082, "depends": ["users"] }
},
"env": {
"development": { "STORE_ADDRESS": "file://./data" }
}
}
```
### Without Configuration
If no `micro.mu` or `micro.json` exists, `micro run` discovers all `main.go` files and runs them (original behavior).
## Describe the service
Describe the service to see available endpoints
```
micro describe helloworld
```
Output
```
{
"name": "helloworld",
"version": "latest",
"metadata": null,
"endpoints": [
{
"request": {
"name": "Request",
"type": "Request",
"values": [
{
"name": "name",
"type": "string",
"values": null
}
]
},
"response": {
"name": "Response",
"type": "Response",
"values": [
{
"name": "msg",
"type": "string",
"values": null
}
]
},
"metadata": {},
"name": "Helloworld.Call"
},
{
"request": {
"name": "Context",
"type": "Context",
"values": null
},
"response": {
"name": "Stream",
"type": "Stream",
"values": null
},
"metadata": {
"stream": "true"
},
"name": "Helloworld.Stream"
}
],
"nodes": [
{
"metadata": {
"broker": "http",
"protocol": "mucp",
"registry": "mdns",
"server": "mucp",
"transport": "http"
},
"id": "helloworld-31e55be7-ac83-4810-89c8-a6192fb3ae83",
"address": "127.0.0.1:39963"
}
]
}
```
## Call the service
Call via RPC endpoint
```
micro call helloworld Helloworld.Call '{"name": "Asim"}'
```
## Create a client
Create a client to call the service
```go
package main
import (
"context"
"fmt"
"go-micro.dev/v5"
)
type Request struct {
Name string
}
type Response struct {
Message string
}
func main() {
client := micro.New("helloworld").Client()
req := client.NewRequest("helloworld", "Helloworld.Call", &Request{Name: "John"})
var rsp Response
err := client.Call(context.TODO(), req, &rsp)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(rsp.Message)
}
```
## Building and Deployment
### Build Binaries
Build Go binaries for deployment:
```bash
micro build # Build for current OS
micro build --os linux # Cross-compile for Linux
micro build --os linux --arch arm64 # For ARM64
micro build --output ./dist # Custom output directory
```
### Deploy to Server
Deploy to any Linux server with systemd:
```bash
# First time: set up the server
ssh user@server
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
exit
# Deploy from your laptop
micro deploy user@server
```
The deploy command:
1. Builds binaries for linux/amd64
2. Copies via SSH to `/opt/micro/bin/`
3. Sets up systemd services (`micro@<service>`)
4. Restarts and verifies services are running
### Named Deploy Targets
Add deploy targets to `micro.mu`:
```
deploy prod
ssh deploy@prod.example.com
deploy staging
ssh deploy@staging.example.com
```
Then:
```bash
micro deploy prod # Deploy to production
micro deploy staging # Deploy to staging
```
### Managing Deployed Services
```bash
# Check status
micro status --remote user@server
# View logs
micro logs --remote user@server
micro logs myservice --remote user@server -f
# Stop a service
micro stop myservice --remote user@server
```
See [docs/deployment.md](../../docs/deployment.md) for the full deployment guide.
## Protobuf
Use protobuf for code generation with [protoc-gen-micro](https://github.com/micro/go-micro/tree/master/cmd/protoc-gen-micro)
## Server
The micro server is an api and web dashboard that provide a fixed entrypoint for seeing and querying services.
Run it like so
```
micro server
```
Then browse to [localhost:8080](http://localhost:8080)
### API Endpoints
The API provides a fixed HTTP entrypoint for calling services
```
curl http://localhost:8080/api/helloworld/Helloworld/Call -d '{"name": "John"}'
```
See /api for more details and documentation for each service
### Web Dashboard
The web dashboard provides a modern, secure UI for managing and exploring your Micro services. Major features include:
- **Dynamic Service & Endpoint Forms**: Browse all registered services and endpoints. For each endpoint, a dynamic form is generated for easy testing and exploration.
- **API Documentation**: The `/api` page lists all available services and endpoints, with request/response schemas and a sidebar for quick navigation. A documentation banner explains authentication requirements.
- **JWT Authentication**: All login and token management uses a custom JWT utility. Passwords are securely stored with bcrypt. All `/api/x` endpoints and authenticated pages require an `Authorization: Bearer <token>` header (or `micro_token` cookie as fallback).
- **Token Management**: The `/auth/tokens` page allows you to generate, view (obfuscated), and copy JWT tokens. Tokens are stored and can be revoked. When a user is deleted, all their tokens are revoked immediately.
- **User Management**: The `/auth/users` page allows you to create, list, and delete users. Passwords are never shown or stored in plaintext.
- **Token Revocation**: JWT tokens are stored and checked for revocation on every request. Revoked or deleted tokens are immediately invalidated.
- **Security**: All protected endpoints use consistent authentication logic. Unauthorized or revoked tokens receive a 401 error. All sensitive actions require authentication.
- **Logs & Status**: View service logs and status (PID, uptime, etc) directly from the dashboard.
To get started, run:
```
micro server
```
Then browse to [localhost:8080](http://localhost:8080) and log in with the default admin account (`admin`/`micro`).
> **Note:** See the `/api` page for details on API authentication and how to generate tokens for use with the HTTP API
+225
View File
@@ -0,0 +1,225 @@
# Micro [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
A Go microservices toolkit
## Overview
Micro is a toolkit for Go microservices development. It provides the foundation for building services in the cloud.
The core of Micro is the [Go Micro](https://github.com/micro/go-micro) framework, which developers import and use in their code to
write services. Surrounding this we introduce a number of tools to make it easy to serve and consume services.
## Install the CLI
Install `micro` via `go install`
```
go install go-micro.dev/v5/cmd/micro@v5.13.0
```
> **Note:** Use a specific version instead of `@latest` to avoid module path conflicts. See [releases](https://github.com/micro/go-micro/releases) for the latest version.
Or via install script
```
wget -q https://raw.githubusercontent.com/micro/micro/master/scripts/install.sh -O - | /bin/bash
```
For releases see the [latest](https://go-micro.dev/releases/latest) tag
## Create a service
Create your service (all setup is now automatic!):
```
micro new helloworld
```
This will:
- Create a new service in the `helloworld` directory
- Automatically run `go mod tidy` and `make proto` for you
- Show the updated project tree including generated files
- Warn you if `protoc` is not installed, with install instructions
## Run the service
Run the service
```
micro run
```
List services to see it's running and registered itself
```
micro services
```
## Describe the service
Describe the service to see available endpoints
```
micro describe helloworld
```
Output
```
{
"name": "helloworld",
"version": "latest",
"metadata": null,
"endpoints": [
{
"request": {
"name": "Request",
"type": "Request",
"values": [
{
"name": "name",
"type": "string",
"values": null
}
]
},
"response": {
"name": "Response",
"type": "Response",
"values": [
{
"name": "msg",
"type": "string",
"values": null
}
]
},
"metadata": {},
"name": "Helloworld.Call"
},
{
"request": {
"name": "Context",
"type": "Context",
"values": null
},
"response": {
"name": "Stream",
"type": "Stream",
"values": null
},
"metadata": {
"stream": "true"
},
"name": "Helloworld.Stream"
}
],
"nodes": [
{
"metadata": {
"broker": "http",
"protocol": "mucp",
"registry": "mdns",
"server": "mucp",
"transport": "http"
},
"id": "helloworld-31e55be7-ac83-4810-89c8-a6192fb3ae83",
"address": "127.0.0.1:39963"
}
]
}
```
## Call the service
Call via RPC endpoint
```
micro call helloworld Helloworld.Call '{"name": "Asim"}'
```
## Create a client
Create a client to call the service
```go
package main
import (
"context"
"fmt"
"go-micro.dev/v5"
)
type Request struct {
Name string
}
type Response struct {
Message string
}
func main() {
client := micro.New("helloworld").Client()
req := client.NewRequest("helloworld", "Helloworld.Call", &Request{Name: "John"})
var rsp Response
err := client.Call(context.TODO(), req, &rsp)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(rsp.Message)
}
```
## Protobuf
Use protobuf for code generation with [protoc-gen-micro](https://go-micro.dev/tree/master/cmd/protoc-gen-micro)
## Server
The micro server is an api and web dashboard that provide a fixed entrypoint for seeing and querying services.
Run it like so
```
micro server
```
Then browse to [localhost:8080](http://localhost:8080)
### API Endpoints
The API provides a fixed HTTP entrypoint for calling services
```
curl http://localhost:8080/api/helloworld/Helloworld/Call -d '{"name": "John"}'
```
See /api for more details and documentation for each service
### Web Dashboard
The web dashboard provides a modern, secure UI for managing and exploring your Micro services. Major features include:
- **Dynamic Service & Endpoint Forms**: Browse all registered services and endpoints. For each endpoint, a dynamic form is generated for easy testing and exploration.
- **API Documentation**: The `/api` page lists all available services and endpoints, with request/response schemas and a sidebar for quick navigation. A documentation banner explains authentication requirements.
- **JWT Authentication**: All login and token management uses a custom JWT utility. Passwords are securely stored with bcrypt. All `/api/x` endpoints and authenticated pages require an `Authorization: Bearer <token>` header (or `micro_token` cookie as fallback).
- **Token Management**: The `/auth/tokens` page allows you to generate, view (obfuscated), and copy JWT tokens. Tokens are stored and can be revoked. When a user is deleted, all their tokens are revoked immediately.
- **User Management**: The `/auth/users` page allows you to create, list, and delete users. Passwords are never shown or stored in plaintext.
- **Token Revocation**: JWT tokens are stored and checked for revocation on every request. Revoked or deleted tokens are immediately invalidated.
- **Security**: All protected endpoints use consistent authentication logic. Unauthorized or revoked tokens receive a 401 error. All sensitive actions require authentication.
- **Logs & Status**: View service logs and status (PID, uptime, etc) directly from the dashboard.
To get started, run:
```
micro server
```
Then browse to [localhost:8080](http://localhost:8080) and log in with the default admin account (`admin`/`micro`).
> **Note:** See the `/api` page for details on API authentication and how to generate tokens for use with the HTTP API
+343
View File
@@ -0,0 +1,343 @@
// Package build provides the micro build command for building service binaries
package build
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
"go-micro.dev/v5/cmd/micro/run/config"
)
// Build builds Go binaries for services
func Build(c *cli.Context) error {
dir := c.Args().Get(0)
if dir == "" {
dir = "."
}
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("failed to get absolute path: %w", err)
}
// Load config
cfg, err := config.Load(absDir)
if err != nil {
return fmt.Errorf("failed to load config: %w", err)
}
// Output directory
outDir := c.String("output")
if outDir == "" {
outDir = filepath.Join(absDir, "bin")
}
if err := os.MkdirAll(outDir, 0755); err != nil {
return fmt.Errorf("failed to create output dir: %w", err)
}
// Target OS/ARCH
targetOS := c.String("os")
targetArch := c.String("arch")
if targetOS == "" {
targetOS = runtime.GOOS
}
if targetArch == "" {
targetArch = runtime.GOARCH
}
if cfg != nil && len(cfg.Services) > 0 {
// Build each service from config
sorted, err := cfg.TopologicalSort()
if err != nil {
return err
}
for _, svc := range sorted {
svcDir := filepath.Join(absDir, svc.Path)
if err := buildService(svc.Name, svcDir, outDir, targetOS, targetArch); err != nil {
return fmt.Errorf("failed to build %s: %w", svc.Name, err)
}
}
} else {
// Build single service from current directory
name := filepath.Base(absDir)
if err := buildService(name, absDir, outDir, targetOS, targetArch); err != nil {
return err
}
}
fmt.Printf("\n✓ Built to %s\n", outDir)
return nil
}
func buildService(name, dir, outDir, targetOS, targetArch string) error {
binName := name
if targetOS == "windows" {
binName += ".exe"
}
outPath := filepath.Join(outDir, binName)
fmt.Printf("Building %s (%s/%s)...\n", name, targetOS, targetArch)
// Build command
buildCmd := exec.Command("go", "build", "-o", outPath, ".")
buildCmd.Dir = dir
buildCmd.Env = append(os.Environ(),
"GOOS="+targetOS,
"GOARCH="+targetArch,
"CGO_ENABLED=0",
)
buildCmd.Stdout = os.Stdout
buildCmd.Stderr = os.Stderr
if err := buildCmd.Run(); err != nil {
return fmt.Errorf("go build failed: %w", err)
}
fmt.Printf("✓ %s\n", outPath)
return nil
}
// Docker builds container images (optional)
func Docker(c *cli.Context) error {
dir := c.Args().Get(0)
if dir == "" {
dir = "."
}
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("failed to get absolute path: %w", err)
}
cfg, err := config.Load(absDir)
if err != nil {
return fmt.Errorf("failed to load config: %w", err)
}
tag := c.String("tag")
if tag == "" {
tag = "latest"
}
registry := c.String("registry")
push := c.Bool("push")
if cfg != nil && len(cfg.Services) > 0 {
for name, svc := range cfg.Services {
svcDir := filepath.Join(absDir, svc.Path)
if err := buildDockerImage(name, svcDir, svc.Port, tag, registry, push); err != nil {
return fmt.Errorf("failed to build %s: %w", name, err)
}
}
} else {
name := filepath.Base(absDir)
if err := buildDockerImage(name, absDir, 8080, tag, registry, push); err != nil {
return err
}
}
return nil
}
const dockerfileTemplate = `FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /service .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /service /service
EXPOSE %d
CMD ["/service"]
`
func buildDockerImage(name, dir string, port int, tag, registry string, push bool) error {
if port == 0 {
port = 8080
}
// Generate Dockerfile if not exists
dockerfilePath := filepath.Join(dir, "Dockerfile")
if _, err := os.Stat(dockerfilePath); os.IsNotExist(err) {
fmt.Printf("Generating Dockerfile for %s...\n", name)
dockerfile := fmt.Sprintf(dockerfileTemplate, port)
if err := os.WriteFile(dockerfilePath, []byte(dockerfile), 0644); err != nil {
return fmt.Errorf("failed to write Dockerfile: %w", err)
}
}
imageName := name + ":" + tag
if registry != "" {
imageName = registry + "/" + imageName
}
fmt.Printf("Building %s...\n", imageName)
buildCmd := exec.Command("docker", "build", "-t", imageName, dir)
buildCmd.Stdout = os.Stdout
buildCmd.Stderr = os.Stderr
if err := buildCmd.Run(); err != nil {
return fmt.Errorf("docker build failed: %w", err)
}
fmt.Printf("✓ Built %s\n", imageName)
if push {
fmt.Printf("Pushing %s...\n", imageName)
pushCmd := exec.Command("docker", "push", imageName)
pushCmd.Stdout = os.Stdout
pushCmd.Stderr = os.Stderr
if err := pushCmd.Run(); err != nil {
return fmt.Errorf("docker push failed: %w", err)
}
fmt.Printf("✓ Pushed %s\n", imageName)
}
return nil
}
// Compose generates docker-compose.yml (optional)
func Compose(c *cli.Context) error {
dir := c.Args().Get(0)
if dir == "" {
dir = "."
}
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("failed to get absolute path: %w", err)
}
cfg, err := config.Load(absDir)
if err != nil {
return fmt.Errorf("failed to load config: %w", err)
}
if cfg == nil || len(cfg.Services) == 0 {
return fmt.Errorf("no services found in micro.mu or micro.json")
}
registry := c.String("registry")
tag := c.String("tag")
if tag == "" {
tag = "latest"
}
var sb strings.Builder
sb.WriteString("# Generated by micro build --compose\n")
sb.WriteString("version: '3.8'\n\nservices:\n")
sorted, err := cfg.TopologicalSort()
if err != nil {
return err
}
for _, svc := range sorted {
imageName := svc.Name + ":" + tag
if registry != "" {
imageName = registry + "/" + imageName
}
sb.WriteString(fmt.Sprintf(" %s:\n", svc.Name))
sb.WriteString(fmt.Sprintf(" image: %s\n", imageName))
if svc.Port > 0 {
sb.WriteString(fmt.Sprintf(" ports:\n - \"%d:%d\"\n", svc.Port, svc.Port))
}
if len(svc.Depends) > 0 {
sb.WriteString(" depends_on:\n")
for _, dep := range svc.Depends {
sb.WriteString(fmt.Sprintf(" - %s\n", dep))
}
}
sb.WriteString(" environment:\n - MICRO_REGISTRY=mdns\n\n")
}
output := filepath.Join(absDir, "docker-compose.yml")
if err := os.WriteFile(output, []byte(sb.String()), 0644); err != nil {
return fmt.Errorf("failed to write docker-compose.yml: %w", err)
}
fmt.Printf("✓ Generated %s\n", output)
return nil
}
func init() {
cmd.Register(&cli.Command{
Name: "build",
Usage: "Build Go binaries for services",
Description: `Build compiles Go binaries for your services.
With a micro.mu config, builds all services. Without, builds the current directory.
Output goes to ./bin/ by default.
Examples:
micro build # Build for current OS/arch
micro build --os linux # Cross-compile for Linux
micro build --os linux --arch arm64 # For ARM64
micro build --output ./dist # Custom output directory
Docker (optional):
micro build --docker # Build container images
micro build --docker --push # Build and push
micro build --compose # Generate docker-compose.yml`,
Action: func(c *cli.Context) error {
if c.Bool("docker") {
return Docker(c)
}
if c.Bool("compose") {
return Compose(c)
}
return Build(c)
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "output",
Aliases: []string{"o"},
Usage: "Output directory (default: ./bin)",
},
&cli.StringFlag{
Name: "os",
Usage: "Target OS (linux, darwin, windows)",
},
&cli.StringFlag{
Name: "arch",
Usage: "Target architecture (amd64, arm64)",
},
// Docker options (optional)
&cli.BoolFlag{
Name: "docker",
Usage: "Build Docker container images instead",
},
&cli.StringFlag{
Name: "tag",
Aliases: []string{"t"},
Usage: "Docker image tag (default: latest)",
Value: "latest",
},
&cli.StringFlag{
Name: "registry",
Aliases: []string{"r"},
Usage: "Docker registry (e.g., docker.io/myuser)",
},
&cli.BoolFlag{
Name: "push",
Usage: "Push Docker images after building",
},
&cli.BoolFlag{
Name: "compose",
Usage: "Generate docker-compose.yml",
},
},
})
}
+191
View File
@@ -0,0 +1,191 @@
package microcli
import (
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/client"
"go-micro.dev/v5/cmd"
"go-micro.dev/v5/codec/bytes"
"go-micro.dev/v5/genai"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/cmd/micro/cli/new"
"go-micro.dev/v5/cmd/micro/cli/util"
// Import packages that register commands via init()
_ "go-micro.dev/v5/cmd/micro/cli/build"
_ "go-micro.dev/v5/cmd/micro/cli/deploy"
_ "go-micro.dev/v5/cmd/micro/cli/init"
_ "go-micro.dev/v5/cmd/micro/cli/remote"
)
var (
// version is set by the release action
// this is the default for local builds
version = "5.0.0-dev"
)
func genProtoHandler(c *cli.Context) error {
cmd := exec.Command("find", ".", "-name", "*.proto", "-exec", "protoc", "--proto_path=.", "--micro_out=.", "--go_out=.", `{}`, `;`)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func genTextHandler(c *cli.Context) error {
prompt := c.String("prompt")
if len(prompt) == 0 {
return nil
}
gen := genai.DefaultGenAI
if gen.String() == "noop" {
return nil
}
ctx := context.Background()
res, err := gen.Generate(ctx, prompt)
if err != nil {
return err
}
fmt.Println(res.Text)
return nil
}
func init() {
cmd.Register([]*cli.Command{
{
Name: "new",
Usage: "Create a new service",
Action: new.Run,
},
{
Name: "gen",
Usage: "Generate various things",
Subcommands: []*cli.Command{
{
Name: "text",
Usage: "Generate text via an LLM",
Action: genTextHandler,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "prompt",
Aliases: []string{"p"},
Usage: "The prompt to generate text from",
},
},
},
{
Name: "proto",
Usage: "Generate proto requires protoc and protoc-gen-micro",
Action: genProtoHandler,
},
},
},
{
Name: "services",
Usage: "List available services",
Action: func(ctx *cli.Context) error {
services, err := registry.ListServices()
if err != nil {
return err
}
for _, service := range services {
fmt.Println(service.Name)
}
return nil
},
},
{
Name: "call",
Usage: "Call a service",
Action: func(ctx *cli.Context) error {
args := ctx.Args()
if args.Len() < 2 {
return fmt.Errorf("Usage: [service] [endpoint] [request]")
}
service := args.Get(0)
endpoint := args.Get(1)
request := `{}`
if args.Len() == 3 {
request = args.Get(2)
}
req := client.NewRequest(service, endpoint, &bytes.Frame{Data: []byte(request)})
var rsp bytes.Frame
err := client.Call(context.TODO(), req, &rsp)
if err != nil {
return err
}
fmt.Print(string(rsp.Data))
return nil
},
},
{
Name: "describe",
Usage: "Describe a service",
Action: func(ctx *cli.Context) error {
args := ctx.Args()
if args.Len() != 1 {
return fmt.Errorf("Usage: [service]")
}
service := args.Get(0)
services, err := registry.GetService(service)
if err != nil {
return err
}
if len(services) == 0 {
return nil
}
b, _ := json.MarshalIndent(services[0], "", " ")
fmt.Println(string(b))
return nil
},
},
// Note: The following commands are registered in their respective packages:
// - status, logs, stop: remote/remote.go
// - build: build/build.go
// - deploy: deploy/deploy.go
// - init: init/init.go
}...)
cmd.App().Action = func(c *cli.Context) error {
if c.Args().Len() == 0 {
return nil
}
v, err := exec.LookPath("micro-" + c.Args().First())
if err == nil {
ce := exec.Command(v, c.Args().Slice()[1:]...)
ce.Stdout = os.Stdout
ce.Stderr = os.Stderr
return ce.Run()
}
command := c.Args().Get(0)
args := c.Args().Slice()
if srv, err := util.LookupService(command); err != nil {
return util.CliError(err)
} else if srv != nil && util.ShouldRenderHelp(args) {
return cli.Exit(util.FormatServiceUsage(srv, c), 0)
} else if srv != nil {
err := util.CallService(srv, args)
return util.CliError(err)
}
return nil
}
}
+447
View File
@@ -0,0 +1,447 @@
// Package deploy provides the micro deploy command for deploying services
package deploy
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
"go-micro.dev/v5/cmd/micro/run/config"
)
const (
defaultRemotePath = "/opt/micro"
)
// Deploy deploys services to a target
func Deploy(c *cli.Context) error {
// Get target from args or flag
target := c.Args().First()
if target == "" {
target = c.String("ssh")
}
// Load config to check for deploy targets
dir := "."
absDir, _ := filepath.Abs(dir)
cfg, _ := config.Load(absDir)
// If still no target, check config for named targets
if target == "" && cfg != nil && len(cfg.Deploy) > 0 {
// Show available targets
return showDeployTargets(cfg)
}
if target == "" {
return showDeployHelp()
}
// Check if target is a named target from config
if cfg != nil {
if dt, ok := cfg.Deploy[target]; ok {
target = dt.SSH
}
}
return deploySSH(c, target, cfg)
}
func showDeployHelp() error {
return fmt.Errorf(`No deployment target specified.
To deploy, you need a server running micro. Quick setup:
1. On your server (Ubuntu/Debian):
ssh user@your-server
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
2. Then deploy from here:
micro deploy user@your-server
Or add to micro.mu:
deploy prod
ssh user@your-server
Run 'micro deploy --help' for more options.`)
}
func showDeployTargets(cfg *config.Config) error {
var sb strings.Builder
sb.WriteString("Available deploy targets:\n\n")
for name, dt := range cfg.Deploy {
sb.WriteString(fmt.Sprintf(" %s -> %s\n", name, dt.SSH))
}
sb.WriteString("\nDeploy with: micro deploy <target>")
return fmt.Errorf("%s", sb.String())
}
func deploySSH(c *cli.Context, target string, cfg *config.Config) error {
dir := c.Args().Get(1)
if dir == "" {
dir = "."
}
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("failed to get absolute path: %w", err)
}
// Load config if not passed
if cfg == nil {
cfg, _ = config.Load(absDir)
}
remotePath := c.String("path")
if remotePath == "" {
remotePath = defaultRemotePath
}
fmt.Printf("Deploying to %s...\n\n", target)
// Step 1: Check SSH connectivity
fmt.Print(" Checking SSH connection... ")
if err := checkSSH(target); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Println("\u2713")
// Step 2: Check server is initialized
fmt.Print(" Checking server setup... ")
if err := checkServerInit(target, remotePath); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Println("\u2713")
// Step 3: Build binaries
var services []string
if cfg != nil && len(cfg.Services) > 0 {
sorted, err := cfg.TopologicalSort()
if err != nil {
return err
}
for _, svc := range sorted {
services = append(services, svc.Name)
}
} else {
services = []string{filepath.Base(absDir)}
}
fmt.Printf(" Building binaries... ")
if err := buildBinaries(absDir, cfg, c.Bool("build")); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Printf("\u2713 %s\n", strings.Join(services, ", "))
// Step 4: Copy binaries
fmt.Printf(" Copying binaries... ")
if err := copyBinaries(target, filepath.Join(absDir, "bin"), remotePath); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Printf("\u2713 %d services\n", len(services))
// Step 5: Setup and restart services via systemd
fmt.Printf(" Updating systemd... ")
if err := setupSystemdServices(target, remotePath, services); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Printf("\u2713 %s\n", strings.Join(prefixServices(services), ", "))
// Step 6: Restart services
fmt.Printf(" Restarting services... ")
if err := restartServices(target, services); err != nil {
fmt.Println("\u2717")
return err
}
fmt.Println("\u2713")
// Step 7: Check health
fmt.Printf(" Checking health... ")
time.Sleep(2 * time.Second) // Give services time to start
healthy, unhealthy := checkServicesHealth(target, services)
if len(unhealthy) > 0 {
fmt.Printf("\u26a0 %d/%d healthy\n", len(healthy), len(services))
} else {
fmt.Println("\u2713 all healthy")
}
fmt.Println()
fmt.Printf("\u2713 Deployed to %s\n", target)
fmt.Println()
fmt.Printf(" Status: micro status --remote %s\n", target)
fmt.Printf(" Logs: micro logs --remote %s\n", target)
if len(unhealthy) > 0 {
fmt.Println()
fmt.Printf("\u26a0 Some services may have issues: %s\n", strings.Join(unhealthy, ", "))
fmt.Printf(" Check logs: micro logs %s --remote %s\n", unhealthy[0], target)
}
return nil
}
func prefixServices(services []string) []string {
result := make([]string, len(services))
for i, s := range services {
result[i] = "micro@" + s
}
return result
}
func checkSSH(host string) error {
testCmd := exec.Command("ssh", "-o", "ConnectTimeout=5", "-o", "BatchMode=yes", host, "echo ok")
output, err := testCmd.CombinedOutput()
if err != nil {
return fmt.Errorf(`
\u2717 Cannot connect to %s
SSH connection failed. Check that:
\u2022 The server is reachable: ping %s
\u2022 SSH is configured: ssh %s
\u2022 Your key is added: ssh-add -l
Common fixes:
\u2022 Add SSH key: ssh-copy-id %s
\u2022 Check hostname in ~/.ssh/config
Error: %s`, host, host, host, host, strings.TrimSpace(string(output)))
}
return nil
}
func checkServerInit(host, remotePath string) error {
checkCmd := fmt.Sprintf("test -f %s/.micro-initialized", remotePath)
sshCmd := exec.Command("ssh", host, checkCmd)
if err := sshCmd.Run(); err != nil {
return fmt.Errorf(`
\u2717 Server not initialized
micro is not set up on %s.
Run this on the server:
ssh %s
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
Or initialize remotely (requires sudo):
micro init --server --remote %s`, host, host, host)
}
return nil
}
func buildBinaries(absDir string, cfg *config.Config, forceBuild bool) error {
binDir := filepath.Join(absDir, "bin")
// Check if we already have binaries and don't need to rebuild
if !forceBuild {
if _, err := os.Stat(binDir); err == nil {
// Check if binaries are for linux
// For now, just rebuild to be safe
}
}
// Always build for linux/amd64
targetOS := "linux"
targetArch := "amd64"
if err := os.MkdirAll(binDir, 0755); err != nil {
return err
}
if cfg != nil && len(cfg.Services) > 0 {
sorted, err := cfg.TopologicalSort()
if err != nil {
return err
}
for _, svc := range sorted {
svcDir := filepath.Join(absDir, svc.Path)
outPath := filepath.Join(binDir, svc.Name)
buildCmd := exec.Command("go", "build", "-o", outPath, ".")
buildCmd.Dir = svcDir
buildCmd.Env = append(os.Environ(),
"GOOS="+targetOS,
"GOARCH="+targetArch,
"CGO_ENABLED=0",
)
if output, err := buildCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to build %s:\n%s", svc.Name, string(output))
}
}
} else {
name := filepath.Base(absDir)
outPath := filepath.Join(binDir, name)
buildCmd := exec.Command("go", "build", "-o", outPath, ".")
buildCmd.Dir = absDir
buildCmd.Env = append(os.Environ(),
"GOOS="+targetOS,
"GOARCH="+targetArch,
"CGO_ENABLED=0",
)
if output, err := buildCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to build:\n%s", string(output))
}
}
return nil
}
func copyBinaries(target, binDir, remotePath string) error {
// Ensure remote bin directory exists
mkdirCmd := exec.Command("ssh", target, fmt.Sprintf("mkdir -p %s/bin", remotePath))
if err := mkdirCmd.Run(); err != nil {
return fmt.Errorf("failed to create remote directory: %w", err)
}
// Use rsync for efficient copy
// --omit-dir-times avoids permission errors on directory timestamps
rsyncArgs := []string{
"-avz", "--delete", "--omit-dir-times",
binDir + "/",
fmt.Sprintf("%s:%s/bin/", target, remotePath),
}
rsyncCmd := exec.Command("rsync", rsyncArgs...)
output, err := rsyncCmd.CombinedOutput()
if err != nil {
outputStr := string(output)
// Fall back to scp if rsync not available
if strings.Contains(outputStr, "command not found") {
scpCmd := exec.Command("scp", "-r", binDir+"/", fmt.Sprintf("%s:%s/bin/", target, remotePath))
if scpOutput, scpErr := scpCmd.CombinedOutput(); scpErr != nil {
return fmt.Errorf("copy failed: %s", string(scpOutput))
}
return nil
}
// rsync exit code 23 means some files failed to transfer, but if we see our files listed, it's ok
// rsync exit code 24 means some files vanished during transfer (harmless)
exitErr, ok := err.(*exec.ExitError)
if ok && (exitErr.ExitCode() == 23 || exitErr.ExitCode() == 24) {
// Check if it's just permission warnings on metadata, not actual file transfer failures
if !strings.Contains(outputStr, "Permission denied (13)") ||
strings.Contains(outputStr, "failed to set times") ||
strings.Contains(outputStr, "chgrp") {
// These are acceptable warnings
return nil
}
}
return fmt.Errorf("copy failed: %s", outputStr)
}
return nil
}
func setupSystemdServices(target, remotePath string, services []string) error {
for _, svc := range services {
// Enable the service using the template
enableCmd := fmt.Sprintf("sudo systemctl enable micro@%s 2>/dev/null || true", svc)
sshCmd := exec.Command("ssh", target, enableCmd)
sshCmd.Run() // Ignore errors, service might already be enabled
}
// Reload systemd
reloadCmd := exec.Command("ssh", target, "sudo systemctl daemon-reload")
if err := reloadCmd.Run(); err != nil {
return fmt.Errorf("failed to reload systemd: %w", err)
}
return nil
}
func restartServices(target string, services []string) error {
for _, svc := range services {
restartCmd := fmt.Sprintf("sudo systemctl restart micro@%s", svc)
sshCmd := exec.Command("ssh", target, restartCmd)
if output, err := sshCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to restart %s: %s", svc, string(output))
}
}
return nil
}
func checkServicesHealth(target string, services []string) (healthy, unhealthy []string) {
for _, svc := range services {
checkCmd := fmt.Sprintf("systemctl is-active micro@%s", svc)
sshCmd := exec.Command("ssh", target, checkCmd)
if err := sshCmd.Run(); err != nil {
unhealthy = append(unhealthy, svc)
} else {
healthy = append(healthy, svc)
}
}
return
}
// Ensure we're not on Windows for deploy
func checkPlatform() error {
if runtime.GOOS == "windows" {
return fmt.Errorf("micro deploy requires SSH and rsync, which work best on Linux/macOS.\nConsider using WSL on Windows.")
}
return nil
}
func init() {
cmd.Register(&cli.Command{
Name: "deploy",
Usage: "Deploy services to a remote server",
Description: `Deploy copies binaries to a remote server and manages them with systemd.
Before deploying, initialize the server:
ssh user@server 'curl -fsSL https://go-micro.dev/install.sh | sh && sudo micro init --server'
Then deploy:
micro deploy user@server
With a micro.mu config, you can define named targets:
deploy prod
ssh user@prod.example.com
deploy staging
ssh user@staging.example.com
Then: micro deploy prod
The deploy process:
1. Builds binaries for linux/amd64
2. Copies to /opt/micro/bin/ via rsync
3. Enables and restarts systemd services
4. Verifies services are healthy`,
Action: func(c *cli.Context) error {
if err := checkPlatform(); err != nil {
return err
}
return Deploy(c)
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "ssh",
Usage: "Deploy target as user@host (can also be positional arg)",
},
&cli.StringFlag{
Name: "path",
Usage: "Remote path (default: /opt/micro)",
Value: "/opt/micro",
},
&cli.BoolFlag{
Name: "build",
Usage: "Force rebuild of binaries",
},
},
})
}
+311
View File
@@ -0,0 +1,311 @@
// Package generate provides code generation commands for micro
package gen
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"text/template"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
"go-micro.dev/v5/genai"
)
var handlerTemplate = `package handler
import (
"context"
log "go-micro.dev/v5/logger"
)
type {{.Name}} struct{}
func New{{.Name}}() *{{.Name}} {
return &{{.Name}}{}
}
{{range .Methods}}
// {{.Name}} handles {{.Name}} requests
func (h *{{$.Name}}) {{.Name}}(ctx context.Context, req *{{.RequestType}}, rsp *{{.ResponseType}}) error {
log.Infof("Received {{$.Name}}.{{.Name}} request")
// TODO: implement
return nil
}
{{end}}
`
var endpointTemplate = `package handler
import (
"context"
"encoding/json"
"net/http"
log "go-micro.dev/v5/logger"
)
// {{.Name}}Request is the request for {{.Name}}
type {{.Name}}Request struct {
// Add request fields here
}
// {{.Name}}Response is the response for {{.Name}}
type {{.Name}}Response struct {
// Add response fields here
}
// {{.Name}} handles HTTP {{.Method}} requests to /{{.Path}}
func {{.Name}}(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
log.Infof("Received {{.Name}} request")
var req {{.Name}}Request
if r.Method != http.MethodGet {
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
}
// TODO: implement handler logic
_ = ctx
_ = req
rsp := {{.Name}}Response{}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(rsp)
}
`
var modelTemplate = `package model
import (
"context"
"time"
)
// {{.Name}} represents a {{lower .Name}} in the system
type {{.Name}} struct {
ID string ` + "`json:\"id\"`" + `
CreatedAt time.Time ` + "`json:\"created_at\"`" + `
UpdatedAt time.Time ` + "`json:\"updated_at\"`" + `
// Add your fields here
}
// {{.Name}}Repository defines the interface for {{lower .Name}} storage
type {{.Name}}Repository interface {
Create(ctx context.Context, m *{{.Name}}) error
Get(ctx context.Context, id string) (*{{.Name}}, error)
Update(ctx context.Context, m *{{.Name}}) error
Delete(ctx context.Context, id string) error
List(ctx context.Context, offset, limit int) ([]*{{.Name}}, error)
}
`
type handlerData struct {
Name string
Methods []methodData
}
type methodData struct {
Name string
RequestType string
ResponseType string
}
type endpointData struct {
Name string
Method string
Path string
}
type modelData struct {
Name string
}
func generateHandler(c *cli.Context) error {
name := c.Args().First()
if name == "" {
return fmt.Errorf("handler name required: micro generate handler <name>")
}
name = strings.Title(strings.ToLower(name))
// Parse methods if provided
methods := []methodData{}
for _, m := range c.StringSlice("method") {
methods = append(methods, methodData{
Name: strings.Title(m),
RequestType: strings.Title(m) + "Request",
ResponseType: strings.Title(m) + "Response",
})
}
if len(methods) == 0 {
methods = []methodData{
{Name: "Handle", RequestType: "Request", ResponseType: "Response"},
}
}
data := handlerData{
Name: name,
Methods: methods,
}
return generateFile("handler", strings.ToLower(name)+".go", handlerTemplate, data)
}
func generateEndpoint(c *cli.Context) error {
name := c.Args().First()
if name == "" {
return fmt.Errorf("endpoint name required: micro generate endpoint <name>")
}
data := endpointData{
Name: strings.Title(strings.ToLower(name)),
Method: strings.ToUpper(c.String("method")),
Path: c.String("path"),
}
if data.Path == "" {
data.Path = strings.ToLower(name)
}
return generateFile("handler", strings.ToLower(name)+"_endpoint.go", endpointTemplate, data)
}
func generateModel(c *cli.Context) error {
name := c.Args().First()
if name == "" {
return fmt.Errorf("model name required: micro generate model <name>")
}
data := modelData{
Name: strings.Title(strings.ToLower(name)),
}
return generateFile("model", strings.ToLower(name)+".go", modelTemplate, data)
}
func generateWithAI(c *cli.Context) error {
prompt := c.Args().First()
if prompt == "" {
return fmt.Errorf("description required: micro generate ai <description>")
}
gen := genai.DefaultGenAI
if gen.String() == "noop" {
return fmt.Errorf("no AI provider configured. Set OPENAI_API_KEY or GEMINI_API_KEY")
}
aiPrompt := fmt.Sprintf(`Generate Go code for a micro service handler based on this description: %s
Use the go-micro.dev/v5 framework. Include:
- Proper imports
- Handler struct with methods
- Context handling
- Logging with go-micro.dev/v5/logger
- Error handling
Only output the Go code, no explanations.`, prompt)
ctx := context.Background()
res, err := gen.Generate(ctx, aiPrompt)
if err != nil {
return fmt.Errorf("AI generation failed: %w", err)
}
fmt.Println(res.Text)
return nil
}
func generateFile(dir, filename, tmplStr string, data interface{}) error {
// Create directory if it doesn't exist
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("failed to create directory %s: %w", dir, err)
}
filepath := filepath.Join(dir, filename)
// Check if file exists
if _, err := os.Stat(filepath); err == nil {
return fmt.Errorf("file %s already exists", filepath)
}
fn := template.FuncMap{
"title": strings.Title,
"lower": strings.ToLower,
}
tmpl, err := template.New("gen").Funcs(fn).Parse(tmplStr)
if err != nil {
return fmt.Errorf("failed to parse template: %w", err)
}
f, err := os.Create(filepath)
if err != nil {
return fmt.Errorf("failed to create file: %w", err)
}
defer f.Close()
if err := tmpl.Execute(f, data); err != nil {
return fmt.Errorf("failed to execute template: %w", err)
}
fmt.Printf("Created %s\n", filepath)
return nil
}
func init() {
cmd.Register(&cli.Command{
Name: "generate",
Usage: "Generate code scaffolding (like Rails generators)",
Aliases: []string{"gen"},
Subcommands: []*cli.Command{
{
Name: "handler",
Usage: "Generate a handler: micro g handler <name>",
Action: generateHandler,
Flags: []cli.Flag{
&cli.StringSliceFlag{
Name: "method",
Aliases: []string{"m"},
Usage: "Methods to generate (can be repeated)",
},
},
},
{
Name: "endpoint",
Usage: "Generate an HTTP endpoint: micro g endpoint <name>",
Action: generateEndpoint,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "method",
Aliases: []string{"m"},
Usage: "HTTP method (GET, POST, etc.)",
Value: "POST",
},
&cli.StringFlag{
Name: "path",
Aliases: []string{"p"},
Usage: "URL path for the endpoint",
},
},
},
{
Name: "model",
Usage: "Generate a model: micro g model <name>",
Action: generateModel,
},
{
Name: "ai",
Usage: "Generate code using AI: micro g ai <description>",
Action: generateWithAI,
},
},
})
}
+269
View File
@@ -0,0 +1,269 @@
// Package initcmd provides the micro init command for server setup
package initcmd
import (
"fmt"
"os"
"os/exec"
"os/user"
"path/filepath"
"runtime"
"strings"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
)
const systemdTemplate = `[Unit]
Description=Micro service: %%i
After=network.target
[Service]
Type=simple
User=%s
Group=%s
WorkingDirectory=%s
ExecStart=%s/bin/%%i
Restart=on-failure
RestartSec=5
EnvironmentFile=-%s/config/%%i.env
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=micro-%%i
# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=%s/data
[Install]
WantedBy=multi-user.target
`
// Init initializes a server to receive micro deployments
func Init(c *cli.Context) error {
if !c.Bool("server") {
return fmt.Errorf("usage: micro init --server\n\nInitialize this machine to receive micro deployments")
}
// Check if we're on Linux
if runtime.GOOS != "linux" {
return fmt.Errorf("micro init --server is only supported on Linux")
}
// Check for remote init
remoteHost := c.String("remote")
if remoteHost != "" {
return initRemote(c, remoteHost)
}
basePath := c.String("path")
userName := c.String("user")
fmt.Println("Initializing micro server...")
fmt.Println()
// Check if running as root (needed for systemd and creating users)
if os.Geteuid() != 0 {
return fmt.Errorf(`micro init --server requires root privileges.
Run with sudo:
sudo micro init --server`)
}
// Create user if needed
if userName == "micro" {
if err := createMicroUser(); err != nil {
return err
}
}
// Create directories
fmt.Println("Creating directories:")
dirs := []string{
filepath.Join(basePath, "bin"),
filepath.Join(basePath, "data"),
filepath.Join(basePath, "config"),
}
for _, dir := range dirs {
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("failed to create %s: %w", dir, err)
}
fmt.Printf(" ✓ %s\n", dir)
}
// Set ownership
if userName != "root" {
u, err := user.Lookup(userName)
if err != nil {
return fmt.Errorf("user %s not found: %w", userName, err)
}
// chown -R user:user /opt/micro
chownCmd := exec.Command("chown", "-R", fmt.Sprintf("%s:%s", u.Username, u.Username), basePath)
if err := chownCmd.Run(); err != nil {
return fmt.Errorf("failed to set ownership: %w", err)
}
}
fmt.Println()
// Create systemd template
fmt.Println("Creating systemd template:")
unitContent := fmt.Sprintf(systemdTemplate, userName, userName, basePath, basePath, basePath, basePath)
unitPath := "/etc/systemd/system/micro@.service"
if err := os.WriteFile(unitPath, []byte(unitContent), 0644); err != nil {
return fmt.Errorf("failed to write systemd unit: %w", err)
}
fmt.Printf(" ✓ %s\n", unitPath)
// Reload systemd
reloadCmd := exec.Command("systemctl", "daemon-reload")
if err := reloadCmd.Run(); err != nil {
return fmt.Errorf("failed to reload systemd: %w", err)
}
fmt.Println(" ✓ systemd daemon-reload")
// Write marker file so deploy can detect initialization
markerPath := filepath.Join(basePath, ".micro-initialized")
if err := os.WriteFile(markerPath, []byte("1\n"), 0644); err != nil {
return fmt.Errorf("failed to write marker: %w", err)
}
fmt.Println()
fmt.Println("Server ready!")
fmt.Println()
fmt.Println(" Deploy from your machine:")
fmt.Printf(" micro deploy user@%s\n", getHostname())
fmt.Println()
fmt.Println(" Manage services:")
fmt.Println(" sudo systemctl status micro@myservice")
fmt.Println(" sudo journalctl -u micro@myservice -f")
fmt.Println()
return nil
}
func createMicroUser() error {
// Check if user exists
if _, err := user.Lookup("micro"); err == nil {
return nil // user already exists
}
fmt.Println("Creating micro user:")
createCmd := exec.Command("useradd", "--system", "--no-create-home", "--shell", "/bin/false", "micro")
if err := createCmd.Run(); err != nil {
// Check if it's just because user already exists
if _, lookupErr := user.Lookup("micro"); lookupErr == nil {
return nil
}
return fmt.Errorf("failed to create micro user: %w", err)
}
fmt.Println(" ✓ Created user 'micro'")
return nil
}
func initRemote(c *cli.Context, host string) error {
fmt.Printf("Initializing micro on %s...\n\n", host)
// Check SSH connectivity first
if err := checkSSH(host); err != nil {
return err
}
basePath := c.String("path")
userName := c.String("user")
// Run micro init --server on remote
initCmd := fmt.Sprintf("sudo micro init --server --path %s --user %s", basePath, userName)
sshCmd := exec.Command("ssh", host, initCmd)
sshCmd.Stdout = os.Stdout
sshCmd.Stderr = os.Stderr
if err := sshCmd.Run(); err != nil {
return fmt.Errorf("remote init failed: %w", err)
}
return nil
}
func checkSSH(host string) error {
// Quick SSH test
testCmd := exec.Command("ssh", "-o", "ConnectTimeout=5", "-o", "BatchMode=yes", host, "echo ok")
output, err := testCmd.CombinedOutput()
if err != nil {
return fmt.Errorf(`✗ Cannot connect to %s
SSH connection failed. Check that:
• The server is reachable: ping %s
• SSH is configured: ssh %s
• Your key is added: ssh-add -l
Common fixes:
• Add SSH key: ssh-copy-id %s
• Check hostname in ~/.ssh/config
Error: %s`, host, host, host, host, strings.TrimSpace(string(output)))
}
return nil
}
func getHostname() string {
name, err := os.Hostname()
if err != nil {
return "this-server"
}
return name
}
func init() {
cmd.Register(&cli.Command{
Name: "init",
Usage: "Initialize micro for development or server deployment",
Description: `Initialize micro on a server to receive deployments.
Server setup:
sudo micro init --server
This creates:
• /opt/micro/bin/ - service binaries
• /opt/micro/data/ - persistent data
• /opt/micro/config/ - environment files
• systemd template for managing services
Remote setup:
micro init --server --remote user@host
After init, deploy with:
micro deploy user@host`,
Action: Init,
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "server",
Usage: "Initialize as a deployment server",
},
&cli.StringFlag{
Name: "path",
Usage: "Base path for micro (default: /opt/micro)",
Value: "/opt/micro",
},
&cli.StringFlag{
Name: "user",
Usage: "User to run services as (default: micro)",
Value: "micro",
},
&cli.StringFlag{
Name: "remote",
Usage: "Initialize a remote server via SSH",
},
},
})
}
+257
View File
@@ -0,0 +1,257 @@
// Package new generates micro service templates
package new
import (
"fmt"
"go/build"
"os"
"os/exec"
"path"
"path/filepath"
"runtime"
"strings"
"text/template"
"time"
"github.com/urfave/cli/v2"
"github.com/xlab/treeprint"
tmpl "go-micro.dev/v5/cmd/micro/cli/new/template"
)
func protoComments(goDir, alias string) []string {
return []string{
"\ndownload protoc zip packages (protoc-$VERSION-$PLATFORM.zip) and install:\n",
"visit https://github.com/protocolbuffers/protobuf/releases",
"\ncompile the proto file " + alias + ".proto:\n",
"cd " + alias,
"go mod tidy",
"make proto\n",
}
}
type config struct {
// foo
Alias string
// github.com/micro/foo
Dir string
// $GOPATH/src/github.com/micro/foo
GoDir string
// $GOPATH
GoPath string
// UseGoPath
UseGoPath bool
// Files
Files []file
// Comments
Comments []string
}
type file struct {
Path string
Tmpl string
}
func write(c config, file, tmpl string) error {
fn := template.FuncMap{
"title": func(s string) string {
return strings.ReplaceAll(strings.Title(s), "-", "")
},
"dehyphen": func(s string) string {
return strings.ReplaceAll(s, "-", "")
},
"lower": func(s string) string {
return strings.ToLower(s)
},
}
f, err := os.Create(file)
if err != nil {
return err
}
defer f.Close()
t, err := template.New("f").Funcs(fn).Parse(tmpl)
if err != nil {
return err
}
return t.Execute(f, c)
}
func create(c config) error {
// check if dir exists
if _, err := os.Stat(c.Dir); !os.IsNotExist(err) {
return fmt.Errorf("%s already exists", c.Dir)
}
fmt.Printf("Creating service %s\n\n", c.Alias)
t := treeprint.New()
// write the files
for _, file := range c.Files {
f := filepath.Join(c.Dir, file.Path)
dir := filepath.Dir(f)
if _, err := os.Stat(dir); os.IsNotExist(err) {
if err := os.MkdirAll(dir, 0755); err != nil {
return err
}
}
addFileToTree(t, file.Path)
if err := write(c, f, file.Tmpl); err != nil {
return err
}
}
// print tree
fmt.Println(t.String())
for _, comment := range c.Comments {
fmt.Println(comment)
}
// just wait
<-time.After(time.Millisecond * 250)
return nil
}
func addFileToTree(root treeprint.Tree, file string) {
split := strings.Split(file, "/")
curr := root
for i := 0; i < len(split)-1; i++ {
n := curr.FindByValue(split[i])
if n != nil {
curr = n
} else {
curr = curr.AddBranch(split[i])
}
}
if curr.FindByValue(split[len(split)-1]) == nil {
curr.AddNode(split[len(split)-1])
}
}
func Run(ctx *cli.Context) error {
dir := ctx.Args().First()
if len(dir) == 0 {
fmt.Println("specify service name")
return nil
}
// check if the path is absolute, we don't want this
// we want to a relative path so we can install in GOPATH
if path.IsAbs(dir) {
fmt.Println("require relative path as service will be installed in GOPATH")
return nil
}
// Check for protoc
if _, err := exec.LookPath("protoc"); err != nil {
fmt.Println("WARNING: protoc is not installed or not in your PATH.")
fmt.Println("Please install protoc from https://github.com/protocolbuffers/protobuf/releases")
fmt.Println("After installing, re-run 'make proto' in your service directory if needed.")
}
var goPath string
var goDir string
goPath = build.Default.GOPATH
// don't know GOPATH, runaway....
if len(goPath) == 0 {
fmt.Println("unknown GOPATH")
return nil
}
// attempt to split path if not windows
if runtime.GOOS == "windows" {
goPath = strings.Split(goPath, ";")[0]
} else {
goPath = strings.Split(goPath, ":")[0]
}
goDir = filepath.Join(goPath, "src", path.Clean(dir))
c := config{
Alias: dir,
Comments: nil, // Remove redundant protoComments
Dir: dir,
GoDir: goDir,
GoPath: goPath,
UseGoPath: false,
Files: []file{
{"main.go", tmpl.MainSRV},
{"handler/" + dir + ".go", tmpl.HandlerSRV},
{"proto/" + dir + ".proto", tmpl.ProtoSRV},
{"Makefile", tmpl.Makefile},
{"README.md", tmpl.Readme},
{".gitignore", tmpl.GitIgnore},
},
}
// set gomodule
if os.Getenv("GO111MODULE") != "off" {
c.Files = append(c.Files, file{"go.mod", tmpl.Module})
}
// create the files
if err := create(c); err != nil {
return err
}
// Run go mod tidy and make proto
fmt.Println("\nRunning 'go mod tidy' and 'make proto'...")
if err := runInDir(dir, "go mod tidy"); err != nil {
fmt.Printf("Error running 'go mod tidy': %v\n", err)
}
if err := runInDir(dir, "make proto"); err != nil {
fmt.Printf("Error running 'make proto': %v\n", err)
}
// Print updated tree including generated files
fmt.Println("\nProject structure after 'make proto':")
printTree(dir)
fmt.Println("\nService created successfully! Start coding in your new service directory.")
return nil
}
func runInDir(dir, cmd string) error {
parts := strings.Fields(cmd)
c := exec.Command(parts[0], parts[1:]...)
c.Dir = dir
c.Stdout = os.Stdout
c.Stderr = os.Stderr
return c.Run()
}
func printTree(dir string) {
t := treeprint.New()
walk := func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
rel, _ := filepath.Rel(dir, path)
if rel == "." {
return nil
}
parts := strings.Split(rel, string(os.PathSeparator))
curr := t
for i := 0; i < len(parts)-1; i++ {
n := curr.FindByValue(parts[i])
if n != nil {
curr = n
} else {
curr = curr.AddBranch(parts[i])
}
}
if !info.IsDir() {
curr.AddNode(parts[len(parts)-1])
}
return nil
}
filepath.Walk(dir, walk)
fmt.Println(t.String())
}
+66
View File
@@ -0,0 +1,66 @@
package template
var (
HandlerSRV = `package handler
import (
"context"
log "go-micro.dev/v5/logger"
pb "{{.Dir}}/proto"
)
type {{title .Alias}} struct{}
// Return a new handler
func New() *{{title .Alias}} {
return &{{title .Alias}}{}
}
// Call is a single request handler called via client.Call or the generated client code
func (e *{{title .Alias}}) Call(ctx context.Context, req *pb.Request, rsp *pb.Response) error {
log.Info("Received {{title .Alias}}.Call request")
rsp.Msg = "Hello " + req.Name
return nil
}
// Stream is a server side stream handler called via client.Stream or the generated client code
func (e *{{title .Alias}}) Stream(ctx context.Context, req *pb.StreamingRequest, stream pb.{{title .Alias}}_StreamStream) error {
log.Infof("Received {{title .Alias}}.Stream request with count: %d", req.Count)
for i := 0; i < int(req.Count); i++ {
log.Infof("Responding: %d", i)
if err := stream.Send(&pb.StreamingResponse{
Count: int64(i),
}); err != nil {
return err
}
}
return nil
}
`
SubscriberSRV = `package subscriber
import (
"context"
log "go-micro.dev/v5/logger"
pb "{{.Dir}}/proto"
)
type {{title .Alias}} struct{}
func (e *{{title .Alias}}) Handle(ctx context.Context, msg *pb.Message) error {
log.Info("Handler Received message: ", msg.Say)
return nil
}
func Handler(ctx context.Context, msg *pb.Message) error {
log.Info("Function Received message: ", msg.Say)
return nil
}
`
)
+7
View File
@@ -0,0 +1,7 @@
package template
var (
GitIgnore = `
{{.Alias}}
`
)
+27
View File
@@ -0,0 +1,27 @@
package template
var (
MainSRV = `package main
import (
"{{.Dir}}/handler"
pb "{{.Dir}}/proto"
"go-micro.dev/v5"
)
func main() {
// Create service
service := micro.New("{{lower .Alias}}")
// Initialize service
service.Init()
// Register handler
pb.Register{{title .Alias}}Handler(service.Server(), handler.New())
// Run service
service.Run()
}
`
)
+59
View File
@@ -0,0 +1,59 @@
package template
var Makefile = `.PHONY: proto build run test clean docker
# Generate protobuf files
proto:
protoc --proto_path=. --micro_out=. --go_out=. proto/*.proto
# Build the service
build:
go build -o bin/{{.Alias}} .
# Run the service
run:
go run .
# Run with hot reload (requires air: go install github.com/air-verse/air@latest)
dev:
air
# Run tests
test:
go test -v ./...
# Run tests with coverage
test-coverage:
go test -v -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html
# Clean build artifacts
clean:
rm -rf bin/ coverage.out coverage.html
# Build Docker image
docker:
docker build -t {{.Alias}}:latest .
# Run with Docker Compose
docker-up:
docker-compose up -d
# Stop Docker Compose
docker-down:
docker-compose down
# Lint code
lint:
golangci-lint run ./...
# Format code
fmt:
go fmt ./...
goimports -w .
# Update dependencies
deps:
go mod tidy
go mod download
`
+14
View File
@@ -0,0 +1,14 @@
package template
var (
Module = `module {{.Dir}}
go 1.18
require (
go-micro.dev/v5 latest
github.com/golang/protobuf latest
google.golang.org/protobuf latest
)
`
)
+35
View File
@@ -0,0 +1,35 @@
package template
var (
ProtoSRV = `syntax = "proto3";
package {{dehyphen .Alias}};
option go_package = "./proto;{{dehyphen .Alias}}";
service {{title .Alias}} {
rpc Call(Request) returns (Response) {}
rpc Stream(StreamingRequest) returns (stream StreamingResponse) {}
}
message Message {
string say = 1;
}
message Request {
string name = 1;
}
message Response {
string msg = 1;
}
message StreamingRequest {
int64 count = 1;
}
message StreamingResponse {
int64 count = 1;
}
`
)
+30
View File
@@ -0,0 +1,30 @@
package template
var (
Readme = `# {{title .Alias}} Service
This is the {{title .Alias}} service
Generated with
` + "```" +
`
micro new {{.Alias}}
` + "```" + `
## Usage
Generate the proto code
` + "```" +
`
make proto
` + "```" + `
Run the service
` + "```" +
`
micro run .
` + "```"
)
+367
View File
@@ -0,0 +1,367 @@
// Package remote provides remote server operations for micro
package remote
import (
"bufio"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"syscall"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
)
const defaultRemotePath = "/opt/micro"
// Status shows status of services (local or remote)
func Status(c *cli.Context) error {
remoteHost := c.String("remote")
if remoteHost != "" {
return remoteStatus(remoteHost)
}
return localStatus(c)
}
func localStatus(c *cli.Context) error {
homeDir, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("failed to get home dir: %w", err)
}
runDir := filepath.Join(homeDir, "micro", "run")
files, err := os.ReadDir(runDir)
if err != nil {
fmt.Println("No services running locally.")
fmt.Println("\nStart services with: micro run")
return nil
}
var hasServices bool
fmt.Printf("%-20s %-10s %-8s %s\n", "SERVICE", "STATUS", "PID", "DIRECTORY")
fmt.Println(strings.Repeat("-", 70))
for _, f := range files {
if f.IsDir() || !strings.HasSuffix(f.Name(), ".pid") {
continue
}
hasServices = true
service := f.Name()[:len(f.Name())-4]
pidFilePath := filepath.Join(runDir, f.Name())
pidFile, err := os.Open(pidFilePath)
if err != nil {
continue
}
var pid int
var dir string
scanner := bufio.NewScanner(pidFile)
if scanner.Scan() {
fmt.Sscanf(scanner.Text(), "%d", &pid)
}
if scanner.Scan() {
dir = scanner.Text()
}
pidFile.Close()
status := "\u2717 stopped"
if pid > 0 {
proc, err := os.FindProcess(pid)
if err == nil {
if err := proc.Signal(syscall.Signal(0)); err == nil {
status = "\u25cf running"
}
}
}
fmt.Printf("%-20s %-10s %-8d %s\n", service, status, pid, dir)
}
if !hasServices {
fmt.Println("No services running locally.")
fmt.Println("\nStart services with: micro run")
}
return nil
}
func remoteStatus(host string) error {
// Get list of micro services via systemctl
listCmd := exec.Command("ssh", host, "systemctl list-units 'micro@*' --no-legend --no-pager 2>/dev/null || true")
output, err := listCmd.Output()
if err != nil {
return fmt.Errorf("failed to get status from %s: %w", host, err)
}
lines := strings.Split(strings.TrimSpace(string(output)), "\n")
if len(lines) == 0 || (len(lines) == 1 && lines[0] == "") {
fmt.Printf("%s\n", host)
fmt.Println(strings.Repeat("\u2501", 50))
fmt.Println("\nNo services deployed.")
fmt.Println("\nDeploy with: micro deploy " + host)
return nil
}
fmt.Printf("%s\n", host)
fmt.Println(strings.Repeat("\u2501", 50))
fmt.Println()
for _, line := range lines {
if line == "" {
continue
}
parts := strings.Fields(line)
if len(parts) < 4 {
continue
}
unit := parts[0]
loadState := parts[1]
activeState := parts[2]
subState := parts[3]
// Extract service name from micro@servicename.service
serviceName := strings.TrimPrefix(unit, "micro@")
serviceName = strings.TrimSuffix(serviceName, ".service")
// Get more details
statusIcon := "\u25cf"
statusText := subState
if activeState != "active" || subState != "running" {
statusIcon = "\u2717"
}
_ = loadState // unused but parsed
fmt.Printf(" %-15s %s %s\n", serviceName, statusIcon, statusText)
}
fmt.Println()
return nil
}
// Logs shows logs for services (local or remote)
func Logs(c *cli.Context) error {
remoteHost := c.String("remote")
service := c.Args().First()
follow := c.Bool("follow") || c.Bool("f")
lines := c.Int("lines")
if remoteHost != "" {
return remoteLogs(remoteHost, service, follow, lines)
}
return localLogs(c, service, follow, lines)
}
func localLogs(c *cli.Context, service string, follow bool, lines int) error {
homeDir, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("failed to get home dir: %w", err)
}
logDir := filepath.Join(homeDir, "micro", "logs")
if service == "" {
// List available logs
files, err := os.ReadDir(logDir)
if err != nil {
fmt.Println("No logs available.")
return nil
}
fmt.Println("Available logs:")
for _, f := range files {
if strings.HasSuffix(f.Name(), ".log") {
name := strings.TrimSuffix(f.Name(), ".log")
fmt.Printf(" %s\n", name)
}
}
fmt.Println("\nView logs: micro logs <service>")
return nil
}
logPath := filepath.Join(logDir, service+".log")
if _, err := os.Stat(logPath); os.IsNotExist(err) {
return fmt.Errorf("no logs for service '%s'", service)
}
if follow {
cmd := exec.Command("tail", "-f", logPath)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
if lines == 0 {
lines = 100
}
cmd := exec.Command("tail", "-n", fmt.Sprintf("%d", lines), logPath)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func remoteLogs(host, service string, follow bool, lines int) error {
var journalCmd string
if service == "" {
// All micro services
journalCmd = "journalctl -u 'micro@*'"
} else {
journalCmd = fmt.Sprintf("journalctl -u 'micro@%s'", service)
}
if follow {
journalCmd += " -f"
} else {
if lines == 0 {
lines = 100
}
journalCmd += fmt.Sprintf(" -n %d", lines)
}
journalCmd += " --no-pager"
sshCmd := exec.Command("ssh", host, journalCmd)
sshCmd.Stdout = os.Stdout
sshCmd.Stderr = os.Stderr
return sshCmd.Run()
}
// Stop stops a running service
func Stop(c *cli.Context) error {
if c.Args().Len() != 1 {
return fmt.Errorf("Usage: micro stop <service>")
}
service := c.Args().First()
remoteHost := c.String("remote")
if remoteHost != "" {
return remoteStop(remoteHost, service)
}
return localStop(service)
}
func localStop(service string) error {
homeDir, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("failed to get home dir: %w", err)
}
runDir := filepath.Join(homeDir, "micro", "run")
pidFilePath := filepath.Join(runDir, service+".pid")
pidFile, err := os.Open(pidFilePath)
if err != nil {
return fmt.Errorf("service '%s' is not running", service)
}
var pid int
scanner := bufio.NewScanner(pidFile)
if scanner.Scan() {
fmt.Sscanf(scanner.Text(), "%d", &pid)
}
pidFile.Close()
if pid <= 0 {
_ = os.Remove(pidFilePath)
return fmt.Errorf("service '%s' is not running", service)
}
proc, err := os.FindProcess(pid)
if err != nil {
_ = os.Remove(pidFilePath)
return fmt.Errorf("could not find process for '%s'", service)
}
if err := proc.Signal(syscall.SIGTERM); err != nil {
_ = os.Remove(pidFilePath)
return fmt.Errorf("failed to stop service '%s': %v", service, err)
}
_ = os.Remove(pidFilePath)
fmt.Printf("Stopped %s (pid %d)\n", service, pid)
return nil
}
func remoteStop(host, service string) error {
stopCmd := fmt.Sprintf("sudo systemctl stop micro@%s", service)
sshCmd := exec.Command("ssh", host, stopCmd)
if output, err := sshCmd.CombinedOutput(); err != nil {
return fmt.Errorf("failed to stop %s: %s", service, string(output))
}
fmt.Printf("Stopped %s on %s\n", service, host)
return nil
}
func init() {
cmd.Register(&cli.Command{
Name: "status",
Usage: "Check status of running services",
Description: `Show status of running services.
Local status:
micro status
Remote status:
micro status --remote user@host`,
Action: Status,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "remote",
Usage: "Check status on remote server",
},
},
})
cmd.Register(&cli.Command{
Name: "logs",
Usage: "Show logs for a service",
Description: `View service logs.
Local logs:
micro logs # list available logs
micro logs myservice # show logs for myservice
micro logs myservice -f # follow logs
Remote logs:
micro logs --remote user@host
micro logs myservice --remote user@host -f`,
Action: Logs,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "remote",
Usage: "View logs on remote server",
},
&cli.BoolFlag{
Name: "follow",
Aliases: []string{"f"},
Usage: "Follow log output",
},
&cli.IntFlag{
Name: "lines",
Aliases: []string{"n"},
Usage: "Number of lines to show (default: 100)",
Value: 100,
},
},
})
cmd.Register(&cli.Command{
Name: "stop",
Usage: "Stop a running service",
Description: `Stop a running service.
Local:
micro stop myservice
Remote:
micro stop myservice --remote user@host`,
Action: Stop,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "remote",
Usage: "Stop service on remote server",
},
},
})
}
+416
View File
@@ -0,0 +1,416 @@
package util
import (
"bytes"
"context"
"encoding/json"
"fmt"
"math"
"os"
"sort"
"strconv"
"strings"
"unicode"
"github.com/stretchr/objx"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/client"
"go-micro.dev/v5/registry"
)
// LookupService queries the service for a service with the given alias. If
// no services are found for a given alias, the registry will return nil and
// the error will also be nil. An error is only returned if there was an issue
// listing from the registry.
func LookupService(name string) (*registry.Service, error) {
// return a lookup in the default domain as a catch all
return serviceWithName(name)
}
// FormatServiceUsage returns a string containing the service usage.
func FormatServiceUsage(srv *registry.Service, c *cli.Context) string {
alias := c.Args().First()
subcommand := c.Args().Get(1)
commands := make([]string, len(srv.Endpoints))
endpoints := make([]*registry.Endpoint, len(srv.Endpoints))
for i, e := range srv.Endpoints {
// map "Helloworld.Call" to "helloworld.call"
parts := strings.Split(e.Name, ".")
for i, part := range parts {
parts[i] = lowercaseInitial(part)
}
name := strings.Join(parts, ".")
// remove the prefix if it is the service name, e.g. rather than
// "micro run helloworld helloworld call", it would be
// "micro run helloworld call".
name = strings.TrimPrefix(name, alias+".")
// instead of "micro run helloworld foo.bar", the command should
// be "micro run helloworld foo bar".
commands[i] = strings.Replace(name, ".", " ", 1)
endpoints[i] = e
}
result := ""
if len(subcommand) > 0 && subcommand != "--help" {
result += fmt.Sprintf("NAME:\n\tmicro %v %v\n\n", alias, subcommand)
result += fmt.Sprintf("USAGE:\n\tmicro %v %v [flags]\n\n", alias, subcommand)
result += fmt.Sprintf("FLAGS:\n")
for i, command := range commands {
if command == subcommand {
result += renderFlags(endpoints[i])
}
}
} else {
// sort the command names alphabetically
sort.Strings(commands)
result += fmt.Sprintf("NAME:\n\tmicro %v\n\n", alias)
result += fmt.Sprintf("VERSION:\n\t%v\n\n", srv.Version)
result += fmt.Sprintf("USAGE:\n\tmicro %v [command]\n\n", alias)
result += fmt.Sprintf("COMMANDS:\n\t%v\n", strings.Join(commands, "\n\t"))
}
return result
}
func lowercaseInitial(str string) string {
for i, v := range str {
return string(unicode.ToLower(v)) + str[i+1:]
}
return ""
}
func renderFlags(endpoint *registry.Endpoint) string {
ret := ""
for _, value := range endpoint.Request.Values {
ret += renderValue([]string{}, value) + "\n"
}
return ret
}
func renderValue(path []string, value *registry.Value) string {
if len(value.Values) > 0 {
renders := []string{}
for _, v := range value.Values {
renders = append(renders, renderValue(append(path, value.Name), v))
}
return strings.Join(renders, "\n")
}
return fmt.Sprintf("\t--%v %v", strings.Join(append(path, value.Name), "_"), value.Type)
}
// CallService will call a service using the arguments and flags provided
// in the context. It will print the result or error to stdout. If there
// was an error performing the call, it will be returned.
func CallService(srv *registry.Service, args []string) error {
// parse the flags and args
args, flags, err := splitCmdArgs(args)
if err != nil {
return err
}
// construct the endpoint
endpoint, err := constructEndpoint(args)
if err != nil {
return err
}
// ensure the endpoint exists on the service
var ep *registry.Endpoint
for _, e := range srv.Endpoints {
if e.Name == endpoint {
ep = e
break
}
}
if ep == nil {
return fmt.Errorf("Endpoint %v not found for service %v", endpoint, srv.Name)
}
// parse the flags
body, err := FlagsToRequest(flags, ep.Request)
if err != nil {
return err
}
// create a context for the call based on the cli context
callCtx := context.TODO()
// TODO: parse out --header or --metadata
// construct and execute the request using the json content type
req := client.DefaultClient.NewRequest(srv.Name, endpoint, body, client.WithContentType("application/json"))
var rsp json.RawMessage
if err := client.DefaultClient.Call(callCtx, req, &rsp); err != nil {
return err
}
// format the response
var out bytes.Buffer
defer out.Reset()
if err := json.Indent(&out, rsp, "", "\t"); err != nil {
return err
}
out.Write([]byte("\n"))
out.WriteTo(os.Stdout)
return nil
}
// splitCmdArgs takes a cli context and parses out the args and flags, for
// example "micro helloworld --name=foo call apple" would result in "call",
// "apple" as args and {"name":"foo"} as the flags.
func splitCmdArgs(arguments []string) ([]string, map[string][]string, error) {
args := []string{}
flags := map[string][]string{}
prev := ""
for _, a := range arguments {
if !strings.HasPrefix(a, "--") {
if len(prev) == 0 {
args = append(args, a)
continue
}
_, exists := flags[prev]
if !exists {
flags[prev] = []string{}
}
flags[prev] = append(flags[prev], a)
prev = ""
continue
}
// comps would be "foo", "bar" for "--foo=bar"
comps := strings.Split(strings.TrimPrefix(a, "--"), "=")
_, exists := flags[comps[0]]
if !exists {
flags[comps[0]] = []string{}
}
switch len(comps) {
case 1:
prev = comps[0]
case 2:
flags[comps[0]] = append(flags[comps[0]], comps[1])
default:
return nil, nil, fmt.Errorf("Invalid flag: %v. Expected format: --foo=bar", a)
}
}
return args, flags, nil
}
// constructEndpoint takes a slice of args and converts it into a valid endpoint
// such as Helloworld.Call or Foo.Bar, it will return an error if an invalid number
// of arguments were provided
func constructEndpoint(args []string) (string, error) {
var epComps []string
switch len(args) {
case 1:
epComps = append(args, "call")
case 2:
epComps = args
case 3:
epComps = args[1:3]
default:
return "", fmt.Errorf("Incorrect number of arguments")
}
// transform the endpoint components, e.g ["helloworld", "call"] to the
// endpoint name: "Helloworld.Call".
return fmt.Sprintf("%v.%v", strings.Title(epComps[0]), strings.Title(epComps[1])), nil
}
// ShouldRenderHelp returns true if the help flag was passed
func ShouldRenderHelp(args []string) bool {
args, flags, _ := splitCmdArgs(args)
// only 1 arg e.g micro helloworld
if len(args) == 1 {
return true
}
for key := range flags {
if key == "help" {
return true
}
}
return false
}
// FlagsToRequest parses a set of flags, e.g {name:"Foo", "options_surname","Bar"} and
// converts it into a request body. If the key is not a valid object in the request, an
// error will be returned.
//
// This function constructs []interface{} slices
// as opposed to typed ([]string etc) slices for easier testing
func FlagsToRequest(flags map[string][]string, req *registry.Value) (map[string]interface{}, error) {
coerceValue := func(valueType string, value []string) (interface{}, error) {
switch valueType {
case "bool":
if len(value) == 0 || len(strings.TrimSpace(value[0])) == 0 {
return true, nil
}
return strconv.ParseBool(value[0])
case "int32":
i, err := strconv.Atoi(value[0])
if err != nil {
return nil, err
}
if i < math.MinInt32 || i > math.MaxInt32 {
return nil, fmt.Errorf("value out of range for int32: %d", i)
}
return int32(i), nil
case "int64":
return strconv.ParseInt(value[0], 0, 64)
case "float64":
return strconv.ParseFloat(value[0], 64)
case "[]bool":
// length is one if it's a `,` separated int slice
if len(value) == 1 {
value = strings.Split(value[0], ",")
}
ret := []interface{}{}
for _, v := range value {
i, err := strconv.ParseBool(v)
if err != nil {
return nil, err
}
ret = append(ret, i)
}
return ret, nil
case "[]int32":
// length is one if it's a `,` separated int slice
if len(value) == 1 {
value = strings.Split(value[0], ",")
}
ret := []interface{}{}
for _, v := range value {
i, err := strconv.Atoi(v)
if err != nil {
return nil, err
}
if i < math.MinInt32 || i > math.MaxInt32 {
return nil, fmt.Errorf("value out of range for int32: %d", i)
}
ret = append(ret, int32(i))
}
return ret, nil
case "[]int64":
// length is one if it's a `,` separated int slice
if len(value) == 1 {
value = strings.Split(value[0], ",")
}
ret := []interface{}{}
for _, v := range value {
i, err := strconv.ParseInt(v, 0, 64)
if err != nil {
return nil, err
}
ret = append(ret, i)
}
return ret, nil
case "[]float64":
// length is one if it's a `,` separated float slice
if len(value) == 1 {
value = strings.Split(value[0], ",")
}
ret := []interface{}{}
for _, v := range value {
i, err := strconv.ParseFloat(v, 64)
if err != nil {
return nil, err
}
ret = append(ret, i)
}
return ret, nil
case "[]string":
// length is one it's a `,` separated string slice
if len(value) == 1 {
value = strings.Split(value[0], ",")
}
ret := []interface{}{}
for _, v := range value {
ret = append(ret, v)
}
return ret, nil
case "string":
return value[0], nil
case "map[string]string":
var val map[string]string
if err := json.Unmarshal([]byte(value[0]), &val); err != nil {
return value[0], nil
}
return val, nil
default:
return value, nil
}
return nil, nil
}
result := objx.MustFromJSON("{}")
var flagType func(key string, values []*registry.Value, path ...string) (string, bool)
flagType = func(key string, values []*registry.Value, path ...string) (string, bool) {
for _, attr := range values {
if strings.Join(append(path, attr.Name), "-") == key {
return attr.Type, true
}
if attr.Values != nil {
typ, found := flagType(key, attr.Values, append(path, attr.Name)...)
if found {
return typ, found
}
}
}
return "", false
}
for key, value := range flags {
ty, found := flagType(key, req.Values)
if !found {
return nil, fmt.Errorf("Unknown flag: %v", key)
}
parsed, err := coerceValue(ty, value)
if err != nil {
return nil, err
}
// objx.Set does not create the path,
// so we do that here
if strings.Contains(key, "-") {
parts := strings.Split(key, "-")
for i, _ := range parts {
pToCreate := strings.Join(parts[0:i], ".")
if i > 0 && i < len(parts) && !result.Has(pToCreate) {
result.Set(pToCreate, map[string]interface{}{})
}
}
}
path := strings.Replace(key, "-", ".", -1)
result.Set(path, parsed)
}
return result, nil
}
// find a service in a domain matching the name
func serviceWithName(name string) (*registry.Service, error) {
srvs, err := registry.GetService(name)
if err == registry.ErrNotFound {
return nil, nil
} else if err != nil {
return nil, err
}
if len(srvs) == 0 {
return nil, nil
}
return srvs[0], nil
}
+379
View File
@@ -0,0 +1,379 @@
package util
import (
"reflect"
"strings"
"testing"
"github.com/davecgh/go-spew/spew"
goregistry "go-micro.dev/v5/registry"
)
type parseCase struct {
args []string
values *goregistry.Value
expected map[string]interface{}
}
func TestDynamicFlagParsing(t *testing.T) {
cases := []parseCase{
{
args: []string{"--ss=a,b"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "ss",
Type: "[]string",
},
},
},
expected: map[string]interface{}{
"ss": []interface{}{"a", "b"},
},
},
{
args: []string{"--ss", "a,b"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "ss",
Type: "[]string",
},
},
},
expected: map[string]interface{}{
"ss": []interface{}{"a", "b"},
},
},
{
args: []string{"--ss=a", "--ss=b"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "ss",
Type: "[]string",
},
},
},
expected: map[string]interface{}{
"ss": []interface{}{"a", "b"},
},
},
{
args: []string{"--ss", "a", "--ss", "b"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "ss",
Type: "[]string",
},
},
},
expected: map[string]interface{}{
"ss": []interface{}{"a", "b"},
},
},
{
args: []string{"--bs=true,false"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "bs",
Type: "[]bool",
},
},
},
expected: map[string]interface{}{
"bs": []interface{}{true, false},
},
},
{
args: []string{"--bs", "true,false"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "bs",
Type: "[]bool",
},
},
},
expected: map[string]interface{}{
"bs": []interface{}{true, false},
},
},
{
args: []string{"--bs=true", "--bs=false"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "bs",
Type: "[]bool",
},
},
},
expected: map[string]interface{}{
"bs": []interface{}{true, false},
},
},
{
args: []string{"--bs", "true", "--bs", "false"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "bs",
Type: "[]bool",
},
},
},
expected: map[string]interface{}{
"bs": []interface{}{true, false},
},
},
{
args: []string{"--is=10,20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int32",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int32(10), int32(20)},
},
},
{
args: []string{"--is", "10,20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int32",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int32(10), int32(20)},
},
},
{
args: []string{"--is=10", "--is=20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int32",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int32(10), int32(20)},
},
},
{
args: []string{"--is", "10", "--is", "20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int32",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int32(10), int32(20)},
},
},
{
args: []string{"--is=10,20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int64",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int64(10), int64(20)},
},
},
{
args: []string{"--is", "10,20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int64",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int64(10), int64(20)},
},
},
{
args: []string{"--is=10", "--is=20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int64",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int64(10), int64(20)},
},
},
{
args: []string{"--is", "10", "--is", "20"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "is",
Type: "[]int64",
},
},
},
expected: map[string]interface{}{
"is": []interface{}{int64(10), int64(20)},
},
},
{
args: []string{"--fs=10.1,20.2"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "fs",
Type: "[]float64",
},
},
},
expected: map[string]interface{}{
"fs": []interface{}{float64(10.1), float64(20.2)},
},
},
{
args: []string{"--fs", "10.1,20.2"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "fs",
Type: "[]float64",
},
},
},
expected: map[string]interface{}{
"fs": []interface{}{float64(10.1), float64(20.2)},
},
},
{
args: []string{"--fs=10.1", "--fs=20.2"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "fs",
Type: "[]float64",
},
},
},
expected: map[string]interface{}{
"fs": []interface{}{float64(10.1), float64(20.2)},
},
},
{
args: []string{"--fs", "10.1", "--fs", "20.2"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "fs",
Type: "[]float64",
},
},
},
expected: map[string]interface{}{
"fs": []interface{}{float64(10.1), float64(20.2)},
},
},
{
args: []string{"--user_email=someemail"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "user_email",
Type: "string",
},
},
},
expected: map[string]interface{}{
"user_email": "someemail",
},
},
{
args: []string{"--user_email=someemail", "--user_name=somename"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "user_email",
Type: "string",
},
{
Name: "user_name",
Type: "string",
},
},
},
expected: map[string]interface{}{
"user_email": "someemail",
"user_name": "somename",
},
},
{
args: []string{"--b"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "b",
Type: "bool",
},
},
},
expected: map[string]interface{}{
"b": true,
},
},
{
args: []string{"--user_friend_email=hi"},
values: &goregistry.Value{
Values: []*goregistry.Value{
{
Name: "user_friend_email",
Type: "string",
},
},
},
expected: map[string]interface{}{
"user_friend_email": "hi",
},
},
}
for _, c := range cases {
t.Run(strings.Join(c.args, " "), func(t *testing.T) {
_, flags, err := splitCmdArgs(c.args)
if err != nil {
t.Fatal(err)
}
req, err := FlagsToRequest(flags, c.values)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(c.expected, req) {
spew.Dump("Expected:", c.expected, "got: ", req)
t.Fatalf("Expected %v, got %v", c.expected, req)
}
})
}
}
+72
View File
@@ -0,0 +1,72 @@
// Package cliutil contains methods used across all cli commands
// @todo: get rid of os.Exits and use errors instread
package util
import (
"fmt"
"regexp"
"strings"
"github.com/urfave/cli/v2"
merrors "go-micro.dev/v5/errors"
)
type Exec func(*cli.Context, []string) ([]byte, error)
func Print(e Exec) func(*cli.Context) error {
return func(c *cli.Context) error {
rsp, err := e(c, c.Args().Slice())
if err != nil {
return CliError(err)
}
if len(rsp) > 0 {
fmt.Printf("%s\n", string(rsp))
}
return nil
}
}
// CliError returns a user friendly message from error. If we can't determine a good one returns an error with code 128
func CliError(err error) cli.ExitCoder {
if err == nil {
return nil
}
// if it's already a cli.ExitCoder we use this
cerr, ok := err.(cli.ExitCoder)
if ok {
return cerr
}
// grpc errors
if mname := regexp.MustCompile(`malformed method name: \\?"(\w+)\\?"`).FindStringSubmatch(err.Error()); len(mname) > 0 {
return cli.Exit(fmt.Sprintf(`Method name "%s" invalid format. Expecting service.endpoint`, mname[1]), 3)
}
if service := regexp.MustCompile(`service ([\w\.]+): route not found`).FindStringSubmatch(err.Error()); len(service) > 0 {
return cli.Exit(fmt.Sprintf(`Service "%s" not found`, service[1]), 4)
}
if service := regexp.MustCompile(`unknown service ([\w\.]+)`).FindStringSubmatch(err.Error()); len(service) > 0 {
if strings.Contains(service[0], ".") {
return cli.Exit(fmt.Sprintf(`Service method "%s" not found`, service[1]), 5)
}
return cli.Exit(fmt.Sprintf(`Service "%s" not found`, service[1]), 5)
}
if address := regexp.MustCompile(`Error while dialing dial tcp.*?([\w]+\.[\w:\.]+): `).FindStringSubmatch(err.Error()); len(address) > 0 {
return cli.Exit(fmt.Sprintf(`Failed to connect to micro server at %s`, address[1]), 4)
}
merr, ok := err.(*merrors.Error)
if !ok {
return cli.Exit(err, 128)
}
switch merr.Code {
case 408:
return cli.Exit("Request timed out", 1)
case 401:
// TODO check if not signed in, prompt to sign in
return cli.Exit("Not authorized to perform this request", 2)
}
// fallback to using the detail from the merr
return cli.Exit(merr.Detail, 127)
}
+28
View File
@@ -0,0 +1,28 @@
package main
import (
"embed"
"go-micro.dev/v5/cmd"
_ "go-micro.dev/v5/cmd/micro/cli"
_ "go-micro.dev/v5/cmd/micro/cli/build"
_ "go-micro.dev/v5/cmd/micro/cli/deploy"
_ "go-micro.dev/v5/cmd/micro/run"
"go-micro.dev/v5/cmd/micro/server"
)
//go:embed web/styles.css web/main.js web/templates/*
var webFS embed.FS
var version = "5.0.0-dev"
func init() {
server.HTML = webFS
}
func main() {
cmd.Init(
cmd.Name("micro"),
cmd.Version(version),
)
}
+284
View File
@@ -0,0 +1,284 @@
// Package config handles micro.mu and micro.json configuration parsing
package config
import (
"bufio"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
)
// Config represents the micro run configuration
type Config struct {
Services map[string]*Service `json:"services"`
Envs map[string]map[string]string `json:"env"`
Deploy map[string]*DeployTarget `json:"deploy"`
}
// DeployTarget represents a deployment target configuration
type DeployTarget struct {
Name string `json:"-"`
SSH string `json:"ssh"`
Path string `json:"path,omitempty"`
}
// Service represents a service configuration
type Service struct {
Name string `json:"-"`
Path string `json:"path"`
Port int `json:"port,omitempty"`
Depends []string `json:"depends,omitempty"`
}
// Load attempts to load configuration from micro.mu or micro.json in the given directory
func Load(dir string) (*Config, error) {
// Try micro.mu first (preferred)
muPath := filepath.Join(dir, "micro.mu")
if _, err := os.Stat(muPath); err == nil {
return ParseMu(muPath)
}
// Fall back to micro.json
jsonPath := filepath.Join(dir, "micro.json")
if _, err := os.Stat(jsonPath); err == nil {
return ParseJSON(jsonPath)
}
return nil, nil // No config file, not an error
}
// ParseJSON parses a micro.json configuration file
func ParseJSON(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("failed to read %s: %w", path, err)
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("failed to parse %s: %w", path, err)
}
// Set service names from map keys
for name, svc := range cfg.Services {
svc.Name = name
}
return &cfg, nil
}
// ParseMu parses a micro.mu DSL configuration file
//
// Format:
//
// service users
// path ./users
// port 8081
//
// service posts
// path ./posts
// port 8082
// depends users
//
// env development
// STORE_ADDRESS file://./data
func ParseMu(path string) (*Config, error) {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("failed to open %s: %w", path, err)
}
defer file.Close()
cfg := &Config{
Services: make(map[string]*Service),
Envs: make(map[string]map[string]string),
Deploy: make(map[string]*DeployTarget),
}
var currentService *Service
var currentEnv string
var currentEnvMap map[string]string
var currentDeploy *DeployTarget
scanner := bufio.NewScanner(file)
lineNum := 0
for scanner.Scan() {
lineNum++
line := scanner.Text()
// Skip empty lines and comments
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
continue
}
// Check indentation
indented := strings.HasPrefix(line, " ") || strings.HasPrefix(line, "\t")
if !indented {
// Top-level declaration
parts := strings.Fields(trimmed)
if len(parts) < 2 {
return nil, fmt.Errorf("%s:%d: expected 'service <name>' or 'env <name>'", path, lineNum)
}
keyword := parts[0]
name := parts[1]
switch keyword {
case "service":
// Save previous env if any
if currentEnv != "" && currentEnvMap != nil {
cfg.Envs[currentEnv] = currentEnvMap
}
currentEnv = ""
currentEnvMap = nil
currentService = &Service{Name: name}
cfg.Services[name] = currentService
case "env":
// Save previous env if any
if currentEnv != "" && currentEnvMap != nil {
cfg.Envs[currentEnv] = currentEnvMap
}
currentService = nil
currentDeploy = nil
currentEnv = name
currentEnvMap = make(map[string]string)
case "deploy":
// Save previous env if any
if currentEnv != "" && currentEnvMap != nil {
cfg.Envs[currentEnv] = currentEnvMap
}
currentService = nil
currentEnv = ""
currentEnvMap = nil
currentDeploy = &DeployTarget{Name: name}
cfg.Deploy[name] = currentDeploy
default:
return nil, fmt.Errorf("%s:%d: unknown keyword '%s'", path, lineNum, keyword)
}
} else {
// Indented property
parts := strings.Fields(trimmed)
if len(parts) < 2 {
return nil, fmt.Errorf("%s:%d: expected 'key value'", path, lineNum)
}
key := parts[0]
value := strings.Join(parts[1:], " ")
if currentService != nil {
switch key {
case "path":
currentService.Path = value
case "port":
port, err := strconv.Atoi(value)
if err != nil {
return nil, fmt.Errorf("%s:%d: invalid port '%s'", path, lineNum, value)
}
currentService.Port = port
case "depends":
currentService.Depends = parts[1:]
default:
return nil, fmt.Errorf("%s:%d: unknown service property '%s'", path, lineNum, key)
}
} else if currentDeploy != nil {
switch key {
case "ssh":
currentDeploy.SSH = value
case "path":
currentDeploy.Path = value
default:
return nil, fmt.Errorf("%s:%d: unknown deploy property '%s'", path, lineNum, key)
}
} else if currentEnvMap != nil {
// Environment variable
currentEnvMap[key] = value
} else {
return nil, fmt.Errorf("%s:%d: property outside of service, deploy, or env block", path, lineNum)
}
}
}
// Save final env if any
if currentEnv != "" && currentEnvMap != nil {
cfg.Envs[currentEnv] = currentEnvMap
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading %s: %w", path, err)
}
return cfg, nil
}
// TopologicalSort returns services in dependency order
func (c *Config) TopologicalSort() ([]*Service, error) {
if c == nil || len(c.Services) == 0 {
return nil, nil
}
// Build adjacency list and in-degree count
inDegree := make(map[string]int)
for name := range c.Services {
inDegree[name] = 0
}
for _, svc := range c.Services {
for _, dep := range svc.Depends {
if _, ok := c.Services[dep]; !ok {
return nil, fmt.Errorf("service '%s' depends on unknown service '%s'", svc.Name, dep)
}
inDegree[svc.Name]++
}
}
// Kahn's algorithm
var queue []string
for name, degree := range inDegree {
if degree == 0 {
queue = append(queue, name)
}
}
var result []*Service
for len(queue) > 0 {
name := queue[0]
queue = queue[1:]
result = append(result, c.Services[name])
// Reduce in-degree for dependents
for _, svc := range c.Services {
for _, dep := range svc.Depends {
if dep == name {
inDegree[svc.Name]--
if inDegree[svc.Name] == 0 {
queue = append(queue, svc.Name)
}
}
}
}
}
if len(result) != len(c.Services) {
return nil, fmt.Errorf("circular dependency detected")
}
return result, nil
}
// GetEnv returns environment variables for the given environment name
func (c *Config) GetEnv(name string) map[string]string {
if c == nil || c.Envs == nil {
return nil
}
return c.Envs[name]
}
+217
View File
@@ -0,0 +1,217 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestParseMu(t *testing.T) {
content := `# Micro configuration
service users
path ./users
port 8081
service posts
path ./posts
port 8082
depends users
service web
path ./web
port 8089
depends users posts
env development
STORE_ADDRESS file://./data
DEBUG true
env production
STORE_ADDRESS postgres://localhost/db
`
tmpDir := t.TempDir()
muPath := filepath.Join(tmpDir, "micro.mu")
if err := os.WriteFile(muPath, []byte(content), 0644); err != nil {
t.Fatal(err)
}
cfg, err := ParseMu(muPath)
if err != nil {
t.Fatalf("ParseMu failed: %v", err)
}
// Check services
if len(cfg.Services) != 3 {
t.Errorf("expected 3 services, got %d", len(cfg.Services))
}
users := cfg.Services["users"]
if users == nil {
t.Fatal("users service not found")
}
if users.Path != "./users" {
t.Errorf("users.Path = %q, want %q", users.Path, "./users")
}
if users.Port != 8081 {
t.Errorf("users.Port = %d, want %d", users.Port, 8081)
}
posts := cfg.Services["posts"]
if posts == nil {
t.Fatal("posts service not found")
}
if len(posts.Depends) != 1 || posts.Depends[0] != "users" {
t.Errorf("posts.Depends = %v, want [users]", posts.Depends)
}
web := cfg.Services["web"]
if web == nil {
t.Fatal("web service not found")
}
if len(web.Depends) != 2 {
t.Errorf("web.Depends = %v, want [users posts]", web.Depends)
}
// Check envs
if len(cfg.Envs) != 2 {
t.Errorf("expected 2 envs, got %d", len(cfg.Envs))
}
dev := cfg.GetEnv("development")
if dev == nil {
t.Fatal("development env not found")
}
if dev["STORE_ADDRESS"] != "file://./data" {
t.Errorf("STORE_ADDRESS = %q, want %q", dev["STORE_ADDRESS"], "file://./data")
}
if dev["DEBUG"] != "true" {
t.Errorf("DEBUG = %q, want %q", dev["DEBUG"], "true")
}
}
func TestParseJSON(t *testing.T) {
content := `{
"services": {
"users": {
"path": "./users",
"port": 8081
},
"posts": {
"path": "./posts",
"port": 8082,
"depends": ["users"]
}
},
"env": {
"development": {
"STORE_ADDRESS": "file://./data"
}
}
}`
tmpDir := t.TempDir()
jsonPath := filepath.Join(tmpDir, "micro.json")
if err := os.WriteFile(jsonPath, []byte(content), 0644); err != nil {
t.Fatal(err)
}
cfg, err := ParseJSON(jsonPath)
if err != nil {
t.Fatalf("ParseJSON failed: %v", err)
}
if len(cfg.Services) != 2 {
t.Errorf("expected 2 services, got %d", len(cfg.Services))
}
users := cfg.Services["users"]
if users == nil {
t.Fatal("users service not found")
}
if users.Port != 8081 {
t.Errorf("users.Port = %d, want %d", users.Port, 8081)
}
}
func TestTopologicalSort(t *testing.T) {
cfg := &Config{
Services: map[string]*Service{
"web": {Name: "web", Depends: []string{"users", "posts"}},
"posts": {Name: "posts", Depends: []string{"users"}},
"users": {Name: "users"},
},
}
sorted, err := cfg.TopologicalSort()
if err != nil {
t.Fatalf("TopologicalSort failed: %v", err)
}
if len(sorted) != 3 {
t.Fatalf("expected 3 services, got %d", len(sorted))
}
// users must come before posts and web
// posts must come before web
positions := make(map[string]int)
for i, svc := range sorted {
positions[svc.Name] = i
}
if positions["users"] > positions["posts"] {
t.Error("users should come before posts")
}
if positions["users"] > positions["web"] {
t.Error("users should come before web")
}
if positions["posts"] > positions["web"] {
t.Error("posts should come before web")
}
}
func TestCircularDependency(t *testing.T) {
cfg := &Config{
Services: map[string]*Service{
"a": {Name: "a", Depends: []string{"b"}},
"b": {Name: "b", Depends: []string{"a"}},
},
}
_, err := cfg.TopologicalSort()
if err == nil {
t.Error("expected circular dependency error")
}
}
func TestLoad(t *testing.T) {
// Test with no config file
tmpDir := t.TempDir()
cfg, err := Load(tmpDir)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg != nil {
t.Error("expected nil config when no file exists")
}
// Test with micro.mu
muContent := `service test
path ./test
port 8080
`
if err := os.WriteFile(filepath.Join(tmpDir, "micro.mu"), []byte(muContent), 0644); err != nil {
t.Fatal(err)
}
cfg, err = Load(tmpDir)
if err != nil {
t.Fatalf("Load failed: %v", err)
}
if cfg == nil {
t.Fatal("expected config to be loaded")
}
if cfg.Services["test"] == nil {
t.Error("test service not found")
}
}
+276
View File
@@ -0,0 +1,276 @@
// Package gateway provides an HTTP gateway for micro run
package gateway
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"sync"
"time"
"go-micro.dev/v5/client"
"go-micro.dev/v5/codec/bytes"
"go-micro.dev/v5/health"
"go-micro.dev/v5/registry"
)
// Gateway provides HTTP access to micro services
type Gateway struct {
addr string
server *http.Server
services []ServiceInfo
mu sync.RWMutex
}
// ServiceInfo holds information about a running service
type ServiceInfo struct {
Name string `json:"name"`
Address string `json:"address"`
Port int `json:"port,omitempty"`
}
// New creates a new gateway
func New(addr string) *Gateway {
return &Gateway{
addr: addr,
}
}
// SetServices updates the list of known services
func (g *Gateway) SetServices(services []ServiceInfo) {
g.mu.Lock()
g.services = services
g.mu.Unlock()
}
// Start starts the gateway HTTP server
func (g *Gateway) Start() error {
mux := http.NewServeMux()
// Health endpoint - aggregates all service health
mux.HandleFunc("/health", g.healthHandler)
mux.HandleFunc("/health/live", g.liveHandler)
mux.HandleFunc("/health/ready", g.readyHandler)
// API endpoint - HTTP to RPC proxy
mux.HandleFunc("/api/", g.apiHandler)
// Services list
mux.HandleFunc("/services", g.servicesHandler)
// Home page
mux.HandleFunc("/", g.homeHandler)
g.server = &http.Server{
Addr: g.addr,
Handler: mux,
}
go func() {
if err := g.server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
fmt.Printf("Gateway error: %v\n", err)
}
}()
return nil
}
// Stop stops the gateway
func (g *Gateway) Stop() {
if g.server != nil {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
g.server.Shutdown(ctx)
}
}
// Addr returns the gateway address
func (g *Gateway) Addr() string {
return g.addr
}
func (g *Gateway) homeHandler(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
g.mu.RLock()
services := g.services
g.mu.RUnlock()
// Get services from registry
regServices, _ := registry.ListServices()
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprintf(w, `<!DOCTYPE html>
<html>
<head>
<title>Micro</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; background: #f5f5f5; color: #333; }
.container { max-width: 800px; margin: 0 auto; padding: 40px 20px; }
h1 { font-size: 2em; margin-bottom: 10px; }
.subtitle { color: #666; margin-bottom: 30px; }
.card { background: white; border-radius: 8px; padding: 20px; margin-bottom: 20px; box-shadow: 0 1px 3px rgba(0,0,0,0.1); }
.card h2 { font-size: 1.2em; margin-bottom: 15px; color: #333; }
.service { display: flex; justify-content: space-between; align-items: center; padding: 10px 0; border-bottom: 1px solid #eee; }
.service:last-child { border-bottom: none; }
.service-name { font-weight: 500; }
.service-addr { color: #666; font-family: monospace; font-size: 0.9em; }
.endpoints { margin-top: 10px; }
.endpoint { display: block; padding: 5px 10px; margin: 5px 0; background: #f0f0f0; border-radius: 4px; font-family: monospace; font-size: 0.85em; text-decoration: none; color: #333; }
.endpoint:hover { background: #e0e0e0; }
.try-it { background: #f9f9f9; padding: 15px; border-radius: 6px; margin-top: 20px; }
.try-it h3 { font-size: 1em; margin-bottom: 10px; }
code { background: #333; color: #0f0; padding: 10px 15px; display: block; border-radius: 4px; font-size: 0.85em; overflow-x: auto; }
.links { margin-top: 20px; }
.links a { color: #0066cc; margin-right: 15px; }
</style>
</head>
<body>
<div class="container">
<h1>Micro</h1>
<p class="subtitle">Services are running</p>
<div class="card">
<h2>Services (%d)</h2>
`, len(regServices))
if len(regServices) > 0 {
for _, svc := range regServices {
fmt.Fprintf(w, ` <div class="service">
<span class="service-name">%s</span>
</div>
`, svc.Name)
// Get endpoints for this service
if details, err := registry.GetService(svc.Name); err == nil && len(details) > 0 {
if len(details[0].Endpoints) > 0 {
fmt.Fprintf(w, ` <div class="endpoints">`)
for _, ep := range details[0].Endpoints {
fmt.Fprintf(w, ` <a class="endpoint" href="/api/%s/%s">POST /api/%s/%s</a>\n`,
svc.Name, ep.Name, svc.Name, ep.Name)
}
fmt.Fprintf(w, ` </div>`)
}
}
}
} else if len(services) > 0 {
for _, svc := range services {
fmt.Fprintf(w, ` <div class="service">
<span class="service-name">%s</span>
<span class="service-addr">%s</span>
</div>
`, svc.Name, svc.Address)
}
} else {
fmt.Fprintf(w, ` <p style="color: #666; padding: 10px 0;">No services registered yet...</p>`)
}
fmt.Fprintf(w, ` </div>
<div class="card">
<h2>Quick Links</h2>
<div class="links">
<a href="/health">Health Check</a>
<a href="/services">Services JSON</a>
</div>
</div>
<div class="try-it">
<h3>Try it</h3>
<code>curl -X POST http://localhost%s/api/{service}/{Endpoint} -d '{}'</code>
</div>
</div>
</body>
</html>`, g.addr)
}
func (g *Gateway) servicesHandler(w http.ResponseWriter, r *http.Request) {
services, err := registry.ListServices()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
var result []map[string]interface{}
for _, svc := range services {
details, _ := registry.GetService(svc.Name)
var endpoints []string
if len(details) > 0 {
for _, ep := range details[0].Endpoints {
endpoints = append(endpoints, ep.Name)
}
}
result = append(result, map[string]interface{}{
"name": svc.Name,
"endpoints": endpoints,
})
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(result)
}
func (g *Gateway) healthHandler(w http.ResponseWriter, r *http.Request) {
resp := health.Run(r.Context())
w.Header().Set("Content-Type", "application/json")
if resp.Status == health.StatusUp {
w.WriteHeader(http.StatusOK)
} else {
w.WriteHeader(http.StatusServiceUnavailable)
}
json.NewEncoder(w).Encode(resp)
}
func (g *Gateway) liveHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write([]byte(`{"status":"up"}`))
}
func (g *Gateway) readyHandler(w http.ResponseWriter, r *http.Request) {
g.healthHandler(w, r)
}
func (g *Gateway) apiHandler(w http.ResponseWriter, r *http.Request) {
// Parse path: /api/{service}/{endpoint}
path := strings.TrimPrefix(r.URL.Path, "/api/")
parts := strings.SplitN(path, "/", 2)
if len(parts) < 2 {
http.Error(w, `{"error": "usage: /api/{service}/{endpoint}"}`, http.StatusBadRequest)
return
}
service := parts[0]
endpoint := parts[1]
// Read request body
body, err := io.ReadAll(r.Body)
if err != nil {
http.Error(w, fmt.Sprintf(`{"error": "%s"}`, err.Error()), http.StatusBadRequest)
return
}
if len(body) == 0 {
body = []byte("{}")
}
// Create RPC request
req := client.NewRequest(service, endpoint, &bytes.Frame{Data: body})
var rsp bytes.Frame
if err := client.Call(r.Context(), req, &rsp); err != nil {
http.Error(w, fmt.Sprintf(`{"error": "%s"}`, err.Error()), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
w.Write(rsp.Data)
}
+519
View File
@@ -0,0 +1,519 @@
package run
import (
"bufio"
"crypto/md5"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
"time"
"github.com/urfave/cli/v2"
"go-micro.dev/v5/cmd"
"go-micro.dev/v5/cmd/micro/run/config"
"go-micro.dev/v5/cmd/micro/run/gateway"
"go-micro.dev/v5/cmd/micro/run/watcher"
)
// Color codes for log output
var colors = []string{
"\033[31m", // red
"\033[32m", // green
"\033[33m", // yellow
"\033[34m", // blue
"\033[35m", // magenta
"\033[36m", // cyan
}
const colorReset = "\033[0m"
func colorFor(idx int) string {
return colors[idx%len(colors)]
}
// serviceProcess tracks a running service
type serviceProcess struct {
name string
dir string
binPath string
pidFile string
logFile string
cmd *exec.Cmd
pipeWriter *io.PipeWriter
color string
port int
env []string
mu sync.Mutex
running bool
}
func (s *serviceProcess) start(logDir string) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
return nil
}
// Build
buildCmd := exec.Command("go", "build", "-o", s.binPath, ".")
buildCmd.Dir = s.dir
buildOut, buildErr := buildCmd.CombinedOutput()
if buildErr != nil {
return fmt.Errorf("build failed: %s\n%s", buildErr, string(buildOut))
}
// Open log file
logFile, err := os.OpenFile(s.logFile, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
return fmt.Errorf("failed to open log file: %w", err)
}
// Start process
s.cmd = exec.Command(s.binPath)
s.cmd.Dir = s.dir
s.cmd.Env = append(os.Environ(), s.env...)
pr, pw := io.Pipe()
s.pipeWriter = pw
s.cmd.Stdout = pw
s.cmd.Stderr = pw
// Stream output
go func(name string, color string, pr *io.PipeReader, logFile *os.File) {
defer logFile.Close()
scanner := bufio.NewScanner(pr)
for scanner.Scan() {
line := scanner.Text()
fmt.Printf("%s[%s]%s %s\n", color, name, colorReset, line)
logFile.WriteString("[" + name + "] " + line + "\n")
}
}(s.name, s.color, pr, logFile)
if err := s.cmd.Start(); err != nil {
pw.Close()
return fmt.Errorf("failed to start: %w", err)
}
// Write PID file
os.WriteFile(s.pidFile, []byte(fmt.Sprintf("%d\n%s\n%s\n%s\n",
s.cmd.Process.Pid, s.dir, s.name, time.Now().Format(time.RFC3339))), 0644)
s.running = true
fmt.Printf("%s[%s]%s started (pid %d)\n", s.color, s.name, colorReset, s.cmd.Process.Pid)
return nil
}
func (s *serviceProcess) stop() {
s.mu.Lock()
defer s.mu.Unlock()
if !s.running || s.cmd == nil || s.cmd.Process == nil {
return
}
fmt.Printf("%s[%s]%s stopping...\n", s.color, s.name, colorReset)
// Graceful shutdown
s.cmd.Process.Signal(syscall.SIGTERM)
// Wait with timeout
done := make(chan error, 1)
go func() {
done <- s.cmd.Wait()
}()
select {
case <-done:
case <-time.After(5 * time.Second):
s.cmd.Process.Kill()
<-done
}
if s.pipeWriter != nil {
s.pipeWriter.Close()
}
os.Remove(s.pidFile)
s.running = false
}
func (s *serviceProcess) restart(logDir string) error {
s.stop()
return s.start(logDir)
}
// waitForHealth waits for a service's health endpoint to respond
func waitForHealth(port int, timeout time.Duration) bool {
if port == 0 {
return true // No port configured, assume ready
}
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
resp, err := http.Get(fmt.Sprintf("http://localhost:%d/health", port))
if err == nil {
resp.Body.Close()
if resp.StatusCode == 200 {
return true
}
}
time.Sleep(100 * time.Millisecond)
}
return false
}
func Run(c *cli.Context) error {
dir := c.Args().Get(0)
if dir == "" {
dir = "."
}
// Handle git URLs
if strings.HasPrefix(dir, "github.com/") || strings.HasPrefix(dir, "https://github.com/") {
repo := strings.TrimPrefix(dir, "https://")
tmp, err := os.MkdirTemp("", "micro-run-")
if err != nil {
return fmt.Errorf("failed to create temp dir: %w", err)
}
defer os.RemoveAll(tmp)
cloneURL := "https://" + repo
cloneCmd := exec.Command("git", "clone", "--depth", "1", cloneURL, tmp)
cloneCmd.Stdout = os.Stdout
cloneCmd.Stderr = os.Stderr
if err := cloneCmd.Run(); err != nil {
return fmt.Errorf("failed to clone %s: %w", cloneURL, err)
}
dir = tmp
}
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("failed to get absolute path: %w", err)
}
// Setup directories
homeDir, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("failed to get home dir: %w", err)
}
logsDir := filepath.Join(homeDir, "micro", "logs")
runDir := filepath.Join(homeDir, "micro", "run")
binDir := filepath.Join(homeDir, "micro", "bin")
for _, d := range []string{logsDir, runDir, binDir} {
if err := os.MkdirAll(d, 0755); err != nil {
return fmt.Errorf("failed to create %s: %w", d, err)
}
}
// Load configuration
cfg, err := config.Load(absDir)
if err != nil {
return fmt.Errorf("failed to load config: %w", err)
}
// Get environment
envName := c.String("env")
if envName == "" {
envName = os.Getenv("MICRO_ENV")
}
if envName == "" {
envName = "development"
}
var envVars []string
if cfg != nil {
if envMap := cfg.GetEnv(envName); envMap != nil {
for k, v := range envMap {
envVars = append(envVars, k+"="+v)
}
}
}
// Discover services
var services []*serviceProcess
servicesByDir := make(map[string]*serviceProcess)
if cfg != nil && len(cfg.Services) > 0 {
// Use configured services in dependency order
sorted, err := cfg.TopologicalSort()
if err != nil {
return fmt.Errorf("dependency error: %w", err)
}
for i, svc := range sorted {
svcDir := filepath.Join(absDir, svc.Path)
absSvcDir, _ := filepath.Abs(svcDir)
hash := fmt.Sprintf("%x", md5.Sum([]byte(absSvcDir)))[:8]
sp := &serviceProcess{
name: svc.Name,
dir: absSvcDir,
binPath: filepath.Join(binDir, svc.Name+"-"+hash),
pidFile: filepath.Join(runDir, svc.Name+"-"+hash+".pid"),
logFile: filepath.Join(logsDir, svc.Name+"-"+hash+".log"),
color: colorFor(i),
port: svc.Port,
env: envVars,
}
services = append(services, sp)
servicesByDir[absSvcDir] = sp
}
} else {
// Auto-discover from main.go files
var mainFiles []string
filepath.Walk(absDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
if info.Name() == "main.go" {
mainFiles = append(mainFiles, path)
}
return nil
})
if len(mainFiles) == 0 {
return fmt.Errorf("no main.go files found in %s", absDir)
}
for i, mainFile := range mainFiles {
svcDir := filepath.Dir(mainFile)
absSvcDir, _ := filepath.Abs(svcDir)
var name string
if absSvcDir == absDir {
name = filepath.Base(absDir)
} else {
name = filepath.Base(svcDir)
}
hash := fmt.Sprintf("%x", md5.Sum([]byte(absSvcDir)))[:8]
sp := &serviceProcess{
name: name,
dir: absSvcDir,
binPath: filepath.Join(binDir, name+"-"+hash),
pidFile: filepath.Join(runDir, name+"-"+hash+".pid"),
logFile: filepath.Join(logsDir, name+"-"+hash+".log"),
color: colorFor(i),
env: envVars,
}
services = append(services, sp)
servicesByDir[absSvcDir] = sp
}
}
if len(services) == 0 {
return fmt.Errorf("no services found")
}
// Start gateway unless disabled
var gw *gateway.Gateway
gatewayAddr := c.String("address")
if gatewayAddr == "" {
gatewayAddr = ":8080"
}
if !c.Bool("no-gateway") {
gw = gateway.New(gatewayAddr)
var svcInfos []gateway.ServiceInfo
for _, svc := range services {
svcInfos = append(svcInfos, gateway.ServiceInfo{
Name: svc.name,
Port: svc.port,
})
}
gw.SetServices(svcInfos)
if err := gw.Start(); err != nil {
return fmt.Errorf("failed to start gateway: %w", err)
}
}
// Start services
for _, svc := range services {
if err := svc.start(logsDir); err != nil {
fmt.Fprintf(os.Stderr, "[%s] %v\n", svc.name, err)
continue
}
// Wait for health if port configured
if svc.port > 0 {
if !waitForHealth(svc.port, 10*time.Second) {
fmt.Fprintf(os.Stderr, "[%s] health check timeout\n", svc.name)
}
}
}
// Print startup banner
printBanner(services, gw, !c.Bool("no-watch"))
// Setup signal handling
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
// Watch mode
watchEnabled := !c.Bool("no-watch")
var watch *watcher.Watcher
if watchEnabled {
var dirs []string
for _, svc := range services {
dirs = append(dirs, svc.dir)
}
watch = watcher.New(dirs)
watch.Start()
go func() {
for event := range watch.Events() {
if svc, ok := servicesByDir[event.Dir]; ok {
fmt.Printf("%s[%s]%s rebuilding...\n", svc.color, svc.name, colorReset)
if err := svc.restart(logsDir); err != nil {
fmt.Fprintf(os.Stderr, "%s[%s]%s restart failed: %v\n", svc.color, svc.name, colorReset, err)
}
}
}
}()
}
// Wait for signal
<-sigCh
fmt.Println("\nShutting down...")
if watch != nil {
watch.Stop()
}
if gw != nil {
gw.Stop()
}
// Stop services in reverse order
for i := len(services) - 1; i >= 0; i-- {
services[i].stop()
}
return nil
}
// Helper functions
func parsePid(pidStr string) int {
pid, _ := strconv.Atoi(pidStr)
return pid
}
func processRunning(pidStr string) bool {
pid := parsePid(pidStr)
if pid <= 0 {
return false
}
proc, err := os.FindProcess(pid)
if err != nil {
return false
}
return proc.Signal(syscall.Signal(0)) == nil
}
func printBanner(services []*serviceProcess, gw *gateway.Gateway, watching bool) {
fmt.Println()
fmt.Println(" ┌─────────────────────────────────────────────────────────────┐")
fmt.Println(" │ │")
fmt.Println(" │ \033[1mMicro\033[0m │")
fmt.Println(" │ │")
if gw != nil {
fmt.Printf(" │ Web: \033[36mhttp://localhost%s\033[0m │\n", gw.Addr())
fmt.Printf(" │ API: \033[36mhttp://localhost%s/api/{service}/{method}\033[0m │\n", gw.Addr())
fmt.Printf(" │ Health: \033[36mhttp://localhost%s/health\033[0m │\n", gw.Addr())
}
fmt.Println(" │ │")
fmt.Println(" │ Services: │")
for _, svc := range services {
status := "\033[32m●\033[0m" // green dot
if !svc.running {
status = "\033[31m●\033[0m" // red dot
}
name := svc.name
if len(name) > 20 {
name = name[:17] + "..."
}
fmt.Printf(" │ %s %-20s │\n", status, name)
}
fmt.Println(" │ │")
if watching {
fmt.Println(" │ \033[33mWatching for changes...\033[0m │")
fmt.Println(" │ │")
}
if gw != nil && len(services) > 0 {
svc := services[0]
fmt.Println(" │ Try: │")
fmt.Printf(" │ \033[90mcurl -X POST http://localhost%s/api/%s/...\033[0m │\n", gw.Addr(), svc.name)
fmt.Println(" │ │")
}
fmt.Println(" └─────────────────────────────────────────────────────────────┘")
fmt.Println()
}
func init() {
cmd.Register(&cli.Command{
Name: "run",
Usage: "Run services with API gateway and hot reload",
Description: `Run discovers and runs services in a directory.
Starts an HTTP gateway on :8080 providing:
- Web dashboard at /
- API proxy at /api/{service}/{endpoint}
- Health checks at /health
With a micro.mu or micro.json config file, services start in dependency order.
Without config, all main.go files are discovered and run.
Examples:
micro run # Run with gateway on :8080
micro run --address :3000 # Gateway on custom port
micro run --no-gateway # Services only, no HTTP gateway
micro run --no-watch # Disable hot reload
micro run --env production # Use production environment`,
Action: Run,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "address",
Aliases: []string{"a"},
Usage: "Gateway address (default :8080)",
Value: ":8080",
},
&cli.BoolFlag{
Name: "no-gateway",
Usage: "Disable HTTP gateway",
},
&cli.BoolFlag{
Name: "no-watch",
Usage: "Disable hot reload (file watching)",
},
&cli.StringFlag{
Name: "env",
Aliases: []string{"e"},
Usage: "Environment to use (default: development)",
EnvVars: []string{"MICRO_ENV"},
},
},
})
}
+168
View File
@@ -0,0 +1,168 @@
// Package watcher provides file watching for hot reload
package watcher
import (
"os"
"path/filepath"
"strings"
"sync"
"time"
)
// Event represents a file change event
type Event struct {
Path string
Dir string // The service directory that was affected
}
// Watcher watches directories for file changes
type Watcher struct {
dirs []string
events chan Event
done chan struct{}
interval time.Duration
debounce time.Duration
mu sync.Mutex
modTimes map[string]time.Time
}
// Option configures the watcher
type Option func(*Watcher)
// WithInterval sets the polling interval
func WithInterval(d time.Duration) Option {
return func(w *Watcher) {
w.interval = d
}
}
// WithDebounce sets the debounce duration for rapid changes
func WithDebounce(d time.Duration) Option {
return func(w *Watcher) {
w.debounce = d
}
}
// New creates a new file watcher for the given directories
func New(dirs []string, opts ...Option) *Watcher {
w := &Watcher{
dirs: dirs,
events: make(chan Event, 100),
done: make(chan struct{}),
interval: 500 * time.Millisecond,
debounce: 300 * time.Millisecond,
modTimes: make(map[string]time.Time),
}
for _, opt := range opts {
opt(w)
}
return w
}
// Events returns the channel of file change events
func (w *Watcher) Events() <-chan Event {
return w.events
}
// Start begins watching for file changes
func (w *Watcher) Start() {
// Initial scan to populate mod times
w.scan(false)
go w.watch()
}
// Stop stops the watcher
func (w *Watcher) Stop() {
close(w.done)
}
func (w *Watcher) watch() {
ticker := time.NewTicker(w.interval)
defer ticker.Stop()
// Track pending events per directory for debouncing
pending := make(map[string]time.Time)
var pendingMu sync.Mutex
for {
select {
case <-w.done:
return
case <-ticker.C:
changed := w.scan(true)
now := time.Now()
pendingMu.Lock()
for _, dir := range changed {
pending[dir] = now
}
// Emit events for directories that have been stable
for dir, t := range pending {
if now.Sub(t) >= w.debounce {
select {
case w.events <- Event{Dir: dir}:
default:
// Channel full, skip
}
delete(pending, dir)
}
}
pendingMu.Unlock()
}
}
}
func (w *Watcher) scan(notify bool) []string {
w.mu.Lock()
defer w.mu.Unlock()
var changed []string
changedDirs := make(map[string]bool)
for _, dir := range w.dirs {
absDir, err := filepath.Abs(dir)
if err != nil {
continue
}
filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return nil
}
// Skip hidden directories and vendor
if info.IsDir() {
name := info.Name()
if strings.HasPrefix(name, ".") || name == "vendor" || name == "node_modules" {
return filepath.SkipDir
}
return nil
}
// Only watch .go files
if !strings.HasSuffix(path, ".go") {
return nil
}
modTime := info.ModTime()
if oldTime, exists := w.modTimes[path]; exists {
if modTime.After(oldTime) && notify {
if !changedDirs[absDir] {
changedDirs[absDir] = true
changed = append(changed, absDir)
}
}
}
w.modTimes[path] = modTime
return nil
})
}
return changed
}
File diff suppressed because it is too large Load Diff
+91
View File
@@ -0,0 +1,91 @@
package server
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"errors"
"os"
"time"
"github.com/golang-jwt/jwt/v5"
)
var (
jwtPrivateKey *rsa.PrivateKey
jwtPublicKey *rsa.PublicKey
)
// Load or generate RSA keys for JWT
func InitJWTKeys(privPath, pubPath string) error {
var err error
if _, err = os.Stat(privPath); os.IsNotExist(err) {
priv, _ := rsa.GenerateKey(rand.Reader, 2048)
privBytes := x509.MarshalPKCS1PrivateKey(priv)
privPem := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: privBytes})
os.WriteFile(privPath, privPem, 0600)
pubBytes, _ := x509.MarshalPKIXPublicKey(&priv.PublicKey)
pubPem := pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: pubBytes})
os.WriteFile(pubPath, pubPem, 0644)
}
privPem, err := os.ReadFile(privPath)
if err != nil {
return err
}
block, _ := pem.Decode(privPem)
if block == nil {
return errors.New("invalid private key PEM")
}
jwtPrivateKey, err = x509.ParsePKCS1PrivateKey(block.Bytes)
if err != nil {
return err
}
pubPem, err := os.ReadFile(pubPath)
if err != nil {
return err
}
block, _ = pem.Decode(pubPem)
if block == nil {
return errors.New("invalid public key PEM")
}
pub, err := x509.ParsePKIXPublicKey(block.Bytes)
if err != nil {
return err
}
var ok bool
jwtPublicKey, ok = pub.(*rsa.PublicKey)
if !ok {
return errors.New("not RSA public key")
}
return nil
}
// Generate a JWT for a user
func GenerateJWT(userID, userType string, scopes []string, expiry time.Duration) (string, error) {
claims := jwt.MapClaims{
"sub": userID,
"type": userType,
"scopes": scopes,
"exp": time.Now().Add(expiry).Unix(),
}
token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
return token.SignedString(jwtPrivateKey)
}
// Parse and validate a JWT, returns claims if valid
func ParseJWT(tokenStr string) (jwt.MapClaims, error) {
token, err := jwt.Parse(tokenStr, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
return nil, errors.New("unexpected signing method")
}
return jwtPublicKey, nil
})
if err != nil {
return nil, err
}
if claims, ok := token.Claims.(jwt.MapClaims); ok && token.Valid {
return claims, nil
}
return nil, errors.New("invalid token")
}
+34
View File
@@ -0,0 +1,34 @@
// Minimal JS for reactive form submissions
document.addEventListener('DOMContentLoaded', function() {
document.querySelectorAll('form[data-reactive]')?.forEach(function(form) {
form.addEventListener('submit', async function(e) {
e.preventDefault();
const formData = new FormData(form);
const params = {};
for (const [key, value] of formData.entries()) {
params[key] = value;
}
const action = form.getAttribute('action');
const method = form.getAttribute('method') || 'POST';
try {
const resp = await fetch(action, {
method,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(params)
});
const data = await resp.json();
// Find or create a response container
let respDiv = form.querySelector('.js-response');
if (!respDiv) {
respDiv = document.createElement('div');
respDiv.className = 'js-response';
form.appendChild(respDiv);
}
respDiv.innerHTML = '<pre>' + JSON.stringify(data, null, 2) + '</pre>';
} catch (err) {
alert('Error: ' + err);
}
});
});
});
+236
View File
@@ -0,0 +1,236 @@
body {
background: #fff;
color: #111;
font-family: 'Inter', 'Segoe UI', 'Arial', 'Helvetica Neue', Arial, sans-serif;
font-size: 15px;
margin: 0;
padding: 0;
line-height: 1.7;
}
header, nav, footer {
background: #fff;
color: #111;
padding: 1.2em 2em 1.2em 2em;
margin-bottom: 2em;
}
nav {
margin: 20px;
border-radius: 20px;
}
main {
max-width: 1400px;
margin: 0 auto;
padding: 2em 1em 3em 1em;
background: #fff;
margin-left: 100px; /* leave space for sidebar */
margin-right: 100px;
}
h1, h2, h3, h4, h5, h6 {
color: #111;
font-weight: 600;
margin-top: 2em;
margin-bottom: 0.5em;
letter-spacing: -0.01em;
}
h1 {
font-size: 2.2em;
margin-top: 0;
}
h2 {
font-size: 1.4em;
}
hr {
border: none;
border-top: 1px solid #222;
margin: 2em 0;
}
a {
color: #111;
text-decoration: none;
transition: background 0.2s;
}
a:hover {
font-weight: bold;
}
ul, ol {
margin: 1em 0 1em 2em;
padding: 0;
}
li {
margin-bottom: 0.5em;
}
pre, code {
background: #f7f7f7;
color: #111;
font-family: inherit;
font-size: 0.98em;
border-radius: 5px;
padding: 0.2em 0.4em;
}
pre {
padding: 1em;
overflow-x: auto;
border-radius: 0;
margin: 1.5em 0;
}
form {
background: #fff;
border: 1px solid #222;
padding: 1.5em 1.5em 1em 1.5em;
margin: 2em 0;
border-radius: 10px;
box-shadow: none;
}
input, select, textarea {
background: #fff;
color: #111;
border: 1px solid #222;
border-radius: 7px;
font-size: 1em;
padding: 0.5em 0.7em;
margin-bottom: 1em;
width: 100%;
box-sizing: border-box;
outline: none;
transition: border 0.2s;
}
input:focus, select:focus, textarea:focus {
border: 1.5px solid #111;
}
button, input[type="submit"], .button {
background: #fff;
color: #111;
border: 1.5px solid #111;
border-radius: 7px;
font-size: 1em;
padding: 0.5em 1.2em;
margin: 0.5em 0.2em 0.5em 0;
cursor: pointer;
font-family: inherit;
transition: background 0.2s, color 0.2s;
}
button:hover, input[type="submit"]:hover, .button:hover {
background: #111;
color: #fff;
}
.table, table {
width: 100%;
border-collapse: collapse;
background: #fff;
margin: 2em 0;
}
table th, table td {
border: none;
padding: 0.7em 1em;
text-align: left;
}
table th {
background: #f7f7f7;
color: #111;
font-weight: 600;
}
table tr:nth-child(even) {
background: #f7f7f7;
}
.no-bullets {
list-style: none;
margin: 0;
padding: 0;
}
.copy-btn {
background: #fff;
color: #111;
border: 1px solid #222;
border-radius: 7px;
font-size: 0.95em;
padding: 0.2em 0.7em;
margin-left: 0.5em;
cursor: pointer;
transition: background 0.2s, color 0.2s;
}
.copy-btn:hover {
background: #111;
color: #fff;
}
.alert, .error, .success {
background: #fff;
color: #111;
border: 1px solid #222;
padding: 1em 1.5em;
margin: 2em 0;
border-radius: 10px;
}
::-webkit-scrollbar {
width: 8px;
background: #fff;
}
::-webkit-scrollbar-thumb {
background: #222;
}
@media (max-width: 800px) {
main {
max-width: 98vw;
padding: 1em 0.2em 2em 0.2em;
margin-left: 0;
}
}
/* Inline/unstyled form for delete button */
.form-inline, .form-plain {
display: inline;
background: none;
border: none;
padding: 0;
margin: 0;
box-shadow: none;
}
.form-inline input, .form-inline button, .form-plain input, .form-plain button {
margin: 0;
padding: 0.3em 1em;
border-radius: 7px;
font-size: 1em;
}
.delete-btn, .form-inline .delete-btn, .form-plain .delete-btn {
background: #fff;
color: #c00;
border: 1.5px solid #c00;
border-radius: 7px;
font-size: 1em;
padding: 0.3em 1em;
margin: 0 0.2em;
cursor: pointer;
font-family: inherit;
transition: background 0.2s, color 0.2s;
}
.delete-btn:hover {
background: #c00;
color: #fff;
}
#title {
text-decoration: none;
}
.log-link:hover {
font-weight: normal;
text-decoration: underline;
}
+34
View File
@@ -0,0 +1,34 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">API</h2>
<p class="api-auth-info" style="background:#f8f8e8; border:1px solid #e0e0b0; padding:1em; margin-bottom:2em; font-size:1.08em; border-radius:6px;">
<b>API Authentication Required:</b> All API calls to <code>/api/...</code> endpoints (except this page) must include an <b>Authorization: Bearer &lt;token&gt;</b> header.<br>
You can generate tokens on the <a href="/auth/tokens">Tokens page</a>.
</p>
{{range .Services}}
<h3 id="{{.Anchor}}" style="margin-top:3em; font-size:1.2em; font-weight:bold;">{{.Name}}</h3>
{{if .Endpoints}}
<div style="margin-bottom:3em;">
{{range .Endpoints}}
<div style="margin-bottom:2.8em; padding:1.3em 1.5em; background:#fafbfc; border-radius:7px; border:1px solid #eee;">
<div style="font-size:1.12em; margin-bottom:0.7em;"><a href="{{.Path}}" class="micro-link" style="font-weight:bold;">{{.Name}}</a></div>
<div style="margin-bottom:0.8em; color:#888; font-size:1em;">
<b>HTTP Path:</b> <code>{{.Path}}</code>
</div>
<div style="display:flex; gap:3em; flex-wrap:wrap;">
<div style="min-width:240px;">
<b>Request:</b>
<pre style="background:#f4f4f4; border-radius:5px; padding:1em 1.2em; margin:0.5em 0 1em 0; font-size:1em;">{{.Params}}</pre>
</div>
<div style="min-width:240px;">
<b>Response:</b>
<pre style="background:#f4f4f4; border-radius:5px; padding:1em 1.2em; margin:0.5em 0 1em 0; font-size:1em;">{{.Response}}</pre>
</div>
</div>
</div>
{{end}}
</div>
{{else}}
<p style="color:#888;">No endpoints</p>
{{end}}
{{end}}
{{end}}
+15
View File
@@ -0,0 +1,15 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Login</h2>
<form method="POST" action="/auth/login" style="max-width:340px; margin:2em 0;">
<div style="margin-bottom:1.2em;">
<input name="id" placeholder="Username" required style="width:100%; padding:0.7em;">
</div>
<div style="margin-bottom:1.2em;">
<input name="password" type="password" placeholder="Password" required style="width:100%; padding:0.7em;">
</div>
<button type="submit" style="width:100%; padding:0.7em;">Login</button>
</form>
{{if .Error}}
<div style="color:#c00; margin-top:1em;">{{.Error}}</div>
{{end}}
{{end}}
+63
View File
@@ -0,0 +1,63 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Auth Tokens</h2>
<table style="margin-bottom:2em;">
<thead>
<tr><th>ID</th><th>Type</th><th>Scopes</th><th>Metadata</th><th>Token</th><th>Delete</th></tr>
</thead>
<tbody>
{{range .Tokens}}
<tr>
<td>{{.ID}}</td>
<td>{{.Type}}</td>
<td>{{range .Scopes}}<code>{{.}}</code> {{end}}</td>
<td>
{{range $k, $v := .Metadata}}
{{if and (ne $k "password_hash") (ne $k "token")}}
<b>{{$k}}</b>: {{$v}}
{{end}}
{{end}}
</td>
<td style="max-width:320px; word-break:break-all;">
{{if .Token}}
<span class="obfuscated-token" data-token="{{.Token}}">
{{if .TokenSuffix}}
{{.TokenPrefix}}...{{.TokenSuffix}}
{{else}}
{{.Token}}
{{end}}
</span>
<button onclick="copyToken(this)" data-token="{{.Token}}" style="margin-left:0.5em;">Copy</button>
{{end}}
</td>
<td>
<form method="POST" action="/auth/tokens" style="display:inline; padding: 0; border: 0">
<input type="hidden" name="delete" value="{{.ID}}">
<button type="submit" onclick="return confirm('Delete token {{.ID}}?')">Delete</button>
</form>
</td>
</tr>
{{end}}
</tbody>
</table>
<h3 style="margin-bottom:1em;">Create New Token</h3>
<form method="POST" action="/auth/tokens">
<input name="id" placeholder="Name/ID" required style="margin-right:1em;">
<select name="type" style="margin-right:1em;">
<option value="user">User</option>
<option value="admin">Admin</option>
<option value="service">Service</option>
</select>
<input name="scopes" placeholder="Scopes (comma separated)" style="margin-right:1em;">
<button type="submit">Create</button>
</form>
<script>
function copyToken(btn) {
const token = btn.getAttribute('data-token');
if (navigator.clipboard) {
navigator.clipboard.writeText(token);
btn.textContent = 'Copied!';
setTimeout(() => { btn.textContent = 'Copy'; }, 1200);
}
}
</script>
{{end}}
+40
View File
@@ -0,0 +1,40 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">User Accounts</h2>
<table style="margin-bottom:2em;">
<thead>
<tr><th>ID</th><th>Type</th><th>Scopes</th><th>Metadata</th><th>Delete</th></tr>
</thead>
<tbody>
{{range .Users}}
<tr>
<td>{{.ID}}</td>
<td>{{.Type}}</td>
<td>{{range .Scopes}}<code>{{.}}</code> {{end}}</td>
<td>
{{range $k, $v := .Metadata}}
{{if ne $k "password_hash"}}
<b>{{$k}}</b>: {{$v}}
{{end}}
{{end}}
</td>
<td>
<form method="POST" action="/auth/users" style="display:inline; padding: 0; border: 0">
<input type="hidden" name="delete" value="{{.ID}}">
<button type="submit" onclick="return confirm('Delete user {{.ID}}?')">Delete</button>
</form>
</td>
</tr>
{{end}}
</tbody>
</table>
<h3 style="margin-bottom:1em;">Create New User</h3>
<form method="POST" action="/auth/users">
<input name="id" placeholder="Username" required style="margin-right:1em;">
<input name="password" type="password" placeholder="Password" required style="margin-right:1em;">
<select name="type" style="margin-right:1em;">
<option value="user">User</option>
<option value="admin">Admin</option>
</select>
<button type="submit">Create</button>
</form>
{{end}}
+52
View File
@@ -0,0 +1,52 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<title>{{.Title}}</title>
<link rel="stylesheet" href="/styles.css">
</head>
<body>
<div id="layout" style="display:flex; min-height:100vh;">
{{if not .HideSidebar}}
<nav id="sidebar" style="width:220px; background:#f5f5f5; padding:2em 1.5em 2em 2em; border:1px solid #eee;">
<h1 style="margin-bottom:1em;"><a href="/" id="title">Micro</a></h1>
{{if .User}}
<div style="margin-bottom:1.5em; font-size:1.05em;">
<span style="color:#888;">Logged in as</span>
<b>{{.User.ID}}</b>
<form method="POST" action="/auth/logout" style="margin-top:0.7em; display:block; background:none; box-shadow:none; padding:0; border:none;">
<button type="submit" style="padding:0.25em 0.8em; font-size:0.97em; border-radius:4px; margin:0; cursor:pointer;">Logout</button>
</form>
</div>
{{else}}
<div style="margin-bottom:1.5em;">
<a href="/auth/login" class="micro-link">Login</a>
</div>
{{end}}
<ul class="no-bullets" style="padding-left:0;">
<li><a href="/" class="micro-link">Home</a></li>
<li><a href="/services" class="micro-link">Services</a></li>
<li><a href="/logs" class="micro-link">Logs</a></li>
<li><a href="/status" class="micro-link">Status</a></li>
<li><a href="/api" class="micro-link">API</a></li>
<li><a href="/auth/tokens" class="micro-link">Tokens</a></li>
<li><a href="/auth/users" class="micro-link">Users</a></li>
</ul>
{{if and .SidebarEndpoints .SidebarEndpointsEnabled}}
<hr style="margin:2em 0 1em 0;">
<div style="font-weight:bold; margin-bottom:0.5em;">API Endpoints</div>
<div style="max-height:40vh; overflow-y:auto; font-size:0.97em;">
{{range .SidebarEndpoints}}
<div style="margin-bottom:0.3em;"><a href="#{{.Anchor}}" class="micro-link">{{.Name}}</a></div>
{{end}}
</div>
{{end}}
</nav>
{{end}}
<main class="container" style="flex:1; min-width:0;">
{{template "content" .}}
</main>
</div>
</body>
</html>
+23
View File
@@ -0,0 +1,23 @@
{{define "content"}}
<h2>{{.ServiceName}}</h2>
<form action="/{{.Action}}" method="POST" data-reactive>
<h3 class="text-lg font-bold mb-2">{{.EndpointName}}</h3>
{{range .Inputs}}
<label class="block font-semibold">{{.Label}}</label>
<input name="{{.Name}}" placeholder="{{.Placeholder}}" class="border rounded px-2 py-1 mb-2 w-full" value="{{.Value}}">
{{end}}
<button class="micro-link mt-2" type="submit">Submit</button>
<div class="js-response"></div>
</form>
{{if .Error}}
<div class="mt-4 text-red-600 font-bold">Error: {{.Error}}</div>
{{end}}
{{if .Response}}
<div class="mt-4">
<h4 class="font-bold mb-2">Response</h4>
{{.ResponseTable}}
<pre class="bg-gray-100 rounded p-2 mt-2">{{.ResponseJSON}}</pre>
</div>
{{end}}
<script src="/main.js"></script>
{{end}}
+21
View File
@@ -0,0 +1,21 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Dashboard</h2>
<div style="display:flex; align-items:center; gap:2em; margin-bottom:2em;">
<div style="display:flex; align-items:center; gap:0.5em;">
<span style="font-size:2.2em; vertical-align:middle;">
{{if eq .StatusDot "green"}}
<span style="display:inline-block; width:1em; height:1em; background:#2ecc40; border-radius:50%;"></span>
{{else if eq .StatusDot "yellow"}}
<span style="display:inline-block; width:1em; height:1em; background:#ffcc00; border-radius:50%;"></span>
{{else}}
<span style="display:inline-block; width:1em; height:1em; background:#ff4136; border-radius:50%;"></span>
{{end}}
</span>
<span style="font-size:1.2em; font-weight:bold;">Status</span>
</div>
<div style="font-size:1.1em;">Services: <b>{{.ServiceCount}}</b></div>
<div style="font-size:1.1em; color:#2ecc40;">Running: <b>{{.RunningCount}}</b></div>
<div style="font-size:1.1em; color:#ff4136;">Stopped: <b>{{.StoppedCount}}</b></div>
</div>
<p>Welcome to the Micro dashboard. Use the sidebar to navigate services, logs, status, and API.</p>
{{end}}
+5
View File
@@ -0,0 +1,5 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Logs for {{.Service}}</h2>
<pre class="bg-gray-100 rounded p-2 mt-2" style="max-height: 60vh; overflow-y: auto;">{{.Log}}</pre>
<a href="/logs" class="micro-link">Back to logs</a>
{{end}}
+8
View File
@@ -0,0 +1,8 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Logs</h2>
<ul class="no-bullets">
{{range .Services}}
<li><a href="/logs/{{.}}" class="micro-link">{{.}}</a></li>
{{end}}
</ul>
{{end}}
+26
View File
@@ -0,0 +1,26 @@
{{define "content"}}
{{if .ServiceName}}
<h2 class="text-xl font-bold mb-2">{{.ServiceName}}</h2>
<h4 class="font-semibold mb-2">Endpoints</h4>
{{if .Endpoints}}
{{range .Endpoints}}
<div><a href="{{.Path}}" class="micro-link">{{.Name}}</a></div>
{{end}}
{{else}}
<p>No endpoints registered</p>
{{end}}
<h4 class="font-semibold mt-4 mb-2">Description</h4>
<pre class="bg-gray-100 rounded p-2">{{.Description}}</pre>
{{else}}
<h2 class="text-2xl font-bold mb-4">Services</h2>
{{if .Services}}
<ul class="no-bullets">
{{range .Services}}
<li><a href="/{{.}}" class="micro-link">{{.}}</a></li>
{{end}}
</ul>
{{else}}
<p>No services registered</p>
{{end}}
{{end}}
{{end}}
+29
View File
@@ -0,0 +1,29 @@
{{define "content"}}
<h2 class="text-2xl font-bold mb-4">Service Status</h2>
<table>
<thead>
<tr>
<th>Service</th>
<th>Directory</th>
<th>Status</th>
<th>PID</th>
<th>Uptime</th>
<th>ID</th>
<th>Logs</th>
</tr>
</thead>
<tbody>
{{range .Statuses}}
<tr>
<td>{{.Service}}</td>
<td><code>{{.Dir}}</code></td>
<td>{{.Status}}</td>
<td>{{.PID}}</td>
<td>{{.Uptime}}</td>
<td style="font-size:0.9em; color:#888;">{{.ID}}</td>
<td><a href="/logs/{{.ID}}" class="log-link">View logs</a></td>
</tr>
{{end}}
</tbody>
</table>
{{end}}
+44 -15
View File
@@ -10,6 +10,7 @@ import (
"go-micro.dev/v5/config"
"go-micro.dev/v5/debug/profile"
"go-micro.dev/v5/debug/trace"
"go-micro.dev/v5/events"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/selector"
"go-micro.dev/v5/server"
@@ -21,27 +22,28 @@ type Options struct {
// Other options for implementations of the interface
// can be stored in a context
Context context.Context
Auth *auth.Auth
Selector *selector.Selector
Profile *profile.Profile
Context context.Context
Auth *auth.Auth
Selector *selector.Selector
DebugProfile *profile.Profile
Registry *registry.Registry
Brokers map[string]func(...broker.Option) broker.Broker
Transport *transport.Transport
Cache *cache.Cache
Config *config.Config
Client *client.Client
Server *server.Server
Caches map[string]func(...cache.Option) cache.Cache
Tracer *trace.Tracer
Profiles map[string]func(...profile.Option) profile.Profile
Brokers map[string]func(...broker.Option) broker.Broker
Transport *transport.Transport
Cache *cache.Cache
Config *config.Config
Client *client.Client
Server *server.Server
Caches map[string]func(...cache.Option) cache.Cache
Tracer *trace.Tracer
DebugProfiles map[string]func(...profile.Option) profile.Profile
// We need pointers to things so we can swap them out if needed.
Broker *broker.Broker
Auths map[string]func(...auth.Option) auth.Auth
Store *store.Store
Stream *events.Stream
Configs map[string]func(...config.Option) (config.Config, error)
Clients map[string]func(...client.Option) client.Client
Registries map[string]func(...registry.Option) registry.Registry
@@ -49,6 +51,7 @@ type Options struct {
Servers map[string]func(...server.Option) server.Server
Transports map[string]func(...transport.Option) transport.Transport
Stores map[string]func(...store.Option) store.Store
Streams map[string]func(...events.Option) events.Stream
Tracers map[string]func(...trace.Option) trace.Tracer
Version string
@@ -81,72 +84,91 @@ func Version(v string) Option {
func Broker(b *broker.Broker) Option {
return func(o *Options) {
o.Broker = b
broker.DefaultBroker = *b
}
}
func Cache(c *cache.Cache) Option {
return func(o *Options) {
o.Cache = c
cache.DefaultCache = *c
}
}
func Config(c *config.Config) Option {
return func(o *Options) {
o.Config = c
config.DefaultConfig = *c
}
}
func Selector(s *selector.Selector) Option {
return func(o *Options) {
o.Selector = s
selector.DefaultSelector = *s
}
}
func Registry(r *registry.Registry) Option {
return func(o *Options) {
o.Registry = r
registry.DefaultRegistry = *r
}
}
func Transport(t *transport.Transport) Option {
return func(o *Options) {
o.Transport = t
transport.DefaultTransport = *t
}
}
func Client(c *client.Client) Option {
return func(o *Options) {
o.Client = c
client.DefaultClient = *c
}
}
func Server(s *server.Server) Option {
return func(o *Options) {
o.Server = s
server.DefaultServer = *s
}
}
func Store(s *store.Store) Option {
return func(o *Options) {
o.Store = s
store.DefaultStore = *s
}
}
func Stream(s *events.Stream) Option {
return func(o *Options) {
o.Stream = s
events.DefaultStream = *s
}
}
func Tracer(t *trace.Tracer) Option {
return func(o *Options) {
o.Tracer = t
trace.DefaultTracer = *t
}
}
func Auth(a *auth.Auth) Option {
return func(o *Options) {
o.Auth = a
auth.DefaultAuth = *a
}
}
func Profile(p *profile.Profile) Option {
return func(o *Options) {
o.Profile = p
o.DebugProfile = p
profile.DefaultProfile = *p
}
}
@@ -157,6 +179,13 @@ func NewBroker(name string, b func(...broker.Option) broker.Broker) Option {
}
}
// New stream func.
func NewStream(name string, b func(...events.Option) events.Stream) Option {
return func(o *Options) {
o.Streams[name] = b
}
}
// New cache func.
func NewCache(name string, c func(...cache.Option) cache.Cache) Option {
return func(o *Options) {
@@ -223,6 +252,6 @@ func NewConfig(name string, t func(...config.Option) (config.Config, error)) Opt
// New profile func.
func NewProfile(name string, t func(...profile.Option) profile.Profile) Option {
return func(o *Options) {
o.Profiles[name] = t
o.DebugProfiles[name] = t
}
}
+1 -1
View File
@@ -5,7 +5,7 @@ This is protobuf code generation for go-micro. We use protoc-gen-micro to reduce
## Install
```
go install github.com/micro/go-micro/cmd/protoc-gen-micro@latest
go install go-micro.dev/v5/cmd/protoc-gen-micro@v5.10.0
```
Also required:
@@ -1,13 +1,12 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.27.1
// protoc v3.15.6
// protoc-gen-go v1.32.0
// protoc v4.25.3
// source: greeter.proto
package greeter
import (
_ "google.golang.org/genproto/googleapis/api/annotations"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
@@ -126,23 +125,20 @@ func (x *Response) GetMsg() string {
var File_greeter_proto protoreflect.FileDescriptor
var file_greeter_proto_rawDesc = []byte{
0x0a, 0x0d, 0x67, 0x72, 0x65, 0x65, 0x74, 0x65, 0x72, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a,
0x1c, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x61, 0x6e, 0x6e, 0x6f,
0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x3c, 0x0a,
0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65,
0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x15, 0x0a, 0x03,
0x6d, 0x73, 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x03, 0x6d, 0x73, 0x67,
0x88, 0x01, 0x01, 0x42, 0x06, 0x0a, 0x04, 0x5f, 0x6d, 0x73, 0x67, 0x22, 0x1c, 0x0a, 0x08, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x10, 0x0a, 0x03, 0x6d, 0x73, 0x67, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6d, 0x73, 0x67, 0x32, 0x6e, 0x0a, 0x07, 0x47, 0x72, 0x65,
0x65, 0x74, 0x65, 0x72, 0x12, 0x2f, 0x0a, 0x05, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x12, 0x08, 0x2e,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x09, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x22, 0x11, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x0b, 0x22, 0x06, 0x2f, 0x68, 0x65, 0x6c,
0x6c, 0x6f, 0x3a, 0x01, 0x2a, 0x12, 0x32, 0x0a, 0x06, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x12,
0x0a, 0x0d, 0x67, 0x72, 0x65, 0x65, 0x74, 0x65, 0x72, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22,
0x3c, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61,
0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x15,
0x0a, 0x03, 0x6d, 0x73, 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x03, 0x6d,
0x73, 0x67, 0x88, 0x01, 0x01, 0x42, 0x06, 0x0a, 0x04, 0x5f, 0x6d, 0x73, 0x67, 0x22, 0x1c, 0x0a,
0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x10, 0x0a, 0x03, 0x6d, 0x73, 0x67,
0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6d, 0x73, 0x67, 0x32, 0x4e, 0x0a, 0x07, 0x47,
0x72, 0x65, 0x65, 0x74, 0x65, 0x72, 0x12, 0x1e, 0x0a, 0x05, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x12,
0x08, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x09, 0x2e, 0x52, 0x65, 0x73, 0x70,
0x6f, 0x6e, 0x73, 0x65, 0x22, 0x0f, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x09, 0x12, 0x07, 0x2f, 0x73,
0x74, 0x72, 0x65, 0x61, 0x6d, 0x28, 0x01, 0x30, 0x01, 0x42, 0x0c, 0x5a, 0x0a, 0x2e, 0x2e, 0x2f,
0x67, 0x72, 0x65, 0x65, 0x74, 0x65, 0x72, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x23, 0x0a, 0x06, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d,
0x12, 0x08, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x09, 0x2e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30, 0x01, 0x42, 0x0c, 0x5a, 0x0a, 0x2e,
0x2e, 0x2f, 0x67, 0x72, 0x65, 0x65, 0x74, 0x65, 0x72, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x33,
}
var (
@@ -5,7 +5,6 @@ package greeter
import (
fmt "fmt"
_ "google.golang.org/genproto/googleapis/api/annotations"
proto "google.golang.org/protobuf/proto"
math "math"
)
@@ -68,6 +67,7 @@ type Greeter_StreamService interface {
Context() context.Context
SendMsg(interface{}) error
RecvMsg(interface{}) error
CloseSend() error
Close() error
Send(*Request) error
Recv() (*Response, error)
@@ -77,6 +77,10 @@ type greeterServiceStream struct {
stream client.Stream
}
func (x *greeterServiceStream) CloseSend() error {
return x.stream.CloseSend()
}
func (x *greeterServiceStream) Close() error {
return x.stream.Close()
}

Some files were not shown because too many files have changed in this diff Show More