Compare commits

...

396 Commits

Author SHA1 Message Date
Shelley 1af588df2f Fix systemd ProtectHome setting
Run Tests / Unit Tests (push) Waiting to run
Run Tests / Etcd Integration Tests (push) Waiting to run
ProtectHome=true prevents services from writing to /home, which breaks
the default micro store that uses ~/.micro. Changed to ProtectHome=false.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:40:18 +00:00
Shelley 2d65ceac6b Fix non-constant format string in deploy error
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:27:45 +00:00
Shelley f9236b4e6b Add install script for micro CLI
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:22:54 +00:00
Shelley 45feec63aa Fix systemd template escaping and rsync permission warnings
- Fix %i escaping in systemd template (was being interpreted by fmt.Sprintf)
- Handle rsync exit code 23/24 gracefully (metadata permission warnings)
- Add --omit-dir-times to rsync to avoid directory timestamp errors

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:22:32 +00:00
Shelley b27dca0a69 Add deployment section to CLI documentation
Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:14:38 +00:00
Shelley de7a34b16e Add deployment documentation
- docs/deployment.md: Comprehensive guide for server deployment
- README.md: Updated deployment section with full workflow

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:13:43 +00:00
Shelley 603c5db778 Add systemd-based deployment support
- micro init --server: Initialize server to receive deployments
  - Creates /opt/micro/{bin,data,config} directories
  - Generates systemd template unit (micro@.service)
  - Creates 'micro' system user

- micro deploy: Deploy services via SSH + systemd
  - Builds linux/amd64 binaries automatically
  - Copies via rsync/scp to server
  - Manages services via systemctl
  - Helpful error messages for common issues

- micro status --remote: Check remote service status
- micro logs --remote: Stream remote logs via journalctl
- micro stop --remote: Stop services on remote server

- Config: Added 'deploy' blocks to micro.mu for named targets

The deployment model:
- systemd is the process supervisor (battle-tested)
- SSH is the transport (standard, secure)
- No custom daemons or platforms needed

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 13:12:44 +00:00
Asim Aslam 239dbfc27e fix: make build/deploy Go-native, Docker optional (#2835)
micro build:
  - Default: builds Go binaries to ./bin/
  - Cross-compile with --os and --arch
  - Docker is optional via --docker flag

micro deploy:
  - Requires --ssh user@host
  - Copies pre-built binaries (if ./bin/ exists)
  - Or syncs source and builds on remote
  - No Docker dependency

Go binaries are self-contained. No runtime needed.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:47:07 +00:00
Asim Aslam de2b3031f3 feat: add micro build and micro deploy commands (#2834)
micro build:
  - Generates Dockerfiles for services (if not present)
  - Builds container images for all services in micro.mu
  - Supports --tag, --registry, --push flags
  - --compose flag generates docker-compose.yml

micro deploy:
  - Default: deploys with docker-compose
  - --ssh user@host: deploys via SSH (rsync + build on remote)
  - --build: rebuild images before deploying

Complete workflow:
  micro run          # Develop locally
  micro build        # Build images
  micro deploy       # Deploy

Or for simple SSH deploys:
  micro deploy --ssh user@host

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:40:04 +00:00
Asim Aslam 40c3ec0e32 feat: add testing package for in-process service testing (#2833)
Provides a test harness for running micro services in isolation:

  h := testing.NewHarness(t)
  defer h.Stop()

  h.Name("users").Register(new(UsersHandler))
  h.Start()

  var rsp Response
  h.Call("UsersHandler.Create", &req, &rsp)

Features:
- Isolated registry, transport, and broker per harness
- Simple API: Name(), Register(), Start(), Stop()
- Call helpers: Call(), CallContext()
- Assertions: AssertServiceRunning(), AssertCallSucceeds(), AssertCallFails()
- Access to underlying Client(), Server(), Registry()

Note: Due to go-micro's global defaults, each harness tests one service.
For multi-service testing, use integration tests or mocks.

Also fixes README.md example showing conflicting port 8080.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:35:05 +00:00
Asim Aslam 139e70e880 feat(run): integrate HTTP gateway with micro run (#2832)
micro run now starts an HTTP gateway alongside your services:

  - Web dashboard at http://localhost:8080
  - API proxy at /api/{service}/{method}
  - Health checks at /health
  - Service listing at /services

The experience is now:
  $ micro new helloworld
  $ cd helloworld
  $ micro run

  Open http://localhost:8080 to see and call your services.

New flags:
  --address :3000    # Custom gateway port
  --no-gateway       # Disable gateway (services only)

Updated documentation to make this the central experience.

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:22:45 +00:00
Asim Aslam 94e83e57f8 feat: add health check package for K8s-style probes (#2831)
Adds a new health package providing:
- /health endpoint for overall health status
- /health/live for Kubernetes liveness probes
- /health/ready for Kubernetes readiness probes
- Built-in checks: PingCheck, TCPCheck, HTTPCheck, DNSCheck
- Critical vs non-critical checks
- Concurrent check execution
- Configurable timeouts
- Service info metadata

Integrates with micro run's health check waiting when services
specify a port in their micro.mu configuration.

Example usage:

    health.Register("database", health.PingCheck(db.Ping))
    health.Register("redis", health.TCPCheck("localhost:6379", time.Second))
    health.RegisterHandlers(mux)

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:10:38 +00:00
Asim Aslam 39484560ea docs: add micro run documentation with hot reload and config file guide (#2830)
- Update main README with micro run quick start
- Expand cmd/micro/README.md with configuration options
- Add detailed guide at internal/website/docs/guides/micro-run.md

Documents:
- Hot reload with file watching
- micro.mu DSL configuration
- micro.json alternative
- Dependency ordering
- Environment management
- Graceful shutdown

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:04:15 +00:00
Asim Aslam 7690c41d5f feat(run): add hot reload, config files, and dependency ordering (#2829)
- Add micro.mu DSL and micro.json config file support
- Implement hot reload with file watching (--no-watch to disable)
- Start services in dependency order (topological sort)
- Environment management (--env flag, MICRO_ENV var)
- Health check waiting before starting dependents
- Graceful shutdown in reverse dependency order

Config file example (micro.mu):

    service users
        path ./users
        port 8081

    service posts
        path ./posts
        port 8082
        depends users

    env development
        STORE_ADDRESS file://./data

Closes #2828

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 12:01:47 +00:00
Asim Aslam cae6fbbe76 Framework hardening: security, reliability, and developer experience improvements (#2826)
* fix: remove deprecated rand.Seed calls

Go 1.20+ automatically seeds the global random number generator.
These calls are no-ops and generate warnings with newer Go versions.

Removed from:
- selector/strategy.go
- registry/cache/cache.go
- broker/memory.go
- broker/http.go
- cmd/cmd.go
- transport/memory.go

Co-authored-by: Shelley <shelley@exe.dev>

* fix: handle previously ignored errors

- MySQL store: properly handle prepared statement errors in initDB()
- Consul registry: handle client creation errors in Client() method

These silent failures could cause hard-to-debug issues in production.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(genai): improve provider interface with context and streaming

Breaking changes:
- Generate() and Stream() now require context.Context as first parameter
- Stream.Close() added for proper resource cleanup

Improvements:
- Proper context support for cancellation and timeouts
- Real SSE streaming for OpenAI and Gemini text generation
- Better error handling with wrapped errors and API error responses
- Thread-safe provider registry with sync.RWMutex
- New options: WithMaxTokens, WithTemperature, WithTimeout
- Stream has proper Close() method for cleanup
- Results can include Error field for per-chunk errors

Provider updates:
- OpenAI: true streaming with SSE parsing, proper HTTP client with timeout
- Gemini: true streaming with streamGenerateContent endpoint
- Default model updated to gpt-4o-mini (OpenAI) and gemini-2.0-flash (Gemini)

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): make TLS secure by default, configurable via environment

BREAKING: TLS now verifies certificates by default. Set MICRO_TLS_INSECURE=true
to restore previous behavior (NOT recommended for production).

Changes:
- Add util/tls.Config(), SecureConfig(), InsecureConfig(), ConfigFromEnv() helpers
- Update all components to use ConfigFromEnv() instead of hardcoded InsecureSkipVerify
- Set MinVersion to TLS 1.2 for all TLS configs

Affected components:
- broker/http
- broker/rabbitmq
- registry/etcd
- registry/consul
- transport/grpc

This improves security posture while allowing opt-out for development environments.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): add TLS helpers with opt-in secure mode

NOT a breaking change - keeps InsecureSkipVerify=true as default for
local development compatibility.

New util/tls helpers:
- Config() - returns config based on MICRO_TLS_SECURE env var
- SecureConfig() - certificate verification enabled
- InsecureConfig() - certificate verification disabled (dev only)

For production security, use one of:
- Set MICRO_TLS_SECURE=true with proper CA-signed certs
- Use a service mesh (Istio, Linkerd) for automatic mTLS
- Configure TLSConfig directly with your certificates

Also: Changed CLI alias from 'g' to 'gen' for clarity
- micro generate handler -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

* refactor(cli): rename generate directory to gen for consistency

Directory name now matches the command alias:
  cmd/micro/cli/gen/ -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

---------

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 10:39:25 +00:00
Asim Aslam a32cdbc117 feat(web): add SSE broadcaster for real-time event streaming (#2825)
This adds Server-Sent Events (SSE) support to the web package, enabling
real-time updates in web applications built with go-micro.

Features:
- SSEBroadcaster manages connected clients and broadcasts events
- Integrates with events.Stream for subscribing to topics
- Supports JSON events and raw HTML (for htmx/datastar)
- Keep-alive mechanism for long-lived connections
- Thread-safe client management

Usage:
  broadcaster := web.NewSSEBroadcaster(
    web.WithStream(stream),
    web.WithTopics("posts", "comments"),
  )
  broadcaster.Start()
  router.GET("/events", broadcaster.Handler())

This enables real-time features requested in #2788, including:
- Live updates without page refresh
- htmx/datastar integration
- Foundation for MCP (Model Context Protocol) support

Closes #2788

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 06:29:34 +00:00
Copilot baeb282cf1 Enhance cache rate limiting to prevent registry overload during rolling deployments (#2824)
* Initial plan

* Enhance cache rate limiting to protect against rolling deployment scenarios

Per @asim's feedback, rate limiting now applies to ALL cache refresh attempts
(not just error cases) to prevent registry overload during rolling deployments.

Changes:
- Rate limit ALL refresh attempts using lastRefreshAttempt tracking
- Always return stale cache if available (even if expired) when rate limiting
- Only return ErrNotFound when no cache exists during rate limit period
- Rate limiting happens inside singleflight to avoid race conditions
- Update test to reflect new behavior (no retry when stale cache + rate limit)
- Enhanced documentation with rolling deployment scenario examples

This addresses the scenario where all upstream services expire their cache
simultaneously during a downstream rolling deployment, which would previously
cause a stampede to the registry under high QPS.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Remove unused failedAttempts and consecutiveFailures fields

Per @asim's feedback, these fields are no longer needed since the new
rate limiting strategy uses lastRefreshAttempt to track ALL refresh
attempts, not just failed ones.

Removed:
- failedAttempts map[string]time.Time
- consecutiveFailures int
- All code that sets these fields

The rate limiting logic now only uses lastRefreshAttempt, making these
fields redundant. All tests continue to pass.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-14 10:57:14 +00:00
Copilot 2869cc16d1 Add adaptive throttling to registry cache to prevent penetration when no stale cache exists (#2823)
* Initial plan

* Add adaptive throttling to prevent cache penetration without stale cache

Implement rate limiting mechanism to protect etcd when:
- Cache is empty (no stale data available)
- Registry is failing/overloaded
- Multiple sequential requests arrive

Changes:
- Add MinimumRetryInterval option (default 5s)
- Track last failed attempt time per service
- Throttle requests within retry interval when no cache exists
- Clear throttling on successful lookup
- Add comprehensive tests validating throttling behavior

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix WaitGroup race condition in events stream tests

The WaitGroup.Add() calls were happening after goroutines started,
causing a race where Done() could be called before Add(), resulting
in "negative WaitGroup counter" panics.

Fixes:
- Move wg.Add(1) before goroutine in TestConsumeTopic
- Move wg.Add(2) before first goroutine in TestConsumeGroup

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Update registry cache documentation with adaptive throttling feature

Document the new adaptive throttling feature that prevents cache
penetration when registry is failing and no stale cache exists.

Includes:
- Feature overview
- Usage examples with configuration options
- Explanation of throttling behavior
- Example scenario demonstrating protection

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-09 12:27:55 +00:00
Copilot dbb66ec938 Fix etcd auth overload and cache stampede with KeepAlive channels (#2822)
* Initial plan

* Replace KeepAliveOnce with KeepAlive to reduce etcd auth requests

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add tests to verify cache penetration protection via singleflight

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add etcd integration tests to CI workflow

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix race conditions and improve code quality based on review feedback

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add workflow permissions to fix security scan finding

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Add comprehensive documentation for performance improvements

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix memory store limit/offset bug causing events test failure

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix memory store to filter before limiting in prefix/suffix reads

The previous fix had a logic error where limit/offset were applied before
prefix/suffix filtering. This could cause incorrect results when the first
N items in the unfiltered list don't match the search criteria.

Now filters first to get all matching keys, then applies limit/offset to
the filtered results. This ensures ReadLimit(1) always returns 1 matching
record if available, regardless of map iteration order.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-09 11:12:06 +00:00
Copilot 80345fe63d docs: clarify gRPC server option ordering and service name usage (#2820)
* Initial plan

* docs: clarify gRPC server option ordering and service name usage

- Fix all examples to show Server option before Name option
- Add note explaining why option ordering matters
- Add new section on "Option Ordering Issue" in Common Errors
- Add new section on "Service Name vs Package Name" in Common Errors
- Update transport.md to show proper ordering with comment

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* docs: refine comments based on code review feedback

- Clarify that Server ordering before Name is mandatory
- Remove confusing comment about Client ordering being for consistency

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2025-11-25 09:26:59 +00:00
Copilot 00a119496e Add documentation for native gRPC compatibility vs transport (#2819)
* Initial plan

* Add documentation for native gRPC compatibility with grpc client/server packages

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix go_package path in proto example to use relative path

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2025-11-25 07:28:48 +00:00
Asim Aslam 3e25be984d x 2025-11-13 20:48:01 +00:00
Asim Aslam 4e8934c230 Get Badge 2025-11-13 20:46:20 +00:00
Asim Aslam a368be82cd add showcase 2025-11-13 20:42:45 +00:00
Asim Aslam 43386d4ec1 x 2025-11-13 20:37:54 +00:00
Asim Aslam a84f92c907 mobile hamburger menu 2025-11-13 20:34:09 +00:00
Asim Aslam 751de9e7c0 fix mobile rendering for website 2025-11-13 20:31:29 +00:00
Asim Aslam be1cf9c9d8 x 2025-11-13 20:06:28 +00:00
Asim Aslam d7699845bb Add docs link 2025-11-13 20:01:32 +00:00
Asim Aslam 520a0fa140 Update features 2025-11-13 19:59:28 +00:00
Asim Aslam 8a1591af0c x 2025-11-13 19:58:38 +00:00
Asim Aslam 1f1d9875a2 update landing 2025-11-13 19:56:01 +00:00
Asim Aslam cd4c881db5 x 2025-11-13 19:52:06 +00:00
Asim Aslam de03bbbf81 x 2025-11-13 19:49:34 +00:00
Asim Aslam ee4f656fa5 fix typo in search documentation 2025-11-13 19:26:19 +00:00
Asim Aslam 2cce3e5e1a fix links 2025-11-13 19:21:27 +00:00
Asim Aslam 0526a42efa changes to theme 2025-11-13 19:14:01 +00:00
Asim Aslam dc1e1eb45c fix config file 2025-11-13 19:04:30 +00:00
Asim Aslam 06b31f545a update the docs to easily navigate 2025-11-13 18:59:34 +00:00
Asim Aslam 8cda829320 major docs overhaul 2025-11-13 18:34:40 +00:00
Asim Aslam e755e4a823 updates for syntax highlighting 2025-11-13 18:17:45 +00:00
Asim Aslam eef06fcf01 server docs 2025-11-13 18:12:15 +00:00
Asim Aslam 9e36df224b new copilot generated documentation 2025-11-13 18:11:29 +00:00
Asim Aslam 2da1cc0edd Delete .github/PULL_REQUEST_TEMPLATE.md 2025-11-13 17:19:17 +00:00
Asim Aslam 0ef0143537 Delete .github/ISSUE_TEMPLATE/question.md 2025-11-13 17:19:04 +00:00
Asim Aslam 30dc01523a Delete .github/ISSUE_TEMPLATE/feature-request---enhancement.md 2025-11-13 17:18:54 +00:00
Asim Aslam 4a0648536f Delete .github/ISSUE_TEMPLATE/bug_report.md 2025-11-13 17:18:43 +00:00
nswdewy 01ed999b2a Change template paths from 'html' to 'web' (#2816) 2025-11-06 10:17:30 +00:00
Asim Aslam acfb4e2639 Update README.md 2025-10-29 10:46:41 +00:00
Asim Aslam cd735aba56 Update sponsorship link in README.md 2025-10-29 10:46:14 +00:00
Asim Aslam 8c995ea8ac Update sponsorship link in README 2025-10-29 10:45:28 +00:00
Asim Aslam e9ce3487b6 Update README.md 2025-10-29 10:44:16 +00:00
Asim Aslam 79f5a7a7f6 Fix README formatting and sponsor button placement 2025-10-29 10:43:48 +00:00
Asim Aslam 48c310f70b Update README.md 2025-10-29 10:43:32 +00:00
Asim Aslam 5ffb711dfe Add funding configuration for GitHub 2025-10-29 10:42:01 +00:00
Asim Aslam de5057e35d Update README to remove install script section
Removed mention of install script from README.
2025-10-22 21:19:31 +01:00
Asim Aslam 91d91eef1f Fix for https://github.com/micro/go-micro/issues/2687 2025-10-22 06:38:16 +00:00
asim 48479228b1 Fixes https://github.com/micro/go-micro/issues/2738 2025-10-22 07:18:37 +01:00
asim 5a07370970 Fix https://github.com/micro/go-micro/issues/2769 2025-10-22 07:10:41 +01:00
Asim Aslam db9b1e8d79 Update installation command to specific version 2025-10-20 11:41:26 +01:00
Asim Aslam 3e05108882 Update installation command to specific version 2025-10-20 11:41:04 +01:00
Asim Aslam 85cd80a467 Delete cmd/README.md 2025-10-14 19:34:27 +01:00
Eng Zer Jun bc40df0c9e Update github.com/imdario/mergo to dario.cat/mergo (#2810)
Mergo v1 is released with a new module path URL. No breaking changes,
only the module path update and bug fixes.

Reference: https://github.com/darccio/mergo/releases/tag/v1.0.0
Reference: https://github.com/darccio/mergo/releases/tag/v1.0.1
Reference: https://github.com/darccio/mergo/releases/tag/v1.0.2

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-10-14 11:18:28 +01:00
asim 66014eac68 . 2025-10-14 11:17:03 +01:00
asim 26b9d56165 . 2025-10-14 11:15:52 +01:00
asim 6574180b0d . 2025-10-14 11:14:36 +01:00
asim e6c9f4fa39 . 2025-10-14 11:13:53 +01:00
asim 1dde737b64 move micro cli and protoc-gen-micro to cmd/ 2025-10-14 11:13:35 +01:00
Asim Aslam 26adf49e55 Remove Command Line section from README
Removed command line section from README.
2025-10-14 10:33:20 +01:00
Eng Zer Jun 7a9c445321 Update github.com/streadway/amqp to github.com/rabbitmq/amqp091-go (#2811)
The `github.com/streadway/amqp` module is no longer actively maintained.
The new module is now maintained by the RabbitMQ core team under a
different package name.

Reference: https://github.com/rabbitmq/amqp091-go

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-10-13 21:34:07 +01:00
Asim Aslam 44cc31345d use buf pool 2025-10-07 16:38:09 +00:00
Asim Aslam aa2ed9aa91 Patch for potential deadlock 2025-10-07 16:20:45 +00:00
Asim Aslam 3d6b6521ab fix atomic int alignment bug 2025-10-07 15:57:43 +00:00
Kenley Wang 7b0fa8b6ae fix: new pool cannot customize closetimeout (#2808) 2025-10-02 14:28:06 +01:00
asim 059d317969 . 2025-09-29 10:28:59 +01:00
Asim Aslam e0af3b61a5 Change 'Toolkit' to 'Command Line' in README
Updated section headers in README for clarity.
2025-09-28 09:55:43 +01:00
Asim Aslam 5bbc24abe1 Clarify micro CLI usage and genai package details
Updated the README to clarify the usage of the micro CLI and the genai package.
2025-09-28 08:51:30 +01:00
Asim Aslam dfcaae7bf8 Update link text from 'Docs' to 'Go Doc' 2025-09-27 09:45:52 +01:00
Ak-Army 9070b3befd [feature] ability to use wildcard in topic name (factory-events.*.*) (#2804) 2025-09-09 12:51:39 +01:00
ExtinctPatronageDeodorize e5cd820c71 fix data race in memory.update (#2802)
Co-authored-by: ExtinctPatronageDeodorize <ExtinctPatronageDeodorize@users.noreply.github.com>
2025-08-29 14:14:11 +01:00
Tsln 479bd58c3f fix consul context key mismatch (#2801) 2025-08-29 07:39:20 +01:00
Byron Silvers 95540b7859 fix consul watcher address handling (#2800) 2025-08-19 09:27:34 +01:00
Asim Aslam be2559c555 Update index.html 2025-08-05 14:15:06 +01:00
Asim Aslam 6e53f541d3 Update README.md 2025-08-05 14:14:28 +01:00
Asim Aslam a6ede13f73 Update index.html 2025-08-04 11:56:59 +01:00
Asim Aslam 31d2d39f76 Update README.md 2025-07-24 21:24:04 +01:00
Roman Perekhod ad73add529 fix the concurrent map writes error (#2794) 2025-07-16 07:27:45 +01:00
asim f99a205b2b add string function 2025-07-15 20:43:58 +01:00
Asim Aslam 3cf5540b0f Update README.md 2025-07-08 19:24:19 +01:00
Asim Aslam 2a140858a8 Update README.md 2025-07-08 19:23:49 +01:00
Asim Aslam 2d19a304fc Update README.md 2025-07-08 19:23:15 +01:00
asim 7b48449a85 fix options 2025-07-07 10:23:23 +01:00
asim 680987aeeb . 2025-07-02 09:56:34 +01:00
Asim Aslam ee9f3afe37 GenAI interface (#2790)
* genai interface

* x

* x

* text to speech

* Re-add events package (#2761)

* Re-add events package

* run redis as a dep

* remove redis events

* fix: data race on event subscriber

* fix: data race in tests

* fix: store errors

* fix: lint issues

* feat: default stream

* Update file.go

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>

* .

* copilot couldn't make it compile so I did

* copilot couldn't make it compile so I did

* x

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>
2025-06-20 10:24:31 +01:00
Asim Aslam 7e1bba2baf Re-add events package (#2761)
* Re-add events package

* run redis as a dep

* remove redis events

* fix: data race on event subscriber

* fix: data race in tests

* fix: store errors

* fix: lint issues

* feat: default stream

* Update file.go

---------

Co-authored-by: Brian Ketelsen <bketelsen@gmail.com>
2025-06-18 17:12:02 +01:00
blacksheepaul dd0944bf68 Update getting-started.md (#2787) 2025-06-12 09:56:34 +01:00
Ak-Army 88f38eaef6 Subscribe error handling (#2785)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [feature] always subscribes to all topics and if there is an error in one of them, the unsuccessful topics will be subscribed to again

---------

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
2025-06-04 08:55:17 +01:00
Asim Aslam 0e45edf439 Set service name as table name (#2780)
* add docs layout

* update all to use _layouts

* update the styling

* always use service name as table name
2025-05-21 22:43:29 +01:00
Asim Aslam 788dcd05b2 Docs (#2779)
* add docs layout

* update all to use _layouts

* update the styling
2025-05-21 13:48:03 +01:00
Asim Aslam 456cd7e092 Add links to readme contents 2025-05-21 12:06:25 +00:00
Asim Aslam 3fd52c66e7 Generated docs for the following files: 2025-05-21 12:03:24 +00:00
Asim Aslam 3c3ba55c45 Create README.md (#2778) 2025-05-21 11:22:11 +01:00
Asim Aslam 1a8074467a Update and rename docs.yml to website.yml (#2777) 2025-05-21 11:18:54 +01:00
Asim Aslam b9665e32c5 Move whole website (#2776)
* adding docs

* minor getting started typo

* move the whole website
2025-05-21 11:17:04 +01:00
Asim Aslam 11b7eb0727 Docs (#2775)
* minor getting started typo
2025-05-21 10:59:04 +01:00
Asim Aslam 5a409f2607 Create docs.yml 2025-05-21 10:49:59 +01:00
Asim Aslam 8e771c57e1 adding docs (#2774) 2025-05-21 10:47:47 +01:00
Brian Ketelsen ddc34801ee Plugins and profiles (#2764)
* feat: more plugins

* chore(ci): split out benchmarks

Attempt to resolve too many open files in ci

* chore(ci): split out benchmarks

* fix(ci): Attempt to resolve too many open files in ci

* fix: set DefaultX for cli flag and service option

* fix: restore http broker

* fix: default http broker

* feat: full nats profile

* chore: still ugly, not ready

* fix: better initialization for profiles

* fix(tests): comment out flaky listen tests

* fix: disable benchmarks on gha

* chore: cleanup, comments

* chore: add nats config source
2025-05-20 13:24:06 -04:00
Brian Ketelsen e12504ce3a feat: re-add profiles (#2772)
* feat: re-add profiles

* fix: make profile separate package

* fix: make profile separate package

* fix: profile flag

* fix: defaults
2025-05-19 13:59:28 -04:00
asim a03ab5f601 switch location of store dir to home dir 2025-05-19 09:28:13 +01:00
Asim Aslam 44ff301d2d syntactically easy way to register commands (#2770) 2025-05-18 09:45:49 +01:00
Asim Aslam 37bb1a8ab6 Revert "WIP: move file store (do not merge) (#2766)" (#2768)
This reverts commit 97275d3db9.
2025-05-16 19:06:45 +01:00
BombartSimon 1fe2638298 Implement MDNS Registry (#2767)
- Removed existing mDNS test file and replaced it with a new implementation.
- Added a new `mdns_registry.go` file that contains the MDNS registry logic.
- Updated the default registry to use the new MDNS registry.
- Refactored tests to accommodate the new MDNS registry implementation.
- Implemented service registration, deregistration, and service discovery using mDNS.
- Added encoding and decoding functions for mDNS TXT records.
- Implemented a watcher for monitoring service changes in the MDNS registry.
2025-05-16 19:03:36 +01:00
Asim Aslam 97275d3db9 WIP: move file store (do not merge) (#2766)
* move file store

* fix build
2025-05-16 15:08:13 +01:00
BombartSimon e29159e836 Rename 'Users' section to 'Adopters' in README (#2765) 2025-05-15 22:20:10 +01:00
Asim Aslam caba761c7b fix extraction field naming with standard Go types (#2763) 2025-05-15 18:59:10 +01:00
Brian Ketelsen cd2b40ca4a Add a selection of plugins to the core repo (#2755)
* WIP

* fix: default memory registry, add registrations for mdns, nats

* fix: same for broker

* fix: add more

* fix: http port

* rename redis

* chore: linting
2025-05-15 18:47:35 +01:00
Asim Aslam 7c04b7cfd6 Update README.md (#2762) 2025-05-15 16:13:27 +01:00
Asim Aslam b4a87d05f7 Update README.md (#2759) 2025-05-15 09:37:12 +01:00
asim 0e47cbf1a2 fix conflicting port range test 2025-05-14 22:11:23 +01:00
asim 12dfa797dc drop the separate go mods 2025-05-14 22:03:38 +01:00
Asim Aslam 230505bf5a Add grpc plugin (#2758)
* Add grpc plugin

* remove stale readmes
2025-05-14 21:04:42 +01:00
Asim Aslam 01b8394c81 Update README.md 2025-05-07 19:39:11 +01:00
Asim Aslam f9d08a14f3 Update README.md (#2756) 2025-05-07 19:35:20 +01:00
Asim Aslam a997f738dd Update README.md 2025-05-07 19:34:21 +01:00
Asim Aslam 46db7df218 Update README.md 2025-05-07 17:07:20 +01:00
Brian Ketelsen 17c04258a4 fix: go version breaking unit tests (#2754) 2025-05-05 21:12:34 -04:00
asim 8eb280126a add go mod 2025-05-05 22:02:32 +01:00
asim f51cd8d883 default to file store 2025-05-05 22:02:11 +01:00
asim ef4dc8b5b0 add json.NewValues to config 2025-05-05 14:39:54 +01:00
asim 23b14123ea errors for go 1.24wq 2025-05-04 21:48:02 +01:00
asim 2388f662cf break config.Get and return error with value 2025-05-04 21:31:17 +01:00
asim 60474ed38f add store encode/decode for record 2025-05-04 20:13:58 +01:00
asim 484eb3d15e public functions for store 2025-05-04 20:00:44 +01:00
asim c51095a074 store.NewRecord 2025-05-04 19:55:10 +01:00
Asim Aslam 03782bc9b3 move service to service dir to make life not suck (#2753) 2025-05-04 19:44:56 +01:00
Asim Aslam 1040b7c58e Update README.md 2025-05-01 10:27:59 +01:00
Asim Aslam bd0e9ca7cb Update README.md 2025-05-01 10:27:37 +01:00
Asim Aslam f5399d56c3 Update README.md 2025-05-01 10:25:44 +01:00
Asim Aslam 0f4c238804 protoc-gen-micro has moved to micro/micro/cmd/protoc-gen-micro (#2751) 2025-04-30 16:50:27 +01:00
Asim Aslam 66c169076c Update README.md 2025-04-25 09:43:41 +01:00
Asim Aslam 280eb5b46d Update README.md 2025-04-25 09:42:43 +01:00
Asim Aslam 4ead4ff953 Update README.md (#2750) 2025-04-25 09:41:35 +01:00
Asim Aslam 200a3cb0ce Update README.md (#2749) 2025-04-23 16:04:24 +01:00
Asim Aslam 049dea6804 Update README.md 2025-04-23 15:56:47 +01:00
asim 3fa2a38d76 syntatic sugar for micro.New("helloworld") 2025-04-23 12:21:21 +01:00
asim 65af48823f yolo development is bad 2025-04-23 12:16:34 +01:00
asim 156a968253 directly support handler in the service interface 2025-04-23 12:13:23 +01:00
Morya 517b2b0855 Update README.md (#2748)
go install github.com/micro/go-micro/cmd/protoc-gen-micro@latest

should be used, instand of 

`go install github.com/micro/go-micro/cmd/protoc-gen-micro`
2025-04-22 17:09:21 +01:00
Pavel Ivanov 4702afe57d Fix http transport client blocking recv (#2744)
* reproduce blocking recv

* fix blocking recv call on httpTransportClient

* prevent race condition

* refactoring
2025-04-10 15:24:54 +01:00
Jörn Friedrich Dreyer e032a6aafd update ttl for updated nodes only (#2740)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-11-15 14:08:17 +00:00
Jörn Friedrich Dreyer 14a1791011 invalidate service if node was not updated (#2736)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-10-28 15:34:38 +00:00
Morya b318b7f097 go mod tidy (#2730) 2024-08-26 07:20:29 +01:00
Ak-Army 0433e98dbc Better connection pool handling (#2725)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [fix] Use pool connection close timeout

* [fix] replace Close with private function

* [fix] Do not close the transport client twice in stream connection , the transport client is closed in the rpc codec

* [fix] tests

---------

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
2024-07-23 12:19:43 +01:00
Asim Aslam 1c6c1ff1a3 Update README.md (#2722) 2024-07-15 11:55:17 +01:00
Asim Aslam 4c34451125 Fix data race (#2721) 2024-07-12 09:36:43 +01:00
asim 9a7cd8ce66 update readme 2024-07-08 18:52:06 +01:00
asim dd0145fa18 unused packages 2024-07-08 18:48:52 +01:00
asim 72df27b7d1 remove util/sync 2024-07-08 18:46:34 +01:00
asim e9a52070e6 k8s not needed 2024-07-08 18:44:42 +01:00
asim 6e393f6abf move cmd package back to top level. Strip grpc plugin 2024-07-07 22:38:11 +01:00
asim 90531337db . 2024-07-07 19:14:04 +01:00
asim e756fd32e6 Fix panic 2024-07-07 19:09:19 +01:00
asim 7c3f272001 remove sync package 2024-07-07 19:03:18 +01:00
asim 66b3fd8893 fix retry test 2024-07-07 19:01:43 +01:00
asim c64cb1cb03 more references to runtime 2024-07-07 18:50:52 +01:00
asim 8876002e57 more references to runtime 2024-07-07 18:49:39 +01:00
asim a3809afbdf more references to runtime 2024-07-07 18:44:18 +01:00
asim 563e0d265d meh 2024-07-07 18:40:24 +01:00
asim 3d5f87c01b no one is using that 2024-07-07 18:40:15 +01:00
asim bac34aaec1 still more fixes 2024-07-07 18:36:04 +01:00
asim db0fa9fe1f fix bugs 2024-07-07 18:32:26 +01:00
asim 3676232df1 strip runtime 2024-07-07 18:30:48 +01:00
asim 29d79d748d remove runtime that no one uses 2024-07-07 18:28:38 +01:00
asim 627066baf9 remove events entirely 2024-07-07 18:26:30 +01:00
asim d84da9a8eb meh 2024-07-07 18:19:50 +01:00
asim 6c0a073e33 Merge branch 'master' of ssh://github.com/micro/go-micro 2024-07-07 18:19:19 +01:00
asim 12e8fcd057 Fix parallel test 2024-07-07 18:18:43 +01:00
Asim Aslam 4dcf2e58c0 Update README.md (#2720) 2024-07-07 08:05:08 +01:00
Asim Aslam b56cfaf818 Revert license to Apache 2 2024-07-07 08:04:29 +01:00
Asim Aslam ff52e9f5d0 Delete .github/FUNDING.yml 2024-07-07 07:18:47 +01:00
Asim Aslam c05c11e0c3 Update README.md (#2719) 2024-07-04 10:29:05 +01:00
asim fc614aef2d add back generator 2024-07-04 09:18:52 +01:00
Ak-Army 1515db5240 Fix stream eos error (#2716)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [fix] Do not send error when stream send eos header, just close the connection

* [fix] Do not overwrite the error in http client

---------

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
2024-07-02 13:06:14 +01:00
littlecoderonway 4d5b3b084f Fix registry cache broken (#2715)
* services is shared between routines, when application change services returned by get function may lead to other routine set a changed services to cache

* services is shared between routines, when application change services returned by get function may lead to other routine set a changed services to cache

---------

Co-authored-by: Shuaihu Liu <shuaihu.liu@shopee.com>
2024-07-02 09:30:51 +01:00
Asim Aslam e9ceb66e32 Update README.md 2024-06-05 18:28:17 +01:00
asim 610c00859f v5 2024-06-04 21:40:43 +01:00
Asim Aslam e11395c8f5 Update LICENSE (#2712)
BSL license
2024-06-04 21:37:52 +01:00
Di Wu 63f2507944 fix(sec): CVE-2024-24786 (#2699) 2024-06-04 20:23:15 +01:00
Asim Aslam 0c6c907c58 Delete CHANGELOG.md
Not being maintained
2024-06-04 20:21:45 +01:00
Asim Aslam 53f691da30 Update README.md (#2711) 2024-06-04 20:21:05 +01:00
Asim Aslam 8f43ae86dd Update FUNDING.yml (#2710) 2024-06-04 20:01:41 +01:00
dependabot[bot] f9ce51c522 chore(deps): bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#2706)
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.5 to 1.1.12.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 18:51:46 +01:00
dependabot[bot] 08d98adcd2 chore(deps): bump golang.org/x/net from 0.17.0 to 0.23.0 (#2705)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 13:56:58 +01:00
guangwu 186b8351dc fix: close dockerfile (#2703) 2024-04-17 07:45:39 +01:00
Z.Q.K b1e58a1fe8 fix typo (#2701) 2024-03-26 03:39:45 +00:00
Ak-Army f1a8b39309 Fix double close in stream client (#2693)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [fix] Do not close the transport client twice in stream connection , the transport client is closed in the rpc codec

---------

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2024-02-15 20:26:36 +00:00
Asim Aslam 6e55bb1a0e Update README.md (#2692) 2024-02-10 18:47:38 +00:00
Morya f28468a59c feat: allow to set API listen address (#2680) 2023-12-18 10:29:47 +00:00
Asim Aslam 3a4790b3c5 Update tests.yaml (#2681) 2023-12-18 10:28:44 +00:00
Eng Zer Jun cbd45b12dc Avoid allocations with (*regexp.Regexp).MatchString (#2679)
We should use `(*regexp.Regexp).MatchString` instead of
`(*regexp.Regexp).Match([]byte(...))` when matching string to avoid
unnecessary `[]byte` conversions and reduce allocations.

func BenchmarkMatch(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := versionRe.Match([]byte("v1")); !match {
			b.Fail()
		}
	}
}

func BenchmarkMatchString(b *testing.B) {
	for i := 0; i < b.N; i++ {
		if match := versionRe.MatchString("v1"); !match {
			b.Fail()
		}
	}
}

goos: linux
goarch: amd64
pkg: go-micro.dev/v4/api/handler/event
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkMatch-16          	11430127	       127.4 ns/op	       2 B/op	       1 allocs/op
BenchmarkMatchString-16    	12220628	        97.54 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	go-micro.dev/v4/api/handler/event	3.822s

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-12-04 16:00:01 +00:00
guangwu 252385e39c chore: replace for loop with call to copy (#2678) 2023-11-30 11:18:05 +00:00
ChengDaqi2023 68c026c5d8 update golang.org/x/net v0.8.0 to 0.17.0 (#2677) 2023-11-29 10:24:36 +00:00
Elmut ca6190f5f2 append to subscribers (#2640)
* append to subscribers

* Update rpc_router.go

error correction log
2023-11-26 11:08:28 +01:00
Guillaume Bour 674b9822e0 util/addr: Fixes findIP to return public IP if present. (#2673) 2023-11-26 11:06:55 +01:00
Christian Richter 7392705af8 bump urfave cli & fix race condition (#2659)
* bump urfave cli

Signed-off-by: Christian Richter <crichter@owncloud.com>

* Fix race condition

Co-authored-by: André Duffeck <andre.duffeck@firondu.de>

Signed-off-by: Christian Richter <crichter@owncloud.com>

---------

Signed-off-by: Christian Richter <crichter@owncloud.com>
2023-10-09 10:33:57 +01:00
Asim Aslam 27c488712e Update README.md (#2652) 2023-08-07 09:46:15 +01:00
Asim Aslam c3d7c65c88 Update README.md (#2650) 2023-07-20 07:49:53 +01:00
Asim Aslam 384814c885 Update README.md (#2648) 2023-07-17 09:33:34 +01:00
Asim Aslam 8a3a98a8b8 Update README.md (#2647) 2023-07-16 12:02:53 +01:00
asim 416f65e814 rename test dir 2023-07-11 09:25:35 +01:00
clearcode 8020303017 Update README.md (#2635) 2023-05-06 21:39:24 +02:00
mamadeusia 67d48b205e Add Context in event options (#2634)
Co-authored-by: mamadeusia <timadues7775@gmail.com>
2023-05-03 13:24:36 +01:00
dependabot[bot] 31135d4696 Bump github.com/docker/docker from 20.10.7+incompatible to 20.10.24+incompatible (#2625)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-26 02:22:47 +02:00
Coder 1478a8e46d fix(config/source/cli): mergo.Map error, src and dst must be of same … (#2628) 2023-04-26 02:21:08 +02:00
Mohamed MHAMDI ad8fca255b fix(config): fix file source watcher stop behavior when Stop is called (#2630)
Co-authored-by: Mohamed MHAMDI <mmhamdi@hubside.com>
2023-04-26 02:19:52 +02:00
Lukasz Raczylo a7522e7d6c fix: struct field alignment (#2632) 2023-04-26 02:16:34 +02:00
Asim Aslam 0f9b2f00c9 Update README.md 2023-04-12 11:09:56 +01:00
Asim Aslam 7fe95e8732 Update README.md 2023-04-12 11:09:40 +01:00
Asim Aslam d44ed328d1 Add Handle option (#2627) 2023-04-11 10:29:03 +01:00
Thomas Chandelle d392e72021 fix(api/handler): avoid zombie goroutine when connection unexpectedly closes (#2624) 2023-04-05 22:50:24 +02:00
dependabot[bot] 683e2729b4 Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5 (#2623)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 20:19:21 +02:00
asim 6f8930926f gofmt 2023-03-22 16:42:54 +00:00
Asim Aslam 2117672933 Update FUNDING.yml 2023-03-21 20:51:42 +00:00
Asim Aslam d7e692b5d1 Update README.md 2023-03-21 15:19:07 +00:00
asim d949258b2f update the logo 2023-03-20 16:48:44 +00:00
fuyou d376656528 fix(sec): upgrade github.com/containerd/containerd to 1.6.18 (#2617)
* update github.com/containerd/containerd v1.4.3 to 1.6.18

* fix(sec): upgrade containerd

* fix(sec): go mod tidy

---------

Co-authored-by: Davincible <david.brouwer.99@gmail.com>
2023-03-07 17:28:22 +01:00
dependabot[bot] 415b3e3a2f Bump golang.org/x/net from 0.0.0-20210510120150-4163338589ed to 0.7.0 (#2615)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20210510120150-4163338589ed to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/commits/v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-07 17:11:10 +01:00
Hellis e337eb2c3d fix(api): add WithRegistry option for api (#2618) 2023-03-07 17:05:25 +01:00
dependabot[bot] 521e6b644c Bump golang.org/x/crypto from 0.0.0-20210513164829-c07d793c2f9a to 0.1.0 (#2619)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20210513164829-c07d793c2f9a to 0.1.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/commits/v0.1.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-07 17:04:20 +01:00
Stanislav ad0b4f103d feat(errors): append errors use variadic arguments (#2606) 2023-01-03 12:58:03 +01:00
Davincible 68a6425ad8 fix(service): profile stop error scope 2022-11-16 05:07:41 +01:00
leoujz 0a91ba7b5d api gateway handing http request adds Content-Type application/x-www-form-urlencoded, and extract endpoints from path if no endpoint matched (#2592)
Co-authored-by: l <l@x>
2022-11-09 04:28:39 +01:00
645775992 a17eaf64da update gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b to 3.0.0 (#2589) 2022-11-04 12:06:13 +01:00
645775992 6df085fd6d update github.com/opencontainers/runc v0.1.1 to 1.1.2 (#2590) 2022-11-04 12:05:41 +01:00
David Brouwer 697ea1b21e Update README.md 2022-11-02 21:45:35 +01:00
JellyTony b442dbb56b fix: fix configuration version data competition (#2586)
Co-authored-by: JeffreyBool <zhanggaoyuan@mediatrack.cn>
2022-10-26 12:12:44 +02:00
Sagan Yaroslav b988a78bae fix: store table initialization (#2584) 2022-10-20 14:59:08 +02:00
David Brouwer a3980c2308 feat: add test framework & refactor RPC server (#2579)
Co-authored-by: Rene Jochum <rene@jochum.dev>
2022-10-20 13:00:50 +02:00
keepstep c25dee7c8a fix(registry/cache): do not watch when ttl=0 eg: some custom registry no s… (#2580)
Co-authored-by: keep <keep@pa.com>
2022-10-08 15:19:48 +02:00
Rene Jochum 24dfcef49a fix(transport/memory): Improve the memory transport, 4x speed (#2581) 2022-10-07 23:54:09 +02:00
Rene Jochum 065f9714e9 fix: easy lint fixes to api/ (#2567) 2022-10-01 10:50:11 +02:00
Rene Jochum 010b1d9f11 fix: linting issues (#2566) 2022-09-30 20:32:55 +02:00
David Brouwer 85c0b0b8eb fix: some linting issues (#2563) 2022-09-30 16:27:07 +02:00
David Brouwer 47e6a8d725 feat(CI): add linting and pretty test output (#2562) 2022-09-30 02:08:42 +02:00
Mohamed MHAMDI 1db36357d5 feat(logger): add logger option to all micro components (override DefaultLogger) closes #2556 (#2559)
Run tests / Test repo (push) Waiting to run
* feat(logger): add logger option to all components

* fix: refactor api/rpc.go

* fix: refactor api/stream.go

* fix: api/options.go comment

* fix(logger): do not use logger.Helper internally

* fix(logger): fix comments

* fix(logger): use level.Enabled method

* fix: rename mlogger to log

* fix: run go fmt

* fix: log level

* fix: factories

Co-authored-by: Mohamed MHAMDI <mmhamdi@hubside.com>
Co-authored-by: Davincible <david.brouwer.99@gmail.com>
2022-09-29 16:44:53 +02:00
David Brouwer 57a0ef5a0f docs: update README shields (#2558)
* docs: add badges to README.md

* docs: new discord badge

* docs: update discord badge

* docs: update discord badge

* docs: remove shield
2022-09-24 03:40:48 +02:00
Mohamed MHAMDI 39e00f11a8 feat(config): add withFs option to file source (#2557)
Co-authored-by: Mohamed MHAMDI <mmhamdi@hubside.com>
2022-09-24 02:46:18 +02:00
David Bereza 34b6434791 fix: prevent returning empty strings for list (#2553) 2022-09-06 20:03:23 +01:00
Asim Aslam a702a1b097 Update README.md 2022-08-17 10:02:52 +01:00
asim d32c8bd0eb move logo
Run tests / Test repo (push) Waiting to run
2022-08-17 09:50:32 +01:00
刘志铭 d7d8123bc7 Fix a problem with the Register Memory component (#2545) 2022-08-13 11:23:42 +01:00
David Bereza f40cfdda82 feat(sync): Add additional context option to sync (#2544) 2022-08-11 13:09:25 +08:00
Arvin 51250bf248 fix: Fix a logic judgment bug. (#2540)
Signed-off-by: Arvin <1458070668@qq.com>
2022-08-02 17:54:51 +01:00
Arvin 63a9b94820 support use of listen options (#2536)
* feat: support use of listen options

* style: Adjust the import order of packages.
2022-07-30 10:56:14 +01:00
Asim Aslam adcc1761d0 Update README.md 2022-07-24 06:39:24 +01:00
Asim Aslam cc53b5e7d7 Update README.md 2022-07-23 11:25:07 +01:00
asim 4d63d61c20 remove function
Run tests / Test repo (push) Waiting to run
2022-07-22 20:12:24 +01:00
asim 678c227061 remove deprecated function 2022-07-22 20:11:27 +01:00
asim ec847fa60c remove generate file 2022-07-22 20:10:06 +01:00
asim 616060876f remove file 2022-07-22 20:08:37 +01:00
asim 641be04fc0 remove plugins dir 2022-07-22 20:07:34 +01:00
Asim Aslam 28f36e8fd6 strip debug handler (#2532)
* strip debug handler

* fix tests
2022-07-22 20:03:27 +01:00
asim a8224e1c67 Merge branch 'master' of ssh://github.com/asim/go-micro 2022-07-22 10:22:58 +01:00
asim 38293f479a strip wrappers for trace/stats 2022-07-22 10:22:50 +01:00
Ak-Army 48eae3b968 Fix http transport panic (#2531)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

* [fix] Read buff before reset it, when the connection is closed

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2022-07-22 08:40:58 +01:00
Asim Aslam e28f5b97f6 Update README.md 2022-07-20 10:34:27 +01:00
Asim Aslam 140f90b354 Update README.md 2022-07-20 10:12:05 +01:00
Asim Aslam 467c8a34e3 Update README.md 2022-07-18 11:40:43 +01:00
Ak-Army 582e3f9310 Http transport stream (#2529)
* [fix] etcd config source prefix issue (#2389)

* http transport data race issue (#2436)

* [fix] #2431 http transport data race issue

* [feature] Ability to close connection while receiving.
Ability to send messages while receiving.
Icreased r channel limit to 100 to more fluently communication.
Do not dropp sent request if r channel is full.

Co-authored-by: Johnson C <chengqiaosheng@gmail.com>
Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2022-07-15 11:04:41 +01:00
Asim Aslam 98375c7ae5 api honours naming convention of other interfaces 2022-07-11 15:49:30 +01:00
Asim Aslam 7e903d80b6 Update README.md 2022-07-11 15:41:03 +01:00
Asim Aslam 6cf2b02f0f move cmd to util (#2527)
* move cmd to util

* go fmt
2022-07-11 14:37:34 +01:00
Asim Aslam b36949f2c1 Update README.md 2022-07-11 13:08:54 +01:00
Asim Aslam 995abaaea6 go mod tidy 2022-07-11 09:51:48 +01:00
Asim Aslam 97d59fbf78 remove generator and move to github.com/go-micro 2022-07-11 09:11:59 +01:00
Asim Aslam ff1312438c Update README.md 2022-07-07 12:49:56 +01:00
Asim Aslam edaa353826 strip body from gateway 2022-07-04 10:15:14 +01:00
Asim Aslam 48d6650f28 rename api to gateway 2022-07-02 14:42:12 +01:00
Asim Aslam 3381a9f3db Remove body from endpoint 2022-07-02 14:38:58 +01:00
Asim Aslam 63d1f9de4a minor changes 2022-07-02 14:25:21 +01:00
Asim Aslam e313863a2a make router configurable in api 2022-07-02 14:15:02 +01:00
Asim Aslam 73dc3723c7 Update api.go 2022-07-02 13:50:45 +01:00
Asim Aslam 667ee9b85e rename field to Versions 2022-07-02 12:35:57 +01:00
Asim Aslam b6b866c0b2 api package (#2522)
* api gateway

* add comment
2022-07-02 12:11:59 +01:00
Johnson C 28298a30e4 feat: remove plugins and related docs (#2521)
* feat: redis implementation of cache

* Revert "feat: redis implementation of cache"

This reverts commit 4b3f25c1fc494d934613992d8a762024fafad7b6.

* feat: remove plugins
2022-07-02 10:25:57 +01:00
Johnson C b884a68347 feat: update out of date docs (#2520)
Run tests / Test repo (push) Waiting to run
* feat: redis implementation of cache

* Revert "feat: redis implementation of cache"

This reverts commit 4b3f25c1fc494d934613992d8a762024fafad7b6.

* feat: update out of date docs
2022-07-02 08:44:32 +01:00
Asim Aslam bfcdfc3c7f remove m3o
Run tests / Test repo (push) Waiting to run
2022-07-01 11:37:13 +01:00
Asim Aslam 392610022a Update README.md 2022-07-01 08:38:57 +01:00
Asim Aslam 34a51c9f29 Update README.md 2022-07-01 08:34:20 +01:00
Asim Aslam a5fb3fab66 Update README.md 2022-07-01 08:33:53 +01:00
JellyTony 96576a612a feat(web): add start and stop functions to the web (#2519) 2022-07-01 08:06:27 +01:00
Thomas Chandelle 87c96bc4d0 Correctly check error on redis command XTRIM (#2518)
Also:
 - add comment about version requirement
 - move minID to var for code readability
2022-06-30 18:24:27 +08:00
JellyTony 65a2e4ed02 fix(registry): fix watch services only first time (#2517) 2022-06-29 08:49:22 +01:00
Abdul Hadi 9960cd1d36 typo siza #2429 (#2515) 2022-06-24 19:18:47 +01:00
David Brouwer 107bd74187 fix: allow event subscribers to use pointers (#2514)
When a subscriber would use a pointer as event type, panics would occur.
2022-06-22 09:01:13 +08:00
Johnson C 7a6a2b6373 feat: redis implementation of cache (#2513)
* feat: redis implementation of cache

* Revert "feat: redis implementation of cache"

This reverts commit 4b3f25c1fc494d934613992d8a762024fafad7b6.

* feat: redis implementation of cache
2022-06-21 20:05:00 +01:00
Asim Aslam cf51ddeb26 Update README.md 2022-05-30 08:50:02 +01:00
Asim Aslam 13b76331ec Update README.md
Run tests / Test repo (push) Waiting to run
2022-05-11 09:55:41 +01:00
Asim Aslam de0b456b46 Update README.md 2022-05-10 06:59:08 +01:00
yulian 072547201c remove unused variable in loop (#2495) 2022-05-03 11:06:13 +08:00
bosima 8293988499 fix: consume and publish blocked after rabbitmq reconnecting (#2492)
* Support direct generation of grpc method when package and service names of proto files are different.

* fix req.Interface() return nil.

* Get rid of dependence on 'Micro-Topic'

* Revert "Get rid of dependence on 'Micro-Topic'"

This reverts commit 3ff69443364d39f5fda3a32fc2249826e2d207dd.

* Revert "fix req.Interface() return nil."

This reverts commit 90a1b34195e07772fa6f2074e1cf22237ac2a87f.

* Revert "Revert "fix req.Interface() return nil.""

This reverts commit e64737b7da8d1767c4456881f6730f1c196cea60.

* Revert "Revert "Get rid of dependence on 'Micro-Topic'""

This reverts commit 141bb0a557c81cb6d1c651b085b3e65483d5e681.

* fix: consume and publish blocked after reconnecting

Co-authored-by: maxinglun <maxinglun@zhijiaxing.net>
2022-04-28 17:55:06 +08:00
Asim Aslam 367771923c Update README.md 2022-04-19 15:47:45 +01:00
Asim Aslam 4ca75f3ab3 Update README.md 2022-04-19 15:47:31 +01:00
Asim Aslam 4a7c16419e Update README.md 2022-04-19 15:31:23 +01:00
Asim Aslam 1d396bdd64 Update README.md 2022-04-19 15:31:11 +01:00
Sergey Galkin ca807e36c5 option to disable file watcher (#2485)
Co-authored-by: Sergey Galkin <v.sergey.galkin@reddit.com>
2022-04-18 19:30:14 +01:00
Asim Aslam cce02927e7 Update README.md 2022-04-18 19:20:45 +01:00
Wang 034ba9a0de typo fix; (#2480) 2022-04-15 09:55:30 +08:00
bosima 1919048c8f Support direct generation of grpc method (#2474)
* Support direct generation of grpc method when package and service names of proto files are different.
* fix req.Interface() return nil.
2022-04-10 22:12:39 +08:00
Asim Aslam 62c2981baf remove cli 2022-04-07 11:46:07 +01:00
Asim Aslam efa2c4f2d2 remove dashboard 2022-04-07 11:45:58 +01:00
Asim Aslam 27b98c450e Remove examples 2022-04-07 11:45:39 +01:00
Asim Aslam e8604f065c remove services 2022-04-07 11:45:22 +01:00
Asim Aslam aaca7c27da Update README.md 2022-04-07 11:44:47 +01:00
Asim Aslam 664ee31b1a Update README.md 2022-04-07 08:20:57 +01:00
Wang 6dedee5d8c Add header suppor for Kafka broker plugin; (#2470) 2022-04-04 19:54:19 +01:00
Wang d1806e2883 Fix codec/bytes (#2466)
* Update marshaler.go

Byes codec always return error "invalid message" now.

* Update go.mod

Update package name

* Update go.mod
2022-04-02 18:11:12 +01:00
Morya 1f086a3002 fix https://github.com/asim/go-micro/issues/2344 (#2462)
fix https://github.com/asim/go-micro/issues/2344
2022-04-01 07:10:51 +01:00
Muhammad Iqbal Alaydrus 73eda3346d Add Wait option support for sync/etcd plugins (#2459)
* Add Wait support for sync/etcd plugins

* Use ErrLockTimeout if context deadline exceeded
2022-03-25 10:28:19 +08:00
Chris Moran c6d352c832 #2453 Fix with associated test update (#2454) 2022-03-17 10:22:05 +08:00
baerwang 356448017f style:arrays pre-allocation (#2449) 2022-03-14 14:03:14 +08:00
dependabot[bot] d6a74c1a8e Bump github.com/nats-io/nats-server/v2 in /plugins/events/natsjs (#2447)
Bumps [github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) from 2.6.2 to 2.7.4.
- [Release notes](https://github.com/nats-io/nats-server/releases)
- [Changelog](https://github.com/nats-io/nats-server/blob/main/.goreleaser.yml)
- [Commits](https://github.com/nats-io/nats-server/compare/v2.6.2...v2.7.4)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-server/v2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-14 12:02:12 +08:00
Willy Kloucek e5a35d38f9 fix natsjs syntax error, remove TODOs and enable tests (#2446) 2022-03-11 16:03:35 +08:00
Willy Kloucek a2f6fac852 add NATS JetStream events plugin (#2433)
* add NATS JetStream plugin

* add notes about de-duplication and inprogress call

* fix typo
2022-03-04 14:53:15 +00:00
Ak-Army dca3a3b553 [fix] http transport deadlock (#2441)
Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2022-02-24 09:32:09 +00:00
AlkaidChen 4154ed6a80 add tls option for sync etcd plugin (#2440) 2022-02-24 17:07:14 +08:00
Chris Moran 69b4b5d1c6 Support for -micro_out=module=<module_prefix> for protoc-gen-micro (#2435) 2022-02-23 13:14:50 +08:00
Asim Aslam d47c2d984b add new services 2022-02-16 08:43:36 +00:00
Asim Aslam 91cfd18373 Update README.md 2022-02-16 08:35:22 +00:00
Asim Aslam 9e0be6c85d add service updates (#2418)
Run tests / Test repo (push) Waiting to run
2022-01-18 15:27:36 +00:00
Asim Aslam 23f1de80c5 Update README.md 2022-01-17 13:35:01 +00:00
Asim Aslam 6bc97c16ee Update README.md 2022-01-17 13:34:30 +00:00
Asim Aslam a612e09a34 Add files via upload 2022-01-15 20:26:27 +00:00
Asim Aslam 1e2d197c7d default the content type to json (#2412) 2022-01-11 13:36:20 +00:00
Asim Aslam 3a999a0db6 Add files via upload 2022-01-11 12:07:54 +00:00
Asim Aslam b8c2eb2b28 Update README.md 2022-01-11 08:35:55 +00:00
Asim Aslam 6f15174a2f Update README.md 2022-01-11 08:35:16 +00:00
Asim Aslam 748fb39cac Add files via upload 2022-01-11 08:34:58 +00:00
simon 5f2251cfad Add Kafka asynchronous send support (#2409)
* Add Kafka asynchronous send support

* Add Kafka asynchronous send support

* Upgrade sarama to 1.30.1

* Example
2022-01-09 14:12:10 +00:00
Amir Khazaie ef7528ebdb use read lock and unlock instead of write ones (#2410)
Co-authored-by: Amir Khazaie <a.khazaie@digikala.com>
2022-01-09 10:20:08 +00:00
Asim Aslam 696a0ba714 Merge branch 'master' of ssh://github.com/asim/go-micro 2022-01-03 11:21:29 +00:00
Asim Aslam 267da35259 move the api client 2022-01-03 11:21:20 +00:00
Asim Aslam d152870bdb Update client_test.go 2021-12-31 12:39:12 +00:00
Asim Aslam da41ab146e switch services client 2021-12-31 10:11:49 +00:00
Asim Aslam 757d0fe343 fix client 2021-12-30 09:47:40 +00:00
Asim Aslam c8d94c44f0 remove the client 2021-12-30 09:42:08 +00:00
Asim Aslam 5fa6e89f4f fix go mod 2021-12-30 09:35:21 +00:00
Asim Aslam 85f1c44000 Delete go.sum 2021-12-30 08:58:45 +00:00
Asim Aslam 3ba69f11ff Delete go.mod 2021-12-30 08:58:34 +00:00
Asim Aslam af5e70a1d6 add service interfaces 2021-12-30 08:44:34 +00:00
Johnson C 5fe884e59f [FEATURE] add changelog (#2400)
change logs for release history
2021-12-29 11:01:40 +08:00
isfk 415016c6e4 nats config plugin (#2397) 2021-12-28 11:27:36 +08:00
Asim Aslam e0de23c546 Update and rename m3o.go to services.go 2021-12-24 11:43:20 +00:00
Asim Aslam 7af0eb4b7a Update services/ (#2392) 2021-12-21 13:18:29 +00:00
isfk 8e52761edb fix context value nil (#2391) 2021-12-20 16:31:48 +08:00
Johnson C 37de747d19 [fix] nats deregister issue (#2384)
Run tests / Test repo (push) Waiting to run
2021-12-10 19:32:21 +08:00
Johnson C a40f6e8fae [fix] fixing f.IsExported undefined issue (#2382)
IsExported needs go1.17, replace with PkgPath
2021-12-07 17:30:48 +08:00
Johnson C b00c8368b9 [feat] dashboard update (#2381)
* [feature] dashboard update
2021-12-07 10:42:31 +08:00
Ak-Army 97f169c424 fix http_transport Recv and Close race condition on buff (#2374)
fix rpc_stream panic override with the "Unlock of unlocked RWMutex" panic

Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2021-12-05 11:56:17 +00:00
Ak-Army 81244a41f1 Extend client mock with ability to test publish, and a few useful method like SetResponse and SetSubscriber (#2375)
Co-authored-by: Hunyadvári Péter <peter.hunyadvari@vcc.live>
2021-12-05 11:55:54 +00:00
JeffreyBool 229b369702 [fix] update protoc-gen-micro install doc 2021-12-04 12:52:02 +08:00
Johnson C c0e0b2b225 [FEATURE] dashboard support publish messages (#2373)
* [feature] dashboard

* [feat] support publish message
2021-12-02 13:15:38 +08:00
Johnson C 1e4dd94b71 [fix] zookeeper registry delete event (#2372)
* [fix] #2358 zookeeper delete event
2021-12-01 16:26:31 +08:00
zhaoyang b25d744f5c feat: delete redundant lines (#2364) 2021-11-24 17:41:54 +00:00
zhaoyang 685f9979a1 feat: modify the dependencies urls (#2363) 2021-11-24 16:43:05 +00:00
Asim Aslam 6fc5e45626 Update README.md 2021-11-24 07:32:51 +00:00
Asim Aslam 2c6515152a Update load.go 2021-11-24 07:32:20 +00:00
Johnson C 70d2bd8a6b [feature] dashboard (#2361) 2021-11-24 11:27:23 +08:00
Johnson C 90b3e4af0b [fix] ignore unexported field (#2354)
ignore unexported field when register endpoints
2021-11-18 17:07:00 +08:00
Niek den Breeje 4f1a571704 Fix Micro CLI's proto comments (#2353) 2021-11-18 06:10:43 +01:00
陈杨文 799b8d6a65 upgrade to go 1.17 (#2346)
Run tests / Test repo (push) Waiting to run
2021-11-11 14:03:34 +00:00
Asim Aslam 8e312801a1 Update README.md 2021-11-08 09:03:37 +00:00
Asim Aslam 335c4e54a1 Merge branch 'master' of ssh://github.com/asim/go-micro 2021-11-08 09:01:56 +00:00
Asim Aslam e6d17257b0 add nats and redis events plugins 2021-11-08 08:59:14 +00:00
Asim Aslam 0c2041e439 add events package (#2341)
* add events package

* update go version
2021-11-08 08:52:39 +00:00
Johnson C c5be9f560c fix(#2333): etcd grpc marshal issue (#2334)
make protocodec compatible with legacy proto message type
2021-11-03 10:58:05 +08:00
Asim Aslam 62b985f49c Update README.md 2021-11-01 21:44:52 +00:00
Asim Aslam 2b9a6f9aeb flatten cli (#2332)
Run tests / Test repo (push) Waiting to run
2021-11-01 14:34:09 +00:00
Asim Aslam 3c70d23a1d m3o client changed 2021-11-01 11:00:21 +00:00
Asim Aslam adaa98e6cf use vanity url for cli command 2021-11-01 09:00:14 +00:00
无相 ed90a65783 fix broker nsq plugin nil pointer error (#2329)
Co-authored-by: longhaoteng <longhaoteng@kingsoft.com>
2021-11-01 08:47:22 +00:00
无相 268278c53e fix config json slice parsing (#2330)
Co-authored-by: longhaoteng <longhaoteng@kingsoft.com>
2021-11-01 08:47:00 +00:00
Benjamin 5d5aee1f08 replace ioutil with io and os (#2327)
set the go version to 1.16 in pr.yml and tests.yml, so as to be consistent with the version in go.mod.
2021-10-30 19:24:40 +01:00
Johnson C ed690ed838 fixing #2308 (#2326)
IPV6 too many colons in address
net.SplitHostPort need ipv6 address in [host]:port format
2021-10-28 17:03:48 +08:00
Niek den Breeje e155029a4b Fix Micro CLI by removing replace line in go.mod 2021-10-27 17:40:26 +02:00
Asim Aslam 880d751df4 Update README.md 2021-10-27 16:34:55 +01:00
Niek den Breeje a3202b00ee Rename gomu to micro (#2325)
* Rename Gomu to Micro

* docs: Remove license reference in CLI's README
2021-10-27 16:31:29 +01:00
Asim Aslam efcda9a48c Update README.md 2021-10-26 20:17:07 +01:00
Asim Aslam 2c73c7fcf3 strip protoc-gen-micro go mod 2021-10-26 19:46:25 +01:00
Johnson C d2a51d05c4 [feature] stream CloseSend (#2323)
* support stream CloseSend

* move CloseSend into Closer
2021-10-26 15:07:08 +01:00
Johnson C af3cfa0a4c remove unnecessary dependencies between plugins
remove unnecessary dependencies between plugins
upgrade go-micro.dev/v4 to v4.2.1
2021-10-23 16:20:42 +08:00
Johnson C f96b48dad9 1. use default memory registry in grpc plugins (#2317)
2. try fixing grpc plugin failed to get issue
use v4.0.0-v4.0.0-00010101000000-000000000000 instead of specific version
3. kafka panic on disconnect
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x18 pc=0x1266c50]

goroutine 31 [running]:
github.com/asim/go-micro/plugins/broker/kafka/v3.(*kBroker).Disconnect(0xc0002400c0)
        C:/Workshop/Go/pkg/mod/github.com/asim/go-micro/plugins/broker/kafka/v3@v3.7.0/kafka.go:130 +0xd0
github.com/asim/go-micro/plugins/server/grpc/v3.(*grpcServer).Start.func2()
        C:/Workshop/Go/pkg/mod/github.com/asim/go-micro/plugins/server/grpc/v3@v3.0.0-20210712061837-0532fd9de8ae/grpc.go:998 +0xc8d
created by github.com/asim/go-micro/plugins/server/grpc/v3.(*grpcServer).Start
        C:/Workshop/Go/pkg/mod/github.com/asim/go-micro/plugins/server/grpc/v3@v3.0.0-20210712061837-0532fd9de8ae/grpc.go:917 +0xcaf
exit status 2
2021-10-22 22:30:28 +08:00
Niek den Breeje 9edc569e68 Add update rule to Makefile (#2315)
I realized that when writing `require go-micro.dev/v4 v4.1.0` in the
`go.mod` file, `go mod tidy` will install that exact version of
`go-micro.dev/v4`. As of right now, v4.1.0 is an outdated version, and
preferably we shouldn't be updating gomu's Go Modules template everytime
a new release is tagged, but still want gomu users to generate projects
using the latest Go Micro version.

In an attempt to solve this problem, I'm opting to add a new Makefile
rule for new projects generated by gomu, `update` that runs `go get -u`.
This aggressively updates any dependencies your Go Modules project may
have.  `go mod tidy` is then able to prune the `go.mod` and `go.sum`
files by removing the unnecessary checksums and transitive dependencies
(e.g. `// indirect`), that were added to by go get -u due to newer
semver available.

Put one and one together and you get two. In addition to adding an
`update` rule to the Makefile generated for new projects by gomu, I'm
also updating the proto and client comments printed when new projects
have been generated to promote the `update` rule.

References:

- https://stackoverflow.com/questions/67201708/go-update-all-modules
2021-10-19 21:12:42 +02:00
Johnson C 29fefbad4e Plugins (#2311)
* release plugins

* add window plugins release bat script

Co-authored-by: Johnson C <johnson.cheng@scenestek.com>
2021-10-16 05:56:51 +01:00
Asim Aslam 9f4770e7fd fix generation (#2312)
Run tests / Test repo (push) Waiting to run
2021-10-15 14:03:40 +01:00
1375 changed files with 33136 additions and 162387 deletions
-2
View File
@@ -1,3 +1 @@
# These are supported funding model platforms
github: asim
+34 -14
View File
@@ -1,24 +1,44 @@
---
name: Bug report
about: For reporting bugs in go-micro
title: "[BUG]"
labels: ''
about: Create a report to help us improve
title: '[BUG] '
labels: bug
assignees: ''
---
**Describe the bug**
## Describe the bug
A clear and concise description of what the bug is.
1. What are you trying to do?
2. What did you expect to happen?
3. What happens instead?
## To Reproduce
Steps to reproduce the behavior:
1. Create service with '...'
2. Configure plugin '...'
3. Run command '...'
4. See error
**How to reproduce the bug:**
## Expected behavior
A clear and concise description of what you expected to happen.
If possible, please include a minimal code snippet here.
**Environment:**
Go Version: please paste `go version` output here
## Code sample
```go
// Minimal reproducible code
```
please paste `go env` output here
## Environment
- Go Micro version: [e.g. v5.3.0]
- Go version: [e.g. 1.21.0]
- OS: [e.g. Ubuntu 22.04]
- Plugins used: [e.g. consul registry, nats broker]
## Logs
```
Paste relevant logs here
```
## Additional context
Add any other context about the problem here.
## Resources
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Examples](https://github.com/micro/go-micro/tree/master/internal/website/docs/examples)
- [API Reference](https://pkg.go.dev/go-micro.dev/v5)
@@ -1,17 +0,0 @@
---
name: Feature request / Enhancement
about: If you have a need not served by go-micro
title: "[FEATURE]"
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Additional context**
Add any other context or screenshots about the feature request here.
+30
View File
@@ -0,0 +1,30 @@
---
name: Feature request
about: Suggest an idea for this project
title: '[FEATURE] '
labels: enhancement
assignees: ''
---
## Is your feature request related to a problem?
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
## Describe the solution you'd like
A clear and concise description of what you want to happen.
## Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
## Use case
Describe how this feature would be used in practice. What problem does it solve?
## Additional context
Add any other context, code examples, or screenshots about the feature request here.
## Willing to contribute?
- [ ] I'd be willing to submit a PR for this feature
## Resources
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Plugins](https://github.com/micro/go-micro/tree/master/internal/website/docs/plugins.md)
- [Roadmap](https://github.com/micro/go-micro/blob/master/ROADMAP.md)
+25 -8
View File
@@ -1,14 +1,31 @@
---
name: Question
about: Ask a question about go-micro
title: ''
labels: ''
about: Ask a question about using Go Micro
title: '[QUESTION] '
labels: question
assignees: ''
---
Before asking, please check if your question has already been answered:
## Your question
A clear and concise question about Go Micro usage.
1. Check the documentation - https://micro.mu/docs/
2. Check the examples and plugins - https://github.com/micro/examples & https://github.com/micro/go-plugins
3. Search existing issues
## What have you tried?
Describe what you've already attempted or researched.
## Code sample (if applicable)
```go
// Your code here
```
## Context
Provide any additional context that might help answer your question.
## Resources you've checked
- [ ] [Getting Started Guide](https://github.com/micro/go-micro/tree/master/internal/website/docs/getting-started.md)
- [ ] [Examples](https://github.com/micro/go-micro/tree/master/internal/website/docs/examples)
- [ ] [API Documentation](https://pkg.go.dev/go-micro.dev/v5)
- [ ] Searched existing issues
## Helpful links
- [Documentation](https://github.com/micro/go-micro/tree/master/internal/website/docs)
- [Plugins Guide](https://github.com/micro/go-micro/tree/master/internal/website/docs/plugins.md)
-10
View File
@@ -1,10 +0,0 @@
## Pull Request template
Please, go through these steps before clicking submit on this PR.
1. Make sure this PR targets the `develop` branch. We follow the git-flow branching model.
2. Give a descriptive title to your PR.
3. Provide a description of your changes.
4. Make sure you have some relevant tests.
5. Put `closes #XXXX` in your comment to auto-close the issue that your PR fixes (if applicable).
**PLEASE REMOVE THIS TEMPLATE BEFORE SUBMITTING**
-15
View File
@@ -1,15 +0,0 @@
#!/bin/bash -e
find . -type f -name '*.pb.*.go' -o -name '*.pb.go' -a ! -name 'message.pb.go' -delete
PROTOS=$(find . -type f -name '*.proto' | grep -v proto/google/api)
mkdir -p proto/google/api
curl -s -o proto/google/api/annotations.proto -L https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/annotations.proto
curl -s -o proto/google/api/http.proto -L https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/http.proto
for PROTO in $PROTOS; do
echo $PROTO
protoc -I./proto -I. -I$(dirname $PROTO) --go_out=plugins=grpc,paths=source_relative:. --micro_out=paths=source_relative:. $PROTO
done
rm -r proto
-28
View File
@@ -1,28 +0,0 @@
name: PR Sanity Check
on: pull_request
jobs:
prtest:
name: PR sanity check
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.15
uses: actions/setup-go@v1
with:
go-version: 1.15
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Get dependencies
run: |
go get -v -t -d ./...
- name: Run tests
id: tests
env:
IN_TRAVIS_CI: yes
run: go test -v ./...
+74
View File
@@ -0,0 +1,74 @@
name: Run Tests
on:
push:
branches:
- "**"
pull_request:
types:
- opened
- reopened
branches:
- "**"
jobs:
unittests:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.24
check-latest: true
cache: true
- name: Get dependencies
run: |
go install github.com/kyoh86/richgo@latest
go get -v -t -d ./...
- name: Run tests
id: tests
run: richgo test -v -race -cover ./...
env:
IN_TRAVIS_CI: yes
RICHGO_FORCE_COLOR: 1
etcd-integration:
name: Etcd Integration Tests
runs-on: ubuntu-latest
permissions:
contents: read
services:
etcd:
image: quay.io/coreos/etcd:v3.5.2
env:
ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379
ETCD_ADVERTISE_CLIENT_URLS: http://0.0.0.0:2379
ports:
- 2379:2379
options: >-
--health-cmd "etcdctl endpoint health"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.24
check-latest: true
cache: true
- name: Get dependencies
run: |
go install github.com/kyoh86/richgo@latest
go get -v -t -d ./...
- name: Wait for etcd
run: |
timeout 30 bash -c 'until curl -s http://localhost:2379/health; do sleep 1; done'
- name: Run etcd integration tests
run: richgo test -v -race ./registry/etcd/...
env:
ETCD_ADDRESS: localhost:2379
IN_TRAVIS_CI: yes
RICHGO_FORCE_COLOR: 1
-29
View File
@@ -1,29 +0,0 @@
name: Run tests
on: [push]
jobs:
test:
name: Test repo
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.15
uses: actions/setup-go@v1
with:
go-version: 1.15
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Get dependencies
run: |
go get -v -t -d ./...
- name: Run tests
id: tests
env:
IN_TRAVIS_CI: yes
run: go test -v ./...
+51
View File
@@ -0,0 +1,51 @@
# Sample workflow for building and deploying a Jekyll site to GitHub Pages
name: Deploy Jekyll with GitHub Pages dependencies preinstalled
on:
# Runs on pushes targeting the default branch
push:
branches: ["master"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Build with Jekyll
uses: actions/jekyll-build-pages@v1
with:
source: ./internal/website
destination: ./_site
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
+5
View File
@@ -1,6 +1,7 @@
# Develop tools
/.vscode/
/.idea/
/.trunk
# Binaries for programs and plugins
*.exe
@@ -34,3 +35,7 @@ _cgo_export.*
*~
*.swp
*.swo
# go work files
go.work
go.work.sum
+251
View File
@@ -0,0 +1,251 @@
# This file contains all available configuration options
# with their default values.
# options for analysis running
run:
# go: '1.18'
# default concurrency is a available CPU number
# concurrency: 4
# timeout for analysis, e.g. 30s, 5m, default is 1m
deadline: 10m
# exit code when at least one issue was found, default is 1
issues-exit-code: 1
# include test files or not, default is true
tests: true
# which files to skip: they will be analyzed, but issues from them
# won't be reported. Default value is empty list, but there is
# no need to include all autogenerated files, we confidently recognize
# autogenerated files. If it's not please let us know.
skip-files:
[]
# - .*\\.pb\\.go$
allow-parallel-runners: true
# list of build tags, all linters use it. Default is empty list.
build-tags: []
# output configuration options
output:
# Format: colored-line-number|line-number|json|tab|checkstyle|code-climate|junit-xml|github-actions
#
# Multiple can be specified by separating them by comma, output can be provided
# for each of them by separating format name and path by colon symbol.
# Output path can be either `stdout`, `stderr` or path to the file to write to.
# Example: "checkstyle:report.json,colored-line-number"
#
# Default: colored-line-number
format: colored-line-number
# Print lines of code with issue.
# Default: true
print-issued-lines: true
# Print linter name in the end of issue text.
# Default: true
print-linter-name: true
# Make issues output unique by line.
# Default: true
uniq-by-line: true
# Add a prefix to the output file references.
# Default is no prefix.
path-prefix: ""
# Sort results by: filepath, line and column.
sort-results: true
# all available settings of specific linters
linters-settings:
wsl:
allow-cuddle-with-calls: ["Lock", "RLock", "defer"]
funlen:
lines: 80
statements: 60
varnamelen:
# The longest distance, in source lines, that is being considered a "small scope".
# Variables used in at most this many lines will be ignored.
# Default: 5
max-distance: 26
ignore-names:
- err
- id
- ch
- wg
- mu
ignore-decls:
- c echo.Context
- t testing.T
- f *foo.Bar
- e error
- i int
- const C
- T any
- m map[string]int
errcheck:
# report about not checking of errors in type assetions: `a := b.(MyStruct)`;
# default is false: such cases aren't reported by default.
check-type-assertions: true
# report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;
# default is false: such cases aren't reported by default.
check-blank: true
govet:
# report about shadowed variables
check-shadowing: false
gofmt:
# simplify code: gofmt with `-s` option, true by default
simplify: true
gocyclo:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 15
maligned:
# print struct with more effective memory layout or not, false by default
suggest-new: true
dupl:
# tokens count to trigger issue, 150 by default
threshold: 100
goconst:
# minimal length of string constant, 3 by default
min-len: 3
# minimal occurrences count to trigger, 3 by default
min-occurrences: 3
depguard:
list-type: blacklist
# Packages listed here will reported as error if imported
packages:
- github.com/golang/protobuf/proto
misspell:
# Correct spellings using locale preferences for US or UK.
# Default is to use a neutral variety of English.
# Setting locale to US will correct the British spelling of 'colour' to 'color'.
locale: US
lll:
# max line length, lines longer will be reported. Default is 120.
# '\t' is counted as 1 character by default, and can be changed with the tab-width option
line-length: 120
# tab width in spaces. Default to 1.
tab-width: 1
unused:
# treat code as a program (not a library) and report unused exported identifiers; default is false.
# XXX: if you enable this setting, unused will report a lot of false-positives in text editors:
# if it's called for subdir of a project it can't find funcs usages. All text editor integrations
# with golangci-lint call it on a directory with the changed file.
check-exported: false
unparam:
# call graph construction algorithm (cha, rta). In general, use cha for libraries,
# and rta for programs with main packages. Default is cha.
algo: cha
# Inspect exported functions, default is false. Set to true if no external program/library imports your code.
# XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
# if it's called for subdir of a project it can't find external interfaces. All text editor integrations
# with golangci-lint call it on a directory with the changed file.
check-exported: false
nakedret:
# make an issue if func has more lines of code than this setting and it has naked returns; default is 30
max-func-lines: 60
nolintlint:
allow-unused: false
allow-leading-space: false
allow-no-explanation: []
require-explanation: false
require-specific: true
prealloc:
# XXX: we don't recommend using this linter before doing performance profiling.
# For most programs usage of prealloc will be a premature optimization.
# Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.
# True by default.
simple: true
range-loops: true # Report preallocation suggestions on range loops, true by default
for-loops: false # Report preallocation suggestions on for loops, false by default
cyclop:
# the maximal code complexity to report
max-complexity: 20
gomoddirectives:
replace-local: true
retract-allow-no-explanation: false
exclude-forbidden: true
linters:
enable-all: true
disable-all: false
fast: false
disable:
- golint
- varcheck
- ifshort
- structcheck
- deadcode
# - nosnakecase
- interfacer
- maligned
- scopelint
- exhaustivestruct
- testpackage
- promlinter
- nonamedreturns
- makezero
- gofumpt
- nlreturn
- thelper
# Can be considered to be enabled
- gochecknoinits
- gochecknoglobals # RIP
- dogsled
- wrapcheck
- paralleltest
- ireturn
- gomnd
- goerr113
- exhaustruct
- containedctx
- godox
- forcetypeassert
- gci
- lll
issues:
# List of regexps of issue texts to exclude, empty list by default.
# But independently from this option we use default exclude patterns,
# it can be disabled by `exclude-use-default: false`. To list all
# excluded by default patterns execute `golangci-lint run --help`
# exclude:
# - package comment should be of the form "Package services ..." # revive
# - ^ST1000 # ST1000: at least one file in a package should have a package comment (stylecheck)
# exclude-rules:
# - path: internal/app/machined/pkg/system/services
# linters:
# - dupl
exclude-rules:
- path: _test\.go
linters:
- gocyclo
- dupl
- gosec
- funlen
- varnamelen
- wsl
# Independently from option `exclude` we use default exclude patterns,
# it can be disabled by this option. To list all
# excluded by default patterns execute `golangci-lint run --help`.
# Default value for this option is true.
exclude-use-default: false
# Maximum issues count per one linter. Set to 0 to disable. Default is 50.
max-issues-per-linter: 0
# Maximum count of issues with the same text. Set to 0 to disable. Default is 3.
max-same-issues: 0
# Show only new issues: if there are unstaged changes or untracked files,
# only those changes are analyzed, else only changes in HEAD~ are analyzed.
# It's a super-useful option for integration of golangci-lint into existing
# large codebase. It's not practical to fix all existing issues at the moment
# of integration: much better don't allow issues in new code.
# Default is false.
new: false
+29
View File
@@ -0,0 +1,29 @@
labelType: long
coverThreshold: 70
buildStyle:
bold: true
foreground: yellow
startStyle:
foreground: lightBlack
passStyle:
foreground: green
failStyle:
bold: true
foreground: "#821515"
skipStyle:
foreground: lightBlack
passPackageStyle:
foreground: green
hide: false
failPackageStyle:
bold: true
foreground: "#821515"
coveredStyle:
foreground: green
uncoveredStyle:
bold: true
foreground: yellow
fileStyle:
foreground: cyan
lineStyle:
foreground: magenta
+207
View File
@@ -0,0 +1,207 @@
# Contributing to Go Micro
Thank you for your interest in contributing to Go Micro! This document provides guidelines and instructions for contributing.
## Code of Conduct
Be respectful, inclusive, and collaborative. We're all here to build great software together.
## Getting Started
1. Fork the repository
2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/go-micro.git`
3. Add upstream remote: `git remote add upstream https://github.com/micro/go-micro.git`
4. Create a feature branch: `git checkout -b feature/my-feature`
## Development Setup
```bash
# Install dependencies
go mod download
# Run tests
go test ./...
# Run tests with coverage
go test -race -coverprofile=coverage.out ./...
# Run linter (install golangci-lint first)
golangci-lint run
```
## Making Changes
### Code Guidelines
- Follow standard Go conventions (use `gofmt`, `golint`)
- Write clear, descriptive commit messages
- Add tests for new functionality
- Update documentation for API changes
- Keep PRs focused - one feature/fix per PR
### Commit Messages
Use conventional commits format:
```
type(scope): subject
body
footer
```
Types:
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation changes
- `test`: Test additions/changes
- `refactor`: Code refactoring
- `perf`: Performance improvements
- `chore`: Maintenance tasks
Examples:
```
feat(registry): add kubernetes registry plugin
fix(broker): resolve nats connection leak
docs(examples): add streaming example
```
### Testing
- Write unit tests for all new code
- Ensure existing tests pass
- Add integration tests for plugin implementations
- Test with multiple Go versions (1.20+)
```bash
# Run specific package tests
go test ./registry/...
# Run with verbose output
go test -v ./...
# Run specific test
go test -run TestMyFunction ./pkg/...
```
### Documentation
- Update relevant markdown files in `internal/website/docs/`
- Add examples to `internal/website/docs/examples/` for new features
- Update README.md for major features
- Add godoc comments for exported functions/types
## Pull Request Process
1. **Update your branch**
```bash
git fetch upstream
git rebase upstream/master
```
2. **Run tests and linting**
```bash
go test ./...
golangci-lint run
```
3. **Push to your fork**
```bash
git push origin feature/my-feature
```
4. **Create Pull Request**
- Use a descriptive title
- Reference any related issues
- Describe what changed and why
- Add screenshots for UI changes
- Mark as draft if work in progress
5. **PR Review**
- Respond to feedback promptly
- Make requested changes
- Re-request review after updates
### PR Checklist
- [ ] Tests pass locally
- [ ] Code follows Go conventions
- [ ] Documentation updated
- [ ] Commit messages are clear
- [ ] Branch is up to date with master
- [ ] No merge conflicts
## Adding Plugins
New plugins should:
1. Live in the appropriate interface directory (e.g., `registry/myplugin/`)
2. Implement the interface completely
3. Include comprehensive tests
4. Provide usage examples
5. Document configuration options (env vars, options)
6. Add to plugin documentation
Example structure:
```
registry/myplugin/
├── myplugin.go # Main implementation
├── myplugin_test.go # Tests
├── options.go # Plugin-specific options
└── README.md # Usage and configuration
```
## Reporting Issues
Before creating an issue:
1. Search existing issues
2. Check documentation
3. Try the latest version
When reporting bugs:
- Use the bug report template
- Include minimal reproduction code
- Specify versions (Go, Go Micro, plugins)
- Provide relevant logs
## Documentation Contributions
Documentation improvements are always welcome!
- Fix typos and grammar
- Improve clarity
- Add missing examples
- Update outdated information
Documentation lives in `internal/website/docs/`. Preview locally with Jekyll:
```bash
cd internal/website
bundle install
bundle exec jekyll serve --livereload
```
## Community
- GitHub Issues: Bug reports and feature requests
- GitHub Discussions: Questions, ideas, and community chat
- Sponsorship: [GitHub Sponsors](https://github.com/sponsors/micro)
## Release Process
Maintainers handle releases:
1. Update CHANGELOG.md
2. Tag release: `git tag -a v5.x.x -m "Release v5.x.x"`
3. Push tag: `git push origin v5.x.x`
4. GitHub Actions creates release
## Questions?
- Check [documentation](internal/website/docs/)
- Browse [examples](internal/website/docs/examples/)
- Open a [question issue](.github/ISSUE_TEMPLATE/question.md)
Thank you for contributing to Go Micro! 🎉
+12 -2
View File
@@ -1,4 +1,3 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
@@ -176,7 +175,18 @@
END OF TERMS AND CONDITIONS
Copyright 2015 Asim Aslam.
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+185 -56
View File
@@ -1,92 +1,221 @@
# Go Micro [![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Go.Dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go-micro.dev/v4?tab=doc)
# Go Micro [![Go.Dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go-micro.dev/v5?tab=doc) [![Go Report Card](https://goreportcard.com/badge/github.com/go-micro/go-micro)](https://goreportcard.com/report/github.com/go-micro/go-micro)
Go Micro is a framework for distributed systems development.
**[📖 Documentation](https://go-micro.dev/docs/)** | [Sponsor the project](https://github.com/sponsors/micro)
## Overview
Go Micro provides the core requirements for distributed systems development including RPC and Event driven communication.
The **Micro** philosophy is sane defaults with a pluggable architecture. We provide defaults to get you started quickly
but everything can be easily swapped out.
Go Micro provides the core requirements for distributed systems development including RPC and Event driven communication.
The Go Micro philosophy is sane defaults with a pluggable architecture. We provide defaults to get you started quickly
but everything can be easily swapped out.
## Features
Go Micro abstracts away the details of distributed systems. Here are the main features.
- **Authentication** - Auth is built in as a first class citizen. Authentication and authorization enable secure
zero trust networking by providing every service an identity and certificates. This additionally includes rule
based access control.
- **Authentication** - Auth is built in as a first class citizen. Authentication and authorization enable secure
zero trust networking by providing every service an identity and certificates. This additionally includes rule
based access control.
- **Dynamic Config** - Load and hot reload dynamic config from anywhere. The config interface provides a way to load application
level config from any source such as env vars, file, etcd. You can merge the sources and even define fallbacks.
- **Dynamic Config** - Load and hot reload dynamic config from anywhere. The config interface provides a way to load application
level config from any source such as env vars, file, etcd. You can merge the sources and even define fallbacks.
- **Data Storage** - A simple data store interface to read, write and delete records. It includes support for memory, file and
CockroachDB by default. State and persistence becomes a core requirement beyond prototyping and Micro looks to build that into the framework.
- **Data Storage** - A simple data store interface to read, write and delete records. It includes support for many storage backends
in the plugins repo. State and persistence becomes a core requirement beyond prototyping and Micro looks to build that into the framework.
- **Service Discovery** - Automatic service registration and name resolution. Service discovery is at the core of micro service
development. When service A needs to speak to service B it needs the location of that service. The default discovery mechanism is
multicast DNS (mdns), a zeroconf system.
- **Service Discovery** - Automatic service registration and name resolution. Service discovery is at the core of micro service
development. When service A needs to speak to service B it needs the location of that service. The default discovery mechanism is
multicast DNS (mdns), a zeroconf system.
- **Load Balancing** - Client side load balancing built on service discovery. Once we have the addresses of any number of instances
of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution
across the services and retry a different node if there's a problem.
- **Load Balancing** - Client side load balancing built on service discovery. Once we have the addresses of any number of instances
of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution
across the services and retry a different node if there's a problem.
- **Message Encoding** - Dynamic message encoding based on content-type. The client and server will use codecs along with content-type
to seamlessly encode and decode Go types for you. Any variety of messages could be encoded and sent from different clients. The client
and server handle this by default. This includes protobuf and json by default.
- **Message Encoding** - Dynamic message encoding based on content-type. The client and server will use codecs along with content-type
to seamlessly encode and decode Go types for you. Any variety of messages could be encoded and sent from different clients. The client
and server handle this by default. This includes protobuf and json by default.
- **RPC Client/Server** - RPC based request/response with support for bidirectional streaming. We provide an abstraction for synchronous
communication. A request made to a service will be automatically resolved, load balanced, dialled and streamed.
- **RPC Client/Server** - RPC based request/response with support for bidirectional streaming. We provide an abstraction for synchronous
communication. A request made to a service will be automatically resolved, load balanced, dialled and streamed.
- **Async Messaging** - PubSub is built in as a first class citizen for asynchronous communication and event driven architectures.
Event notifications are a core pattern in micro service development. The default messaging system is a HTTP event message broker.
- **Async Messaging** - PubSub is built in as a first class citizen for asynchronous communication and event driven architectures.
Event notifications are a core pattern in micro service development. The default messaging system is a HTTP event message broker.
- **Synchronization** - Distributed systems are often built in an eventually consistent manner. Support for distributed locking and
leadership are built in as a Sync interface. When using an eventually consistent database or scheduling use the Sync interface.
- **Pluggable Interfaces** - Go Micro makes use of Go interfaces for each distributed system abstraction. Because of this these interfaces
are pluggable and allows Go Micro to be runtime agnostic. You can plugin any underlying technology.
- **Pluggable Interfaces** - Go Micro makes use of Go interfaces for each distributed system abstraction. Because of this these interfaces
are pluggable and allows Go Micro to be runtime agnostic. You can plugin any underlying technology.
## Getting Started
To make use of Go Micro
To make use of Go Micro
```golang
import "go-micro.dev/v4"
// create a new service
service := micro.NewService(
micro.Name("helloworld"),
)
// initialise flags
service.Init()
// start the service
service.Run()
```bash
go get go-micro.dev/v5@latest
```
See the [examples](https://github.com/micro/go-micro/tree/master/examples) for detailed information on usage.
Create a service and register a handler
## Command Line Interface
```go
package main
See [cmd/gomu](https://github.com/asim/go-micro/tree/master/cmd/gomu) for the command line interface.
import (
"go-micro.dev/v5"
)
## Code Generation
type Request struct {
Name string `json:"name"`
}
See [cmd/protoc-gen-micro](https://github.com/micro/go-micro/tree/master/cmd/protoc-gen-micro) for protobuf code generation.
type Response struct {
Message string `json:"message"`
}
## Example Usage
type Say struct{}
See [examples](https://github.com/micro/go-micro/tree/master/examples) directory for usage examples.
func (h *Say) Hello(ctx context.Context, req *Request, rsp *Response) error {
rsp.Message = "Hello " + req.Name
return nil
}
## Plugins
func main() {
// create the service
service := micro.New("helloworld")
See [plugins](https://github.com/micro/go-micro/tree/master/plugins) directory for all the plugins.
// register handler
service.Handle(new(Say))
## Services
// run the service
service.Run()
}
```
See [services](https://github.com/micro/go-micro/tree/master/services) directory for third party services.
Set a fixed address
## License
```go
service := micro.NewService(
micro.Name("helloworld"),
micro.Address(":8080"),
)
```
Go Micro is Apache 2.0 licensed.
Call it via curl
```bash
curl -XPOST \
-H 'Content-Type: application/json' \
-H 'Micro-Endpoint: Say.Hello' \
-d '{"name": "alice"}' \
http://localhost:8080
```
## Experimental
There's a new `genai` package for generative AI capabilities.
## Protobuf
Install the code generator and see usage in the docs:
```bash
go install go-micro.dev/v5/cmd/protoc-gen-micro@latest
```
Docs: [`internal/website/docs/getting-started.md`](internal/website/docs/getting-started.md)
## Command Line
Install the CLI:
```
go install go-micro.dev/v5/cmd/micro@latest
```
### Quick Start
```bash
micro new helloworld # Create a new service
cd helloworld
micro run # Run with API gateway
```
Then open http://localhost:8080 to see your service and call it from the browser.
### micro run
`micro run` starts your services with:
- **API Gateway** - HTTP to RPC proxy at `/api/{service}/{method}`
- **Web Dashboard** - Browse and call services at `/`
- **Health Checks** - Aggregated health at `/health`
- **Hot Reload** - Auto-rebuild on file changes
```bash
micro run # Gateway on :8080
micro run --address :3000 # Custom gateway port
micro run --no-gateway # Services only
micro run --env production # Use production environment
```
### Configuration
For multi-service projects, create a `micro.mu` file:
```
service users
path ./users
port 8081
service posts
path ./posts
port 8082
depends users
env development
DATABASE_URL sqlite://./dev.db
```
The gateway runs on :8080 by default, so services should use other ports.
### Deployment
Deploy to any Linux server with systemd:
```bash
# On your server (one-time setup)
curl -fsSL https://go-micro.dev/install.sh | sh
sudo micro init --server
# From your laptop
micro deploy user@your-server
```
The deploy command:
1. Builds binaries for Linux
2. Copies via SSH to the server
3. Sets up systemd services
4. Verifies services are healthy
Manage deployed services:
```bash
micro status --remote user@server # Check status
micro logs --remote user@server # View logs
micro logs myservice --remote user@server -f # Follow specific service
```
No Docker required. No Kubernetes. Just systemd.
See [docs/deployment.md](docs/deployment.md) for full deployment guide.
See [cmd/micro/README.md](cmd/micro/README.md) for full CLI documentation.
Docs: [`internal/website/docs`](internal/website/docs)
Package reference: https://pkg.go.dev/go-micro.dev/v5
Selected topics:
- Getting Started: [`internal/website/docs/getting-started.md`](internal/website/docs/getting-started.md)
- Plugins overview: [`internal/website/docs/plugins.md`](internal/website/docs/plugins.md)
- Learn by Example: [`internal/website/docs/examples/index.md`](internal/website/docs/examples/index.md)
## Adopters
- [Sourse](https://sourse.eu) - Work in the field of earth observation, including embedded Kubernetes running onboard aircraft, and weve built a mission management SaaS platform using Go Micro.
+163
View File
@@ -0,0 +1,163 @@
# Go Micro Roadmap
This roadmap outlines the planned features and improvements for Go Micro. Community feedback and contributions are welcome!
## Current Focus (Q1 2026)
### Documentation & Developer Experience
- [x] Modernize documentation structure
- [x] Add learn-by-example guides
- [x] Update issue templates
- [ ] Create video tutorials
- [ ] Interactive documentation site
- [ ] Plugin discovery dashboard
### Observability
- [ ] OpenTelemetry native support
- [ ] Auto-instrumentation for handlers
- [ ] Metrics export standardization
- [ ] Distributed tracing examples
- [ ] Integration with popular observability platforms
### Developer Tools
- [ ] `micro dev` with hot reload
- [ ] Service templates (`micro new --template`)
- [ ] Better error messages with suggestions
- [ ] Debug tooling improvements
- [ ] VS Code extension for Go Micro
## Q2 2026
### Production Readiness
- [ ] Health check standardization
- [ ] Graceful shutdown improvements
- [ ] Resource cleanup best practices
- [ ] Load testing framework integration
- [ ] Performance benchmarking suite
### Cloud Native
- [ ] Kubernetes operator
- [ ] Helm charts for common setups
- [ ] Service mesh integration guides (Istio, Linkerd)
- [ ] Cloud provider quickstarts (AWS, GCP, Azure)
- [ ] Multi-cluster patterns
### Security
- [ ] mTLS by default option
- [ ] Secret management integration (Vault, AWS Secrets Manager)
- [ ] RBAC improvements
- [ ] Security audit and hardening
- [ ] CVE scanning and response process
## Q3 2026
### Plugin Ecosystem
- [ ] Plugin marketplace/registry
- [ ] Plugin quality standards
- [ ] Community plugin contributions
- [ ] Plugin compatibility matrix
- [ ] Auto-discovery of available plugins
### Streaming & Async
- [ ] Improved streaming support
- [ ] Server-sent events (SSE) support
- [ ] WebSocket plugin
- [ ] Event sourcing patterns
- [ ] CQRS examples
### Testing
- [ ] Mock generation tooling
- [ ] Integration test helpers
- [ ] Contract testing support
- [ ] Chaos engineering examples
- [ ] E2E testing framework
## Q4 2026
### Performance
- [ ] Connection pooling optimizations
- [ ] Zero-allocation paths
- [ ] gRPC performance improvements
- [ ] Caching strategies guide
- [ ] Performance profiling tools
### Developer Productivity
- [ ] Code generation improvements
- [ ] Better IDE support
- [ ] Debugging tools
- [ ] Migration automation tools
- [ ] Upgrade helpers
### Community
- [ ] Regular blog posts and case studies
- [ ] Community spotlight program
- [ ] Contribution rewards
- [ ] Monthly community calls
- [ ] Conference presence
## Long-term Vision
### Core Framework
- Maintain backward compatibility (Go Micro v5+)
- Progressive disclosure of complexity
- Best-in-class developer experience
- Production-grade reliability
- Comprehensive plugin ecosystem
### Ecosystem Goals
- 100+ production deployments documented
- 50+ community plugins
- Active contributor community
- Regular releases (monthly patches, quarterly features)
- Comprehensive benchmarks vs alternatives
### Differentiation
- **Batteries included, fully swappable** - Start simple, scale complex
- **Zero-config local development** - No infrastructure required to start
- **Plugin ecosystem in-repo** - No version compatibility hell
- **Progressive complexity** - Learn as you grow
- **Cloud-native first** - Built for Kubernetes and containers
## Contributing
We welcome contributions to any roadmap items! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
### High Priority Areas
1. Documentation improvements
2. Real-world examples
3. Plugin development
4. Performance optimizations
5. Testing infrastructure
### How to Contribute
- Pick an item from the roadmap
- Open an issue to discuss approach
- Submit a PR with implementation
- Help review others' contributions
## Feedback
Have suggestions for the roadmap?
- Open a [feature request](.github/ISSUE_TEMPLATE/feature_request.md)
- Start a discussion in GitHub Discussions
- Comment on existing roadmap issues
## Version Compatibility
We follow semantic versioning:
- Major versions (v5 → v6): Breaking changes
- Minor versions (v5.3 → v5.4): New features, backward compatible
- Patch versions (v5.3.0 → v5.3.1): Bug fixes, no API changes
## Support Timeline
- v5: Active development (current)
- v4: Security fixes only (until v6 release)
- v3: End of life
---
Last updated: November 2025
This roadmap is subject to change based on community needs and priorities. Star the repo to stay updated! ⭐
-1
View File
@@ -1 +0,0 @@
theme: jekyll-theme-architect
-187
View File
@@ -1,187 +0,0 @@
package api
import (
"errors"
"regexp"
"strings"
"go-micro.dev/v4/registry"
"go-micro.dev/v4/server"
)
type Api interface {
// Initialise options
Init(...Option) error
// Get the options
Options() Options
// Register a http handler
Register(*Endpoint) error
// Register a route
Deregister(*Endpoint) error
// Implemenation of api
String() string
}
type Options struct{}
type Option func(*Options) error
// Endpoint is a mapping between an RPC method and HTTP endpoint
type Endpoint struct {
// RPC Method e.g. Greeter.Hello
Name string
// Description e.g what's this endpoint for
Description string
// API Handler e.g rpc, proxy
Handler string
// HTTP Host e.g example.com
Host []string
// HTTP Methods e.g GET, POST
Method []string
// HTTP Path e.g /greeter. Expect POSIX regex
Path []string
// Body destination
// "*" or "" - top level message value
// "string" - inner message value
Body string
// Stream flag
Stream bool
}
// Service represents an API service
type Service struct {
// Name of service
Name string
// The endpoint for this service
Endpoint *Endpoint
// Versions of this service
Services []*registry.Service
}
func strip(s string) string {
return strings.TrimSpace(s)
}
func slice(s string) []string {
var sl []string
for _, p := range strings.Split(s, ",") {
if str := strip(p); len(str) > 0 {
sl = append(sl, strip(p))
}
}
return sl
}
// Encode encodes an endpoint to endpoint metadata
func Encode(e *Endpoint) map[string]string {
if e == nil {
return nil
}
// endpoint map
ep := make(map[string]string)
// set vals only if they exist
set := func(k, v string) {
if len(v) == 0 {
return
}
ep[k] = v
}
set("endpoint", e.Name)
set("description", e.Description)
set("handler", e.Handler)
set("method", strings.Join(e.Method, ","))
set("path", strings.Join(e.Path, ","))
set("host", strings.Join(e.Host, ","))
return ep
}
// Decode decodes endpoint metadata into an endpoint
func Decode(e map[string]string) *Endpoint {
if e == nil {
return nil
}
return &Endpoint{
Name: e["endpoint"],
Description: e["description"],
Method: slice(e["method"]),
Path: slice(e["path"]),
Host: slice(e["host"]),
Handler: e["handler"],
}
}
// Validate validates an endpoint to guarantee it won't blow up when being served
func Validate(e *Endpoint) error {
if e == nil {
return errors.New("endpoint is nil")
}
if len(e.Name) == 0 {
return errors.New("name required")
}
for _, p := range e.Path {
ps := p[0]
pe := p[len(p)-1]
if ps == '^' && pe == '$' {
_, err := regexp.CompilePOSIX(p)
if err != nil {
return err
}
} else if ps == '^' && pe != '$' {
return errors.New("invalid path")
} else if ps != '^' && pe == '$' {
return errors.New("invalid path")
}
}
if len(e.Handler) == 0 {
return errors.New("invalid handler")
}
return nil
}
/*
Design ideas
// Gateway is an api gateway interface
type Gateway interface {
// Register a http handler
Handle(pattern string, http.Handler)
// Register a route
RegisterRoute(r Route)
// Init initialises the command line.
// It also parses further options.
Init(...Option) error
// Run the gateway
Run() error
}
// NewGateway returns a new api gateway
func NewGateway() Gateway {
return newGateway()
}
*/
// WithEndpoint returns a server.HandlerOption with endpoint metadata set
//
// Usage:
//
// proto.RegisterHandler(service.Server(), new(Handler), api.WithEndpoint(
// &api.Endpoint{
// Name: "Greeter.Hello",
// Path: []string{"/greeter"},
// },
// ))
func WithEndpoint(e *Endpoint) server.HandlerOption {
return server.EndpointMetadata(e.Name, Encode(e))
}
-152
View File
@@ -1,152 +0,0 @@
package api
import (
"strings"
"testing"
)
func TestEncoding(t *testing.T) {
testData := []*Endpoint{
nil,
{
Name: "Foo.Bar",
Description: "A test endpoint",
Handler: "meta",
Host: []string{"foo.com"},
Method: []string{"GET"},
Path: []string{"/test"},
},
}
compare := func(expect, got []string) bool {
// no data to compare, return true
if len(expect) == 0 && len(got) == 0 {
return true
}
// no data expected but got some return false
if len(expect) == 0 && len(got) > 0 {
return false
}
// compare expected with what we got
for _, e := range expect {
var seen bool
for _, g := range got {
if e == g {
seen = true
break
}
}
if !seen {
return false
}
}
// we're done, return true
return true
}
for _, d := range testData {
// encode
e := Encode(d)
// decode
de := Decode(e)
// nil endpoint returns nil
if d == nil {
if e != nil {
t.Fatalf("expected nil got %v", e)
}
if de != nil {
t.Fatalf("expected nil got %v", de)
}
continue
}
// check encoded map
name := e["endpoint"]
desc := e["description"]
method := strings.Split(e["method"], ",")
path := strings.Split(e["path"], ",")
host := strings.Split(e["host"], ",")
handler := e["handler"]
if name != d.Name {
t.Fatalf("expected %v got %v", d.Name, name)
}
if desc != d.Description {
t.Fatalf("expected %v got %v", d.Description, desc)
}
if handler != d.Handler {
t.Fatalf("expected %v got %v", d.Handler, handler)
}
if ok := compare(d.Method, method); !ok {
t.Fatalf("expected %v got %v", d.Method, method)
}
if ok := compare(d.Path, path); !ok {
t.Fatalf("expected %v got %v", d.Path, path)
}
if ok := compare(d.Host, host); !ok {
t.Fatalf("expected %v got %v", d.Host, host)
}
if de.Name != d.Name {
t.Fatalf("expected %v got %v", d.Name, de.Name)
}
if de.Description != d.Description {
t.Fatalf("expected %v got %v", d.Description, de.Description)
}
if de.Handler != d.Handler {
t.Fatalf("expected %v got %v", d.Handler, de.Handler)
}
if ok := compare(d.Method, de.Method); !ok {
t.Fatalf("expected %v got %v", d.Method, de.Method)
}
if ok := compare(d.Path, de.Path); !ok {
t.Fatalf("expected %v got %v", d.Path, de.Path)
}
if ok := compare(d.Host, de.Host); !ok {
t.Fatalf("expected %v got %v", d.Host, de.Host)
}
}
}
func TestValidate(t *testing.T) {
epPcre := &Endpoint{
Name: "Foo.Bar",
Description: "A test endpoint",
Handler: "meta",
Host: []string{"foo.com"},
Method: []string{"GET"},
Path: []string{"^/test/?$"},
}
if err := Validate(epPcre); err != nil {
t.Fatal(err)
}
epGpath := &Endpoint{
Name: "Foo.Bar",
Description: "A test endpoint",
Handler: "meta",
Host: []string{"foo.com"},
Method: []string{"GET"},
Path: []string{"/test/{id}"},
}
if err := Validate(epGpath); err != nil {
t.Fatal(err)
}
epPcreInvalid := &Endpoint{
Name: "Foo.Bar",
Description: "A test endpoint",
Handler: "meta",
Host: []string{"foo.com"},
Method: []string{"GET"},
Path: []string{"/test/?$"},
}
if err := Validate(epPcreInvalid); err == nil {
t.Fatalf("invalid pcre %v", epPcreInvalid.Path[0])
}
}
-123
View File
@@ -1,123 +0,0 @@
// Package api provides an http-rpc handler which provides the entire http request over rpc
package api
import (
"net/http"
goapi "go-micro.dev/v4/api"
"go-micro.dev/v4/api/handler"
api "go-micro.dev/v4/api/proto"
"go-micro.dev/v4/client"
"go-micro.dev/v4/errors"
"go-micro.dev/v4/selector"
"go-micro.dev/v4/util/ctx"
)
type apiHandler struct {
opts handler.Options
s *goapi.Service
}
const (
Handler = "api"
)
// API handler is the default handler which takes api.Request and returns api.Response
func (a *apiHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
bsize := handler.DefaultMaxRecvSize
if a.opts.MaxRecvSize > 0 {
bsize = a.opts.MaxRecvSize
}
r.Body = http.MaxBytesReader(w, r.Body, bsize)
request, err := requestToProto(r)
if err != nil {
er := errors.InternalServerError("go.micro.api", err.Error())
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(500)
w.Write([]byte(er.Error()))
return
}
var service *goapi.Service
if a.s != nil {
// we were given the service
service = a.s
} else if a.opts.Router != nil {
// try get service from router
s, err := a.opts.Router.Route(r)
if err != nil {
er := errors.InternalServerError("go.micro.api", err.Error())
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(500)
w.Write([]byte(er.Error()))
return
}
service = s
} else {
// we have no way of routing the request
er := errors.InternalServerError("go.micro.api", "no route found")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(500)
w.Write([]byte(er.Error()))
return
}
// create request and response
c := a.opts.Client
req := c.NewRequest(service.Name, service.Endpoint.Name, request)
rsp := &api.Response{}
// create the context from headers
cx := ctx.FromRequest(r)
// create strategy
so := selector.WithStrategy(strategy(service.Services))
if err := c.Call(cx, req, rsp, client.WithSelectOption(so)); err != nil {
w.Header().Set("Content-Type", "application/json")
ce := errors.Parse(err.Error())
switch ce.Code {
case 0:
w.WriteHeader(500)
default:
w.WriteHeader(int(ce.Code))
}
w.Write([]byte(ce.Error()))
return
} else if rsp.StatusCode == 0 {
rsp.StatusCode = http.StatusOK
}
for _, header := range rsp.GetHeader() {
for _, val := range header.Values {
w.Header().Add(header.Key, val)
}
}
if len(w.Header().Get("Content-Type")) == 0 {
w.Header().Set("Content-Type", "application/json")
}
w.WriteHeader(int(rsp.StatusCode))
w.Write([]byte(rsp.Body))
}
func (a *apiHandler) String() string {
return "api"
}
func NewHandler(opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &apiHandler{
opts: options,
}
}
func WithService(s *goapi.Service, opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &apiHandler{
opts: options,
s: s,
}
}
-119
View File
@@ -1,119 +0,0 @@
package api
import (
"fmt"
"mime"
"net"
"net/http"
"strings"
api "go-micro.dev/v4/api/proto"
"go-micro.dev/v4/registry"
"go-micro.dev/v4/selector"
"github.com/oxtoacart/bpool"
)
var (
// need to calculate later to specify useful defaults
bufferPool = bpool.NewSizedBufferPool(1024, 8)
)
func requestToProto(r *http.Request) (*api.Request, error) {
if err := r.ParseForm(); err != nil {
return nil, fmt.Errorf("Error parsing form: %v", err)
}
req := &api.Request{
Path: r.URL.Path,
Method: r.Method,
Header: make(map[string]*api.Pair),
Get: make(map[string]*api.Pair),
Post: make(map[string]*api.Pair),
Url: r.URL.String(),
}
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
if err != nil {
ct = "text/plain; charset=UTF-8" //default CT is text/plain
r.Header.Set("Content-Type", ct)
}
//set the body:
if r.Body != nil {
switch ct {
case "application/x-www-form-urlencoded":
// expect form vals in Post data
default:
buf := bufferPool.Get()
defer bufferPool.Put(buf)
if _, err = buf.ReadFrom(r.Body); err != nil {
return nil, err
}
req.Body = buf.String()
}
}
// Set X-Forwarded-For if it does not exist
if ip, _, err := net.SplitHostPort(r.RemoteAddr); err == nil {
if prior, ok := r.Header["X-Forwarded-For"]; ok {
ip = strings.Join(prior, ", ") + ", " + ip
}
// Set the header
req.Header["X-Forwarded-For"] = &api.Pair{
Key: "X-Forwarded-For",
Values: []string{ip},
}
}
// Host is stripped from net/http Headers so let's add it
req.Header["Host"] = &api.Pair{
Key: "Host",
Values: []string{r.Host},
}
// Get data
for key, vals := range r.URL.Query() {
header, ok := req.Get[key]
if !ok {
header = &api.Pair{
Key: key,
}
req.Get[key] = header
}
header.Values = vals
}
// Post data
for key, vals := range r.PostForm {
header, ok := req.Post[key]
if !ok {
header = &api.Pair{
Key: key,
}
req.Post[key] = header
}
header.Values = vals
}
for key, vals := range r.Header {
header, ok := req.Header[key]
if !ok {
header = &api.Pair{
Key: key,
}
req.Header[key] = header
}
header.Values = vals
}
return req, nil
}
// strategy is a hack for selection
func strategy(services []*registry.Service) selector.Strategy {
return func(_ []*registry.Service) selector.Next {
// ignore input to this function, use services above
return selector.Random(services)
}
}
-46
View File
@@ -1,46 +0,0 @@
package api
import (
"net/http"
"net/url"
"testing"
)
func TestRequestToProto(t *testing.T) {
testData := []*http.Request{
{
Method: "GET",
Header: http.Header{
"Header": []string{"test"},
},
URL: &url.URL{
Scheme: "http",
Host: "localhost",
Path: "/foo/bar",
RawQuery: "param1=value1",
},
},
}
for _, d := range testData {
p, err := requestToProto(d)
if err != nil {
t.Fatal(err)
}
if p.Path != d.URL.Path {
t.Fatalf("Expected path %s got %s", d.URL.Path, p.Path)
}
if p.Method != d.Method {
t.Fatalf("Expected method %s got %s", d.Method, p.Method)
}
for k, v := range d.Header {
if val, ok := p.Header[k]; !ok {
t.Fatalf("Expected header %s", k)
} else {
if val.Values[0] != v[0] {
t.Fatalf("Expected val %s, got %s", val.Values[0], v[0])
}
}
}
}
}
-141
View File
@@ -1,141 +0,0 @@
// Package event provides a handler which publishes an event
package event
import (
"encoding/json"
"fmt"
"net/http"
"path"
"regexp"
"strings"
"time"
"go-micro.dev/v4/api/handler"
proto "go-micro.dev/v4/api/proto"
"go-micro.dev/v4/util/ctx"
"github.com/google/uuid"
"github.com/oxtoacart/bpool"
)
var (
bufferPool = bpool.NewSizedBufferPool(1024, 8)
)
type event struct {
opts handler.Options
}
var (
Handler = "event"
versionRe = regexp.MustCompilePOSIX("^v[0-9]+$")
)
func eventName(parts []string) string {
return strings.Join(parts, ".")
}
func evRoute(ns, p string) (string, string) {
p = path.Clean(p)
p = strings.TrimPrefix(p, "/")
if len(p) == 0 {
return ns, "event"
}
parts := strings.Split(p, "/")
// no path
if len(parts) == 0 {
// topic: namespace
// action: event
return strings.Trim(ns, "."), "event"
}
// Treat /v[0-9]+ as versioning
// /v1/foo/bar => topic: v1.foo action: bar
if len(parts) >= 2 && versionRe.Match([]byte(parts[0])) {
topic := ns + "." + strings.Join(parts[:2], ".")
action := eventName(parts[1:])
return topic, action
}
// /foo => topic: ns.foo action: foo
// /foo/bar => topic: ns.foo action: bar
topic := ns + "." + strings.Join(parts[:1], ".")
action := eventName(parts[1:])
return topic, action
}
func (e *event) ServeHTTP(w http.ResponseWriter, r *http.Request) {
bsize := handler.DefaultMaxRecvSize
if e.opts.MaxRecvSize > 0 {
bsize = e.opts.MaxRecvSize
}
r.Body = http.MaxBytesReader(w, r.Body, bsize)
// request to topic:event
// create event
// publish to topic
topic, action := evRoute(e.opts.Namespace, r.URL.Path)
// create event
ev := &proto.Event{
Name: action,
// TODO: dedupe event
Id: fmt.Sprintf("%s-%s-%s", topic, action, uuid.New().String()),
Header: make(map[string]*proto.Pair),
Timestamp: time.Now().Unix(),
}
// set headers
for key, vals := range r.Header {
header, ok := ev.Header[key]
if !ok {
header = &proto.Pair{
Key: key,
}
ev.Header[key] = header
}
header.Values = vals
}
// set body
if r.Method == "GET" {
bytes, _ := json.Marshal(r.URL.Query())
ev.Data = string(bytes)
} else {
// Read body
buf := bufferPool.Get()
defer bufferPool.Put(buf)
if _, err := buf.ReadFrom(r.Body); err != nil {
http.Error(w, err.Error(), 500)
return
}
ev.Data = buf.String()
}
// get client
c := e.opts.Client
// create publication
p := c.NewMessage(topic, ev)
// publish event
if err := c.Publish(ctx.FromRequest(r), p); err != nil {
http.Error(w, err.Error(), 500)
return
}
}
func (e *event) String() string {
return "event"
}
func NewHandler(opts ...handler.Option) handler.Handler {
return &event{
opts: handler.NewOptions(opts...),
}
}
-14
View File
@@ -1,14 +0,0 @@
// Package handler provides http handlers
package handler
import (
"net/http"
)
// Handler represents a HTTP handler that manages a request
type Handler interface {
// standard http handler
http.Handler
// name of handler
String() string
}
-100
View File
@@ -1,100 +0,0 @@
// Package http is a http reverse proxy handler
package http
import (
"errors"
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"go-micro.dev/v4/api"
"go-micro.dev/v4/api/handler"
"go-micro.dev/v4/selector"
)
const (
Handler = "http"
)
type httpHandler struct {
options handler.Options
// set with different initialiser
s *api.Service
}
func (h *httpHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
service, err := h.getService(r)
if err != nil {
w.WriteHeader(500)
return
}
if len(service) == 0 {
w.WriteHeader(404)
return
}
rp, err := url.Parse(service)
if err != nil {
w.WriteHeader(500)
return
}
httputil.NewSingleHostReverseProxy(rp).ServeHTTP(w, r)
}
// getService returns the service for this request from the selector
func (h *httpHandler) getService(r *http.Request) (string, error) {
var service *api.Service
if h.s != nil {
// we were given the service
service = h.s
} else if h.options.Router != nil {
// try get service from router
s, err := h.options.Router.Route(r)
if err != nil {
return "", err
}
service = s
} else {
// we have no way of routing the request
return "", errors.New("no route found")
}
// create a random selector
next := selector.Random(service.Services)
// get the next node
s, err := next()
if err != nil {
return "", nil
}
return fmt.Sprintf("http://%s", s.Address), nil
}
func (h *httpHandler) String() string {
return "http"
}
// NewHandler returns a http proxy handler
func NewHandler(opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &httpHandler{
options: options,
}
}
// WithService creates a handler with a service
func WithService(s *api.Service, opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &httpHandler{
options: options,
s: s,
}
}
-126
View File
@@ -1,126 +0,0 @@
package http
import (
"net"
"net/http"
"net/http/httptest"
"testing"
"go-micro.dev/v4/api/handler"
"go-micro.dev/v4/api/resolver"
"go-micro.dev/v4/api/resolver/vpath"
"go-micro.dev/v4/api/router"
regRouter "go-micro.dev/v4/api/router/registry"
"go-micro.dev/v4/registry"
)
func testHttp(t *testing.T, path, service, ns string) {
r := registry.NewMemoryRegistry()
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer l.Close()
s := &registry.Service{
Name: service,
Nodes: []*registry.Node{
{
Id: service + "-1",
Address: l.Addr().String(),
},
},
}
r.Register(s)
defer r.Deregister(s)
// setup the test handler
m := http.NewServeMux()
m.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`you got served`))
})
// start http test serve
go http.Serve(l, m)
// create new request and writer
w := httptest.NewRecorder()
req, err := http.NewRequest("POST", path, nil)
if err != nil {
t.Fatal(err)
}
// initialise the handler
rt := regRouter.NewRouter(
router.WithHandler("http"),
router.WithRegistry(r),
router.WithResolver(vpath.NewResolver(
resolver.WithNamespace(resolver.StaticNamespace(ns)),
)),
)
p := NewHandler(handler.WithRouter(rt))
// execute the handler
p.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("Expected 200 response got %d %s", w.Code, w.Body.String())
}
if w.Body.String() != "you got served" {
t.Fatalf("Expected body: you got served. Got: %s", w.Body.String())
}
}
func TestHttpHandler(t *testing.T) {
testData := []struct {
path string
service string
namespace string
}{
{
"/test/foo",
"go.micro.api.test",
"go.micro.api",
},
{
"/test/foo/baz",
"go.micro.api.test",
"go.micro.api",
},
{
"/v1/foo",
"go.micro.api.v1.foo",
"go.micro.api",
},
{
"/v1/foo/bar",
"go.micro.api.v1.foo",
"go.micro.api",
},
{
"/v2/baz",
"go.micro.api.v2.baz",
"go.micro.api",
},
{
"/v2/baz/bar",
"go.micro.api.v2.baz",
"go.micro.api",
},
{
"/v2/baz/bar",
"v2.baz",
"",
},
}
for _, d := range testData {
t.Run(d.service, func(t *testing.T) {
testHttp(t, d.path, d.service, d.namespace)
})
}
}
-69
View File
@@ -1,69 +0,0 @@
package handler
import (
"go-micro.dev/v4/api/router"
"go-micro.dev/v4/client"
)
var (
DefaultMaxRecvSize int64 = 1024 * 1024 * 100 // 10Mb
)
type Options struct {
MaxRecvSize int64
Namespace string
Router router.Router
Client client.Client
}
type Option func(o *Options)
// NewOptions fills in the blanks
func NewOptions(opts ...Option) Options {
var options Options
for _, o := range opts {
o(&options)
}
if options.Client == nil {
WithClient(client.NewClient())(&options)
}
// set namespace if blank
if len(options.Namespace) == 0 {
WithNamespace("go.micro.api")(&options)
}
if options.MaxRecvSize == 0 {
options.MaxRecvSize = DefaultMaxRecvSize
}
return options
}
// WithNamespace specifies the namespace for the handler
func WithNamespace(s string) Option {
return func(o *Options) {
o.Namespace = s
}
}
// WithRouter specifies a router to be used by the handler
func WithRouter(r router.Router) Option {
return func(o *Options) {
o.Router = r
}
}
func WithClient(c client.Client) Option {
return func(o *Options) {
o.Client = c
}
}
// WithmaxRecvSize specifies max body size
func WithMaxRecvSize(size int64) Option {
return func(o *Options) {
o.MaxRecvSize = size
}
}
-522
View File
@@ -1,522 +0,0 @@
// Package rpc is a go-micro rpc handler.
package rpc
import (
"encoding/json"
"io"
"net/http"
"net/textproto"
"strconv"
"strings"
"go-micro.dev/v4/api"
"go-micro.dev/v4/api/handler"
"go-micro.dev/v4/api/internal/proto"
"go-micro.dev/v4/client"
"go-micro.dev/v4/codec"
"go-micro.dev/v4/codec/jsonrpc"
"go-micro.dev/v4/codec/protorpc"
"go-micro.dev/v4/errors"
"go-micro.dev/v4/logger"
"go-micro.dev/v4/metadata"
"go-micro.dev/v4/registry"
"go-micro.dev/v4/selector"
"go-micro.dev/v4/util/ctx"
"go-micro.dev/v4/util/qson"
jsonpatch "github.com/evanphx/json-patch/v5"
"github.com/oxtoacart/bpool"
)
const (
Handler = "rpc"
)
var (
// supported json codecs
jsonCodecs = []string{
"application/grpc+json",
"application/json",
"application/json-rpc",
}
// support proto codecs
protoCodecs = []string{
"application/grpc",
"application/grpc+proto",
"application/proto",
"application/protobuf",
"application/proto-rpc",
"application/octet-stream",
}
bufferPool = bpool.NewSizedBufferPool(1024, 8)
)
type rpcHandler struct {
opts handler.Options
s *api.Service
}
type buffer struct {
io.ReadCloser
}
func (b *buffer) Write(_ []byte) (int, error) {
return 0, nil
}
// strategy is a hack for selection
func strategy(services []*registry.Service) selector.Strategy {
return func(_ []*registry.Service) selector.Next {
// ignore input to this function, use services above
return selector.Random(services)
}
}
func (h *rpcHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
bsize := handler.DefaultMaxRecvSize
if h.opts.MaxRecvSize > 0 {
bsize = h.opts.MaxRecvSize
}
r.Body = http.MaxBytesReader(w, r.Body, bsize)
defer r.Body.Close()
var service *api.Service
if h.s != nil {
// we were given the service
service = h.s
} else if h.opts.Router != nil {
// try get service from router
s, err := h.opts.Router.Route(r)
if err != nil {
writeError(w, r, errors.InternalServerError("go.micro.api", err.Error()))
return
}
service = s
} else {
// we have no way of routing the request
writeError(w, r, errors.InternalServerError("go.micro.api", "no route found"))
return
}
ct := r.Header.Get("Content-Type")
// Strip charset from Content-Type (like `application/json; charset=UTF-8`)
if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx]
}
// micro client
c := h.opts.Client
// create context
cx := ctx.FromRequest(r)
// get context from http handler wrappers
md, ok := metadata.FromContext(r.Context())
if !ok {
md = make(metadata.Metadata)
}
// fill contex with http headers
md["Host"] = r.Host
md["Method"] = r.Method
// get canonical headers
for k := range r.Header {
// may be need to get all values for key like r.Header.Values() provide in go 1.14
md[textproto.CanonicalMIMEHeaderKey(k)] = r.Header.Get(k)
}
// merge context with overwrite
cx = metadata.MergeContext(cx, md, true)
// set merged context to request
*r = *r.Clone(cx)
// if stream we currently only support json
if isStream(r, service) {
// drop older context as it can have timeouts and create new
// md, _ := metadata.FromContext(cx)
//serveWebsocket(context.TODO(), w, r, service, c)
serveWebsocket(cx, w, r, service, c)
return
}
// create strategy
so := selector.WithStrategy(strategy(service.Services))
// walk the standard call path
// get payload
br, err := requestPayload(r)
if err != nil {
writeError(w, r, err)
return
}
var rsp []byte
switch {
// proto codecs
case hasCodec(ct, protoCodecs):
request := &proto.Message{}
// if the extracted payload isn't empty lets use it
if len(br) > 0 {
request = proto.NewMessage(br)
}
// create request/response
response := &proto.Message{}
req := c.NewRequest(
service.Name,
service.Endpoint.Name,
request,
client.WithContentType(ct),
)
// make the call
if err := c.Call(cx, req, response, client.WithSelectOption(so)); err != nil {
writeError(w, r, err)
return
}
// marshall response
rsp, err = response.Marshal()
if err != nil {
writeError(w, r, err)
return
}
default:
// if json codec is not present set to json
if !hasCodec(ct, jsonCodecs) {
ct = "application/json"
}
// default to trying json
var request json.RawMessage
// if the extracted payload isn't empty lets use it
if len(br) > 0 {
request = json.RawMessage(br)
}
// create request/response
var response json.RawMessage
req := c.NewRequest(
service.Name,
service.Endpoint.Name,
&request,
client.WithContentType(ct),
)
// make the call
if err := c.Call(cx, req, &response, client.WithSelectOption(so)); err != nil {
writeError(w, r, err)
return
}
// marshall response
rsp, err = response.MarshalJSON()
if err != nil {
writeError(w, r, err)
return
}
}
// write the response
writeResponse(w, r, rsp)
}
func (rh *rpcHandler) String() string {
return "rpc"
}
func hasCodec(ct string, codecs []string) bool {
for _, codec := range codecs {
if ct == codec {
return true
}
}
return false
}
// requestPayload takes a *http.Request.
// If the request is a GET the query string parameters are extracted and marshaled to JSON and the raw bytes are returned.
// If the request method is a POST the request body is read and returned
func requestPayload(r *http.Request) ([]byte, error) {
var err error
// we have to decode json-rpc and proto-rpc because we suck
// well actually because there's no proxy codec right now
ct := r.Header.Get("Content-Type")
switch {
case strings.Contains(ct, "application/json-rpc"):
msg := codec.Message{
Type: codec.Request,
Header: make(map[string]string),
}
c := jsonrpc.NewCodec(&buffer{r.Body})
if err = c.ReadHeader(&msg, codec.Request); err != nil {
return nil, err
}
var raw json.RawMessage
if err = c.ReadBody(&raw); err != nil {
return nil, err
}
return ([]byte)(raw), nil
case strings.Contains(ct, "application/proto-rpc"), strings.Contains(ct, "application/octet-stream"):
msg := codec.Message{
Type: codec.Request,
Header: make(map[string]string),
}
c := protorpc.NewCodec(&buffer{r.Body})
if err = c.ReadHeader(&msg, codec.Request); err != nil {
return nil, err
}
var raw proto.Message
if err = c.ReadBody(&raw); err != nil {
return nil, err
}
return raw.Marshal()
case strings.Contains(ct, "application/www-x-form-urlencoded"):
r.ParseForm()
// generate a new set of values from the form
vals := make(map[string]string)
for k, v := range r.Form {
vals[k] = strings.Join(v, ",")
}
// marshal
return json.Marshal(vals)
// TODO: application/grpc
}
// otherwise as per usual
ctx := r.Context()
// dont user meadata.FromContext as it mangles names
md, ok := metadata.FromContext(ctx)
if !ok {
md = make(map[string]string)
}
// allocate maximum
matches := make(map[string]interface{}, len(md))
bodydst := ""
// get fields from url path
for k, v := range md {
k = strings.ToLower(k)
// filter own keys
if strings.HasPrefix(k, "x-api-field-") {
matches[strings.TrimPrefix(k, "x-api-field-")] = v
delete(md, k)
} else if k == "x-api-body" {
bodydst = v
delete(md, k)
}
}
// map of all fields
req := make(map[string]interface{}, len(md))
// get fields from url values
if len(r.URL.RawQuery) > 0 {
umd := make(map[string]interface{})
err = qson.Unmarshal(&umd, r.URL.RawQuery)
if err != nil {
return nil, err
}
for k, v := range umd {
matches[k] = v
}
}
// restore context without fields
*r = *r.Clone(metadata.NewContext(ctx, md))
for k, v := range matches {
ps := strings.Split(k, ".")
if len(ps) == 1 {
req[k] = v
continue
}
em := make(map[string]interface{})
em[ps[len(ps)-1]] = v
for i := len(ps) - 2; i > 0; i-- {
nm := make(map[string]interface{})
nm[ps[i]] = em
em = nm
}
if vm, ok := req[ps[0]]; ok {
// nested map
nm := vm.(map[string]interface{})
for vk, vv := range em {
nm[vk] = vv
}
req[ps[0]] = nm
} else {
req[ps[0]] = em
}
}
pathbuf := []byte("{}")
if len(req) > 0 {
pathbuf, err = json.Marshal(req)
if err != nil {
return nil, err
}
}
urlbuf := []byte("{}")
out, err := jsonpatch.MergeMergePatches(urlbuf, pathbuf)
if err != nil {
return nil, err
}
switch r.Method {
case "GET":
// empty response
if strings.Contains(ct, "application/json") && string(out) == "{}" {
return out, nil
} else if string(out) == "{}" && !strings.Contains(ct, "application/json") {
return []byte{}, nil
}
return out, nil
case "PATCH", "POST", "PUT", "DELETE":
bodybuf := []byte("{}")
buf := bufferPool.Get()
defer bufferPool.Put(buf)
if _, err := buf.ReadFrom(r.Body); err != nil {
return nil, err
}
if b := buf.Bytes(); len(b) > 0 {
bodybuf = b
}
if bodydst == "" || bodydst == "*" {
if out, err = jsonpatch.MergeMergePatches(out, bodybuf); err == nil {
return out, nil
}
}
var jsonbody map[string]interface{}
if json.Valid(bodybuf) {
if err = json.Unmarshal(bodybuf, &jsonbody); err != nil {
return nil, err
}
}
dstmap := make(map[string]interface{})
ps := strings.Split(bodydst, ".")
if len(ps) == 1 {
if jsonbody != nil {
dstmap[ps[0]] = jsonbody
} else {
// old unexpected behaviour
dstmap[ps[0]] = bodybuf
}
} else {
em := make(map[string]interface{})
if jsonbody != nil {
em[ps[len(ps)-1]] = jsonbody
} else {
// old unexpected behaviour
em[ps[len(ps)-1]] = bodybuf
}
for i := len(ps) - 2; i > 0; i-- {
nm := make(map[string]interface{})
nm[ps[i]] = em
em = nm
}
dstmap[ps[0]] = em
}
bodyout, err := json.Marshal(dstmap)
if err != nil {
return nil, err
}
if out, err = jsonpatch.MergeMergePatches(out, bodyout); err == nil {
return out, nil
}
//fallback to previous unknown behaviour
return bodybuf, nil
}
return []byte{}, nil
}
func writeError(w http.ResponseWriter, r *http.Request, err error) {
ce := errors.Parse(err.Error())
switch ce.Code {
case 0:
// assuming it's totally screwed
ce.Code = 500
ce.Id = "go.micro.api"
ce.Status = http.StatusText(500)
ce.Detail = "error during request: " + ce.Detail
w.WriteHeader(500)
default:
w.WriteHeader(int(ce.Code))
}
// response content type
w.Header().Set("Content-Type", "application/json")
// Set trailers
if strings.Contains(r.Header.Get("Content-Type"), "application/grpc") {
w.Header().Set("Trailer", "grpc-status")
w.Header().Set("Trailer", "grpc-message")
w.Header().Set("grpc-status", "13")
w.Header().Set("grpc-message", ce.Detail)
}
_, werr := w.Write([]byte(ce.Error()))
if werr != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(werr)
}
}
}
func writeResponse(w http.ResponseWriter, r *http.Request, rsp []byte) {
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
w.Header().Set("Content-Length", strconv.Itoa(len(rsp)))
// Set trailers
if strings.Contains(r.Header.Get("Content-Type"), "application/grpc") {
w.Header().Set("Trailer", "grpc-status")
w.Header().Set("Trailer", "grpc-message")
w.Header().Set("grpc-status", "0")
w.Header().Set("grpc-message", "")
}
// write 204 status if rsp is nil
if len(rsp) == 0 {
w.WriteHeader(http.StatusNoContent)
}
// write response
_, err := w.Write(rsp)
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
}
}
func NewHandler(opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &rpcHandler{
opts: options,
}
}
func WithService(s *api.Service, opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &rpcHandler{
opts: options,
s: s,
}
}
-122
View File
@@ -1,122 +0,0 @@
package rpc
import (
"bytes"
"encoding/json"
"net/http"
"reflect"
"testing"
go_api "go-micro.dev/v4/api/proto"
"github.com/golang/protobuf/proto"
)
func TestRequestPayloadFromRequest(t *testing.T) {
// our test event so that we can validate serialising / deserializing of true protos works
protoEvent := go_api.Event{
Name: "Test",
}
protoBytes, err := proto.Marshal(&protoEvent)
if err != nil {
t.Fatal("Failed to marshal proto", err)
}
jsonBytes, err := json.Marshal(protoEvent)
if err != nil {
t.Fatal("Failed to marshal proto to JSON ", err)
}
type jsonUrl struct {
Key1 string `json:"key1"`
Key2 string `json:"key2"`
Name string `json:"name"`
}
jUrl := &jsonUrl{Key1: "val1", Key2: "val2", Name: "Test"}
t.Run("extracting a json from a POST request with url params", func(t *testing.T) {
r, err := http.NewRequest("POST", "http://localhost/my/path?key1=val1&key2=val2", bytes.NewReader(jsonBytes))
if err != nil {
t.Fatalf("Failed to created http.Request: %v", err)
}
extByte, err := requestPayload(r)
if err != nil {
t.Fatalf("Failed to extract payload from request: %v", err)
}
extJUrl := &jsonUrl{}
if err := json.Unmarshal(extByte, extJUrl); err != nil {
t.Fatalf("Failed to unmarshal payload from request: %v", err)
}
if !reflect.DeepEqual(extJUrl, jUrl) {
t.Fatalf("Expected %v and %v to match", extJUrl, jUrl)
}
})
t.Run("extracting a proto from a POST request", func(t *testing.T) {
r, err := http.NewRequest("POST", "http://localhost/my/path", bytes.NewReader(protoBytes))
if err != nil {
t.Fatalf("Failed to created http.Request: %v", err)
}
extByte, err := requestPayload(r)
if err != nil {
t.Fatalf("Failed to extract payload from request: %v", err)
}
if string(extByte) != string(protoBytes) {
t.Fatalf("Expected %v and %v to match", string(extByte), string(protoBytes))
}
})
t.Run("extracting JSON from a POST request", func(t *testing.T) {
r, err := http.NewRequest("POST", "http://localhost/my/path", bytes.NewReader(jsonBytes))
if err != nil {
t.Fatalf("Failed to created http.Request: %v", err)
}
extByte, err := requestPayload(r)
if err != nil {
t.Fatalf("Failed to extract payload from request: %v", err)
}
if string(extByte) != string(jsonBytes) {
t.Fatalf("Expected %v and %v to match", string(extByte), string(jsonBytes))
}
})
t.Run("extracting params from a GET request", func(t *testing.T) {
r, err := http.NewRequest("GET", "http://localhost/my/path", nil)
if err != nil {
t.Fatalf("Failed to created http.Request: %v", err)
}
q := r.URL.Query()
q.Add("name", "Test")
r.URL.RawQuery = q.Encode()
extByte, err := requestPayload(r)
if err != nil {
t.Fatalf("Failed to extract payload from request: %v", err)
}
if string(extByte) != string(jsonBytes) {
t.Fatalf("Expected %v and %v to match", string(extByte), string(jsonBytes))
}
})
t.Run("GET request with no params", func(t *testing.T) {
r, err := http.NewRequest("GET", "http://localhost/my/path", nil)
if err != nil {
t.Fatalf("Failed to created http.Request: %v", err)
}
extByte, err := requestPayload(r)
if err != nil {
t.Fatalf("Failed to extract payload from request: %v", err)
}
if string(extByte) != "" {
t.Fatalf("Expected %v and %v to match", string(extByte), "")
}
})
}
-259
View File
@@ -1,259 +0,0 @@
package rpc
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"strings"
"time"
"go-micro.dev/v4/api"
"go-micro.dev/v4/client"
raw "go-micro.dev/v4/codec/bytes"
"go-micro.dev/v4/logger"
"go-micro.dev/v4/selector"
"github.com/gobwas/httphead"
"github.com/gobwas/ws"
"github.com/gobwas/ws/wsutil"
)
// serveWebsocket will stream rpc back over websockets assuming json
func serveWebsocket(ctx context.Context, w http.ResponseWriter, r *http.Request, service *api.Service, c client.Client) {
var op ws.OpCode
ct := r.Header.Get("Content-Type")
// Strip charset from Content-Type (like `application/json; charset=UTF-8`)
if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx]
}
// check proto from request
switch ct {
case "application/json":
op = ws.OpText
default:
op = ws.OpBinary
}
hdr := make(http.Header)
if proto, ok := r.Header["Sec-Websocket-Protocol"]; ok {
for _, p := range proto {
switch p {
case "binary":
hdr["Sec-WebSocket-Protocol"] = []string{"binary"}
op = ws.OpBinary
}
}
}
payload, err := requestPayload(r)
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
upgrader := ws.HTTPUpgrader{Timeout: 5 * time.Second,
Protocol: func(proto string) bool {
if strings.Contains(proto, "binary") {
return true
}
// fallback to support all protocols now
return true
},
Extension: func(httphead.Option) bool {
// disable extensions for compatibility
return false
},
Header: hdr,
}
conn, rw, _, err := upgrader.Upgrade(r, w)
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
defer func() {
if err := conn.Close(); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
}()
var request interface{}
if !bytes.Equal(payload, []byte(`{}`)) {
switch ct {
case "application/json", "":
m := json.RawMessage(payload)
request = &m
default:
request = &raw.Frame{Data: payload}
}
}
// we always need to set content type for message
if ct == "" {
ct = "application/json"
}
req := c.NewRequest(
service.Name,
service.Endpoint.Name,
request,
client.WithContentType(ct),
client.StreamingRequest(),
)
so := selector.WithStrategy(strategy(service.Services))
// create a new stream
stream, err := c.Stream(ctx, req, client.WithSelectOption(so))
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
if request != nil {
if err = stream.Send(request); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
}
go writeLoop(rw, stream)
rsp := stream.Response()
// receive from stream and send to client
for {
select {
case <-ctx.Done():
return
case <-stream.Context().Done():
return
default:
// read backend response body
buf, err := rsp.Read()
if err != nil {
// wants to avoid import grpc/status.Status
if strings.Contains(err.Error(), "context canceled") {
return
}
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
// write the response
if err := wsutil.WriteServerMessage(rw, op, buf); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
if err = rw.Flush(); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
}
}
}
// writeLoop
func writeLoop(rw io.ReadWriter, stream client.Stream) {
// close stream when done
defer stream.Close()
for {
select {
case <-stream.Context().Done():
return
default:
buf, op, err := wsutil.ReadClientData(rw)
if err != nil {
if wserr, ok := err.(wsutil.ClosedError); ok {
switch wserr.Code {
case ws.StatusGoingAway:
// this happens when user leave the page
return
case ws.StatusNormalClosure, ws.StatusNoStatusRcvd:
// this happens when user close ws connection, or we don't get any status
return
}
}
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
switch op {
default:
// not relevant
continue
case ws.OpText, ws.OpBinary:
break
}
// send to backend
// default to trying json
// if the extracted payload isn't empty lets use it
request := &raw.Frame{Data: buf}
if err := stream.Send(request); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
return
}
}
}
}
func isStream(r *http.Request, srv *api.Service) bool {
// check if it's a web socket
if !isWebSocket(r) {
return false
}
// check if the endpoint supports streaming
for _, service := range srv.Services {
for _, ep := range service.Endpoints {
// skip if it doesn't match the name
if ep.Name != srv.Endpoint.Name {
continue
}
// matched if the name
if v := ep.Metadata["stream"]; v == "true" {
return true
}
}
}
return false
}
func isWebSocket(r *http.Request) bool {
contains := func(key, val string) bool {
vv := strings.Split(r.Header.Get(key), ",")
for _, v := range vv {
if val == strings.ToLower(strings.TrimSpace(v)) {
return true
}
}
return false
}
if contains("Connection", "upgrade") && contains("Upgrade", "websocket") {
return true
}
return false
}
-177
View File
@@ -1,177 +0,0 @@
// Package web contains the web handler including websocket support
package web
import (
"errors"
"fmt"
"io"
"net"
"net/http"
"net/http/httputil"
"net/url"
"strings"
"go-micro.dev/v4/api"
"go-micro.dev/v4/api/handler"
"go-micro.dev/v4/selector"
)
const (
Handler = "web"
)
type webHandler struct {
opts handler.Options
s *api.Service
}
func (wh *webHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
service, err := wh.getService(r)
if err != nil {
w.WriteHeader(500)
return
}
if len(service) == 0 {
w.WriteHeader(404)
return
}
rp, err := url.Parse(service)
if err != nil {
w.WriteHeader(500)
return
}
if isWebSocket(r) {
wh.serveWebSocket(rp.Host, w, r)
return
}
httputil.NewSingleHostReverseProxy(rp).ServeHTTP(w, r)
}
// getService returns the service for this request from the selector
func (wh *webHandler) getService(r *http.Request) (string, error) {
var service *api.Service
if wh.s != nil {
// we were given the service
service = wh.s
} else if wh.opts.Router != nil {
// try get service from router
s, err := wh.opts.Router.Route(r)
if err != nil {
return "", err
}
service = s
} else {
// we have no way of routing the request
return "", errors.New("no route found")
}
// create a random selector
next := selector.Random(service.Services)
// get the next node
s, err := next()
if err != nil {
return "", nil
}
return fmt.Sprintf("http://%s", s.Address), nil
}
// serveWebSocket used to serve a web socket proxied connection
func (wh *webHandler) serveWebSocket(host string, w http.ResponseWriter, r *http.Request) {
req := new(http.Request)
*req = *r
if len(host) == 0 {
http.Error(w, "invalid host", 500)
return
}
// set x-forward-for
if clientIP, _, err := net.SplitHostPort(r.RemoteAddr); err == nil {
if ips, ok := req.Header["X-Forwarded-For"]; ok {
clientIP = strings.Join(ips, ", ") + ", " + clientIP
}
req.Header.Set("X-Forwarded-For", clientIP)
}
// connect to the backend host
conn, err := net.Dial("tcp", host)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
// hijack the connection
hj, ok := w.(http.Hijacker)
if !ok {
http.Error(w, "failed to connect", 500)
return
}
nc, _, err := hj.Hijack()
if err != nil {
return
}
defer nc.Close()
defer conn.Close()
if err = req.Write(conn); err != nil {
return
}
errCh := make(chan error, 2)
cp := func(dst io.Writer, src io.Reader) {
_, err := io.Copy(dst, src)
errCh <- err
}
go cp(conn, nc)
go cp(nc, conn)
<-errCh
}
func isWebSocket(r *http.Request) bool {
contains := func(key, val string) bool {
vv := strings.Split(r.Header.Get(key), ",")
for _, v := range vv {
if val == strings.ToLower(strings.TrimSpace(v)) {
return true
}
}
return false
}
if contains("Connection", "upgrade") && contains("Upgrade", "websocket") {
return true
}
return false
}
func (wh *webHandler) String() string {
return "web"
}
func NewHandler(opts ...handler.Option) handler.Handler {
return &webHandler{
opts: handler.NewOptions(opts...),
}
}
func WithService(s *api.Service, opts ...handler.Option) handler.Handler {
options := handler.NewOptions(opts...)
return &webHandler{
opts: options,
s: s,
}
}
-28
View File
@@ -1,28 +0,0 @@
package proto
type Message struct {
data []byte
}
func (m *Message) ProtoMessage() {}
func (m *Message) Reset() {
*m = Message{}
}
func (m *Message) String() string {
return string(m.data)
}
func (m *Message) Marshal() ([]byte, error) {
return m.data, nil
}
func (m *Message) Unmarshal(data []byte) error {
m.data = data
return nil
}
func NewMessage(data []byte) *Message {
return &Message{data}
}
-335
View File
@@ -1,335 +0,0 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: api/proto/api.proto
package go_api
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
type Pair struct {
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"`
Values []string `protobuf:"bytes,2,rep,name=values,proto3" json:"values,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Pair) Reset() { *m = Pair{} }
func (m *Pair) String() string { return proto.CompactTextString(m) }
func (*Pair) ProtoMessage() {}
func (*Pair) Descriptor() ([]byte, []int) {
return fileDescriptor_2df576b66d12087a, []int{0}
}
func (m *Pair) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Pair.Unmarshal(m, b)
}
func (m *Pair) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Pair.Marshal(b, m, deterministic)
}
func (m *Pair) XXX_Merge(src proto.Message) {
xxx_messageInfo_Pair.Merge(m, src)
}
func (m *Pair) XXX_Size() int {
return xxx_messageInfo_Pair.Size(m)
}
func (m *Pair) XXX_DiscardUnknown() {
xxx_messageInfo_Pair.DiscardUnknown(m)
}
var xxx_messageInfo_Pair proto.InternalMessageInfo
func (m *Pair) GetKey() string {
if m != nil {
return m.Key
}
return ""
}
func (m *Pair) GetValues() []string {
if m != nil {
return m.Values
}
return nil
}
// A HTTP request as RPC
// Forward by the api handler
type Request struct {
Method string `protobuf:"bytes,1,opt,name=method,proto3" json:"method,omitempty"`
Path string `protobuf:"bytes,2,opt,name=path,proto3" json:"path,omitempty"`
Header map[string]*Pair `protobuf:"bytes,3,rep,name=header,proto3" json:"header,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
Get map[string]*Pair `protobuf:"bytes,4,rep,name=get,proto3" json:"get,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
Post map[string]*Pair `protobuf:"bytes,5,rep,name=post,proto3" json:"post,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
Body string `protobuf:"bytes,6,opt,name=body,proto3" json:"body,omitempty"`
Url string `protobuf:"bytes,7,opt,name=url,proto3" json:"url,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Request) Reset() { *m = Request{} }
func (m *Request) String() string { return proto.CompactTextString(m) }
func (*Request) ProtoMessage() {}
func (*Request) Descriptor() ([]byte, []int) {
return fileDescriptor_2df576b66d12087a, []int{1}
}
func (m *Request) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Request.Unmarshal(m, b)
}
func (m *Request) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Request.Marshal(b, m, deterministic)
}
func (m *Request) XXX_Merge(src proto.Message) {
xxx_messageInfo_Request.Merge(m, src)
}
func (m *Request) XXX_Size() int {
return xxx_messageInfo_Request.Size(m)
}
func (m *Request) XXX_DiscardUnknown() {
xxx_messageInfo_Request.DiscardUnknown(m)
}
var xxx_messageInfo_Request proto.InternalMessageInfo
func (m *Request) GetMethod() string {
if m != nil {
return m.Method
}
return ""
}
func (m *Request) GetPath() string {
if m != nil {
return m.Path
}
return ""
}
func (m *Request) GetHeader() map[string]*Pair {
if m != nil {
return m.Header
}
return nil
}
func (m *Request) GetGet() map[string]*Pair {
if m != nil {
return m.Get
}
return nil
}
func (m *Request) GetPost() map[string]*Pair {
if m != nil {
return m.Post
}
return nil
}
func (m *Request) GetBody() string {
if m != nil {
return m.Body
}
return ""
}
func (m *Request) GetUrl() string {
if m != nil {
return m.Url
}
return ""
}
// A HTTP response as RPC
// Expected response for the api handler
type Response struct {
StatusCode int32 `protobuf:"varint,1,opt,name=statusCode,proto3" json:"statusCode,omitempty"`
Header map[string]*Pair `protobuf:"bytes,2,rep,name=header,proto3" json:"header,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
Body string `protobuf:"bytes,3,opt,name=body,proto3" json:"body,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Response) Reset() { *m = Response{} }
func (m *Response) String() string { return proto.CompactTextString(m) }
func (*Response) ProtoMessage() {}
func (*Response) Descriptor() ([]byte, []int) {
return fileDescriptor_2df576b66d12087a, []int{2}
}
func (m *Response) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Response.Unmarshal(m, b)
}
func (m *Response) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Response.Marshal(b, m, deterministic)
}
func (m *Response) XXX_Merge(src proto.Message) {
xxx_messageInfo_Response.Merge(m, src)
}
func (m *Response) XXX_Size() int {
return xxx_messageInfo_Response.Size(m)
}
func (m *Response) XXX_DiscardUnknown() {
xxx_messageInfo_Response.DiscardUnknown(m)
}
var xxx_messageInfo_Response proto.InternalMessageInfo
func (m *Response) GetStatusCode() int32 {
if m != nil {
return m.StatusCode
}
return 0
}
func (m *Response) GetHeader() map[string]*Pair {
if m != nil {
return m.Header
}
return nil
}
func (m *Response) GetBody() string {
if m != nil {
return m.Body
}
return ""
}
// A HTTP event as RPC
// Forwarded by the event handler
type Event struct {
// e.g login
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
// uuid
Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
// unix timestamp of event
Timestamp int64 `protobuf:"varint,3,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
// event headers
Header map[string]*Pair `protobuf:"bytes,4,rep,name=header,proto3" json:"header,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
// the event data
Data string `protobuf:"bytes,5,opt,name=data,proto3" json:"data,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Event) Reset() { *m = Event{} }
func (m *Event) String() string { return proto.CompactTextString(m) }
func (*Event) ProtoMessage() {}
func (*Event) Descriptor() ([]byte, []int) {
return fileDescriptor_2df576b66d12087a, []int{3}
}
func (m *Event) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Event.Unmarshal(m, b)
}
func (m *Event) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Event.Marshal(b, m, deterministic)
}
func (m *Event) XXX_Merge(src proto.Message) {
xxx_messageInfo_Event.Merge(m, src)
}
func (m *Event) XXX_Size() int {
return xxx_messageInfo_Event.Size(m)
}
func (m *Event) XXX_DiscardUnknown() {
xxx_messageInfo_Event.DiscardUnknown(m)
}
var xxx_messageInfo_Event proto.InternalMessageInfo
func (m *Event) GetName() string {
if m != nil {
return m.Name
}
return ""
}
func (m *Event) GetId() string {
if m != nil {
return m.Id
}
return ""
}
func (m *Event) GetTimestamp() int64 {
if m != nil {
return m.Timestamp
}
return 0
}
func (m *Event) GetHeader() map[string]*Pair {
if m != nil {
return m.Header
}
return nil
}
func (m *Event) GetData() string {
if m != nil {
return m.Data
}
return ""
}
func init() {
proto.RegisterType((*Pair)(nil), "go.api.Pair")
proto.RegisterType((*Request)(nil), "go.api.Request")
proto.RegisterMapType((map[string]*Pair)(nil), "go.api.Request.GetEntry")
proto.RegisterMapType((map[string]*Pair)(nil), "go.api.Request.HeaderEntry")
proto.RegisterMapType((map[string]*Pair)(nil), "go.api.Request.PostEntry")
proto.RegisterType((*Response)(nil), "go.api.Response")
proto.RegisterMapType((map[string]*Pair)(nil), "go.api.Response.HeaderEntry")
proto.RegisterType((*Event)(nil), "go.api.Event")
proto.RegisterMapType((map[string]*Pair)(nil), "go.api.Event.HeaderEntry")
}
func init() { proto.RegisterFile("api/proto/api.proto", fileDescriptor_2df576b66d12087a) }
var fileDescriptor_2df576b66d12087a = []byte{
// 393 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xac, 0x53, 0xcd, 0xce, 0xd3, 0x30,
0x10, 0x54, 0xe2, 0x24, 0x6d, 0xb6, 0x08, 0x21, 0x23, 0x21, 0x53, 0x2a, 0x54, 0xe5, 0x54, 0x21,
0x91, 0x42, 0xcb, 0x01, 0x71, 0x85, 0xaa, 0x1c, 0x2b, 0xbf, 0x81, 0xab, 0x58, 0x6d, 0x44, 0x13,
0x9b, 0xd8, 0xa9, 0xd4, 0x87, 0xe3, 0xc0, 0x63, 0xf0, 0x36, 0xc8, 0x1b, 0xf7, 0xe7, 0xab, 0xfa,
0x5d, 0xbe, 0xaf, 0xb7, 0x89, 0x3d, 0x3b, 0x3b, 0x3b, 0xeb, 0xc0, 0x6b, 0xa1, 0xcb, 0xa9, 0x6e,
0x94, 0x55, 0x53, 0xa1, 0xcb, 0x1c, 0x11, 0x4d, 0x36, 0x2a, 0x17, 0xba, 0xcc, 0x3e, 0x41, 0xb4,
0x12, 0x65, 0x43, 0x5f, 0x01, 0xf9, 0x25, 0x0f, 0x2c, 0x18, 0x07, 0x93, 0x94, 0x3b, 0x48, 0xdf,
0x40, 0xb2, 0x17, 0xbb, 0x56, 0x1a, 0x16, 0x8e, 0xc9, 0x24, 0xe5, 0xfe, 0x2b, 0xfb, 0x4b, 0xa0,
0xc7, 0xe5, 0xef, 0x56, 0x1a, 0xeb, 0x38, 0x95, 0xb4, 0x5b, 0x55, 0xf8, 0x42, 0xff, 0x45, 0x29,
0x44, 0x5a, 0xd8, 0x2d, 0x0b, 0xf1, 0x14, 0x31, 0x9d, 0x43, 0xb2, 0x95, 0xa2, 0x90, 0x0d, 0x23,
0x63, 0x32, 0x19, 0xcc, 0xde, 0xe5, 0x9d, 0x85, 0xdc, 0x8b, 0xe5, 0x3f, 0xf1, 0x76, 0x51, 0xdb,
0xe6, 0xc0, 0x3d, 0x95, 0x7e, 0x00, 0xb2, 0x91, 0x96, 0x45, 0x58, 0xc1, 0xae, 0x2b, 0x96, 0xd2,
0x76, 0x74, 0x47, 0xa2, 0x1f, 0x21, 0xd2, 0xca, 0x58, 0x16, 0x23, 0xf9, 0xed, 0x35, 0x79, 0xa5,
0x8c, 0x67, 0x23, 0xcd, 0x79, 0x5c, 0xab, 0xe2, 0xc0, 0x92, 0xce, 0xa3, 0xc3, 0x2e, 0x85, 0xb6,
0xd9, 0xb1, 0x5e, 0x97, 0x42, 0xdb, 0xec, 0x86, 0x4b, 0x18, 0x5c, 0xf8, 0xba, 0x11, 0x53, 0x06,
0x31, 0x06, 0x83, 0xb3, 0x0e, 0x66, 0x2f, 0x8e, 0x6d, 0x5d, 0xaa, 0xbc, 0xbb, 0xfa, 0x16, 0x7e,
0x0d, 0x86, 0x3f, 0xa0, 0x7f, 0xb4, 0xfb, 0x0c, 0x95, 0x05, 0xa4, 0xa7, 0x39, 0x9e, 0x2e, 0x93,
0xfd, 0x09, 0xa0, 0xcf, 0xa5, 0xd1, 0xaa, 0x36, 0x92, 0xbe, 0x07, 0x30, 0x56, 0xd8, 0xd6, 0x7c,
0x57, 0x85, 0x44, 0xb5, 0x98, 0x5f, 0x9c, 0xd0, 0x2f, 0xa7, 0xc5, 0x85, 0x98, 0xec, 0xe8, 0x9c,
0x6c, 0xa7, 0x70, 0x73, 0x73, 0xc7, 0x78, 0xc9, 0x39, 0xde, 0xbb, 0x85, 0x99, 0xfd, 0x0b, 0x20,
0x5e, 0xec, 0x65, 0x8d, 0x5b, 0xac, 0x45, 0x25, 0xbd, 0x08, 0x62, 0xfa, 0x12, 0xc2, 0xb2, 0xf0,
0x6f, 0x2f, 0x2c, 0x0b, 0x3a, 0x82, 0xd4, 0x96, 0x95, 0x34, 0x56, 0x54, 0x1a, 0xfd, 0x10, 0x7e,
0x3e, 0xa0, 0x9f, 0x4f, 0xe3, 0x45, 0x0f, 0x1f, 0x0e, 0x36, 0x78, 0x6c, 0xb6, 0x42, 0x58, 0xc1,
0xe2, 0xae, 0xa9, 0xc3, 0x77, 0x9b, 0x6d, 0x9d, 0xe0, 0x0f, 0x3a, 0xff, 0x1f, 0x00, 0x00, 0xff,
0xff, 0xd4, 0x6d, 0x70, 0x51, 0xb7, 0x03, 0x00, 0x00,
}
-21
View File
@@ -1,21 +0,0 @@
// Code generated by protoc-gen-micro. DO NOT EDIT.
// source: api/proto/api.proto
package go_api
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
-43
View File
@@ -1,43 +0,0 @@
syntax = "proto3";
package go.api;
message Pair {
string key = 1;
repeated string values = 2;
}
// A HTTP request as RPC
// Forward by the api handler
message Request {
string method = 1;
string path = 2;
map<string, Pair> header = 3;
map<string, Pair> get = 4;
map<string, Pair> post = 5;
string body = 6; // raw request body; if not application/x-www-form-urlencoded
string url = 7;
}
// A HTTP response as RPC
// Expected response for the api handler
message Response {
int32 statusCode = 1;
map<string, Pair> header = 2;
string body = 3;
}
// A HTTP event as RPC
// Forwarded by the event handler
message Event {
// e.g login
string name = 1;
// uuid
string id = 2;
// unix timestamp of event
int64 timestamp = 3;
// event headers
map<string, Pair> header = 4;
// the event data
string data = 5;
}
-38
View File
@@ -1,38 +0,0 @@
// Package grpc resolves a grpc service like /greeter.Say/Hello to greeter service
package grpc
import (
"errors"
"net/http"
"strings"
"go-micro.dev/v4/api/resolver"
)
type Resolver struct{}
func (r *Resolver) Resolve(req *http.Request) (*resolver.Endpoint, error) {
// /foo.Bar/Service
if req.URL.Path == "/" {
return nil, errors.New("unknown name")
}
// [foo.Bar, Service]
parts := strings.Split(req.URL.Path[1:], "/")
// [foo, Bar]
name := strings.Split(parts[0], ".")
// foo
return &resolver.Endpoint{
Name: strings.Join(name[:len(name)-1], "."),
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
func (r *Resolver) String() string {
return "grpc"
}
func NewResolver(opts ...resolver.Option) resolver.Resolver {
return &Resolver{}
}
-29
View File
@@ -1,29 +0,0 @@
// Package host resolves using http host
package host
import (
"net/http"
"go-micro.dev/v4/api/resolver"
)
type Resolver struct {
opts resolver.Options
}
func (r *Resolver) Resolve(req *http.Request) (*resolver.Endpoint, error) {
return &resolver.Endpoint{
Name: req.Host,
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
func (r *Resolver) String() string {
return "host"
}
func NewResolver(opts ...resolver.Option) resolver.Resolver {
return &Resolver{opts: resolver.NewOptions(opts...)}
}
-33
View File
@@ -1,33 +0,0 @@
package resolver
import (
"net/http"
)
// NewOptions returns new initialised options
func NewOptions(opts ...Option) Options {
var options Options
for _, o := range opts {
o(&options)
}
if options.Namespace == nil {
options.Namespace = StaticNamespace("go.micro")
}
return options
}
// WithHandler sets the handler being used
func WithHandler(h string) Option {
return func(o *Options) {
o.Handler = h
}
}
// WithNamespace sets the function which determines the namespace for a request
func WithNamespace(n func(*http.Request) string) Option {
return func(o *Options) {
o.Namespace = n
}
}
-37
View File
@@ -1,37 +0,0 @@
// Package path resolves using http path
package path
import (
"net/http"
"strings"
"go-micro.dev/v4/api/resolver"
)
type Resolver struct {
opts resolver.Options
}
func (r *Resolver) Resolve(req *http.Request) (*resolver.Endpoint, error) {
if req.URL.Path == "/" {
return nil, resolver.ErrNotFound
}
parts := strings.Split(req.URL.Path[1:], "/")
ns := r.opts.Namespace(req)
return &resolver.Endpoint{
Name: ns + "." + parts[0],
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
func (r *Resolver) String() string {
return "path"
}
func NewResolver(opts ...resolver.Option) resolver.Resolver {
return &Resolver{opts: resolver.NewOptions(opts...)}
}
-44
View File
@@ -1,44 +0,0 @@
// Package resolver resolves a http request to an endpoint
package resolver
import (
"errors"
"net/http"
)
var (
ErrNotFound = errors.New("not found")
ErrInvalidPath = errors.New("invalid path")
)
// Resolver resolves requests to endpoints
type Resolver interface {
Resolve(r *http.Request) (*Endpoint, error)
String() string
}
// Endpoint is the endpoint for a http request
type Endpoint struct {
// e.g greeter
Name string
// HTTP Host e.g example.com
Host string
// HTTP Methods e.g GET, POST
Method string
// HTTP Path e.g /greeter.
Path string
}
type Options struct {
Handler string
Namespace func(*http.Request) string
}
type Option func(o *Options)
// StaticNamespace returns the same namespace for each request
func StaticNamespace(ns string) func(*http.Request) string {
return func(*http.Request) string {
return ns
}
}
-69
View File
@@ -1,69 +0,0 @@
// Package vpath resolves using http path and recognised versioned urls
package vpath
import (
"errors"
"net/http"
"regexp"
"strings"
"go-micro.dev/v4/api/resolver"
)
func NewResolver(opts ...resolver.Option) resolver.Resolver {
return &Resolver{opts: resolver.NewOptions(opts...)}
}
type Resolver struct {
opts resolver.Options
}
var (
re = regexp.MustCompile("^v[0-9]+$")
)
func (r *Resolver) Resolve(req *http.Request) (*resolver.Endpoint, error) {
if req.URL.Path == "/" {
return nil, errors.New("unknown name")
}
parts := strings.Split(req.URL.Path[1:], "/")
if len(parts) == 1 {
return &resolver.Endpoint{
Name: r.withNamespace(req, parts...),
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
// /v1/foo
if re.MatchString(parts[0]) {
return &resolver.Endpoint{
Name: r.withNamespace(req, parts[0:2]...),
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
return &resolver.Endpoint{
Name: r.withNamespace(req, parts[0]),
Host: req.Host,
Method: req.Method,
Path: req.URL.Path,
}, nil
}
func (r *Resolver) String() string {
return "path"
}
func (r *Resolver) withNamespace(req *http.Request, parts ...string) string {
ns := r.opts.Namespace(req)
if len(ns) == 0 {
return strings.Join(parts, ".")
}
return strings.Join(append([]string{ns}, parts...), ".")
}
-52
View File
@@ -1,52 +0,0 @@
package router
import (
"go-micro.dev/v4/api/resolver"
"go-micro.dev/v4/api/resolver/vpath"
"go-micro.dev/v4/registry"
)
type Options struct {
Handler string
Registry registry.Registry
Resolver resolver.Resolver
}
type Option func(o *Options)
func NewOptions(opts ...Option) Options {
options := Options{
Handler: "meta",
Registry: registry.DefaultRegistry,
}
for _, o := range opts {
o(&options)
}
if options.Resolver == nil {
options.Resolver = vpath.NewResolver(
resolver.WithHandler(options.Handler),
)
}
return options
}
func WithHandler(h string) Option {
return func(o *Options) {
o.Handler = h
}
}
func WithRegistry(r registry.Registry) Option {
return func(o *Options) {
o.Registry = r
}
}
func WithResolver(r resolver.Resolver) Option {
return func(o *Options) {
o.Resolver = r
}
}
-498
View File
@@ -1,498 +0,0 @@
// Package registry provides a dynamic api service router
package registry
import (
"errors"
"fmt"
"net/http"
"regexp"
"strings"
"sync"
"time"
"go-micro.dev/v4/api"
"go-micro.dev/v4/api/router"
"go-micro.dev/v4/api/router/util"
"go-micro.dev/v4/logger"
"go-micro.dev/v4/metadata"
"go-micro.dev/v4/registry"
"go-micro.dev/v4/registry/cache"
)
// endpoint struct, that holds compiled pcre
type endpoint struct {
hostregs []*regexp.Regexp
pathregs []util.Pattern
pcreregs []*regexp.Regexp
}
// router is the default router
type registryRouter struct {
exit chan bool
opts router.Options
// registry cache
rc cache.Cache
sync.RWMutex
eps map[string]*api.Service
// compiled regexp for host and path
ceps map[string]*endpoint
}
func (r *registryRouter) isClosed() bool {
select {
case <-r.exit:
return true
default:
return false
}
}
// refresh list of api services
func (r *registryRouter) refresh() {
var attempts int
for {
services, err := r.opts.Registry.ListServices()
if err != nil {
attempts++
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("unable to list services: %v", err)
}
time.Sleep(time.Duration(attempts) * time.Second)
continue
}
attempts = 0
// for each service, get service and store endpoints
for _, s := range services {
service, err := r.rc.GetService(s.Name)
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("unable to get service: %v", err)
}
continue
}
r.store(service)
}
// refresh list in 10 minutes... cruft
// use registry watching
select {
case <-time.After(time.Minute * 10):
case <-r.exit:
return
}
}
}
// process watch event
func (r *registryRouter) process(res *registry.Result) {
// skip these things
if res == nil || res.Service == nil {
return
}
// get entry from cache
service, err := r.rc.GetService(res.Service.Name)
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("unable to get service: %v", err)
}
return
}
// update our local endpoints
r.store(service)
}
// store local endpoint cache
func (r *registryRouter) store(services []*registry.Service) {
// endpoints
eps := map[string]*api.Service{}
// services
names := map[string]bool{}
// create a new endpoint mapping
for _, service := range services {
// set names we need later
names[service.Name] = true
// map per endpoint
for _, sep := range service.Endpoints {
// create a key service:endpoint_name
key := fmt.Sprintf("%s.%s", service.Name, sep.Name)
// decode endpoint
end := api.Decode(sep.Metadata)
// if we got nothing skip
if err := api.Validate(end); err != nil {
if logger.V(logger.TraceLevel, logger.DefaultLogger) {
logger.Tracef("endpoint validation failed: %v", err)
}
continue
}
// try get endpoint
ep, ok := eps[key]
if !ok {
ep = &api.Service{Name: service.Name}
}
// overwrite the endpoint
ep.Endpoint = end
// append services
ep.Services = append(ep.Services, service)
// store it
eps[key] = ep
}
}
r.Lock()
defer r.Unlock()
// delete any existing eps for services we know
for key, service := range r.eps {
// skip what we don't care about
if !names[service.Name] {
continue
}
// ok we know this thing
// delete delete delete
delete(r.eps, key)
}
// now set the eps we have
for name, ep := range eps {
r.eps[name] = ep
cep := &endpoint{}
for _, h := range ep.Endpoint.Host {
if h == "" || h == "*" {
continue
}
hostreg, err := regexp.CompilePOSIX(h)
if err != nil {
if logger.V(logger.TraceLevel, logger.DefaultLogger) {
logger.Tracef("endpoint have invalid host regexp: %v", err)
}
continue
}
cep.hostregs = append(cep.hostregs, hostreg)
}
for _, p := range ep.Endpoint.Path {
var pcreok bool
if p[0] == '^' && p[len(p)-1] == '$' {
pcrereg, err := regexp.CompilePOSIX(p)
if err == nil {
cep.pcreregs = append(cep.pcreregs, pcrereg)
pcreok = true
}
}
rule, err := util.Parse(p)
if err != nil && !pcreok {
if logger.V(logger.TraceLevel, logger.DefaultLogger) {
logger.Tracef("endpoint have invalid path pattern: %v", err)
}
continue
} else if err != nil && pcreok {
continue
}
tpl := rule.Compile()
pathreg, err := util.NewPattern(tpl.Version, tpl.OpCodes, tpl.Pool, "")
if err != nil {
if logger.V(logger.TraceLevel, logger.DefaultLogger) {
logger.Tracef("endpoint have invalid path pattern: %v", err)
}
continue
}
cep.pathregs = append(cep.pathregs, pathreg)
}
r.ceps[name] = cep
}
}
// watch for endpoint changes
func (r *registryRouter) watch() {
var attempts int
for {
if r.isClosed() {
return
}
// watch for changes
w, err := r.opts.Registry.Watch()
if err != nil {
attempts++
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("error watching endpoints: %v", err)
}
time.Sleep(time.Duration(attempts) * time.Second)
continue
}
ch := make(chan bool)
go func() {
select {
case <-ch:
w.Stop()
case <-r.exit:
w.Stop()
}
}()
// reset if we get here
attempts = 0
for {
// process next event
res, err := w.Next()
if err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("error getting next endoint: %v", err)
}
close(ch)
break
}
r.process(res)
}
}
}
func (r *registryRouter) Options() router.Options {
return r.opts
}
func (r *registryRouter) Close() error {
select {
case <-r.exit:
return nil
default:
close(r.exit)
r.rc.Stop()
}
return nil
}
func (r *registryRouter) Register(ep *api.Endpoint) error {
return nil
}
func (r *registryRouter) Deregister(ep *api.Endpoint) error {
return nil
}
func (r *registryRouter) Endpoint(req *http.Request) (*api.Service, error) {
if r.isClosed() {
return nil, errors.New("router closed")
}
r.RLock()
defer r.RUnlock()
var idx int
if len(req.URL.Path) > 0 && req.URL.Path != "/" {
idx = 1
}
path := strings.Split(req.URL.Path[idx:], "/")
// use the first match
// TODO: weighted matching
for n, e := range r.eps {
cep, ok := r.ceps[n]
if !ok {
continue
}
ep := e.Endpoint
var mMatch, hMatch, pMatch bool
// 1. try method
for _, m := range ep.Method {
if m == req.Method {
mMatch = true
break
}
}
if !mMatch {
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api method match %s", req.Method)
}
// 2. try host
if len(ep.Host) == 0 {
hMatch = true
} else {
for idx, h := range ep.Host {
if h == "" || h == "*" {
hMatch = true
break
} else {
if cep.hostregs[idx].MatchString(req.URL.Host) {
hMatch = true
break
}
}
}
}
if !hMatch {
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api host match %s", req.URL.Host)
}
// 3. try path via google.api path matching
for _, pathreg := range cep.pathregs {
matches, err := pathreg.Match(path, "")
if err != nil {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api gpath not match %s != %v", path, pathreg)
}
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api gpath match %s = %v", path, pathreg)
}
pMatch = true
ctx := req.Context()
md, ok := metadata.FromContext(ctx)
if !ok {
md = make(metadata.Metadata)
}
for k, v := range matches {
md[fmt.Sprintf("x-api-field-%s", k)] = v
}
md["x-api-body"] = ep.Body
*req = *req.Clone(metadata.NewContext(ctx, md))
break
}
if !pMatch {
// 4. try path via pcre path matching
for _, pathreg := range cep.pcreregs {
if !pathreg.MatchString(req.URL.Path) {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api pcre path not match %s != %v", path, pathreg)
}
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api pcre path match %s != %v", path, pathreg)
}
pMatch = true
break
}
}
if !pMatch {
continue
}
// TODO: Percentage traffic
// we got here, so its a match
return e, nil
}
// no match
return nil, errors.New("not found")
}
func (r *registryRouter) Route(req *http.Request) (*api.Service, error) {
if r.isClosed() {
return nil, errors.New("router closed")
}
// try get an endpoint
ep, err := r.Endpoint(req)
if err == nil {
return ep, nil
}
// error not nil
// ignore that shit
// TODO: don't ignore that shit
// get the service name
rp, err := r.opts.Resolver.Resolve(req)
if err != nil {
return nil, err
}
// service name
name := rp.Name
// get service
services, err := r.rc.GetService(name)
if err != nil {
return nil, err
}
// only use endpoint matching when the meta handler is set aka api.Default
switch r.opts.Handler {
// rpc handlers
case "meta", "api", "rpc":
handler := r.opts.Handler
// set default handler to api
if r.opts.Handler == "meta" {
handler = "rpc"
}
// construct api service
return &api.Service{
Name: name,
Endpoint: &api.Endpoint{
Name: rp.Method,
Handler: handler,
},
Services: services,
}, nil
// http handler
case "http", "proxy", "web":
// construct api service
return &api.Service{
Name: name,
Endpoint: &api.Endpoint{
Name: req.URL.String(),
Handler: r.opts.Handler,
Host: []string{req.Host},
Method: []string{req.Method},
Path: []string{req.URL.Path},
},
Services: services,
}, nil
}
return nil, errors.New("unknown handler")
}
func newRouter(opts ...router.Option) *registryRouter {
options := router.NewOptions(opts...)
r := &registryRouter{
exit: make(chan bool),
opts: options,
rc: cache.New(options.Registry),
eps: make(map[string]*api.Service),
ceps: make(map[string]*endpoint),
}
go r.watch()
go r.refresh()
return r
}
// NewRouter returns the default router
func NewRouter(opts ...router.Option) router.Router {
return newRouter(opts...)
}
-34
View File
@@ -1,34 +0,0 @@
package registry
import (
"testing"
"go-micro.dev/v4/registry"
"github.com/stretchr/testify/assert"
)
func TestStoreRegex(t *testing.T) {
router := newRouter()
router.store([]*registry.Service{
{
Name: "Foobar",
Version: "latest",
Endpoints: []*registry.Endpoint{
{
Name: "foo",
Metadata: map[string]string{
"endpoint": "FooEndpoint",
"description": "Some description",
"method": "POST",
"path": "^/foo/$",
"handler": "rpc",
},
},
},
Metadata: map[string]string{},
},
},
)
assert.Len(t, router.ceps["Foobar.foo"].pcreregs, 1)
}
-24
View File
@@ -1,24 +0,0 @@
// Package router provides api service routing
package router
import (
"net/http"
"go-micro.dev/v4/api"
)
// Router is used to determine an endpoint for a request
type Router interface {
// Returns options
Options() Options
// Stop the router
Close() error
// Endpoint returns an api.Service endpoint or an error if it does not exist
Endpoint(r *http.Request) (*api.Service, error)
// Register endpoint in router
Register(ep *api.Endpoint) error
// Deregister endpoint from router
Deregister(ep *api.Endpoint) error
// Route returns an api.Service route
Route(r *http.Request) (*api.Service, error)
}
-356
View File
@@ -1,356 +0,0 @@
package static
import (
"errors"
"fmt"
"net/http"
"regexp"
"strings"
"sync"
"go-micro.dev/v4/api"
"go-micro.dev/v4/api/router"
"go-micro.dev/v4/api/router/util"
"go-micro.dev/v4/logger"
"go-micro.dev/v4/metadata"
"go-micro.dev/v4/registry"
rutil "go-micro.dev/v4/util/registry"
)
type endpoint struct {
apiep *api.Endpoint
hostregs []*regexp.Regexp
pathregs []util.Pattern
pcreregs []*regexp.Regexp
}
// router is the default router
type staticRouter struct {
exit chan bool
opts router.Options
sync.RWMutex
eps map[string]*endpoint
}
func (r *staticRouter) isClosed() bool {
select {
case <-r.exit:
return true
default:
return false
}
}
/*
// watch for endpoint changes
func (r *staticRouter) watch() {
var attempts int
for {
if r.isClosed() {
return
}
// watch for changes
w, err := r.opts.Registry.Watch()
if err != nil {
attempts++
log.Println("Error watching endpoints", err)
time.Sleep(time.Duration(attempts) * time.Second)
continue
}
ch := make(chan bool)
go func() {
select {
case <-ch:
w.Stop()
case <-r.exit:
w.Stop()
}
}()
// reset if we get here
attempts = 0
for {
// process next event
res, err := w.Next()
if err != nil {
log.Println("Error getting next endpoint", err)
close(ch)
break
}
r.process(res)
}
}
}
*/
func (r *staticRouter) Register(ep *api.Endpoint) error {
if err := api.Validate(ep); err != nil {
return err
}
var pathregs []util.Pattern
var hostregs []*regexp.Regexp
var pcreregs []*regexp.Regexp
for _, h := range ep.Host {
if h == "" || h == "*" {
continue
}
hostreg, err := regexp.CompilePOSIX(h)
if err != nil {
return err
}
hostregs = append(hostregs, hostreg)
}
for _, p := range ep.Path {
var pcreok bool
// pcre only when we have start and end markers
if p[0] == '^' && p[len(p)-1] == '$' {
pcrereg, err := regexp.CompilePOSIX(p)
if err == nil {
pcreregs = append(pcreregs, pcrereg)
pcreok = true
}
}
rule, err := util.Parse(p)
if err != nil && !pcreok {
return err
} else if err != nil && pcreok {
continue
}
tpl := rule.Compile()
pathreg, err := util.NewPattern(tpl.Version, tpl.OpCodes, tpl.Pool, "")
if err != nil {
return err
}
pathregs = append(pathregs, pathreg)
}
r.Lock()
r.eps[ep.Name] = &endpoint{
apiep: ep,
pcreregs: pcreregs,
pathregs: pathregs,
hostregs: hostregs,
}
r.Unlock()
return nil
}
func (r *staticRouter) Deregister(ep *api.Endpoint) error {
if err := api.Validate(ep); err != nil {
return err
}
r.Lock()
delete(r.eps, ep.Name)
r.Unlock()
return nil
}
func (r *staticRouter) Options() router.Options {
return r.opts
}
func (r *staticRouter) Close() error {
select {
case <-r.exit:
return nil
default:
close(r.exit)
}
return nil
}
func (r *staticRouter) Endpoint(req *http.Request) (*api.Service, error) {
ep, err := r.endpoint(req)
if err != nil {
return nil, err
}
epf := strings.Split(ep.apiep.Name, ".")
services, err := r.opts.Registry.GetService(epf[0])
if err != nil {
return nil, err
}
// hack for stream endpoint
if ep.apiep.Stream {
svcs := rutil.Copy(services)
for _, svc := range svcs {
if len(svc.Endpoints) == 0 {
e := &registry.Endpoint{}
e.Name = strings.Join(epf[1:], ".")
e.Metadata = make(map[string]string)
e.Metadata["stream"] = "true"
svc.Endpoints = append(svc.Endpoints, e)
}
for _, e := range svc.Endpoints {
e.Name = strings.Join(epf[1:], ".")
e.Metadata = make(map[string]string)
e.Metadata["stream"] = "true"
}
}
services = svcs
}
svc := &api.Service{
Name: epf[0],
Endpoint: &api.Endpoint{
Name: strings.Join(epf[1:], "."),
Handler: "rpc",
Host: ep.apiep.Host,
Method: ep.apiep.Method,
Path: ep.apiep.Path,
Body: ep.apiep.Body,
Stream: ep.apiep.Stream,
},
Services: services,
}
return svc, nil
}
func (r *staticRouter) endpoint(req *http.Request) (*endpoint, error) {
if r.isClosed() {
return nil, errors.New("router closed")
}
r.RLock()
defer r.RUnlock()
var idx int
if len(req.URL.Path) > 0 && req.URL.Path != "/" {
idx = 1
}
path := strings.Split(req.URL.Path[idx:], "/")
// use the first match
// TODO: weighted matching
for _, ep := range r.eps {
var mMatch, hMatch, pMatch bool
// 1. try method
for _, m := range ep.apiep.Method {
if m == req.Method {
mMatch = true
break
}
}
if !mMatch {
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api method match %s", req.Method)
}
// 2. try host
if len(ep.apiep.Host) == 0 {
hMatch = true
} else {
for idx, h := range ep.apiep.Host {
if h == "" || h == "*" {
hMatch = true
break
} else {
if ep.hostregs[idx].MatchString(req.URL.Host) {
hMatch = true
break
}
}
}
}
if !hMatch {
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api host match %s", req.URL.Host)
}
// 3. try google.api path
for _, pathreg := range ep.pathregs {
matches, err := pathreg.Match(path, "")
if err != nil {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api gpath not match %s != %v", path, pathreg)
}
continue
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api gpath match %s = %v", path, pathreg)
}
pMatch = true
ctx := req.Context()
md, ok := metadata.FromContext(ctx)
if !ok {
md = make(metadata.Metadata)
}
for k, v := range matches {
md[fmt.Sprintf("x-api-field-%s", k)] = v
}
md["x-api-body"] = ep.apiep.Body
*req = *req.Clone(metadata.NewContext(ctx, md))
break
}
if !pMatch {
// 4. try path via pcre path matching
for _, pathreg := range ep.pcreregs {
if !pathreg.MatchString(req.URL.Path) {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("api pcre path not match %s != %v", req.URL.Path, pathreg)
}
continue
}
pMatch = true
break
}
}
if !pMatch {
continue
}
// TODO: Percentage traffic
// we got here, so its a match
return ep, nil
}
// no match
return nil, fmt.Errorf("endpoint not found for %v", req.URL)
}
func (r *staticRouter) Route(req *http.Request) (*api.Service, error) {
if r.isClosed() {
return nil, errors.New("router closed")
}
// try get an endpoint
ep, err := r.Endpoint(req)
if err != nil {
return nil, err
}
return ep, nil
}
func NewRouter(opts ...router.Option) *staticRouter {
options := router.NewOptions(opts...)
r := &staticRouter{
exit: make(chan bool),
opts: options,
eps: make(map[string]*endpoint),
}
//go r.watch()
//go r.refresh()
return r
}
-27
View File
@@ -1,27 +0,0 @@
Copyright (c) 2015, Gengo, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Gengo, Inc. nor the names of its
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-115
View File
@@ -1,115 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/compile.go
const (
opcodeVersion = 1
)
// Template is a compiled representation of path templates.
type Template struct {
// Version is the version number of the format.
Version int
// OpCodes is a sequence of operations.
OpCodes []int
// Pool is a constant pool
Pool []string
// Verb is a VERB part in the template.
Verb string
// Fields is a list of field paths bound in this template.
Fields []string
// Original template (example: /v1/a_bit_of_everything)
Template string
}
// Compiler compiles utilities representation of path templates into marshallable operations.
// They can be unmarshalled by runtime.NewPattern.
type Compiler interface {
Compile() Template
}
type op struct {
// code is the opcode of the operation
code OpCode
// str is a string operand of the code.
// operand is ignored if str is not empty.
str string
// operand is a numeric operand of the code.
operand int
}
func (w wildcard) compile() []op {
return []op{
{code: OpPush},
}
}
func (w deepWildcard) compile() []op {
return []op{
{code: OpPushM},
}
}
func (l literal) compile() []op {
return []op{
{
code: OpLitPush,
str: string(l),
},
}
}
func (v variable) compile() []op {
var ops []op
for _, s := range v.segments {
ops = append(ops, s.compile()...)
}
ops = append(ops, op{
code: OpConcatN,
operand: len(v.segments),
}, op{
code: OpCapture,
str: v.path,
})
return ops
}
func (t template) Compile() Template {
var rawOps []op
for _, s := range t.segments {
rawOps = append(rawOps, s.compile()...)
}
var (
ops []int
pool []string
fields []string
)
consts := make(map[string]int)
for _, op := range rawOps {
ops = append(ops, int(op.code))
if op.str == "" {
ops = append(ops, op.operand)
} else {
if _, ok := consts[op.str]; !ok {
consts[op.str] = len(pool)
pool = append(pool, op.str)
}
ops = append(ops, consts[op.str])
}
if op.code == OpCapture {
fields = append(fields, op.str)
}
}
return Template{
Version: opcodeVersion,
OpCodes: ops,
Pool: pool,
Verb: t.verb,
Fields: fields,
Template: t.template,
}
}
-122
View File
@@ -1,122 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/compile_test.go
import (
"reflect"
"testing"
)
const (
operandFiller = 0
)
func TestCompile(t *testing.T) {
for _, spec := range []struct {
segs []segment
verb string
ops []int
pool []string
fields []string
}{
{},
{
segs: []segment{
wildcard{},
},
ops: []int{int(OpPush), operandFiller},
},
{
segs: []segment{
deepWildcard{},
},
ops: []int{int(OpPushM), operandFiller},
},
{
segs: []segment{
literal("v1"),
},
ops: []int{int(OpLitPush), 0},
pool: []string{"v1"},
},
{
segs: []segment{
literal("v1"),
},
verb: "LOCK",
ops: []int{int(OpLitPush), 0},
pool: []string{"v1"},
},
{
segs: []segment{
variable{
path: "name.nested",
segments: []segment{
wildcard{},
},
},
},
ops: []int{
int(OpPush), operandFiller,
int(OpConcatN), 1,
int(OpCapture), 0,
},
pool: []string{"name.nested"},
fields: []string{"name.nested"},
},
{
segs: []segment{
literal("obj"),
variable{
path: "name.nested",
segments: []segment{
literal("a"),
wildcard{},
literal("b"),
},
},
variable{
path: "obj",
segments: []segment{
deepWildcard{},
},
},
},
ops: []int{
int(OpLitPush), 0,
int(OpLitPush), 1,
int(OpPush), operandFiller,
int(OpLitPush), 2,
int(OpConcatN), 3,
int(OpCapture), 3,
int(OpPushM), operandFiller,
int(OpConcatN), 1,
int(OpCapture), 0,
},
pool: []string{"obj", "a", "b", "name.nested"},
fields: []string{"name.nested", "obj"},
},
} {
tmpl := template{
segments: spec.segs,
verb: spec.verb,
}
compiled := tmpl.Compile()
if got, want := compiled.Version, opcodeVersion; got != want {
t.Errorf("tmpl.Compile().Version = %d; want %d; segs=%#v, verb=%q", got, want, spec.segs, spec.verb)
}
if got, want := compiled.OpCodes, spec.ops; !reflect.DeepEqual(got, want) {
t.Errorf("tmpl.Compile().OpCodes = %v; want %v; segs=%#v, verb=%q", got, want, spec.segs, spec.verb)
}
if got, want := compiled.Pool, spec.pool; !reflect.DeepEqual(got, want) {
t.Errorf("tmpl.Compile().Pool = %q; want %q; segs=%#v, verb=%q", got, want, spec.segs, spec.verb)
}
if got, want := compiled.Verb, spec.verb; got != want {
t.Errorf("tmpl.Compile().Verb = %q; want %q; segs=%#v, verb=%q", got, want, spec.segs, spec.verb)
}
if got, want := compiled.Fields, spec.fields; !reflect.DeepEqual(got, want) {
t.Errorf("tmpl.Compile().Fields = %q; want %q; segs=%#v, verb=%q", got, want, spec.segs, spec.verb)
}
}
}
-363
View File
@@ -1,363 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/parse.go
import (
"fmt"
"strings"
"go-micro.dev/v4/logger"
)
// InvalidTemplateError indicates that the path template is not valid.
type InvalidTemplateError struct {
tmpl string
msg string
}
func (e InvalidTemplateError) Error() string {
return fmt.Sprintf("%s: %s", e.msg, e.tmpl)
}
// Parse parses the string representation of path template
func Parse(tmpl string) (Compiler, error) {
if !strings.HasPrefix(tmpl, "/") {
return template{}, InvalidTemplateError{tmpl: tmpl, msg: "no leading /"}
}
tokens, verb := tokenize(tmpl[1:])
p := parser{tokens: tokens}
segs, err := p.topLevelSegments()
if err != nil {
return template{}, InvalidTemplateError{tmpl: tmpl, msg: err.Error()}
}
return template{
segments: segs,
verb: verb,
template: tmpl,
}, nil
}
func tokenize(path string) (tokens []string, verb string) {
if path == "" {
return []string{eof}, ""
}
const (
init = iota
field
nested
)
var (
st = init
)
for path != "" {
var idx int
switch st {
case init:
idx = strings.IndexAny(path, "/{")
case field:
idx = strings.IndexAny(path, ".=}")
case nested:
idx = strings.IndexAny(path, "/}")
}
if idx < 0 {
tokens = append(tokens, path)
break
}
switch r := path[idx]; r {
case '/', '.':
case '{':
st = field
case '=':
st = nested
case '}':
st = init
}
if idx == 0 {
tokens = append(tokens, path[idx:idx+1])
} else {
tokens = append(tokens, path[:idx], path[idx:idx+1])
}
path = path[idx+1:]
}
l := len(tokens)
t := tokens[l-1]
if idx := strings.LastIndex(t, ":"); idx == 0 {
tokens, verb = tokens[:l-1], t[1:]
} else if idx > 0 {
tokens[l-1], verb = t[:idx], t[idx+1:]
}
tokens = append(tokens, eof)
return tokens, verb
}
// parser is a parser of the template syntax defined in github.com/googleapis/googleapis/google/api/http.proto.
type parser struct {
tokens []string
accepted []string
}
// topLevelSegments is the target of this parser.
func (p *parser) topLevelSegments() ([]segment, error) {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("Parsing %q", p.tokens)
}
segs, err := p.segments()
if err != nil {
return nil, err
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("accept segments: %q; %q", p.accepted, p.tokens)
}
if _, err := p.accept(typeEOF); err != nil {
return nil, fmt.Errorf("unexpected token %q after segments %q", p.tokens[0], strings.Join(p.accepted, ""))
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("accept eof: %q; %q", p.accepted, p.tokens)
}
return segs, nil
}
func (p *parser) segments() ([]segment, error) {
s, err := p.segment()
if err != nil {
return nil, err
}
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("accept segment: %q; %q", p.accepted, p.tokens)
}
segs := []segment{s}
for {
if _, err := p.accept("/"); err != nil {
return segs, nil
}
s, err := p.segment()
if err != nil {
return segs, err
}
segs = append(segs, s)
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("accept segment: %q; %q", p.accepted, p.tokens)
}
}
}
func (p *parser) segment() (segment, error) {
if _, err := p.accept("*"); err == nil {
return wildcard{}, nil
}
if _, err := p.accept("**"); err == nil {
return deepWildcard{}, nil
}
if l, err := p.literal(); err == nil {
return l, nil
}
v, err := p.variable()
if err != nil {
return nil, fmt.Errorf("segment neither wildcards, literal or variable: %v", err)
}
return v, err
}
func (p *parser) literal() (segment, error) {
lit, err := p.accept(typeLiteral)
if err != nil {
return nil, err
}
return literal(lit), nil
}
func (p *parser) variable() (segment, error) {
if _, err := p.accept("{"); err != nil {
return nil, err
}
path, err := p.fieldPath()
if err != nil {
return nil, err
}
var segs []segment
if _, err := p.accept("="); err == nil {
segs, err = p.segments()
if err != nil {
return nil, fmt.Errorf("invalid segment in variable %q: %v", path, err)
}
} else {
segs = []segment{wildcard{}}
}
if _, err := p.accept("}"); err != nil {
return nil, fmt.Errorf("unterminated variable segment: %s", path)
}
return variable{
path: path,
segments: segs,
}, nil
}
func (p *parser) fieldPath() (string, error) {
c, err := p.accept(typeIdent)
if err != nil {
return "", err
}
components := []string{c}
for {
if _, err = p.accept("."); err != nil {
return strings.Join(components, "."), nil
}
c, err := p.accept(typeIdent)
if err != nil {
return "", fmt.Errorf("invalid field path component: %v", err)
}
components = append(components, c)
}
}
// A termType is a type of terminal symbols.
type termType string
// These constants define some of valid values of termType.
// They improve readability of parse functions.
//
// You can also use "/", "*", "**", "." or "=" as valid values.
const (
typeIdent = termType("ident")
typeLiteral = termType("literal")
typeEOF = termType("$")
)
const (
// eof is the terminal symbol which always appears at the end of token sequence.
eof = "\u0000"
)
// accept tries to accept a token in "p".
// This function consumes a token and returns it if it matches to the specified "term".
// If it doesn't match, the function does not consume any tokens and return an error.
func (p *parser) accept(term termType) (string, error) {
t := p.tokens[0]
switch term {
case "/", "*", "**", ".", "=", "{", "}":
if t != string(term) && t != "/" {
return "", fmt.Errorf("expected %q but got %q", term, t)
}
case typeEOF:
if t != eof {
return "", fmt.Errorf("expected EOF but got %q", t)
}
case typeIdent:
if err := expectIdent(t); err != nil {
return "", err
}
case typeLiteral:
if err := expectPChars(t); err != nil {
return "", err
}
default:
return "", fmt.Errorf("unknown termType %q", term)
}
p.tokens = p.tokens[1:]
p.accepted = append(p.accepted, t)
return t, nil
}
// expectPChars determines if "t" consists of only pchars defined in RFC3986.
//
// https://www.ietf.org/rfc/rfc3986.txt, P.49
// pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
// unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
// sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
// / "*" / "+" / "," / ";" / "="
// pct-encoded = "%" HEXDIG HEXDIG
func expectPChars(t string) error {
const (
init = iota
pct1
pct2
)
st := init
for _, r := range t {
if st != init {
if !isHexDigit(r) {
return fmt.Errorf("invalid hexdigit: %c(%U)", r, r)
}
switch st {
case pct1:
st = pct2
case pct2:
st = init
}
continue
}
// unreserved
switch {
case 'A' <= r && r <= 'Z':
continue
case 'a' <= r && r <= 'z':
continue
case '0' <= r && r <= '9':
continue
}
switch r {
case '-', '.', '_', '~':
// unreserved
case '!', '$', '&', '\'', '(', ')', '*', '+', ',', ';', '=':
// sub-delims
case ':', '@':
// rest of pchar
case '%':
// pct-encoded
st = pct1
default:
return fmt.Errorf("invalid character in path segment: %q(%U)", r, r)
}
}
if st != init {
return fmt.Errorf("invalid percent-encoding in %q", t)
}
return nil
}
// expectIdent determines if "ident" is a valid identifier in .proto schema ([[:alpha:]_][[:alphanum:]_]*).
func expectIdent(ident string) error {
if ident == "" {
return fmt.Errorf("empty identifier")
}
for pos, r := range ident {
switch {
case '0' <= r && r <= '9':
if pos == 0 {
return fmt.Errorf("identifier starting with digit: %s", ident)
}
continue
case 'A' <= r && r <= 'Z':
continue
case 'a' <= r && r <= 'z':
continue
case r == '_':
continue
default:
return fmt.Errorf("invalid character %q(%U) in identifier: %s", r, r, ident)
}
}
return nil
}
func isHexDigit(r rune) bool {
switch {
case '0' <= r && r <= '9':
return true
case 'A' <= r && r <= 'F':
return true
case 'a' <= r && r <= 'f':
return true
}
return false
}
-321
View File
@@ -1,321 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/parse_test.go
import (
"flag"
"fmt"
"reflect"
"testing"
"go-micro.dev/v4/logger"
)
func TestTokenize(t *testing.T) {
for _, spec := range []struct {
src string
tokens []string
}{
{
src: "",
tokens: []string{eof},
},
{
src: "v1",
tokens: []string{"v1", eof},
},
{
src: "v1/b",
tokens: []string{"v1", "/", "b", eof},
},
{
src: "v1/endpoint/*",
tokens: []string{"v1", "/", "endpoint", "/", "*", eof},
},
{
src: "v1/endpoint/**",
tokens: []string{"v1", "/", "endpoint", "/", "**", eof},
},
{
src: "v1/b/{bucket_name=*}",
tokens: []string{
"v1", "/",
"b", "/",
"{", "bucket_name", "=", "*", "}",
eof,
},
},
{
src: "v1/b/{bucket_name=buckets/*}",
tokens: []string{
"v1", "/",
"b", "/",
"{", "bucket_name", "=", "buckets", "/", "*", "}",
eof,
},
},
{
src: "v1/b/{bucket_name=buckets/*}/o",
tokens: []string{
"v1", "/",
"b", "/",
"{", "bucket_name", "=", "buckets", "/", "*", "}", "/",
"o",
eof,
},
},
{
src: "v1/b/{bucket_name=buckets/*}/o/{name}",
tokens: []string{
"v1", "/",
"b", "/",
"{", "bucket_name", "=", "buckets", "/", "*", "}", "/",
"o", "/", "{", "name", "}",
eof,
},
},
{
src: "v1/a=b&c=d;e=f:g/endpoint.rdf",
tokens: []string{
"v1", "/",
"a=b&c=d;e=f:g", "/",
"endpoint.rdf",
eof,
},
},
} {
tokens, verb := tokenize(spec.src)
if got, want := tokens, spec.tokens; !reflect.DeepEqual(got, want) {
t.Errorf("tokenize(%q) = %q, _; want %q, _", spec.src, got, want)
}
if got, want := verb, ""; got != want {
t.Errorf("tokenize(%q) = _, %q; want _, %q", spec.src, got, want)
}
src := fmt.Sprintf("%s:%s", spec.src, "LOCK")
tokens, verb = tokenize(src)
if got, want := tokens, spec.tokens; !reflect.DeepEqual(got, want) {
t.Errorf("tokenize(%q) = %q, _; want %q, _", src, got, want)
}
if got, want := verb, "LOCK"; got != want {
t.Errorf("tokenize(%q) = _, %q; want _, %q", src, got, want)
}
}
}
func TestParseSegments(t *testing.T) {
flag.Set("v", "3")
for _, spec := range []struct {
tokens []string
want []segment
}{
{
tokens: []string{"v1", eof},
want: []segment{
literal("v1"),
},
},
{
tokens: []string{"/", eof},
want: []segment{
wildcard{},
},
},
{
tokens: []string{"-._~!$&'()*+,;=:@", eof},
want: []segment{
literal("-._~!$&'()*+,;=:@"),
},
},
{
tokens: []string{"%e7%ac%ac%e4%b8%80%e7%89%88", eof},
want: []segment{
literal("%e7%ac%ac%e4%b8%80%e7%89%88"),
},
},
{
tokens: []string{"v1", "/", "*", eof},
want: []segment{
literal("v1"),
wildcard{},
},
},
{
tokens: []string{"v1", "/", "**", eof},
want: []segment{
literal("v1"),
deepWildcard{},
},
},
{
tokens: []string{"{", "name", "}", eof},
want: []segment{
variable{
path: "name",
segments: []segment{
wildcard{},
},
},
},
},
{
tokens: []string{"{", "name", "=", "*", "}", eof},
want: []segment{
variable{
path: "name",
segments: []segment{
wildcard{},
},
},
},
},
{
tokens: []string{"{", "field", ".", "nested", ".", "nested2", "=", "*", "}", eof},
want: []segment{
variable{
path: "field.nested.nested2",
segments: []segment{
wildcard{},
},
},
},
},
{
tokens: []string{"{", "name", "=", "a", "/", "b", "/", "*", "}", eof},
want: []segment{
variable{
path: "name",
segments: []segment{
literal("a"),
literal("b"),
wildcard{},
},
},
},
},
{
tokens: []string{
"v1", "/",
"{",
"name", ".", "nested", ".", "nested2",
"=",
"a", "/", "b", "/", "*",
"}", "/",
"o", "/",
"{",
"another_name",
"=",
"a", "/", "b", "/", "*", "/", "c",
"}", "/",
"**",
eof},
want: []segment{
literal("v1"),
variable{
path: "name.nested.nested2",
segments: []segment{
literal("a"),
literal("b"),
wildcard{},
},
},
literal("o"),
variable{
path: "another_name",
segments: []segment{
literal("a"),
literal("b"),
wildcard{},
literal("c"),
},
},
deepWildcard{},
},
},
} {
p := parser{tokens: spec.tokens}
segs, err := p.topLevelSegments()
if err != nil {
t.Errorf("parser{%q}.segments() failed with %v; want success", spec.tokens, err)
continue
}
if got, want := segs, spec.want; !reflect.DeepEqual(got, want) {
t.Errorf("parser{%q}.segments() = %#v; want %#v", spec.tokens, got, want)
}
if got := p.tokens; len(got) > 0 {
t.Errorf("p.tokens = %q; want []; spec.tokens=%q", got, spec.tokens)
}
}
}
func TestParseSegmentsWithErrors(t *testing.T) {
flag.Set("v", "3")
for _, spec := range []struct {
tokens []string
}{
{
// double slash
tokens: []string{"//", eof},
},
{
// invalid literal
tokens: []string{"a?b", eof},
},
{
// invalid percent-encoding
tokens: []string{"%", eof},
},
{
// invalid percent-encoding
tokens: []string{"%2", eof},
},
{
// invalid percent-encoding
tokens: []string{"a%2z", eof},
},
{
// empty segments
tokens: []string{eof},
},
{
// unterminated variable
tokens: []string{"{", "name", eof},
},
{
// unterminated variable
tokens: []string{"{", "name", "=", eof},
},
{
// unterminated variable
tokens: []string{"{", "name", "=", "*", eof},
},
{
// empty component in field path
tokens: []string{"{", "name", ".", "}", eof},
},
{
// empty component in field path
tokens: []string{"{", "name", ".", ".", "nested", "}", eof},
},
{
// invalid character in identifier
tokens: []string{"{", "field-name", "}", eof},
},
{
// no slash between segments
tokens: []string{"v1", "endpoint", eof},
},
{
// no slash between segments
tokens: []string{"v1", "{", "name", "}", eof},
},
} {
p := parser{tokens: spec.tokens}
segs, err := p.topLevelSegments()
if err == nil {
t.Errorf("parser{%q}.segments() succeeded; want InvalidTemplateError; accepted %#v", spec.tokens, segs)
continue
}
logger.Info(err)
}
}
-24
View File
@@ -1,24 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/utilities/pattern.go
// An OpCode is a opcode of compiled path patterns.
type OpCode int
// These constants are the valid values of OpCode.
const (
// OpNop does nothing
OpNop = OpCode(iota)
// OpPush pushes a component to stack
OpPush
// OpLitPush pushes a component to stack if it matches to the literal
OpLitPush
// OpPushM concatenates the remaining components and pushes it to stack
OpPushM
// OpConcatN pops N items from stack, concatenates them and pushes it back to stack
OpConcatN
// OpCapture pops an item and binds it to the variable
OpCapture
// OpEnd is the least positive invalid opcode.
OpEnd
)
-283
View File
@@ -1,283 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/runtime/pattern.go
import (
"errors"
"fmt"
"strings"
"go-micro.dev/v4/logger"
)
var (
// ErrNotMatch indicates that the given HTTP request path does not match to the pattern.
ErrNotMatch = errors.New("not match to the path pattern")
// ErrInvalidPattern indicates that the given definition of Pattern is not valid.
ErrInvalidPattern = errors.New("invalid pattern")
)
type rop struct {
code OpCode
operand int
}
// Pattern is a template pattern of http request paths defined in github.com/googleapis/googleapis/google/api/http.proto.
type Pattern struct {
// ops is a list of operations
ops []rop
// pool is a constant pool indexed by the operands or vars.
pool []string
// vars is a list of variables names to be bound by this pattern
vars []string
// stacksize is the max depth of the stack
stacksize int
// tailLen is the length of the fixed-size segments after a deep wildcard
tailLen int
// verb is the VERB part of the path pattern. It is empty if the pattern does not have VERB part.
verb string
// assumeColonVerb indicates whether a path suffix after a final
// colon may only be interpreted as a verb.
assumeColonVerb bool
}
type patternOptions struct {
assumeColonVerb bool
}
// PatternOpt is an option for creating Patterns.
type PatternOpt func(*patternOptions)
// NewPattern returns a new Pattern from the given definition values.
// "ops" is a sequence of op codes. "pool" is a constant pool.
// "verb" is the verb part of the pattern. It is empty if the pattern does not have the part.
// "version" must be 1 for now.
// It returns an error if the given definition is invalid.
func NewPattern(version int, ops []int, pool []string, verb string, opts ...PatternOpt) (Pattern, error) {
options := patternOptions{
assumeColonVerb: true,
}
for _, o := range opts {
o(&options)
}
if version != 1 {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("unsupported version: %d", version)
}
return Pattern{}, ErrInvalidPattern
}
l := len(ops)
if l%2 != 0 {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("odd number of ops codes: %d", l)
}
return Pattern{}, ErrInvalidPattern
}
var (
typedOps []rop
stack, maxstack int
tailLen int
pushMSeen bool
vars []string
)
for i := 0; i < l; i += 2 {
op := rop{code: OpCode(ops[i]), operand: ops[i+1]}
switch op.code {
case OpNop:
continue
case OpPush:
if pushMSeen {
tailLen++
}
stack++
case OpPushM:
if pushMSeen {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debug("pushM appears twice")
}
return Pattern{}, ErrInvalidPattern
}
pushMSeen = true
stack++
case OpLitPush:
if op.operand < 0 || len(pool) <= op.operand {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("negative literal index: %d", op.operand)
}
return Pattern{}, ErrInvalidPattern
}
if pushMSeen {
tailLen++
}
stack++
case OpConcatN:
if op.operand <= 0 {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("negative concat size: %d", op.operand)
}
return Pattern{}, ErrInvalidPattern
}
stack -= op.operand
if stack < 0 {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debug("stack underflow")
}
return Pattern{}, ErrInvalidPattern
}
stack++
case OpCapture:
if op.operand < 0 || len(pool) <= op.operand {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("variable name index out of bound: %d", op.operand)
}
return Pattern{}, ErrInvalidPattern
}
v := pool[op.operand]
op.operand = len(vars)
vars = append(vars, v)
stack--
if stack < 0 {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debug("stack underflow")
}
return Pattern{}, ErrInvalidPattern
}
default:
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Debugf("invalid opcode: %d", op.code)
}
return Pattern{}, ErrInvalidPattern
}
if maxstack < stack {
maxstack = stack
}
typedOps = append(typedOps, op)
}
return Pattern{
ops: typedOps,
pool: pool,
vars: vars,
stacksize: maxstack,
tailLen: tailLen,
verb: verb,
assumeColonVerb: options.assumeColonVerb,
}, nil
}
// MustPattern is a helper function which makes it easier to call NewPattern in variable initialization.
func MustPattern(p Pattern, err error) Pattern {
if err != nil {
if logger.V(logger.DebugLevel, logger.DefaultLogger) {
logger.Fatalf("Pattern initialization failed: %v", err)
}
}
return p
}
// Match examines components if it matches to the Pattern.
// If it matches, the function returns a mapping from field paths to their captured values.
// If otherwise, the function returns an error.
func (p Pattern) Match(components []string, verb string) (map[string]string, error) {
if p.verb != verb {
if p.assumeColonVerb || p.verb != "" {
return nil, ErrNotMatch
}
if len(components) == 0 {
components = []string{":" + verb}
} else {
components = append([]string{}, components...)
components[len(components)-1] += ":" + verb
}
verb = ""
}
var pos int
stack := make([]string, 0, p.stacksize)
captured := make([]string, len(p.vars))
l := len(components)
for _, op := range p.ops {
switch op.code {
case OpNop:
continue
case OpPush, OpLitPush:
if pos >= l {
return nil, ErrNotMatch
}
c := components[pos]
if op.code == OpLitPush {
if lit := p.pool[op.operand]; c != lit {
return nil, ErrNotMatch
}
}
stack = append(stack, c)
pos++
case OpPushM:
end := len(components)
if end < pos+p.tailLen {
return nil, ErrNotMatch
}
end -= p.tailLen
stack = append(stack, strings.Join(components[pos:end], "/"))
pos = end
case OpConcatN:
n := op.operand
l := len(stack) - n
stack = append(stack[:l], strings.Join(stack[l:], "/"))
case OpCapture:
n := len(stack) - 1
captured[op.operand] = stack[n]
stack = stack[:n]
}
}
if pos < l {
return nil, ErrNotMatch
}
bindings := make(map[string]string)
for i, val := range captured {
bindings[p.vars[i]] = val
}
return bindings, nil
}
// Verb returns the verb part of the Pattern.
func (p Pattern) Verb() string { return p.verb }
func (p Pattern) String() string {
var stack []string
for _, op := range p.ops {
switch op.code {
case OpNop:
continue
case OpPush:
stack = append(stack, "*")
case OpLitPush:
stack = append(stack, p.pool[op.operand])
case OpPushM:
stack = append(stack, "**")
case OpConcatN:
n := op.operand
l := len(stack) - n
stack = append(stack[:l], strings.Join(stack[l:], "/"))
case OpCapture:
n := len(stack) - 1
stack[n] = fmt.Sprintf("{%s=%s}", p.vars[op.operand], stack[n])
}
}
segs := strings.Join(stack, "/")
if p.verb != "" {
return fmt.Sprintf("/%s:%s", segs, p.verb)
}
return "/" + segs
}
// AssumeColonVerbOpt indicates whether a path suffix after a final
// colon may only be interpreted as a verb.
func AssumeColonVerbOpt(val bool) PatternOpt {
return PatternOpt(func(o *patternOptions) {
o.assumeColonVerb = val
})
}
-62
View File
@@ -1,62 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/types.go
import (
"fmt"
"strings"
)
type template struct {
segments []segment
verb string
template string
}
type segment interface {
fmt.Stringer
compile() (ops []op)
}
type wildcard struct{}
type deepWildcard struct{}
type literal string
type variable struct {
path string
segments []segment
}
func (wildcard) String() string {
return "*"
}
func (deepWildcard) String() string {
return "**"
}
func (l literal) String() string {
return string(l)
}
func (v variable) String() string {
var segs []string
for _, s := range v.segments {
segs = append(segs, s.String())
}
return fmt.Sprintf("{%s=%s}", v.path, strings.Join(segs, "/"))
}
func (t template) String() string {
var segs []string
for _, s := range t.segments {
segs = append(segs, s.String())
}
str := strings.Join(segs, "/")
if t.verb != "" {
str = fmt.Sprintf("%s:%s", str, t.verb)
}
return "/" + str
}
-93
View File
@@ -1,93 +0,0 @@
package util
// download from https://raw.githubusercontent.com/grpc-ecosystem/grpc-gateway/master/protoc-gen-grpc-gateway/httprule/types_test.go
import (
"fmt"
"testing"
)
func TestTemplateStringer(t *testing.T) {
for _, spec := range []struct {
segs []segment
want string
}{
{
segs: []segment{
literal("v1"),
},
want: "/v1",
},
{
segs: []segment{
wildcard{},
},
want: "/*",
},
{
segs: []segment{
deepWildcard{},
},
want: "/**",
},
{
segs: []segment{
variable{
path: "name",
segments: []segment{
literal("a"),
},
},
},
want: "/{name=a}",
},
{
segs: []segment{
variable{
path: "name",
segments: []segment{
literal("a"),
wildcard{},
literal("b"),
},
},
},
want: "/{name=a/*/b}",
},
{
segs: []segment{
literal("v1"),
variable{
path: "name",
segments: []segment{
literal("a"),
wildcard{},
literal("b"),
},
},
literal("c"),
variable{
path: "field.nested",
segments: []segment{
wildcard{},
literal("d"),
},
},
wildcard{},
literal("e"),
deepWildcard{},
},
want: "/v1/{name=a/*/b}/c/{field.nested=*/d}/*/e/**",
},
} {
tmpl := template{segments: spec.segs}
if got, want := tmpl.String(), spec.want; got != want {
t.Errorf("%#v.String() = %q; want %q", tmpl, got, want)
}
tmpl.verb = "LOCK"
if got, want := tmpl.String(), fmt.Sprintf("%s:LOCK", spec.want); got != want {
t.Errorf("%#v.String() = %q; want %q", tmpl, got, want)
}
}
}
-28
View File
@@ -1,28 +0,0 @@
// Package acme abstracts away various ACME libraries
package acme
import (
"crypto/tls"
"errors"
"net"
)
var (
// ErrProviderNotImplemented can be returned when attempting to
// instantiate an unimplemented provider
ErrProviderNotImplemented = errors.New("Provider not implemented")
)
// Provider is a ACME provider interface
type Provider interface {
// Listen returns a new listener
Listen(...string) (net.Listener, error)
// TLSConfig returns a tls config
TLSConfig(...string) (*tls.Config, error)
}
// The Let's Encrypt ACME endpoints
const (
LetsEncryptStagingCA = "https://acme-staging-v02.api.letsencrypt.org/directory"
LetsEncryptProductionCA = "https://acme-v02.api.letsencrypt.org/directory"
)
-46
View File
@@ -1,46 +0,0 @@
// Package autocert is the ACME provider from golang.org/x/crypto/acme/autocert
// This provider does not take any config.
package autocert
import (
"crypto/tls"
"net"
"os"
"go-micro.dev/v4/api/server/acme"
"go-micro.dev/v4/logger"
"golang.org/x/crypto/acme/autocert"
)
// autoCertACME is the ACME provider from golang.org/x/crypto/acme/autocert
type autocertProvider struct{}
// Listen implements acme.Provider
func (a *autocertProvider) Listen(hosts ...string) (net.Listener, error) {
return autocert.NewListener(hosts...), nil
}
// TLSConfig returns a new tls config
func (a *autocertProvider) TLSConfig(hosts ...string) (*tls.Config, error) {
// create a new manager
m := &autocert.Manager{
Prompt: autocert.AcceptTOS,
}
if len(hosts) > 0 {
m.HostPolicy = autocert.HostWhitelist(hosts...)
}
dir := cacheDir()
if err := os.MkdirAll(dir, 0700); err != nil {
if logger.V(logger.InfoLevel, logger.DefaultLogger) {
logger.Infof("warning: autocert not using a cache: %v", err)
}
} else {
m.Cache = autocert.DirCache(dir)
}
return m.TLSConfig(), nil
}
// New returns an autocert acme.Provider
func NewProvider() acme.Provider {
return &autocertProvider{}
}
-16
View File
@@ -1,16 +0,0 @@
package autocert
import (
"testing"
)
func TestAutocert(t *testing.T) {
l := NewProvider()
if _, ok := l.(*autocertProvider); !ok {
t.Error("NewProvider() didn't return an autocertProvider")
}
// TODO: Travis CI doesn't let us bind :443
// if _, err := l.NewListener(); err != nil {
// t.Error(err.Error())
// }
}
-37
View File
@@ -1,37 +0,0 @@
package autocert
import (
"os"
"path/filepath"
"runtime"
)
func homeDir() string {
if runtime.GOOS == "windows" {
return os.Getenv("HOMEDRIVE") + os.Getenv("HOMEPATH")
}
if h := os.Getenv("HOME"); h != "" {
return h
}
return "/"
}
func cacheDir() string {
const base = "golang-autocert"
switch runtime.GOOS {
case "darwin":
return filepath.Join(homeDir(), "Library", "Caches", base)
case "windows":
for _, ev := range []string{"APPDATA", "CSIDL_APPDATA", "TEMP", "TMP"} {
if v := os.Getenv(ev); v != "" {
return filepath.Join(v, base)
}
}
// Worst case:
return filepath.Join(homeDir(), base)
}
if xdg := os.Getenv("XDG_CACHE_HOME"); xdg != "" {
return filepath.Join(xdg, base)
}
return filepath.Join(homeDir(), ".cache", base)
}
-73
View File
@@ -1,73 +0,0 @@
package acme
import "github.com/go-acme/lego/v4/challenge"
// Option (or Options) are passed to New() to configure providers
type Option func(o *Options)
// Options represents various options you can present to ACME providers
type Options struct {
// AcceptTLS must be set to true to indicate that you have read your
// provider's terms of service.
AcceptToS bool
// CA is the CA to use
CA string
// ChallengeProvider is a go-acme/lego challenge provider. Set this if you
// want to use DNS Challenges. Otherwise, tls-alpn-01 will be used
ChallengeProvider challenge.Provider
// Issue certificates for domains on demand. Otherwise, certs will be
// retrieved / issued on start-up.
OnDemand bool
// Cache is a storage interface. Most ACME libraries have an cache, but
// there's no defined interface, so if you consume this option
// sanity check it before using.
Cache interface{}
}
// AcceptToS indicates whether you accept your CA's terms of service
func AcceptToS(b bool) Option {
return func(o *Options) {
o.AcceptToS = b
}
}
// CA sets the CA of an acme.Options
func CA(CA string) Option {
return func(o *Options) {
o.CA = CA
}
}
// ChallengeProvider sets the Challenge provider of an acme.Options
// if set, it enables the DNS challenge, otherwise tls-alpn-01 will be used.
func ChallengeProvider(p challenge.Provider) Option {
return func(o *Options) {
o.ChallengeProvider = p
}
}
// OnDemand enables on-demand certificate issuance. Not recommended for use
// with the DNS challenge, as the first connection may be very slow.
func OnDemand(b bool) Option {
return func(o *Options) {
o.OnDemand = b
}
}
// Cache provides a cache / storage interface to the underlying ACME library
// as there is no standard, this needs to be validated by the underlying
// implentation.
func Cache(c interface{}) Option {
return func(o *Options) {
o.Cache = c
}
}
// DefaultOptions uses the Let's Encrypt Production CA, with DNS Challenge disabled.
func DefaultOptions() Options {
return Options{
AcceptToS: true,
CA: LetsEncryptProductionCA,
OnDemand: true,
}
}
-57
View File
@@ -1,57 +0,0 @@
package cors
import (
"net/http"
)
type Config struct {
AllowOrigin string
AllowCredentials bool
AllowMethods string
AllowHeaders string
}
// CombinedCORSHandler wraps a server and provides CORS headers
func CombinedCORSHandler(h http.Handler, config *Config) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if config != nil {
SetHeaders(w, r, config)
}
if r.Method == "OPTIONS" {
return
}
h.ServeHTTP(w, r)
})
}
// SetHeaders sets the CORS headers
func SetHeaders(w http.ResponseWriter, _ *http.Request, config *Config) {
set := func(w http.ResponseWriter, k, v string) {
if v := w.Header().Get(k); len(v) > 0 {
return
}
w.Header().Set(k, v)
}
//For forward-compatible code, default values may not be provided in the future
if config.AllowCredentials {
set(w, "Access-Control-Allow-Credentials", "true")
} else {
set(w, "Access-Control-Allow-Credentials", "false")
}
if config.AllowOrigin == "" {
set(w, "Access-Control-Allow-Origin", "*")
} else {
set(w, "Access-Control-Allow-Origin", config.AllowOrigin)
}
if config.AllowMethods == "" {
set(w, "Access-Control-Allow-Methods", "POST, PATCH, GET, OPTIONS, PUT, DELETE")
} else {
set(w, "Access-Control-Allow-Methods", config.AllowMethods)
}
if config.AllowHeaders == "" {
set(w, "Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
} else {
set(w, "Access-Control-Allow-Headers", config.AllowHeaders)
}
}
-120
View File
@@ -1,120 +0,0 @@
// Package http provides a http server with features; acme, cors, etc
package http
import (
"crypto/tls"
"net"
"net/http"
"os"
"sync"
"go-micro.dev/v4/api/server"
"go-micro.dev/v4/api/server/cors"
"go-micro.dev/v4/logger"
"github.com/gorilla/handlers"
)
type httpServer struct {
mux *http.ServeMux
opts server.Options
mtx sync.RWMutex
address string
exit chan chan error
}
func NewServer(address string, opts ...server.Option) server.Server {
var options server.Options
for _, o := range opts {
o(&options)
}
return &httpServer{
opts: options,
mux: http.NewServeMux(),
address: address,
exit: make(chan chan error),
}
}
func (s *httpServer) Address() string {
s.mtx.RLock()
defer s.mtx.RUnlock()
return s.address
}
func (s *httpServer) Init(opts ...server.Option) error {
for _, o := range opts {
o(&s.opts)
}
return nil
}
func (s *httpServer) Handle(path string, handler http.Handler) {
// TODO: move this stuff out to one place with ServeHTTP
// apply the wrappers, e.g. auth
for _, wrapper := range s.opts.Wrappers {
handler = wrapper(handler)
}
// wrap with cors
if s.opts.EnableCORS {
handler = cors.CombinedCORSHandler(handler, s.opts.CORSConfig)
}
// wrap with logger
handler = handlers.CombinedLoggingHandler(os.Stdout, handler)
s.mux.Handle(path, handler)
}
func (s *httpServer) Start() error {
var l net.Listener
var err error
if s.opts.EnableACME && s.opts.ACMEProvider != nil {
// should we check the address to make sure its using :443?
l, err = s.opts.ACMEProvider.Listen(s.opts.ACMEHosts...)
} else if s.opts.EnableTLS && s.opts.TLSConfig != nil {
l, err = tls.Listen("tcp", s.address, s.opts.TLSConfig)
} else {
// otherwise plain listen
l, err = net.Listen("tcp", s.address)
}
if err != nil {
return err
}
if logger.V(logger.InfoLevel, logger.DefaultLogger) {
logger.Infof("HTTP API Listening on %s", l.Addr().String())
}
s.mtx.Lock()
s.address = l.Addr().String()
s.mtx.Unlock()
go func() {
if err := http.Serve(l, s.mux); err != nil {
// temporary fix
//logger.Fatal(err)
}
}()
go func() {
ch := <-s.exit
ch <- l.Close()
}()
return nil
}
func (s *httpServer) Stop() error {
ch := make(chan error)
s.exit <- ch
return <-ch
}
func (s *httpServer) String() string {
return "http"
}
-114
View File
@@ -1,114 +0,0 @@
package http
import (
"fmt"
"go-micro.dev/v4/api/server"
"go-micro.dev/v4/api/server/cors"
"io/ioutil"
"net/http"
"testing"
)
func TestHTTPServer(t *testing.T) {
testResponse := "hello world"
s := NewServer("localhost:0")
s.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, testResponse)
}))
if err := s.Start(); err != nil {
t.Fatal(err)
}
rsp, err := http.Get(fmt.Sprintf("http://%s/", s.Address()))
if err != nil {
t.Fatal(err)
}
defer rsp.Body.Close()
b, err := ioutil.ReadAll(rsp.Body)
if err != nil {
t.Fatal(err)
}
if string(b) != testResponse {
t.Fatalf("Unexpected response, got %s, expected %s", string(b), testResponse)
}
if err := s.Stop(); err != nil {
t.Fatal(err)
}
}
func TestCORSHTTPServer(t *testing.T) {
testResponse := "hello world"
testAllowOrigin := "*"
testAllowCredentials := true
testAllowMethods := "GET"
testAllowHeaders := "Accept, Content-Type, Content-Length"
s := NewServer("localhost:0",
server.EnableCORS(true),
server.CORSConfig(&cors.Config{
AllowCredentials: testAllowCredentials,
AllowOrigin: testAllowOrigin,
AllowMethods: testAllowMethods,
AllowHeaders: testAllowHeaders,
}),
)
s.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, testResponse)
}))
if err := s.Start(); err != nil {
t.Fatal(err)
}
rsp, err := http.Get(fmt.Sprintf("http://%s/", s.Address()))
if err != nil {
t.Fatal(err)
}
defer rsp.Body.Close()
b, err := ioutil.ReadAll(rsp.Body)
if err != nil {
t.Fatal(err)
}
if string(b) != testResponse {
t.Fatalf("Unexpected response, got %s, expected %s", string(b), testResponse)
}
allowCredentials := rsp.Header.Get("Access-Control-Allow-Credentials")
getTestCredentialsStr := func() string {
if testAllowCredentials == true {
return "true"
} else {
return "false"
}
}
if getTestCredentialsStr() != allowCredentials {
t.Fatalf("Unexpected Access-Control-Allow-Credentials, got %s, expected %s", allowCredentials, getTestCredentialsStr())
}
allowOrigin := rsp.Header.Get("Access-Control-Allow-Origin")
if testAllowOrigin != allowOrigin {
t.Fatalf("Unexpected Access-Control-Allow-Origins, got %s, expected %s", allowOrigin, testAllowOrigin)
}
allowMethods := rsp.Header.Get("Access-Control-Allow-Methods")
if testAllowMethods != allowMethods {
t.Fatalf("Unexpected Access-Control-Allow-Methods, got %s, expected %s", allowMethods, testAllowMethods)
}
allowHeaders := rsp.Header.Get("Access-Control-Allow-Headers")
if testAllowHeaders != allowHeaders {
t.Fatalf("Unexpected Access-Control-Allow-Headers, got %s, expected %s", allowHeaders, testAllowHeaders)
}
if err := s.Stop(); err != nil {
t.Fatal(err)
}
}
-80
View File
@@ -1,80 +0,0 @@
package server
import (
"crypto/tls"
"go-micro.dev/v4/api/server/cors"
"net/http"
"go-micro.dev/v4/api/resolver"
"go-micro.dev/v4/api/server/acme"
)
type Option func(o *Options)
type Options struct {
EnableACME bool
EnableCORS bool
CORSConfig *cors.Config
ACMEProvider acme.Provider
EnableTLS bool
ACMEHosts []string
TLSConfig *tls.Config
Resolver resolver.Resolver
Wrappers []Wrapper
}
type Wrapper func(h http.Handler) http.Handler
func WrapHandler(w Wrapper) Option {
return func(o *Options) {
o.Wrappers = append(o.Wrappers, w)
}
}
func EnableCORS(b bool) Option {
return func(o *Options) {
o.EnableCORS = b
}
}
func CORSConfig(c *cors.Config) Option {
return func(o *Options) {
o.CORSConfig = c
}
}
func EnableACME(b bool) Option {
return func(o *Options) {
o.EnableACME = b
}
}
func ACMEHosts(hosts ...string) Option {
return func(o *Options) {
o.ACMEHosts = hosts
}
}
func ACMEProvider(p acme.Provider) Option {
return func(o *Options) {
o.ACMEProvider = p
}
}
func EnableTLS(b bool) Option {
return func(o *Options) {
o.EnableTLS = b
}
}
func TLSConfig(t *tls.Config) Option {
return func(o *Options) {
o.TLSConfig = t
}
}
func Resolver(r resolver.Resolver) Option {
return func(o *Options) {
o.Resolver = r
}
}
-15
View File
@@ -1,15 +0,0 @@
// Package server provides an API gateway server which handles inbound requests
package server
import (
"net/http"
)
// Server serves api requests
type Server interface {
Address() string
Init(opts ...Option) error
Handle(path string, handler http.Handler)
Start() error
Stop() error
}
+27 -27
View File
@@ -8,22 +8,22 @@ import (
)
const (
// BearerScheme used for Authorization header
// BearerScheme used for Authorization header.
BearerScheme = "Bearer "
// ScopePublic is the scope applied to a rule to allow access to the public
// ScopePublic is the scope applied to a rule to allow access to the public.
ScopePublic = ""
// ScopeAccount is the scope applied to a rule to limit to users with any valid account
// ScopeAccount is the scope applied to a rule to limit to users with any valid account.
ScopeAccount = "*"
)
var (
// ErrInvalidToken is when the token provided is not valid
// ErrInvalidToken is when the token provided is not valid.
ErrInvalidToken = errors.New("invalid token provided")
// ErrForbidden is when a user does not have the necessary scope to access a resource
// ErrForbidden is when a user does not have the necessary scope to access a resource.
ErrForbidden = errors.New("resource forbidden")
)
// Auth provides authentication and authorization
// Auth provides authentication and authorization.
type Auth interface {
// Init the auth
Init(opts ...Option)
@@ -39,7 +39,7 @@ type Auth interface {
String() string
}
// Rules manages access to resources
// Rules manages access to resources.
type Rules interface {
// Verify an account has access to a resource using the rules
Verify(acc *Account, res *Resource, opts ...VerifyOption) error
@@ -51,40 +51,40 @@ type Rules interface {
List(...ListOption) ([]*Rule, error)
}
// Account provided by an auth provider
// Account provided by an auth provider.
type Account struct {
// Any other associated metadata
Metadata map[string]string `json:"metadata"`
// ID of the account e.g. email
ID string `json:"id"`
// Type of the account, e.g. service
Type string `json:"type"`
// Issuer of the account
Issuer string `json:"issuer"`
// Any other associated metadata
Metadata map[string]string `json:"metadata"`
// Scopes the account has access to
Scopes []string `json:"scopes"`
// Secret for the account, e.g. the password
Secret string `json:"secret"`
// Scopes the account has access to
Scopes []string `json:"scopes"`
}
// Token can be short or long lived
// Token can be short or long lived.
type Token struct {
// The token to be used for accessing resources
AccessToken string `json:"access_token"`
// RefreshToken to be used to generate a new token
RefreshToken string `json:"refresh_token"`
// Time of token creation
Created time.Time `json:"created"`
// Time of token expiry
Expiry time.Time `json:"expiry"`
// The token to be used for accessing resources
AccessToken string `json:"access_token"`
// RefreshToken to be used to generate a new token
RefreshToken string `json:"refresh_token"`
}
// Expired returns a boolean indicating if the token needs to be refreshed
// Expired returns a boolean indicating if the token needs to be refreshed.
func (t *Token) Expired() bool {
return t.Expiry.Unix() < time.Now().Unix()
}
// Resource is an entity such as a user or
// Resource is an entity such as a user or.
type Resource struct {
// Name of the resource, e.g. go.micro.service.notes
Name string `json:"name"`
@@ -94,25 +94,25 @@ type Resource struct {
Endpoint string `json:"endpoint"`
}
// Access defines the type of access a rule grants
// Access defines the type of access a rule grants.
type Access int
const (
// AccessGranted to a resource
// AccessGranted to a resource.
AccessGranted Access = iota
// AccessDenied to a resource
// AccessDenied to a resource.
AccessDenied
)
// Rule is used to verify access to a resource
// Rule is used to verify access to a resource.
type Rule struct {
// Resource the rule applies to
Resource *Resource
// ID of the rule, e.g. "public"
ID string
// Scope the rule requires, a blank scope indicates open to the public and * indicates the rule
// applies to any valid account
Scope string
// Resource the rule applies to
Resource *Resource
// Access determines if the rule grants or denies access to the resource
Access Access
// Priority the rule should take when verifying a request, the higher the value the sooner the
@@ -125,13 +125,13 @@ type accountKey struct{}
// AccountFromContext gets the account from the context, which
// is set by the auth wrapper at the start of a call. If the account
// is not set, a nil account will be returned. The error is only returned
// when there was a problem retrieving an account
// when there was a problem retrieving an account.
func AccountFromContext(ctx context.Context) (*Account, bool) {
acc, ok := ctx.Value(accountKey{}).(*Account)
return acc, ok
}
// ContextWithAccount sets the account in the context
// ContextWithAccount sets the account in the context.
func ContextWithAccount(ctx context.Context, account *Account) context.Context {
return context.WithValue(ctx, accountKey{}, account)
}
+5 -5
View File
@@ -4,16 +4,16 @@ import (
"sync"
"time"
"go-micro.dev/v4/auth"
"go-micro.dev/v4/cmd"
jwtToken "github.com/asim/go-micro/plugins/auth/jwt/v4/token"
jwtToken "github.com/micro/plugins/v5/auth/jwt/token"
"go-micro.dev/v5/auth"
"go-micro.dev/v5/cmd"
)
func init() {
cmd.DefaultAuths["jwt"] = NewAuth
}
// NewAuth returns a new instance of the Auth service
// NewAuth returns a new instance of the Auth service.
func NewAuth(opts ...auth.Option) auth.Auth {
j := new(jwt)
j.Init(opts...)
@@ -92,7 +92,7 @@ func (j *jwtRules) Revoke(rule *auth.Rule) error {
j.Lock()
defer j.Unlock()
rules := []*auth.Rule{}
rules := make([]*auth.Rule, 0, len(j.rules))
for _, r := range j.rules {
if r.ID != rule.ID {
rules = append(rules, r)
@@ -5,10 +5,10 @@ import (
"time"
"github.com/dgrijalva/jwt-go"
"go-micro.dev/v4/auth"
"go-micro.dev/v5/auth"
)
// authClaims to be encoded in the JWT
// authClaims to be encoded in the JWT.
type authClaims struct {
Type string `json:"type"`
Scopes []string `json:"scopes"`
@@ -17,19 +17,19 @@ type authClaims struct {
jwt.StandardClaims
}
// JWT implementation of token provider
// JWT implementation of token provider.
type JWT struct {
opts Options
}
// New returns an initialized basic provider
// New returns an initialized basic provider.
func New(opts ...Option) Provider {
return &JWT{
opts: NewOptions(opts...),
}
}
// Generate a new JWT
// Generate a new JWT.
func (j *JWT) Generate(acc *auth.Account, opts ...GenerateOption) (*Token, error) {
// decode the private key
priv, err := base64.StdEncoding.DecodeString(j.opts.PrivateKey)
@@ -68,7 +68,7 @@ func (j *JWT) Generate(acc *auth.Account, opts ...GenerateOption) (*Token, error
}, nil
}
// Inspect a JWT
// Inspect a JWT.
func (j *JWT) Inspect(t string) (*auth.Account, error) {
// decode the public key
pub, err := base64.StdEncoding.DecodeString(j.opts.PublicKey)
@@ -103,7 +103,7 @@ func (j *JWT) Inspect(t string) (*auth.Account, error) {
}, nil
}
// String returns JWT
// String returns JWT.
func (j *JWT) String() string {
return "jwt"
}
@@ -1,15 +1,15 @@
package token
import (
"io/ioutil"
"os"
"testing"
"time"
"go-micro.dev/v4/auth"
"go-micro.dev/v5/auth"
)
func TestGenerate(t *testing.T) {
privKey, err := ioutil.ReadFile("test/sample_key")
privKey, err := os.ReadFile("test/sample_key")
if err != nil {
t.Fatalf("Unable to read private key: %v", err)
}
@@ -25,11 +25,11 @@ func TestGenerate(t *testing.T) {
}
func TestInspect(t *testing.T) {
pubKey, err := ioutil.ReadFile("test/sample_key.pub")
pubKey, err := os.ReadFile("test/sample_key.pub")
if err != nil {
t.Fatalf("Unable to read public key: %v", err)
}
privKey, err := ioutil.ReadFile("test/sample_key")
privKey, err := os.ReadFile("test/sample_key")
if err != nil {
t.Fatalf("Unable to read private key: %v", err)
}
@@ -82,5 +82,4 @@ func TestInspect(t *testing.T) {
t.Fatalf("Inspect returned %v error, expected %v", err, ErrInvalidToken)
}
})
}
@@ -3,7 +3,7 @@ package token
import (
"time"
"go-micro.dev/v4/store"
"go-micro.dev/v5/store"
)
type Options struct {
@@ -17,21 +17,21 @@ type Options struct {
type Option func(o *Options)
// WithStore sets the token providers store
// WithStore sets the token providers store.
func WithStore(s store.Store) Option {
return func(o *Options) {
o.Store = s
}
}
// WithPublicKey sets the JWT public key
// WithPublicKey sets the JWT public key.
func WithPublicKey(key string) Option {
return func(o *Options) {
o.PublicKey = key
}
}
// WithPrivateKey sets the JWT private key
// WithPrivateKey sets the JWT private key.
func WithPrivateKey(key string) Option {
return func(o *Options) {
o.PrivateKey = key
@@ -43,7 +43,7 @@ func NewOptions(opts ...Option) Options {
for _, o := range opts {
o(&options)
}
//set default store
// set default store
if options.Store == nil {
options.Store = store.DefaultStore
}
@@ -57,20 +57,20 @@ type GenerateOptions struct {
type GenerateOption func(o *GenerateOptions)
// WithExpiry for the generated account's token expires
// WithExpiry for the generated account's token expires.
func WithExpiry(d time.Duration) GenerateOption {
return func(o *GenerateOptions) {
o.Expiry = d
}
}
// NewGenerateOptions from a slice of options
// NewGenerateOptions from a slice of options.
func NewGenerateOptions(opts ...GenerateOption) GenerateOptions {
var options GenerateOptions
for _, o := range opts {
o(&options)
}
//set default Expiry of token
// set default Expiry of token
if options.Expiry == 0 {
options.Expiry = time.Minute * 15
}
+1
View File
@@ -0,0 +1 @@
LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS3dJQkFBS0NBZ0VBOFNiSlA1WGJFaWRSbTViMnNOcExHbzJlV2ZVNU9KZTBpemdySHdEOEg3RjZQa1BkCi9SbDkvMXBNVjdNaU8zTEh3dGhIQzJCUllxcisxd0Zkb1pDR0JZckxhWHVYRnFLMHZ1WmhQcUUzYXpqdUlIUXUKMEJIL2xYUU1xeUVxRjVNSTJ6ZWpDNHpNenIxNU9OK2dFNEpuaXBqcC9DZGpPUEFEbUpHK0JKOXFlRS9RUGVtLwptVWRJVC9MYUY3a1F4eVlLNVZLbitOZ09Xek1sektBQXBDbjdUVEtCVWU4RlpHNldTWDdMVjBlTEdIc29pYnhsCm85akRqbFk1b0JPY3pmcWVOV0hLNUdYQjdRd3BMTmg5NDZQelpucW9hcFdVZStZL1JPaUhpekpUY3I1Wk1TTDUKd2xFcThoTmhtaG01Tk5lL08rR2dqQkROU2ZVaDA2K3E0bmdtYm1OWDVoODM4QmJqUmN5YzM2ZHd6NkpVK2R1bwpSdFFoZ2lZOTEwcFBmOWJhdVhXcXdVQ1VhNHFzSHpqS1IwTC9OMVhYQXlsQ0RqeWVnWnp6Y093MkNIOFNrZkZVCnJnTHJQYkVCOWVnY0drMzgrYnBLczNaNlJyNSt0bkQxQklQSUZHTGVJMFVPQzAreGlCdjBvenhJRE9GbldhOVUKVEdEeFV4OG9qOFZJZVJuV0RxNk1jMWlKcDhVeWNpQklUUnR3NGRabzcweG1mbmVJV3pyM0tTTmFoU29nSmRSMApsYVF6QXVQM2FpV1hJTXAyc2M4U2MrQmwrTGpYbUJveEJyYUJIaDlLa0pKRWNnQUZ3czJib2pDbEpPWXhvRi9YCmdGS1NzSW5IRHJIVk95V1BCZTNmYWRFYzc3YituYi9leE96cjFFcnhoR2c5akZtcmtPK3M0eEdodjZNQ0F3RUEKQVFLQ0FnRUFqUzc1Q2VvUlRRcUtBNzZaaFNiNGEzNVlKRENtcEpSazFsRTNKYnFzNFYxRnhXaDBjZmJYeG9VMgpSdTRRYjUrZWhsdWJGSFQ2a1BxdG9uRWhRVExjMUNmVE9WbHJOb3hocDVZM2ZyUmlQcnNnNXcwK1R3RUtrcFJUCnltanJQTXdQbGxCM2U0NmVaYmVXWGc3R3FFVmptMGcxVFRRK0tocVM4R0w3VGJlTFhRN1ZTem9ydTNCNVRKMVEKeEN6TVB0dnQ2eDYrU3JrcmhvZG1iT3VNRkpDam1TbWxmck9pZzQ4Zkc3NUpERHRObXpLWHBEUVJpYUNodFJhVQpQRHpmUTlTamhYdFFqdkZvWFFFT3BqdkZVRjR2WldNUWNQNUw1VklDM3JRSWp4MFNzQTN6S0FwakVUbjJHNjN2CktZby8zVWttbzhkUCtGRHA3NCs5a3pLNHFFaFJycEl3bEtiN0VOZWtDUXZqUFl1K3pyKzMyUXdQNTJ2L2FveWQKdjJJaUY3M2laTU1vZDhhYjJuQStyVEI2T0cvOVlSYk5kV21tay9VTi9jUHYrN214TmZ6Y1d1ZU1XcThxMXh4eAptNTNpR0NSQ29PQ1lDQk4zcUFkb1JwYW5xd3lCOUxrLzFCQjBHUld3MjgxK3VhNXNYRnZBVDBKeTVURnduMncvClU1MlJKWFlNOXVhMFBvd214b0RDUWRuNFZYVkdNZGdXaHN4aXhHRlYwOUZObWJJQWJaN0xaWGtkS1gzc1ZVbTcKWU1WYWIzVVo2bEhtdXYzT1NzcHNVUlRqN1hiRzZpaVVlaDU1aW91OENWbnRndWtFcnEzQTQwT05FVzhjNDBzOQphVTBGaSs4eWZpQTViaVZHLzF0bWlucUVERkhuQStnWk1xNEhlSkZxcWZxaEZKa1JwRGtDZ2dFQkFQeGR1NGNKCm5Da1duZDdPWFlHMVM3UDdkVWhRUzgwSDlteW9uZFc5bGFCQm84RWRPeTVTZzNOUmsxQ2pNZFZ1a3FMcjhJSnkKeStLWk15SVpvSlJvbllaMEtIUUVMR3ZLbzFOS2NLQ1FJbnYvWHVCdFJpRzBVb1pQNVkwN0RpRFBRQWpYUjlXUwpBc0EzMmQ1eEtFOC91Y3h0MjVQVzJFakNBUmtVeHQ5d0tKazN3bC9JdXVYRlExTDdDWjJsOVlFUjlHeWxUbzhNCmxXUEY3YndtUFV4UVNKaTNVS0FjTzZweTVUU1lkdWQ2aGpQeXJwSXByNU42VGpmTlRFWkVBeU9LbXVpOHVkUkoKMUg3T3RQVEhGZElKQjNrNEJnRDZtRE1HbjB2SXBLaDhZN3NtRUZBbFkvaXlCZjMvOHk5VHVMb1BycEdqR3RHbgp4Y2RpMHFud2p0SGFNbFVDZ2dFQkFQU2Z0dVFCQ2dTU2JLUSswUEFSR2VVeEQyTmlvZk1teENNTmdHUzJ5Ull3CjRGaGV4ZWkwMVJoaFk1NjE3UjduR1dzb0czd1RQa3dvRTJtbE1aQkoxeWEvUU9RRnQ3WG02OVl0RGh0T2FWbDgKL0o4dlVuSTBtWmxtT2pjTlRoYnVPZDlNSDlRdGxIRUMxMlhYdHJNb3Fsb0U2a05TT0pJalNxYm9wcDRXc1BqcApvZTZ0Nkdyd1RhOHBHeUJWWS90Mi85Ym5ORHVPVlpjODBaODdtY2gzcDNQclBqU3h5di9saGxYMFMwYUdHTkhTCk1XVjdUa25OaGo1TWlIRXFnZ1pZemtBWTkyd1JoVENnU1A2M0VNcitUWXFudXVuMXJHbndPYm95TDR2aFRpV0UKcU42UDNCTFlCZ1FpMllDTDludEJrOEl6RHZyd096dW5GVnhhZ0g5SVVoY0NnZ0VCQUwzQXlLa1BlOENWUmR6cQpzL284VkJDZmFSOFhhUGRnSGxTek1BSXZpNXEwNENqckRyMlV3MHZwTVdnM1hOZ0xUT3g5bFJpd3NrYk9SRmxHCmhhd3hRUWlBdkk0SE9WTlBTU0R1WHVNTG5USTQ0S0RFNlMrY2cxU0VMS2pWbDVqcDNFOEpkL1RJMVpLc0xBQUsKZTNHakM5UC9ZbE8xL21ndW4xNjVkWk01cFAwWHBPb2FaeFV2RHFFTktyekR0V1g0RngyOTZlUzdaSFJodFpCNwovQ2t1VUhlcmxrN2RDNnZzdWhTaTh2eTM3c0tPbmQ0K3c4cVM4czhZYVZxSDl3ZzVScUxxakp0bmJBUnc3alVDCm9KQ053M1hNdnc3clhaYzRTbnhVQUNMRGJNV2lLQy9xL1ZGWW9oTEs2WkpUVkJscWd5cjBSYzBRWmpDMlNJb0kKMjRwRWt3VUNnZ0VCQUpqb0FJVVNsVFY0WlVwaExXN3g4WkxPa01UWjBVdFFyd2NPR0hSYndPUUxGeUNGMVFWNQppejNiR2s4SmZyZHpVdk1sTmREZm9uQXVHTHhQa3VTVEUxWlg4L0xVRkJveXhyV3dvZ0cxaUtwME11QTV6em90CjROai9DbUtCQVkvWnh2anA5M2RFS21aZGxWQkdmeUFMeWpmTW5MWUovZXh5L09YSnhPUktZTUttSHg4M08zRWsKMWhvb0FwbTZabTIzMjRGME1iVU1ham5Idld2ZjhHZGJTNk5zcHd4L0dkbk1tYVMrdUJMVUhVMkNLbmc1bEIwVAp4OWJITmY0dXlPbTR0dXRmNzhCd1R5V3UreEdrVW0zZ2VZMnkvR1hqdDZyY2l1ajFGNzFDenZzcXFmZThTcDdJCnd6SHdxcTNzVHR5S2lCYTZuYUdEYWpNR1pKYSt4MVZJV204Q2dnRUJBT001ajFZR25Ba0pxR0czQWJSVDIvNUMKaVVxN0loYkswOGZsSGs5a2YwUlVjZWc0ZVlKY3dIRXJVaE4rdWQyLzE3MC81dDYra0JUdTVZOUg3bkpLREtESQpoeEg5SStyamNlVkR0RVNTRkluSXdDQ1lrOHhOUzZ0cHZMV1U5b0pibGFKMlZsalV2NGRFWGVQb0hkREh1Zk9ZClVLa0lsV2E3Uit1QzNEOHF5U1JrQnFLa3ZXZ1RxcFNmTVNkc1ZTeFIzU2Q4SVhFSHFjTDNUNEtMWGtYNEdEamYKMmZOSTFpZkx6ekhJMTN3Tk5IUTVRNU9SUC9pell2QzVzZkx4U2ZIUXJiMXJZVkpKWkI5ZjVBUjRmWFpHSVFsbApjMG8xd0JmZFlqMnZxVDlpR09IQnNSSTlSL2M2RzJQcUt3aFRpSzJVR2lmVFNEUVFuUkF6b2tpQVkrbE8vUjQ9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
@@ -4,19 +4,19 @@ import (
"errors"
"time"
"go-micro.dev/v4/auth"
"go-micro.dev/v5/auth"
)
var (
// ErrNotFound is returned when a token cannot be found
// ErrNotFound is returned when a token cannot be found.
ErrNotFound = errors.New("token not found")
// ErrEncodingToken is returned when the service encounters an error during encoding
// ErrEncodingToken is returned when the service encounters an error during encoding.
ErrEncodingToken = errors.New("error encoding the token")
// ErrInvalidToken is returned when the token provided is not valid
// ErrInvalidToken is returned when the token provided is not valid.
ErrInvalidToken = errors.New("invalid token provided")
)
// Provider generates and inspects tokens
// Provider generates and inspects tokens.
type Provider interface {
Generate(account *auth.Account, opts ...GenerateOption) (*Token, error)
Inspect(token string) (*auth.Account, error)
+9 -9
View File
@@ -30,24 +30,24 @@ type noop struct {
type noopRules struct{}
// String returns the name of the implementation
// String returns the name of the implementation.
func (n *noop) String() string {
return "noop"
}
// Init the auth
// Init the auth.
func (n *noop) Init(opts ...Option) {
for _, o := range opts {
o(&n.opts)
}
}
// Options set for auth
// Options set for auth.
func (n *noop) Options() Options {
return n.opts
}
// Generate a new account
// Generate a new account.
func (n *noop) Generate(id string, opts ...GenerateOption) (*Account, error) {
options := NewGenerateOptions(opts...)
@@ -60,18 +60,18 @@ func (n *noop) Generate(id string, opts ...GenerateOption) (*Account, error) {
}, nil
}
// Grant access to a resource
// Grant access to a resource.
func (n *noopRules) Grant(rule *Rule) error {
return nil
}
// Revoke access to a resource
// Revoke access to a resource.
func (n *noopRules) Revoke(rule *Rule) error {
return nil
}
// Rules used to verify requests
// Verify an account has access to a resource
// Verify an account has access to a resource.
func (n *noopRules) Verify(acc *Account, res *Resource, opts ...VerifyOption) error {
return nil
}
@@ -80,12 +80,12 @@ func (n *noopRules) List(opts ...ListOption) ([]*Rule, error) {
return []*Rule{}, nil
}
// Inspect a token
// Inspect a token.
func (n *noop) Inspect(token string) (*Account, error) {
return &Account{ID: uuid.New().String(), Issuer: n.Options().Namespace}, nil
}
// Token generation using an account id and secret
// Token generation using an account id and secret.
func (n *noop) Token(opts ...TokenOption) (*Token, error) {
return &Token{}, nil
}
+35 -20
View File
@@ -3,25 +3,33 @@ package auth
import (
"context"
"time"
"go-micro.dev/v5/logger"
)
func NewOptions(opts ...Option) Options {
var options Options
options := Options{
Logger: logger.DefaultLogger,
}
for _, o := range opts {
o(&options)
}
return options
}
type Options struct {
// Logger is the underline logger
Logger logger.Logger
// Token is the services token used to authenticate itself
Token *Token
// Namespace the service belongs to
Namespace string
// ID is the services auth ID
ID string
// Secret is used to authenticate the service
Secret string
// Token is the services token used to authenticate itself
Token *Token
// PublicKey for decoding JWTs
PublicKey string
// PrivateKey for encoding JWTs
@@ -32,35 +40,42 @@ type Options struct {
type Option func(o *Options)
// Addrs is the auth addresses to use
// Addrs is the auth addresses to use.
func Addrs(addrs ...string) Option {
return func(o *Options) {
o.Addrs = addrs
}
}
// Namespace the service belongs to
// Namespace the service belongs to.
func Namespace(n string) Option {
return func(o *Options) {
o.Namespace = n
}
}
// PublicKey is the JWT public key
// PublicKey is the JWT public key.
func PublicKey(key string) Option {
return func(o *Options) {
o.PublicKey = key
}
}
// PrivateKey is the JWT private key
// PrivateKey is the JWT private key.
func PrivateKey(key string) Option {
return func(o *Options) {
o.PrivateKey = key
}
}
// Credentials sets the auth credentials
// WithLogger sets the underline logger.
func WithLogger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// Credentials sets the auth credentials.
func Credentials(id, secret string) Option {
return func(o *Options) {
o.ID = id
@@ -68,7 +83,7 @@ func Credentials(id, secret string) Option {
}
}
// ClientToken sets the auth token to use when making requests
// ClientToken sets the auth token to use when making requests.
func ClientToken(token *Token) Option {
return func(o *Options) {
o.Token = token
@@ -78,54 +93,54 @@ func ClientToken(token *Token) Option {
type GenerateOptions struct {
// Metadata associated with the account
Metadata map[string]string
// Scopes the account has access too
Scopes []string
// Provider of the account, e.g. oauth
Provider string
// Type of the account, e.g. user
Type string
// Secret used to authenticate the account
Secret string
// Scopes the account has access too
Scopes []string
}
type GenerateOption func(o *GenerateOptions)
// WithSecret for the generated account
// WithSecret for the generated account.
func WithSecret(s string) GenerateOption {
return func(o *GenerateOptions) {
o.Secret = s
}
}
// WithType for the generated account
// WithType for the generated account.
func WithType(t string) GenerateOption {
return func(o *GenerateOptions) {
o.Type = t
}
}
// WithMetadata for the generated account
// WithMetadata for the generated account.
func WithMetadata(md map[string]string) GenerateOption {
return func(o *GenerateOptions) {
o.Metadata = md
}
}
// WithProvider for the generated account
// WithProvider for the generated account.
func WithProvider(p string) GenerateOption {
return func(o *GenerateOptions) {
o.Provider = p
}
}
// WithScopes for the generated account
// WithScopes for the generated account.
func WithScopes(s ...string) GenerateOption {
return func(o *GenerateOptions) {
o.Scopes = s
}
}
// NewGenerateOptions from a slice of options
// NewGenerateOptions from a slice of options.
func NewGenerateOptions(opts ...GenerateOption) GenerateOptions {
var options GenerateOptions
for _, o := range opts {
@@ -147,7 +162,7 @@ type TokenOptions struct {
type TokenOption func(o *TokenOptions)
// WithExpiry for the token
// WithExpiry for the token.
func WithExpiry(ex time.Duration) TokenOption {
return func(o *TokenOptions) {
o.Expiry = ex
@@ -167,14 +182,14 @@ func WithToken(rt string) TokenOption {
}
}
// NewTokenOptions from a slice of options
// NewTokenOptions from a slice of options.
func NewTokenOptions(opts ...TokenOption) TokenOptions {
var options TokenOptions
for _, o := range opts {
o(&options)
}
// set defualt expiry of token
// set default expiry of token
if options.Expiry == 0 {
options.Expiry = time.Minute
}
+1 -1
View File
@@ -8,7 +8,7 @@ import (
// Verify an account has access to a resource using the rules provided. If the account does not have
// access an error will be returned. If there are no rules provided which match the resource, an error
// will be returned
// will be returned.
func Verify(rules []*Rule, acc *Account, res *Resource) error {
// the rule is only to be applied if the type matches the resource or is catch-all (*)
validTypes := []string{"*", res.Type}
+19 -19
View File
@@ -42,7 +42,7 @@ func TestVerify(t *testing.T) {
Account: &Account{},
Resource: srvResource,
Rules: []*Rule{
&Rule{
{
Scope: "",
Resource: catchallResource,
},
@@ -52,7 +52,7 @@ func TestVerify(t *testing.T) {
Name: "CatchallPublicNoAccount",
Resource: srvResource,
Rules: []*Rule{
&Rule{
{
Scope: "",
Resource: catchallResource,
},
@@ -63,7 +63,7 @@ func TestVerify(t *testing.T) {
Account: &Account{},
Resource: srvResource,
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
},
@@ -73,7 +73,7 @@ func TestVerify(t *testing.T) {
Name: "CatchallPrivateNoAccount",
Resource: srvResource,
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
},
@@ -85,7 +85,7 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: srvResource.Type,
@@ -100,7 +100,7 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: srvResource.Type,
@@ -118,7 +118,7 @@ func TestVerify(t *testing.T) {
Scopes: []string{"neededscope"},
},
Rules: []*Rule{
&Rule{
{
Scope: "neededscope",
Resource: srvResource,
},
@@ -131,7 +131,7 @@ func TestVerify(t *testing.T) {
Scopes: []string{"neededscope"},
},
Rules: []*Rule{
&Rule{
{
Scope: "invalidscope",
Resource: srvResource,
},
@@ -143,7 +143,7 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessDenied,
@@ -156,7 +156,7 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessDenied,
@@ -169,13 +169,13 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessGranted,
Priority: 1,
},
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessDenied,
@@ -188,13 +188,13 @@ func TestVerify(t *testing.T) {
Resource: srvResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessGranted,
Priority: 0,
},
&Rule{
{
Scope: "*",
Resource: catchallResource,
Access: AccessDenied,
@@ -208,7 +208,7 @@ func TestVerify(t *testing.T) {
Resource: webResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: webResource,
},
@@ -219,7 +219,7 @@ func TestVerify(t *testing.T) {
Resource: webResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: webResource.Type,
@@ -235,7 +235,7 @@ func TestVerify(t *testing.T) {
Resource: webResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: webResource.Type,
@@ -250,7 +250,7 @@ func TestVerify(t *testing.T) {
Resource: webResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: webResource.Type,
@@ -265,7 +265,7 @@ func TestVerify(t *testing.T) {
Resource: webResource,
Account: &Account{},
Rules: []*Rule{
&Rule{
{
Scope: "*",
Resource: &Resource{
Type: webResource.Type,
+6 -3
View File
@@ -18,12 +18,13 @@ type Broker interface {
// message and optional Ack method to acknowledge receipt of the message.
type Handler func(Event) error
// Message is a message send/received from the broker.
type Message struct {
Header map[string]string
Body []byte
}
// Event is given to a subscription handler for processing
// Event is given to a subscription handler for processing.
type Event interface {
Topic() string
Message() *Message
@@ -31,7 +32,7 @@ type Event interface {
Error() error
}
// Subscriber is a convenience return type for the Subscribe method
// Subscriber is a convenience return type for the Subscribe method.
type Subscriber interface {
Options() SubscribeOptions
Topic() string
@@ -39,7 +40,8 @@ type Subscriber interface {
}
var (
DefaultBroker Broker = NewBroker()
// DefaultBroker is the default Broker.
DefaultBroker = NewHttpBroker()
)
func Init(opts ...Option) error {
@@ -62,6 +64,7 @@ func Subscribe(topic string, handler Handler, opts ...SubscribeOption) (Subscrib
return DefaultBroker.Subscribe(topic, handler, opts...)
}
// String returns the name of the Broker.
func String() string {
return DefaultBroker.String()
}
+37 -40
View File
@@ -1,14 +1,11 @@
// Package http provides a http based message broker
package broker
import (
"bytes"
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"io/ioutil"
"math/rand"
"net"
"net/http"
@@ -17,51 +14,54 @@ import (
"sync"
"time"
"go-micro.dev/v4/codec/json"
merr "go-micro.dev/v4/errors"
"go-micro.dev/v4/registry"
"go-micro.dev/v4/registry/cache"
maddr "go-micro.dev/v4/util/addr"
mnet "go-micro.dev/v4/util/net"
mls "go-micro.dev/v4/util/tls"
"github.com/google/uuid"
"go-micro.dev/v5/codec/json"
merr "go-micro.dev/v5/errors"
"go-micro.dev/v5/registry"
"go-micro.dev/v5/registry/cache"
"go-micro.dev/v5/transport/headers"
maddr "go-micro.dev/v5/util/addr"
mnet "go-micro.dev/v5/util/net"
mls "go-micro.dev/v5/util/tls"
"golang.org/x/net/http2"
)
// HTTP Broker is a point to point async broker
// HTTP Broker is a point to point async broker.
type httpBroker struct {
id string
address string
opts Options
opts Options
r registry.Registry
mux *http.ServeMux
c *http.Client
r registry.Registry
sync.RWMutex
c *http.Client
subscribers map[string][]*httpSubscriber
running bool
exit chan chan error
inbox map[string][][]byte
id string
address string
sync.RWMutex
// offline message inbox
mtx sync.RWMutex
inbox map[string][][]byte
mtx sync.RWMutex
running bool
}
type httpSubscriber struct {
opts SubscribeOptions
id string
topic string
fn Handler
svc *registry.Service
hb *httpBroker
id string
topic string
}
type httpEvent struct {
err error
m *Message
t string
err error
}
var (
@@ -74,14 +74,12 @@ var (
)
func init() {
rand.Seed(time.Now().Unix())
}
func newTransport(config *tls.Config) *http.Transport {
if config == nil {
config = &tls.Config{
InsecureSkipVerify: true,
}
// Use environment-based config - secure by default
config = mls.Config()
}
dialTLS := func(network string, addr string) (net.Conn, error) {
@@ -108,11 +106,10 @@ func newTransport(config *tls.Config) *http.Transport {
}
func newHttpBroker(opts ...Option) Broker {
options := Options{
Codec: json.Marshaler{},
Context: context.TODO(),
Registry: registry.DefaultRegistry,
}
options := *NewOptions(opts...)
options.Registry = registry.DefaultRegistry
options.Codec = json.Marshaler{}
for _, o := range opts {
o(&options)
@@ -300,7 +297,7 @@ func (h *httpBroker) ServeHTTP(w http.ResponseWriter, req *http.Request) {
req.ParseForm()
b, err := ioutil.ReadAll(req.Body)
b, err := io.ReadAll(req.Body)
if err != nil {
errr := merr.InternalServerError("go.micro.broker", "Error reading request body: %v", err)
w.WriteHeader(500)
@@ -316,8 +313,8 @@ func (h *httpBroker) ServeHTTP(w http.ResponseWriter, req *http.Request) {
return
}
topic := m.Header["Micro-Topic"]
//delete(m.Header, ":topic")
topic := m.Header[headers.Message]
// delete(m.Header, ":topic")
if len(topic) == 0 {
errr := merr.InternalServerError("go.micro.broker", "Topic not found")
@@ -520,7 +517,7 @@ func (h *httpBroker) Publish(topic string, msg *Message, opts ...PublishOption)
m.Header[k] = v
}
m.Header["Micro-Topic"] = topic
m.Header[headers.Message] = topic
// encode the message
b, err := h.opts.Codec.Marshal(m)
@@ -558,7 +555,7 @@ func (h *httpBroker) Publish(topic string, msg *Message, opts ...PublishOption)
}
// discard response body
io.Copy(ioutil.Discard, r.Body)
io.Copy(io.Discard, r.Body)
r.Body.Close()
return nil
}
@@ -705,7 +702,7 @@ func (h *httpBroker) String() string {
return "http"
}
// NewBroker returns a new http broker
func NewBroker(opts ...Option) Broker {
// NewHttpBroker returns a new http broker.
func NewHttpBroker(opts ...Option) Broker {
return newHttpBroker(opts...)
}
+52 -60
View File
@@ -5,13 +5,13 @@ import (
"testing"
"time"
"go-micro.dev/v4/broker"
"go-micro.dev/v4/registry"
"github.com/google/uuid"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/registry"
)
var (
// mock data
// mock data.
testData = map[string][]*registry.Service{
"foo": {
{
@@ -56,19 +56,19 @@ func newTestRegistry() registry.Registry {
return registry.NewMemoryRegistry(registry.Services(testData))
}
func sub(be *testing.B, c int) {
be.StopTimer()
func sub(b *testing.B, c int) {
b.StopTimer()
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
brker := broker.NewHttpBroker(broker.Registry(m))
topic := uuid.New().String()
if err := b.Init(); err != nil {
be.Fatalf("Unexpected init error: %v", err)
if err := brker.Init(); err != nil {
b.Fatalf("Unexpected init error: %v", err)
}
if err := b.Connect(); err != nil {
be.Fatalf("Unexpected connect error: %v", err)
if err := brker.Connect(); err != nil {
b.Fatalf("Unexpected connect error: %v", err)
}
msg := &broker.Message{
@@ -82,52 +82,54 @@ func sub(be *testing.B, c int) {
done := make(chan bool, c)
for i := 0; i < c; i++ {
sub, err := b.Subscribe(topic, func(p broker.Event) error {
sub, err := brker.Subscribe(topic, func(p broker.Event) error {
done <- true
m := p.Message()
if string(m.Body) != string(msg.Body) {
be.Fatalf("Unexpected msg %s, expected %s", string(m.Body), string(msg.Body))
b.Fatalf("Unexpected msg %s, expected %s", string(m.Body), string(msg.Body))
}
return nil
}, broker.Queue("shared"))
if err != nil {
be.Fatalf("Unexpected subscribe error: %v", err)
b.Fatalf("Unexpected subscribe error: %v", err)
}
subs = append(subs, sub)
}
for i := 0; i < be.N; i++ {
be.StartTimer()
if err := b.Publish(topic, msg); err != nil {
be.Fatalf("Unexpected publish error: %v", err)
for i := 0; i < b.N; i++ {
b.StartTimer()
if err := brker.Publish(topic, msg); err != nil {
b.Fatalf("Unexpected publish error: %v", err)
}
<-done
be.StopTimer()
b.StopTimer()
}
for _, sub := range subs {
sub.Unsubscribe()
if err := sub.Unsubscribe(); err != nil {
b.Fatalf("Unexpected unsubscribe error: %v", err)
}
}
if err := b.Disconnect(); err != nil {
be.Fatalf("Unexpected disconnect error: %v", err)
if err := brker.Disconnect(); err != nil {
b.Fatalf("Unexpected disconnect error: %v", err)
}
}
func pub(be *testing.B, c int) {
be.StopTimer()
func pub(b *testing.B, c int) {
b.StopTimer()
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
brk := broker.NewHttpBroker(broker.Registry(m))
topic := uuid.New().String()
if err := b.Init(); err != nil {
be.Fatalf("Unexpected init error: %v", err)
if err := brk.Init(); err != nil {
b.Fatalf("Unexpected init error: %v", err)
}
if err := b.Connect(); err != nil {
be.Fatalf("Unexpected connect error: %v", err)
if err := brk.Connect(); err != nil {
b.Fatalf("Unexpected connect error: %v", err)
}
msg := &broker.Message{
@@ -139,27 +141,27 @@ func pub(be *testing.B, c int) {
done := make(chan bool, c*4)
sub, err := b.Subscribe(topic, func(p broker.Event) error {
sub, err := brk.Subscribe(topic, func(p broker.Event) error {
done <- true
m := p.Message()
if string(m.Body) != string(msg.Body) {
be.Fatalf("Unexpected msg %s, expected %s", string(m.Body), string(msg.Body))
b.Fatalf("Unexpected msg %s, expected %s", string(m.Body), string(msg.Body))
}
return nil
}, broker.Queue("shared"))
if err != nil {
be.Fatalf("Unexpected subscribe error: %v", err)
b.Fatalf("Unexpected subscribe error: %v", err)
}
var wg sync.WaitGroup
ch := make(chan int, c*4)
be.StartTimer()
b.StartTimer()
for i := 0; i < c; i++ {
go func() {
for range ch {
if err := b.Publish(topic, msg); err != nil {
be.Fatalf("Unexpected publish error: %v", err)
if err := brk.Publish(topic, msg); err != nil {
b.Fatalf("Unexpected publish error: %v", err)
}
select {
case <-done:
@@ -170,25 +172,25 @@ func pub(be *testing.B, c int) {
}()
}
for i := 0; i < be.N; i++ {
for i := 0; i < b.N; i++ {
wg.Add(1)
ch <- i
}
wg.Wait()
be.StopTimer()
b.StopTimer()
sub.Unsubscribe()
close(ch)
close(done)
if err := b.Disconnect(); err != nil {
be.Fatalf("Unexpected disconnect error: %v", err)
if err := brk.Disconnect(); err != nil {
b.Fatalf("Unexpected disconnect error: %v", err)
}
}
func TestBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -226,7 +228,9 @@ func TestBroker(t *testing.T) {
}
<-done
sub.Unsubscribe()
if err := sub.Unsubscribe(); err != nil {
t.Fatalf("Unexpected unsubscribe error: %v", err)
}
if err := b.Disconnect(); err != nil {
t.Fatalf("Unexpected disconnect error: %v", err)
@@ -235,7 +239,7 @@ func TestBroker(t *testing.T) {
func TestConcurrentSubBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -282,7 +286,9 @@ func TestConcurrentSubBroker(t *testing.T) {
wg.Wait()
for _, sub := range subs {
sub.Unsubscribe()
if err := sub.Unsubscribe(); err != nil {
t.Fatalf("Unexpected unsubscribe error: %v", err)
}
}
if err := b.Disconnect(); err != nil {
@@ -292,7 +298,7 @@ func TestConcurrentSubBroker(t *testing.T) {
func TestConcurrentPubBroker(t *testing.T) {
m := newTestRegistry()
b := broker.NewBroker(broker.Registry(m))
b := broker.NewHttpBroker(broker.Registry(m))
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error: %v", err)
@@ -336,7 +342,9 @@ func TestConcurrentPubBroker(t *testing.T) {
wg.Wait()
sub.Unsubscribe()
if err := sub.Unsubscribe(); err != nil {
t.Fatalf("Unexpected unsubscribe error: %v", err)
}
if err := b.Disconnect(); err != nil {
t.Fatalf("Unexpected disconnect error: %v", err)
@@ -354,14 +362,6 @@ func BenchmarkSub32(b *testing.B) {
sub(b, 32)
}
func BenchmarkSub64(b *testing.B) {
sub(b, 64)
}
func BenchmarkSub128(b *testing.B) {
sub(b, 128)
}
func BenchmarkPub1(b *testing.B) {
pub(b, 1)
}
@@ -373,11 +373,3 @@ func BenchmarkPub8(b *testing.B) {
func BenchmarkPub32(b *testing.B) {
pub(b, 32)
}
func BenchmarkPub64(b *testing.B) {
pub(b, 64)
}
func BenchmarkPub128(b *testing.B) {
pub(b, 128)
}
@@ -1,51 +1,44 @@
// Package memory provides a memory broker
package memory
package broker
import (
"context"
"errors"
"math/rand"
"sync"
"time"
"github.com/google/uuid"
"go-micro.dev/v4/cmd"
"go-micro.dev/v4/broker"
"go-micro.dev/v4/logger"
maddr "go-micro.dev/v4/util/addr"
mnet "go-micro.dev/v4/util/net"
log "go-micro.dev/v5/logger"
maddr "go-micro.dev/v5/util/addr"
mnet "go-micro.dev/v5/util/net"
)
func init() {
cmd.DefaultBrokers["memory"] = NewBroker
}
type memoryBroker struct {
opts broker.Options
opts *Options
Subscribers map[string][]*memorySubscriber
addr string
sync.RWMutex
connected bool
Subscribers map[string][]*memorySubscriber
connected bool
}
type memoryEvent struct {
opts broker.Options
topic string
err error
message interface{}
opts *Options
topic string
}
type memorySubscriber struct {
opts SubscribeOptions
exit chan bool
handler Handler
id string
topic string
exit chan bool
handler broker.Handler
opts broker.SubscribeOptions
}
func (m *memoryBroker) Options() broker.Options {
return m.opts
func (m *memoryBroker) Options() Options {
return *m.opts
}
func (m *memoryBroker) Address() string {
@@ -88,14 +81,14 @@ func (m *memoryBroker) Disconnect() error {
return nil
}
func (m *memoryBroker) Init(opts ...broker.Option) error {
func (m *memoryBroker) Init(opts ...Option) error {
for _, o := range opts {
o(&m.opts)
o(m.opts)
}
return nil
}
func (m *memoryBroker) Publish(topic string, msg *broker.Message, opts ...broker.PublishOption) error {
func (m *memoryBroker) Publish(topic string, msg *Message, opts ...PublishOption) error {
m.RLock()
if !m.connected {
m.RUnlock()
@@ -139,7 +132,7 @@ func (m *memoryBroker) Publish(topic string, msg *broker.Message, opts ...broker
return nil
}
func (m *memoryBroker) Subscribe(topic string, handler broker.Handler, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
func (m *memoryBroker) Subscribe(topic string, handler Handler, opts ...SubscribeOption) (Subscriber, error) {
m.RLock()
if !m.connected {
m.RUnlock()
@@ -147,7 +140,7 @@ func (m *memoryBroker) Subscribe(topic string, handler broker.Handler, opts ...b
}
m.RUnlock()
var options broker.SubscribeOptions
var options SubscribeOptions
for _, o := range opts {
o(&options)
}
@@ -189,16 +182,14 @@ func (m *memoryEvent) Topic() string {
return m.topic
}
func (m *memoryEvent) Message() *broker.Message {
func (m *memoryEvent) Message() *Message {
switch v := m.message.(type) {
case *broker.Message:
case *Message:
return v
case []byte:
msg := &broker.Message{}
msg := &Message{}
if err := m.opts.Codec.Unmarshal(v, msg); err != nil {
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("[memory]: failed to unmarshal: %v\n", err)
}
m.opts.Logger.Logf(log.ErrorLevel, "[memory]: failed to unmarshal: %v\n", err)
return nil
}
return msg
@@ -215,7 +206,7 @@ func (m *memoryEvent) Error() error {
return m.err
}
func (m *memorySubscriber) Options() broker.SubscribeOptions {
func (m *memorySubscriber) Options() SubscribeOptions {
return m.opts
}
@@ -228,15 +219,9 @@ func (m *memorySubscriber) Unsubscribe() error {
return nil
}
func NewBroker(opts ...broker.Option) broker.Broker {
options := broker.Options{
Context: context.Background(),
}
func NewMemoryBroker(opts ...Option) Broker {
options := NewOptions(opts...)
rand.Seed(time.Now().UnixNano())
for _, o := range opts {
o(&options)
}
return &memoryBroker{
opts: options,
@@ -1,14 +1,14 @@
package memory
package broker_test
import (
"fmt"
"testing"
"go-micro.dev/v4/broker"
"go-micro.dev/v5/broker"
)
func TestMemoryBroker(t *testing.T) {
b := NewBroker()
b := broker.NewMemoryBroker()
if err := b.Connect(); err != nil {
t.Fatalf("Unexpected connect error %v", err)
@@ -3,10 +3,10 @@ package nats
import (
"context"
"go-micro.dev/v4/broker"
"go-micro.dev/v5/broker"
)
// setBrokerOption returns a function to setup a context with given value
// setBrokerOption returns a function to setup a context with given value.
func setBrokerOption(k, v interface{}) broker.Option {
return func(o *broker.Options) {
if o.Context == nil {
@@ -7,18 +7,13 @@ import (
"strings"
"sync"
"go-micro.dev/v4/broker"
"go-micro.dev/v4/cmd"
"go-micro.dev/v4/codec/json"
"go-micro.dev/v4/logger"
"go-micro.dev/v4/registry"
nats "github.com/nats-io/nats.go"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/codec/json"
"go-micro.dev/v5/logger"
"go-micro.dev/v5/registry"
)
func init() {
cmd.DefaultBrokers["nats"] = NewBroker
}
type natsBroker struct {
sync.Once
sync.RWMutex
@@ -27,9 +22,9 @@ type natsBroker struct {
connected bool
addrs []string
conn *nats.Conn
conn *natsp.Conn
opts broker.Options
nopts nats.Options
nopts natsp.Options
// should we drain the connection
drain bool
@@ -37,7 +32,7 @@ type natsBroker struct {
}
type subscriber struct {
s *nats.Subscription
s *natsp.Subscription
opts broker.SubscribeOptions
}
@@ -101,7 +96,7 @@ func (n *natsBroker) setAddrs(addrs []string) []string {
cAddrs = append(cAddrs, addr)
}
if len(cAddrs) == 0 {
cAddrs = []string{nats.DefaultURL}
cAddrs = []string{natsp.DefaultURL}
}
return cAddrs
}
@@ -114,13 +109,13 @@ func (n *natsBroker) Connect() error {
return nil
}
status := nats.CLOSED
status := natsp.CLOSED
if n.conn != nil {
status = n.conn.Status()
}
switch status {
case nats.CONNECTED, nats.RECONNECTING, nats.CONNECTING:
case natsp.CONNECTED, natsp.RECONNECTING, natsp.CONNECTING:
n.connected = true
return nil
default: // DISCONNECTED or CLOSED or DRAINING
@@ -204,7 +199,7 @@ func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...bro
o(&opt)
}
fn := func(msg *nats.Msg) {
fn := func(msg *natsp.Msg) {
var m broker.Message
pub := &publication{t: msg.Subject}
eh := n.opts.ErrorHandler
@@ -213,9 +208,7 @@ func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...bro
pub.m = &m
if err != nil {
m.Body = msg.Data
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
n.opts.Logger.Log(logger.ErrorLevel, err)
if eh != nil {
eh(pub)
}
@@ -223,16 +216,14 @@ func (n *natsBroker) Subscribe(topic string, handler broker.Handler, opts ...bro
}
if err := handler(pub); err != nil {
pub.err = err
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
n.opts.Logger.Log(logger.ErrorLevel, err)
if eh != nil {
eh(pub)
}
}
}
var sub *nats.Subscription
var sub *natsp.Subscription
var err error
n.RLock()
@@ -258,10 +249,10 @@ func (n *natsBroker) setOption(opts ...broker.Option) {
}
n.Once.Do(func() {
n.nopts = nats.GetDefaultOptions()
n.nopts = natsp.GetDefaultOptions()
})
if nopts, ok := n.opts.Context.Value(optionsKey{}).(nats.Options); ok {
if nopts, ok := n.opts.Context.Value(optionsKey{}).(natsp.Options); ok {
n.nopts = nopts
}
@@ -290,28 +281,29 @@ func (n *natsBroker) setOption(opts ...broker.Option) {
}
}
func (n *natsBroker) onClose(conn *nats.Conn) {
func (n *natsBroker) onClose(conn *natsp.Conn) {
n.closeCh <- nil
}
func (n *natsBroker) onAsyncError(conn *nats.Conn, sub *nats.Subscription, err error) {
func (n *natsBroker) onAsyncError(conn *natsp.Conn, sub *natsp.Subscription, err error) {
// There are kinds of different async error nats might callback, but we are interested
// in ErrDrainTimeout only here.
if err == nats.ErrDrainTimeout {
if err == natsp.ErrDrainTimeout {
n.closeCh <- err
}
}
func (n *natsBroker) onDisconnectedError(conn *nats.Conn, err error) {
func (n *natsBroker) onDisconnectedError(conn *natsp.Conn, err error) {
n.closeCh <- err
}
func NewBroker(opts ...broker.Option) broker.Broker {
func NewNatsBroker(opts ...broker.Option) broker.Broker {
options := broker.Options{
// Default codec
Codec: json.Marshaler{},
Context: context.Background(),
Registry: registry.DefaultRegistry,
Logger: logger.DefaultLogger,
}
n := &natsBroker{
@@ -4,8 +4,8 @@ import (
"fmt"
"testing"
"go-micro.dev/v4/broker"
nats "github.com/nats-io/nats.go"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
)
var addrTestCases = []struct {
@@ -45,10 +45,8 @@ var addrTestCases = []struct {
// TestInitAddrs tests issue #100. Ensures that if the addrs is set by an option in init it will be used.
func TestInitAddrs(t *testing.T) {
for _, tc := range addrTestCases {
t.Run(fmt.Sprintf("%s: %s", tc.name, tc.description), func(t *testing.T) {
var br broker.Broker
var addrs []string
@@ -59,19 +57,19 @@ func TestInitAddrs(t *testing.T) {
switch tc.name {
case "brokerOpts":
// we know that there are just two addrs in the dict
br = NewBroker(broker.Addrs(addrs[0], addrs[1]))
br = NewNatsBroker(broker.Addrs(addrs[0], addrs[1]))
br.Init()
case "brokerInit":
br = NewBroker()
br = NewNatsBroker()
// we know that there are just two addrs in the dict
br.Init(broker.Addrs(addrs[0], addrs[1]))
case "natsOpts":
nopts := nats.GetDefaultOptions()
nopts := natsp.GetDefaultOptions()
nopts.Servers = addrs
br = NewBroker(Options(nopts))
br = NewNatsBroker(Options(nopts))
br.Init()
case "default":
br = NewBroker()
br = NewNatsBroker()
br.Init()
}
@@ -1,19 +1,19 @@
package nats
import (
"go-micro.dev/v4/broker"
nats "github.com/nats-io/nats.go"
natsp "github.com/nats-io/nats.go"
"go-micro.dev/v5/broker"
)
type optionsKey struct{}
type drainConnectionKey struct{}
// Options accepts nats.Options
func Options(opts nats.Options) broker.Option {
// Options accepts nats.Options.
func Options(opts natsp.Options) broker.Option {
return setBrokerOption(optionsKey{}, opts)
}
// DrainConnection will drain subscription on close
// DrainConnection will drain subscription on close.
func DrainConnection() broker.Option {
return setBrokerOption(drainConnectionKey{}, struct{}{})
}
+50 -24
View File
@@ -4,25 +4,30 @@ import (
"context"
"crypto/tls"
"go-micro.dev/v4/codec"
"go-micro.dev/v4/registry"
"go-micro.dev/v5/codec"
"go-micro.dev/v5/logger"
"go-micro.dev/v5/registry"
)
type Options struct {
Addrs []string
Secure bool
Codec codec.Marshaler
Codec codec.Marshaler
// Logger is the underlying logger
Logger logger.Logger
// Registry used for clustering
Registry registry.Registry
// Other options for implementations of the interface
// can be stored in a context
Context context.Context
// Handler executed when error happens in broker mesage
// processing
ErrorHandler Handler
TLSConfig *tls.Config
// Registry used for clustering
Registry registry.Registry
// Other options for implementations of the interface
// can be stored in a context
Context context.Context
Addrs []string
Secure bool
}
type PublishOptions struct {
@@ -32,24 +37,25 @@ type PublishOptions struct {
}
type SubscribeOptions struct {
// AutoAck defaults to true. When a handler returns
// with a nil error the message is acked.
AutoAck bool
// Other options for implementations of the interface
// can be stored in a context
Context context.Context
// Subscribers with the same queue name
// will create a shared subscription where each
// receives a subset of messages.
Queue string
// Other options for implementations of the interface
// can be stored in a context
Context context.Context
// AutoAck defaults to true. When a handler returns
// with a nil error the message is acked.
AutoAck bool
}
type Option func(*Options)
type PublishOption func(*PublishOptions)
// PublishContext set context
// PublishContext set context.
func PublishContext(ctx context.Context) PublishOption {
return func(o *PublishOptions) {
o.Context = ctx
@@ -58,6 +64,19 @@ func PublishContext(ctx context.Context) PublishOption {
type SubscribeOption func(*SubscribeOptions)
func NewOptions(opts ...Option) *Options {
options := Options{
Context: context.Background(),
Logger: logger.DefaultLogger,
}
for _, o := range opts {
o(&options)
}
return &options
}
func NewSubscribeOptions(opts ...SubscribeOption) SubscribeOptions {
opt := SubscribeOptions{
AutoAck: true,
@@ -70,7 +89,7 @@ func NewSubscribeOptions(opts ...SubscribeOption) SubscribeOptions {
return opt
}
// Addrs sets the host addresses to be used by the broker
// Addrs sets the host addresses to be used by the broker.
func Addrs(addrs ...string) Option {
return func(o *Options) {
o.Addrs = addrs
@@ -78,7 +97,7 @@ func Addrs(addrs ...string) Option {
}
// Codec sets the codec used for encoding/decoding used where
// a broker does not support headers
// a broker does not support headers.
func Codec(c codec.Marshaler) Option {
return func(o *Options) {
o.Codec = c
@@ -94,14 +113,14 @@ func DisableAutoAck() SubscribeOption {
}
// ErrorHandler will catch all broker errors that cant be handled
// in normal way, for example Codec errors
// in normal way, for example Codec errors.
func ErrorHandler(h Handler) Option {
return func(o *Options) {
o.ErrorHandler = h
}
}
// Queue sets the name of the queue to share messages on
// Queue sets the name of the queue to share messages on.
func Queue(name string) SubscribeOption {
return func(o *SubscribeOptions) {
o.Queue = name
@@ -114,21 +133,28 @@ func Registry(r registry.Registry) Option {
}
}
// Secure communication with the broker
// Secure communication with the broker.
func Secure(b bool) Option {
return func(o *Options) {
o.Secure = b
}
}
// Specify TLS Config
// Specify TLS Config.
func TLSConfig(t *tls.Config) Option {
return func(o *Options) {
o.TLSConfig = t
}
}
// SubscribeContext set context
// Logger sets the underline logger.
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// SubscribeContext set context.
func SubscribeContext(ctx context.Context) SubscribeOption {
return func(o *SubscribeOptions) {
o.Context = ctx
@@ -6,18 +6,21 @@ package rabbitmq
import (
"errors"
"sync"
"github.com/google/uuid"
"github.com/streadway/amqp"
amqp "github.com/rabbitmq/amqp091-go"
)
type rabbitMQChannel struct {
uuid string
connection *amqp.Connection
channel *amqp.Channel
uuid string
connection *amqp.Connection
channel *amqp.Channel
confirmPublish chan amqp.Confirmation
mtx sync.Mutex
}
func newRabbitChannel(conn *amqp.Connection, prefetchCount int, prefetchGlobal bool) (*rabbitMQChannel, error) {
func newRabbitChannel(conn *amqp.Connection, prefetchCount int, prefetchGlobal bool, confirmPublish bool) (*rabbitMQChannel, error) {
id, err := uuid.NewRandom()
if err != nil {
return nil, err
@@ -26,23 +29,33 @@ func newRabbitChannel(conn *amqp.Connection, prefetchCount int, prefetchGlobal b
uuid: id.String(),
connection: conn,
}
if err := rabbitCh.Connect(prefetchCount, prefetchGlobal); err != nil {
if err := rabbitCh.Connect(prefetchCount, prefetchGlobal, confirmPublish); err != nil {
return nil, err
}
return rabbitCh, nil
}
func (r *rabbitMQChannel) Connect(prefetchCount int, prefetchGlobal bool) error {
func (r *rabbitMQChannel) Connect(prefetchCount int, prefetchGlobal bool, confirmPublish bool) error {
var err error
r.channel, err = r.connection.Channel()
if err != nil {
return err
}
err = r.channel.Qos(prefetchCount, 0, prefetchGlobal)
if err != nil {
return err
}
if confirmPublish {
r.confirmPublish = r.channel.NotifyPublish(make(chan amqp.Confirmation, 1))
err = r.channel.Confirm(false)
if err != nil {
return err
}
}
return nil
}
@@ -57,30 +70,52 @@ func (r *rabbitMQChannel) Publish(exchange, key string, message amqp.Publishing)
if r.channel == nil {
return errors.New("Channel is nil")
}
return r.channel.Publish(exchange, key, false, false, message)
if r.confirmPublish != nil {
r.mtx.Lock()
defer r.mtx.Unlock()
}
err := r.channel.Publish(exchange, key, false, false, message)
if err != nil {
return err
}
if r.confirmPublish != nil {
confirmation, ok := <-r.confirmPublish
if !ok {
return errors.New("Channel closed before could receive confirmation of publish")
}
if !confirmation.Ack {
return errors.New("Could not publish message, received nack from broker on confirmation")
}
}
return nil
}
func (r *rabbitMQChannel) DeclareExchange(exchange string) error {
func (r *rabbitMQChannel) DeclareExchange(ex Exchange) error {
return r.channel.ExchangeDeclare(
exchange, // name
"topic", // kind
false, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
ex.Name, // name
string(ex.Type), // kind
ex.Durable, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
)
}
func (r *rabbitMQChannel) DeclareDurableExchange(exchange string) error {
func (r *rabbitMQChannel) DeclareDurableExchange(ex Exchange) error {
return r.channel.ExchangeDeclare(
exchange, // name
"topic", // kind
true, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
ex.Name, // name
string(ex.Type), // kind
true, // durable
false, // autoDelete
false, // internal
false, // noWait
nil, // args
)
}
@@ -5,28 +5,39 @@ package rabbitmq
//
import (
"crypto/tls"
"regexp"
"strings"
"sync"
"time"
"go-micro.dev/v4/logger"
"github.com/streadway/amqp"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/logger"
mtls "go-micro.dev/v5/util/tls"
)
type MQExchangeType string
const (
ExchangeTypeFanout MQExchangeType = "fanout"
ExchangeTypeTopic = "topic"
ExchangeTypeDirect = "direct"
)
var (
DefaultExchange = Exchange{
Name: "micro",
Type: ExchangeTypeTopic,
}
DefaultRabbitURL = "amqp://guest:guest@127.0.0.1:5672"
DefaultPrefetchCount = 0
DefaultPrefetchGlobal = false
DefaultRequeueOnError = false
DefaultRabbitURL = "amqp://guest:guest@127.0.0.1:5672"
DefaultPrefetchCount = 0
DefaultPrefetchGlobal = false
DefaultRequeueOnError = false
DefaultConfirmPublish = false
DefaultWithoutExchange = false
// The amqp library does not seem to set these when using amqp.DialConfig
// (even though it says so in the comments) so we set them manually to make
// sure to not brake any existing functionality
// sure to not brake any existing functionality.
defaultHeartbeat = 10 * time.Second
defaultLocale = "en_US"
@@ -45,26 +56,32 @@ type rabbitMQConn struct {
Channel *rabbitMQChannel
ExchangeChannel *rabbitMQChannel
exchange Exchange
withoutExchange bool
url string
prefetchCount int
prefetchGlobal bool
confirmPublish bool
sync.Mutex
connected bool
close chan bool
waitConnection chan struct{}
logger logger.Logger
}
// Exchange is the rabbitmq exchange
// Exchange is the rabbitmq exchange.
type Exchange struct {
// Name of the exchange
Name string
// Type of the exchange
Type MQExchangeType
// Whether its persistent
Durable bool
}
func newRabbitMQConn(ex Exchange, urls []string, prefetchCount int, prefetchGlobal bool) *rabbitMQConn {
func newRabbitMQConn(ex Exchange, urls []string, prefetchCount int, prefetchGlobal bool, confirmPublish bool, withoutExchange bool, logger logger.Logger) *rabbitMQConn {
var url string
if len(urls) > 0 && regexp.MustCompile("^amqp(s)?://.*").MatchString(urls[0]) {
@@ -74,12 +91,15 @@ func newRabbitMQConn(ex Exchange, urls []string, prefetchCount int, prefetchGlob
}
ret := &rabbitMQConn{
exchange: ex,
url: url,
prefetchCount: prefetchCount,
prefetchGlobal: prefetchGlobal,
close: make(chan bool),
waitConnection: make(chan struct{}),
exchange: ex,
url: url,
withoutExchange: withoutExchange,
prefetchCount: prefetchCount,
prefetchGlobal: prefetchGlobal,
confirmPublish: confirmPublish,
close: make(chan bool),
waitConnection: make(chan struct{}),
logger: logger,
}
// its bad case of nil == waitConnection, so close it at start
close(ret.waitConnection)
@@ -118,7 +138,7 @@ func (r *rabbitMQConn) reconnect(secure bool, config *amqp.Config) {
r.Lock()
r.connected = true
r.Unlock()
//unblock resubscribe cycle - close channel
// unblock resubscribe cycle - close channel
//at this point channel is created and unclosed - close it without any additional checks
close(r.waitConnection)
}
@@ -127,44 +147,38 @@ func (r *rabbitMQConn) reconnect(secure bool, config *amqp.Config) {
notifyClose := make(chan *amqp.Error)
r.Connection.NotifyClose(notifyClose)
chanNotifyClose := make(chan *amqp.Error)
channel := r.ExchangeChannel.channel
var channel *amqp.Channel
if !r.withoutExchange {
channel = r.ExchangeChannel.channel
} else {
channel = r.Channel.channel
}
channel.NotifyClose(chanNotifyClose)
channelNotifyReturn := make(chan amqp.Return)
channel.NotifyReturn(channelNotifyReturn)
// block until closed
select {
case result, ok := <-channelNotifyReturn:
if !ok {
// Channel closed, probably also the channel or connection.
// To avoid deadlocks it is necessary to consume the messages from all channels.
for notifyClose != nil || chanNotifyClose != nil {
// block until closed
select {
case err := <-chanNotifyClose:
r.logger.Log(logger.ErrorLevel, err)
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
chanNotifyClose = nil
case err := <-notifyClose:
r.logger.Log(logger.ErrorLevel, err)
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
notifyClose = nil
case <-r.close:
return
}
// Do what you need with messageFailing.
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Errorf("notify error reason: %s, description: %s", result.ReplyText, result.Exchange)
}
case err := <-chanNotifyClose:
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
case err := <-notifyClose:
if logger.V(logger.ErrorLevel, logger.DefaultLogger) {
logger.Error(err)
}
// block all resubscribe attempt - they are useless because there is no connection to rabbitmq
// create channel 'waitConnection' (at this point channel is nil or closed, create it without unnecessary checks)
r.Lock()
r.connected = false
r.waitConnection = make(chan struct{})
r.Unlock()
case <-r.close:
return
}
}
}
@@ -218,9 +232,8 @@ func (r *rabbitMQConn) tryConnect(secure bool, config *amqp.Config) error {
if secure || config.TLSClientConfig != nil || strings.HasPrefix(r.url, "amqps://") {
if config.TLSClientConfig == nil {
config.TLSClientConfig = &tls.Config{
InsecureSkipVerify: true,
}
// Use environment-based config - secure by default
config.TLSClientConfig = mtls.Config()
}
url = strings.Replace(r.url, "amqp://", "amqps://", 1)
@@ -232,22 +245,23 @@ func (r *rabbitMQConn) tryConnect(secure bool, config *amqp.Config) error {
return err
}
if r.Channel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal); err != nil {
if r.Channel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish); err != nil {
return err
}
if r.exchange.Durable {
r.Channel.DeclareDurableExchange(r.exchange.Name)
} else {
r.Channel.DeclareExchange(r.exchange.Name)
if !r.withoutExchange {
if r.exchange.Durable {
r.Channel.DeclareDurableExchange(r.exchange)
} else {
r.Channel.DeclareExchange(r.exchange)
}
r.ExchangeChannel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish)
}
r.ExchangeChannel, err = newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal)
return err
}
func (r *rabbitMQConn) Consume(queue, key string, headers amqp.Table, qArgs amqp.Table, autoAck, durableQueue bool) (*rabbitMQChannel, <-chan amqp.Delivery, error) {
consumerChannel, err := newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal)
consumerChannel, err := newRabbitChannel(r.Connection, r.prefetchCount, r.prefetchGlobal, r.confirmPublish)
if err != nil {
return nil, nil, err
}
@@ -267,14 +281,19 @@ func (r *rabbitMQConn) Consume(queue, key string, headers amqp.Table, qArgs amqp
return nil, nil, err
}
err = consumerChannel.BindQueue(queue, key, r.exchange.Name, headers)
if err != nil {
return nil, nil, err
if !r.withoutExchange {
err = consumerChannel.BindQueue(queue, key, r.exchange.Name, headers)
if err != nil {
return nil, nil, err
}
}
return consumerChannel, deliveries, nil
}
func (r *rabbitMQConn) Publish(exchange, key string, msg amqp.Publishing) error {
if r.withoutExchange {
return r.Channel.Publish("", key, msg)
}
return r.ExchangeChannel.Publish(exchange, key, msg)
}
@@ -5,7 +5,8 @@ import (
"errors"
"testing"
"github.com/streadway/amqp"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/logger"
)
func TestNewRabbitMQConnURL(t *testing.T) {
@@ -22,7 +23,7 @@ func TestNewRabbitMQConnURL(t *testing.T) {
}
for _, test := range testcases {
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, 0, false)
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, 0, false, false, false, logger.DefaultLogger)
if have, want := conn.url, test.want; have != want {
t.Errorf("%s: invalid url, want %q, have %q", test.title, want, have)
@@ -38,7 +39,6 @@ func TestTryToConnectTLS(t *testing.T) {
)
dialConfig = func(_ string, c amqp.Config) (*amqp.Connection, error) {
if c.TLSClientConfig != nil {
dialTLSCount++
return nil, err
@@ -64,7 +64,7 @@ func TestTryToConnectTLS(t *testing.T) {
for _, test := range testcases {
dialCount, dialTLSCount = 0, 0
conn := newRabbitMQConn(Exchange{Name: "exchange"}, []string{test.url}, 0, false)
conn := newRabbitMQConn(Exchange{Name: "exchange"}, []string{test.url}, 0, false, false, false, logger.DefaultLogger)
conn.tryConnect(test.secure, test.amqpConfig)
have := dialCount
@@ -78,22 +78,23 @@ func TestTryToConnectTLS(t *testing.T) {
}
}
func TestNewRabbitMQPrefetch(t *testing.T) {
func TestNewRabbitMQPrefetchConfirmPublish(t *testing.T) {
testcases := []struct {
title string
urls []string
prefetchCount int
prefetchGlobal bool
confirmPublish bool
}{
{"Multiple URLs", []string{"amqp://example.com/one", "amqp://example.com/two"}, 1, true},
{"Insecure URL", []string{"amqp://example.com"}, 1, true},
{"Secure URL", []string{"amqps://example.com"}, 1, true},
{"Invalid URL", []string{"http://example.com"}, 1, true},
{"No URLs", []string{}, 1, true},
{"Multiple URLs", []string{"amqp://example.com/one", "amqp://example.com/two"}, 1, true, true},
{"Insecure URL", []string{"amqp://example.com"}, 1, true, true},
{"Secure URL", []string{"amqps://example.com"}, 1, true, true},
{"Invalid URL", []string{"http://example.com"}, 1, true, true},
{"No URLs", []string{}, 1, true, true},
}
for _, test := range testcases {
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, test.prefetchCount, test.prefetchGlobal)
conn := newRabbitMQConn(Exchange{Name: "exchange"}, test.urls, test.prefetchCount, test.prefetchGlobal, test.confirmPublish, false, logger.DefaultLogger)
if have, want := conn.prefetchCount, test.prefetchCount; have != want {
t.Errorf("%s: invalid prefetch count, want %d, have %d", test.title, want, have)
@@ -102,5 +103,9 @@ func TestNewRabbitMQPrefetch(t *testing.T) {
if have, want := conn.prefetchGlobal, test.prefetchGlobal; have != want {
t.Errorf("%s: invalid prefetch global setting, want %t, have %t", test.title, want, have)
}
if have, want := conn.confirmPublish, test.confirmPublish; have != want {
t.Errorf("%s: invalid confirm setting, want %t, have %t", test.title, want, have)
}
}
}
@@ -3,11 +3,11 @@ package rabbitmq
import (
"context"
"go-micro.dev/v4/broker"
"go-micro.dev/v4/server"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/server"
)
// setSubscribeOption returns a function to setup a context with given value
// setSubscribeOption returns a function to setup a context with given value.
func setSubscribeOption(k, v interface{}) broker.SubscribeOption {
return func(o *broker.SubscribeOptions) {
if o.Context == nil {
@@ -17,7 +17,7 @@ func setSubscribeOption(k, v interface{}) broker.SubscribeOption {
}
}
// setBrokerOption returns a function to setup a context with given value
// setBrokerOption returns a function to setup a context with given value.
func setBrokerOption(k, v interface{}) broker.Option {
return func(o *broker.Options) {
if o.Context == nil {
@@ -27,7 +27,7 @@ func setBrokerOption(k, v interface{}) broker.Option {
}
}
// setBrokerOption returns a function to setup a context with given value
// setBrokerOption returns a function to setup a context with given value.
func setServerSubscriberOption(k, v interface{}) server.SubscriberOption {
return func(o *server.SubscriberOptions) {
if o.Context == nil {
@@ -37,7 +37,7 @@ func setServerSubscriberOption(k, v interface{}) server.SubscriberOption {
}
}
// setPublishOption returns a function to setup a context with given value
// setPublishOption returns a function to setup a context with given value.
func setPublishOption(k, v interface{}) broker.PublishOption {
return func(o *broker.PublishOptions) {
if o.Context == nil {
@@ -4,7 +4,9 @@ import (
"context"
"time"
"go-micro.dev/v4/broker"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/client"
"go-micro.dev/v5/server"
)
type durableQueueKey struct{}
@@ -12,7 +14,10 @@ type headersKey struct{}
type queueArgumentsKey struct{}
type prefetchCountKey struct{}
type prefetchGlobalKey struct{}
type confirmPublishKey struct{}
type exchangeKey struct{}
type exchangeTypeKey struct{}
type withoutExchangeKey struct{}
type requeueOnErrorKey struct{}
type deliveryMode struct{}
type priorityKey struct{}
@@ -29,36 +34,56 @@ type appID struct{}
type externalAuth struct{}
type durableExchange struct{}
// ServerDurableQueue provide durable queue option for micro.RegisterSubscriber
func ServerDurableQueue() server.SubscriberOption {
return setServerSubscriberOption(durableQueueKey{}, true)
}
// ServerAckOnSuccess export AckOnSuccess server.SubscriberOption
func ServerAckOnSuccess() server.SubscriberOption {
return setServerSubscriberOption(ackSuccessKey{}, true)
}
// DurableQueue creates a durable queue when subscribing.
func DurableQueue() broker.SubscribeOption {
return setSubscribeOption(durableQueueKey{}, true)
}
// DurableExchange is an option to set the Exchange to be durable
// DurableExchange is an option to set the Exchange to be durable.
func DurableExchange() broker.Option {
return setBrokerOption(durableExchange{}, true)
}
// Headers adds headers used by the headers exchange
// Headers adds headers used by the headers exchange.
func Headers(h map[string]interface{}) broker.SubscribeOption {
return setSubscribeOption(headersKey{}, h)
}
// QueueArguments sets arguments for queue creation
// QueueArguments sets arguments for queue creation.
func QueueArguments(h map[string]interface{}) broker.SubscribeOption {
return setSubscribeOption(queueArgumentsKey{}, h)
}
// RequeueOnError calls Nack(muliple:false, requeue:true) on amqp delivery when handler returns error
func RequeueOnError() broker.SubscribeOption {
return setSubscribeOption(requeueOnErrorKey{}, true)
}
// ExchangeName is an option to set the ExchangeName
// ExchangeName is an option to set the ExchangeName.
func ExchangeName(e string) broker.Option {
return setBrokerOption(exchangeKey{}, e)
}
// WithoutExchange is an option to use the rabbitmq default exchange.
// means it would not create any custom exchange.
func WithoutExchange() broker.Option {
return setBrokerOption(withoutExchangeKey{}, true)
}
// ExchangeType is an option to set the rabbitmq exchange type.
func ExchangeType(t MQExchangeType) broker.Option {
return setBrokerOption(exchangeTypeKey{}, t)
}
// PrefetchCount ...
func PrefetchCount(c int) broker.Option {
return setBrokerOption(prefetchCountKey{}, c)
@@ -69,62 +94,67 @@ func PrefetchGlobal() broker.Option {
return setBrokerOption(prefetchGlobalKey{}, true)
}
// DeliveryMode sets a delivery mode for publishing
// ConfirmPublish ensures all published messages are confirmed by waiting for an ack/nack from the broker.
func ConfirmPublish() broker.Option {
return setBrokerOption(confirmPublishKey{}, true)
}
// DeliveryMode sets a delivery mode for publishing.
func DeliveryMode(value uint8) broker.PublishOption {
return setPublishOption(deliveryMode{}, value)
}
// Priority sets a priority level for publishing
// Priority sets a priority level for publishing.
func Priority(value uint8) broker.PublishOption {
return setPublishOption(priorityKey{}, value)
}
// ContentType sets a property MIME content type for publishing
// ContentType sets a property MIME content type for publishing.
func ContentType(value string) broker.PublishOption {
return setPublishOption(contentType{}, value)
}
// ContentEncoding sets a property MIME content encoding for publishing
// ContentEncoding sets a property MIME content encoding for publishing.
func ContentEncoding(value string) broker.PublishOption {
return setPublishOption(contentEncoding{}, value)
}
// CorrelationID sets a property correlation ID for publishing
// CorrelationID sets a property correlation ID for publishing.
func CorrelationID(value string) broker.PublishOption {
return setPublishOption(correlationID{}, value)
}
// ReplyTo sets a property address to to reply to (ex: RPC) for publishing
// ReplyTo sets a property address to to reply to (ex: RPC) for publishing.
func ReplyTo(value string) broker.PublishOption {
return setPublishOption(replyTo{}, value)
}
// Expiration sets a property message expiration spec for publishing
// Expiration sets a property message expiration spec for publishing.
func Expiration(value string) broker.PublishOption {
return setPublishOption(expiration{}, value)
}
// MessageId sets a property message identifier for publishing
// MessageId sets a property message identifier for publishing.
func MessageId(value string) broker.PublishOption {
return setPublishOption(messageID{}, value)
}
// Timestamp sets a property message timestamp for publishing
// Timestamp sets a property message timestamp for publishing.
func Timestamp(value time.Time) broker.PublishOption {
return setPublishOption(timestamp{}, value)
}
// TypeMsg sets a property message type name for publishing
// TypeMsg sets a property message type name for publishing.
func TypeMsg(value string) broker.PublishOption {
return setPublishOption(typeMsg{}, value)
}
// UserID sets a property user id for publishing
// UserID sets a property user id for publishing.
func UserID(value string) broker.PublishOption {
return setPublishOption(userID{}, value)
}
// AppID sets a property application id for publishing
// AppID sets a property application id for publishing.
func AppID(value string) broker.PublishOption {
return setPublishOption(appID{}, value)
}
@@ -135,14 +165,25 @@ func ExternalAuth() broker.Option {
type subscribeContextKey struct{}
// SubscribeContext set the context for broker.SubscribeOption
// SubscribeContext set the context for broker.SubscribeOption.
func SubscribeContext(ctx context.Context) broker.SubscribeOption {
return setSubscribeOption(subscribeContextKey{}, ctx)
}
type ackSuccessKey struct{}
// AckOnSuccess will automatically acknowledge messages when no error is returned
// AckOnSuccess will automatically acknowledge messages when no error is returned.
func AckOnSuccess() broker.SubscribeOption {
return setSubscribeOption(ackSuccessKey{}, true)
}
// PublishDeliveryMode client.PublishOption for setting message "delivery mode"
// mode , Transient (0 or 1) or Persistent (2)
func PublishDeliveryMode(mode uint8) client.PublishOption {
return func(o *client.PublishOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, deliveryMode{}, mode)
}
}
@@ -4,12 +4,14 @@ package rabbitmq
import (
"context"
"errors"
"fmt"
"net/url"
"sync"
"time"
"go-micro.dev/v4/broker"
"go-micro.dev/v4/cmd"
"github.com/streadway/amqp"
amqp "github.com/rabbitmq/amqp091-go"
"go-micro.dev/v5/broker"
"go-micro.dev/v5/logger"
)
type rbroker struct {
@@ -24,7 +26,7 @@ type rbroker struct {
type subscriber struct {
mtx sync.Mutex
mayRun bool
unsub chan bool
opts broker.SubscribeOptions
topic string
ch *rabbitMQChannel
@@ -33,6 +35,7 @@ type subscriber struct {
r *rbroker
fn func(msg amqp.Delivery)
headers map[string]interface{}
wg sync.WaitGroup
}
type publication struct {
@@ -42,10 +45,6 @@ type publication struct {
err error
}
func init() {
cmd.DefaultBrokers["rabbitmq"] = NewBroker
}
func (p *publication) Ack() error {
return p.d.Ack(false)
}
@@ -71,9 +70,17 @@ func (s *subscriber) Topic() string {
}
func (s *subscriber) Unsubscribe() error {
s.unsub <- true
// Need to wait on subscriber to exit if autoack is disabled
// since closing the channel will prevent the ack/nack from
// being sent upon handler completion.
if !s.opts.AutoAck {
s.wg.Wait()
}
s.mtx.Lock()
defer s.mtx.Unlock()
s.mayRun = false
if s.ch != nil {
return s.ch.Close()
}
@@ -81,27 +88,29 @@ func (s *subscriber) Unsubscribe() error {
}
func (s *subscriber) resubscribe() {
s.wg.Add(1)
defer s.wg.Done()
minResubscribeDelay := 100 * time.Millisecond
maxResubscribeDelay := 30 * time.Second
expFactor := time.Duration(2)
reSubscribeDelay := minResubscribeDelay
//loop until unsubscribe
// loop until unsubscribe
for {
s.mtx.Lock()
mayRun := s.mayRun
s.mtx.Unlock()
if !mayRun {
// we are unsubscribed, showdown routine
return
}
select {
//check shutdown case
case <-s.r.conn.close:
//yep, its shutdown case
// unsubscribe case
case <-s.unsub:
return
//wait until we reconect to rabbit
// check shutdown case
case <-s.r.conn.close:
// yep, its shutdown case
return
// wait until we reconect to rabbit
case <-s.r.conn.waitConnection:
// When the connection is disconnected, the waitConnection will be re-assigned, so '<-s.r.conn.waitConnection' maybe blocked.
// Here, it returns once a second, and then the latest waitconnection will be used
case <-time.After(time.Second):
continue
}
// it may crash (panic) in case of Consume without connection, so recheck it
@@ -135,10 +144,20 @@ func (s *subscriber) resubscribe() {
reSubscribeDelay *= expFactor
continue
}
for d := range sub {
s.r.wg.Add(1)
s.fn(d)
s.r.wg.Done()
SubLoop:
for {
select {
case <-s.unsub:
return
case d, ok := <-sub:
if !ok {
break SubLoop
}
s.r.wg.Add(1)
s.fn(d)
s.r.wg.Done()
}
}
}
}
@@ -164,6 +183,7 @@ func (r *rbroker) Publish(topic string, msg *broker.Message, opts ...broker.Publ
}
if value, ok := options.Context.Value(contentType{}).(string); ok {
m.Headers["Content-Type"] = value
m.ContentType = value
}
@@ -202,13 +222,16 @@ func (r *rbroker) Publish(topic string, msg *broker.Message, opts ...broker.Publ
if value, ok := options.Context.Value(appID{}).(string); ok {
m.AppId = value
}
}
for k, v := range msg.Header {
m.Headers[k] = v
}
if r.getWithoutExchange() {
m.Headers["Micro-Topic"] = topic
}
if r.conn == nil {
return errors.New("connection is nil")
}
@@ -265,8 +288,15 @@ func (r *rbroker) Subscribe(topic string, handler broker.Handler, opts ...broker
fn := func(msg amqp.Delivery) {
header := make(map[string]string)
for k, v := range msg.Headers {
header[k], _ = v.(string)
header[k] = fmt.Sprintf("%v", v)
}
// Get rid of dependence on 'Micro-Topic'
msgTopic := header["Micro-Topic"]
if msgTopic == "" {
header["Micro-Topic"] = msg.RoutingKey
}
m := &broker.Message{
Header: header,
Body: msg.Body,
@@ -280,8 +310,9 @@ func (r *rbroker) Subscribe(topic string, handler broker.Handler, opts ...broker
}
}
sret := &subscriber{topic: topic, opts: opt, mayRun: true, r: r,
durableQueue: durableQueue, fn: fn, headers: headers, queueArgs: qArgs}
sret := &subscriber{topic: topic, opts: opt, unsub: make(chan bool), r: r,
durableQueue: durableQueue, fn: fn, headers: headers, queueArgs: qArgs,
wg: sync.WaitGroup{}}
go sret.resubscribe()
@@ -298,7 +329,12 @@ func (r *rbroker) String() string {
func (r *rbroker) Address() string {
if len(r.addrs) > 0 {
return r.addrs[0]
u, err := url.Parse(r.addrs[0])
if err != nil {
return ""
}
return u.Redacted()
}
return ""
}
@@ -313,7 +349,15 @@ func (r *rbroker) Init(opts ...broker.Option) error {
func (r *rbroker) Connect() error {
if r.conn == nil {
r.conn = newRabbitMQConn(r.getExchange(), r.opts.Addrs, r.getPrefetchCount(), r.getPrefetchGlobal())
r.conn = newRabbitMQConn(
r.getExchange(),
r.opts.Addrs,
r.getPrefetchCount(),
r.getPrefetchGlobal(),
r.getConfirmPublish(),
r.getWithoutExchange(),
r.opts.Logger,
)
}
conf := defaultAmqpConfig
@@ -339,6 +383,7 @@ func (r *rbroker) Disconnect() error {
func NewBroker(opts ...broker.Option) broker.Broker {
options := broker.Options{
Context: context.Background(),
Logger: logger.DefaultLogger,
}
for _, o := range opts {
@@ -352,13 +397,16 @@ func NewBroker(opts ...broker.Option) broker.Broker {
}
func (r *rbroker) getExchange() Exchange {
ex := DefaultExchange
if e, ok := r.opts.Context.Value(exchangeKey{}).(string); ok {
ex.Name = e
}
if t, ok := r.opts.Context.Value(exchangeTypeKey{}).(MQExchangeType); ok {
ex.Type = t
}
if d, ok := r.opts.Context.Value(durableExchange{}).(bool); ok {
ex.Durable = d
}
@@ -379,3 +427,17 @@ func (r *rbroker) getPrefetchGlobal() bool {
}
return DefaultPrefetchGlobal
}
func (r *rbroker) getConfirmPublish() bool {
if e, ok := r.opts.Context.Value(confirmPublishKey{}).(bool); ok {
return e
}
return DefaultConfirmPublish
}
func (r *rbroker) getWithoutExchange() bool {
if e, ok := r.opts.Context.Value(withoutExchangeKey{}).(bool); ok {
return e
}
return DefaultWithoutExchange
}
+305
View File
@@ -0,0 +1,305 @@
package rabbitmq_test
import (
"context"
"encoding/json"
"os"
"testing"
"time"
"go-micro.dev/v5/logger"
micro "go-micro.dev/v5"
broker "go-micro.dev/v5/broker"
rabbitmq "go-micro.dev/v5/broker/rabbitmq"
server "go-micro.dev/v5/server"
)
type Example struct{}
func init() {
rabbitmq.DefaultRabbitURL = "amqp://rabbitmq:rabbitmq@127.0.0.1:5672"
}
type TestEvent struct {
Name string `json:"name"`
Age int `json:"age"`
Time time.Time `json:"time"`
}
func (e *Example) Handler(ctx context.Context, r interface{}) error {
return nil
}
func TestDurable(t *testing.T) {
if tr := os.Getenv("TRAVIS"); len(tr) > 0 {
t.Skip()
}
brkrSub := broker.NewSubscribeOptions(
broker.Queue("queue.default"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
b := rabbitmq.NewBroker()
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
h := &Example{}
// Register a subscriber
micro.RegisterSubscriber(
"topic",
service.Server(),
h.Handler,
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("queue.default"),
)
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestWithoutExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.WithoutExchange())
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("direct.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"direct.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("direct.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("direct.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestFanoutExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.ExchangeType(rabbitmq.ExchangeTypeFanout), rabbitmq.ExchangeName("fanout.test"))
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("fanout.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"fanout.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("fanout.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("fanout.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestDirectExchange(t *testing.T) {
b := rabbitmq.NewBroker(rabbitmq.ExchangeType(rabbitmq.ExchangeTypeDirect), rabbitmq.ExchangeName("direct.test"))
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("direct.exchange.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"direct.exchange.queue",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("direct.exchange.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("direct.exchange.queue", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
func TestTopicExchange(t *testing.T) {
b := rabbitmq.NewBroker()
b.Init()
if err := b.Connect(); err != nil {
t.Logf("cant conect to broker, skip: %v", err)
t.Skip()
}
s := server.NewServer(server.Broker(b))
service := micro.NewService(
micro.Server(s),
micro.Broker(b),
)
brkrSub := broker.NewSubscribeOptions(
broker.Queue("topic.exchange.queue"),
broker.DisableAutoAck(),
rabbitmq.DurableQueue(),
)
// Register a subscriber
err := micro.RegisterSubscriber(
"my-test-topic",
service.Server(),
func(ctx context.Context, evt *TestEvent) error {
logger.Logf(logger.InfoLevel, "receive event: %+v", evt)
return nil
},
server.SubscriberContext(brkrSub.Context),
server.SubscriberQueue("topic.exchange.queue"),
)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(5 * time.Second)
logger.Logf(logger.InfoLevel, "pub event")
jsonData, _ := json.Marshal(&TestEvent{
Name: "test",
Age: 16,
})
err := b.Publish("my-test-topic", &broker.Message{
Body: jsonData,
},
rabbitmq.DeliveryMode(2),
rabbitmq.ContentType("application/json"))
if err != nil {
t.Fatal(err)
}
}()
// service.Init()
if err := service.Run(); err != nil {
t.Fatal(err)
}
}
+8 -9
View File
@@ -22,16 +22,15 @@ var (
)
// Cache is the interface that wraps the cache.
//
// Context specifies the context for the cache.
// Get gets a cached value by key.
// Put stores a key-value pair into cache.
// Delete removes a key from cache.
type Cache interface {
Context(ctx context.Context) Cache
Get(key string) (interface{}, time.Time, error)
Put(key string, val interface{}, d time.Duration) error
Delete(key string) error
// Get gets a cached value by key.
Get(ctx context.Context, key string) (interface{}, time.Time, error)
// Put stores a key-value pair into cache.
Put(ctx context.Context, key string, val interface{}, d time.Duration) error
// Delete removes a key from cache.
Delete(ctx context.Context, key string) error
// String returns the name of the implementation.
String() string
}
// Item represents an item stored in the cache.
+10 -12
View File
@@ -8,20 +8,14 @@ import (
type memCache struct {
opts Options
sync.RWMutex
ctx context.Context
items map[string]Item
sync.RWMutex
}
func (c *memCache) Context(ctx context.Context) Cache {
c.ctx = ctx
return c
}
func (c *memCache) Get(key string) (interface{}, time.Time, error) {
c.RWMutex.Lock()
defer c.RWMutex.Unlock()
func (c *memCache) Get(ctx context.Context, key string) (interface{}, time.Time, error) {
c.RWMutex.RLock()
defer c.RWMutex.RUnlock()
item, found := c.items[key]
if !found {
@@ -34,7 +28,7 @@ func (c *memCache) Get(key string) (interface{}, time.Time, error) {
return item.Value, time.Unix(0, item.Expiration), nil
}
func (c *memCache) Put(key string, val interface{}, d time.Duration) error {
func (c *memCache) Put(ctx context.Context, key string, val interface{}, d time.Duration) error {
var e int64
if d == DefaultExpiration {
d = c.opts.Expiration
@@ -54,7 +48,7 @@ func (c *memCache) Put(key string, val interface{}, d time.Duration) error {
return nil
}
func (c *memCache) Delete(key string) error {
func (c *memCache) Delete(ctx context.Context, key string) error {
c.RWMutex.Lock()
defer c.RWMutex.Unlock()
@@ -66,3 +60,7 @@ func (c *memCache) Delete(key string) error {
delete(c.items, key)
return nil
}
func (m *memCache) String() string {
return "memory"
}

Some files were not shown because too many files have changed in this diff Show More