chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,7 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
- 生成文档除了 README.md 以外其他都放在 docs 目录
|
||||
- 单元测试文件存放在 docs
|
||||
- 不要自动生成测试文件，只有在用户手动提示的情况下再自动创建
|
||||
@@ -0,0 +1,44 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, "optimize/**"]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: pytest (py${{ matrix.python-version }} / ${{ matrix.os }})
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, windows-latest]
|
||||
python-version: ["3.10", "3.11", "3.12", "3.13"]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --extra dev --extra test
|
||||
|
||||
- name: Run unit tests
|
||||
run: uv run pytest tests/unit -q --cov=myboot --cov-report=term-missing
|
||||
|
||||
- name: Run integration tests
|
||||
shell: bash
|
||||
# 退出码 5 = 未收集到用例（集成测试目录暂为空时不视为失败）
|
||||
run: uv run pytest tests/integration -q -m integration || [ $? -eq 5 ]
|
||||
|
||||
- name: Upload coverage report
|
||||
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-report
|
||||
path: .coverage
|
||||
include-hidden-files: true
|
||||
@@ -0,0 +1,40 @@
|
||||
name: Publish to PyPI
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*" # 匹配所有以 v 开头的 tag，如 v1.0.0, v0.1.0
|
||||
|
||||
jobs:
|
||||
build-and-publish:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # 用于 trusted publishing (OIDC)
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Install build dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install build hatchling
|
||||
|
||||
- name: Build distribution packages
|
||||
run: |
|
||||
python -m build
|
||||
|
||||
- name: Publish to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
# 使用 trusted publishing (推荐方式，无需 token)
|
||||
# 需要在 PyPI 上配置 trusted publishing
|
||||
# 参考: https://docs.pypi.org/trusted-publishers/
|
||||
print-hash: true # 显示上传文件的哈希值，便于验证
|
||||
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# 虚拟环境
|
||||
venv/
|
||||
ENV/
|
||||
env/
|
||||
.venv/
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# 日志
|
||||
logs/
|
||||
*.log
|
||||
|
||||
# 数据库
|
||||
*.db
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
|
||||
# 配置文件（可能包含敏感信息）
|
||||
.env
|
||||
.env.local
|
||||
|
||||
# 测试
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.tox/
|
||||
|
||||
# 系统文件
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
.myboot_cache_*.json
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# 贡献指南
|
||||
|
||||
感谢你对 MyBoot 的关注！欢迎通过 Issue 和 Pull Request 参与贡献。
|
||||
|
||||
## 开发环境
|
||||
|
||||
- Python 3.9+（CI 覆盖 3.9–3.13）
|
||||
- [uv](https://github.com/astral-sh/uv) 作为包管理器
|
||||
|
||||
```bash
|
||||
git clone https://github.com/TrumanDu/myboot.git
|
||||
cd myboot
|
||||
uv sync --extra dev --extra test
|
||||
```
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
# 全量单元测试
|
||||
uv run pytest tests/unit -q
|
||||
|
||||
# 含覆盖率
|
||||
uv run pytest tests/unit -q --cov=myboot --cov-report=term-missing
|
||||
|
||||
# 集成测试（涉及真实多进程，较慢）
|
||||
uv run pytest tests/integration -q -m integration
|
||||
```
|
||||
|
||||
提交 PR 前请确保 `tests/unit` 全部通过。CI 会在 ubuntu/windows × Python 3.9–3.13 矩阵上运行。
|
||||
|
||||
## 测试约定
|
||||
|
||||
- `tests/unit/` 中包含**特征测试**（characterization tests）：固化框架公开行为，作为兼容性闸门。如果你的改动有意变更某个公开行为，请同步更新对应测试并在注释中说明原因（参考现有 `issue #14` 风格的注释）。
|
||||
- 修复 bug 时请附带回归测试。
|
||||
|
||||
## 代码风格
|
||||
|
||||
- 遵循现有代码风格（中文 docstring、loguru 日志）
|
||||
- 格式化：`uv run black myboot tests`，import 排序：`uv run isort myboot tests`
|
||||
- 公开 API 需带类型注解
|
||||
|
||||
## 提交规范
|
||||
|
||||
- 提交信息使用 `type: subject` 格式，type 取 `feat` / `fix` / `test` / `docs` / `refactor` / `chore`
|
||||
- 一个 PR 聚焦一件事；破坏性变更需在 PR 描述中明确标注
|
||||
|
||||
## 报告问题
|
||||
|
||||
提 Issue 时请附上：
|
||||
|
||||
- myboot 版本、Python 版本、操作系统
|
||||
- 最小复现代码
|
||||
- 完整报错堆栈（启动时加 `logging.level: DEBUG` 获取更多信息）
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目：`TrumanDu/myboot`
|
||||
- 原始仓库：https://github.com/TrumanDu/myboot
|
||||
- 导入方式：上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1,34 @@
|
||||
# MyBoot 应用配置文件
|
||||
|
||||
# 应用配置
|
||||
app:
|
||||
name: "MyBoot App"
|
||||
version: "0.1.0"
|
||||
|
||||
# 服务器配置
|
||||
server:
|
||||
port: 8000
|
||||
reload: false
|
||||
workers: 1
|
||||
keep_alive_timeout: 60
|
||||
graceful_timeout: 30
|
||||
# CORS 配置（可选，如果不存在则不启用 CORS）
|
||||
cors:
|
||||
allow_origins: ["*"]
|
||||
allow_credentials: true
|
||||
allow_methods: ["*"]
|
||||
allow_headers: ["*"]
|
||||
response_format:
|
||||
enabled: true # 是否启用自动格式化
|
||||
exclude_paths: # 排除的路径（这些路径不会自动格式化）
|
||||
- "/docs"
|
||||
|
||||
# 日志配置
|
||||
logging:
|
||||
level: "debug"
|
||||
|
||||
# 任务调度配置
|
||||
scheduler:
|
||||
enabled: true
|
||||
timezone: "UTC"
|
||||
max_workers: 10
|
||||
@@ -0,0 +1,579 @@
|
||||
# 配置管理使用说明
|
||||
|
||||
MyBoot 使用 [Dynaconf](https://www.dynaconf.com/) 管理配置，支持 YAML 文件、多文件合并、环境变量覆盖与远程配置文件。实现见 `myboot/core/config.py`。
|
||||
|
||||
## 1. 快速开始
|
||||
|
||||
在项目根目录或 `conf/` 目录放置 `config.yaml`，应用启动时自动加载：
|
||||
|
||||
```yaml
|
||||
# conf/config.yaml
|
||||
app:
|
||||
name: "MyBoot App"
|
||||
version: "0.1.0"
|
||||
|
||||
server:
|
||||
port: 8000
|
||||
reload: false
|
||||
|
||||
logging:
|
||||
level: "INFO"
|
||||
```
|
||||
|
||||
代码中读取：
|
||||
|
||||
```python
|
||||
from myboot.core.config import get_settings, get_config, get_config_bool
|
||||
|
||||
# 点号路径（推荐）
|
||||
port = get_config("server.port", 8000)
|
||||
debug = get_config_bool("app.debug", False)
|
||||
|
||||
# Dynaconf 对象
|
||||
settings = get_settings()
|
||||
name = settings.app.name # 属性访问
|
||||
port = settings.get("server.port") # 与 get_config 等价
|
||||
```
|
||||
|
||||
通过 `create_app` 创建应用时，配置挂在 `app.config` 上，与全局 `get_settings()` 为同一实例（首次初始化后）：
|
||||
|
||||
```python
|
||||
from myboot import create_app
|
||||
|
||||
app = create_app(name="我的应用", config_file="conf/config.yaml")
|
||||
port = app.config.get("server.port", 8000)
|
||||
```
|
||||
|
||||
## 2. 配置文件位置与合并
|
||||
|
||||
### 2.1 自动发现的文件（按加载顺序）
|
||||
|
||||
Dynaconf **按列表顺序加载**，**后加载的文件覆盖先加载的同名字段**：
|
||||
|
||||
| 顺序 | 路径 | 说明 |
|
||||
| ---- | -------------------------------------------------------------------------- | ------------------------ |
|
||||
| 1 | `{项目根}/config.yaml` 或 `config.yml` | 基底配置 |
|
||||
| 2 | `{项目根}/conf/config.yaml` 或 `conf/config.yml` | 覆盖根目录同名键 |
|
||||
| 3 | `create_app(config_file=...)` / `get_settings(config_file=...)` 传入的路径 | 覆盖上述文件 |
|
||||
| 4 | 环境变量 `CONFIG_FILE` 指向的路径或 URL | **文件来源中优先级最高** |
|
||||
|
||||
示例：根目录 `server.port: 8000`，`conf/config.yaml` 中 `server.port: 9000`，最终以 **9000** 为准（若未再被 `CONFIG_FILE` 或环境变量键覆盖）。
|
||||
|
||||
同一嵌套对象（如 `server.cors`）在多个文件中出现时，还会受全局 `merge_enabled` 与 `dynaconf_merge` 影响，见下文 **第 3 节**。
|
||||
|
||||
### 2.2 指定配置文件
|
||||
|
||||
```bash
|
||||
# 本地文件
|
||||
export CONFIG_FILE=/etc/myboot/production.yaml
|
||||
|
||||
# 远程 URL（会下载到系统临时目录缓存，失败时尝试使用缓存）
|
||||
export CONFIG_FILE=https://example.com/config.yaml
|
||||
```
|
||||
|
||||
```python
|
||||
from myboot import create_app
|
||||
|
||||
app = create_app(config_file="conf/config.prod.yaml")
|
||||
```
|
||||
|
||||
远程配置下载逻辑见 `_download_config`：网络失败且存在缓存时回退到缓存文件。
|
||||
|
||||
### 2.3 内置默认值
|
||||
|
||||
未在任何 YAML 或环境变量中声明时，使用 `config.py` 中 `default_settings`（节选）：
|
||||
|
||||
| 键 | 默认值 |
|
||||
| -------------------------------- | -------------- |
|
||||
| `app.name` | `"MyBoot App"` |
|
||||
| `app.version` | `"0.1.0"` |
|
||||
| `server.host` | `"0.0.0.0"` |
|
||||
| `server.port` | `8000` |
|
||||
| `server.reload` | `true` |
|
||||
| `server.workers` | `1` |
|
||||
| `server.response_format.enabled` | `true` |
|
||||
| `logging.level` | `"INFO"` |
|
||||
| `scheduler.enabled` | `true` |
|
||||
| `scheduler.timezone` | `"UTC"` |
|
||||
| `scheduler.max_workers` | `10` |
|
||||
|
||||
## 3. 字典/列表合并与 `dynaconf_merge`
|
||||
|
||||
MyBoot 在 `create_settings()` 中启用了 Dynaconf 全局合并：
|
||||
|
||||
```python
|
||||
# myboot/core/config.py
|
||||
merge_enabled=True,
|
||||
```
|
||||
|
||||
因此，**多个配置文件**（或带 `@merge` 的环境变量）加载到**同一嵌套字典/列表**时，默认会**深合并**，而不是简单地把后一个文件里的整块 YAML 盖掉前一个。
|
||||
|
||||
### 3.1 默认合并行为（`merge_enabled=True`）
|
||||
|
||||
| 类型 | 后加载文件中的同路径值 | 结果 |
|
||||
| --------------------------------- | ---------------------- | ---------------------------------------- |
|
||||
| 标量（`str` / `int` / `bool` 等） | 新值 | **覆盖**旧值 |
|
||||
| 字典 | 新键与旧键 | **递归合并**（保留旧文件中未出现的子键） |
|
||||
| 列表 | 新元素 | **拼接合并**（可能产生重复项） |
|
||||
|
||||
**示例**：根目录与 `conf/` 均定义 `server.cors` 时，若后加载文件只改 `allow_origins`，未写 `allow_methods`，合并后仍会保留先前的 `allow_methods`、`allow_headers`。
|
||||
|
||||
```yaml
|
||||
# config.yaml（先加载）
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["*"]
|
||||
allow_methods: ["*"]
|
||||
allow_headers: ["*"]
|
||||
|
||||
# conf/config.yaml（后加载，未使用 dynaconf_merge: false）
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://localhost:3000"]
|
||||
```
|
||||
|
||||
合并结果等价于：
|
||||
|
||||
```yaml
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["*", "http://localhost:3000"] # 列表被拼接
|
||||
allow_methods: ["*"]
|
||||
allow_headers: ["*"]
|
||||
```
|
||||
|
||||
### 3.2 `dynaconf_merge: false` 的作用
|
||||
|
||||
在需要**整块替换**某个字典（或列表）而不是与旧值合并时，在该字典（或列表）内加上 **`dynaconf_merge: false`**。
|
||||
该标记是 Dynaconf 的**合并控制元数据**，不会作为业务配置键出现在 `get_config()` 结果中。
|
||||
|
||||
| 写法 | 作用范围 |
|
||||
| ----------------------------- | ---------------------------------------------------- |
|
||||
| 写在某个 **dict / list 内部** | 仅该节点：后加载内容**整体替换**先加载的同级对象 |
|
||||
| 写在 **YAML 文件顶层** | 整个后加载文件相对先前来源按「替换」策略处理（慎用） |
|
||||
|
||||
**示例：用 `false` 完全替换 `server.cors`**
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["*"]
|
||||
allow_methods: ["*"]
|
||||
allow_headers: ["*"]
|
||||
|
||||
# conf/config.prod.yaml（后加载）
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["https://api.example.com"]
|
||||
allow_credentials: true
|
||||
dynaconf_merge: false # 整块替换 cors，不保留 allow_methods / allow_headers
|
||||
```
|
||||
|
||||
最终 `server.cors` 仅包含：
|
||||
|
||||
```yaml
|
||||
allow_origins: ["https://api.example.com"]
|
||||
allow_credentials: true
|
||||
```
|
||||
|
||||
**对比**：若去掉 `dynaconf_merge: false`，后加载文件里未写的 `allow_methods`、`allow_headers` 会**继续保留**，且 `allow_origins` 可能与旧列表**拼接**。
|
||||
|
||||
### 3.3 `dynaconf_merge: true`
|
||||
|
||||
显式声明「与已有字典/列表合并」。在 `merge_enabled=True` 时，多数嵌套 dict/list **已默认合并**，一般仅在需要强调或配合 `@merge` 环境变量时使用：
|
||||
|
||||
```yaml
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://localhost:3000"]
|
||||
dynaconf_merge: true
|
||||
```
|
||||
|
||||
环境变量也可使用 Dynaconf 的 `@merge` 标记（TOML/JSON 形式），例如：
|
||||
|
||||
```bash
|
||||
export SERVER__CORS='@merge {"allow_origins": ["http://localhost:3000"]}'
|
||||
```
|
||||
|
||||
### 3.4 何时用 `false`、何时用环境变量
|
||||
|
||||
| 场景 | 建议 |
|
||||
| ------------------------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
| 生产配置要**换一整段** `server.cors` / `logging.third_party` | 后加载 YAML 中对应该段加 `dynaconf_merge: false` |
|
||||
| 只改**一两个嵌套字段** | 直接写子键，或 `SERVER__CORS__ALLOW_ORIGINS=...` |
|
||||
| 列表不想与旧值拼接 | 对该 list 使用 `dynaconf_merge: false`，或写完整列表并 `false` |
|
||||
| 不确定合并结果 | 用 `get_settings().server.cors` 或 `get_config("server.cors")` 在本地打印验证 |
|
||||
|
||||
### 3.5 注意
|
||||
|
||||
1. **`dynaconf_merge` 只影响合并策略**，不改变「后加载文件优先」的顺序（见第 2 节）。
|
||||
2. **标量字段**始终是覆盖，与 `dynaconf_merge` 无关。
|
||||
3. 对**深层嵌套**的精细控制，除 `dynaconf_merge: false` 外，也可用环境变量 `__` 逐键覆盖（见第 5 节环境变量）。
|
||||
4. 更多语法见 [Dynaconf 合并文档](https://www.dynaconf.com/merging/)。
|
||||
|
||||
## 4. 优先级总览
|
||||
|
||||
从高到低（后者覆盖前者）：
|
||||
|
||||
```
|
||||
环境变量键（如 SERVER__PORT） > CONFIG_FILE 指向的文件 > config_file 参数 > conf/config.yaml > 根目录 config.yaml > default_settings
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[default_settings] --> B[根目录 config.yaml]
|
||||
B --> C[conf/config.yaml]
|
||||
C --> D[config_file 参数]
|
||||
D --> E[CONFIG_FILE]
|
||||
E --> F[环境变量 SERVER__PORT 等]
|
||||
```
|
||||
|
||||
## 5. 环境变量
|
||||
|
||||
### 5.1 规则
|
||||
|
||||
| 项 | 说明 |
|
||||
| ------------ | ----------------------------------------------------------- |
|
||||
| 前缀 | **无**（`envvar_prefix=False`），变量名即配置路径的大写形式 |
|
||||
| 嵌套分隔符 | **双下划线 `__`**，对应 YAML 中的层级 |
|
||||
| 单下划线 `_` | 仅作为**同一层键名**的一部分（如 `keep_alive_timeout`） |
|
||||
| 类型 | `env_parse_values=True`，自动尝试解析布尔、数字等 |
|
||||
| 未知变量 | `ignore_unknown_envvars=True`，**仅当该键已在配置树中存在时**才接受环境变量覆盖（见下） |
|
||||
|
||||
#### 已知键限制（`ignore_unknown_envvars=True`）
|
||||
|
||||
MyBoot 在 `config.py` 中启用了 `ignore_unknown_envvars=True`。环境变量（含 `.env` 加载进 `os.environ` 的项）**不会**凭空新增配置项，只能**覆盖**下列来源里**已经出现过**的键路径：
|
||||
|
||||
1. 已加载的 YAML（`config.yaml`、`conf/config.yaml`、`CONFIG_FILE` 等）
|
||||
2. `config.py` 中 `default_settings` 内置项（如 `app.name`、`server.port`、`logging.level`、`scheduler.timezone` 等）
|
||||
|
||||
因此：
|
||||
|
||||
- ✅ `SERVER__PORT=9000` 有效（`server.port` 在默认配置或 YAML 中已有）
|
||||
- ❌ `DATABASE__URL=...` **无效**，若 YAML / 默认配置里**没有** `database` 段——变量会进入 `os.environ`，但 **MyBoot 不会读入** `get_config("database.url")`
|
||||
- ❌ 任意「只在 `.env` 里出现、YAML 从未声明」的键都会被**静默忽略**
|
||||
|
||||
**正确做法**：需要靠 `.env` 注入的新配置，先在 YAML 中**声明结构**（值可写占位符），再用环境变量覆盖：
|
||||
|
||||
```yaml
|
||||
# conf/config.yaml — 先声明键结构
|
||||
database:
|
||||
url: "" # 占位，真实值放在 .env / .local.env
|
||||
app:
|
||||
secret_key: "" # 敏感项同理
|
||||
jobs:
|
||||
cleanup_task:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
```bash
|
||||
# .local.env — 再覆盖
|
||||
DATABASE__URL=postgresql://user:pass@127.0.0.1:5432/mydb
|
||||
APP__SECRET_KEY=your-local-secret-key
|
||||
JOBS__CLEANUP_TASK__ENABLED=false
|
||||
```
|
||||
|
||||
若希望环境变量**无需**在 YAML 预声明即可生效，需修改 `myboot/core/config.py` 将 `ignore_unknown_envvars` 设为 `False`（当前框架默认未开启）。
|
||||
|
||||
### 5.2 示例
|
||||
|
||||
| YAML 路径 | 环境变量 |
|
||||
| --------------------------- | --------------------------------------------------------- |
|
||||
| `app.name` | `APP__NAME=MyApp` |
|
||||
| `server.port` | `SERVER__PORT=9000` |
|
||||
| `logging.level` | `LOGGING__LEVEL=DEBUG` |
|
||||
| `server.keep_alive_timeout` | `SERVER__KEEP_ALIVE_TIMEOUT=60` |
|
||||
| `server.cors.allow_origins` | `SERVER__CORS__ALLOW_ORIGINS='["http://localhost:3000"]'` |
|
||||
|
||||
```bash
|
||||
export APP__NAME="生产应用"
|
||||
export SERVER__PORT=8080
|
||||
export LOGGING__LEVEL=WARNING
|
||||
export SCHEDULER__TIMEZONE=Asia/Shanghai
|
||||
```
|
||||
|
||||
**不要**用单下划线表示嵌套层级，例如 `SERVER_PORT` **不会**映射到 `server.port`（除非你在 YAML 里真有名为 `server_port` 的顶层键）。
|
||||
|
||||
### 5.3 布尔值
|
||||
|
||||
`get_config_bool` 将字符串视为真：`true`、`1`、`yes`、`on`（不区分大小写）。
|
||||
|
||||
### 5.4 MyBoot 与 `.env` 的关系（0.2.0 起自动加载）
|
||||
|
||||
**0.2.0 起 MyBoot 自动加载项目根目录的 `.env` 文件**（通过 Dynaconf 的
|
||||
`load_dotenv`），`main.py` 无需再手动调用 `load_dotenv()`。
|
||||
|
||||
加载语义：
|
||||
|
||||
- 路径：**项目根目录**（含 `pyproject.toml` 的目录）下的 `.env`；
|
||||
- 优先级：**真实环境变量 > `.env` > YAML 配置 > 内置默认值**
|
||||
（`dotenv_override=False`：`.env` 不覆盖已存在的真实环境变量，与容器部署习惯一致）；
|
||||
- 仍受 **`ignore_unknown_envvars=True`** 约束：`.env` 里的变量名必须对应
|
||||
**YAML 或内置默认配置中已声明过的键路径**，否则不会进入 `get_config()`
|
||||
（见上文 **5.1 已知键限制**）。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[".env（项目根）"] -->|"框架自动加载"| C["os.environ"]
|
||||
B["真实环境变量"] -->|"优先"| C
|
||||
C --> D["Dynaconf / MyBoot"]
|
||||
E["config.yaml"] --> D
|
||||
```
|
||||
|
||||
**注意：**
|
||||
|
||||
| 事项 | 说明 |
|
||||
| ------------------- | -------------------------------------------------------------------- |
|
||||
| 修改 `.env` 后 | 需**重启进程**；不会热更新 |
|
||||
| 版本控制 | `.env` 加入 `.gitignore`；可提交 `.env.example` 作模板 |
|
||||
| 键必须已声明 | `.env` 中每一项都须在 YAML（或内置默认值）中有对应路径，见 5.1 |
|
||||
| 多个 env 文件 | 框架只自动加载根目录 `.env`；需要 `.local.env` 分层时仍可在 main.py 顶部自行 `load_dotenv(..., override=True)` |
|
||||
|
||||
### 5.5 旧版本（≤0.1.x）手动加载方式
|
||||
|
||||
0.1.x 不自动加载 `.env`，需在 `create_app()` 之前手动调用：
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv(Path(__file__).resolve().parent / ".env")
|
||||
|
||||
from myboot import create_app # 必须在 load_dotenv 之后
|
||||
app = create_app(name="我的应用")
|
||||
```
|
||||
|
||||
升级到 0.2.0 后这段代码可以删除（保留也无害——重复加载是幂等的）。
|
||||
|
||||
### 5.6 `.env` 文件如何书写
|
||||
|
||||
`.env` 使用 **dotenv 格式**（不是 YAML）：每行 `键=值`，`#` 开头为注释。**不要**写 `export`（那是 shell 脚本写法）。
|
||||
|
||||
> **重要**：`.env` 只用于**覆盖**已有配置键，不能代替 YAML 做「首次声明」。未在 YAML / `default_settings` 中出现的键（如仅写 `DATABASE__URL` 而 `config.yaml` 里没有 `database.url`）将被忽略。
|
||||
|
||||
#### 命名规则（与 MyBoot 一致）
|
||||
|
||||
- 嵌套配置用 **`__`（双下划线）** 对应 YAML 层级。
|
||||
- 键名一般**大写**（与常见约定一致；写入环境后 Dynaconf 会映射到配置树）。
|
||||
- 单下划线 `_` 只表示键名的一部分，例如 `KEEP_ALIVE_TIMEOUT`。
|
||||
|
||||
#### 示例 `.env`（非敏感、可提交 `.env.example`）
|
||||
|
||||
```bash
|
||||
# 应用
|
||||
APP__NAME=MyBoot App
|
||||
APP__VERSION=0.1.0
|
||||
APP__DEBUG=false
|
||||
|
||||
# 服务
|
||||
SERVER__HOST=0.0.0.0
|
||||
SERVER__PORT=8000
|
||||
SERVER__RELOAD=true
|
||||
|
||||
# 日志
|
||||
LOGGING__LEVEL=INFO
|
||||
|
||||
# 调度器
|
||||
SCHEDULER__ENABLED=true
|
||||
SCHEDULER__TIMEZONE=Asia/Shanghai
|
||||
```
|
||||
|
||||
#### 示例 `.local.env`（敏感信息，勿提交 Git）
|
||||
|
||||
以下变量均要求 `conf/config.yaml`（或根目录 `config.yaml`）中**已有同名路径**；示例见上一节 YAML 占位声明。
|
||||
|
||||
```bash
|
||||
# 覆盖 .env 中的端口（server.port 已在 YAML/默认配置中存在）
|
||||
SERVER__PORT=9000
|
||||
|
||||
# 以下键须先在 YAML 中声明 database.url、app.secret_key
|
||||
DATABASE__URL=postgresql://user:password@127.0.0.1:5432/mydb
|
||||
APP__SECRET_KEY=your-local-secret-key
|
||||
```
|
||||
|
||||
#### 与 YAML 的对应关系
|
||||
|
||||
| `.env` 中的写法 | 等价 YAML 路径 | 代码读取 |
|
||||
| ------------------------------------- | --------------------------- | ----------------------------------------- |
|
||||
| `SERVER__PORT=9000` | `server.port` | `get_config("server.port")` |
|
||||
| `LOGGING__LEVEL=DEBUG` | `logging.level` | `get_config("logging.level")` |
|
||||
| `DATABASE__URL=...` | `database.url`（**须先在 YAML 声明**） | `get_config("database.url")` |
|
||||
| `SERVER__CORS__ALLOW_ORIGINS='["*"]'` | `server.cors.allow_origins` | `get_config("server.cors.allow_origins")` |
|
||||
|
||||
#### 值的书写建议
|
||||
|
||||
```bash
|
||||
# 字符串含空格或特殊字符时用引号
|
||||
APP__NAME="My Boot App"
|
||||
|
||||
# 布尔（会被 env_parse_values 解析）
|
||||
APP__DEBUG=true
|
||||
SERVER__RELOAD=false
|
||||
|
||||
# 数字
|
||||
SERVER__PORT=8080
|
||||
SCHEDULER__MAX_WORKERS=20
|
||||
|
||||
# 列表 / 字典建议用 JSON 字符串（尤其嵌套较深时）
|
||||
SERVER__CORS__ALLOW_ORIGINS=["http://localhost:3000","http://127.0.0.1:3000"]
|
||||
```
|
||||
|
||||
**错误示例（不会映射到预期配置）：**
|
||||
|
||||
```bash
|
||||
# 错误：单下划线不能表示 server.port
|
||||
SERVER_PORT=8000
|
||||
|
||||
# 错误：dotenv 行内不要用 export
|
||||
export SERVER__PORT=8000
|
||||
|
||||
# 错误：YAML 缩进语法不能用在 .env
|
||||
server:
|
||||
port: 8000
|
||||
|
||||
# 错误：YAML 中未声明 database 段时，此行会被 ignore_unknown_envvars 忽略
|
||||
DATABASE__URL=postgresql://...
|
||||
```
|
||||
|
||||
**内置默认已存在、可直接用 `.env` 覆盖的键（无需 YAML）** 包括：`app.name`、`app.version`、`server.host`、`server.port`、`server.reload`、`server.workers`、`logging.level`、`scheduler.enabled`、`scheduler.timezone`、`scheduler.max_workers` 等，完整列表见 `myboot/core/config.py` 中 `default_settings`。
|
||||
|
||||
#### 验证是否加载成功
|
||||
|
||||
```python
|
||||
# 在 load_dotenv 且 create_app 之后执行
|
||||
from myboot.core.config import get_config
|
||||
|
||||
# server.port 在 YAML/默认配置中存在 → 能读到 .env 覆盖值
|
||||
print(get_config("server.port"))
|
||||
|
||||
# database.url 仅当 YAML 中已声明 database.url 时才能读到 .env 值
|
||||
print(get_config("database.url", "(YAML 未声明或 .env 未覆盖)"))
|
||||
```
|
||||
|
||||
`echo $DATABASE__URL` 能看到 shell 环境变量，但 **`get_config` 仍可能取不到**——说明键未进入 MyBoot 配置树，请检查 YAML 是否已声明该路径。
|
||||
|
||||
或在 shell 中确认环境变量已写入（Windows Git Bash）：
|
||||
|
||||
```bash
|
||||
echo $SERVER__PORT
|
||||
```
|
||||
|
||||
## 6. 读取 API
|
||||
|
||||
| 函数 / 对象 | 用途 |
|
||||
| ------------------------------------- | ---------------------------------------- |
|
||||
| `get_settings(config_file=None)` | 获取全局 `Dynaconf` 单例 |
|
||||
| `get_config(key, default=None)` | 按点号路径取值 |
|
||||
| `get_config_str(key, default="")` | 转为字符串 |
|
||||
| `get_config_int(key, default=0)` | 转为整数，失败返回 default |
|
||||
| `get_config_bool(key, default=False)` | 转为布尔 |
|
||||
| `reload_config()` | 清空单例，下次 `get_settings()` 重新加载 |
|
||||
| `app.config` | 应用持有的同一配置对象 |
|
||||
|
||||
```python
|
||||
# 运行时修改（仅当前进程内存，不写回文件）
|
||||
app.config.set("server.port", 9001)
|
||||
```
|
||||
|
||||
### 6.1 单例注意
|
||||
|
||||
`get_settings()` 使用模块级单例，**首次调用**时确定加载了哪些文件；之后传入不同的 `config_file` 不会生效，除非先调用 `reload_config()`。
|
||||
|
||||
```python
|
||||
from myboot.core.config import get_settings, reload_config
|
||||
|
||||
settings = get_settings() # 已固定加载结果
|
||||
reload_config()
|
||||
settings = get_settings("other.yaml") # 重新加载
|
||||
```
|
||||
|
||||
## 7. 常用配置项
|
||||
|
||||
与框架行为直接相关的键（完整示例可参考 `conf/config.yaml` 与 README）：
|
||||
|
||||
### 7.1 `app`
|
||||
|
||||
| 键 | 说明 |
|
||||
| ------------- | -------------------- |
|
||||
| `app.name` | 应用名称 |
|
||||
| `app.version` | 版本号 |
|
||||
| `app.debug` | 调试开关（按需使用） |
|
||||
|
||||
### 7.2 `server`
|
||||
|
||||
| 键 | 说明 |
|
||||
| -------------------------------------- | ----------------------------------- |
|
||||
| `server.host` | 监听地址 |
|
||||
| `server.port` | 端口 |
|
||||
| `server.reload` | 热重载 |
|
||||
| `server.workers` | Worker 数量 |
|
||||
| `server.keep_alive_timeout` | Keep-Alive 超时 |
|
||||
| `server.graceful_timeout` | 优雅关闭超时 |
|
||||
| `server.cors` | CORS 子对象，存在时启用 CORS 中间件 |
|
||||
| `server.response_format.enabled` | 是否统一响应格式 |
|
||||
| `server.response_format.exclude_paths` | 排除路径列表 |
|
||||
|
||||
### 7.3 `logging`
|
||||
|
||||
| 键 | 说明 |
|
||||
| --------------------- | -------------------- |
|
||||
| `logging.level` | 日志级别 |
|
||||
| `logging.format` | 日志格式 |
|
||||
| `logging.file` | 日志文件路径（可选） |
|
||||
| `logging.third_party` | 第三方库日志级别映射 |
|
||||
|
||||
### 7.4 `scheduler`
|
||||
|
||||
| 键 | 说明 |
|
||||
| -------------------------- | ---------------------------------- |
|
||||
| `scheduler.enabled` | 是否启用调度器 |
|
||||
| `scheduler.timezone` | 时区（建议安装 `pytz`） |
|
||||
| `scheduler.max_workers` | 调度线程池大小 |
|
||||
| `scheduler.on_all_workers` | 多进程时是否在每个 worker 运行调度 |
|
||||
|
||||
Cron 与任务装饰器详见 [scheduler.md](./scheduler.md)。
|
||||
|
||||
### 7.5 `jobs`（可选）
|
||||
|
||||
用于在 `@cron` / `@interval` / `@once` 的 `enabled` 参数中按任务开关：
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
cleanup_task:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
```python
|
||||
from myboot.core.config import get_config
|
||||
|
||||
@cron("0 2 * * *", enabled=get_config("jobs.cleanup_task.enabled", True))
|
||||
def cleanup(self):
|
||||
...
|
||||
```
|
||||
|
||||
## 8. 配置与装饰器、组件
|
||||
|
||||
- **定时任务**：`enabled=get_config('jobs.xxx.enabled', True)` 在类定义时求值，修改环境变量后需**重启进程**才生效。
|
||||
- **日志**：`Application` 构造时调用 `setup_logging(self.config)`，之后改 `logging.*` 需自行处理或重启。
|
||||
- **依赖注入**：`get_config` 可在模块级或组件方法内使用；与 `@component` 无冲突。
|
||||
|
||||
## 9. 最佳实践
|
||||
|
||||
1. **开发**用 `conf/config.yaml`，**生产**用 `CONFIG_FILE` 或环境变量注入敏感项，避免密钥进仓库。
|
||||
2. **嵌套键**统一用 `__` 环境变量，避免与 `snake_case` 字段混淆。
|
||||
3. **显式设置** `logging.level`、`scheduler.timezone`，减少环境差异。
|
||||
4. 列表、字典类配置在环境变量中优先使用 **JSON 字符串**（如 CORS origins）。
|
||||
5. 需要切换配置源时调用 `reload_config()`，或保证在任意 `get_settings()` 之前设置好 `CONFIG_FILE`。
|
||||
6. 多文件拆分时保持「基底 → 环境专用」的加载顺序意识：后加载覆盖先加载。
|
||||
7. 多文件共用同一嵌套段（如 `server.cors`）且不想**拼接列表、保留旧子键**时，在后加载文件中对应该段添加 **`dynaconf_merge: false`**（见第 3 节）。
|
||||
8. 本地敏感项用 **`.local.env`** + `main.py` 中 `load_dotenv`（见第 5.5、5.6 节）；敏感值放 `.env`，但**键名须先在 YAML 声明**（`ignore_unknown_envvars=True`），结构可进 `config.yaml`，秘密不进 Git。
|
||||
|
||||
## 10. 相关文档与代码
|
||||
|
||||
| 资源 | 说明 |
|
||||
| ---------------------------------------------------- | ---------------------------- |
|
||||
| `myboot/core/config.py` | 配置加载与便捷函数 |
|
||||
| `conf/config.yaml` | 项目默认配置示例 |
|
||||
| [scheduler.md](./scheduler.md) | 调度器配置与 Cron |
|
||||
| [dependency-injection.md](./dependency-injection.md) | 组件与 `get_config` 结合示例 |
|
||||
@@ -0,0 +1,627 @@
|
||||
# 依赖注入使用指南
|
||||
|
||||
MyBoot 框架提供了基于 `dependency_injector` 的自动依赖注入功能，让您可以轻松管理服务之间的依赖关系，无需手动获取和传递依赖。
|
||||
|
||||
## 目录
|
||||
|
||||
- [快速开始](#快速开始)
|
||||
- [基本用法](#基本用法)
|
||||
- [声明依赖](#1-声明依赖)
|
||||
- [服务命名规则](#2-服务命名规则)
|
||||
- [多级依赖](#3-多级依赖)
|
||||
- [可选依赖](#4-可选依赖)
|
||||
- [Client 依赖注入](#5-client-依赖注入)
|
||||
- [Component 组件](#6-component-组件)
|
||||
- [高级特性](#高级特性)
|
||||
- [最佳实践](#最佳实践)
|
||||
- [常见问题](#常见问题)
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 安装依赖
|
||||
|
||||
确保已安装 `dependency_injector`：
|
||||
|
||||
```bash
|
||||
pip install dependency-injector
|
||||
```
|
||||
|
||||
### 基本示例
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import service
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
"""用户服务"""
|
||||
def __init__(self):
|
||||
self.users = {}
|
||||
|
||||
def get_user(self, user_id: int):
|
||||
return self.users.get(user_id)
|
||||
|
||||
@service()
|
||||
class EmailService:
|
||||
"""邮件服务"""
|
||||
def send_email(self, to: str, subject: str):
|
||||
print(f"发送邮件到 {to}: {subject}")
|
||||
|
||||
@service()
|
||||
class OrderService:
|
||||
"""订单服务 - 自动注入 UserService 和 EmailService"""
|
||||
def __init__(self, user_service: UserService, email_service: EmailService):
|
||||
self.user_service = user_service
|
||||
self.email_service = email_service
|
||||
|
||||
def create_order(self, user_id: int, product: str):
|
||||
user = self.user_service.get_user(user_id)
|
||||
self.email_service.send_email(user['email'], "订单创建", f"您的订单 {product} 已创建")
|
||||
```
|
||||
|
||||
框架会自动：
|
||||
|
||||
1. 检测 `OrderService` 的依赖（`UserService` 和 `EmailService`）
|
||||
2. 按正确的顺序初始化服务
|
||||
3. 自动注入依赖到 `OrderService` 的构造函数
|
||||
|
||||
## 基本用法
|
||||
|
||||
### 1. 声明依赖
|
||||
|
||||
通过类型注解声明依赖是最简单的方式：
|
||||
|
||||
```python
|
||||
@service()
|
||||
class ProductService:
|
||||
def __init__(self, user_service: UserService, cache_service: CacheService):
|
||||
self.user_service = user_service
|
||||
self.cache_service = cache_service
|
||||
```
|
||||
|
||||
框架会自动：
|
||||
|
||||
- 从类型注解中识别依赖的服务类
|
||||
- 将类名转换为服务名（如 `UserService` → `user_service`）
|
||||
- 自动注入对应的服务实例
|
||||
|
||||
### 2. 服务命名规则
|
||||
|
||||
服务名称遵循以下规则：
|
||||
|
||||
- **默认命名**：类名自动转换为下划线分隔的小写形式
|
||||
|
||||
- `UserService` → `user_service`
|
||||
- `EmailService` → `email_service`
|
||||
- `DatabaseClient` → `database_client`
|
||||
|
||||
- **自定义命名**：通过装饰器参数指定
|
||||
```python
|
||||
@service('custom_user_service')
|
||||
class UserService:
|
||||
pass
|
||||
```
|
||||
|
||||
### 3. 多级依赖
|
||||
|
||||
支持多级依赖，框架会自动处理依赖顺序：
|
||||
|
||||
```python
|
||||
@service()
|
||||
class DatabaseClient:
|
||||
def __init__(self):
|
||||
self.connection = None
|
||||
|
||||
@service()
|
||||
class UserRepository:
|
||||
def __init__(self, db: DatabaseClient):
|
||||
self.db = db
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
def __init__(self, user_repo: UserRepository):
|
||||
self.user_repo = user_repo
|
||||
```
|
||||
|
||||
依赖顺序：`DatabaseClient` → `UserRepository` → `UserService`
|
||||
|
||||
### 4. 可选依赖
|
||||
|
||||
使用 `Optional` 类型注解声明可选依赖：
|
||||
|
||||
```python
|
||||
from typing import Optional
|
||||
|
||||
@service()
|
||||
class CacheService:
|
||||
pass
|
||||
|
||||
@service()
|
||||
class ProductService:
|
||||
# cache_service 是可选的，如果不存在则为 None
|
||||
def __init__(self, cache_service: Optional[CacheService] = None):
|
||||
self.cache_service = cache_service
|
||||
if self.cache_service:
|
||||
# 使用缓存服务
|
||||
pass
|
||||
```
|
||||
|
||||
### 5. Client 依赖注入
|
||||
|
||||
除了 Service 之间的依赖注入，框架还支持将 Client 注入到 Controller 或 Service 中：
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import client, service, rest_controller, get
|
||||
|
||||
@client()
|
||||
class HttpClient:
|
||||
"""HTTP 客户端"""
|
||||
def request(self, url: str):
|
||||
return {"url": url}
|
||||
|
||||
@client(name="redis_client") # 自定义名称
|
||||
class RedisClient:
|
||||
"""Redis 客户端"""
|
||||
def get(self, key: str):
|
||||
return None
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
"""注入 Client 到 Service"""
|
||||
def __init__(self, http_client: HttpClient):
|
||||
self.http_client = http_client
|
||||
|
||||
@rest_controller("/api")
|
||||
class UserController:
|
||||
"""注入 Client 和 Service 到 Controller"""
|
||||
def __init__(self, user_service: UserService, redis_client: RedisClient):
|
||||
self.user_service = user_service
|
||||
self.redis_client = redis_client
|
||||
|
||||
@get("/users")
|
||||
def list_users(self):
|
||||
return []
|
||||
```
|
||||
|
||||
#### Client 命名规则
|
||||
|
||||
- **默认命名**：类名自动转换为下划线形式
|
||||
|
||||
- `HttpClient` → `http_client`
|
||||
- `RedisClient` → `redis_client`
|
||||
|
||||
- **自定义命名**：通过装饰器参数指定
|
||||
```python
|
||||
@client(name="my_redis")
|
||||
class RedisClient:
|
||||
pass
|
||||
```
|
||||
|
||||
#### Client 查找方式
|
||||
|
||||
框架支持多种方式查找 Client 依赖：
|
||||
|
||||
```python
|
||||
@client(name="my_http") # 自定义名称
|
||||
class HttpClient:
|
||||
pass
|
||||
|
||||
@rest_controller("/api")
|
||||
class MyController:
|
||||
# 以下方式都可以成功注入：
|
||||
|
||||
# 方式1：按自定义名称（参数名匹配）
|
||||
def __init__(self, my_http: HttpClient):
|
||||
pass
|
||||
|
||||
# 方式2：按自动转换名称
|
||||
def __init__(self, http_client: HttpClient):
|
||||
pass
|
||||
|
||||
# 方式3：按类型匹配（参数名任意）
|
||||
def __init__(self, client: HttpClient):
|
||||
pass
|
||||
|
||||
# 方式4：显式指定名称
|
||||
def __init__(self, x: Provide['my_http']):
|
||||
pass
|
||||
```
|
||||
|
||||
### 6. Component 组件
|
||||
|
||||
`@component` 装饰器用于注册通用组件，支持依赖注入。它可用于任意需要托管的类（工具类、配置类、包含定时任务的类等）。
|
||||
|
||||
#### 基本用法
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import component
|
||||
|
||||
@component()
|
||||
class EmailHelper:
|
||||
"""邮件工具类"""
|
||||
def send(self, to: str, content: str):
|
||||
print(f"发送邮件到 {to}: {content}")
|
||||
```
|
||||
|
||||
#### 带依赖注入
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import component, client
|
||||
|
||||
@client()
|
||||
class SmtpClient:
|
||||
def send_mail(self, to: str, subject: str, body: str):
|
||||
pass
|
||||
|
||||
@component(name='email_helper')
|
||||
class EmailHelper:
|
||||
"""带依赖注入的组件"""
|
||||
def __init__(self, smtp_client: SmtpClient):
|
||||
self.smtp = smtp_client
|
||||
|
||||
def send(self, to: str, subject: str, body: str):
|
||||
self.smtp.send_mail(to, subject, body)
|
||||
```
|
||||
|
||||
#### 包含定时任务的组件
|
||||
|
||||
**重要**：定时任务（`@cron`、`@interval`、`@once`）**必须**在 `@component` 装饰的类中定义。这是定义定时任务的唯一方式，不再支持模块级函数或 `@service` 类中的定时任务。
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import component, service, cron, interval
|
||||
|
||||
@service()
|
||||
class DataService:
|
||||
def sync(self):
|
||||
print("同步数据...")
|
||||
|
||||
def health_check(self):
|
||||
print("健康检查...")
|
||||
|
||||
@component()
|
||||
class DataSyncJobs:
|
||||
"""数据同步任务集合 - 自动注入 DataService"""
|
||||
|
||||
def __init__(self, data_service: DataService):
|
||||
self.data_service = data_service
|
||||
|
||||
@cron("0 2 * * *") # 每天凌晨 2 点
|
||||
def sync_daily_data(self):
|
||||
"""每日数据同步"""
|
||||
self.data_service.sync()
|
||||
|
||||
@interval(hours=1) # 每小时
|
||||
def check_data_health(self):
|
||||
"""数据健康检查"""
|
||||
self.data_service.health_check()
|
||||
```
|
||||
|
||||
**注意**：
|
||||
- 定时任务方法会在组件注册时自动扫描并注册到调度器
|
||||
- 组件支持依赖注入，可以在构造函数中注入所需的服务
|
||||
|
||||
#### Component 配置选项
|
||||
|
||||
```python
|
||||
@component(
|
||||
name='my_component', # 组件名称，默认使用类名的 snake_case
|
||||
scope='singleton', # 生命周期：'singleton'（默认）或 'prototype'
|
||||
lazy=False, # 是否懒加载
|
||||
primary=False # 当按类型获取有多个匹配时，是否为首选
|
||||
)
|
||||
class MyComponent:
|
||||
pass
|
||||
```
|
||||
|
||||
#### 从容器获取组件
|
||||
|
||||
```python
|
||||
from myboot.core.application import app
|
||||
|
||||
# 方式1：通过 container 获取
|
||||
email_helper = app().container.get('email_helper')
|
||||
|
||||
# 方式2：通过 Application 直接获取
|
||||
email_helper = app().get_component('email_helper')
|
||||
|
||||
# 方式3：依赖注入（推荐）
|
||||
@component()
|
||||
class NotificationService:
|
||||
def __init__(self, email_helper: EmailHelper):
|
||||
self.email_helper = email_helper
|
||||
```
|
||||
|
||||
## 高级特性
|
||||
|
||||
### 1. 显式指定服务名
|
||||
|
||||
如果服务名与类名转换规则不匹配，可以使用 `Provide` 类型提示：
|
||||
|
||||
```python
|
||||
from myboot.core.di import Provide
|
||||
|
||||
@service('custom_user_service')
|
||||
class UserService:
|
||||
pass
|
||||
|
||||
@service()
|
||||
class OrderService:
|
||||
def __init__(self, user_service: Provide['custom_user_service']):
|
||||
self.user_service = user_service
|
||||
```
|
||||
|
||||
### 2. 服务生命周期
|
||||
|
||||
通过 `scope` 参数控制服务的生命周期：
|
||||
|
||||
```python
|
||||
# 单例模式（默认）
|
||||
@service(scope='singleton')
|
||||
class UserService:
|
||||
pass
|
||||
|
||||
# 工厂模式（每次创建新实例）
|
||||
@service(scope='factory')
|
||||
class TaskService:
|
||||
pass
|
||||
```
|
||||
|
||||
### 3. 循环依赖检测
|
||||
|
||||
框架会自动检测循环依赖并抛出清晰的错误：
|
||||
|
||||
```python
|
||||
@service()
|
||||
class ServiceA:
|
||||
def __init__(self, service_b: ServiceB):
|
||||
pass
|
||||
|
||||
@service()
|
||||
class ServiceB:
|
||||
def __init__(self, service_a: ServiceA):
|
||||
pass
|
||||
```
|
||||
|
||||
错误信息：
|
||||
|
||||
```
|
||||
ValueError: 检测到循环依赖: service_a -> service_b -> service_a。
|
||||
请重构代码以消除循环依赖。
|
||||
```
|
||||
|
||||
### 4. 获取服务实例
|
||||
|
||||
在路由或其他地方获取服务实例：
|
||||
|
||||
```python
|
||||
from myboot.core.application import get_service
|
||||
|
||||
@get('/users/{user_id}')
|
||||
def get_user(user_id: int):
|
||||
user_service = get_service('user_service')
|
||||
return user_service.get_user(user_id)
|
||||
```
|
||||
|
||||
## 最佳实践
|
||||
|
||||
### 1. 使用类型注解
|
||||
|
||||
推荐使用类型注解声明依赖，代码更清晰：
|
||||
|
||||
```python
|
||||
# ✅ 推荐
|
||||
@service()
|
||||
class OrderService:
|
||||
def __init__(self, user_service: UserService, email_service: EmailService):
|
||||
self.user_service = user_service
|
||||
self.email_service = email_service
|
||||
|
||||
# ❌ 不推荐（需要手动获取）
|
||||
@service()
|
||||
class OrderService:
|
||||
def __init__(self):
|
||||
from myboot.core.application import get_service
|
||||
self.user_service = get_service('user_service')
|
||||
self.email_service = get_service('email_service')
|
||||
```
|
||||
|
||||
### 2. 避免循环依赖
|
||||
|
||||
设计服务时避免循环依赖：
|
||||
|
||||
```python
|
||||
# ✅ 好的设计
|
||||
@service()
|
||||
class UserService:
|
||||
def __init__(self, user_repo: UserRepository):
|
||||
self.user_repo = user_repo
|
||||
|
||||
@service()
|
||||
class OrderService:
|
||||
def __init__(self, user_service: UserService, order_repo: OrderRepository):
|
||||
self.user_service = user_service
|
||||
self.order_repo = order_repo
|
||||
|
||||
# ❌ 避免循环依赖
|
||||
@service()
|
||||
class ServiceA:
|
||||
def __init__(self, service_b: ServiceB):
|
||||
pass
|
||||
|
||||
@service()
|
||||
class ServiceB:
|
||||
def __init__(self, service_a: ServiceA):
|
||||
pass
|
||||
```
|
||||
|
||||
### 3. 使用接口而非具体实现
|
||||
|
||||
虽然 Python 没有接口，但可以通过抽象基类或协议定义接口：
|
||||
|
||||
```python
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
class IUserRepository(ABC):
|
||||
@abstractmethod
|
||||
def get_user(self, user_id: int):
|
||||
pass
|
||||
|
||||
@service()
|
||||
class UserRepository(IUserRepository):
|
||||
def get_user(self, user_id: int):
|
||||
return {"id": user_id}
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
def __init__(self, user_repo: IUserRepository):
|
||||
self.user_repo = user_repo
|
||||
```
|
||||
|
||||
### 4. 合理使用可选依赖
|
||||
|
||||
对于非必需的依赖，使用 `Optional`：
|
||||
|
||||
```python
|
||||
from typing import Optional
|
||||
|
||||
@service()
|
||||
class ProductService:
|
||||
def __init__(
|
||||
self,
|
||||
db: DatabaseClient, # 必需依赖
|
||||
cache: Optional[CacheService] = None # 可选依赖
|
||||
):
|
||||
self.db = db
|
||||
self.cache = cache
|
||||
```
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q1: 依赖注入失败怎么办？
|
||||
|
||||
如果依赖注入失败，框架会自动回退到传统方式（直接实例化）。检查日志中的错误信息：
|
||||
|
||||
1. **依赖的服务未注册**：确保依赖的服务已使用 `@service()` 装饰器
|
||||
2. **服务名不匹配**：检查服务名是否正确（类名转下划线命名）
|
||||
3. **循环依赖**：重构代码消除循环依赖
|
||||
|
||||
### Q2: 如何调试依赖关系？
|
||||
|
||||
框架会在日志中输出依赖关系信息：
|
||||
|
||||
```
|
||||
已注册服务提供者: user_service (依赖: set())
|
||||
已注册服务提供者: order_service (依赖: {'user_service', 'email_service'})
|
||||
```
|
||||
|
||||
### Q3: 可以在运行时动态获取服务吗？
|
||||
|
||||
可以，使用 `get_service()` 函数：
|
||||
|
||||
```python
|
||||
from myboot.core.application import get_service
|
||||
|
||||
def some_function():
|
||||
user_service = get_service('user_service')
|
||||
if user_service:
|
||||
# 使用服务
|
||||
pass
|
||||
```
|
||||
|
||||
### Q4: 支持异步服务吗？
|
||||
|
||||
目前依赖注入主要支持同步服务。对于异步服务，建议在服务内部处理异步逻辑。
|
||||
|
||||
### Q5: 如何测试带依赖的服务？
|
||||
|
||||
在测试中，可以手动创建服务实例并注入 mock 对象：
|
||||
|
||||
```python
|
||||
def test_order_service():
|
||||
# 创建 mock 依赖
|
||||
mock_user_service = MockUserService()
|
||||
mock_email_service = MockEmailService()
|
||||
|
||||
# 创建服务实例
|
||||
order_service = OrderService(mock_user_service, mock_email_service)
|
||||
|
||||
# 测试
|
||||
assert order_service is not None
|
||||
```
|
||||
|
||||
## 完整示例
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import service, get
|
||||
from myboot.core.application import get_service
|
||||
from typing import Optional
|
||||
|
||||
# 基础服务
|
||||
@service()
|
||||
class DatabaseClient:
|
||||
def __init__(self):
|
||||
self.connection = "connected"
|
||||
print("✅ DatabaseClient 已初始化")
|
||||
|
||||
@service()
|
||||
class CacheService:
|
||||
def __init__(self):
|
||||
self.cache = {}
|
||||
print("✅ CacheService 已初始化")
|
||||
|
||||
# 仓储层
|
||||
@service()
|
||||
class UserRepository:
|
||||
def __init__(self, db: DatabaseClient):
|
||||
self.db = db
|
||||
print("✅ UserRepository 已初始化（依赖: DatabaseClient）")
|
||||
|
||||
def find_by_id(self, user_id: int):
|
||||
return {"id": user_id, "name": f"用户{user_id}"}
|
||||
|
||||
# 服务层
|
||||
@service()
|
||||
class UserService:
|
||||
def __init__(
|
||||
self,
|
||||
user_repo: UserRepository,
|
||||
cache: Optional[CacheService] = None
|
||||
):
|
||||
self.user_repo = user_repo
|
||||
self.cache = cache
|
||||
print("✅ UserService 已初始化（依赖: UserRepository, CacheService）")
|
||||
|
||||
def get_user(self, user_id: int):
|
||||
# 尝试从缓存获取
|
||||
if self.cache and user_id in self.cache.cache:
|
||||
return self.cache.cache[user_id]
|
||||
|
||||
# 从数据库获取
|
||||
user = self.user_repo.find_by_id(user_id)
|
||||
|
||||
# 存入缓存
|
||||
if self.cache:
|
||||
self.cache.cache[user_id] = user
|
||||
|
||||
return user
|
||||
|
||||
# 路由层
|
||||
@get('/users/{user_id}')
|
||||
def get_user(user_id: int):
|
||||
user_service = get_service('user_service')
|
||||
return user_service.get_user(user_id)
|
||||
```
|
||||
|
||||
## 总结
|
||||
|
||||
依赖注入功能让您能够：
|
||||
|
||||
- ✅ 自动管理服务依赖关系
|
||||
- ✅ 无需手动获取和传递依赖
|
||||
- ✅ 支持多级依赖和可选依赖
|
||||
- ✅ 自动检测循环依赖
|
||||
- ✅ 支持 Client 注入到 Service 和 Controller
|
||||
- ✅ 支持 Component 组件，可包含定时任务
|
||||
- ✅ 支持多种依赖查找方式（名称、类型）
|
||||
- ✅ 保持向后兼容，现有代码无需修改
|
||||
|
||||
开始使用依赖注入，让代码更加清晰和可维护！
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
# Prometheus Metrics 指南
|
||||
|
||||
myboot 0.2.0 起内置 Prometheus 指标支持：零代码获得 `/metrics` 端点与 HTTP
|
||||
请求指标，多 worker 模式下自动聚合全部进程的指标，业务代码可用轻量 API 记录
|
||||
自定义指标。
|
||||
|
||||
## 1. 开启
|
||||
|
||||
```bash
|
||||
pip install myboot[metrics] # prometheus-client 为可选依赖
|
||||
```
|
||||
|
||||
```yaml
|
||||
# conf/config.yaml
|
||||
metrics:
|
||||
enabled: true # 默认 false，不开启则一切如旧
|
||||
```
|
||||
|
||||
不需要在 main.py 写任何 metrics 代码——框架自动完成 `/metrics` 挂载、HTTP
|
||||
中间件注册、多进程聚合配置与进程退出清理。
|
||||
|
||||
> `metrics.enabled: true` 但未安装 prometheus-client 时只打 warning，
|
||||
> 应用照常启动（指标功能禁用）。
|
||||
|
||||
## 2. 配置项
|
||||
|
||||
| 配置 | 默认 | 说明 |
|
||||
|---|---|---|
|
||||
| `metrics.enabled` | `false` | 总开关 |
|
||||
| `metrics.path` | `/metrics` | 暴露端点路径 |
|
||||
| `metrics.http_metrics` | `true` | 是否注册内置 HTTP 请求指标中间件 |
|
||||
| `metrics.multiproc_dir` | 自动（系统临时目录） | 多进程指标共享目录，一般无需配置 |
|
||||
|
||||
## 3. 内置 HTTP 指标
|
||||
|
||||
开启后自动采集（无需写代码）：
|
||||
|
||||
```
|
||||
# 请求计数（按方法、路由模板、状态码）
|
||||
myboot_http_requests_total{method="POST", path="/api/items/{id}", status="200"}
|
||||
|
||||
# 请求延迟直方图
|
||||
myboot_http_request_duration_seconds_bucket{method="POST", path="/api/items/{id}", le="0.1"}
|
||||
```
|
||||
|
||||
- `path` 标签使用**路由模板**（`/api/items/{id}`），而非真实 URL
|
||||
（`/api/items/123`），避免标签基数爆炸；
|
||||
- 未匹配任何路由的请求（如 404）统一归入 `path="unmatched"`；
|
||||
- `/metrics` 端点自身不计入统计。
|
||||
|
||||
## 4. 自定义指标 API
|
||||
|
||||
```python
|
||||
from myboot.metrics import get_counter, get_histogram, observe_stage, time_stage
|
||||
|
||||
# 计数器（同名重复调用返回同一对象，无需担心重复注册）
|
||||
cache_miss = get_counter("cache_miss_total", "缓存未命中数", ["source"])
|
||||
cache_miss.labels(source="redis").inc()
|
||||
|
||||
# 直方图
|
||||
latency = get_histogram("external_api_seconds", "外部接口耗时", ["api"])
|
||||
latency.labels(api="item-feature").observe(0.123)
|
||||
|
||||
# 阶段计时：内置 myboot_stage_duration_seconds{stage} 直方图的便捷封装
|
||||
with time_stage("sasrec"):
|
||||
candidates = model.infer(request)
|
||||
|
||||
with time_stage("rerank"):
|
||||
result = reranker.score(candidates)
|
||||
|
||||
# 或手动上报耗时（秒）
|
||||
observe_stage("persist", 0.05)
|
||||
```
|
||||
|
||||
重要特性：**未开启 `metrics.enabled` 或未安装 prometheus-client 时，以上调用
|
||||
全部是静默 no-op**——业务代码不需要任何条件判断，开发环境关掉指标也不会报错。
|
||||
|
||||
## 5. 多 Worker 聚合原理
|
||||
|
||||
多进程下 Prometheus 官方方案要求：
|
||||
|
||||
1. `PROMETHEUS_MULTIPROC_DIR` 必须在 `prometheus_client` 首次 import **之前**设置；
|
||||
2. 各 worker 把指标值写入该目录的 mmap 文件，`/metrics` 端点用
|
||||
`MultiProcessCollector` 聚合读取；
|
||||
3. worker 退出时调用 `mark_process_dead` 清理 gauge 类残留。
|
||||
|
||||
这三步时序极易写错，myboot 已全部内置：环境变量在 `Application.__init__`
|
||||
阶段设置（fork 子进程继承、spawn 子进程重新执行 `__init__`，两种模式都来得及），
|
||||
陈旧文件由父进程启动时清理，`mark_process_dead` 挂在 lifespan 关闭末尾。
|
||||
|
||||
唯一注意事项：**不要在 `create_app()` 之前 import prometheus_client**
|
||||
（框架检测到会打 warning）。
|
||||
|
||||
### Windows 限制
|
||||
|
||||
Windows 多 worker（spawn + terminate）下 multiproc 模式不可靠，框架自动降级为
|
||||
各进程独立指标：`/metrics` 只反映处理该次请求的 worker。Linux/macOS 生产部署
|
||||
不受影响；Windows 单 worker 完全正常。
|
||||
|
||||
## 6. Prometheus 抓取配置
|
||||
|
||||
```yaml
|
||||
scrape_configs:
|
||||
- job_name: my-app
|
||||
scrape_interval: 15s
|
||||
static_configs:
|
||||
- targets: ["my-app:8000"] # 抓取到的即全 worker 聚合值
|
||||
```
|
||||
|
||||
## 7. 常用查询示例（PromQL）
|
||||
|
||||
```promql
|
||||
# QPS（按路由）
|
||||
sum(rate(myboot_http_requests_total[1m])) by (path)
|
||||
|
||||
# P99 延迟
|
||||
histogram_quantile(0.99, sum(rate(myboot_http_request_duration_seconds_bucket[5m])) by (path, le))
|
||||
|
||||
# 错误率
|
||||
sum(rate(myboot_http_requests_total{status=~"5.."}[5m]))
|
||||
/ sum(rate(myboot_http_requests_total[5m]))
|
||||
|
||||
# 自定义阶段耗时 P95
|
||||
histogram_quantile(0.95, sum(rate(myboot_stage_duration_seconds_bucket[5m])) by (stage, le))
|
||||
```
|
||||
@@ -0,0 +1,136 @@
|
||||
# 多 Worker 模式指南
|
||||
|
||||
myboot 0.2.0 起，多 worker（`workers > 1`）模式下**每个 worker 进程独立完成组件实例化**：
|
||||
client / service / controller 都在各自的 worker 进程内创建，不再共享父进程 fork 前的实例。
|
||||
本文是多 worker 相关能力的一站式说明。
|
||||
|
||||
## 启动方式
|
||||
|
||||
```python
|
||||
# main.py
|
||||
app = create_app(name="my-app", auto_discover_package="app")
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run(workers=4, app_path="main:app.get_fastapi_app()")
|
||||
```
|
||||
|
||||
```yaml
|
||||
# conf/config.yaml 亦可
|
||||
server:
|
||||
workers: 4
|
||||
```
|
||||
|
||||
无需再手动调用 `auto_discover()` / `apply_auto_configuration()`——框架在每个
|
||||
worker 内自动完成引导（`bootstrap_worker()`）。
|
||||
|
||||
## Worker 信息
|
||||
|
||||
```python
|
||||
app.worker_id # 当前 worker ID，从 1 开始；单进程为 1
|
||||
app.worker_count # worker 总数
|
||||
app.is_primary_worker # 是否主 worker（适合"只跑一份"的逻辑）
|
||||
```
|
||||
|
||||
## Client 生命周期
|
||||
|
||||
**建连可以直接写在 `__init__` 里**——0.2.0 起 client 在各 worker 内实例化，
|
||||
不存在 fork 共享连接问题：
|
||||
|
||||
```python
|
||||
@client()
|
||||
class RedisClient:
|
||||
def __init__(self):
|
||||
self.conn = redis.Redis(...) # 安全：本 worker 进程内新建
|
||||
|
||||
def close(self): # 可选：定义即获得自动清理
|
||||
self.conn.close()
|
||||
```
|
||||
|
||||
定义了 `close()` 方法（同步或 async）的 client，框架会在 worker 停止时
|
||||
（`worker_stop_hooks` 之后、`shutdown_hooks` 之前）自动调用。若你也在自己的
|
||||
shutdown 钩子里关闭，建议把 `close()` 实现为幂等——框架的兜底调用会把二次
|
||||
close 的异常降为 warning，不影响关闭流程。
|
||||
|
||||
## Worker 生命周期钩子
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import on_worker_start, on_worker_stop
|
||||
|
||||
@on_worker_start
|
||||
def warm_up():
|
||||
"""每个 worker 启动时各触发一次（startup_hooks 之后、调度器启动之前）"""
|
||||
|
||||
@on_worker_stop(order=1)
|
||||
def cleanup():
|
||||
"""每个 worker 停止时各触发一次（调度器停止之后）"""
|
||||
```
|
||||
|
||||
> Windows 限制：多 worker 下父进程以 `terminate()` 硬终止 worker，
|
||||
> `@on_worker_stop` 与 client 自动 close 在 Windows 上不保证执行；
|
||||
> 关键清理逻辑不要只依赖它们。
|
||||
|
||||
## Primary-first 初始化协调：`run_primary_first`
|
||||
|
||||
典型场景：4 个 worker 都去下载同一个 3GB 模型是浪费——应该 primary 下载，
|
||||
其余 worker 等它完成后直接从本地加载：
|
||||
|
||||
```python
|
||||
from myboot.utils import run_primary_first
|
||||
from myboot.core.decorators import on_worker_start
|
||||
|
||||
@on_worker_start
|
||||
def load_model():
|
||||
run_primary_first(
|
||||
"sasrec-model", # 协调标识（同一标识共享一次协调）
|
||||
primary_fn=download_and_load, # 仅 primary 执行
|
||||
secondary_fn=load_from_local, # 其余 worker 等 primary 完成后执行
|
||||
timeout=600, # 等待上限（秒）
|
||||
)
|
||||
```
|
||||
|
||||
语义保证：
|
||||
|
||||
- `secondary_fn` 缺省时其余 worker 也执行 `primary_fn`（即"等完再做同样的事"）；
|
||||
- primary 失败 → 其余 worker 立即抛 `RuntimeError`（含原始异常信息），不会傻等到超时；
|
||||
- 等待超时 → `TimeoutError`；
|
||||
- 单进程模式退化为直接调用 `primary_fn()`，代码无需分支；
|
||||
- 基于临时目录标记文件实现，跨 spawn/fork、跨平台（不依赖 fcntl），
|
||||
但**不支持跨机器**协调。
|
||||
|
||||
## 定时任务与多 worker：任务级 `all_workers`
|
||||
|
||||
默认所有定时任务**只在 primary worker 执行一份**（防止重复）。但"刷新本 worker
|
||||
内存态"的任务必须每个 worker 各跑一份，否则其余 worker 永远用旧数据：
|
||||
|
||||
```python
|
||||
@component()
|
||||
class RefreshJobs:
|
||||
@interval(hours=24, all_workers=True) # 每个 worker 各执行
|
||||
def refresh_local_cache(self):
|
||||
self.items = load_items() # 刷新的是本进程内存
|
||||
|
||||
@cron("0 2 * * *") # 默认：仅 primary 执行一份
|
||||
def daily_report(self):
|
||||
send_report()
|
||||
```
|
||||
|
||||
怎么选：
|
||||
|
||||
| 任务类型 | 配置 | 原因 |
|
||||
|---|---|---|
|
||||
| 刷新进程内缓存/内存清单 | `all_workers=True` | 数据在各 worker 自己内存里 |
|
||||
| 发报表 / 写数据库 / 调外部 API | 默认 | 跑多份会重复副作用 |
|
||||
|
||||
全局开关 `scheduler.on_all_workers: true`（所有任务都在全部 worker 跑）仍然有效，
|
||||
但通常任务级参数是更对的粒度。`scheduler.enabled: false` 全局禁用一切定时任务。
|
||||
|
||||
## 服务作用域 `scope`
|
||||
|
||||
```python
|
||||
@service() # 默认 singleton：每个 worker 进程内一个实例
|
||||
@service(scope="request") # 每个请求（asyncio 任务上下文）一个实例
|
||||
@service(scope="factory") # 每次注入解析都创建新实例
|
||||
```
|
||||
|
||||
`request` 作用域适合存放请求级上下文，避免单例服务里的共享可变状态。
|
||||
`@client` 同样支持 `scope` 参数。
|
||||
@@ -0,0 +1,162 @@
|
||||
# PyPI 自动发布配置指南
|
||||
|
||||
本文档说明如何配置 GitHub Actions 自动发布到 PyPI。
|
||||
|
||||
## 工作流说明
|
||||
|
||||
项目已配置 GitHub Actions 工作流 (`.github/workflows/publish.yml`)，当创建新的 tag（以 `v` 开头，如 `v1.0.0`）时，会自动构建并发布到 PyPI。
|
||||
|
||||
## 配置 Trusted Publishing（推荐方式）
|
||||
|
||||
Trusted Publishing 是 PyPI 推荐的安全发布方式，无需使用 API token，通过 OIDC 进行身份验证。
|
||||
|
||||
### 步骤 1: 在 PyPI 上配置 Trusted Publisher
|
||||
|
||||
1. 登录 [PyPI](https://pypi.org/)
|
||||
2. 进入您的项目页面
|
||||
3. 点击左侧菜单的 **"Publishing"** 或 **"Manage"** → **"Publishing"**
|
||||
4. 在 **"Trusted publishers"** 部分，点击 **"Add"**
|
||||
5. 填写以下信息：
|
||||
- **PyPI project name**: `myboot`（您的项目名称）
|
||||
- **Publisher name**: 自定义名称，如 `github-actions`
|
||||
- **Workflow filename**: `publish.yml`
|
||||
- **Environment name**: 留空（或填写特定环境名称）
|
||||
- **GitHub Owner**: 您的 GitHub 用户名或组织名
|
||||
- **GitHub Repository**: `myboot`（您的仓库名）
|
||||
- **Workflow filename**: `publish.yml`
|
||||
6. 点击 **"Add"** 保存
|
||||
|
||||
### 步骤 2: 验证配置
|
||||
|
||||
配置完成后，当您创建新的 tag 时，GitHub Actions 会自动触发发布流程：
|
||||
|
||||
```bash
|
||||
# 创建并推送 tag
|
||||
git tag v1.0.0
|
||||
git push origin v1.0.0
|
||||
```
|
||||
|
||||
工作流会自动：
|
||||
|
||||
1. 检出代码
|
||||
2. 安装构建依赖
|
||||
3. 构建分发包（wheel 和 sdist）
|
||||
4. 使用 trusted publishing 发布到 PyPI
|
||||
|
||||
## 使用 API Token 方式（备选）
|
||||
|
||||
如果您不想使用 trusted publishing，也可以使用传统的 API token 方式：
|
||||
|
||||
### 步骤 1: 创建 PyPI API Token
|
||||
|
||||
1. 登录 [PyPI](https://pypi.org/)
|
||||
2. 进入 **"Account settings"** → **"API tokens"**
|
||||
3. 点击 **"Add API token"**
|
||||
4. 填写：
|
||||
- **Token name**: 如 `github-actions-publish`
|
||||
- **Scope**: 选择 **"Project: myboot"**（项目范围）或 **"Entire account"**（账户范围）
|
||||
5. 复制生成的 token（只显示一次，请妥善保存）
|
||||
|
||||
### 步骤 2: 在 GitHub 中配置 Secret
|
||||
|
||||
1. 进入您的 GitHub 仓库
|
||||
2. 点击 **"Settings"** → **"Secrets and variables"** → **"Actions"**
|
||||
3. 点击 **"New repository secret"**
|
||||
4. 填写：
|
||||
- **Name**: `PYPI_API_TOKEN`
|
||||
- **Secret**: 粘贴刚才复制的 API token
|
||||
5. 点击 **"Add secret"**
|
||||
|
||||
### 步骤 3: 修改工作流文件
|
||||
|
||||
修改 `.github/workflows/publish.yml`，添加 `password` 参数：
|
||||
|
||||
```yaml
|
||||
- name: Publish to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_API_TOKEN }}
|
||||
print-hash: true
|
||||
```
|
||||
|
||||
**注意**: 使用 API token 会禁用 trusted publishing，但两种方式不能同时使用。
|
||||
|
||||
## Tag 命名规则
|
||||
|
||||
工作流配置为匹配以 `v` 开头的 tag：
|
||||
|
||||
- ✅ `v1.0.0` - 匹配
|
||||
- ✅ `v0.1.0` - 匹配
|
||||
- ✅ `v2.3.4` - 匹配
|
||||
- ❌ `1.0.0` - 不匹配（缺少 `v` 前缀）
|
||||
- ❌ `release-1.0.0` - 不匹配
|
||||
|
||||
## 发布流程
|
||||
|
||||
1. **更新版本号**: 在 `pyproject.toml` 中更新 `version` 字段
|
||||
2. **提交更改**: 提交并推送代码到仓库
|
||||
3. **创建 Tag**: 创建并推送以 `v` 开头的 tag
|
||||
4. **自动发布**: GitHub Actions 会自动触发发布流程
|
||||
|
||||
```bash
|
||||
# 示例流程
|
||||
# 1. 更新 pyproject.toml 中的版本号
|
||||
# version = "1.0.0"
|
||||
|
||||
# 2. 提交更改
|
||||
git add pyproject.toml
|
||||
git commit -m "Bump version to 1.0.0"
|
||||
git push
|
||||
|
||||
# 3. 创建并推送 tag
|
||||
git tag v1.0.0
|
||||
git push origin v1.0.0
|
||||
```
|
||||
|
||||
## 验证发布
|
||||
|
||||
发布完成后，您可以：
|
||||
|
||||
1. 在 [PyPI 项目页面](https://pypi.org/project/myboot/) 查看新版本
|
||||
2. 使用 pip 安装测试：
|
||||
```bash
|
||||
pip install myboot==1.0.0
|
||||
```
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 问题 1: Trusted Publishing 配置失败
|
||||
|
||||
**错误信息**: `403 Client Error: Invalid or non-existent authentication information`
|
||||
|
||||
**解决方案**:
|
||||
|
||||
- 检查 PyPI 上的 trusted publisher 配置是否正确
|
||||
- 确认 GitHub 仓库名称、工作流文件名等信息匹配
|
||||
- 确保工作流文件中的 `permissions` 包含 `id-token: write`
|
||||
|
||||
### 问题 2: 版本已存在
|
||||
|
||||
**错误信息**: `File already exists`
|
||||
|
||||
**解决方案**:
|
||||
|
||||
- 检查 PyPI 上是否已存在该版本
|
||||
- 如果确实需要重新发布，需要先删除 PyPI 上的版本（不推荐）
|
||||
- 或者使用新的版本号
|
||||
|
||||
### 问题 3: 构建失败
|
||||
|
||||
**错误信息**: 构建步骤失败
|
||||
|
||||
**解决方案**:
|
||||
|
||||
- 检查 `pyproject.toml` 配置是否正确
|
||||
- 确认所有必需的文件都已包含在分发包中
|
||||
- 查看 GitHub Actions 日志获取详细错误信息
|
||||
|
||||
## 参考资源
|
||||
|
||||
- [PyPI Trusted Publishers 文档](https://docs.pypi.org/trusted-publishers/)
|
||||
- [pypa/gh-action-pypi-publish 项目](https://github.com/pypa/gh-action-pypi-publish)
|
||||
- [PyPI 发布指南](https://packaging.python.org/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/)
|
||||
@@ -0,0 +1,505 @@
|
||||
# REST API 中使用异步任务
|
||||
|
||||
在 REST API 中，当需要执行耗时操作时，可以使用异步任务来避免阻塞请求响应。MyBoot 提供了多种方式来在 REST API 中使用异步任务。
|
||||
|
||||
## 目录
|
||||
|
||||
- [快速启动后台任务](#快速启动后台任务)
|
||||
- [使用 ScheduledJob](#使用-scheduledjob)
|
||||
- [异步路由处理](#异步路由处理)
|
||||
- [任务状态查询](#任务状态查询)
|
||||
- [完整示例](#完整示例)
|
||||
|
||||
## 快速启动后台任务
|
||||
|
||||
使用 `async_run` 函数可以快速在后台启动异步任务，适用于不需要跟踪任务状态的场景。
|
||||
|
||||
### 基本用法
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import post, rest_controller
|
||||
from myboot.utils.async_utils import async_run
|
||||
import time
|
||||
|
||||
def process_data(data: dict):
|
||||
"""耗时的数据处理任务"""
|
||||
print(f"开始处理数据: {data}")
|
||||
time.sleep(5) # 模拟耗时操作
|
||||
print(f"数据处理完成: {data}")
|
||||
return {"processed": True, "data": data}
|
||||
|
||||
@rest_controller('/api/tasks')
|
||||
class TaskController:
|
||||
"""任务控制器"""
|
||||
|
||||
@post('/process')
|
||||
def create_process_task(self, data: dict):
|
||||
"""创建数据处理任务"""
|
||||
# 立即返回，任务在后台执行
|
||||
task = async_run(process_data, data, task_name="数据处理任务")
|
||||
|
||||
return {
|
||||
"message": "任务已创建，正在后台处理",
|
||||
"task_id": str(id(task)),
|
||||
"status": "pending"
|
||||
}
|
||||
```
|
||||
|
||||
### 带参数的任务
|
||||
|
||||
```python
|
||||
from myboot.utils.async_utils import async_run
|
||||
|
||||
def send_email(to: str, subject: str, content: str):
|
||||
"""发送邮件任务"""
|
||||
print(f"发送邮件到 {to}: {subject}")
|
||||
# 模拟邮件发送
|
||||
time.sleep(2)
|
||||
return {"sent": True, "to": to}
|
||||
|
||||
@post('/api/emails')
|
||||
def send_email_async(to: str, subject: str, content: str):
|
||||
"""异步发送邮件"""
|
||||
# 启动后台任务
|
||||
async_run(send_email, to, subject, content, task_name=f"发送邮件给{to}")
|
||||
|
||||
return {
|
||||
"message": "邮件发送任务已创建",
|
||||
"recipient": to
|
||||
}
|
||||
```
|
||||
|
||||
## 使用 ScheduledJob
|
||||
|
||||
对于需要跟踪和管理任务状态的场景，建议使用 `ScheduledJob`。
|
||||
|
||||
### 使用 ScheduledJob
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import post, get, rest_controller
|
||||
from myboot.jobs.scheduled_job import ScheduledJob
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
import time
|
||||
|
||||
@rest_controller('/api/reports')
|
||||
class ReportController:
|
||||
"""报告控制器"""
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = get_scheduler()
|
||||
|
||||
@post('/generate')
|
||||
def generate_report_task(self, report_type: str, filters: dict = None):
|
||||
"""创建报告生成任务"""
|
||||
# 创建自定义 ScheduledJob
|
||||
class ReportJob(ScheduledJob):
|
||||
def __init__(self, report_type: str, filters: dict):
|
||||
super().__init__(
|
||||
name=f"生成{report_type}报告",
|
||||
description=f"生成类型为 {report_type} 的报告",
|
||||
max_retries=3,
|
||||
timeout=300 # 5分钟超时
|
||||
)
|
||||
self.report_type = report_type
|
||||
self.filters = filters or {}
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
"""生成报告任务"""
|
||||
print(f"开始生成 {self.report_type} 报告")
|
||||
time.sleep(10) # 模拟报告生成
|
||||
return {
|
||||
"type": self.report_type,
|
||||
"filters": self.filters,
|
||||
"status": "completed"
|
||||
}
|
||||
|
||||
# 创建任务实例
|
||||
job = ReportJob(report_type, filters)
|
||||
|
||||
# 添加到调度器（用于状态跟踪，非定时任务）
|
||||
job_id = self.scheduler.add_job_object(job)
|
||||
|
||||
# 在后台执行任务
|
||||
import threading
|
||||
thread = threading.Thread(target=job.execute)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
return {
|
||||
"message": "报告生成任务已创建",
|
||||
"job_id": job_id,
|
||||
"status": "pending"
|
||||
}
|
||||
|
||||
@get('/status/{job_id}')
|
||||
def get_report_status(self, job_id: str):
|
||||
"""查询任务状态"""
|
||||
job = self.scheduler.get_scheduled_job(job_id)
|
||||
|
||||
if not job:
|
||||
return {
|
||||
"error": "任务不存在"
|
||||
}
|
||||
|
||||
job_info = job.get_info()
|
||||
return {
|
||||
"job_id": job_id,
|
||||
"status": job_info["status"],
|
||||
"progress": self._calculate_progress(job_info),
|
||||
"created_at": job_info["created_at"],
|
||||
"started_at": job_info["started_at"],
|
||||
"completed_at": job_info["completed_at"]
|
||||
}
|
||||
|
||||
def _calculate_progress(self, job_info: dict) -> float:
|
||||
"""计算任务进度（示例）"""
|
||||
if job_info["status"] == "completed":
|
||||
return 100.0
|
||||
elif job_info["status"] == "running":
|
||||
# 可以根据实际业务逻辑计算进度
|
||||
return 50.0
|
||||
else:
|
||||
return 0.0
|
||||
```
|
||||
|
||||
### 使用自定义 ScheduledJob 类
|
||||
|
||||
```python
|
||||
from myboot.jobs.scheduled_job import ScheduledJob
|
||||
from myboot.core.decorators import post, get, rest_controller
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
|
||||
class DataImportJob(ScheduledJob):
|
||||
"""数据导入任务"""
|
||||
|
||||
def __init__(self, file_path: str, **kwargs):
|
||||
super().__init__(
|
||||
name="数据导入",
|
||||
description=f"从文件 {file_path} 导入数据",
|
||||
**kwargs
|
||||
)
|
||||
self.file_path = file_path
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
"""执行数据导入"""
|
||||
import time
|
||||
print(f"开始导入文件: {self.file_path}")
|
||||
|
||||
# 模拟数据导入过程
|
||||
for i in range(10):
|
||||
time.sleep(1)
|
||||
print(f"导入进度: {(i+1)*10}%")
|
||||
|
||||
return {
|
||||
"file_path": self.file_path,
|
||||
"records_imported": 1000,
|
||||
"status": "completed"
|
||||
}
|
||||
|
||||
@rest_controller('/api/import')
|
||||
class ImportController:
|
||||
"""数据导入控制器"""
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = get_scheduler()
|
||||
|
||||
@post('/start')
|
||||
def start_import(self, file_path: str):
|
||||
"""启动数据导入任务"""
|
||||
job = DataImportJob(file_path)
|
||||
|
||||
# 添加到调度器（用于状态跟踪，非定时任务）
|
||||
job_id = self.scheduler.add_job_object(job)
|
||||
|
||||
# 在后台执行
|
||||
import threading
|
||||
thread = threading.Thread(target=job.execute)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
return {
|
||||
"message": "数据导入任务已启动",
|
||||
"job_id": job_id,
|
||||
"file_path": file_path
|
||||
}
|
||||
|
||||
@get('/jobs')
|
||||
def list_jobs(self):
|
||||
"""列出所有任务"""
|
||||
# 获取所有 ScheduledJob 对象
|
||||
jobs = self.scheduler.get_all_scheduled_jobs()
|
||||
all_jobs = [job.get_info() for job in jobs]
|
||||
|
||||
return {
|
||||
"jobs": all_jobs,
|
||||
"total": len(all_jobs)
|
||||
}
|
||||
```
|
||||
|
||||
## 任务状态查询
|
||||
|
||||
### 使用调度器查询
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import get, rest_controller
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
|
||||
@rest_controller('/api/jobs')
|
||||
class JobStatusController:
|
||||
"""任务状态控制器"""
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = get_scheduler()
|
||||
|
||||
@get('/{job_id}')
|
||||
def get_job_status(self, job_id: str):
|
||||
"""获取任务状态"""
|
||||
job = self.scheduler.get_scheduled_job(job_id)
|
||||
|
||||
if not job:
|
||||
return {
|
||||
"error": "任务不存在"
|
||||
}
|
||||
|
||||
return job.get_info()
|
||||
|
||||
@get('/')
|
||||
def list_all_jobs(self):
|
||||
"""列出所有任务"""
|
||||
# 获取所有 ScheduledJob 对象
|
||||
jobs = self.scheduler.get_all_scheduled_jobs()
|
||||
all_jobs = [job.get_info() for job in jobs]
|
||||
|
||||
# 计算统计信息
|
||||
total = len(all_jobs)
|
||||
running = sum(1 for j in all_jobs if j["status"] == "running")
|
||||
completed = sum(1 for j in all_jobs if j["status"] == "completed")
|
||||
failed = sum(1 for j in all_jobs if j["status"] == "failed")
|
||||
|
||||
statistics = {
|
||||
"total": total,
|
||||
"running": running,
|
||||
"completed": completed,
|
||||
"failed": failed,
|
||||
"success_rate": completed / total if total > 0 else 0
|
||||
}
|
||||
|
||||
return {
|
||||
"jobs": all_jobs,
|
||||
"statistics": statistics
|
||||
}
|
||||
|
||||
@get('/statistics')
|
||||
def get_statistics(self):
|
||||
"""获取任务统计信息"""
|
||||
# 获取所有 ScheduledJob 对象
|
||||
jobs = self.scheduler.get_all_scheduled_jobs()
|
||||
all_jobs = [job.get_info() for job in jobs]
|
||||
|
||||
total = len(all_jobs)
|
||||
running = sum(1 for j in all_jobs if j["status"] == "running")
|
||||
completed = sum(1 for j in all_jobs if j["status"] == "completed")
|
||||
failed = sum(1 for j in all_jobs if j["status"] == "failed")
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"running": running,
|
||||
"completed": completed,
|
||||
"failed": failed,
|
||||
"success_rate": completed / total if total > 0 else 0
|
||||
}
|
||||
```
|
||||
|
||||
以下是一个完整的示例，展示如何在 REST API 中实现文件上传和异步处理：
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import post, get, rest_controller
|
||||
from myboot.jobs.scheduled_job import ScheduledJob
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
from myboot.utils.async_utils import async_run
|
||||
import time
|
||||
import uuid
|
||||
|
||||
@rest_controller('/api/files')
|
||||
class FileController:
|
||||
"""文件处理控制器"""
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = get_scheduler()
|
||||
self._file_storage = {} # 简单的存储，实际应使用数据库
|
||||
|
||||
@post('/upload')
|
||||
def upload_file(self, file_path: str, options: dict = None):
|
||||
"""上传文件并创建处理任务"""
|
||||
# 生成任务 ID
|
||||
task_id = str(uuid.uuid4())
|
||||
|
||||
# 创建自定义 ScheduledJob
|
||||
class FileProcessJob(ScheduledJob):
|
||||
def __init__(self, file_path: str, options: dict, task_id: str):
|
||||
super().__init__(
|
||||
name=f"处理文件-{task_id}",
|
||||
description=f"处理上传的文件: {file_path}",
|
||||
max_retries=3,
|
||||
timeout=600 # 10分钟超时
|
||||
)
|
||||
self.file_path = file_path
|
||||
self.options = options or {}
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
"""处理上传的文件"""
|
||||
print(f"开始处理文件: {self.file_path}")
|
||||
|
||||
# 模拟文件处理过程
|
||||
for i in range(20):
|
||||
time.sleep(0.5)
|
||||
print(f"处理进度: {(i+1)*5}%")
|
||||
|
||||
return {
|
||||
"file_path": self.file_path,
|
||||
"processed": True,
|
||||
"records": 1000,
|
||||
"options": self.options
|
||||
}
|
||||
|
||||
# 创建处理任务
|
||||
job = FileProcessJob(file_path, options, task_id)
|
||||
|
||||
# 添加到调度器（用于状态跟踪，非定时任务）
|
||||
job_id = self.scheduler.add_job_object(job)
|
||||
|
||||
# 保存文件信息
|
||||
self._file_storage[task_id] = {
|
||||
"job_id": job_id,
|
||||
"file_path": file_path,
|
||||
"status": "pending",
|
||||
"created_at": time.time()
|
||||
}
|
||||
|
||||
# 在后台执行任务
|
||||
import threading
|
||||
thread = threading.Thread(target=self._execute_job, args=(job, task_id))
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
return {
|
||||
"message": "文件上传成功，处理任务已创建",
|
||||
"task_id": task_id,
|
||||
"job_id": job_id,
|
||||
"status": "pending"
|
||||
}
|
||||
|
||||
def _execute_job(self, job, task_id: str):
|
||||
"""执行任务并更新状态"""
|
||||
try:
|
||||
result = job.execute()
|
||||
self._file_storage[task_id]["status"] = "completed"
|
||||
self._file_storage[task_id]["result"] = result
|
||||
except Exception as e:
|
||||
self._file_storage[task_id]["status"] = "failed"
|
||||
self._file_storage[task_id]["error"] = str(e)
|
||||
|
||||
@get('/status/{task_id}')
|
||||
def get_file_status(self, task_id: str):
|
||||
"""查询文件处理状态"""
|
||||
if task_id not in self._file_storage:
|
||||
return {
|
||||
"error": "任务不存在"
|
||||
}
|
||||
|
||||
file_info = self._file_storage[task_id]
|
||||
job = self.scheduler.get_scheduled_job(file_info["job_id"])
|
||||
job_info = job.get_info() if job else None
|
||||
|
||||
return {
|
||||
"task_id": task_id,
|
||||
"file_path": file_info["file_path"],
|
||||
"status": file_info.get("status", "unknown"),
|
||||
"job_info": job_info,
|
||||
"result": file_info.get("result"),
|
||||
"error": file_info.get("error")
|
||||
}
|
||||
|
||||
@get('/tasks')
|
||||
def list_tasks(self):
|
||||
"""列出所有文件处理任务"""
|
||||
return {
|
||||
"tasks": list(self._file_storage.values()),
|
||||
"total": len(self._file_storage)
|
||||
}
|
||||
```
|
||||
|
||||
## 最佳实践
|
||||
|
||||
### 1. 选择合适的异步方式
|
||||
|
||||
- **简单任务，无需跟踪**：使用 `async_run`
|
||||
- **需要跟踪状态**：使用 `ScheduledJob`（继承并实现 `run` 方法）
|
||||
- **需要定时执行**：使用 `@component` + `@cron`/`@interval`/`@once` 装饰器
|
||||
|
||||
### 2. 任务超时设置
|
||||
|
||||
任务超时功能支持跨平台（Windows、Linux、macOS），使用 `ThreadPoolExecutor` 实现：
|
||||
|
||||
```python
|
||||
class MyTask(ScheduledJob):
|
||||
def __init__(self):
|
||||
super().__init__(
|
||||
name="我的任务",
|
||||
timeout=300 # 设置5分钟超时
|
||||
)
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
# 任务逻辑
|
||||
pass
|
||||
```
|
||||
|
||||
**注意**：
|
||||
- 超时功能在 Windows、Linux 和 macOS 上均可正常工作
|
||||
- 超时后会抛出 `TimeoutError` 异常
|
||||
- 由于 Python GIL 的限制，超时后任务线程可能仍在后台运行，但不会再等待其结果
|
||||
|
||||
### 3. 错误处理和重试
|
||||
|
||||
```python
|
||||
class MyTask(ScheduledJob):
|
||||
def __init__(self):
|
||||
super().__init__(
|
||||
name="我的任务",
|
||||
max_retries=3,
|
||||
retry_delay=5.0 # 失败后等待5秒再重试
|
||||
)
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
# 任务逻辑
|
||||
pass
|
||||
```
|
||||
|
||||
### 4. 资源清理
|
||||
|
||||
```python
|
||||
from myboot.utils.async_utils import cleanup_async_executor
|
||||
|
||||
# 在应用关闭时清理
|
||||
@app.add_shutdown_hook
|
||||
def shutdown_hook():
|
||||
cleanup_async_executor()
|
||||
```
|
||||
|
||||
### 5. 任务状态管理
|
||||
|
||||
建议使用数据库或 Redis 来持久化任务状态，而不是内存存储。
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. **线程安全**：`Scheduler` 是线程安全的，可以在多个线程中使用
|
||||
2. **任务执行**：使用 `threading.Thread` 在后台执行任务，避免阻塞主线程
|
||||
3. **资源管理**：长时间运行的应用应定期清理已完成的任务
|
||||
4. **错误处理**：确保任务函数有适当的错误处理，避免任务失败影响系统
|
||||
5. **ScheduledJob 使用**：对于非定时任务，可以直接创建 `ScheduledJob` 实例并执行，无需添加到调度器
|
||||
|
||||
## 相关文档
|
||||
|
||||
- [异步工具使用指南](../myboot/utils/async_utils.py)
|
||||
- [调度器文档](../myboot/core/scheduler.py)
|
||||
- [ScheduledJob 基类文档](../myboot/jobs/scheduled_job.py)
|
||||
@@ -0,0 +1,263 @@
|
||||
# REST API 统一响应格式
|
||||
|
||||
MyBoot 框架提供了统一的 REST API 响应格式封装，确保所有 API 返回一致的格式。
|
||||
|
||||
## 响应格式结构
|
||||
|
||||
### 成功响应
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"code": 200,
|
||||
"message": "操作成功",
|
||||
"data": {
|
||||
// 实际业务数据
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 错误响应
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"code": 422,
|
||||
"message": "参数校验失败",
|
||||
"data": {
|
||||
"fieldErrors": [
|
||||
{
|
||||
"field": "username",
|
||||
"message": "用户名长度必须在3-20个字符之间"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 使用方法
|
||||
|
||||
### 1. 自动格式化（推荐）
|
||||
|
||||
默认情况下，框架会自动将所有路由的响应包装为统一格式。你只需要在路由函数中返回业务数据即可：
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import get, post
|
||||
from myboot.core.application import app
|
||||
|
||||
@get('/users')
|
||||
def get_users():
|
||||
"""获取用户列表"""
|
||||
users = [{"id": 1, "name": "张三"}, {"id": 2, "name": "李四"}]
|
||||
return {"users": users} # 自动包装为统一格式
|
||||
```
|
||||
|
||||
实际返回：
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"code": 200,
|
||||
"message": "操作成功",
|
||||
"data": {
|
||||
"users": [
|
||||
{ "id": 1, "name": "张三" },
|
||||
{ "id": 2, "name": "李四" }
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 手动使用响应包装器
|
||||
|
||||
如果你需要自定义响应消息，可以使用响应包装器：
|
||||
|
||||
```python
|
||||
from myboot.web.response import response
|
||||
|
||||
@get('/users/{user_id}')
|
||||
def get_user(user_id: int):
|
||||
"""获取单个用户"""
|
||||
user = {"id": user_id, "name": "张三"}
|
||||
|
||||
# 使用响应包装器
|
||||
return response.success(
|
||||
data=user,
|
||||
message="查询成功",
|
||||
code=200
|
||||
)
|
||||
```
|
||||
|
||||
### 3. 便捷方法
|
||||
|
||||
响应包装器提供了多个便捷方法：
|
||||
|
||||
```python
|
||||
from myboot.web.response import response
|
||||
|
||||
# 创建成功（201）
|
||||
@post('/users')
|
||||
def create_user(name: str, email: str):
|
||||
user = create_user_service(name, email)
|
||||
return response.created(data=user, message="用户创建成功")
|
||||
|
||||
# 更新成功
|
||||
@put('/users/{user_id}')
|
||||
def update_user(user_id: int, name: str):
|
||||
user = update_user_service(user_id, name)
|
||||
return response.updated(data=user, message="用户更新成功")
|
||||
|
||||
# 删除成功
|
||||
@delete('/users/{user_id}')
|
||||
def delete_user(user_id: int):
|
||||
delete_user_service(user_id)
|
||||
return response.deleted(message="用户删除成功")
|
||||
|
||||
# 分页响应
|
||||
@get('/users')
|
||||
def get_users(page: int = 1, size: int = 10):
|
||||
users, total = get_users_service(page, size)
|
||||
return response.pagination(
|
||||
data=users,
|
||||
total=total,
|
||||
page=page,
|
||||
size=size,
|
||||
message="查询成功"
|
||||
)
|
||||
```
|
||||
|
||||
## 配置选项
|
||||
|
||||
### 启用/禁用自动格式化
|
||||
|
||||
在配置文件中设置：
|
||||
|
||||
```yaml
|
||||
server:
|
||||
response_format:
|
||||
enabled: true # 是否启用自动格式化
|
||||
exclude_paths: # 排除的路径（这些路径不会自动格式化）
|
||||
- "/custom/path"
|
||||
- "/another/path"
|
||||
```
|
||||
|
||||
或者在代码中（通过配置参数）：
|
||||
|
||||
```python
|
||||
app = Application(
|
||||
name="My App"
|
||||
)
|
||||
# 配置在 config.yaml 中设置：
|
||||
# server:
|
||||
# response_format:
|
||||
# enabled: true
|
||||
# exclude_paths:
|
||||
# - "/custom/path"
|
||||
```
|
||||
|
||||
### 默认排除的路径
|
||||
|
||||
以下路径默认不会被格式化（系统路径和文档路径）：
|
||||
|
||||
- `/docs`
|
||||
- `/openapi.json`
|
||||
- `/redoc`
|
||||
- `/health`
|
||||
- `/health/ready`
|
||||
- `/health/live`
|
||||
|
||||
## 异常响应格式
|
||||
|
||||
框架已经统一了异常响应格式，所有异常都会自动返回统一格式：
|
||||
|
||||
### 验证错误（422）
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"code": 422,
|
||||
"message": "Validation Error",
|
||||
"data": {
|
||||
"fieldErrors": [
|
||||
{
|
||||
"field": "username",
|
||||
"message": "用户名格式不正确"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### HTTP 错误（4xx, 5xx）
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"code": 404,
|
||||
"message": "HTTP Error",
|
||||
"data": {}
|
||||
}
|
||||
```
|
||||
|
||||
### 服务器错误（500）
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"code": 500,
|
||||
"message": "Internal Server Error",
|
||||
"data": {
|
||||
"type": "ExceptionClassName"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. **已经是统一格式的响应不会被重复包装**：如果你手动返回统一格式的响应，中间件会检测并直接返回。
|
||||
|
||||
2. **非 JSON 响应不会格式化**：只有 `JSONResponse` 类型的响应会被格式化。
|
||||
|
||||
3. **排除路径不会被格式化**：配置在排除列表中的路径不会被自动格式化，适用于需要返回原始格式的接口。
|
||||
|
||||
4. **中间件执行顺序**：响应格式化中间件是最后添加的，因此会最先执行（FastAPI 中间件是 LIFO）。
|
||||
|
||||
## 示例代码
|
||||
|
||||
完整示例：
|
||||
|
||||
```python
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.decorators import get, post, put, delete
|
||||
from myboot.web.response import response
|
||||
|
||||
app = Application(name="My API")
|
||||
|
||||
# 自动格式化
|
||||
@get('/users')
|
||||
def get_users():
|
||||
return {"users": [...]} # 自动包装
|
||||
|
||||
# 手动格式化（自定义消息）
|
||||
@get('/users/{user_id}')
|
||||
def get_user(user_id: int):
|
||||
user = get_user_by_id(user_id)
|
||||
return response.success(data=user, message="查询成功")
|
||||
|
||||
# 创建操作
|
||||
@post('/users')
|
||||
def create_user(name: str, email: str):
|
||||
user = create_user(name, email)
|
||||
return response.created(data=user, message="用户创建成功")
|
||||
|
||||
# 分页
|
||||
@get('/posts')
|
||||
def get_posts(page: int = 1, size: int = 10):
|
||||
posts, total = get_posts_paged(page, size)
|
||||
return response.pagination(
|
||||
data=posts,
|
||||
total=total,
|
||||
page=page,
|
||||
size=size
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,271 @@
|
||||
# 任务调度器使用说明
|
||||
|
||||
MyBoot 内置基于 [APScheduler](https://apscheduler.readthedocs.io/) 的任务调度器，支持 Cron、固定间隔与一次性任务。应用启动时，若已注册任务且调度器已启用，会自动启动调度。
|
||||
|
||||
## 1. 快速开始（推荐）
|
||||
|
||||
定时任务**必须**定义在 `@component` 装饰的类中，由自动配置在组件注册时扫描并加入 `app.scheduler`。
|
||||
|
||||
```python
|
||||
from myboot.core.decorators import component, cron, interval, once
|
||||
from myboot.core.config import get_config
|
||||
|
||||
@component()
|
||||
class DataSyncJobs:
|
||||
"""定时任务组件"""
|
||||
|
||||
@cron("0 2 * * *") # 每天 02:00（5 位 Cron）
|
||||
def sync_daily(self):
|
||||
print("每日同步")
|
||||
|
||||
@interval(minutes=30) # 每 30 分钟
|
||||
def health_check(self):
|
||||
print("健康检查")
|
||||
|
||||
@once("2025-12-31 23:59:59") # 指定时刻执行一次
|
||||
def year_end_task(self):
|
||||
print("年末任务")
|
||||
|
||||
@cron("0 */5 * * *", enabled=get_config("jobs.report.enabled", True))
|
||||
def report(self):
|
||||
"""enabled 可从配置读取，为 False 时不注册"""
|
||||
print("报表任务")
|
||||
```
|
||||
|
||||
**要求与约定：**
|
||||
|
||||
| 项 | 说明 |
|
||||
|----|------|
|
||||
| 类装饰器 | 必须使用 `@component()`，不支持模块级函数或 `@service` 类中的 `@cron` |
|
||||
| 方法可见性 | 以 `_` 开头的私有方法不会被扫描 |
|
||||
| 依赖注入 | 可在组件 `__init__` 中注入 `@service`，在任务方法内使用 |
|
||||
| 包扫描 | 任务类需位于自动发现包内（默认 `app`），见项目启动与 `auto_discover` 配置 |
|
||||
|
||||
应用生命周期中，存在已注册任务且调度器启用时，会在启动钩子阶段调用 `scheduler.start()`，关闭时 `scheduler.stop()`。
|
||||
|
||||
## 2. 配置
|
||||
|
||||
在 `config.yaml` 或环境变量中配置（环境变量嵌套键使用双下划线 `__`，详见 [配置管理使用说明](./configuration.md)）。
|
||||
|
||||
```yaml
|
||||
scheduler:
|
||||
enabled: true # 是否允许启动调度器
|
||||
timezone: "Asia/Shanghai" # 任务触发时区（建议显式设置）
|
||||
max_workers: 10 # 线程池大小
|
||||
on_all_workers: false # 多 worker 时是否在非 primary 进程也启用
|
||||
```
|
||||
|
||||
| 配置项 | 环境变量示例 | 默认值 | 说明 |
|
||||
|--------|----------------|--------|------|
|
||||
| `scheduler.enabled` | `SCHEDULER__ENABLED=false` | `true` | 为 `false` 时调度器不启动 |
|
||||
| `scheduler.timezone` | `SCHEDULER__TIMEZONE=Asia/Shanghai` | `UTC` | 所有触发器使用的时区 |
|
||||
| `scheduler.max_workers` | `SCHEDULER__MAX_WORKERS=20` | `10` | 并发执行任务的上限 |
|
||||
| `scheduler.on_all_workers` | `SCHEDULER__ON_ALL_WORKERS=true` | `false` | 见下文「多 Worker」 |
|
||||
|
||||
`config.py` 中默认片段：
|
||||
|
||||
```yaml
|
||||
scheduler:
|
||||
enabled: true
|
||||
timezone: "UTC"
|
||||
max_workers: 10
|
||||
```
|
||||
|
||||
## 3. Cron 表达式
|
||||
|
||||
解析逻辑见 `myboot/core/scheduler.py` 中 `_parse_cron`：
|
||||
|
||||
1. 优先使用 `CronTrigger.from_crontab`（**5 位**，Unix/APScheduler 风格）
|
||||
2. 解析失败时回退为手动构造 `CronTrigger`（支持 **5 位** 或 **6 位**）
|
||||
|
||||
### 3.1 五位格式（推荐）
|
||||
|
||||
顺序：**分 · 时 · 日 · 月 · 周**
|
||||
|
||||
```
|
||||
分 时 日 月 周
|
||||
│ │ │ │ └── 星期（0=周一 … 6=周日，见下表）
|
||||
│ │ │ └────── 月（1-12 或 *）
|
||||
│ │ └────────── 日（1-31 或 *）
|
||||
│ └────────────── 时（0-23）
|
||||
└────────────────── 分（0-59）
|
||||
```
|
||||
|
||||
**星期字段（APScheduler `from_crontab`）与 Linux crontab 不同：**
|
||||
|
||||
| 值 | APScheduler（本项目 5 位默认路径） | 传统 Linux crontab |
|
||||
|----|-----------------------------------|---------------------|
|
||||
| 0 | 周一 | 周日 |
|
||||
| 1 | 周二 | 周一 |
|
||||
| 2 | 周三 | 周二 |
|
||||
| … | … | … |
|
||||
| 6 | 周日 | 周六 |
|
||||
|
||||
> 使用 5 位表达式时，请按 **APScheduler 星期编号**理解，避免与系统 `crontab` 的「周日=0」混淆。
|
||||
|
||||
**常用示例（5 位，时区以 `scheduler.timezone` 为准）：**
|
||||
|
||||
| 表达式 | 含义 |
|
||||
|--------|------|
|
||||
| `0 * * * *` | 每小时整点 |
|
||||
| `0 2 * * *` | 每天 02:00 |
|
||||
| `0 23 * * 1` | 每周二 23:00（周字段 `1` = 周二） |
|
||||
| `30 8 * * 0` | 每周一 08:30 |
|
||||
| `*/15 * * * *` | 每 15 分钟 |
|
||||
| `0 9-17 * * 1-5` | 工作日 09:00–17:00 每小时整点 |
|
||||
|
||||
### 3.2 六位格式（兼容旧写法）
|
||||
|
||||
顺序：**秒 · 分 · 时 · 日 · 月 · 周**
|
||||
|
||||
仅在 `from_crontab` 无法解析时走手动分支，例如：
|
||||
|
||||
| 表达式 | 含义 |
|
||||
|--------|------|
|
||||
| `0 0 * * * *` | 每小时整点（秒=0，分=0） |
|
||||
| `0 */5 * * * *` | 每 5 分钟 |
|
||||
| `0 0 2 * * *` | 每天 02:00:00 |
|
||||
|
||||
### 3.3 字段支持
|
||||
|
||||
支持 APScheduler 常见写法：`*`、`,`、`-`、`/` 及范围。复杂表达式以 [APScheduler Cron 文档](https://apscheduler.readthedocs.io/en/stable/modules/triggers/cron.html) 为准。
|
||||
|
||||
## 4. 装饰器 API
|
||||
|
||||
### 4.1 `@cron`
|
||||
|
||||
```python
|
||||
@cron(cron_expression: str, enabled: bool | None = None, **kwargs)
|
||||
```
|
||||
|
||||
- `cron_expression`：5 位或 6 位 Cron 字符串
|
||||
- `enabled`：`False` 时跳过注册；`None` 默认启用
|
||||
- `**kwargs`：传给 APScheduler `add_job` 的额外参数（如 `name`、`max_instances`）
|
||||
|
||||
### 4.2 `@interval`
|
||||
|
||||
```python
|
||||
@interval(seconds=None, minutes=None, hours=None, enabled=None, **kwargs)
|
||||
```
|
||||
|
||||
三者至少指定其一，内部统一换算为秒。例如 `@interval(minutes=5)` 每 5 分钟执行。
|
||||
|
||||
### 4.3 `@once`
|
||||
|
||||
```python
|
||||
@once(run_date: str, enabled=None, **kwargs)
|
||||
```
|
||||
|
||||
`run_date` 支持格式：
|
||||
|
||||
- `YYYY-MM-DD HH:MM:SS`
|
||||
- `YYYY-MM-DD HH:MM`
|
||||
- `YYYY-MM-DD`（当天 00:00:00）
|
||||
|
||||
时间为 **naive**，由调度器全局时区 `scheduler.timezone` 解释。
|
||||
|
||||
## 5. 多 Worker 与进程模型
|
||||
|
||||
多进程部署（`server.workers > 1`）时：
|
||||
|
||||
- 默认**仅 primary worker**（`MYBOOT_IS_PRIMARY_WORKER=1`）启用调度器，避免重复执行
|
||||
- 设置 `scheduler.on_all_workers: true` 可在每个 worker 都运行调度（一般仅特殊场景需要）
|
||||
|
||||
任务在**线程池**中执行；`max_instances` 默认为 3（APScheduler `job_defaults`），同一任务并发实例数受此限制。
|
||||
|
||||
## 6. 编程式 API
|
||||
|
||||
除装饰器外，可通过 `app.scheduler` 或 `get_scheduler()` 动态管理任务。
|
||||
|
||||
```python
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
|
||||
scheduler = get_scheduler() # 注意：与 Application 可能不是同一实例，推荐用 app.scheduler
|
||||
|
||||
job_id = scheduler.add_cron_job(func=my_func, cron="0 2 * * *")
|
||||
scheduler.add_interval_job(func=my_func, interval=60)
|
||||
scheduler.add_date_job(func=my_func, run_date="2025-12-31 23:59:59")
|
||||
|
||||
scheduler.remove_job(job_id)
|
||||
info = scheduler.get_job_info(job_id)
|
||||
all_jobs = scheduler.list_all_jobs()
|
||||
```
|
||||
|
||||
| 方法 | 说明 |
|
||||
|------|------|
|
||||
| `add_cron_job(func, cron, job_id=None, **kwargs)` | 添加 Cron 任务 |
|
||||
| `add_interval_job(func, interval, job_id=None, **kwargs)` | `interval` 单位为秒 |
|
||||
| `add_date_job(func, run_date, job_id=None, **kwargs)` | 一次性任务 |
|
||||
| `remove_job(job_id)` | 移除任务 |
|
||||
| `get_job_info(job_id)` | 任务类型、下次执行时间等 |
|
||||
| `list_all_jobs()` | 所有任务摘要 |
|
||||
| `start()` / `stop()` | 启停（应用生命周期通常自动处理） |
|
||||
| `is_enabled()` / `is_running()` / `has_jobs()` | 状态查询 |
|
||||
| `get_config()` | 当前调度器配置摘要 |
|
||||
|
||||
未指定 `job_id` 时，默认生成 `cron_{模块名}.{限定名}`（如
|
||||
`cron_app.jobs.SyncJobs.daily_sync`），`interval_`/`date_` 前缀同理。
|
||||
限定名包含类名，因此不同类中的同名方法不会冲突（0.2.0 起，issue #14；
|
||||
此前为 `cron_{函数名}`，同名方法会因 ID 冲突导致启动失败）。
|
||||
|
||||
注册时即校验 ID 唯一性：显式传入重复的 `job_id` 会抛出 `SchedulerError`；
|
||||
自动生成的 ID 重复（同一函数注册多次）会追加 8 位随机后缀并记录 warning。
|
||||
|
||||
### 6.1 `ScheduledJob` 类
|
||||
|
||||
继承 `myboot.jobs.scheduled_job.ScheduledJob` 可实现带重试、状态跟踪的任务，并通过 `add_scheduled_job` 注册：
|
||||
|
||||
```python
|
||||
from myboot.jobs.scheduled_job import ScheduledJob
|
||||
|
||||
class CleanupJob(ScheduledJob):
|
||||
def run(self):
|
||||
# 业务逻辑
|
||||
return "ok"
|
||||
|
||||
job = CleanupJob(name="cleanup", trigger="0 3 * * *") # 或 trigger={'type': 'cron', 'cron': '...'}
|
||||
app.scheduler.add_scheduled_job(job)
|
||||
```
|
||||
|
||||
`trigger` 可为 Cron 字符串或字典：`cron` / `interval`（seconds/minutes/hours/days）/ `date`（`run_date`）。
|
||||
|
||||
## 7. 运维与排查
|
||||
|
||||
```python
|
||||
# 应用内
|
||||
print(app.scheduler.get_config())
|
||||
for job in app.scheduler.list_all_jobs():
|
||||
print(job["job_id"], job.get("type"), job.get("next_run_time"))
|
||||
|
||||
# 健康检查等可结合 application 状态中的 scheduler 字段
|
||||
```
|
||||
|
||||
日志：调度器绑定 logger 名 `scheduler`，注册任务时会输出 `已添加 Cron 任务` 等信息。
|
||||
|
||||
**任务未执行时检查：**
|
||||
|
||||
1. `scheduler.enabled` 是否为 `true`
|
||||
2. 多 worker 下当前进程是否为 primary（或已开启 `on_all_workers`）
|
||||
3. 装饰器 `enabled=False` 是否被跳过
|
||||
4. Cron 位数与星期编号是否符合上文约定
|
||||
5. `timezone` 是否与预期一致
|
||||
6. 应用启动时 `has_jobs()` 是否为真（无任务不会 `start()`）
|
||||
|
||||
## 8. 最佳实践
|
||||
|
||||
1. **统一用 5 位 Cron**，减少与 6 位混用带来的理解成本。
|
||||
2. **显式设置 `scheduler.timezone`**（如 `Asia/Shanghai`），避免默认 UTC 造成「时间差 8 小时」。
|
||||
3. **任务逻辑保持幂等**；错过触发时 APScheduler 有 `misfire_grace_time`（默认 30 秒），但仍可能补跑。
|
||||
4. **长耗时任务**注意 `max_workers` 与 `max_instances`，避免占满线程池。
|
||||
5. **配置开关**用 `enabled=get_config('jobs.xxx.enabled', True)`，便于按环境关闭任务。
|
||||
6. **IO 密集或阻塞操作**在任务内自行控制超时与异常，避免拖垮调度线程。
|
||||
|
||||
## 9. 相关文档与代码
|
||||
|
||||
| 资源 | 说明 |
|
||||
|------|------|
|
||||
| [scheduler_refactor_analysis.md](./scheduler_refactor_analysis.md) | APScheduler 重构与能力对照（偏设计） |
|
||||
| [dependency-injection.md](./dependency-injection.md) | 含定时任务的 `@component` 示例 |
|
||||
| `myboot/core/scheduler.py` | 调度器实现 |
|
||||
| `myboot/core/decorators.py` | `@cron` / `@interval` / `@once` |
|
||||
| `examples/convention_app.py` | 完整示例 `ScheduledJobs` 组件 |
|
||||
@@ -0,0 +1,354 @@
|
||||
# 调度器重构可行性分析：使用 APScheduler
|
||||
|
||||
> **使用说明**：日常开发与配置 Cron/间隔任务请参阅 [scheduler.md](./scheduler.md)。
|
||||
|
||||
## 1. 当前实现分析
|
||||
|
||||
### 1.1 现有功能
|
||||
- **任务类型支持**：
|
||||
- Cron 任务（6位表达式：秒 分 时 日 月 周）
|
||||
- 间隔任务（interval，支持秒/分/小时）
|
||||
- 一次性任务（date，指定日期时间执行）
|
||||
|
||||
- **API 接口**：
|
||||
- `add_cron_job(func, cron, job_id, **kwargs)`
|
||||
- `add_interval_job(func, interval, job_id, **kwargs)`
|
||||
- `add_date_job(func, run_date, job_id, **kwargs)`
|
||||
- `remove_job(job_id)`
|
||||
- `get_job(job_id)`
|
||||
- `list_jobs()`
|
||||
- `start()` / `stop()`
|
||||
- `add_scheduled_job(job: ScheduledJob)`
|
||||
|
||||
- **集成点**：
|
||||
- 与 `Application` 生命周期集成（启动/停止）
|
||||
- 自动配置系统自动注册装饰器标记的任务
|
||||
- 支持 `ScheduledJob` 类对象
|
||||
- 配置支持（时区、最大工作线程数、启用/禁用）
|
||||
|
||||
### 1.2 当前实现的问题
|
||||
- 自定义实现，维护成本高
|
||||
- Cron 表达式解析逻辑复杂，可能存在边界情况
|
||||
- 时区处理需要手动实现（依赖 pytz）
|
||||
- 任务执行状态跟踪有限
|
||||
- 缺少任务持久化能力
|
||||
- 错误处理和重试机制简单
|
||||
|
||||
## 2. APScheduler 能力分析
|
||||
|
||||
### 2.1 APScheduler 核心特性
|
||||
- **触发器（Triggers）**：
|
||||
- `CronTrigger`：完整的 Cron 表达式支持（5位或6位）
|
||||
- `IntervalTrigger`：固定间隔执行
|
||||
- `DateTrigger`：指定日期时间执行
|
||||
|
||||
- **执行器（Executors）**：
|
||||
- `ThreadPoolExecutor`：线程池执行
|
||||
- `ProcessPoolExecutor`：进程池执行
|
||||
- `AsyncIOExecutor`：异步执行
|
||||
- `GeventExecutor`：Gevent 协程执行
|
||||
|
||||
- **任务存储（Job Stores）**：
|
||||
- `MemoryJobStore`：内存存储（默认）
|
||||
- `SQLAlchemyJobStore`：数据库持久化
|
||||
- `RedisJobStore`：Redis 持久化
|
||||
- `MongoDBJobStore`：MongoDB 持久化
|
||||
|
||||
- **调度器（Schedulers）**：
|
||||
- `BlockingScheduler`：阻塞式调度器
|
||||
- `BackgroundScheduler`：后台线程调度器（推荐）
|
||||
- `AsyncIOScheduler`：异步调度器
|
||||
- `GeventScheduler`：Gevent 调度器
|
||||
- `TornadoScheduler`：Tornado 调度器
|
||||
- `TwistedScheduler`：Twisted 调度器
|
||||
|
||||
### 2.2 APScheduler 优势
|
||||
- ✅ 成熟稳定，社区活跃
|
||||
- ✅ 完整的 Cron 表达式支持（包括复杂表达式）
|
||||
- ✅ 内置时区支持（无需手动处理）
|
||||
- ✅ 任务持久化能力
|
||||
- ✅ 丰富的任务状态和事件监听
|
||||
- ✅ 更好的错误处理和日志
|
||||
- ✅ 支持任务暂停/恢复
|
||||
- ✅ 支持任务修改（修改触发器）
|
||||
|
||||
## 3. 功能映射关系
|
||||
|
||||
### 3.1 任务类型映射
|
||||
|
||||
| 当前实现 | APScheduler | 映射方式 |
|
||||
|---------|------------|---------|
|
||||
| `add_cron_job(func, cron, ...)` | `scheduler.add_job(func, CronTrigger.from_crontab(cron), ...)` | ✅ 直接映射 |
|
||||
| `add_interval_job(func, interval, ...)` | `scheduler.add_job(func, IntervalTrigger(seconds=interval), ...)` | ✅ 直接映射 |
|
||||
| `add_date_job(func, run_date, ...)` | `scheduler.add_job(func, DateTrigger(run_date=...), ...)` | ✅ 直接映射 |
|
||||
|
||||
### 3.2 API 接口映射
|
||||
|
||||
| 当前方法 | APScheduler 对应方法 | 兼容性 |
|
||||
|---------|-------------------|--------|
|
||||
| `add_cron_job()` | `add_job(..., trigger=CronTrigger(...))` | ✅ 可封装保持兼容 |
|
||||
| `add_interval_job()` | `add_job(..., trigger=IntervalTrigger(...))` | ✅ 可封装保持兼容 |
|
||||
| `add_date_job()` | `add_job(..., trigger=DateTrigger(...))` | ✅ 可封装保持兼容 |
|
||||
| `remove_job(job_id)` | `remove_job(job_id)` | ✅ 完全兼容 |
|
||||
| `get_job(job_id)` | `get_job(job_id)` | ✅ 完全兼容 |
|
||||
| `list_jobs()` | `get_jobs()` | ✅ 可封装保持兼容 |
|
||||
| `start()` | `start()` | ✅ 完全兼容 |
|
||||
| `stop()` | `shutdown()` | ⚠️ 需要封装（方法名不同） |
|
||||
|
||||
### 3.3 配置映射
|
||||
|
||||
| 当前配置 | APScheduler 配置 | 映射方式 |
|
||||
|---------|-----------------|---------|
|
||||
| `scheduler.enabled` | 通过 `start()` 控制 | ✅ 逻辑控制 |
|
||||
| `scheduler.timezone` | `timezone` 参数 | ✅ 直接映射 |
|
||||
| `scheduler.max_workers` | `executors['default']['max_workers']` | ✅ 直接映射 |
|
||||
|
||||
## 4. 重构方案设计
|
||||
|
||||
### 4.1 方案一：完全封装（推荐）
|
||||
|
||||
**设计思路**：保持现有 API 接口不变，内部使用 APScheduler 实现。
|
||||
|
||||
**优点**：
|
||||
- ✅ 完全向后兼容，现有代码无需修改
|
||||
- ✅ 可以逐步迁移，降低风险
|
||||
- ✅ 保留自定义扩展能力
|
||||
|
||||
**实现要点**：
|
||||
```python
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
from apscheduler.triggers.date import DateTrigger
|
||||
|
||||
class Scheduler:
|
||||
def __init__(self, config):
|
||||
# 创建 APScheduler 实例
|
||||
self._scheduler = BackgroundScheduler(
|
||||
timezone=config.get('scheduler.timezone', 'UTC'),
|
||||
executors={
|
||||
'default': {
|
||||
'type': 'threadpool',
|
||||
'max_workers': config.get('scheduler.max_workers', 10)
|
||||
}
|
||||
}
|
||||
)
|
||||
# 保持兼容性的映射
|
||||
self._jobs = {} # job_id -> APScheduler Job
|
||||
self._scheduled_jobs = {} # ScheduledJob 对象
|
||||
|
||||
def add_cron_job(self, func, cron, job_id=None, **kwargs):
|
||||
# 转换 cron 表达式格式（6位 -> APScheduler 格式）
|
||||
trigger = self._parse_cron(cron)
|
||||
job = self._scheduler.add_job(
|
||||
func,
|
||||
trigger=trigger,
|
||||
id=job_id,
|
||||
**kwargs
|
||||
)
|
||||
self._jobs[job_id] = job
|
||||
return job_id
|
||||
```
|
||||
|
||||
### 4.2 方案二：直接使用 APScheduler
|
||||
|
||||
**设计思路**：直接暴露 APScheduler 的 API，简化封装层。
|
||||
|
||||
**优点**：
|
||||
- ✅ 代码更简洁
|
||||
- ✅ 可以直接使用 APScheduler 的高级特性
|
||||
|
||||
**缺点**：
|
||||
- ❌ 需要修改现有代码
|
||||
- ❌ 破坏向后兼容性
|
||||
|
||||
### 4.3 推荐方案：方案一（完全封装）
|
||||
|
||||
**理由**：
|
||||
1. 保持 API 兼容性，现有代码无需修改
|
||||
2. 可以逐步增强功能（如添加持久化）
|
||||
3. 保留扩展空间
|
||||
|
||||
## 5. 实施步骤
|
||||
|
||||
### 5.1 阶段一：基础重构（保持兼容）
|
||||
1. **创建新的 Scheduler 类**（基于 APScheduler）
|
||||
- 实现所有现有 API 方法
|
||||
- 保持方法签名和返回值一致
|
||||
- 内部使用 APScheduler
|
||||
|
||||
2. **Cron 表达式转换**
|
||||
- 当前格式：`"秒 分 时 日 月 周"`（6位）
|
||||
- APScheduler 格式：`"分 时 日 月 周"`（5位）或 `CronTrigger` 对象
|
||||
- 需要实现转换函数
|
||||
|
||||
3. **测试验证**
|
||||
- 单元测试覆盖所有 API
|
||||
- 集成测试验证现有功能
|
||||
- 确保装饰器和自动配置正常工作
|
||||
|
||||
### 5.2 阶段二：功能增强（可选）
|
||||
1. **添加任务持久化**
|
||||
- 支持 SQLAlchemyJobStore
|
||||
- 支持 RedisJobStore
|
||||
|
||||
2. **增强任务管理**
|
||||
- 任务暂停/恢复
|
||||
- 任务修改触发器
|
||||
- 任务执行历史
|
||||
|
||||
3. **事件监听**
|
||||
- 任务执行成功/失败事件
|
||||
- 任务错过执行事件
|
||||
|
||||
### 5.3 阶段三：优化和文档
|
||||
1. 性能优化
|
||||
2. 文档更新
|
||||
3. 示例代码更新
|
||||
|
||||
## 6. 关键技术点
|
||||
|
||||
### 6.1 Cron 表达式转换
|
||||
|
||||
**当前格式**（6位）：
|
||||
```
|
||||
"秒 分 时 日 月 周"
|
||||
例如："0 0 * * * *" # 每小时
|
||||
```
|
||||
|
||||
**APScheduler 格式**（5位或 CronTrigger）：
|
||||
```python
|
||||
# 方式1：5位字符串（标准 Cron）
|
||||
"分 时 日 月 周"
|
||||
"0 * * * *" # 每小时
|
||||
|
||||
# 方式2：CronTrigger 对象（推荐）
|
||||
CronTrigger(second=0, minute=0, hour='*', day='*', month='*', day_of_week='*')
|
||||
```
|
||||
|
||||
**转换函数**：
|
||||
```python
|
||||
def _parse_cron(self, cron_expr: str) -> CronTrigger:
|
||||
"""将6位 Cron 表达式转换为 APScheduler CronTrigger"""
|
||||
parts = cron_expr.split()
|
||||
if len(parts) == 6:
|
||||
second, minute, hour, day, month, weekday = parts
|
||||
return CronTrigger(
|
||||
second=second,
|
||||
minute=minute,
|
||||
hour=hour,
|
||||
day=day,
|
||||
month=month,
|
||||
day_of_week=weekday
|
||||
)
|
||||
elif len(parts) == 5:
|
||||
# 标准5位格式，秒默认为0
|
||||
return CronTrigger.from_crontab(cron_expr)
|
||||
else:
|
||||
raise ValueError(f"无效的 Cron 表达式: {cron_expr}")
|
||||
```
|
||||
|
||||
### 6.2 ScheduledJob 集成
|
||||
|
||||
需要将 `ScheduledJob.execute()` 方法适配到 APScheduler：
|
||||
|
||||
```python
|
||||
def add_scheduled_job(self, job: ScheduledJob, job_id: Optional[str] = None):
|
||||
# 包装 execute 方法，保持状态跟踪
|
||||
def wrapped_execute():
|
||||
return job.execute()
|
||||
|
||||
# 根据 trigger 类型添加任务
|
||||
if isinstance(job.trigger, str):
|
||||
return self.add_cron_job(wrapped_execute, job.trigger, job_id)
|
||||
# ... 其他类型
|
||||
```
|
||||
|
||||
### 6.3 时区处理
|
||||
|
||||
APScheduler 内置时区支持，无需手动处理：
|
||||
|
||||
```python
|
||||
from pytz import timezone
|
||||
|
||||
scheduler = BackgroundScheduler(
|
||||
timezone=timezone('Asia/Shanghai') # 直接支持时区对象
|
||||
)
|
||||
```
|
||||
|
||||
### 6.4 任务状态管理
|
||||
|
||||
APScheduler 提供任务状态：
|
||||
- `pending`：等待执行
|
||||
- `running`：正在执行
|
||||
- `completed`：已完成
|
||||
- `failed`：执行失败
|
||||
|
||||
可以通过 `get_job(job_id).next_run_time` 获取下次执行时间。
|
||||
|
||||
## 7. 风险评估
|
||||
|
||||
### 7.1 兼容性风险
|
||||
- **风险**：Cron 表达式格式差异
|
||||
- **缓解**：实现转换函数，确保兼容
|
||||
|
||||
### 7.2 行为差异风险
|
||||
- **风险**：APScheduler 的执行时机可能与当前实现略有差异
|
||||
- **缓解**：充分测试，特别是边界情况
|
||||
|
||||
### 7.3 性能风险
|
||||
- **风险**：APScheduler 可能有额外的开销
|
||||
- **缓解**：性能测试，通常 APScheduler 性能更好
|
||||
|
||||
## 8. 测试策略
|
||||
|
||||
### 8.1 单元测试
|
||||
- 所有 API 方法的测试
|
||||
- Cron 表达式转换测试
|
||||
- 任务添加/删除测试
|
||||
|
||||
### 8.2 集成测试
|
||||
- 装饰器自动注册测试
|
||||
- ScheduledJob 集成测试
|
||||
- 应用生命周期集成测试
|
||||
|
||||
### 8.3 兼容性测试
|
||||
- 现有示例代码运行测试
|
||||
- 现有项目迁移测试
|
||||
|
||||
## 9. 结论
|
||||
|
||||
### 9.1 可行性评估
|
||||
✅ **高度可行**
|
||||
|
||||
**理由**：
|
||||
1. APScheduler 功能完全覆盖现有需求
|
||||
2. API 可以完全兼容
|
||||
3. 项目已依赖 APScheduler
|
||||
4. 重构收益明显（稳定性、功能、维护性）
|
||||
|
||||
### 9.2 推荐方案
|
||||
**采用方案一（完全封装）**，分阶段实施：
|
||||
1. 第一阶段：基础重构，保持兼容
|
||||
2. 第二阶段：功能增强（可选）
|
||||
3. 第三阶段：优化和文档
|
||||
|
||||
### 9.3 预期收益
|
||||
- ✅ 减少代码量（约 200+ 行 → 100+ 行）
|
||||
- ✅ 提高稳定性（使用成熟库）
|
||||
- ✅ 增强功能（持久化、事件监听等）
|
||||
- ✅ 降低维护成本
|
||||
- ✅ 更好的错误处理
|
||||
|
||||
### 9.4 实施建议
|
||||
1. **先创建新实现**，保留旧实现作为备份
|
||||
2. **充分测试**后再替换
|
||||
3. **逐步迁移**，可以先支持两种实现并存
|
||||
4. **文档更新**，说明新特性
|
||||
|
||||
## 10. 参考资源
|
||||
|
||||
- APScheduler 官方文档：https://apscheduler.readthedocs.io/
|
||||
- APScheduler GitHub：https://github.com/agronholm/apscheduler
|
||||
- Cron 表达式参考：https://en.wikipedia.org/wiki/Cron
|
||||
|
||||
@@ -0,0 +1,440 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
约定优于配置示例应用
|
||||
|
||||
展示 MyBoot 框架的约定优于配置特性
|
||||
"""
|
||||
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.config import get_config
|
||||
from myboot.core.decorators import (
|
||||
get, post, put, delete,
|
||||
cron, interval, once,
|
||||
service, client, middleware,
|
||||
rest_controller, component
|
||||
)
|
||||
from myboot.jobs.scheduled_job import ScheduledJob
|
||||
from myboot.core.scheduler import get_scheduler
|
||||
|
||||
# 添加项目根目录到 Python 路径
|
||||
project_root = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
# 创建应用实例
|
||||
app = Application(
|
||||
name="约定优于配置示例",
|
||||
auto_configuration=True, # 启用自动配置
|
||||
auto_discover_package="examples.convention_app"
|
||||
)
|
||||
|
||||
|
||||
# ==================== 服务层 ====================
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
"""用户服务 - 自动注册为 'user_service'"""
|
||||
|
||||
def __init__(self):
|
||||
self.users = {}
|
||||
print("✅ UserService 已初始化")
|
||||
|
||||
def get_user(self, user_id: int):
|
||||
"""获取用户"""
|
||||
return self.users.get(user_id, {"id": user_id, "name": f"用户{user_id}"})
|
||||
|
||||
def create_user(self, name: str, email: str):
|
||||
"""创建用户"""
|
||||
user_id = len(self.users) + 1
|
||||
user = {"id": user_id, "name": name, "email": email}
|
||||
self.users[user_id] = user
|
||||
return user
|
||||
|
||||
def update_user(self, user_id: int, **kwargs):
|
||||
"""更新用户"""
|
||||
if user_id in self.users:
|
||||
self.users[user_id].update(kwargs)
|
||||
return self.users[user_id]
|
||||
return None
|
||||
|
||||
def delete_user(self, user_id: int):
|
||||
"""删除用户"""
|
||||
return self.users.pop(user_id, None)
|
||||
|
||||
|
||||
@service('email_service')
|
||||
class EmailService:
|
||||
"""邮件服务 - 注册为 'email_service'"""
|
||||
|
||||
def __init__(self):
|
||||
print("✅ EmailService 已初始化")
|
||||
|
||||
def send_email(self, to: str, subject: str, body: str):
|
||||
"""发送邮件"""
|
||||
print(f"📧 发送邮件到 {to}: {subject}")
|
||||
return {"status": "sent", "to": to, "subject": subject}
|
||||
|
||||
|
||||
# ==================== 客户端层 ====================
|
||||
|
||||
@client()
|
||||
class DatabaseClient:
|
||||
"""数据库客户端 - 自动注册为 'database_client'"""
|
||||
|
||||
def __init__(self):
|
||||
print("✅ DatabaseClient 已初始化")
|
||||
|
||||
def connect(self):
|
||||
"""连接数据库"""
|
||||
print("🔗 连接数据库")
|
||||
return True
|
||||
|
||||
def query(self, sql: str):
|
||||
"""执行查询"""
|
||||
print(f"📊 执行查询: {sql}")
|
||||
return []
|
||||
|
||||
|
||||
@client('redis_client')
|
||||
class RedisClient:
|
||||
"""Redis 客户端 - 注册为 'redis_client'"""
|
||||
|
||||
def __init__(self):
|
||||
print("✅ RedisClient 已初始化")
|
||||
self.value = None
|
||||
|
||||
def get(self, key: str):
|
||||
"""获取缓存"""
|
||||
print(f"📦 获取缓存: {key}")
|
||||
return self.value
|
||||
|
||||
def set(self, key: str, value: str):
|
||||
"""设置缓存"""
|
||||
print(f"💾 设置缓存: {key} = {value}")
|
||||
self.value = value
|
||||
|
||||
|
||||
# ==================== 中间件 ====================
|
||||
|
||||
@middleware(order=1)
|
||||
async def logging_middleware(request, next_handler):
|
||||
"""日志中间件 - 自动注册，处理所有请求"""
|
||||
print(f"📝 请求日志: {request.method} {request.url}")
|
||||
response = await next_handler(request)
|
||||
print(f"📝 响应日志: {response.status_code}")
|
||||
return response
|
||||
|
||||
|
||||
@middleware(order=2)
|
||||
async def timing_middleware(request, next_handler):
|
||||
"""计时中间件 - 自动注册，处理所有请求"""
|
||||
import time
|
||||
start_time = time.time()
|
||||
response = await next_handler(request)
|
||||
end_time = time.time()
|
||||
elapsed = end_time - start_time
|
||||
print(f"⏱️ 请求耗时: {elapsed:.3f}s - {request.method} {request.url.path}")
|
||||
return response
|
||||
|
||||
|
||||
@middleware(order=3, path_filter='/api/*')
|
||||
async def api_middleware(request, next_handler):
|
||||
"""API 中间件 - 只处理 /api/* 路径的请求"""
|
||||
print(f"🔌 API 请求: {request.method} {request.url.path}")
|
||||
# 可以在这里添加 API 特定的逻辑，如认证、限流等
|
||||
response = await next_handler(request)
|
||||
return response
|
||||
|
||||
|
||||
@middleware(order=4, methods=['POST', 'PUT', 'PATCH'])
|
||||
async def modify_middleware(request, next_handler):
|
||||
"""修改操作中间件 - 只处理 POST、PUT、PATCH 请求"""
|
||||
print(f"✏️ 修改操作: {request.method} {request.url.path}")
|
||||
response = await next_handler(request)
|
||||
return response
|
||||
|
||||
|
||||
@middleware(
|
||||
order=5,
|
||||
path_filter=['/users/*', '/api/products/*'],
|
||||
condition=lambda req: req.headers.get('user-agent', '').startswith('Mozilla')
|
||||
)
|
||||
async def browser_only_middleware(request, next_handler):
|
||||
"""浏览器专用中间件 - 只处理浏览器请求且匹配特定路径"""
|
||||
print(f"🌐 浏览器请求: {request.headers.get('user-agent', 'Unknown')}")
|
||||
response = await next_handler(request)
|
||||
return response
|
||||
|
||||
|
||||
# ==================== 定时任务组件 ====================
|
||||
# 注意：定时任务必须在 @component 装饰的类中定义，支持依赖注入
|
||||
|
||||
|
||||
@component()
|
||||
class ScheduledJobs:
|
||||
"""定时任务组件 - 使用 @component 装饰器，支持依赖注入"""
|
||||
|
||||
def __init__(self):
|
||||
print("✅ ScheduledJobs 已初始化")
|
||||
|
||||
@cron('0 */1 * * * *', enabled=True) # 每分钟执行
|
||||
def heartbeat(self):
|
||||
"""心跳任务"""
|
||||
print("💓 心跳检测 - 系统运行正常")
|
||||
|
||||
@interval(minutes=10, enabled=get_config('jobs.cleanup_task.enabled', True))
|
||||
def cleanup_task(self):
|
||||
"""清理任务 - 从配置文件读取 enabled 状态"""
|
||||
print("🧹 执行清理任务")
|
||||
|
||||
@once('2025-12-31 23:59:59')
|
||||
def new_year_task(self):
|
||||
"""新年任务"""
|
||||
print("🎉 新年任务执行")
|
||||
|
||||
|
||||
# ==================== REST 控制器 ====================
|
||||
# 注意：路由必须在 @rest_controller 装饰的类中定义
|
||||
|
||||
@rest_controller('/')
|
||||
class HomeController:
|
||||
"""首页控制器"""
|
||||
|
||||
@get('/')
|
||||
def home(self):
|
||||
"""首页 - GET /"""
|
||||
return {
|
||||
"message": "欢迎使用 MyBoot 约定优于配置示例",
|
||||
"features": [
|
||||
"自动发现和注册组件",
|
||||
"约定优于配置",
|
||||
"零配置启动",
|
||||
"REST 控制器",
|
||||
"依赖注入",
|
||||
"定时任务"
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@rest_controller('/api/users')
|
||||
class UserController:
|
||||
"""用户控制器 - 使用依赖注入"""
|
||||
|
||||
def __init__(self, user_service: UserService, email_service: EmailService):
|
||||
self.user_service = user_service
|
||||
self.email_service = email_service
|
||||
|
||||
@get('/')
|
||||
def list_users(self):
|
||||
"""获取用户列表 - GET /api/users"""
|
||||
return {"users": list(self.user_service.users.values())}
|
||||
|
||||
@get('/{user_id}')
|
||||
def get_user(self, user_id: int):
|
||||
"""获取单个用户 - GET /api/users/{user_id}"""
|
||||
return self.user_service.get_user(user_id)
|
||||
|
||||
@post('/')
|
||||
def create_user(self, name: str, email: str):
|
||||
"""创建用户 - POST /api/users"""
|
||||
user = self.user_service.create_user(name, email)
|
||||
self.email_service.send_email(email, "欢迎注册", f"欢迎 {name} 注册我们的服务！")
|
||||
return {"message": "用户创建成功", "user": user}
|
||||
|
||||
@put('/{user_id}')
|
||||
def update_user(self, user_id: int, name: str = None, email: str = None):
|
||||
"""更新用户 - PUT /api/users/{user_id}"""
|
||||
update_data = {}
|
||||
if name:
|
||||
update_data['name'] = name
|
||||
if email:
|
||||
update_data['email'] = email
|
||||
|
||||
user = self.user_service.update_user(user_id, **update_data)
|
||||
if user:
|
||||
return {"message": "用户更新成功", "user": user}
|
||||
return {"error": "用户不存在"}
|
||||
|
||||
@delete('/{user_id}')
|
||||
def delete_user(self, user_id: int):
|
||||
"""删除用户 - DELETE /api/users/{user_id}"""
|
||||
user = self.user_service.delete_user(user_id)
|
||||
if user:
|
||||
return {"message": "用户删除成功", "user": user}
|
||||
return {"error": "用户不存在"}
|
||||
|
||||
|
||||
@rest_controller('/api/products')
|
||||
class ProductController:
|
||||
"""产品控制器 - 使用依赖注入"""
|
||||
|
||||
def __init__(self, redis_client: RedisClient):
|
||||
self.redis_client = redis_client
|
||||
self.products = {
|
||||
1: {"id": 1, "name": "产品1", "price": 100},
|
||||
2: {"id": 2, "name": "产品2", "price": 200}
|
||||
}
|
||||
|
||||
@get('/')
|
||||
def list_products(self):
|
||||
"""获取产品列表 - GET /api/products"""
|
||||
if self.redis_client:
|
||||
print(self.redis_client.get('app_status'))
|
||||
return {"products": list(self.products.values())}
|
||||
|
||||
@get('/{product_id}')
|
||||
def get_product(self, product_id: int):
|
||||
"""获取单个产品 - GET /api/products/{product_id}"""
|
||||
return self.products.get(product_id, {"error": "产品不存在"})
|
||||
|
||||
@post('/')
|
||||
def create_product(self, name: str, price: float):
|
||||
"""创建产品 - POST /api/products"""
|
||||
product_id = max(self.products.keys()) + 1
|
||||
product = {"id": product_id, "name": name, "price": price}
|
||||
self.products[product_id] = product
|
||||
return {"message": "产品创建成功", "product": product}
|
||||
|
||||
@put('/{product_id}')
|
||||
def update_product(self, product_id: int, name: str = None, price: float = None):
|
||||
"""更新产品 - PUT /api/products/{product_id}"""
|
||||
if product_id not in self.products:
|
||||
return {"error": "产品不存在"}
|
||||
|
||||
if name:
|
||||
self.products[product_id]['name'] = name
|
||||
if price:
|
||||
self.products[product_id]['price'] = price
|
||||
|
||||
return {"message": "产品更新成功", "product": self.products[product_id]}
|
||||
|
||||
@delete('/{product_id}')
|
||||
def delete_product(self, product_id: int):
|
||||
"""删除产品 - DELETE /api/products/{product_id}"""
|
||||
if product_id in self.products:
|
||||
product = self.products.pop(product_id)
|
||||
return {"message": "产品删除成功", "product": product}
|
||||
return {"error": "产品不存在"}
|
||||
|
||||
|
||||
def generate_report(report_type: str):
|
||||
"""生成报告任务"""
|
||||
import time
|
||||
print(f"开始生成 {report_type} 报告")
|
||||
time.sleep(10) # 模拟报告生成
|
||||
return {"type": report_type, "status": "completed"}
|
||||
|
||||
|
||||
@rest_controller('/api/reports')
|
||||
class ReportController:
|
||||
"""报告控制器"""
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = get_scheduler()
|
||||
|
||||
@post('/generate')
|
||||
def create_report(self, report_type: str):
|
||||
"""创建报告生成任务"""
|
||||
# 创建自定义 ScheduledJob
|
||||
class ReportJob(ScheduledJob):
|
||||
def __init__(self, report_type: str):
|
||||
super().__init__(
|
||||
name=f"生成{report_type}报告",
|
||||
timeout=300 # 5分钟超时
|
||||
)
|
||||
self.report_type = report_type
|
||||
|
||||
def run(self, *args, **kwargs):
|
||||
return generate_report(self.report_type)
|
||||
|
||||
# 创建任务实例
|
||||
job = ReportJob(report_type)
|
||||
|
||||
# 添加到调度器并执行
|
||||
job_id = self.scheduler.add_scheduled_job(job)
|
||||
thread = threading.Thread(target=job.execute)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
return {"message": "报告生成任务已创建", "job_id": job_id}
|
||||
|
||||
@get('/status/{job_id}')
|
||||
def get_status(self, job_id: str):
|
||||
"""查询任务状态"""
|
||||
job = self.scheduler.get_scheduled_job(job_id)
|
||||
if job:
|
||||
return job.get_info()
|
||||
return {"error": "任务不存在"}
|
||||
|
||||
|
||||
# ==================== 启动钩子 ====================
|
||||
@app.add_startup_hook
|
||||
def startup_hook():
|
||||
"""启动钩子"""
|
||||
print("🚀 应用启动钩子执行")
|
||||
|
||||
from myboot.core.application import get_client
|
||||
|
||||
# 初始化数据库连接
|
||||
db_client = get_client('database_client')
|
||||
if db_client:
|
||||
db_client.connect()
|
||||
|
||||
# 初始化 Redis 连接
|
||||
redis_client = get_client('redis_client')
|
||||
if redis_client:
|
||||
redis_client.set('app_status', 'running')
|
||||
|
||||
|
||||
@app.add_shutdown_hook
|
||||
def shutdown_hook():
|
||||
"""关闭钩子"""
|
||||
print("🛑 应用关闭钩子执行")
|
||||
|
||||
|
||||
# ==================== 主程序 ====================
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("=" * 60)
|
||||
print("🎯 MyBoot 约定优于配置示例")
|
||||
print("=" * 60)
|
||||
print()
|
||||
print("✨ 特性展示:")
|
||||
print(" • 自动发现和注册组件")
|
||||
print(" • 约定优于配置")
|
||||
print(" • 零配置启动")
|
||||
print(" • 自动服务注入")
|
||||
print(" • 自动路由注册")
|
||||
print(" • 自动任务调度")
|
||||
print()
|
||||
print("🌐 访问地址:")
|
||||
print(" • 应用: http://localhost:8000")
|
||||
print(" • API 文档: http://localhost:8000/docs")
|
||||
print(" • 健康检查: http://localhost:8000/health")
|
||||
print()
|
||||
print("📚 API 端点（通过 @rest_controller 定义）:")
|
||||
print(" • GET / - 首页")
|
||||
print(" • GET /api/users - 用户列表")
|
||||
print(" • GET /api/users/{id} - 获取用户")
|
||||
print(" • POST /api/users - 创建用户")
|
||||
print(" • PUT /api/users/{id} - 更新用户")
|
||||
print(" • DELETE /api/users/{id} - 删除用户")
|
||||
print(" • GET /api/products - 产品列表")
|
||||
print(" • GET /api/products/{id} - 获取产品")
|
||||
print(" • POST /api/products - 创建产品")
|
||||
print(" • PUT /api/products/{id} - 更新产品")
|
||||
print(" • DELETE /api/products/{id} - 删除产品")
|
||||
print()
|
||||
print("⏰ 定时任务（通过 @component 组件定义）:")
|
||||
print(" • 心跳检测 (每分钟)")
|
||||
print(" • 清理任务 (每10分钟)")
|
||||
print(" • 新年任务 (2025-12-31 23:59:59)")
|
||||
print()
|
||||
print("=" * 60)
|
||||
|
||||
# 启动应用
|
||||
app.run(host="0.0.0.0", port=8000, reload=False)
|
||||
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
依赖注入示例应用
|
||||
|
||||
展示 MyBoot 框架的依赖注入功能
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.decorators import service, rest_controller, get, post
|
||||
|
||||
# 添加项目根目录到 Python 路径
|
||||
project_root = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
# 创建应用实例
|
||||
app = Application(
|
||||
name="依赖注入示例",
|
||||
auto_configuration=True,
|
||||
auto_discover_package="examples.dependency_injection_example"
|
||||
)
|
||||
|
||||
|
||||
# ==================== 基础服务层 ====================
|
||||
|
||||
@service()
|
||||
class DatabaseClient:
|
||||
"""数据库客户端"""
|
||||
|
||||
def __init__(self):
|
||||
self.connection = "connected"
|
||||
print("✅ DatabaseClient 已初始化")
|
||||
|
||||
def query(self, sql: str):
|
||||
print(f"📊 执行查询: {sql}")
|
||||
return [{"id": 1, "name": "用户1"}]
|
||||
|
||||
|
||||
@service()
|
||||
class CacheService:
|
||||
"""缓存服务"""
|
||||
|
||||
def __init__(self):
|
||||
self.cache = {}
|
||||
print("✅ CacheService 已初始化")
|
||||
|
||||
def get(self, key: str):
|
||||
return self.cache.get(key)
|
||||
|
||||
def set(self, key: str, value: any):
|
||||
self.cache[key] = value
|
||||
|
||||
|
||||
# ==================== 仓储层 ====================
|
||||
|
||||
@service()
|
||||
class UserRepository:
|
||||
"""用户仓储 - 依赖 DatabaseClient"""
|
||||
|
||||
def __init__(self, db: DatabaseClient):
|
||||
self.db = db
|
||||
print("✅ UserRepository 已初始化（依赖: DatabaseClient）")
|
||||
|
||||
def find_by_id(self, user_id: int):
|
||||
result = self.db.query(f"SELECT * FROM users WHERE id = {user_id}")
|
||||
return result[0] if result else {"id": user_id, "name": f"用户{user_id}"}
|
||||
|
||||
|
||||
# ==================== 服务层 ====================
|
||||
|
||||
@service()
|
||||
class UserService:
|
||||
"""用户服务 - 依赖 UserRepository 和可选的 CacheService"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
user_repo: UserRepository,
|
||||
cache: Optional[CacheService] = None
|
||||
):
|
||||
self.user_repo = user_repo
|
||||
self.cache = cache
|
||||
print("✅ UserService 已初始化（依赖: UserRepository, CacheService）")
|
||||
|
||||
def get_user(self, user_id: int):
|
||||
# 尝试从缓存获取
|
||||
if self.cache:
|
||||
cached = self.cache.get(f"user:{user_id}")
|
||||
if cached:
|
||||
print(f"📦 从缓存获取用户 {user_id}")
|
||||
return cached
|
||||
|
||||
# 从数据库获取
|
||||
user = self.user_repo.find_by_id(user_id)
|
||||
|
||||
# 存入缓存
|
||||
if self.cache:
|
||||
self.cache.set(f"user:{user_id}", user)
|
||||
print(f"💾 用户 {user_id} 已存入缓存")
|
||||
|
||||
return user
|
||||
|
||||
|
||||
@service()
|
||||
class EmailService:
|
||||
"""邮件服务"""
|
||||
|
||||
def __init__(self):
|
||||
print("✅ EmailService 已初始化")
|
||||
|
||||
def send_email(self, to: str, subject: str, body: str):
|
||||
print(f"📧 发送邮件到 {to}: {subject} - {body}")
|
||||
return {"status": "sent", "to": to, "subject": subject}
|
||||
|
||||
|
||||
@service()
|
||||
class OrderService:
|
||||
"""订单服务 - 依赖 UserService 和 EmailService"""
|
||||
|
||||
def __init__(self, user_service: UserService, email_service: EmailService):
|
||||
self.user_service = user_service
|
||||
self.email_service = email_service
|
||||
print("✅ OrderService 已初始化（依赖: UserService, EmailService）")
|
||||
|
||||
def create_order(self, user_id: int, product: str):
|
||||
# 获取用户信息
|
||||
user = self.user_service.get_user(user_id)
|
||||
|
||||
# 发送邮件通知
|
||||
self.email_service.send_email(
|
||||
to=user.get('email', 'user@example.com'),
|
||||
subject="订单创建",
|
||||
body=f"您的订单 {product} 已创建"
|
||||
)
|
||||
|
||||
return {
|
||||
"order_id": 12345,
|
||||
"user_id": user_id,
|
||||
"product": product,
|
||||
"status": "created"
|
||||
}
|
||||
|
||||
|
||||
# ==================== REST 控制器 ====================
|
||||
# 注意：路由必须在 @rest_controller 装饰的类中定义，支持依赖注入
|
||||
|
||||
@rest_controller('/')
|
||||
class HomeController:
|
||||
"""首页控制器"""
|
||||
|
||||
@get('/')
|
||||
def home(self):
|
||||
"""首页（依赖注入示例）"""
|
||||
return {
|
||||
"message": "依赖注入示例应用",
|
||||
"features": [
|
||||
"自动依赖注入",
|
||||
"多级依赖支持",
|
||||
"可选依赖支持",
|
||||
"循环依赖检测",
|
||||
"REST 控制器"
|
||||
],
|
||||
"endpoints": [
|
||||
"GET /api/users/{user_id} - 获取用户信息",
|
||||
"POST /api/orders - 创建订单"
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@rest_controller('/api/users')
|
||||
class UserController:
|
||||
"""用户控制器 - 自动注入 UserService"""
|
||||
|
||||
def __init__(self, user_service: UserService):
|
||||
self.user_service = user_service
|
||||
|
||||
@get('/{user_id}')
|
||||
def get_user(self, user_id: int):
|
||||
"""获取用户信息 - GET /api/users/{user_id}"""
|
||||
return self.user_service.get_user(user_id)
|
||||
|
||||
|
||||
@rest_controller('/api/orders')
|
||||
class OrderController:
|
||||
"""订单控制器 - 自动注入 OrderService"""
|
||||
|
||||
def __init__(self, order_service: OrderService):
|
||||
self.order_service = order_service
|
||||
|
||||
@post('/')
|
||||
def create_order(self, user_id: int, product: str):
|
||||
"""创建订单 - POST /api/orders"""
|
||||
return self.order_service.create_order(user_id, product)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("\n" + "="*60)
|
||||
print("依赖注入示例应用")
|
||||
print("="*60 + "\n")
|
||||
|
||||
# 运行应用
|
||||
app.run(host="0.0.0.0", port=8000, reload=False)
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
MyBoot 应用包
|
||||
|
||||
核心应用代码，包含：
|
||||
- main.py: 应用入口
|
||||
- api/: 路由与视图层
|
||||
- core/: 核心基础设施
|
||||
- models/: ORM 模型层
|
||||
- services/: 业务逻辑层
|
||||
- jobs/: 定时任务
|
||||
- utils/: 工具函数
|
||||
"""
|
||||
|
||||
__all__ = [
|
||||
|
||||
]
|
||||
+531
@@ -0,0 +1,531 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
MyBoot CLI 工具
|
||||
|
||||
提供项目模板初始化功能
|
||||
"""
|
||||
|
||||
import click
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@click.group()
|
||||
def cli():
|
||||
"""MyBoot 命令行工具 - 项目模板初始化"""
|
||||
pass
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.option('--name', prompt='项目名称', help='项目名称')
|
||||
@click.option('--dir', default='.', help='项目目录（默认为当前目录）')
|
||||
@click.option('--template', type=click.Choice(['basic', 'api', 'full']), default='basic',
|
||||
help='项目模板: basic(基础), api(API项目), full(完整项目)')
|
||||
@click.option('--force', is_flag=True, help='如果目录已存在则覆盖')
|
||||
def init(name: str, dir: str, template: str, force: bool):
|
||||
"""初始化新的 MyBoot 项目"""
|
||||
|
||||
project_dir = Path(dir) / name
|
||||
|
||||
# 检查目录是否存在
|
||||
if project_dir.exists() and not force:
|
||||
click.echo(f"❌ 错误: 目录 '{project_dir}' 已存在", err=True)
|
||||
click.echo(" 使用 --force 选项可以覆盖现有目录")
|
||||
sys.exit(1)
|
||||
|
||||
click.echo(f"📦 正在初始化项目: {name}")
|
||||
click.echo(f" 模板: {template}")
|
||||
click.echo(f" 目录: {project_dir}")
|
||||
click.echo()
|
||||
|
||||
try:
|
||||
# 创建目录结构
|
||||
dirs = ['app', 'app/api', 'app/service', 'app/model', 'app/jobs', 'app/client', 'conf', 'tests']
|
||||
for d in dirs:
|
||||
(project_dir / d).mkdir(parents=True, exist_ok=True)
|
||||
# 创建 __init__.py
|
||||
init_file = project_dir / d / '__init__.py'
|
||||
if not init_file.exists():
|
||||
init_file.write_text('', encoding='utf-8')
|
||||
|
||||
click.echo("✓ 创建目录结构")
|
||||
|
||||
# 创建配置文件
|
||||
config_content = f"""# {name} 配置文件
|
||||
|
||||
# 应用配置
|
||||
app:
|
||||
name: "{name}"
|
||||
version: "0.1.0"
|
||||
|
||||
# 服务器配置
|
||||
server:
|
||||
port: 8000
|
||||
reload: true
|
||||
workers: 1
|
||||
keep_alive_timeout: 5
|
||||
graceful_timeout: 30
|
||||
response_format:
|
||||
enabled: true
|
||||
exclude_paths:
|
||||
- "/docs"
|
||||
|
||||
# 日志配置
|
||||
logging:
|
||||
level: "INFO"
|
||||
|
||||
# 任务调度配置
|
||||
scheduler:
|
||||
enabled: true
|
||||
timezone: "UTC"
|
||||
max_workers: 10
|
||||
"""
|
||||
config_file = project_dir / 'conf' / 'config.yaml'
|
||||
config_file.write_text(config_content, encoding='utf-8')
|
||||
click.echo("✓ 创建配置文件: conf/config.yaml")
|
||||
|
||||
# 创建主应用文件（放在根目录）
|
||||
app_content = f'''"""主应用文件"""
|
||||
from myboot.core.application import create_app
|
||||
|
||||
app = create_app(name="{name}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run()
|
||||
'''
|
||||
main_file = project_dir / 'main.py'
|
||||
main_file.write_text(app_content, encoding='utf-8')
|
||||
click.echo("✓ 创建主应用文件: main.py")
|
||||
|
||||
# 根据模板创建不同的文件
|
||||
if template in ['api', 'full']:
|
||||
# 创建示例路由
|
||||
api_content = '''"""API 路由示例"""
|
||||
from myboot.core.decorators import rest_controller, get
|
||||
|
||||
|
||||
@rest_controller('/api')
|
||||
class HelloController:
|
||||
"""Hello 控制器"""
|
||||
|
||||
@get('/')
|
||||
def hello(self):
|
||||
"""Hello World 接口 - GET /api"""
|
||||
return {"message": "Hello, MyBoot!", "status": "success"}
|
||||
|
||||
@get('/health')
|
||||
def health(self):
|
||||
"""健康检查接口 - GET /api/health"""
|
||||
return {"status": "healthy", "service": "running"}
|
||||
'''
|
||||
routes_file = project_dir / 'app' / 'api' / 'routes.py'
|
||||
routes_file.write_text(api_content, encoding='utf-8')
|
||||
click.echo("✓ 创建示例路由: app/api/routes.py")
|
||||
|
||||
if template == 'full':
|
||||
# 创建示例服务
|
||||
service_content = '''"""服务层示例"""
|
||||
from typing import Dict, Any
|
||||
|
||||
|
||||
class UserService:
|
||||
"""用户服务示例"""
|
||||
|
||||
def get_user(self, user_id: int) -> Dict[str, Any]:
|
||||
"""获取用户信息"""
|
||||
return {
|
||||
"id": user_id,
|
||||
"name": "示例用户",
|
||||
"email": "user@example.com"
|
||||
}
|
||||
|
||||
def create_user(self, name: str, email: str) -> Dict[str, Any]:
|
||||
"""创建用户"""
|
||||
return {
|
||||
"id": 1,
|
||||
"name": name,
|
||||
"email": email,
|
||||
"status": "created"
|
||||
}
|
||||
'''
|
||||
service_file = project_dir / 'app' / 'service' / 'user_service.py'
|
||||
service_file.write_text(service_content, encoding='utf-8')
|
||||
click.echo("✓ 创建示例服务: app/service/user_service.py")
|
||||
|
||||
# 创建示例模型
|
||||
model_content = '''"""数据模型示例"""
|
||||
from pydantic import BaseModel, ConfigDict, EmailStr
|
||||
from typing import Optional
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
"""用户模型"""
|
||||
model_config = ConfigDict(
|
||||
json_schema_extra={
|
||||
"example": {
|
||||
"name": "张三",
|
||||
"email": "zhangsan@example.com"
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
id: Optional[int] = None
|
||||
name: str
|
||||
email: EmailStr
|
||||
status: str = "active"
|
||||
'''
|
||||
model_file = project_dir / 'app' / 'model' / 'user.py'
|
||||
model_file.write_text(model_content, encoding='utf-8')
|
||||
click.echo("✓ 创建示例模型: app/model/user.py")
|
||||
|
||||
# 创建示例客户端
|
||||
client_content = '''"""客户端示例"""
|
||||
from typing import Dict, Any
|
||||
import requests
|
||||
|
||||
|
||||
class ApiClient:
|
||||
"""API 客户端示例"""
|
||||
|
||||
def __init__(self, base_url: str):
|
||||
"""初始化客户端"""
|
||||
self.base_url = base_url
|
||||
|
||||
def get(self, endpoint: str) -> Dict[str, Any]:
|
||||
"""GET 请求"""
|
||||
response = requests.get(f"{self.base_url}/{endpoint}")
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def post(self, endpoint: str, data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""POST 请求"""
|
||||
response = requests.post(f"{self.base_url}/{endpoint}", json=data)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
'''
|
||||
client_file = project_dir / 'app' / 'client' / 'api_client.py'
|
||||
client_file.write_text(client_content, encoding='utf-8')
|
||||
click.echo("✓ 创建示例客户端: app/client/api_client.py")
|
||||
|
||||
# 创建示例定时任务
|
||||
job_content = '''"""定时任务示例"""
|
||||
from myboot.core.decorators import cron
|
||||
|
||||
|
||||
@cron("0 */5 * * * *") # 每5分钟执行一次
|
||||
def cleanup_task():
|
||||
"""清理任务示例"""
|
||||
print("执行清理任务...")
|
||||
|
||||
|
||||
@cron("0 0 * * * *") # 每小时执行一次
|
||||
def hourly_task():
|
||||
"""每小时任务示例"""
|
||||
print("执行每小时任务...")
|
||||
'''
|
||||
job_file = project_dir / 'app' / 'jobs' / 'tasks.py'
|
||||
job_file.write_text(job_content, encoding='utf-8')
|
||||
click.echo("✓ 创建示例任务: app/jobs/tasks.py")
|
||||
|
||||
# 创建 README
|
||||
readme_content = f"""# {name}
|
||||
|
||||
MyBoot 项目
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 安装依赖
|
||||
|
||||
```bash
|
||||
pip install myboot
|
||||
```
|
||||
|
||||
### 运行应用
|
||||
|
||||
```bash
|
||||
# 使用默认设置启动
|
||||
python main.py
|
||||
|
||||
# 或者使用 myboot 命令（如果已安装）
|
||||
myboot run
|
||||
```
|
||||
|
||||
### 开发模式
|
||||
|
||||
```bash
|
||||
# 启用自动重载
|
||||
python main.py --reload
|
||||
|
||||
# 或者
|
||||
myboot dev --reload
|
||||
```
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
{name}/
|
||||
├── main.py # 应用入口
|
||||
├── pyproject.toml # 项目配置文件
|
||||
├── .gitignore # Git 忽略文件
|
||||
├── app/ # 应用代码
|
||||
│ ├── api/ # API 路由
|
||||
│ ├── service/ # 业务逻辑层
|
||||
│ ├── model/ # 数据模型
|
||||
│ ├── jobs/ # 定时任务
|
||||
│ └── client/ # 客户端（第三方API调用等）
|
||||
├── conf/ # 配置文件
|
||||
│ └── config.yaml # 主配置文件
|
||||
└── tests/ # 测试代码
|
||||
```
|
||||
|
||||
## 配置说明
|
||||
|
||||
配置文件位于 `conf/config.yaml`，支持以下配置：
|
||||
|
||||
- **app**: 应用配置（名称、版本等）
|
||||
- **server**: 服务器配置（端口、工作进程等）
|
||||
- **logging**: 日志配置
|
||||
- **scheduler**: 任务调度配置
|
||||
|
||||
## 更多信息
|
||||
|
||||
- [MyBoot 文档](https://github.com/your-org/myboot)
|
||||
- [快速开始指南](https://github.com/your-org/myboot/docs)
|
||||
"""
|
||||
readme_file = project_dir / 'README.md'
|
||||
readme_file.write_text(readme_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 README: README.md")
|
||||
|
||||
# 创建 .gitignore
|
||||
gitignore_content = """# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# Virtual Environment
|
||||
venv/
|
||||
ENV/
|
||||
env/
|
||||
.venv
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
logs/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# MyBoot
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
"""
|
||||
gitignore_file = project_dir / '.gitignore'
|
||||
gitignore_file.write_text(gitignore_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 .gitignore")
|
||||
|
||||
# 创建 pyproject.toml
|
||||
project_name = name.lower().replace(' ', '-').replace('_', '-')
|
||||
pyproject_content = f"""[project]
|
||||
name = "{project_name}"
|
||||
version = "0.1.0"
|
||||
description = "{name} - MyBoot 项目"
|
||||
authors = [
|
||||
{{name = "Your Name", email = "your.email@example.com"}}
|
||||
]
|
||||
readme = "README.md"
|
||||
license = {{text = "MIT"}}
|
||||
requires-python = ">=3.9"
|
||||
keywords = ["myboot", "web", "api"]
|
||||
classifiers = [
|
||||
"Development Status :: 3 - Alpha",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"myboot>=0.1.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=7.0",
|
||||
"pytest-cov>=4.0",
|
||||
"pytest-asyncio>=0.21.0",
|
||||
"black>=23.0",
|
||||
"isort>=5.12",
|
||||
"flake8>=6.0",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["app"]
|
||||
|
||||
[tool.black]
|
||||
line-length = 88
|
||||
target-version = ['py39', 'py310', 'py311', 'py312']
|
||||
include = '\\.pyi?$'
|
||||
|
||||
[tool.isort]
|
||||
profile = "black"
|
||||
line_length = 88
|
||||
known_first_party = ["app"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
python_files = ["test_*.py", "*_test.py"]
|
||||
python_classes = ["Test*"]
|
||||
python_functions = ["test_*"]
|
||||
addopts = [
|
||||
"--strict-markers",
|
||||
"--verbose",
|
||||
]
|
||||
"""
|
||||
pyproject_file = project_dir / 'pyproject.toml'
|
||||
pyproject_file.write_text(pyproject_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 pyproject.toml")
|
||||
|
||||
# 创建 Docker 支持文件（issue #1）
|
||||
dockerfile_content = f"""# 多阶段构建：构建层安装依赖，运行层只保留运行所需内容
|
||||
FROM python:3.12-slim AS builder
|
||||
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml ./
|
||||
# 如有 uv.lock 一并复制可获得可复现构建（无锁文件时此层退化为解析安装）
|
||||
RUN uv venv /opt/venv && \\
|
||||
VIRTUAL_ENV=/opt/venv uv pip install --no-cache .
|
||||
|
||||
FROM python:3.12-slim
|
||||
|
||||
ENV VIRTUAL_ENV=/opt/venv \\
|
||||
PATH="/opt/venv/bin:$PATH" \\
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
COPY . .
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
# 配置可用环境变量覆盖（__ 为层级分隔符），如 SERVER__PORT=8080
|
||||
CMD ["python", "main.py"]
|
||||
"""
|
||||
dockerfile = project_dir / 'Dockerfile'
|
||||
dockerfile.write_text(dockerfile_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 Dockerfile")
|
||||
|
||||
dockerignore_content = """__pycache__/
|
||||
*.py[cod]
|
||||
.venv/
|
||||
venv/
|
||||
.git/
|
||||
.gitignore
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
*.log
|
||||
logs/
|
||||
.vscode/
|
||||
.idea/
|
||||
Dockerfile
|
||||
docker-compose.yaml
|
||||
"""
|
||||
dockerignore_file = project_dir / '.dockerignore'
|
||||
dockerignore_file.write_text(dockerignore_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 .dockerignore")
|
||||
|
||||
compose_content = f"""services:
|
||||
{project_name}:
|
||||
build: .
|
||||
ports:
|
||||
- "8000:8000"
|
||||
environment:
|
||||
# MyBoot 配置支持环境变量覆盖，__ 为层级分隔符
|
||||
- LOGGING__LEVEL=INFO
|
||||
# - SERVER__WORKERS=4
|
||||
# - CONFIG_FILE=/app/conf/config.prod.yaml
|
||||
restart: unless-stopped
|
||||
"""
|
||||
compose_file = project_dir / 'docker-compose.yaml'
|
||||
compose_file.write_text(compose_content, encoding='utf-8')
|
||||
click.echo("✓ 创建 docker-compose.yaml")
|
||||
|
||||
click.echo()
|
||||
click.echo(f"✅ 项目 '{name}' 初始化完成！")
|
||||
click.echo()
|
||||
click.echo("下一步:")
|
||||
click.echo(f" cd {name}")
|
||||
click.echo(" python main.py")
|
||||
click.echo()
|
||||
click.echo("Docker 部署:")
|
||||
click.echo(f" docker build -t {project_name} .")
|
||||
click.echo(f" docker run -p 8000:8000 {project_name}")
|
||||
click.echo()
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"❌ 初始化失败: {e}", err=True)
|
||||
import traceback
|
||||
if '--debug' in sys.argv:
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@cli.command()
|
||||
def info():
|
||||
"""显示 MyBoot 信息"""
|
||||
click.echo("🎯 MyBoot - 类似 Spring Boot 的企业级Python快速开发框架")
|
||||
click.echo()
|
||||
click.echo("✨ 主要特性:")
|
||||
click.echo(" • 快速启动和自动配置")
|
||||
click.echo(" • 约定优于配置")
|
||||
click.echo(" • 高性能 Hypercorn 服务器")
|
||||
click.echo(" • Web API 开发")
|
||||
click.echo(" • 定时任务调度")
|
||||
click.echo(" • 日志管理")
|
||||
click.echo(" • 配置管理")
|
||||
click.echo()
|
||||
click.echo("🚀 快速开始:")
|
||||
click.echo(" myboot init # 初始化新项目")
|
||||
click.echo(" myboot init --template api # 使用 API 模板")
|
||||
click.echo(" myboot init --template full # 使用完整模板")
|
||||
click.echo()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
cli()
|
||||
@@ -0,0 +1,33 @@
|
||||
"""
|
||||
核心基础设施模块
|
||||
|
||||
包含配置、数据库、日志、调度器等核心功能
|
||||
"""
|
||||
|
||||
from .config import (
|
||||
get_settings,
|
||||
get_config,
|
||||
get_config_str,
|
||||
get_config_int,
|
||||
get_config_bool,
|
||||
reload_config
|
||||
)
|
||||
from .logger import Logger, get_logger, logger, setup_logging
|
||||
from .scheduler import Scheduler, get_scheduler
|
||||
from .auto_configuration import AutoConfigurationError
|
||||
|
||||
__all__ = [
|
||||
"get_settings",
|
||||
"get_config",
|
||||
"get_config_str",
|
||||
"get_config_int",
|
||||
"get_config_bool",
|
||||
"reload_config",
|
||||
"Logger",
|
||||
"get_logger",
|
||||
"logger", # loguru logger，建议直接使用
|
||||
"setup_logging",
|
||||
"Scheduler",
|
||||
"get_scheduler",
|
||||
"AutoConfigurationError",
|
||||
]
|
||||
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
auto_configuration 内部实现包
|
||||
|
||||
将 auto_configuration.py 的可分离职责拆分到此包，便于维护与测试：
|
||||
- ast_analyzer: AST 静态分析（解析装饰器、扫描文件/包元数据）
|
||||
- component_scanner: 模块发现/导入逻辑
|
||||
- scan_cache: 缓存读写与缓存文件路径计算
|
||||
|
||||
注意：本包内的模块不应反向导入 auto_configuration，以避免循环导入。
|
||||
"""
|
||||
@@ -0,0 +1,99 @@
|
||||
"""
|
||||
AST 静态分析
|
||||
|
||||
在发现阶段使用 AST 静态分析替代 import（不执行模块代码）：
|
||||
- parse_decorators: 提取节点上的装饰器名称
|
||||
- scan_file_ast: 扫描单个文件，提取被 MyBoot 装饰器标注的类/函数元数据
|
||||
- scan_package_ast: 递归扫描整个包
|
||||
|
||||
这些函数为纯分析函数，由调用方传入组件元数据容器与装饰器映射，
|
||||
不依赖 auto_configuration 模块，避免循环导入。
|
||||
"""
|
||||
|
||||
import os
|
||||
import ast
|
||||
from pathlib import Path
|
||||
from typing import Dict, List
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
logger = loguru_logger.bind(name=__name__)
|
||||
|
||||
|
||||
def parse_decorators(node: ast.AST) -> List[str]:
|
||||
"""解析装饰器名称"""
|
||||
decorators = []
|
||||
decorator_list = getattr(node, 'decorator_list', [])
|
||||
for dec in decorator_list:
|
||||
if isinstance(dec, ast.Name):
|
||||
decorators.append(dec.id)
|
||||
elif isinstance(dec, ast.Call):
|
||||
if isinstance(dec.func, ast.Name):
|
||||
decorators.append(dec.func.id)
|
||||
elif isinstance(dec.func, ast.Attribute):
|
||||
decorators.append(dec.func.attr)
|
||||
return decorators
|
||||
|
||||
|
||||
def scan_file_ast(
|
||||
file_path: Path,
|
||||
module_name: str,
|
||||
component_metadata: Dict[str, List[dict]],
|
||||
decorator_mapping: Dict[str, str],
|
||||
) -> None:
|
||||
"""使用 AST 静态分析扫描单个文件（不执行 import）
|
||||
|
||||
将发现的组件元数据就地追加到 component_metadata 中。
|
||||
"""
|
||||
try:
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
source = f.read()
|
||||
tree = ast.parse(source, filename=str(file_path))
|
||||
except Exception as e:
|
||||
logger.warning(f"AST 解析失败 {file_path}: {e}")
|
||||
return
|
||||
|
||||
# 只遍历模块顶层节点（避免 ast.walk 的问题）
|
||||
# 注意：job 方法（@cron/@interval/@once）只能在 @component 类中定义
|
||||
# job 方法的注册在 _auto_register_components 中动态进行，不在 AST 扫描阶段处理
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.ClassDef):
|
||||
decorators = parse_decorators(node)
|
||||
for dec_name in decorators:
|
||||
if dec_name in decorator_mapping:
|
||||
component_type = decorator_mapping[dec_name]
|
||||
component_metadata[component_type].append({
|
||||
'module': module_name,
|
||||
'class_name': node.name,
|
||||
'type': f'class_{dec_name}'
|
||||
})
|
||||
|
||||
elif isinstance(node, ast.FunctionDef):
|
||||
# 模块级函数
|
||||
decorators = parse_decorators(node)
|
||||
for dec_name in decorators:
|
||||
if dec_name in decorator_mapping:
|
||||
component_type = decorator_mapping[dec_name]
|
||||
component_metadata[component_type].append({
|
||||
'module': module_name,
|
||||
'func_name': node.name,
|
||||
'type': f'function_{dec_name}'
|
||||
})
|
||||
|
||||
|
||||
def scan_package_ast(
|
||||
package_path: Path,
|
||||
component_metadata: Dict[str, List[dict]],
|
||||
decorator_mapping: Dict[str, str],
|
||||
) -> None:
|
||||
"""使用 AST 递归扫描包（不执行 import）
|
||||
|
||||
将发现的组件元数据就地追加到 component_metadata 中。
|
||||
"""
|
||||
for item in package_path.rglob("*.py"):
|
||||
if item.name.startswith("__"):
|
||||
continue
|
||||
# 计算模块名
|
||||
rel_path = item.relative_to(package_path.parent)
|
||||
module_name = str(rel_path.with_suffix('')).replace(os.sep, '.')
|
||||
scan_file_ast(item, module_name, component_metadata, decorator_mapping)
|
||||
@@ -0,0 +1,123 @@
|
||||
"""
|
||||
组件模块发现与导入
|
||||
|
||||
负责注册阶段的延迟导入逻辑：
|
||||
- import_modules: 批量导入模块（支持并行/串行），并报告慢模块
|
||||
- build_discovered_components: 将组件元数据转换为包含实际类/函数对象的组件条目
|
||||
|
||||
这些函数为纯函数，由 AutoConfigurationManager 调用并传入所需状态，
|
||||
不依赖 auto_configuration 模块，避免循环导入。
|
||||
"""
|
||||
|
||||
import time
|
||||
import importlib
|
||||
from typing import Dict, List, Any, Set
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
logger = loguru_logger.bind(name=__name__)
|
||||
|
||||
|
||||
def _import_single(module_name: str):
|
||||
"""导入单个模块并返回结果"""
|
||||
try:
|
||||
start = time.perf_counter()
|
||||
module = importlib.import_module(module_name)
|
||||
elapsed = (time.perf_counter() - start) * 1000
|
||||
return module_name, module, elapsed, None
|
||||
except Exception as e:
|
||||
return module_name, None, 0, e
|
||||
|
||||
|
||||
def import_modules(modules_to_import: Set[str], parallel_import: bool = False) -> Dict[str, Any]:
|
||||
"""批量导入模块（支持并行/串行），返回 模块名 -> 模块 的映射
|
||||
|
||||
导入失败的模块会记录 error 日志并被跳过；导入耗时超过 100ms 的模块
|
||||
会汇总到一条 warning 日志中。
|
||||
"""
|
||||
imported_modules: Dict[str, Any] = {}
|
||||
slow_modules = [] # 记录慢模块
|
||||
|
||||
if parallel_import and len(modules_to_import) > 1:
|
||||
# 并行导入（对 I/O 密集的模块有帮助）
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
with ThreadPoolExecutor(max_workers=min(8, len(modules_to_import))) as executor:
|
||||
futures = {executor.submit(_import_single, m): m for m in modules_to_import}
|
||||
for future in as_completed(futures):
|
||||
module_name, module, elapsed, error = future.result()
|
||||
if error:
|
||||
logger.error(f"导入模块失败 {module_name}: {error}")
|
||||
else:
|
||||
imported_modules[module_name] = module
|
||||
if elapsed > 100:
|
||||
slow_modules.append((module_name, elapsed))
|
||||
else:
|
||||
# 串行导入
|
||||
for module_name in modules_to_import:
|
||||
module_name, module, elapsed, error = _import_single(module_name)
|
||||
if error:
|
||||
logger.error(f"导入模块失败 {module_name}: {error}")
|
||||
else:
|
||||
imported_modules[module_name] = module
|
||||
if elapsed > 100:
|
||||
slow_modules.append((module_name, elapsed))
|
||||
|
||||
# 输出慢模块报告
|
||||
if slow_modules:
|
||||
slow_modules.sort(key=lambda x: x[1], reverse=True)
|
||||
report = ", ".join([f"{name}({ms:.0f}ms)" for name, ms in slow_modules[:10]])
|
||||
logger.warning(f"慢模块导入: {report}")
|
||||
|
||||
return imported_modules
|
||||
|
||||
|
||||
def build_discovered_components(
|
||||
component_metadata: Dict[str, List[dict]],
|
||||
parallel_import: bool = False,
|
||||
) -> Dict[str, List[dict]]:
|
||||
"""将组件元数据转换为包含实际类对象的组件条目
|
||||
|
||||
收集需导入的模块并去重，执行导入后将元数据解析为实际的类/函数/方法对象。
|
||||
返回与 component_metadata 同结构的 discovered_components 字典。
|
||||
"""
|
||||
# 收集需要导入的模块（去重）
|
||||
modules_to_import: Set[str] = set()
|
||||
for items in component_metadata.values():
|
||||
for item in items:
|
||||
modules_to_import.add(item['module'])
|
||||
|
||||
# 批量导入模块
|
||||
imported_modules = import_modules(modules_to_import, parallel_import)
|
||||
|
||||
# 将元数据转换为包含实际类对象的组件
|
||||
discovered_components: Dict[str, List[dict]] = {key: [] for key in component_metadata.keys()}
|
||||
for component_type, items in component_metadata.items():
|
||||
for item in items:
|
||||
module = imported_modules.get(item['module'])
|
||||
if not module:
|
||||
continue
|
||||
|
||||
entry = {'module': item['module'], 'type': item['type']}
|
||||
|
||||
if 'class_name' in item:
|
||||
cls = getattr(module, item['class_name'], None)
|
||||
if cls:
|
||||
entry['class'] = cls
|
||||
else:
|
||||
continue
|
||||
|
||||
if 'func_name' in item:
|
||||
func = getattr(module, item['func_name'], None)
|
||||
if func:
|
||||
entry['function'] = func
|
||||
else:
|
||||
continue
|
||||
|
||||
if 'method_name' in item:
|
||||
entry['method_name'] = item['method_name']
|
||||
if 'class' in entry:
|
||||
entry['method'] = getattr(entry['class'], item['method_name'], None)
|
||||
|
||||
discovered_components[component_type].append(entry)
|
||||
|
||||
return discovered_components
|
||||
@@ -0,0 +1,96 @@
|
||||
"""
|
||||
扫描缓存读写
|
||||
|
||||
负责 AST 分析结果（组件元数据）的缓存持久化：
|
||||
- 缓存版本号 _CACHE_VERSION（修改扫描逻辑时递增以使旧缓存失效）
|
||||
- 缓存文件路径计算
|
||||
- 源文件清单收集（用于失效判断）
|
||||
- 缓存有效性检查、保存与加载
|
||||
|
||||
这些函数为纯函数，由 AutoConfigurationManager 调用并传入所需状态，
|
||||
不依赖 auto_configuration 模块，避免循环导入。
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
logger = loguru_logger.bind(name=__name__)
|
||||
|
||||
|
||||
# 缓存版本号，修改扫描逻辑时递增以使旧缓存失效
|
||||
# 3.1: 新增 worker_hooks 组件类型（@on_worker_start/@on_worker_stop）
|
||||
_CACHE_VERSION = "3.1"
|
||||
|
||||
|
||||
def get_cache_path(app_root: str, package_name: str) -> Path:
|
||||
"""获取缓存文件路径"""
|
||||
return Path(app_root) / f".myboot_cache_{package_name}.json"
|
||||
|
||||
|
||||
def collect_source_files(package_path: Path) -> Dict[str, float]:
|
||||
"""收集扫描范围内的源文件及其修改时间（支持单文件或目录）"""
|
||||
if package_path.is_file():
|
||||
return {str(package_path): package_path.stat().st_mtime}
|
||||
|
||||
files = {}
|
||||
for item in package_path.rglob("*.py"):
|
||||
if not item.name.startswith("__"):
|
||||
files[str(item)] = item.stat().st_mtime
|
||||
return files
|
||||
|
||||
|
||||
def is_cache_valid(cache_path: Path, package_path: Path) -> bool:
|
||||
"""检查缓存是否有效"""
|
||||
if not cache_path.exists():
|
||||
return False
|
||||
try:
|
||||
with open(cache_path, 'r', encoding='utf-8') as f:
|
||||
cache = json.load(f)
|
||||
if cache.get('version') != _CACHE_VERSION:
|
||||
return False
|
||||
current_files = collect_source_files(package_path)
|
||||
cached_files = cache.get('source_files', {})
|
||||
return current_files == cached_files
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def save_cache(cache_path: Path, package_path: Path, component_metadata: Dict[str, List[dict]]) -> None:
|
||||
"""保存元数据缓存（不包含类对象）"""
|
||||
try:
|
||||
cache = {
|
||||
'version': _CACHE_VERSION,
|
||||
'source_files': collect_source_files(package_path),
|
||||
'components': component_metadata
|
||||
}
|
||||
with open(cache_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(cache, f, indent=2)
|
||||
except Exception as e:
|
||||
logger.warning(f"保存缓存失败: {e}")
|
||||
|
||||
|
||||
def load_cache(cache_path: Path, component_keys) -> Optional[Dict[str, List[dict]]]:
|
||||
"""从缓存加载元数据（不导入模块）
|
||||
|
||||
成功时返回加载并规范化后的组件元数据字典（补齐 component_keys 中缺失的
|
||||
组件类型键，避免后续 KeyError）；失败时返回 None。
|
||||
|
||||
Args:
|
||||
cache_path: 缓存文件路径
|
||||
component_keys: 需要补齐的组件类型键集合（通常为当前元数据字典的 keys）
|
||||
"""
|
||||
try:
|
||||
with open(cache_path, 'r', encoding='utf-8') as f:
|
||||
cache = json.load(f)
|
||||
loaded = cache.get('components', {})
|
||||
# 规范化：补齐缺失的组件类型键，避免后续 KeyError
|
||||
for key in component_keys:
|
||||
loaded.setdefault(key, [])
|
||||
return loaded
|
||||
except Exception as e:
|
||||
logger.warning(f"加载缓存失败: {e}")
|
||||
return None
|
||||
@@ -0,0 +1,835 @@
|
||||
"""
|
||||
MyBoot 应用程序主类
|
||||
|
||||
提供类似 Spring Boot 的自动配置和快速启动功能
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import signal
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any, Callable, Dict, List, Optional, Type
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse
|
||||
from loguru import logger
|
||||
|
||||
from .auto_configuration import auto_discover, apply_auto_configuration
|
||||
from .config import get_settings
|
||||
from .container import Container
|
||||
from .logger import setup_logging
|
||||
from .scheduler import Scheduler
|
||||
from .server import ServerManager
|
||||
from ..exceptions import MyBootException
|
||||
from ..utils import get_local_ip
|
||||
from ..web.middleware import Middleware
|
||||
|
||||
# 全局应用实例注册表（用于在路由函数中获取当前应用实例）
|
||||
_current_app: Optional['Application'] = None
|
||||
|
||||
|
||||
def app() -> 'Application':
|
||||
"""获取当前应用实例"""
|
||||
if _current_app is None:
|
||||
raise RuntimeError("应用实例未初始化，请确保应用已创建并启动")
|
||||
return _current_app
|
||||
|
||||
|
||||
class Application:
|
||||
"""MyBoot 应用程序主类"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
name: str = "MyBoot App",
|
||||
config_file: Optional[str] = None,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
初始化应用程序
|
||||
|
||||
Args:
|
||||
name: 应用程序名称
|
||||
config_file: 配置文件路径
|
||||
**kwargs: 其他配置参数
|
||||
"""
|
||||
self.name = name
|
||||
self.config = get_settings(config_file)
|
||||
|
||||
# 应用配置
|
||||
self._apply_config(kwargs)
|
||||
|
||||
# 获取应用版本号（从配置文件读取，默认 0.0.1）
|
||||
self.version = self.config.get("app.version", "0.0.1")
|
||||
|
||||
# 初始化 loguru 日志系统（包括第三方库日志级别配置）
|
||||
setup_logging(self.config)
|
||||
|
||||
# Prometheus 多进程指标环境（须在 prometheus_client 被 import 之前配置）
|
||||
from ..metrics import setup_multiproc_env
|
||||
setup_multiproc_env(self.config, self.name)
|
||||
|
||||
self.logger = logger.bind(name=self.name)
|
||||
|
||||
# Worker 信息（多进程模式下由环境变量设置）
|
||||
self._worker_id = int(os.environ.get("MYBOOT_WORKER_ID", "1"))
|
||||
self._worker_count = int(os.environ.get("MYBOOT_WORKER_COUNT", "1"))
|
||||
self._is_primary_worker = os.environ.get("MYBOOT_IS_PRIMARY_WORKER", "1") == "1"
|
||||
|
||||
# Scheduler 默认只在 primary worker 启动（可通过 scheduler.on_all_workers
|
||||
# 全局配置覆盖）。任务级 all_workers=True 的任务在非 primary worker 注册时，
|
||||
# 注册门控会调用 scheduler.enable() 按需启用调度器实例。
|
||||
scheduler_on_all_workers = self.config.get("scheduler.on_all_workers", False)
|
||||
scheduler_globally_enabled = self.config.get("scheduler.enabled", True)
|
||||
self._scheduler_enabled = (
|
||||
self._is_primary_worker or scheduler_on_all_workers
|
||||
) and scheduler_globally_enabled
|
||||
self.scheduler = Scheduler(config=self.config, enabled=self._scheduler_enabled)
|
||||
|
||||
# 中间件列表
|
||||
self.middlewares: List[Middleware] = []
|
||||
|
||||
# 路由处理器
|
||||
self.route_handlers: Dict[str, Callable] = {}
|
||||
|
||||
# 服务注册表
|
||||
self.services: Dict[str, Any] = {}
|
||||
|
||||
# 模型注册表
|
||||
self.models: Dict[str, Any] = {}
|
||||
|
||||
# 客户端注册表
|
||||
self.clients: Dict[str, Any] = {}
|
||||
|
||||
# 组件注册表
|
||||
self.components: Dict[str, Any] = {}
|
||||
|
||||
# 统一容器接口（支持从 container、services、clients 中获取实例）
|
||||
self.container = Container(self)
|
||||
|
||||
# 启动钩子
|
||||
self.startup_hooks: List[Callable] = []
|
||||
self.shutdown_hooks: List[Callable] = []
|
||||
|
||||
# Worker 生命周期钩子（@on_worker_start / @on_worker_stop）
|
||||
# 每个 worker 进程的 lifespan 中各触发一次；单 worker 模式下也触发一次
|
||||
self.worker_start_hooks: List[Callable] = []
|
||||
self.worker_stop_hooks: List[Callable] = []
|
||||
|
||||
# FastAPI 应用实例
|
||||
self._fastapi_app: Optional[FastAPI] = None
|
||||
|
||||
# 服务器实例
|
||||
self._server: Optional[Any] = None
|
||||
|
||||
# 注册信号处理器
|
||||
self._register_signal_handlers()
|
||||
|
||||
# 创建 FastAPI 应用
|
||||
self._fastapi_app = self._create_fastapi_app()
|
||||
|
||||
# 自动配置标志
|
||||
self.auto_configuration_enabled = kwargs.get('auto_configuration', True)
|
||||
self.auto_discover_package = kwargs.get('auto_discover_package', 'app')
|
||||
|
||||
# 服务器管理器
|
||||
self.server_manager = ServerManager()
|
||||
|
||||
# 注册为当前应用实例
|
||||
global _current_app
|
||||
_current_app = self
|
||||
|
||||
def _apply_config(self, kwargs: Dict[str, Any]) -> None:
|
||||
"""应用配置参数"""
|
||||
for key, value in kwargs.items():
|
||||
self.config.set(key, value)
|
||||
|
||||
def _register_signal_handlers(self) -> None:
|
||||
"""注册信号处理器"""
|
||||
signal.signal(signal.SIGINT, self._signal_handler)
|
||||
signal.signal(signal.SIGTERM, self._signal_handler)
|
||||
|
||||
def _signal_handler(self, signum: int, frame) -> None:
|
||||
"""信号处理器"""
|
||||
self.logger.info(f"收到信号 {signum}，开始优雅关闭...")
|
||||
asyncio.create_task(self.shutdown())
|
||||
|
||||
def add_middleware(self, middleware: Middleware) -> None:
|
||||
"""添加中间件"""
|
||||
self.middlewares.append(middleware)
|
||||
self.logger.debug(f"已添加中间件: {middleware.__class__.__name__}")
|
||||
|
||||
def add_startup_hook(self, hook: Callable) -> None:
|
||||
"""添加启动钩子"""
|
||||
self.startup_hooks.append(hook)
|
||||
self.logger.debug(f"已添加启动钩子: {hook.__name__}")
|
||||
|
||||
def add_shutdown_hook(self, hook: Callable) -> None:
|
||||
"""添加关闭钩子"""
|
||||
self.shutdown_hooks.append(hook)
|
||||
self.logger.debug(f"已添加关闭钩子: {hook.__name__}")
|
||||
|
||||
def add_worker_start_hook(self, hook: Callable) -> None:
|
||||
"""添加 worker 启动钩子
|
||||
|
||||
在每个 worker 进程的 lifespan 启动阶段触发（startup_hooks 之后、
|
||||
调度器启动之前），单 worker 模式下也触发一次。
|
||||
"""
|
||||
self.worker_start_hooks.append(hook)
|
||||
self.logger.debug(f"已添加 worker 启动钩子: {hook.__name__}")
|
||||
|
||||
def add_worker_stop_hook(self, hook: Callable) -> None:
|
||||
"""添加 worker 停止钩子
|
||||
|
||||
在每个 worker 进程的 lifespan 关闭阶段触发（调度器停止之后、
|
||||
shutdown_hooks 之前）。
|
||||
|
||||
注意：Windows 多 worker 模式下父进程通过 terminate()（硬终止）
|
||||
清理 worker 进程，lifespan 关闭阶段可能不会执行，
|
||||
因此 worker 停止钩子在 Windows 上不保证触发。
|
||||
"""
|
||||
self.worker_stop_hooks.append(hook)
|
||||
self.logger.debug(f"已添加 worker 停止钩子: {hook.__name__}")
|
||||
|
||||
def register_service(self, name: str, service: Any) -> None:
|
||||
"""注册服务"""
|
||||
self.services[name] = service
|
||||
self.logger.debug(f"已注册服务: {name}")
|
||||
|
||||
def get_service(self, name: str) -> Any:
|
||||
"""获取服务
|
||||
|
||||
优先返回 app.services 中的单例实例；非单例（request/factory）服务
|
||||
不在 app.services 中预存，回退到 DI 容器按需解析。
|
||||
"""
|
||||
if name in self.services:
|
||||
return self.services[name]
|
||||
di_container = getattr(self, 'di_container', None)
|
||||
if di_container is not None and di_container.has_service(name):
|
||||
return di_container.get_service(name)
|
||||
return None
|
||||
|
||||
def has_service(self, name: str) -> bool:
|
||||
"""检查是否有服务"""
|
||||
return name in self.services
|
||||
|
||||
def get_client(self, name: str) -> Any:
|
||||
"""获取客户端"""
|
||||
return self.clients.get(name)
|
||||
|
||||
def has_client(self, name: str) -> bool:
|
||||
"""检查是否有客户端"""
|
||||
return name in self.clients
|
||||
|
||||
def get_component(self, name: str) -> Any:
|
||||
"""获取组件"""
|
||||
return self.components.get(name)
|
||||
|
||||
def has_component(self, name: str) -> bool:
|
||||
"""检查是否有组件"""
|
||||
return name in self.components
|
||||
|
||||
def route(
|
||||
self,
|
||||
path: str,
|
||||
methods: Optional[List[str]] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""
|
||||
装饰器：注册路由
|
||||
|
||||
Args:
|
||||
path: 路由路径
|
||||
methods: HTTP 方法列表
|
||||
**kwargs: 其他 FastAPI 路由参数
|
||||
"""
|
||||
if methods is None:
|
||||
methods = ["GET"]
|
||||
|
||||
def decorator(func: Callable) -> Callable:
|
||||
# 存储路由处理器
|
||||
route_key = f"{','.join(methods)}:{path}"
|
||||
self.route_handlers[route_key] = func
|
||||
|
||||
self.logger.debug(f"已注册路由: {methods} {path} -> {func.__name__}")
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
def get(self, path: str, **kwargs) -> Callable:
|
||||
"""GET 路由装饰器"""
|
||||
return self.route(path, ["GET"], **kwargs)
|
||||
|
||||
def post(self, path: str, **kwargs) -> Callable:
|
||||
"""POST 路由装饰器"""
|
||||
return self.route(path, ["POST"], **kwargs)
|
||||
|
||||
def put(self, path: str, **kwargs) -> Callable:
|
||||
"""PUT 路由装饰器"""
|
||||
return self.route(path, ["PUT"], **kwargs)
|
||||
|
||||
def delete(self, path: str, **kwargs) -> Callable:
|
||||
"""DELETE 路由装饰器"""
|
||||
return self.route(path, ["DELETE"], **kwargs)
|
||||
|
||||
def patch(self, path: str, **kwargs) -> Callable:
|
||||
"""PATCH 路由装饰器"""
|
||||
return self.route(path, ["PATCH"], **kwargs)
|
||||
|
||||
def _create_fastapi_app(self) -> FastAPI:
|
||||
"""创建 FastAPI 应用实例"""
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""应用生命周期管理"""
|
||||
|
||||
# 执行启动钩子
|
||||
for hook in self.startup_hooks:
|
||||
try:
|
||||
if asyncio.iscoroutinefunction(hook):
|
||||
await hook()
|
||||
else:
|
||||
hook()
|
||||
except Exception as e:
|
||||
self.logger.error(f"启动钩子执行失败: {e}", exc_info=True)
|
||||
|
||||
# 执行 worker 启动钩子（每个 worker 进程的 lifespan 各触发一次，
|
||||
# 位于 startup_hooks 之后、调度器启动之前）
|
||||
for hook in self.worker_start_hooks:
|
||||
try:
|
||||
if asyncio.iscoroutinefunction(hook):
|
||||
await hook()
|
||||
else:
|
||||
hook()
|
||||
except Exception as e:
|
||||
self.logger.error(f"Worker 启动钩子执行失败: {e}", exc_info=True)
|
||||
|
||||
# 启动调度器
|
||||
if self.scheduler.has_jobs():
|
||||
self.scheduler.start()
|
||||
self.logger.info("✅ 任务调度器已启动")
|
||||
|
||||
yield
|
||||
|
||||
# 关闭
|
||||
self.logger.info(f"🛑 关闭 {self.name}...")
|
||||
|
||||
# 停止调度器
|
||||
if self.scheduler.is_running():
|
||||
self.scheduler.stop()
|
||||
self.logger.info("✅ 任务调度器已停止")
|
||||
|
||||
# 执行 worker 停止钩子（调度器停止之后、shutdown_hooks 之前）
|
||||
# 注意：Windows 多 worker 模式下父进程使用 terminate() 硬终止
|
||||
# worker，lifespan 关闭阶段可能不执行，stop 钩子不保证触发
|
||||
for hook in self.worker_stop_hooks:
|
||||
try:
|
||||
if asyncio.iscoroutinefunction(hook):
|
||||
await hook()
|
||||
else:
|
||||
hook()
|
||||
except Exception as e:
|
||||
self.logger.error(f"Worker 停止钩子执行失败: {e}", exc_info=True)
|
||||
|
||||
# 自动关闭 client：对具有 close() 方法的 client 实例兜底调用
|
||||
# （worker_stop_hooks 之后，shutdown_hooks 之前）。
|
||||
# 应用若已自行 close，建议实现为幂等；二次 close 的异常降为 warning
|
||||
for client_name, client_instance in list(self.clients.items()):
|
||||
close_method = getattr(client_instance, "close", None)
|
||||
if not callable(close_method):
|
||||
continue
|
||||
try:
|
||||
if asyncio.iscoroutinefunction(close_method):
|
||||
await close_method()
|
||||
else:
|
||||
close_method()
|
||||
self.logger.debug(f"Client 已自动关闭: {client_name}")
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Client '{client_name}' close() 异常（已忽略）: {e}")
|
||||
|
||||
# 执行关闭钩子
|
||||
for hook in self.shutdown_hooks:
|
||||
try:
|
||||
if asyncio.iscoroutinefunction(hook):
|
||||
await hook()
|
||||
else:
|
||||
hook()
|
||||
except Exception as e:
|
||||
self.logger.error(f"关闭钩子执行失败: {e}", exc_info=True)
|
||||
|
||||
# Prometheus multiproc：标记本进程退出，清理 gauge 残留文件
|
||||
# （内部已兜底 try/except，此处再防御一层确保关闭流程不被打断）
|
||||
try:
|
||||
from ..metrics import mark_current_process_dead
|
||||
mark_current_process_dead()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 创建 FastAPI 应用
|
||||
|
||||
app = FastAPI(
|
||||
title=self.name,
|
||||
version=self.version,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
# 添加 CORS 中间件（如果配置了 server.cors）
|
||||
cors_config = self.config.get("server.cors")
|
||||
if cors_config:
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=cors_config.get("allow_origins", ["*"]),
|
||||
allow_credentials=cors_config.get("allow_credentials", True),
|
||||
allow_methods=cors_config.get("allow_methods", ["*"]),
|
||||
allow_headers=cors_config.get("allow_headers", ["*"]),
|
||||
)
|
||||
self.logger.debug("CORS 中间件已启用")
|
||||
|
||||
# Prometheus 指标（metrics.enabled 时启用，可选依赖缺失只告警不报错）
|
||||
from ..metrics import is_available as metrics_available, is_enabled as metrics_enabled
|
||||
metrics_path = str(self.config.get("metrics.path", "/metrics") or "/metrics")
|
||||
metrics_ready = False
|
||||
if metrics_enabled(self.config):
|
||||
if metrics_available():
|
||||
metrics_ready = True
|
||||
from ..metrics import _coerce_bool
|
||||
if _coerce_bool(self.config.get("metrics.http_metrics", True), True):
|
||||
from ..metrics import HttpMetricsMiddleware
|
||||
app.add_middleware(HttpMetricsMiddleware, metrics_path=metrics_path)
|
||||
self.logger.debug("HTTP 指标中间件已启用")
|
||||
else:
|
||||
self.logger.warning(
|
||||
"metrics.enabled=true 但未安装 prometheus-client，"
|
||||
"指标功能不可用。安装方式: pip install myboot[metrics]"
|
||||
)
|
||||
|
||||
# 添加自定义中间件
|
||||
for middleware in self.middlewares:
|
||||
app.add_middleware(middleware.middleware_class, **middleware.kwargs)
|
||||
|
||||
# 添加响应格式化中间件（最后添加，因为它会最先执行）
|
||||
# FastAPI 中间件是后进先出（LIFO），所以最后添加的中间件会最先处理响应
|
||||
response_format_enabled = self.config.get("server.response_format.enabled", True)
|
||||
if response_format_enabled:
|
||||
from myboot.web.middleware import ResponseFormatterMiddleware
|
||||
exclude_paths = list(self.config.get("server.response_format.exclude_paths", []) or [])
|
||||
if metrics_ready:
|
||||
# metrics 端点输出 Prometheus 文本格式，不做 JSON 包装
|
||||
exclude_paths.append(metrics_path)
|
||||
app.add_middleware(
|
||||
ResponseFormatterMiddleware,
|
||||
exclude_paths=exclude_paths,
|
||||
auto_wrap=True
|
||||
)
|
||||
self.logger.debug("响应格式化中间件已启用")
|
||||
|
||||
# 注册路由
|
||||
self._register_routes(app)
|
||||
|
||||
# 注册异常处理器
|
||||
self._register_exception_handlers(app)
|
||||
|
||||
# 添加健康检查端点
|
||||
self._add_health_endpoints(app)
|
||||
|
||||
# 挂载 Prometheus 指标端点（懒初始化，首次请求才 import prometheus_client）
|
||||
if metrics_ready:
|
||||
from ..metrics import make_metrics_asgi_app
|
||||
app.mount(metrics_path, make_metrics_asgi_app())
|
||||
self.logger.debug(f"Prometheus 指标端点已挂载: {metrics_path}")
|
||||
|
||||
return app
|
||||
|
||||
def _register_routes(self, app: FastAPI) -> None:
|
||||
"""注册路由到 FastAPI 应用"""
|
||||
for route_key, handler in self.route_handlers.items():
|
||||
methods, path = route_key.split(":", 1)
|
||||
method_list = methods.split(",")
|
||||
|
||||
# 添加路由到 FastAPI
|
||||
app.add_api_route(
|
||||
path,
|
||||
handler,
|
||||
methods=method_list,
|
||||
name=handler.__name__
|
||||
)
|
||||
|
||||
def _register_exception_handlers(self, app: FastAPI) -> None:
|
||||
"""注册异常处理器"""
|
||||
|
||||
def _extract_error_messages(errors: list) -> list:
|
||||
"""从错误列表中提取错误消息"""
|
||||
messages = []
|
||||
for error in errors:
|
||||
msg = error.get('msg', 'Validation Error')
|
||||
# 移除 "Value error, " 前缀
|
||||
if msg.startswith('Value error, '):
|
||||
msg = msg[len('Value error, '):]
|
||||
messages.append(msg)
|
||||
return messages
|
||||
|
||||
@app.exception_handler(MyBootException)
|
||||
async def myboot_exception_handler(request: Request, exc: MyBootException):
|
||||
"""MyBoot 异常处理器"""
|
||||
self.logger.error(f"MyBoot 异常: {exc}", exc_info=True)
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={
|
||||
"success": False,
|
||||
"code": 500,
|
||||
"message": "Internal Server Error",
|
||||
"data": {
|
||||
"type": exc.__class__.__name__
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
@app.exception_handler(HTTPException)
|
||||
async def http_exception_handler(request: Request, exc: HTTPException):
|
||||
"""HTTP 异常处理器"""
|
||||
self.logger.warning(f"HTTP 异常: {exc.status_code} - {exc.detail}")
|
||||
return JSONResponse(
|
||||
status_code=exc.status_code,
|
||||
content={
|
||||
"success": False,
|
||||
"code": exc.status_code,
|
||||
"message": "HTTP Error"
|
||||
}
|
||||
)
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_exception_handler(request: Request, exc: RequestValidationError):
|
||||
"""请求验证异常处理器 - 返回友好的错误信息"""
|
||||
errors = exc.errors()
|
||||
# 只提取错误消息
|
||||
error_messages = _extract_error_messages(errors)
|
||||
|
||||
# 使用第一个错误消息作为主要错误信息
|
||||
error_msg = error_messages[0] if error_messages else 'Validation Error'
|
||||
|
||||
self.logger.warning(f"请求验证失败: {error_messages}")
|
||||
return JSONResponse(
|
||||
status_code=422,
|
||||
content={
|
||||
"success": False,
|
||||
"code": 422,
|
||||
"message": error_msg,
|
||||
"data": {
|
||||
"fieldErrors": error_messages
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def global_exception_handler(request: Request, exc: Exception):
|
||||
"""全局异常处理器"""
|
||||
self.logger.error(f"未处理的异常: {exc}", exc_info=True)
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={
|
||||
"success": False,
|
||||
"code": 500,
|
||||
"message": "Internal Server Error",
|
||||
"data": {
|
||||
"type": exc.__class__.__name__
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
def _add_health_endpoints(self, app: FastAPI) -> None:
|
||||
"""添加健康检查端点"""
|
||||
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
"""健康检查端点"""
|
||||
return {
|
||||
"status": "healthy",
|
||||
"app": self.name,
|
||||
"version": self.version,
|
||||
"uptime": "running"
|
||||
}
|
||||
|
||||
@app.get("/health/ready")
|
||||
async def readiness_check():
|
||||
"""就绪检查端点"""
|
||||
return {
|
||||
"status": "ready",
|
||||
"app": self.name,
|
||||
"services": {
|
||||
"scheduler": self.scheduler.is_running() if self.scheduler.has_jobs() else "disabled"
|
||||
}
|
||||
}
|
||||
|
||||
@app.get("/health/live")
|
||||
async def liveness_check():
|
||||
"""存活检查端点"""
|
||||
return {
|
||||
"status": "alive",
|
||||
"app": self.name
|
||||
}
|
||||
|
||||
def run(
|
||||
self,
|
||||
host: str = "0.0.0.0",
|
||||
port: int = 8000,
|
||||
reload: bool = False,
|
||||
workers: int = 1,
|
||||
app_path: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> None:
|
||||
"""
|
||||
运行应用程序
|
||||
|
||||
Args:
|
||||
host: 主机地址
|
||||
port: 端口号
|
||||
reload: 是否开启热重载
|
||||
workers: 工作进程数
|
||||
app_path: 应用模块路径（多 workers 模式必需），格式为 "module.path:app_name"
|
||||
例如: "main:app" 表示从 main.py 导入 app 变量
|
||||
如果 app 是通过 get_fastapi_app() 获取，使用 "main:app.get_fastapi_app()"
|
||||
**kwargs: 其他服务器参数
|
||||
|
||||
Example:
|
||||
# 单进程模式（默认）
|
||||
app.run(host="0.0.0.0", port=8000)
|
||||
|
||||
# 多进程模式（4个 workers）
|
||||
app.run(
|
||||
host="0.0.0.0",
|
||||
port=8000,
|
||||
workers=4,
|
||||
app_path="main:app.get_fastapi_app()"
|
||||
)
|
||||
"""
|
||||
# 从配置中获取参数
|
||||
host = self.config.get("server.host", host)
|
||||
port = self.config.get("server.port", port)
|
||||
reload = self.config.get("server.reload", reload)
|
||||
workers = self.config.get("server.workers", workers)
|
||||
app_path = self.config.get("server.app_path", app_path)
|
||||
|
||||
# 白名单方式收集 server.* 中的 Hypercorn 配置
|
||||
# 别名转换（如 max_incomplete_request_size）由 server.py 的 HYPERCORN_CONFIG_ALIASES 统一处理
|
||||
server_section = self.config.get("server") or {}
|
||||
server_kwargs = {}
|
||||
for k in ("backlog", "read_timeout", "keep_alive_timeout", "graceful_timeout",
|
||||
"reloader", "use_reloader", "max_incomplete_request_size"):
|
||||
if k in server_section:
|
||||
server_kwargs[k] = server_section[k]
|
||||
# 合并：配置文件 < run(**kwargs) 覆盖
|
||||
run_kwargs = {**server_kwargs, **kwargs}
|
||||
|
||||
# 自动发现和配置
|
||||
# 多 worker 模式（workers > 1 且提供了 app_path）下，父进程只做 AST 自动发现
|
||||
# （无副作用，预热 .myboot_cache_*.json 供子进程使用），实例化（apply_auto_configuration）
|
||||
# 延迟到每个 worker 进程内的 bootstrap_worker() 执行，
|
||||
# 避免 fork 模式下所有 worker 共享父进程预先创建的 client/service 实例（issue #11）。
|
||||
# 单 worker 模式与缺少 app_path 的回退路径（父进程自己 serve）保持原有行为不变。
|
||||
if self.auto_configuration_enabled:
|
||||
self.logger.info("🔍 开始自动发现组件...")
|
||||
auto_discover(self.auto_discover_package)
|
||||
if workers > 1 and app_path:
|
||||
self.logger.info("⏩ 多 worker 模式：自动配置延迟到各 worker 进程内执行")
|
||||
else:
|
||||
apply_auto_configuration(self)
|
||||
|
||||
# 获取真实 IP 用于日志显示（服务器仍然使用配置的 host 绑定）
|
||||
display_host = get_local_ip() if host == "0.0.0.0" else host
|
||||
|
||||
# 显示服务器信息
|
||||
self.logger.info(f"🌐 服务器启动: http://{display_host}:{port}")
|
||||
self.logger.info(f"📚 API 文档: http://{display_host}:{port}/docs")
|
||||
self.logger.info(f"🔍 健康检查: http://{display_host}:{port}/health")
|
||||
self.logger.info(f"⚙️ 服务器类型: Hypercorn")
|
||||
self.logger.info(f"🔧 工作进程: {workers}")
|
||||
|
||||
if workers > 1 and not app_path:
|
||||
self.logger.warning(
|
||||
"⚠️ 多 workers 模式需要提供 app_path 参数，"
|
||||
"例如: app.run(workers=4, app_path='main:app.get_fastapi_app()')"
|
||||
)
|
||||
|
||||
# 启动服务器
|
||||
try:
|
||||
self.server_manager.start_server(
|
||||
app=self._fastapi_app,
|
||||
host=host,
|
||||
port=port,
|
||||
reload=reload,
|
||||
workers=workers,
|
||||
app_path=app_path,
|
||||
**run_kwargs
|
||||
)
|
||||
except KeyboardInterrupt:
|
||||
self.logger.info("收到中断信号，正在关闭...")
|
||||
finally:
|
||||
asyncio.run(self.shutdown())
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
"""优雅关闭应用程序"""
|
||||
if self._server:
|
||||
# 服务器关闭逻辑
|
||||
pass
|
||||
|
||||
# 停止调度器
|
||||
if self.scheduler.is_running():
|
||||
self.scheduler.stop()
|
||||
|
||||
self.logger.info("应用程序已关闭")
|
||||
|
||||
def add_route(self, path: str, handler: Callable, methods: List[str] = None, **kwargs) -> None:
|
||||
"""添加路由到 FastAPI 应用"""
|
||||
if self._fastapi_app is None:
|
||||
self._fastapi_app = self._create_fastapi_app()
|
||||
|
||||
if methods is None:
|
||||
methods = ['GET']
|
||||
|
||||
# 使用 FastAPI 的 add_api_route 方法
|
||||
self._fastapi_app.add_api_route(path, handler, methods=methods, **kwargs)
|
||||
|
||||
def get_fastapi_app(self) -> FastAPI:
|
||||
"""获取 FastAPI 应用实例"""
|
||||
if self._fastapi_app is None:
|
||||
self._fastapi_app = self._create_fastapi_app()
|
||||
return self._fastapi_app
|
||||
|
||||
def bootstrap_worker(self) -> FastAPI:
|
||||
"""Worker 进程内引导（多 worker 模式下由 server._worker_serve 调用）
|
||||
|
||||
spawn（Windows）与 fork（Linux/macOS）两种模式在此收敛：
|
||||
|
||||
1. 重读 MYBOOT_WORKER_ID 等环境变量——它们在 _worker_serve 中设置，
|
||||
晚于 Application 构造（spawn 子进程在 multiprocessing 引导阶段
|
||||
重新 import 用户主模块时即构造 Application），因此 __init__ 读到
|
||||
的是默认值，必须在这里刷新。
|
||||
2. 重算 scheduler 门控并重建 Scheduler——修复「每个 worker 都自认
|
||||
primary、调度器多跑」的问题；同时丢弃 fork 继承的、绑定到
|
||||
fork 前实例的任务注册。
|
||||
3. 防御性重置——若检测到 fork 继承的预引导状态（父进程曾手动执行
|
||||
apply_auto_configuration），清空 DI 容器与各注册表并重建
|
||||
FastAPI 应用，丢弃绑定到 fork 前控制器实例的路由。
|
||||
4. 在 worker 进程内执行自动配置——所有 client/service/component/
|
||||
controller 在本 worker 内实例化（issue #11 核心修复），同时修复
|
||||
Windows spawn 模式下 worker 无用户路由的问题。
|
||||
|
||||
Returns:
|
||||
本 worker 进程内完成引导的 FastAPI 应用实例
|
||||
"""
|
||||
# 1. 重读 worker 环境变量
|
||||
self._worker_id = int(os.environ.get("MYBOOT_WORKER_ID", "1"))
|
||||
self._worker_count = int(os.environ.get("MYBOOT_WORKER_COUNT", "1"))
|
||||
self._is_primary_worker = os.environ.get("MYBOOT_IS_PRIMARY_WORKER", "1") == "1"
|
||||
|
||||
# 2. 重算调度器门控并重建调度器（默认仅 primary worker 启用，
|
||||
# 可通过 scheduler.on_all_workers 配置覆盖；任务级 all_workers=True
|
||||
# 的任务在本 worker 注册时由注册门控按需调用 scheduler.enable()）
|
||||
scheduler_on_all_workers = self.config.get("scheduler.on_all_workers", False)
|
||||
scheduler_globally_enabled = self.config.get("scheduler.enabled", True)
|
||||
self._scheduler_enabled = (
|
||||
self._is_primary_worker or scheduler_on_all_workers
|
||||
) and scheduler_globally_enabled
|
||||
self.scheduler = Scheduler(config=self.config, enabled=self._scheduler_enabled)
|
||||
|
||||
# 3. 防御性重置：fork 子进程若继承了父进程已引导的状态
|
||||
# （正常延迟引导路径下这些注册表都是空的，不会进入此分支）
|
||||
di_container = getattr(self, 'di_container', None)
|
||||
inherited = bool(
|
||||
(di_container is not None and di_container.service_providers)
|
||||
or self.services or self.clients or self.components
|
||||
)
|
||||
if inherited:
|
||||
self.logger.warning(
|
||||
"检测到 fork 继承的预引导状态，重置注册表后在 worker 内重建实例..."
|
||||
)
|
||||
if di_container is not None:
|
||||
di_container.clear()
|
||||
self.services.clear()
|
||||
self.clients.clear()
|
||||
self.components.clear()
|
||||
if hasattr(self, '_client_type_map'):
|
||||
self._client_type_map.clear()
|
||||
if hasattr(self, '_component_type_map'):
|
||||
self._component_type_map.clear()
|
||||
# 注意：worker 钩子由 _auto_register_worker_hooks 重新注册，
|
||||
# 先清空避免重复（此分支下父进程已经 apply 过一次）
|
||||
self.worker_start_hooks.clear()
|
||||
self.worker_stop_hooks.clear()
|
||||
# 重建 FastAPI 应用，丢弃绑定到 fork 前控制器实例的路由
|
||||
self._fastapi_app = self._create_fastapi_app()
|
||||
|
||||
# 4. 在 worker 进程内执行自动配置
|
||||
# - spawn 子进程：全局管理器是全新的，auto_discover 执行扫描（命中父进程缓存）
|
||||
# - fork 子进程：管理器已发现完成，auto_discover 幂等守卫直接返回
|
||||
if self.auto_configuration_enabled:
|
||||
self.logger.info(f"🔍 Worker-{self._worker_id} 开始进程内引导...")
|
||||
auto_discover(self.auto_discover_package)
|
||||
apply_auto_configuration(self)
|
||||
|
||||
return self.get_fastapi_app()
|
||||
|
||||
# ==================== Worker 信息 ====================
|
||||
|
||||
@property
|
||||
def worker_id(self) -> int:
|
||||
"""
|
||||
获取当前 Worker ID（从 1 开始）
|
||||
|
||||
单进程模式下返回 1
|
||||
"""
|
||||
return self._worker_id
|
||||
|
||||
@property
|
||||
def worker_count(self) -> int:
|
||||
"""
|
||||
获取 Worker 总数
|
||||
|
||||
单进程模式下返回 1
|
||||
"""
|
||||
return self._worker_count
|
||||
|
||||
@property
|
||||
def is_primary_worker(self) -> bool:
|
||||
"""
|
||||
是否为主 Worker（Worker ID = 1）
|
||||
|
||||
适用于只需要在一个 worker 执行的任务（如定时任务、初始化任务）
|
||||
|
||||
Example:
|
||||
if app.is_primary_worker:
|
||||
# 只在主 worker 执行
|
||||
await init_cache()
|
||||
"""
|
||||
return self._is_primary_worker
|
||||
|
||||
@property
|
||||
def is_multi_worker_mode(self) -> bool:
|
||||
"""是否运行在多 Worker 模式"""
|
||||
return self._worker_count > 1
|
||||
|
||||
|
||||
# 便捷函数
|
||||
def create_app(
|
||||
name: str = "MyBoot App",
|
||||
config_file: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Application:
|
||||
"""创建 MyBoot 应用程序实例"""
|
||||
return Application(name, config_file, **kwargs)
|
||||
|
||||
|
||||
def get_service(name: str):
|
||||
return _current_app.get_service(name)
|
||||
|
||||
|
||||
def get_client(name: str):
|
||||
return _current_app.get_client(name)
|
||||
|
||||
|
||||
def get_container() -> Container:
|
||||
"""获取容器实例"""
|
||||
if _current_app is None:
|
||||
raise RuntimeError("应用实例未初始化，请确保应用已创建并启动")
|
||||
return _current_app.container
|
||||
@@ -0,0 +1,962 @@
|
||||
"""
|
||||
自动配置模块
|
||||
|
||||
实现约定优于配置的设计理念，提供自动发现和配置功能
|
||||
|
||||
设计参考 Spring Boot：
|
||||
- 使用 AST 静态分析替代 import（发现阶段不执行代码）
|
||||
- 延迟导入：只在注册阶段才导入需要的模块
|
||||
- 缓存 AST 分析结果，缓存命中时完全跳过分析
|
||||
"""
|
||||
|
||||
import os
|
||||
import time
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Type, Any
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
from ._internal import ast_analyzer, component_scanner, scan_cache
|
||||
from ._internal.scan_cache import _CACHE_VERSION
|
||||
|
||||
logger = loguru_logger.bind(name=__name__)
|
||||
|
||||
|
||||
class AutoConfigurationError(Exception):
|
||||
"""自动配置失败异常
|
||||
|
||||
当自动注册组件失败时抛出此异常，导致应用启动失败
|
||||
"""
|
||||
pass
|
||||
|
||||
# MyBoot 装饰器到组件类型的映射
|
||||
# 注意：@cron/@interval/@once 装饰器只能在 @component 类中使用
|
||||
# job 方法的注册在 _auto_register_components 中动态进行
|
||||
_DECORATOR_MAPPING = {
|
||||
'service': 'services',
|
||||
'client': 'clients',
|
||||
'model': 'models',
|
||||
'component': 'components',
|
||||
'rest_controller': 'rest_controllers',
|
||||
'route': 'routes',
|
||||
'get': 'routes',
|
||||
'post': 'routes',
|
||||
'put': 'routes',
|
||||
'delete': 'routes',
|
||||
'patch': 'routes',
|
||||
'middleware': 'middleware',
|
||||
'on_worker_start': 'worker_hooks',
|
||||
'on_worker_stop': 'worker_hooks',
|
||||
}
|
||||
|
||||
|
||||
from ..utils.naming import camel_to_snake as _camel_to_snake
|
||||
|
||||
|
||||
def _find_project_root() -> str:
|
||||
"""查找项目根目录"""
|
||||
# 从当前文件所在目录开始向上查找
|
||||
current_dir = Path(__file__).parent.absolute()
|
||||
|
||||
# 向上查找，直到找到包含 pyproject.toml 或 requirements.txt 的目录
|
||||
while current_dir.parent != current_dir:
|
||||
if (current_dir / 'pyproject.toml').exists() or (current_dir / 'requirements.txt').exists():
|
||||
return str(current_dir)
|
||||
current_dir = current_dir.parent
|
||||
|
||||
# 如果没找到，返回当前工作目录
|
||||
return os.getcwd()
|
||||
|
||||
|
||||
class AutoConfigurationManager:
|
||||
"""
|
||||
自动配置管理器
|
||||
|
||||
设计参考 Spring Boot：
|
||||
- 发现阶段：使用 AST 静态分析，不执行 import
|
||||
- 注册阶段：延迟导入，只导入需要的模块
|
||||
- 缓存：缓存 AST 分析结果，缓存命中时完全跳过分析
|
||||
"""
|
||||
|
||||
def __init__(self, app_root: str = None, use_cache: bool = True, parallel_import: bool = False):
|
||||
self.app_root = app_root or _find_project_root()
|
||||
self.use_cache = use_cache
|
||||
self.parallel_import = parallel_import
|
||||
# 组件元数据（不包含实际类对象，只有模块路径和名称）
|
||||
self._component_metadata: Dict[str, List[dict]] = {
|
||||
'routes': [],
|
||||
'middleware': [],
|
||||
'services': [],
|
||||
'models': [],
|
||||
'clients': [],
|
||||
'components': [],
|
||||
'rest_controllers': [],
|
||||
'worker_hooks': []
|
||||
}
|
||||
# 已加载的组件（包含实际类对象，延迟填充）
|
||||
self.discovered_components: Dict[str, List[dict]] = {
|
||||
'routes': [],
|
||||
'middleware': [],
|
||||
'services': [],
|
||||
'models': [],
|
||||
'clients': [],
|
||||
'components': [],
|
||||
'rest_controllers': [],
|
||||
'worker_hooks': []
|
||||
}
|
||||
self.auto_configured = False
|
||||
self._modules_loaded = False
|
||||
|
||||
def _get_cache_path(self, package_name: str) -> Path:
|
||||
"""获取缓存文件路径"""
|
||||
return scan_cache.get_cache_path(self.app_root, package_name)
|
||||
|
||||
def _collect_source_files(self, package_path: Path) -> Dict[str, float]:
|
||||
"""收集所有源文件及其修改时间"""
|
||||
return scan_cache.collect_source_files(package_path)
|
||||
|
||||
def _is_cache_valid(self, cache_path: Path, package_path: Path) -> bool:
|
||||
"""检查缓存是否有效"""
|
||||
return scan_cache.is_cache_valid(cache_path, package_path)
|
||||
|
||||
def _save_cache(self, cache_path: Path, package_path: Path) -> None:
|
||||
"""保存元数据缓存（不包含类对象）"""
|
||||
scan_cache.save_cache(cache_path, package_path, self._component_metadata)
|
||||
|
||||
def _load_cache(self, cache_path: Path) -> bool:
|
||||
"""从缓存加载元数据（不导入模块）"""
|
||||
loaded = scan_cache.load_cache(cache_path, self._component_metadata.keys())
|
||||
if loaded is None:
|
||||
return False
|
||||
self._component_metadata = loaded
|
||||
return True
|
||||
|
||||
def _parse_decorators(self, node) -> List[str]:
|
||||
"""解析装饰器名称"""
|
||||
return ast_analyzer.parse_decorators(node)
|
||||
|
||||
def _scan_file_ast(self, file_path: Path, module_name: str) -> None:
|
||||
"""使用 AST 静态分析扫描单个文件（不执行 import）"""
|
||||
ast_analyzer.scan_file_ast(
|
||||
file_path, module_name, self._component_metadata, _DECORATOR_MAPPING
|
||||
)
|
||||
|
||||
def _scan_package_ast(self, package_path: Path) -> None:
|
||||
"""使用 AST 递归扫描包（不执行 import）"""
|
||||
ast_analyzer.scan_package_ast(
|
||||
package_path, self._component_metadata, _DECORATOR_MAPPING
|
||||
)
|
||||
|
||||
def _load_modules(self) -> None:
|
||||
"""延迟加载模块，将元数据转换为实际的类对象"""
|
||||
if self._modules_loaded:
|
||||
return
|
||||
|
||||
self.discovered_components = component_scanner.build_discovered_components(
|
||||
self._component_metadata, self.parallel_import
|
||||
)
|
||||
|
||||
self._modules_loaded = True
|
||||
|
||||
def _resolve_scan_target(self, package_name: str):
|
||||
"""解析扫描目标：单模块文件或包目录
|
||||
|
||||
Returns:
|
||||
(scan_path, scan_type, module_name)
|
||||
scan_type 为 'file' 或 'package'；目录扫描时 module_name 为 None
|
||||
"""
|
||||
app_root = Path(self.app_root)
|
||||
|
||||
if package_name.endswith('.py'):
|
||||
file_path = app_root / package_name.replace('/', os.sep)
|
||||
if file_path.is_file():
|
||||
module_name = str(
|
||||
file_path.relative_to(app_root).with_suffix('')
|
||||
).replace(os.sep, '.')
|
||||
return file_path, 'file', module_name
|
||||
|
||||
dotted_path = app_root / package_name.replace('.', os.sep)
|
||||
py_file = dotted_path.with_suffix('.py')
|
||||
if py_file.is_file():
|
||||
return py_file, 'file', package_name
|
||||
|
||||
dir_path = dotted_path if dotted_path.is_dir() else app_root / package_name
|
||||
if dir_path.is_dir():
|
||||
return dir_path, 'package', None
|
||||
|
||||
return None, None, None
|
||||
|
||||
def auto_discover(self, package_name: str = "app") -> None:
|
||||
"""
|
||||
自动发现应用组件（AST 静态分析，不执行 import）
|
||||
|
||||
模块的实际导入延迟到 apply_auto_configuration 时进行。
|
||||
package_name 支持目录包（如 app、examples/convention）或单模块
|
||||
（如 examples.convention_app、examples/convention_app.py）。
|
||||
"""
|
||||
# 幂等守卫：fork 子进程（bootstrap_worker）重复调用时直接返回，
|
||||
# 避免向 _component_metadata 追加重复条目
|
||||
if self.auto_configured:
|
||||
logger.debug("自动发现已完成，跳过重复扫描")
|
||||
return
|
||||
|
||||
start_time = time.perf_counter()
|
||||
logger.info(f"开始自动发现 {package_name} 包中的组件...")
|
||||
|
||||
try:
|
||||
scan_path, scan_type, module_name = self._resolve_scan_target(package_name)
|
||||
if scan_path is None:
|
||||
logger.warning(f"扫描目标不存在: {package_name}")
|
||||
return
|
||||
|
||||
cache_path = self._get_cache_path(package_name)
|
||||
|
||||
# 尝试使用缓存（只读取元数据，不导入模块）
|
||||
if self.use_cache and self._is_cache_valid(cache_path, scan_path):
|
||||
if self._load_cache(cache_path):
|
||||
self.auto_configured = True
|
||||
elapsed = (time.perf_counter() - start_time) * 1000
|
||||
logger.info(f"自动发现完成（从缓存加载），耗时: {elapsed:.2f}ms")
|
||||
return
|
||||
|
||||
# AST 静态分析扫描（不执行 import）
|
||||
if scan_type == 'file':
|
||||
self._scan_file_ast(scan_path, module_name)
|
||||
else:
|
||||
self._scan_package_ast(scan_path)
|
||||
|
||||
# 保存缓存
|
||||
if self.use_cache:
|
||||
self._save_cache(cache_path, scan_path)
|
||||
|
||||
self.auto_configured = True
|
||||
elapsed = (time.perf_counter() - start_time) * 1000
|
||||
logger.info(f"自动发现完成（AST 扫描），耗时: {elapsed:.2f}ms")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"自动发现失败: {e}", exc_info=True)
|
||||
|
||||
def apply_auto_configuration(self, app) -> None:
|
||||
"""应用自动配置（此时才执行模块导入）"""
|
||||
if not self.auto_configured:
|
||||
logger.warning("自动发现未完成，跳过自动配置")
|
||||
return
|
||||
|
||||
start_time = time.perf_counter()
|
||||
|
||||
# 延迟加载模块（将元数据转换为实际类对象）
|
||||
load_start = time.perf_counter()
|
||||
self._load_modules()
|
||||
load_elapsed = (time.perf_counter() - load_start) * 1000
|
||||
logger.info(f"模块加载完成，耗时: {load_elapsed:.2f}ms")
|
||||
|
||||
self._auto_register_routes(app)
|
||||
self._auto_register_models(app)
|
||||
self._auto_register_clients(app)
|
||||
self._auto_register_services(app) # 先注册服务，支持依赖注入
|
||||
self._auto_register_components(app) # 注册组件并注册其中的 job 方法
|
||||
self._auto_register_middleware(app)
|
||||
self._auto_register_rest_controllers(app)
|
||||
# 最后注册 worker 钩子（此时容器/服务已就绪，钩子内可解析依赖）
|
||||
self._auto_register_worker_hooks(app)
|
||||
|
||||
elapsed = (time.perf_counter() - start_time) * 1000
|
||||
logger.info(f"自动配置应用完成，耗时: {elapsed:.2f}ms")
|
||||
|
||||
def _auto_register_rest_controllers(self, app) -> None:
|
||||
"""自动注册 REST 控制器
|
||||
|
||||
只注册显式使用 @get、@post 等装饰器的方法，不自动根据方法名生成路由
|
||||
"""
|
||||
import inspect as inspect_module
|
||||
|
||||
for controller_info in self.discovered_components['rest_controllers']:
|
||||
try:
|
||||
cls = controller_info['class']
|
||||
controller_config = getattr(cls, '__myboot_rest_controller__')
|
||||
base_path = controller_config['base_path']
|
||||
base_kwargs = controller_config.get('kwargs', {})
|
||||
|
||||
# 创建控制器实例（支持依赖注入）
|
||||
instance = self._get_class_instance(cls, app)
|
||||
|
||||
# 检查类中的所有方法，只处理显式使用路由装饰器的方法
|
||||
for method_name, method in inspect_module.getmembers(
|
||||
instance,
|
||||
predicate=lambda x: inspect_module.ismethod(x) and not x.__name__.startswith('_')
|
||||
):
|
||||
# 只处理有 __myboot_route__ 属性的方法（即显式使用 @get、@post 等装饰器的方法）
|
||||
if hasattr(method, '__myboot_route__'):
|
||||
route_config = getattr(method, '__myboot_route__')
|
||||
method_path = route_config['path']
|
||||
methods = route_config.get('methods', ['GET'])
|
||||
route_kwargs = {**base_kwargs, **route_config.get('kwargs', {})}
|
||||
route_kwargs.setdefault(
|
||||
'operation_id',
|
||||
f"{controller_info['module'].replace('.', '_')}_"
|
||||
f"{cls.__name__}_{method_name}"
|
||||
)
|
||||
|
||||
# 合并基础路径和方法路径
|
||||
# 如果方法路径是绝对路径（以 // 开头），则直接使用（去掉一个 /）
|
||||
# 否则，将方法路径追加到基础路径
|
||||
if method_path.startswith('//'):
|
||||
# 绝对路径，去掉一个 / 后使用
|
||||
full_path = method_path[1:]
|
||||
elif method_path.startswith('/'):
|
||||
# 以 / 开头但非绝对路径，去掉开头的 / 后追加到基础路径
|
||||
full_path = f"{base_path}{method_path}".replace('//', '/')
|
||||
else:
|
||||
# 相对路径，追加到基础路径
|
||||
full_path = f"{base_path}/{method_path}".replace('//', '/')
|
||||
|
||||
# 注册路由
|
||||
app.add_route(
|
||||
path=full_path,
|
||||
handler=method,
|
||||
methods=methods,
|
||||
**route_kwargs
|
||||
)
|
||||
|
||||
logger.debug(f"自动注册 REST 路由: {methods} {full_path} -> {controller_info['module']}.{cls.__name__}.{method_name}")
|
||||
|
||||
logger.info(f"自动注册 REST 控制器: {controller_info['module']}.{cls.__name__}")
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册 REST 控制器失败 {controller_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册 REST 控制器失败 '{controller_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
def _auto_register_routes(self, app) -> None:
|
||||
"""自动注册路由"""
|
||||
for route_info in self.discovered_components['routes']:
|
||||
try:
|
||||
# 前缀匹配：AST 扫描产生的类型为 function_get/function_post/
|
||||
# function_route 等（issue #8 修复同款逻辑，精确匹配会漏掉
|
||||
# 模块级 @get/@post 函数路由）
|
||||
if route_info['type'].startswith('function_'):
|
||||
# 函数路由
|
||||
func = route_info['function']
|
||||
route_config = getattr(func, '__myboot_route__')
|
||||
app.add_route(
|
||||
path=route_config['path'],
|
||||
handler=func,
|
||||
methods=route_config.get('methods', ['GET']),
|
||||
**route_config.get('kwargs', {})
|
||||
)
|
||||
logger.debug(f"自动注册路由: {route_info['module']}.{func.__name__}")
|
||||
elif route_info['type'].startswith('class_'):
|
||||
# 类路由（class 条目没有 'function' 键，日志只在各方法处打印）
|
||||
cls = route_info['class']
|
||||
route_config = getattr(cls, '__myboot_route__')
|
||||
instance = cls()
|
||||
for method_name, method in inspect.getmembers(instance, predicate=inspect.ismethod):
|
||||
if hasattr(method, '__myboot_route__'):
|
||||
method_config = getattr(method, '__myboot_route__')
|
||||
app.add_route(
|
||||
path=method_config['path'],
|
||||
handler=method,
|
||||
methods=method_config.get('methods', ['GET']),
|
||||
**method_config.get('kwargs', {})
|
||||
)
|
||||
logger.debug(f"自动注册路由: {route_info['module']}.{cls.__name__}.{method_name}")
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册路由失败 {route_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册路由失败 '{route_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
def _is_job_enabled(self, _func, job_config: dict) -> bool:
|
||||
"""
|
||||
检查任务是否启用
|
||||
|
||||
Args:
|
||||
_func: 任务函数（保留用于未来扩展，如从函数名读取配置）
|
||||
job_config: 任务配置
|
||||
|
||||
Returns:
|
||||
bool: 任务是否启用
|
||||
"""
|
||||
# 检查装饰器中直接指定的 enabled
|
||||
enabled = job_config.get('enabled')
|
||||
if enabled is not None:
|
||||
# 支持布尔值和字符串
|
||||
if isinstance(enabled, bool):
|
||||
return enabled
|
||||
if isinstance(enabled, str):
|
||||
return enabled.lower() in ('true', '1', 'yes', 'on', 'enabled')
|
||||
return bool(enabled)
|
||||
|
||||
# 默认启用
|
||||
return True
|
||||
|
||||
def _should_register_job_on_this_worker(self, app, job_config: dict) -> bool:
|
||||
"""
|
||||
判断任务是否应在当前 worker 进程注册（任务级 all_workers 门控）
|
||||
|
||||
规则：
|
||||
- scheduler.enabled=false：全局禁用，任何 worker 都不注册
|
||||
- primary worker：注册全部任务（现状行为）
|
||||
- 非 primary worker：
|
||||
- scheduler.on_all_workers=true（全局配置）：注册全部任务
|
||||
- 任务声明 all_workers=True：注册该任务
|
||||
- 其他：跳过
|
||||
"""
|
||||
config = getattr(app, 'config', None)
|
||||
if config is not None and not config.get('scheduler.enabled', True):
|
||||
return False
|
||||
if getattr(app, 'is_primary_worker', True):
|
||||
return True
|
||||
if config is not None and config.get('scheduler.on_all_workers', False):
|
||||
return True
|
||||
return bool(job_config.get('all_workers', False))
|
||||
|
||||
def _get_class_instance(self, cls: Type, app) -> Any:
|
||||
"""
|
||||
获取类实例，支持依赖注入
|
||||
|
||||
从 di_container 获取 service 依赖，从 app.components、app.clients 获取依赖
|
||||
找不到直接报错，不尝试初始化
|
||||
|
||||
Args:
|
||||
cls: 类
|
||||
app: 应用实例
|
||||
|
||||
Returns:
|
||||
类实例
|
||||
|
||||
Raises:
|
||||
RuntimeError: 如果 di_container 不存在或依赖注入失败
|
||||
KeyError: 如果必需依赖未注册
|
||||
"""
|
||||
# 检查 di_container 是否存在
|
||||
if not hasattr(app, 'di_container'):
|
||||
raise RuntimeError(f"无法实例化 {cls.__name__}：应用未配置依赖注入容器")
|
||||
|
||||
di_container = app.di_container
|
||||
|
||||
# 检查类是否有 @service 装饰器
|
||||
if hasattr(cls, '__myboot_service__'):
|
||||
service_config = getattr(cls, '__myboot_service__')
|
||||
service_name = service_config.get('name', _camel_to_snake(cls.__name__))
|
||||
if di_container.has_service(service_name):
|
||||
return di_container.get_service(service_name)
|
||||
raise KeyError(f"服务 '{service_name}' 未在 DI 容器中注册")
|
||||
|
||||
# 检查类是否有 @client 装饰器
|
||||
if hasattr(cls, '__myboot_client__'):
|
||||
client_config = getattr(cls, '__myboot_client__')
|
||||
client_name = client_config.get('name', _camel_to_snake(cls.__name__))
|
||||
if hasattr(app, 'clients') and client_name in app.clients:
|
||||
return app.clients[client_name]
|
||||
raise KeyError(f"客户端 '{client_name}' 未注册")
|
||||
|
||||
# 检查类是否有 @component 装饰器且已注册
|
||||
if hasattr(cls, '__myboot_component__'):
|
||||
component_config = getattr(cls, '__myboot_component__')
|
||||
component_name = component_config.get('name', _camel_to_snake(cls.__name__))
|
||||
if hasattr(app, 'components') and component_name in app.components:
|
||||
return app.components[component_name]
|
||||
# 组件尚未注册，继续创建新实例
|
||||
|
||||
# 检查构造函数是否有依赖
|
||||
from myboot.core.di.decorators import get_injectable_params
|
||||
params = get_injectable_params(cls.__init__)
|
||||
|
||||
if not params:
|
||||
# 没有依赖参数，直接实例化
|
||||
return cls()
|
||||
|
||||
# 有依赖参数，从 DI 容器和 clients 获取依赖
|
||||
dependencies = {}
|
||||
for param_name, param_info in params.items():
|
||||
dependency_name = param_info.get('service_name')
|
||||
if not dependency_name:
|
||||
continue
|
||||
|
||||
is_optional = param_info.get('is_optional', False)
|
||||
dependency_instance = None
|
||||
found = False
|
||||
|
||||
# 优先从 DI 容器获取 service
|
||||
if di_container.has_service(dependency_name):
|
||||
try:
|
||||
dependency_instance = di_container.get_service(dependency_name)
|
||||
logger.debug(f"从 DI 容器注入 service 依赖: {param_name} = {dependency_name}")
|
||||
found = True
|
||||
except Exception as e:
|
||||
if not is_optional:
|
||||
raise RuntimeError(
|
||||
f"无法实例化 {cls.__name__}："
|
||||
f"获取 service 依赖 '{dependency_name}' (参数 '{param_name}') 失败: {e}"
|
||||
) from e
|
||||
logger.debug(f"获取可选 service 依赖 '{dependency_name}' 失败: {e}")
|
||||
|
||||
# 如果没找到 service，尝试从 components 获取
|
||||
if not found and hasattr(app, 'components'):
|
||||
# 先按名称查找
|
||||
if dependency_name in app.components:
|
||||
dependency_instance = app.components[dependency_name]
|
||||
logger.debug(f"从 components 注入依赖: {param_name} = {dependency_name}")
|
||||
found = True
|
||||
# 再尝试按类型查找
|
||||
elif hasattr(app, '_component_type_map'):
|
||||
param_type = param_info.get('type')
|
||||
if param_type and param_type in app._component_type_map:
|
||||
actual_name = app._component_type_map[param_type]
|
||||
dependency_instance = app.components[actual_name]
|
||||
found = True
|
||||
|
||||
# 如果没找到 component，尝试从 clients 获取
|
||||
if not found and hasattr(app, 'clients'):
|
||||
# 先按名称查找
|
||||
if dependency_name in app.clients:
|
||||
dependency_instance = app.clients[dependency_name]
|
||||
logger.debug(f"从 clients 注入依赖: {param_name} = {dependency_name}")
|
||||
found = True
|
||||
# 再尝试按类型查找
|
||||
elif hasattr(app, '_client_type_map'):
|
||||
param_type = param_info.get('type')
|
||||
if param_type and param_type in app._client_type_map:
|
||||
actual_name = app._client_type_map[param_type]
|
||||
dependency_instance = app.clients[actual_name]
|
||||
found = True
|
||||
|
||||
# 如果都没找到且不是可选的，报错
|
||||
if not found and not is_optional:
|
||||
raise KeyError(
|
||||
f"无法实例化 {cls.__name__}："
|
||||
f"必需依赖 '{dependency_name}' (参数 '{param_name}') 未在 DI 容器、components 或 clients 中注册"
|
||||
)
|
||||
|
||||
if found and dependency_instance is not None:
|
||||
dependencies[param_name] = dependency_instance
|
||||
|
||||
# 使用依赖实例化
|
||||
logger.debug(f"使用依赖注入实例化 {cls.__name__}: {list(dependencies.keys())}")
|
||||
return cls(**dependencies)
|
||||
|
||||
def _auto_register_middleware(self, app) -> None:
|
||||
"""自动注册中间件"""
|
||||
from myboot.web.middleware import FunctionMiddleware
|
||||
|
||||
if not self.discovered_components['middleware']:
|
||||
return
|
||||
|
||||
# 按照 order 排序中间件
|
||||
middleware_list = []
|
||||
for middleware_info in self.discovered_components['middleware']:
|
||||
try:
|
||||
func = middleware_info['function']
|
||||
middleware_config = getattr(func, '__myboot_middleware__')
|
||||
order = middleware_config.get('order', 0)
|
||||
middleware_list.append({
|
||||
'func': func,
|
||||
'config': middleware_config,
|
||||
'order': order,
|
||||
'module': middleware_info['module']
|
||||
})
|
||||
except Exception as e:
|
||||
logger.error(f"解析中间件配置失败 {middleware_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"解析中间件配置失败 '{middleware_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
# 按 order 排序
|
||||
middleware_list.sort(key=lambda x: x['order'])
|
||||
|
||||
# 注册中间件（FastAPI 的中间件是后进先出的，所以需要反向注册）
|
||||
for middleware_item in reversed(middleware_list):
|
||||
try:
|
||||
func = middleware_item['func']
|
||||
config = middleware_item['config']
|
||||
module = middleware_item['module']
|
||||
|
||||
# 创建动态中间件类，包装 FunctionMiddleware
|
||||
middleware_name = config.get('name', func.__name__)
|
||||
|
||||
# 使用闭包捕获变量
|
||||
def make_init(middleware_func, middleware_config):
|
||||
def __init__(self, app):
|
||||
FunctionMiddleware.__init__(
|
||||
self,
|
||||
app=app,
|
||||
middleware_func=middleware_func,
|
||||
path_filter=middleware_config.get('path_filter'),
|
||||
methods=middleware_config.get('methods'),
|
||||
condition=middleware_config.get('condition'),
|
||||
order=middleware_config.get('order', 0),
|
||||
**middleware_config.get('kwargs', {})
|
||||
)
|
||||
return __init__
|
||||
|
||||
# 动态创建中间件类
|
||||
middleware_class = type(
|
||||
f"Middleware_{middleware_name}",
|
||||
(FunctionMiddleware,),
|
||||
{'__init__': make_init(func, config)}
|
||||
)
|
||||
|
||||
# 添加到 FastAPI 应用
|
||||
app._fastapi_app.add_middleware(middleware_class)
|
||||
|
||||
logger.info(
|
||||
f"自动注册中间件: '{middleware_name}' "
|
||||
f"(order={config.get('order', 0)}, "
|
||||
f"module={module})"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册中间件失败 {middleware_item['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册中间件失败 '{middleware_item['module']}': {e}"
|
||||
) from e
|
||||
|
||||
logger.info(f"成功注册 {len(middleware_list)} 个中间件")
|
||||
|
||||
def _auto_register_services(self, app) -> None:
|
||||
"""自动注册服务（支持依赖注入）"""
|
||||
try:
|
||||
from myboot.core.di import DependencyContainer
|
||||
|
||||
# 检查应用是否有依赖注入容器
|
||||
if not hasattr(app, 'di_container'):
|
||||
app.di_container = DependencyContainer()
|
||||
|
||||
di_container = app.di_container
|
||||
|
||||
# 第一步：注册所有服务到容器（不创建实例）
|
||||
for service_info in self.discovered_components['services']:
|
||||
try:
|
||||
cls = service_info['class']
|
||||
service_config = getattr(cls, '__myboot_service__')
|
||||
service_name = service_config.get('name', cls.__name__.lower())
|
||||
|
||||
# 获取服务作用域（默认单例）
|
||||
scope = service_config.get('scope', 'singleton')
|
||||
|
||||
# 注册到依赖注入容器
|
||||
di_container.register_service(
|
||||
service_class=cls,
|
||||
service_name=service_name,
|
||||
scope=scope,
|
||||
config=service_config
|
||||
)
|
||||
|
||||
logger.debug(f"已注册服务到容器: '{service_name}' ({service_info['module']}.{cls.__name__})")
|
||||
except Exception as e:
|
||||
logger.error(f"注册服务到容器失败 {service_info['module']}: {e}", exc_info=True)
|
||||
|
||||
# 第二步：构建依赖注入容器
|
||||
try:
|
||||
di_container.build_container()
|
||||
logger.info("依赖注入容器构建成功")
|
||||
except Exception as e:
|
||||
logger.error(f"构建依赖注入容器失败: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(f"构建依赖注入容器失败: {e}") from e
|
||||
|
||||
# 第三步：获取所有服务实例并注册到应用上下文
|
||||
for service_info in self.discovered_components['services']:
|
||||
try:
|
||||
cls = service_info['class']
|
||||
service_config = getattr(cls, '__myboot_service__')
|
||||
service_name = service_config.get('name', cls.__name__.lower())
|
||||
|
||||
# 非单例（request/factory）服务跳过预实例化：
|
||||
# 按需通过 app.di_container / Container.get 解析，
|
||||
# 不会出现在 app.services 字典中
|
||||
scope = service_config.get('scope', 'singleton')
|
||||
if scope != 'singleton':
|
||||
logger.info(
|
||||
f"自动注册服务（{scope} 作用域，按需解析）: '{service_name}' "
|
||||
f"({service_info['module']}.{cls.__name__})"
|
||||
)
|
||||
continue
|
||||
|
||||
# 从容器获取服务实例（自动注入依赖）
|
||||
instance = di_container.get_service(service_name)
|
||||
app.services[service_name] = instance
|
||||
|
||||
# 确保当前应用实例已注册
|
||||
from myboot.core.application import _current_app
|
||||
if _current_app != app:
|
||||
# 更新当前应用实例
|
||||
import myboot.core.application
|
||||
myboot.core.application._current_app = app
|
||||
|
||||
logger.info(f"自动注册服务（依赖注入）: '{service_name}' ({service_info['module']}.{cls.__name__})")
|
||||
except Exception as e:
|
||||
logger.error(f"获取服务实例失败 {service_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册服务失败 '{service_name}' ({service_info['module']}): {e}"
|
||||
) from e
|
||||
|
||||
except ImportError as e:
|
||||
logger.error(f"dependency_injector 未安装: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
"依赖注入需要 dependency_injector 库，请运行: pip install dependency-injector"
|
||||
) from e
|
||||
except AutoConfigurationError:
|
||||
# 重新抛出 AutoConfigurationError
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"依赖注入服务注册失败: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(f"依赖注入服务注册失败: {e}") from e
|
||||
|
||||
def _auto_register_models(self, app) -> None:
|
||||
"""自动注册模型"""
|
||||
for model_info in self.discovered_components['models']:
|
||||
try:
|
||||
cls = model_info['class']
|
||||
model_config = getattr(cls, '__myboot_model__')
|
||||
|
||||
# 注册模型到应用上下文
|
||||
model_name = model_config.get('name', cls.__name__.lower())
|
||||
app.models[model_name] = cls
|
||||
|
||||
logger.info(f"自动注册模型: {model_info['module']}")
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册模型失败 {model_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册模型失败 '{model_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
def _auto_register_clients(self, app) -> None:
|
||||
"""自动注册客户端"""
|
||||
# 初始化类型到名称的映射（用于按类型查找）
|
||||
if not hasattr(app, '_client_type_map'):
|
||||
app._client_type_map = {}
|
||||
|
||||
# 初始化 DI 容器（如果还没有）
|
||||
if not hasattr(app, 'di_container'):
|
||||
from myboot.core.di import DependencyContainer
|
||||
app.di_container = DependencyContainer()
|
||||
|
||||
for client_info in self.discovered_components['clients']:
|
||||
try:
|
||||
cls = client_info['class']
|
||||
client_config = getattr(cls, '__myboot_client__')
|
||||
# 优先使用用户自定义名称，否则使用 _camel_to_snake 自动生成
|
||||
client_name = client_config.get('name', _camel_to_snake(cls.__name__))
|
||||
auto_name = _camel_to_snake(cls.__name__)
|
||||
|
||||
# 非单例（request/factory）客户端：注册类到 DI 容器按需解析，
|
||||
# 不在此处创建实例，也不出现在 app.clients 字典中
|
||||
scope = client_config.get('scope', 'singleton')
|
||||
if scope != 'singleton':
|
||||
app.di_container.register_service(
|
||||
service_class=cls,
|
||||
service_name=client_name,
|
||||
scope=scope,
|
||||
config=client_config
|
||||
)
|
||||
if auto_name != client_name and not app.di_container.has_service(auto_name):
|
||||
app.di_container.register_service(
|
||||
service_class=cls,
|
||||
service_name=auto_name,
|
||||
scope=scope,
|
||||
config=client_config
|
||||
)
|
||||
logger.info(
|
||||
f"自动注册客户端（{scope} 作用域，按需解析）: '{client_name}' "
|
||||
f"({client_info['module']}.{cls.__name__})"
|
||||
)
|
||||
continue
|
||||
|
||||
# 创建客户端实例并注册到应用上下文
|
||||
instance = cls()
|
||||
app.clients[client_name] = instance
|
||||
|
||||
# 同时记录类型映射，用于按类型查找
|
||||
app._client_type_map[cls] = client_name
|
||||
# 也用自动转换的名称注册（如果不同），方便按类型名查找
|
||||
if auto_name != client_name and auto_name not in app.clients:
|
||||
app.clients[auto_name] = instance
|
||||
|
||||
# 将 client 实例注册到 DI 容器（作为已创建的单例）
|
||||
# 这样 Service 可以通过 DI 容器获取 Client 依赖
|
||||
app.di_container.register_instance(client_name, instance)
|
||||
if auto_name != client_name:
|
||||
app.di_container.register_instance(auto_name, instance)
|
||||
|
||||
logger.info(f"自动注册客户端: '{client_name}' ({client_info['module']}.{cls.__name__})")
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册客户端失败 {client_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册客户端失败 '{client_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
def _auto_register_components(self, app) -> None:
|
||||
"""自动注册组件（支持依赖注入）"""
|
||||
# 初始化类型到名称的映射（用于按类型查找）
|
||||
if not hasattr(app, '_component_type_map'):
|
||||
app._component_type_map = {}
|
||||
|
||||
for component_info in self.discovered_components['components']:
|
||||
try:
|
||||
cls = component_info['class']
|
||||
component_config = getattr(cls, '__myboot_component__')
|
||||
|
||||
# 获取组件配置
|
||||
component_name = component_config.get('name', _camel_to_snake(cls.__name__))
|
||||
lazy = component_config.get('lazy', False)
|
||||
# scope 配置用于未来支持 prototype 模式
|
||||
|
||||
# 懒加载的组件跳过立即实例化
|
||||
if lazy:
|
||||
# 记录组件信息，延迟创建
|
||||
app._lazy_components = getattr(app, '_lazy_components', {})
|
||||
app._lazy_components[component_name] = {
|
||||
'class': cls,
|
||||
'config': component_config,
|
||||
'module': component_info['module']
|
||||
}
|
||||
logger.debug(f"已注册懒加载组件: '{component_name}' ({component_info['module']}.{cls.__name__})")
|
||||
continue
|
||||
|
||||
# 创建组件实例（支持依赖注入）
|
||||
instance = self._get_class_instance(cls, app)
|
||||
|
||||
# 注册到组件注册表
|
||||
app.components[component_name] = instance
|
||||
|
||||
# 记录类型映射，用于按类型查找
|
||||
app._component_type_map[cls] = component_name
|
||||
|
||||
# 也用自动转换的名称注册（如果不同）
|
||||
auto_name = _camel_to_snake(cls.__name__)
|
||||
if auto_name != component_name and auto_name not in app.components:
|
||||
app.components[auto_name] = instance
|
||||
|
||||
# 将组件实例注册到 DI 容器（作为已创建的单例）
|
||||
# 这样其他组件可以通过 DI 容器获取依赖
|
||||
if hasattr(app, 'di_container'):
|
||||
app.di_container.register_instance(component_name, instance)
|
||||
if auto_name != component_name:
|
||||
app.di_container.register_instance(auto_name, instance)
|
||||
|
||||
# 注册组件内的 job 方法（@cron/@interval/@once）
|
||||
self._register_component_jobs(app, instance, cls, component_info['module'])
|
||||
|
||||
logger.info(f"自动注册组件: '{component_name}' ({component_info['module']}.{cls.__name__})")
|
||||
except Exception as e:
|
||||
logger.error(f"自动注册组件失败 {component_info['module']}: {e}", exc_info=True)
|
||||
raise AutoConfigurationError(
|
||||
f"自动注册组件失败 '{component_info['module']}': {e}"
|
||||
) from e
|
||||
|
||||
def _register_component_jobs(self, app, instance, cls: Type, module_name: str) -> None:
|
||||
"""
|
||||
注册组件内的 job 方法
|
||||
|
||||
扫描组件实例中使用 @cron/@interval/@once 装饰器的方法，并注册到调度器
|
||||
|
||||
Args:
|
||||
app: 应用实例
|
||||
instance: 组件实例
|
||||
cls: 组件类
|
||||
module_name: 模块名称
|
||||
"""
|
||||
import inspect as inspect_module
|
||||
|
||||
for method_name, method in inspect_module.getmembers(instance, predicate=inspect_module.ismethod):
|
||||
# 跳过私有方法
|
||||
if method_name.startswith('_'):
|
||||
continue
|
||||
|
||||
# 检查是否有 job 装饰器
|
||||
if not hasattr(method, '__myboot_job__'):
|
||||
continue
|
||||
|
||||
job_config = getattr(method, '__myboot_job__')
|
||||
|
||||
# 检查任务是否启用
|
||||
if not self._is_job_enabled(method, job_config):
|
||||
logger.info(f"任务已禁用，跳过注册: {cls.__name__}.{method_name} ({module_name})")
|
||||
continue
|
||||
|
||||
# 任务级 all_workers 门控：非 primary worker 只注册声明了
|
||||
# all_workers=True 的任务（scheduler.on_all_workers=true 可全局覆盖）
|
||||
if not self._should_register_job_on_this_worker(app, job_config):
|
||||
logger.debug(
|
||||
f"当前 worker 非 primary 且任务未声明 all_workers，跳过注册: "
|
||||
f"{cls.__name__}.{method_name} ({module_name})"
|
||||
)
|
||||
continue
|
||||
|
||||
# 非 primary worker 上放行的 all_workers 任务：确保调度器实例启用
|
||||
# （非 primary worker 的调度器默认 enabled=False，仅靠注册门控
|
||||
# 决定实际任务集；scheduler.enabled=false 时上面门控已拦截）
|
||||
if not getattr(app, 'is_primary_worker', True):
|
||||
app.scheduler.enable()
|
||||
|
||||
# 注册任务
|
||||
try:
|
||||
if job_config['type'] == 'cron':
|
||||
app.scheduler.add_cron_job(
|
||||
func=method,
|
||||
cron=job_config['cron'],
|
||||
**job_config.get('kwargs', {})
|
||||
)
|
||||
elif job_config['type'] == 'interval':
|
||||
app.scheduler.add_interval_job(
|
||||
func=method,
|
||||
interval=job_config['interval'],
|
||||
**job_config.get('kwargs', {})
|
||||
)
|
||||
elif job_config['type'] == 'once':
|
||||
app.scheduler.add_date_job(
|
||||
func=method,
|
||||
run_date=job_config['run_date'],
|
||||
**job_config.get('kwargs', {})
|
||||
)
|
||||
|
||||
logger.info(f"自动注册任务（组件方法）: {cls.__name__}.{method_name} ({module_name})")
|
||||
except Exception as e:
|
||||
logger.error(f"注册任务失败 {cls.__name__}.{method_name}: {e}", exc_info=True)
|
||||
|
||||
def _auto_register_worker_hooks(self, app) -> None:
|
||||
"""自动注册 worker 生命周期钩子（@on_worker_start / @on_worker_stop）
|
||||
|
||||
模块级函数钩子按 order 排序后追加到 app.worker_start_hooks /
|
||||
app.worker_stop_hooks，由每个 worker 进程的 lifespan 触发。
|
||||
"""
|
||||
hook_infos = self.discovered_components.get('worker_hooks', [])
|
||||
if not hook_infos:
|
||||
return
|
||||
|
||||
start_hooks = []
|
||||
stop_hooks = []
|
||||
for hook_info in hook_infos:
|
||||
func = hook_info.get('function')
|
||||
if func is None:
|
||||
continue
|
||||
hook_config = getattr(func, '__myboot_worker_hook__', None)
|
||||
if not hook_config:
|
||||
continue
|
||||
order = hook_config.get('order', 0)
|
||||
if hook_config.get('event') == 'start':
|
||||
start_hooks.append((order, func, hook_info['module']))
|
||||
else:
|
||||
stop_hooks.append((order, func, hook_info['module']))
|
||||
|
||||
for order, func, module in sorted(start_hooks, key=lambda x: x[0]):
|
||||
app.add_worker_start_hook(func)
|
||||
logger.info(f"自动注册 worker 启动钩子: {module}.{func.__name__} (order={order})")
|
||||
for order, func, module in sorted(stop_hooks, key=lambda x: x[0]):
|
||||
app.add_worker_stop_hook(func)
|
||||
logger.info(f"自动注册 worker 停止钩子: {module}.{func.__name__} (order={order})")
|
||||
|
||||
|
||||
# 全局自动配置管理器实例
|
||||
_auto_configuration_manager = AutoConfigurationManager()
|
||||
|
||||
|
||||
def auto_discover(package_name: str = "app") -> None:
|
||||
"""自动发现应用组件"""
|
||||
_auto_configuration_manager.auto_discover(package_name)
|
||||
|
||||
|
||||
def apply_auto_configuration(app) -> None:
|
||||
"""应用自动配置"""
|
||||
_auto_configuration_manager.apply_auto_configuration(app)
|
||||
@@ -0,0 +1,226 @@
|
||||
"""
|
||||
配置管理模块
|
||||
|
||||
使用 Dynaconf 提供强大的配置管理功能
|
||||
支持远程加载文件、配置文件优先级、环境变量覆盖等
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
import requests
|
||||
from dynaconf import Dynaconf
|
||||
|
||||
|
||||
def _find_project_root() -> str:
|
||||
"""查找项目根目录"""
|
||||
current_dir = Path(__file__).parent.absolute()
|
||||
|
||||
while current_dir.parent != current_dir:
|
||||
if (current_dir / 'pyproject.toml').exists():
|
||||
return str(current_dir)
|
||||
current_dir = current_dir.parent
|
||||
|
||||
return os.getcwd()
|
||||
|
||||
|
||||
def _is_url(path: str) -> bool:
|
||||
"""检查是否为 URL"""
|
||||
return path and path.startswith(('http://', 'https://'))
|
||||
|
||||
|
||||
def _download_config(url: str, cache_dir: str) -> str:
|
||||
"""下载配置文件到缓存"""
|
||||
import hashlib
|
||||
|
||||
url_hash = hashlib.md5(url.encode()).hexdigest()
|
||||
cache_path = os.path.join(cache_dir, f"{url_hash}.yaml")
|
||||
|
||||
try:
|
||||
print(f"正在下载配置文件: {url}")
|
||||
response = requests.get(url, timeout=30)
|
||||
response.raise_for_status()
|
||||
|
||||
with open(cache_path, 'w', encoding='utf-8') as f:
|
||||
f.write(response.text)
|
||||
|
||||
print(f"配置文件已更新并缓存到: {cache_path}")
|
||||
return cache_path
|
||||
except Exception as e:
|
||||
print(f"下载配置文件失败: {e}")
|
||||
|
||||
if os.path.exists(cache_path):
|
||||
print(f"网络连接失败，使用缓存的配置文件: {cache_path}")
|
||||
return cache_path
|
||||
else:
|
||||
print(f"无可用缓存文件，下载失败: {e}")
|
||||
raise
|
||||
|
||||
|
||||
def _get_config_files(config_file: Optional[str] = None) -> list:
|
||||
"""获取配置文件列表（供 Dynaconf ``settings_files`` 使用）。
|
||||
|
||||
Dynaconf 按列表顺序依次加载，**后加载的文件会覆盖先加载的同名字段**。
|
||||
因此列表顺序为「基底 → 越来越高优先级」，而不是把「最重要」的文件放在最前。
|
||||
|
||||
合并结果上的优先级（后者覆盖前者）：
|
||||
项目根目录 ``config.yaml`` < ``conf/config.yaml`` < 参数 ``config_file`` < 环境变量 ``CONFIG_FILE``
|
||||
"""
|
||||
project_root = _find_project_root()
|
||||
cache_dir = os.path.join(tempfile.gettempdir(), 'myboot_config_cache')
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
|
||||
config_files = []
|
||||
added_paths = set() # 用于去重
|
||||
|
||||
# 与 Dynaconf 加载顺序一致：靠前的先加载（作默认），靠后的覆盖同名键
|
||||
config_paths = [
|
||||
# 1. 项目根目录（最先加载）
|
||||
os.path.join(project_root, 'config.yaml'),
|
||||
os.path.join(project_root, 'config.yml'),
|
||||
# 2. conf 目录
|
||||
os.path.join(project_root, 'conf', 'config.yaml'),
|
||||
os.path.join(project_root, 'conf', 'config.yml'),
|
||||
# 3. 调用方显式传入
|
||||
config_file,
|
||||
# 4. CONFIG_FILE 环境变量（最后加载，覆盖上述来源中的同名字段）
|
||||
os.getenv('CONFIG_FILE'),
|
||||
]
|
||||
|
||||
for config_path in config_paths:
|
||||
if not config_path:
|
||||
continue
|
||||
|
||||
# 处理 URL 配置
|
||||
if _is_url(config_path):
|
||||
downloaded_path = _download_config(config_path, cache_dir)
|
||||
if downloaded_path and downloaded_path not in added_paths:
|
||||
config_files.append(downloaded_path)
|
||||
added_paths.add(downloaded_path)
|
||||
# 处理文件路径
|
||||
elif os.path.exists(config_path) and config_path not in added_paths:
|
||||
config_files.append(config_path)
|
||||
added_paths.add(config_path)
|
||||
|
||||
return config_files
|
||||
|
||||
|
||||
def create_settings(config_file: Optional[str] = None) -> Dynaconf:
|
||||
"""创建 Dynaconf 设置实例"""
|
||||
config_files = _get_config_files(config_file)
|
||||
|
||||
# 创建 Dynaconf 配置
|
||||
settings = Dynaconf(
|
||||
# 配置文件列表
|
||||
settings_files=config_files,
|
||||
|
||||
# 环境变量前缀（禁用前缀）
|
||||
envvar_prefix=False,
|
||||
|
||||
# 环境变量分隔符
|
||||
envvar_separator="__",
|
||||
|
||||
# 是否自动转换环境变量类型
|
||||
env_parse_values=True,
|
||||
|
||||
# 是否忽略空值
|
||||
ignore_unknown_envvars=True,
|
||||
|
||||
# 是否合并环境变量
|
||||
merge_enabled=True,
|
||||
|
||||
# 自动加载项目根目录 .env（不覆盖已存在的真实环境变量，
|
||||
# 与容器部署习惯一致：真实 env > .env > 配置文件）
|
||||
load_dotenv=True,
|
||||
dotenv_path=os.path.join(_find_project_root(), ".env"),
|
||||
dotenv_override=False,
|
||||
|
||||
# 默认值：以大写 kwargs 传入（Dynaconf 将任意大写 kwarg 注册为默认配置项）。
|
||||
# 注意不能用 default_settings= —— Dynaconf 没有该参数，
|
||||
# 它会被当成一个名为 DEFAULT_SETTINGS 的普通配置项，默认值整体失效
|
||||
APP={
|
||||
"name": "MyBoot App",
|
||||
"version": "0.1.0"
|
||||
},
|
||||
SERVER={
|
||||
"host": "0.0.0.0",
|
||||
"port": 8000,
|
||||
"reload": True,
|
||||
"workers": 1,
|
||||
"keep_alive_timeout": 5,
|
||||
"graceful_timeout": 30,
|
||||
"response_format": {
|
||||
"enabled": True,
|
||||
"exclude_paths": ["/docs"]
|
||||
}
|
||||
},
|
||||
LOGGING={
|
||||
"level": "INFO"
|
||||
},
|
||||
METRICS={
|
||||
"enabled": False,
|
||||
"path": "/metrics",
|
||||
"multiproc_dir": None,
|
||||
"http_metrics": True
|
||||
},
|
||||
SCHEDULER={
|
||||
"enabled": True,
|
||||
"timezone": "UTC",
|
||||
"max_workers": 10
|
||||
}
|
||||
)
|
||||
|
||||
return settings
|
||||
|
||||
|
||||
# 全局配置实例
|
||||
_settings: Optional[Dynaconf] = None
|
||||
|
||||
|
||||
def get_settings(config_file: Optional[str] = None) -> Dynaconf:
|
||||
"""获取 Dynaconf 设置实例"""
|
||||
global _settings
|
||||
|
||||
if _settings is None:
|
||||
_settings = create_settings(config_file)
|
||||
|
||||
return _settings
|
||||
|
||||
|
||||
# 为了向后兼容，保留一些便捷函数
|
||||
def get_config(key: str, default: Any = None) -> Any:
|
||||
"""获取配置值的便捷函数"""
|
||||
return get_settings().get(key, default)
|
||||
|
||||
|
||||
def get_config_str(key: str, default: str = "") -> str:
|
||||
"""获取字符串配置值的便捷函数"""
|
||||
value = get_config(key, default)
|
||||
return str(value) if value is not None else default
|
||||
|
||||
|
||||
def get_config_int(key: str, default: int = 0) -> int:
|
||||
"""获取整数配置值的便捷函数"""
|
||||
value = get_config(key, default)
|
||||
try:
|
||||
return int(value)
|
||||
except (ValueError, TypeError):
|
||||
return default
|
||||
|
||||
|
||||
def get_config_bool(key: str, default: bool = False) -> bool:
|
||||
"""获取布尔配置值的便捷函数"""
|
||||
value = get_config(key, default)
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.lower() in ('true', '1', 'yes', 'on')
|
||||
return bool(value)
|
||||
|
||||
|
||||
def reload_config() -> None:
|
||||
"""重新加载配置"""
|
||||
global _settings
|
||||
_settings = None
|
||||
@@ -0,0 +1,198 @@
|
||||
"""
|
||||
统一容器模块
|
||||
|
||||
提供统一的容器接口，支持从 container、components、services、clients 中获取实例
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Type, TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .application import Application
|
||||
|
||||
|
||||
class Container:
|
||||
"""统一容器类，支持从 container、components、services、clients 中获取实例"""
|
||||
|
||||
def __init__(self, app: 'Application'):
|
||||
"""
|
||||
初始化容器
|
||||
|
||||
Args:
|
||||
app: 应用实例
|
||||
"""
|
||||
self._app = app
|
||||
# 容器自己的存储字典
|
||||
self._storage: Dict[str, Any] = {}
|
||||
|
||||
def put(self, name: str, instance: Any) -> None:
|
||||
"""
|
||||
将实例放入容器
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
instance: 实例对象
|
||||
"""
|
||||
self._storage[name] = instance
|
||||
self._app.logger.debug(f"已放入容器: {name} ({type(instance).__name__})")
|
||||
|
||||
def get(self, name: str, default: Any = None) -> Any:
|
||||
"""
|
||||
从容器中获取实例（优先从 container，然后从 components、services，最后从 clients）
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
default: 如果不存在时返回的默认值
|
||||
|
||||
Returns:
|
||||
实例对象，如果不存在则返回 default
|
||||
"""
|
||||
# 优先从 container 中获取
|
||||
if name in self._storage:
|
||||
return self._storage[name]
|
||||
|
||||
# 然后从 components 中获取
|
||||
if name in self._app.components:
|
||||
return self._app.components[name]
|
||||
|
||||
# 然后从 services 中获取
|
||||
if name in self._app.services:
|
||||
return self._app.services[name]
|
||||
|
||||
# 最后从 clients 中获取
|
||||
if name in self._app.clients:
|
||||
return self._app.clients[name]
|
||||
|
||||
# 非单例（request/factory）的 service/client 不在上述字典中预存，
|
||||
# 回退到 DI 容器按需解析
|
||||
di_container = getattr(self._app, 'di_container', None)
|
||||
if di_container is not None and di_container.has_service(name):
|
||||
return di_container.get_service(name)
|
||||
|
||||
return default
|
||||
|
||||
def get_or_raise(self, name: str) -> Any:
|
||||
"""
|
||||
从容器中获取实例，如果不存在则抛出异常
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
|
||||
Returns:
|
||||
实例对象
|
||||
|
||||
Raises:
|
||||
KeyError: 如果实例不存在
|
||||
"""
|
||||
instance = self.get(name)
|
||||
if instance is None:
|
||||
raise KeyError(f"容器中不存在实例: {name} (已检查 container、components、services、clients)")
|
||||
return instance
|
||||
|
||||
def has(self, name: str) -> bool:
|
||||
"""
|
||||
检查容器中是否存在指定名称的实例（检查 container、components、services、clients）
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
|
||||
Returns:
|
||||
是否存在
|
||||
"""
|
||||
return (name in self._storage or
|
||||
name in self._app.components or
|
||||
name in self._app.services or
|
||||
name in self._app.clients)
|
||||
|
||||
def remove(self, name: str) -> bool:
|
||||
"""
|
||||
从容器中移除实例（按顺序从 container、components、services、clients 中移除）
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
|
||||
Returns:
|
||||
是否成功移除
|
||||
"""
|
||||
removed = False
|
||||
if name in self._storage:
|
||||
del self._storage[name]
|
||||
self._app.logger.debug(f"已从容器移除: {name}")
|
||||
removed = True
|
||||
if name in self._app.components:
|
||||
del self._app.components[name]
|
||||
self._app.logger.debug(f"已从组件移除: {name}")
|
||||
removed = True
|
||||
if name in self._app.services:
|
||||
del self._app.services[name]
|
||||
self._app.logger.debug(f"已从服务移除: {name}")
|
||||
removed = True
|
||||
if name in self._app.clients:
|
||||
del self._app.clients[name]
|
||||
self._app.logger.debug(f"已从客户端移除: {name}")
|
||||
removed = True
|
||||
return removed
|
||||
|
||||
def get_by_type(self, instance_type: Type) -> List[Any]:
|
||||
"""
|
||||
根据类型获取容器中所有匹配的实例（从 container、components、services、clients 中查找）
|
||||
|
||||
Args:
|
||||
instance_type: 实例类型
|
||||
|
||||
Returns:
|
||||
匹配的实例列表
|
||||
"""
|
||||
instances = []
|
||||
# 从 container 中查找
|
||||
instances.extend([instance for instance in self._storage.values()
|
||||
if isinstance(instance, instance_type)])
|
||||
# 从 components 中查找
|
||||
instances.extend([instance for instance in self._app.components.values()
|
||||
if isinstance(instance, instance_type)])
|
||||
# 从 services 中查找
|
||||
instances.extend([instance for instance in self._app.services.values()
|
||||
if isinstance(instance, instance_type)])
|
||||
# 从 clients 中查找
|
||||
instances.extend([instance for instance in self._app.clients.values()
|
||||
if isinstance(instance, instance_type)])
|
||||
return instances
|
||||
|
||||
def list_all(self) -> Dict[str, Any]:
|
||||
"""
|
||||
列出容器中所有实例（包括 container、components、services、clients）
|
||||
|
||||
Returns:
|
||||
所有实例的字典（名称 -> 实例）
|
||||
"""
|
||||
all_instances = {}
|
||||
all_instances.update(self._storage)
|
||||
all_instances.update(self._app.components)
|
||||
all_instances.update(self._app.services)
|
||||
all_instances.update(self._app.clients)
|
||||
return all_instances
|
||||
|
||||
def clear(self) -> None:
|
||||
"""清空容器（清空 container、components、services、clients）"""
|
||||
self._storage.clear()
|
||||
self._app.components.clear()
|
||||
self._app.services.clear()
|
||||
self._app.clients.clear()
|
||||
self._app.logger.debug("容器已清空")
|
||||
|
||||
def __getitem__(self, name: str) -> Any:
|
||||
"""支持使用 [] 语法获取实例"""
|
||||
return self.get_or_raise(name)
|
||||
|
||||
def __setitem__(self, name: str, instance: Any) -> None:
|
||||
"""支持使用 [] 语法设置实例"""
|
||||
self.put(name, instance)
|
||||
|
||||
def __contains__(self, name: str) -> bool:
|
||||
"""支持使用 in 语法检查实例是否存在"""
|
||||
return self.has(name)
|
||||
|
||||
def __delitem__(self, name: str) -> None:
|
||||
"""支持使用 del 语法删除实例"""
|
||||
if not self.remove(name):
|
||||
raise KeyError(f"容器中不存在实例: {name}")
|
||||
|
||||
@@ -0,0 +1,436 @@
|
||||
"""
|
||||
装饰器模块
|
||||
|
||||
提供约定优于配置的装饰器，用于自动注册组件
|
||||
"""
|
||||
|
||||
import re
|
||||
from functools import wraps
|
||||
from typing import Any, Dict, List, Optional, Union, Callable
|
||||
|
||||
|
||||
from ..utils.naming import camel_to_snake as _camel_to_snake
|
||||
|
||||
|
||||
def route(path: str, methods: List[str] = None, **kwargs):
|
||||
"""
|
||||
路由装饰器
|
||||
|
||||
Args:
|
||||
path: 路由路径
|
||||
methods: HTTP 方法列表
|
||||
**kwargs: 其他路由参数
|
||||
"""
|
||||
if methods is None:
|
||||
methods = ['GET']
|
||||
|
||||
def decorator(func):
|
||||
func.__myboot_route__ = {
|
||||
'path': path,
|
||||
'methods': methods,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
def get(path: str, **kwargs):
|
||||
"""GET 路由装饰器"""
|
||||
return route(path, methods=['GET'], **kwargs)
|
||||
|
||||
|
||||
def post(path: str, **kwargs):
|
||||
"""POST 路由装饰器"""
|
||||
return route(path, methods=['POST'], **kwargs)
|
||||
|
||||
|
||||
def put(path: str, **kwargs):
|
||||
"""PUT 路由装饰器"""
|
||||
return route(path, methods=['PUT'], **kwargs)
|
||||
|
||||
|
||||
def delete(path: str, **kwargs):
|
||||
"""DELETE 路由装饰器"""
|
||||
return route(path, methods=['DELETE'], **kwargs)
|
||||
|
||||
|
||||
def patch(path: str, **kwargs):
|
||||
"""PATCH 路由装饰器"""
|
||||
return route(path, methods=['PATCH'], **kwargs)
|
||||
|
||||
|
||||
def cron(cron_expression: str, enabled: Optional[bool] = None, all_workers: bool = False, **kwargs):
|
||||
"""
|
||||
Cron 任务装饰器
|
||||
|
||||
支持在函数和类方法中使用：
|
||||
- 函数：@cron("0 0 * * *") 或 @cron("0 0 * * * *")
|
||||
- 类方法：在类的方法上使用 @cron("0 0 * * *")
|
||||
|
||||
Args:
|
||||
cron_expression: Cron 表达式
|
||||
- 标准5位格式：分 时 日 月 周（如 "0 0 * * *" 表示每小时）
|
||||
- 6位格式：秒 分 时 日 月 周（如 "0 0 * * * *" 表示每小时，兼容旧格式）
|
||||
enabled: 是否启用任务，如果为 None 则默认启用
|
||||
可以通过手动获取配置来传递，例如：
|
||||
from myboot.core.config import get_config
|
||||
enabled = get_config('jobs.heartbeat.enabled', True)
|
||||
all_workers: 多 worker 模式下是否在每个 worker 进程都注册执行
|
||||
默认 False（仅 primary worker 执行）
|
||||
**kwargs: 其他任务参数
|
||||
"""
|
||||
def decorator(func):
|
||||
func.__myboot_job__ = {
|
||||
'type': 'cron',
|
||||
'cron': cron_expression,
|
||||
'enabled': enabled,
|
||||
'all_workers': all_workers,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
def interval(seconds: int = None, minutes: int = None, hours: int = None, enabled: Optional[bool] = None, all_workers: bool = False, **kwargs):
|
||||
"""
|
||||
间隔任务装饰器
|
||||
|
||||
支持在函数和类方法中使用：
|
||||
- 函数：@interval(seconds=60)
|
||||
- 类方法：在类的方法上使用 @interval(seconds=60)
|
||||
|
||||
Args:
|
||||
seconds: 秒数
|
||||
minutes: 分钟数
|
||||
hours: 小时数
|
||||
enabled: 是否启用任务，如果为 None 则默认启用
|
||||
可以通过手动获取配置来传递，例如：
|
||||
from myboot.core.config import get_config
|
||||
enabled = get_config('jobs.heartbeat.enabled', True)
|
||||
all_workers: 多 worker 模式下是否在每个 worker 进程都注册执行
|
||||
默认 False（仅 primary worker 执行）
|
||||
**kwargs: 其他任务参数
|
||||
"""
|
||||
def decorator(func):
|
||||
interval_value = seconds or (minutes * 60) or (hours * 3600)
|
||||
func.__myboot_job__ = {
|
||||
'type': 'interval',
|
||||
'interval': interval_value,
|
||||
'enabled': enabled,
|
||||
'all_workers': all_workers,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
def once(run_date: str = None, enabled: Optional[bool] = None, all_workers: bool = False, **kwargs):
|
||||
"""
|
||||
一次性任务装饰器
|
||||
|
||||
支持在函数和类方法中使用：
|
||||
- 函数：@once('2025-12-31 23:59:59')
|
||||
- 类方法：在类的方法上使用 @once('2025-12-31 23:59:59')
|
||||
|
||||
Args:
|
||||
run_date: 运行日期
|
||||
enabled: 是否启用任务，如果为 None 则默认启用
|
||||
可以通过手动获取配置来传递，例如：
|
||||
from myboot.core.config import get_config
|
||||
enabled = get_config('jobs.heartbeat.enabled', True)
|
||||
all_workers: 多 worker 模式下是否在每个 worker 进程都注册执行
|
||||
默认 False（仅 primary worker 执行）
|
||||
**kwargs: 其他任务参数
|
||||
"""
|
||||
def decorator(func):
|
||||
func.__myboot_job__ = {
|
||||
'type': 'once',
|
||||
'run_date': run_date,
|
||||
'enabled': enabled,
|
||||
'all_workers': all_workers,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
# @service/@client 支持的生命周期范围
|
||||
# - singleton: 单例（默认），每个 worker 进程内一个实例
|
||||
# - request: 请求级，基于 contextvars（ContextLocalSingleton），
|
||||
# 同一 HTTP 请求（asyncio 任务）内同一实例，跨请求不同实例
|
||||
# - factory: 每次解析都创建新实例
|
||||
_VALID_SCOPES = {'singleton', 'request', 'factory'}
|
||||
|
||||
|
||||
def _validate_scope(scope: str, decorator_name: str) -> None:
|
||||
"""校验 scope 参数，装饰期即报错，避免拼写错误静默回退为单例"""
|
||||
if scope not in _VALID_SCOPES:
|
||||
raise ValueError(
|
||||
f"@{decorator_name} 不支持的 scope: {scope!r}，"
|
||||
f"可选值: {sorted(_VALID_SCOPES)}"
|
||||
)
|
||||
|
||||
|
||||
def service(name: str = None, scope: str = 'singleton', **kwargs):
|
||||
"""
|
||||
服务装饰器
|
||||
|
||||
Args:
|
||||
name: 服务名称
|
||||
scope: 生命周期范围（singleton/request/factory，默认 singleton）
|
||||
- 'singleton': 每个 worker 进程内单例
|
||||
- 'request': 每个请求（asyncio 任务上下文）内单例
|
||||
- 'factory': 每次解析创建新实例
|
||||
**kwargs: 其他服务参数
|
||||
"""
|
||||
_validate_scope(scope, 'service')
|
||||
|
||||
def decorator(cls):
|
||||
cls.__myboot_service__ = {
|
||||
'name': name or _camel_to_snake(cls.__name__),
|
||||
'scope': scope,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return cls
|
||||
return decorator
|
||||
|
||||
|
||||
def model(name: str = None, **kwargs):
|
||||
"""
|
||||
模型装饰器
|
||||
|
||||
Args:
|
||||
name: 模型名称
|
||||
**kwargs: 其他模型参数
|
||||
"""
|
||||
def decorator(cls):
|
||||
cls.__myboot_model__ = {
|
||||
'name': name or _camel_to_snake(cls.__name__),
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return cls
|
||||
return decorator
|
||||
|
||||
|
||||
def client(name: str = None, scope: str = 'singleton', **kwargs):
|
||||
"""
|
||||
客户端装饰器
|
||||
|
||||
Args:
|
||||
name: 客户端名称
|
||||
scope: 生命周期范围（singleton/request/factory，默认 singleton）
|
||||
- 'singleton': 每个 worker 进程内单例，注册到 app.clients
|
||||
- 'request': 每个请求（asyncio 任务上下文）内单例，
|
||||
通过 DI 容器按需解析，不出现在 app.clients
|
||||
- 'factory': 每次解析创建新实例，不出现在 app.clients
|
||||
**kwargs: 其他客户端参数
|
||||
"""
|
||||
_validate_scope(scope, 'client')
|
||||
|
||||
def decorator(cls):
|
||||
cls.__myboot_client__ = {
|
||||
'name': name or _camel_to_snake(cls.__name__),
|
||||
'scope': scope,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return cls
|
||||
return decorator
|
||||
|
||||
|
||||
def component(
|
||||
name: str = None,
|
||||
scope: str = 'singleton',
|
||||
primary: bool = False,
|
||||
lazy: bool = False,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
组件装饰器
|
||||
|
||||
用于将类注册为通用组件，支持依赖注入。
|
||||
可用于任意需要托管的类（Job 实例、工具类、配置类等）。
|
||||
|
||||
Args:
|
||||
name: 组件名称，默认使用类名的 snake_case 形式
|
||||
scope: 生命周期范围
|
||||
- 'singleton': 单例模式（默认），整个应用生命周期内只创建一个实例
|
||||
- 'prototype': 原型模式，每次获取时创建新实例
|
||||
primary: 当按类型获取有多个匹配时，是否为首选
|
||||
lazy: 是否懒加载，True 时在首次使用时才创建实例
|
||||
**kwargs: 其他组件参数
|
||||
|
||||
Examples:
|
||||
# 基本用法
|
||||
@component()
|
||||
class EmailHelper:
|
||||
def send(self, to: str, content: str):
|
||||
pass
|
||||
|
||||
# 带依赖注入
|
||||
@component(name='redis_cache')
|
||||
class RedisCache:
|
||||
def __init__(self, redis_client: RedisClient):
|
||||
self.redis = redis_client
|
||||
|
||||
# 包含定时任务的组件
|
||||
@component()
|
||||
class DataSyncJobs:
|
||||
def __init__(self, data_service: DataService):
|
||||
self.data_service = data_service
|
||||
|
||||
@cron("0 2 * * *")
|
||||
def sync_daily(self):
|
||||
self.data_service.sync()
|
||||
"""
|
||||
def decorator(cls):
|
||||
cls.__myboot_component__ = {
|
||||
'name': name or _camel_to_snake(cls.__name__),
|
||||
'scope': scope,
|
||||
'primary': primary,
|
||||
'lazy': lazy,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return cls
|
||||
return decorator
|
||||
|
||||
|
||||
def middleware(
|
||||
name: str = None,
|
||||
order: int = 0,
|
||||
path_filter: Union[str, List[str]] = None,
|
||||
methods: List[str] = None,
|
||||
condition: Callable = None,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
中间件装饰器
|
||||
|
||||
Args:
|
||||
name: 中间件名称
|
||||
order: 执行顺序，数字越小越先执行（默认 0）
|
||||
path_filter: 路径过滤，支持字符串、字符串列表或正则表达式模式
|
||||
例如: '/api/*', ['/api/*', '/admin/*']
|
||||
methods: HTTP 方法过滤，如 ['GET', 'POST']（默认 None，处理所有方法）
|
||||
condition: 条件函数，接收 request 对象，返回 bool 决定是否执行中间件
|
||||
**kwargs: 其他中间件参数
|
||||
|
||||
Examples:
|
||||
@middleware(order=1, path_filter='/api/*')
|
||||
def api_middleware(request, next_handler):
|
||||
return next_handler(request)
|
||||
|
||||
@middleware(order=2, methods=['POST', 'PUT'])
|
||||
def post_middleware(request, next_handler):
|
||||
return next_handler(request)
|
||||
"""
|
||||
def decorator(func):
|
||||
func.__myboot_middleware__ = {
|
||||
'name': name or func.__name__,
|
||||
'order': order,
|
||||
'path_filter': path_filter,
|
||||
'methods': methods,
|
||||
'condition': condition,
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
def _worker_hook(event: str, order_or_func):
|
||||
"""on_worker_start/on_worker_stop 的公共实现
|
||||
|
||||
同时支持两种用法：
|
||||
@on_worker_start （裸装饰器）
|
||||
@on_worker_start(order=1) （带参数）
|
||||
"""
|
||||
if callable(order_or_func):
|
||||
# 裸装饰器用法：@on_worker_start
|
||||
func = order_or_func
|
||||
func.__myboot_worker_hook__ = {'event': event, 'order': 0}
|
||||
return func
|
||||
|
||||
order = order_or_func
|
||||
|
||||
def decorator(func):
|
||||
func.__myboot_worker_hook__ = {'event': event, 'order': order}
|
||||
return func
|
||||
return decorator
|
||||
|
||||
|
||||
def on_worker_start(order: Union[int, Callable] = 0):
|
||||
"""
|
||||
Worker 启动钩子装饰器
|
||||
|
||||
被装饰的模块级函数在每个 worker 进程的 lifespan 启动阶段触发一次
|
||||
（startup_hooks 之后、调度器启动之前）。单 worker 模式下也触发一次，
|
||||
语义一致。支持同步和异步函数。
|
||||
|
||||
钩子函数不接收参数，可通过 myboot.core.application.app() 读取
|
||||
worker_id / is_primary_worker 等信息。
|
||||
|
||||
Args:
|
||||
order: 执行顺序，数字越小越先执行（默认 0）
|
||||
|
||||
Examples:
|
||||
@on_worker_start
|
||||
def init_pool():
|
||||
...
|
||||
|
||||
@on_worker_start(order=1)
|
||||
async def warm_cache():
|
||||
...
|
||||
"""
|
||||
return _worker_hook('start', order)
|
||||
|
||||
|
||||
def on_worker_stop(order: Union[int, Callable] = 0):
|
||||
"""
|
||||
Worker 停止钩子装饰器
|
||||
|
||||
被装饰的模块级函数在每个 worker 进程的 lifespan 关闭阶段触发一次
|
||||
（调度器停止之后、shutdown_hooks 之前）。支持同步和异步函数。
|
||||
|
||||
注意：Windows 多 worker 模式下父进程通过 terminate()（硬终止）清理
|
||||
worker 进程，lifespan 关闭阶段可能不会执行，因此 stop 钩子在
|
||||
Windows 上不保证触发。
|
||||
|
||||
Args:
|
||||
order: 执行顺序，数字越小越先执行（默认 0）
|
||||
"""
|
||||
return _worker_hook('stop', order)
|
||||
|
||||
|
||||
def rest_controller(base_path: str, **kwargs):
|
||||
"""
|
||||
REST 控制器装饰器
|
||||
|
||||
用于标记 REST 控制器类，为类中的方法提供基础路径。
|
||||
类中的方法需要显式使用 @get、@post、@put、@delete、@patch 等装饰器才会生成路由。
|
||||
|
||||
路径合并规则：
|
||||
- 方法路径以 // 开头：作为绝对路径使用（去掉一个 /）
|
||||
- 方法路径以 / 开头：去掉开头的 / 后追加到基础路径
|
||||
- 方法路径不以 / 开头：直接追加到基础路径
|
||||
|
||||
示例:
|
||||
@rest_controller('/api/reports')
|
||||
class ReportController:
|
||||
@post('/generate') # 最终路径: POST /api/reports/generate
|
||||
def create_report(self, report_type: str):
|
||||
return {"message": "报告生成任务已创建"}
|
||||
|
||||
@get('/status/{job_id}') # 最终路径: GET /api/reports/status/{job_id}
|
||||
def get_status(self, job_id: str):
|
||||
return {"status": "completed"}
|
||||
|
||||
Args:
|
||||
base_path: 基础路径
|
||||
**kwargs: 其他路由参数
|
||||
"""
|
||||
def decorator(cls):
|
||||
cls.__myboot_rest_controller__ = {
|
||||
'base_path': base_path.rstrip('/'),
|
||||
'kwargs': kwargs
|
||||
}
|
||||
return cls
|
||||
return decorator
|
||||
@@ -0,0 +1,17 @@
|
||||
"""
|
||||
依赖注入模块
|
||||
|
||||
提供基于 dependency_injector 的自动依赖注入功能
|
||||
"""
|
||||
|
||||
from .container import DependencyContainer
|
||||
from .registry import ServiceRegistry
|
||||
from .decorators import inject, Provide
|
||||
|
||||
__all__ = [
|
||||
'DependencyContainer',
|
||||
'ServiceRegistry',
|
||||
'inject',
|
||||
'Provide',
|
||||
]
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
依赖注入容器
|
||||
|
||||
管理 dependency_injector Container 和服务的生命周期
|
||||
"""
|
||||
|
||||
from typing import Dict, Type, Any, Optional
|
||||
from dependency_injector import containers, providers
|
||||
from loguru import logger
|
||||
|
||||
from .registry import ServiceRegistry
|
||||
from .providers import ServiceProvider
|
||||
|
||||
|
||||
class DependencyContainer:
|
||||
"""依赖注入容器管理器"""
|
||||
|
||||
def __init__(self):
|
||||
"""初始化依赖注入容器"""
|
||||
self.container = containers.DynamicContainer()
|
||||
self.registry = ServiceRegistry()
|
||||
self.service_providers: Dict[str, ServiceProvider] = {}
|
||||
self.service_instances: Dict[str, Any] = {}
|
||||
self._wired = False
|
||||
|
||||
def register_service(
|
||||
self,
|
||||
service_class: Type,
|
||||
service_name: str,
|
||||
scope: str = ServiceProvider.SINGLETON,
|
||||
config: dict = None
|
||||
) -> None:
|
||||
"""
|
||||
注册服务到容器
|
||||
|
||||
Args:
|
||||
service_class: 服务类
|
||||
service_name: 服务名称
|
||||
scope: 生命周期范围 (singleton/factory)
|
||||
config: 服务配置
|
||||
"""
|
||||
# 注册到注册表
|
||||
self.registry.register_service(service_class, service_name, config)
|
||||
|
||||
# 创建服务提供者
|
||||
# 注意：装饰器元数据（config）中可能含顶层 'scope' 键，
|
||||
# 与显式 scope 参数冲突，这里剔除（以显式参数为准）
|
||||
config_kwargs = {k: v for k, v in (config or {}).items() if k != 'scope'}
|
||||
provider = ServiceProvider(service_class, service_name, scope, **config_kwargs)
|
||||
self.service_providers[service_name] = provider
|
||||
|
||||
def register_instance(self, name: str, instance: Any) -> None:
|
||||
"""
|
||||
注册已创建的实例到容器（用于 Client 等外部创建的实例）
|
||||
|
||||
Args:
|
||||
name: 实例名称
|
||||
instance: 实例对象
|
||||
"""
|
||||
# 直接缓存实例
|
||||
self.service_instances[name] = instance
|
||||
|
||||
# 通知注册表（避免依赖检查时产生警告）
|
||||
self.registry.known_instances.add(name)
|
||||
|
||||
# 创建一个返回已有实例的提供者
|
||||
di_provider = providers.Object(instance)
|
||||
setattr(self.container, name, di_provider)
|
||||
|
||||
# 创建占位的 ServiceProvider（用于 has_service 检查）
|
||||
placeholder = ServiceProvider(type(instance), name, ServiceProvider.SINGLETON)
|
||||
placeholder._provider = di_provider
|
||||
self.service_providers[name] = placeholder
|
||||
|
||||
def build_container(self) -> None:
|
||||
"""
|
||||
构建依赖注入容器
|
||||
|
||||
按照依赖顺序注册所有服务到 Container
|
||||
"""
|
||||
# 构建依赖图
|
||||
self.registry.build_dependency_graph()
|
||||
|
||||
# 获取初始化顺序
|
||||
try:
|
||||
init_order = self.registry.get_initialization_order()
|
||||
except ValueError as e:
|
||||
logger.error(f"无法构建依赖注入容器: {e}")
|
||||
raise
|
||||
|
||||
# 获取服务的参数信息，用于正确映射依赖
|
||||
service_params = {}
|
||||
for service_name in self.service_providers.keys():
|
||||
service_class = self.registry.get_service_class(service_name)
|
||||
if service_class and hasattr(service_class, '__init__'):
|
||||
from .decorators import get_injectable_params
|
||||
params = get_injectable_params(service_class.__init__)
|
||||
service_params[service_name] = params
|
||||
|
||||
# 按顺序注册服务
|
||||
for service_name in init_order:
|
||||
if service_name not in self.service_providers:
|
||||
continue
|
||||
|
||||
provider = self.service_providers[service_name]
|
||||
deps = self.registry.get_dependencies(service_name)
|
||||
|
||||
# 构建依赖字典（使用参数名作为键）
|
||||
dependencies = {}
|
||||
params = service_params.get(service_name, {})
|
||||
|
||||
for param_name, param_info in params.items():
|
||||
dep_service_name = param_info.get('service_name')
|
||||
if dep_service_name and dep_service_name in self.service_providers:
|
||||
# 从容器中获取依赖的提供者
|
||||
dep_provider = self.service_providers[dep_service_name]
|
||||
if not dep_provider.get_provider():
|
||||
# 如果依赖的提供者还未创建，先创建它
|
||||
dep_deps = self.registry.get_dependencies(dep_service_name)
|
||||
dep_dependencies = {}
|
||||
dep_params = service_params.get(dep_service_name, {})
|
||||
for dep_param_name, dep_param_info in dep_params.items():
|
||||
nested_dep_name = dep_param_info.get('service_name')
|
||||
if nested_dep_name and nested_dep_name in self.service_providers:
|
||||
nested_provider = self.service_providers[nested_dep_name]
|
||||
if nested_provider.get_provider():
|
||||
dep_dependencies[dep_param_name] = nested_provider.get_provider()
|
||||
dep_provider.create_provider(dep_dependencies if dep_dependencies else None)
|
||||
|
||||
# 使用参数名作为键（dependency_injector 需要参数名匹配）
|
||||
dependencies[param_name] = dep_provider.get_provider()
|
||||
|
||||
# 创建提供者
|
||||
di_provider = provider.create_provider(dependencies if dependencies else None)
|
||||
|
||||
# 注册到容器
|
||||
setattr(self.container, service_name, di_provider)
|
||||
|
||||
logger.debug(f"已注册服务提供者: {service_name} (依赖: {deps})")
|
||||
|
||||
def wire_modules(self, *modules) -> None:
|
||||
"""
|
||||
连接模块以启用依赖注入
|
||||
|
||||
Args:
|
||||
*modules: 要连接的模块列表
|
||||
"""
|
||||
if not self._wired:
|
||||
self.container.wire(modules=modules)
|
||||
self._wired = True
|
||||
logger.debug(f"已连接模块: {modules}")
|
||||
|
||||
def unwire_modules(self) -> None:
|
||||
"""断开模块连接"""
|
||||
if self._wired:
|
||||
self.container.unwire()
|
||||
self._wired = False
|
||||
logger.debug("已断开模块连接")
|
||||
|
||||
def get_service(self, service_name: str) -> Any:
|
||||
"""
|
||||
获取服务实例
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
服务实例
|
||||
|
||||
Raises:
|
||||
KeyError: 如果服务不存在
|
||||
"""
|
||||
# 先检查是否有已缓存的实例（包括通过 register_instance 注册的）
|
||||
if service_name in self.service_instances:
|
||||
return self.service_instances[service_name]
|
||||
|
||||
if service_name not in self.service_providers:
|
||||
raise KeyError(f"服务 '{service_name}' 未注册")
|
||||
|
||||
# 如果是单例，缓存实例
|
||||
provider = self.service_providers[service_name]
|
||||
if provider.scope == ServiceProvider.SINGLETON:
|
||||
di_provider = getattr(self.container, service_name, None)
|
||||
if di_provider:
|
||||
self.service_instances[service_name] = di_provider()
|
||||
else:
|
||||
raise RuntimeError(f"服务 '{service_name}' 的提供者未正确配置")
|
||||
return self.service_instances[service_name]
|
||||
else:
|
||||
# 工厂模式，每次创建新实例
|
||||
di_provider = getattr(self.container, service_name, None)
|
||||
if di_provider:
|
||||
return di_provider()
|
||||
else:
|
||||
raise RuntimeError(f"服务 '{service_name}' 的提供者未正确配置")
|
||||
|
||||
def has_service(self, service_name: str) -> bool:
|
||||
"""
|
||||
检查服务是否已注册
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
是否存在
|
||||
"""
|
||||
return service_name in self.service_providers or service_name in self.service_instances
|
||||
|
||||
def get_all_services(self) -> Dict[str, Any]:
|
||||
"""
|
||||
获取所有服务实例（仅单例）
|
||||
|
||||
Returns:
|
||||
服务名称到实例的字典
|
||||
"""
|
||||
result = {}
|
||||
for service_name in self.service_providers.keys():
|
||||
try:
|
||||
result[service_name] = self.get_service(service_name)
|
||||
except Exception as e:
|
||||
logger.warning(f"无法获取服务 '{service_name}': {e}")
|
||||
return result
|
||||
|
||||
def clear(self) -> None:
|
||||
"""清空容器"""
|
||||
self.unwire_modules()
|
||||
self.container = containers.DynamicContainer()
|
||||
self.registry.clear()
|
||||
self.service_providers.clear()
|
||||
self.service_instances.clear()
|
||||
self._wired = False
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
"""
|
||||
依赖注入装饰器
|
||||
|
||||
提供 @inject 装饰器和 Provide 类型提示
|
||||
"""
|
||||
|
||||
from typing import Any, TypeVar, Generic
|
||||
from functools import wraps
|
||||
import inspect
|
||||
|
||||
T = TypeVar('T')
|
||||
|
||||
|
||||
class Provide(Generic[T]):
|
||||
"""
|
||||
依赖提供者类型提示
|
||||
|
||||
用于在类型注解中显式指定需要注入的服务名称
|
||||
|
||||
Example:
|
||||
@service()
|
||||
class OrderService:
|
||||
@inject
|
||||
def __init__(
|
||||
self,
|
||||
user_service: Provide['user_service'],
|
||||
email_service: Provide['email_service']
|
||||
):
|
||||
self.user_service = user_service
|
||||
self.email_service = email_service
|
||||
"""
|
||||
def __class_getitem__(cls, item: str) -> str:
|
||||
"""支持 Provide['service_name'] 语法"""
|
||||
return item
|
||||
|
||||
|
||||
def inject(func):
|
||||
"""
|
||||
依赖注入装饰器
|
||||
|
||||
用于标记需要自动注入依赖的方法（通常是 __init__）
|
||||
|
||||
Example:
|
||||
@service()
|
||||
class OrderService:
|
||||
@inject
|
||||
def __init__(self, user_service: UserService):
|
||||
self.user_service = user_service
|
||||
|
||||
Note:
|
||||
如果使用类型注解，通常不需要显式使用 @inject 装饰器
|
||||
框架会自动检测并注入依赖
|
||||
"""
|
||||
if not hasattr(func, '__myboot_inject__'):
|
||||
func.__myboot_inject__ = True
|
||||
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
def get_injectable_params(func) -> dict:
|
||||
"""
|
||||
获取可注入的参数信息
|
||||
|
||||
Args:
|
||||
func: 函数或方法
|
||||
|
||||
Returns:
|
||||
参数字典，格式: {param_name: {type, service_name, is_optional, default}}
|
||||
"""
|
||||
signature = inspect.signature(func)
|
||||
params = {}
|
||||
|
||||
for param_name, param in signature.parameters.items():
|
||||
if param_name == 'self':
|
||||
continue
|
||||
|
||||
param_type = param.annotation
|
||||
is_optional = False
|
||||
service_name = None
|
||||
|
||||
# 处理类型注解
|
||||
if param_type != inspect.Parameter.empty:
|
||||
# 处理 Provide['service_name'] 类型（字符串形式）
|
||||
if isinstance(param_type, str):
|
||||
if param_type.startswith("Provide['") and param_type.endswith("']"):
|
||||
service_name = param_type[9:-2] # 提取 'service_name'
|
||||
param_type = None
|
||||
# 处理类型对象
|
||||
elif hasattr(param_type, '__origin__'):
|
||||
origin = param_type.__origin__
|
||||
args = getattr(param_type, '__args__', ())
|
||||
|
||||
# 处理 Optional[Type] 或 Union[Type, None]
|
||||
if origin is type(None) or (hasattr(origin, '__name__') and origin.__name__ == 'Union'):
|
||||
# 取第一个非 None 的类型
|
||||
for arg in args:
|
||||
if arg is not type(None):
|
||||
param_type = arg
|
||||
is_optional = True
|
||||
break
|
||||
|
||||
# 处理 Provide[Type] 泛型
|
||||
if hasattr(origin, '__name__'):
|
||||
origin_name = origin.__name__
|
||||
if 'Provide' in origin_name or str(origin).startswith('typing.Union') and any('Provide' in str(arg) for arg in args):
|
||||
# 查找 Provide 类型参数
|
||||
for arg in args:
|
||||
if isinstance(arg, str):
|
||||
service_name = arg
|
||||
param_type = None
|
||||
break
|
||||
elif hasattr(arg, '__origin__') and 'Provide' in str(arg):
|
||||
# 处理嵌套的 Provide
|
||||
nested_args = getattr(arg, '__args__', ())
|
||||
if nested_args and isinstance(nested_args[0], str):
|
||||
service_name = nested_args[0]
|
||||
param_type = None
|
||||
break
|
||||
|
||||
# 如果没有显式指定服务名，尝试从类型推断
|
||||
if service_name is None and param_type != inspect.Parameter.empty:
|
||||
if hasattr(param_type, '__name__'):
|
||||
# 将类名转换为服务名（驼峰转下划线）
|
||||
from myboot.core.decorators import _camel_to_snake
|
||||
service_name = _camel_to_snake(param_type.__name__)
|
||||
elif isinstance(param_type, type):
|
||||
# 处理直接的类型对象
|
||||
from myboot.core.decorators import _camel_to_snake
|
||||
service_name = _camel_to_snake(param_type.__name__)
|
||||
|
||||
# 检查默认值（表示可选）
|
||||
if param.default != inspect.Parameter.empty:
|
||||
is_optional = True
|
||||
|
||||
params[param_name] = {
|
||||
'type': param_type,
|
||||
'service_name': service_name,
|
||||
'is_optional': is_optional,
|
||||
'default': param.default if param.default != inspect.Parameter.empty else None
|
||||
}
|
||||
|
||||
return params
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
"""
|
||||
服务提供者配置
|
||||
|
||||
定义服务的提供者配置和生命周期管理
|
||||
"""
|
||||
|
||||
from typing import Any, Type, Optional
|
||||
from dependency_injector import providers
|
||||
|
||||
|
||||
class ServiceProvider:
|
||||
"""服务提供者配置"""
|
||||
|
||||
SINGLETON = 'singleton'
|
||||
REQUEST = 'request'
|
||||
FACTORY = 'factory'
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
service_class: Type,
|
||||
service_name: str,
|
||||
scope: str = SINGLETON,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
初始化服务提供者
|
||||
|
||||
Args:
|
||||
service_class: 服务类
|
||||
service_name: 服务名称
|
||||
scope: 生命周期范围 (singleton/request/factory)
|
||||
**kwargs: 其他配置参数
|
||||
"""
|
||||
self.service_class = service_class
|
||||
self.service_name = service_name
|
||||
self.scope = scope
|
||||
self.kwargs = kwargs
|
||||
self._provider: Optional[Any] = None
|
||||
|
||||
def create_provider(self, dependencies: dict = None) -> Any:
|
||||
"""
|
||||
创建 dependency_injector Provider
|
||||
|
||||
scope 到 Provider 的映射:
|
||||
- singleton -> providers.Singleton（默认）
|
||||
- request -> providers.ContextLocalSingleton（基于 contextvars，
|
||||
每个 asyncio 任务/HTTP 请求内单例）
|
||||
- 其他 -> providers.Factory（每次创建新实例）
|
||||
|
||||
Args:
|
||||
dependencies: 依赖的服务提供者字典
|
||||
|
||||
Returns:
|
||||
dependency_injector Provider 实例
|
||||
"""
|
||||
if self.scope == self.SINGLETON:
|
||||
provider_class = providers.Singleton
|
||||
elif self.scope == self.REQUEST:
|
||||
provider_class = providers.ContextLocalSingleton
|
||||
else:
|
||||
provider_class = providers.Factory
|
||||
|
||||
if dependencies:
|
||||
self._provider = provider_class(self.service_class, **dependencies)
|
||||
else:
|
||||
self._provider = provider_class(self.service_class)
|
||||
|
||||
return self._provider
|
||||
|
||||
def get_provider(self) -> Any:
|
||||
"""获取提供者实例"""
|
||||
return self._provider
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
"""
|
||||
服务注册表
|
||||
|
||||
负责扫描服务、分析依赖关系、构建依赖图
|
||||
"""
|
||||
|
||||
import inspect
|
||||
from typing import Dict, List, Type, Set, Optional, Tuple
|
||||
from collections import defaultdict, deque
|
||||
from loguru import logger
|
||||
|
||||
from .decorators import get_injectable_params
|
||||
|
||||
|
||||
class ServiceRegistry:
|
||||
"""服务注册表"""
|
||||
|
||||
def __init__(self):
|
||||
"""初始化服务注册表"""
|
||||
self.services: Dict[str, Type] = {} # service_name -> service_class
|
||||
self.service_configs: Dict[str, dict] = {} # service_name -> config
|
||||
self.dependencies: Dict[str, Set[str]] = {} # service_name -> set of dependency names
|
||||
self.dependents: Dict[str, Set[str]] = {} # service_name -> set of dependent names
|
||||
self._dependency_graph: Optional[Dict[str, Set[str]]] = None
|
||||
self.known_instances: Set[str] = set() # 已知的外部实例名称（如 Client）
|
||||
|
||||
def register_service(self, service_class: Type, service_name: str, config: dict = None) -> None:
|
||||
"""
|
||||
注册服务类
|
||||
|
||||
Args:
|
||||
service_class: 服务类
|
||||
service_name: 服务名称
|
||||
config: 服务配置
|
||||
"""
|
||||
self.services[service_name] = service_class
|
||||
self.service_configs[service_name] = config or {}
|
||||
self.dependencies[service_name] = set()
|
||||
# 不覆盖已有 dependents：可能已有「先注册的依赖方」写入的条目，覆盖会导致拓扑序错误
|
||||
self.dependents.setdefault(service_name, set[str]())
|
||||
|
||||
# 分析依赖关系
|
||||
self._analyze_dependencies(service_name, service_class)
|
||||
|
||||
# 注册时即检测循环依赖：只告警不报错（报错时机仍在容器构建时），
|
||||
# 让问题在注册阶段就可见，便于定位
|
||||
for cycle in self._find_cycles_involving(service_name):
|
||||
logger.warning(
|
||||
f"检测到循环依赖: {' -> '.join(cycle)}。"
|
||||
f"容器构建时将会失败，请重构代码以消除循环依赖。"
|
||||
)
|
||||
|
||||
def _find_cycles_involving(self, service_name: str) -> List[List[str]]:
|
||||
"""查找包含指定服务的循环依赖链（基于当前已注册的依赖关系，不使用缓存图）"""
|
||||
cycles: List[List[str]] = []
|
||||
path: List[str] = []
|
||||
rec_stack: Set[str] = set()
|
||||
|
||||
def dfs(node: str) -> None:
|
||||
if node in rec_stack:
|
||||
cycle_start = path.index(node)
|
||||
cycles.append(path[cycle_start:] + [node])
|
||||
return
|
||||
rec_stack.add(node)
|
||||
path.append(node)
|
||||
for neighbor in self.dependencies.get(node, set()):
|
||||
if neighbor in self.services and not cycles:
|
||||
dfs(neighbor)
|
||||
rec_stack.discard(node)
|
||||
path.pop()
|
||||
|
||||
dfs(service_name)
|
||||
return cycles
|
||||
|
||||
def _analyze_dependencies(self, service_name: str, service_class: Type) -> None:
|
||||
"""
|
||||
分析服务的依赖关系
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
service_class: 服务类
|
||||
"""
|
||||
if not hasattr(service_class, '__init__'):
|
||||
return
|
||||
|
||||
init_method = service_class.__init__
|
||||
params = get_injectable_params(init_method)
|
||||
|
||||
for param_name, param_info in params.items():
|
||||
dep_service_name = param_info['service_name']
|
||||
|
||||
if dep_service_name:
|
||||
# 检查依赖的服务是否存在（可能还未注册）
|
||||
# 先记录依赖关系，稍后在构建依赖图时验证
|
||||
self.dependencies[service_name].add(dep_service_name)
|
||||
if dep_service_name not in self.dependents:
|
||||
self.dependents[dep_service_name] = set()
|
||||
self.dependents[dep_service_name].add(service_name)
|
||||
|
||||
def build_dependency_graph(self) -> Dict[str, Set[str]]:
|
||||
"""
|
||||
构建依赖关系图
|
||||
|
||||
Returns:
|
||||
依赖关系图字典
|
||||
"""
|
||||
if self._dependency_graph is not None:
|
||||
return self._dependency_graph
|
||||
|
||||
self._dependency_graph = {}
|
||||
|
||||
# 验证所有依赖的服务都已注册（排除已知的外部实例如 Client）
|
||||
known_names = set(self.services.keys()) | self.known_instances
|
||||
for service_name, deps in self.dependencies.items():
|
||||
missing_deps = deps - known_names
|
||||
if missing_deps:
|
||||
logger.warning(
|
||||
f"服务 '{service_name}' 的依赖 '{missing_deps}' 未找到，"
|
||||
f"将在运行时检查"
|
||||
)
|
||||
|
||||
# 构建依赖图
|
||||
for service_name in self.services.keys():
|
||||
self._dependency_graph[service_name] = self.dependencies.get(service_name, set())
|
||||
|
||||
return self._dependency_graph
|
||||
|
||||
def detect_circular_dependencies(self) -> List[List[str]]:
|
||||
"""
|
||||
检测循环依赖
|
||||
|
||||
Returns:
|
||||
循环依赖列表，每个元素是一个循环依赖链
|
||||
"""
|
||||
graph = self.build_dependency_graph()
|
||||
cycles = []
|
||||
visited = set()
|
||||
rec_stack = set()
|
||||
path = []
|
||||
|
||||
def dfs(node: str) -> None:
|
||||
if node in rec_stack:
|
||||
# 找到循环
|
||||
cycle_start = path.index(node)
|
||||
cycle = path[cycle_start:] + [node]
|
||||
cycles.append(cycle)
|
||||
return
|
||||
|
||||
if node in visited:
|
||||
return
|
||||
|
||||
visited.add(node)
|
||||
rec_stack.add(node)
|
||||
path.append(node)
|
||||
|
||||
for neighbor in graph.get(node, set()):
|
||||
if neighbor in self.services: # 只检查已注册的服务
|
||||
dfs(neighbor)
|
||||
|
||||
rec_stack.remove(node)
|
||||
path.pop()
|
||||
|
||||
for service_name in self.services.keys():
|
||||
if service_name not in visited:
|
||||
dfs(service_name)
|
||||
|
||||
return cycles
|
||||
|
||||
def get_initialization_order(self) -> List[str]:
|
||||
"""
|
||||
获取服务初始化顺序（拓扑排序）
|
||||
|
||||
Returns:
|
||||
服务名称列表，按初始化顺序排列
|
||||
|
||||
Raises:
|
||||
ValueError: 如果存在循环依赖
|
||||
"""
|
||||
cycles = self.detect_circular_dependencies()
|
||||
if cycles:
|
||||
cycle_str = ' -> '.join(cycles[0])
|
||||
raise ValueError(
|
||||
f"检测到循环依赖: {cycle_str}。"
|
||||
f"请重构代码以消除循环依赖。"
|
||||
)
|
||||
|
||||
graph = self.build_dependency_graph()
|
||||
in_degree = defaultdict(int)
|
||||
|
||||
# 计算入度
|
||||
for service_name in self.services.keys():
|
||||
in_degree[service_name] = 0
|
||||
|
||||
for service_name, deps in graph.items():
|
||||
for dep in deps:
|
||||
if dep in self.services: # 只考虑已注册的服务
|
||||
in_degree[service_name] += 1
|
||||
|
||||
# 拓扑排序
|
||||
queue = deque([name for name, degree in in_degree.items() if degree == 0])
|
||||
result = []
|
||||
|
||||
while queue:
|
||||
service_name = queue.popleft()
|
||||
result.append(service_name)
|
||||
|
||||
# 更新依赖此服务的其他服务的入度
|
||||
for dependent in self.dependents.get(service_name, set()):
|
||||
if dependent in in_degree:
|
||||
in_degree[dependent] -= 1
|
||||
if in_degree[dependent] == 0:
|
||||
queue.append(dependent)
|
||||
|
||||
# 检查是否所有服务都已处理
|
||||
if len(result) != len(self.services):
|
||||
remaining = set(self.services.keys()) - set(result)
|
||||
logger.warning(f"以下服务无法确定初始化顺序: {remaining}")
|
||||
# 将剩余的服务添加到末尾
|
||||
result.extend(remaining)
|
||||
|
||||
return result
|
||||
|
||||
def get_dependencies(self, service_name: str) -> Set[str]:
|
||||
"""
|
||||
获取服务的依赖列表
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
依赖的服务名称集合
|
||||
"""
|
||||
return self.dependencies.get(service_name, set())
|
||||
|
||||
def get_dependents(self, service_name: str) -> Set[str]:
|
||||
"""
|
||||
获取依赖此服务的服务列表
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
依赖此服务的服务名称集合
|
||||
"""
|
||||
return self.dependents.get(service_name, set())
|
||||
|
||||
def get_service_class(self, service_name: str) -> Optional[Type]:
|
||||
"""
|
||||
获取服务类
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
服务类，如果不存在则返回 None
|
||||
"""
|
||||
return self.services.get(service_name)
|
||||
|
||||
def get_service_config(self, service_name: str) -> dict:
|
||||
"""
|
||||
获取服务配置
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
服务配置字典
|
||||
"""
|
||||
return self.service_configs.get(service_name, {})
|
||||
|
||||
def has_service(self, service_name: str) -> bool:
|
||||
"""
|
||||
检查服务是否已注册
|
||||
|
||||
Args:
|
||||
service_name: 服务名称
|
||||
|
||||
Returns:
|
||||
是否存在
|
||||
"""
|
||||
return service_name in self.services
|
||||
|
||||
def clear(self) -> None:
|
||||
"""清空注册表"""
|
||||
self.services.clear()
|
||||
self.service_configs.clear()
|
||||
self.dependencies.clear()
|
||||
self.dependents.clear()
|
||||
self._dependency_graph = None
|
||||
|
||||
@@ -0,0 +1,416 @@
|
||||
"""
|
||||
日志管理模块
|
||||
|
||||
基于 loguru 的日志管理，提供初始化配置功能
|
||||
所有代码可以直接使用: from loguru import logger
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from typing import Optional, Union
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
from dynaconf import Dynaconf
|
||||
|
||||
from .config import get_settings
|
||||
|
||||
logger = loguru_logger
|
||||
|
||||
|
||||
def _get_worker_info() -> str:
|
||||
"""
|
||||
获取当前 worker 信息
|
||||
|
||||
Returns:
|
||||
worker 标识字符串，格式为 "Worker-1/4" 或 "Main" (主进程)
|
||||
"""
|
||||
worker_id = os.environ.get("MYBOOT_WORKER_ID")
|
||||
worker_count = os.environ.get("MYBOOT_WORKER_COUNT")
|
||||
|
||||
if worker_id and worker_count:
|
||||
return f"Worker-{worker_id}/{worker_count}"
|
||||
return "Main" # Main process
|
||||
|
||||
|
||||
def _get_log_format_with_worker() -> str:
|
||||
"""
|
||||
获取带 worker 标识的日志格式
|
||||
|
||||
Returns:
|
||||
日志格式字符串
|
||||
"""
|
||||
return (
|
||||
"<green>{time:YYYY-MM-DD HH:mm:ss}</green> | "
|
||||
"<yellow>{extra[worker]}</yellow> | "
|
||||
"<level>{level: <8}</level> | "
|
||||
"<cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - "
|
||||
"<level>{message}</level>"
|
||||
)
|
||||
|
||||
|
||||
def _get_log_format_simple() -> str:
|
||||
"""
|
||||
获取简单日志格式（无 worker 标识）
|
||||
|
||||
Returns:
|
||||
日志格式字符串
|
||||
"""
|
||||
return (
|
||||
"<green>{time:YYYY-MM-DD HH:mm:ss}</green> | "
|
||||
"<level>{level: <8}</level> | "
|
||||
"<cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - "
|
||||
"<level>{message}</level>"
|
||||
)
|
||||
|
||||
|
||||
def _parse_json_config(value) -> bool:
|
||||
"""解析 JSON 配置值为布尔值"""
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.lower() in ("true", "1", "yes", "on")
|
||||
return bool(value)
|
||||
|
||||
|
||||
def _convert_logging_format_to_loguru(user_format: str) -> str:
|
||||
"""
|
||||
将标准 logging 格式转换为 loguru 格式
|
||||
|
||||
Args:
|
||||
user_format: 用户提供的日志格式字符串
|
||||
|
||||
Returns:
|
||||
转换后的 loguru 格式字符串
|
||||
"""
|
||||
format_mapping = {
|
||||
"%(asctime)s": "{time:YYYY-MM-DD HH:mm:ss}",
|
||||
"%(name)s": "{name}",
|
||||
"%(levelname)s": "{level: <8}",
|
||||
"%(message)s": "{message}",
|
||||
"%(filename)s": "{file.name}",
|
||||
"%(funcName)s": "{function}",
|
||||
"%(lineno)d": "{line}",
|
||||
}
|
||||
|
||||
result = user_format
|
||||
for old, new in format_mapping.items():
|
||||
result = result.replace(old, new)
|
||||
return result
|
||||
|
||||
|
||||
def _build_json_handler_kwargs(log_level: str) -> tuple[dict, dict]:
|
||||
"""
|
||||
构建 JSON 格式的 handler 参数
|
||||
|
||||
Returns:
|
||||
(console_kwargs, file_kwargs) 元组
|
||||
"""
|
||||
console_kwargs = {
|
||||
"sink": sys.stdout,
|
||||
"serialize": True,
|
||||
"level": log_level,
|
||||
"backtrace": True,
|
||||
"diagnose": True,
|
||||
}
|
||||
file_kwargs = {
|
||||
"serialize": True,
|
||||
"level": log_level,
|
||||
"rotation": "10 MB",
|
||||
"retention": "7 days",
|
||||
"compression": "zip",
|
||||
"backtrace": True,
|
||||
"diagnose": True,
|
||||
}
|
||||
return console_kwargs, file_kwargs
|
||||
|
||||
|
||||
def _build_text_handler_kwargs(log_format: str, log_level: str) -> tuple[dict, dict]:
|
||||
"""
|
||||
构建文本格式的 handler 参数
|
||||
|
||||
Returns:
|
||||
(console_kwargs, file_kwargs) 元组
|
||||
"""
|
||||
console_kwargs = {
|
||||
"sink": sys.stdout,
|
||||
"format": log_format,
|
||||
"level": log_level,
|
||||
"colorize": True,
|
||||
"backtrace": True,
|
||||
"diagnose": True,
|
||||
}
|
||||
file_kwargs = {
|
||||
"format": log_format,
|
||||
"level": log_level,
|
||||
"rotation": "10 MB",
|
||||
"retention": "7 days",
|
||||
"compression": "zip",
|
||||
"backtrace": True,
|
||||
"diagnose": True,
|
||||
}
|
||||
return console_kwargs, file_kwargs
|
||||
|
||||
|
||||
def _get_third_party_config(config_obj) -> dict:
|
||||
"""获取第三方库日志配置"""
|
||||
try:
|
||||
return config_obj.logging.third_party
|
||||
except (AttributeError, KeyError):
|
||||
pass
|
||||
|
||||
try:
|
||||
return config_obj.get("logging.third_party", {})
|
||||
except (AttributeError, KeyError):
|
||||
return {}
|
||||
|
||||
|
||||
def _configure_third_party_loggers(third_party_config: dict) -> None:
|
||||
"""配置第三方库的日志级别"""
|
||||
if not isinstance(third_party_config, dict):
|
||||
return
|
||||
|
||||
for logger_name, level_name in third_party_config.items():
|
||||
if isinstance(level_name, str):
|
||||
std_logger = logging.getLogger(logger_name)
|
||||
level = getattr(logging, level_name.upper(), logging.INFO)
|
||||
std_logger.setLevel(level)
|
||||
|
||||
|
||||
def _add_file_handler(log_file: str, file_kwargs: dict) -> None:
|
||||
"""添加文件日志 handler"""
|
||||
log_dir = os.path.dirname(log_file)
|
||||
if log_dir:
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
loguru_logger.add(log_file, **file_kwargs)
|
||||
|
||||
|
||||
def setup_logging(config: Optional[Union[str, Dynaconf]] = None, enable_worker_info: bool = True) -> None:
|
||||
"""
|
||||
根据配置文件或配置对象初始化 loguru 日志系统
|
||||
|
||||
Args:
|
||||
config: 配置文件路径或配置对象（Dynaconf），如果为 None 则使用默认配置
|
||||
enable_worker_info: 是否启用 worker 信息显示（多 worker 模式下自动启用）
|
||||
"""
|
||||
# 获取配置对象
|
||||
config_obj = config if isinstance(config, Dynaconf) else get_settings(config)
|
||||
|
||||
# 移除默认的 handler 并配置 worker 上下文
|
||||
loguru_logger.remove()
|
||||
loguru_logger.configure(extra={"worker": _get_worker_info()})
|
||||
|
||||
# 获取日志级别
|
||||
log_level = config_obj.get("logging.level", "INFO").upper()
|
||||
|
||||
# 构建 handler 参数
|
||||
use_json = _parse_json_config(config_obj.get("logging.json", False))
|
||||
if use_json:
|
||||
console_kwargs, file_kwargs = _build_json_handler_kwargs(log_level)
|
||||
else:
|
||||
# 确定日志格式
|
||||
user_format = config_obj.get("logging.format", None)
|
||||
if user_format:
|
||||
log_format = _convert_logging_format_to_loguru(user_format)
|
||||
else:
|
||||
worker_count = os.environ.get("MYBOOT_WORKER_COUNT")
|
||||
is_multi_worker = worker_count and int(worker_count) > 1
|
||||
log_format = _get_log_format_with_worker() if (is_multi_worker and enable_worker_info) else _get_log_format_simple()
|
||||
|
||||
console_kwargs, file_kwargs = _build_text_handler_kwargs(log_format, log_level)
|
||||
|
||||
# 添加 handlers
|
||||
loguru_logger.add(**console_kwargs)
|
||||
|
||||
log_file = config_obj.get("logging.file")
|
||||
if log_file:
|
||||
_add_file_handler(log_file, file_kwargs)
|
||||
|
||||
# 配置第三方库日志
|
||||
_configure_third_party_loggers(_get_third_party_config(config_obj))
|
||||
|
||||
|
||||
def configure_worker_logger(worker_id: int, total_workers: int) -> None:
|
||||
"""
|
||||
配置 worker 进程的日志上下文
|
||||
|
||||
在多 worker 模式下，每个 worker 进程启动时调用此函数，
|
||||
以便在日志中显示 worker 标识
|
||||
|
||||
Args:
|
||||
worker_id: Worker 进程 ID (从 1 开始)
|
||||
total_workers: 总 worker 数量
|
||||
"""
|
||||
worker_info = f"Worker-{worker_id}/{total_workers}"
|
||||
loguru_logger.configure(extra={"worker": worker_info})
|
||||
|
||||
|
||||
def setup_worker_logging(worker_id: int, total_workers: int, config: Optional[Union[str, Dynaconf]] = None) -> None:
|
||||
"""
|
||||
为 worker 进程重新初始化日志系统
|
||||
|
||||
在多 worker 模式下，每个 worker 进程启动后调用此函数，
|
||||
重新配置日志格式以显示 worker 标识
|
||||
|
||||
Args:
|
||||
worker_id: Worker 进程 ID (从 1 开始)
|
||||
total_workers: 总 worker 数量
|
||||
config: 配置文件路径或配置对象（Dynaconf），如果为 None 则使用默认配置
|
||||
"""
|
||||
# 获取配置对象
|
||||
config_obj = config if isinstance(config, Dynaconf) else get_settings(config)
|
||||
|
||||
# 获取日志级别
|
||||
log_level = config_obj.get("logging.level", "INFO").upper()
|
||||
|
||||
# 移除现有的 handler
|
||||
loguru_logger.remove()
|
||||
|
||||
# 设置 worker 上下文
|
||||
worker_info = f"Worker-{worker_id}/{total_workers}"
|
||||
loguru_logger.configure(extra={"worker": worker_info})
|
||||
|
||||
# 使用带 worker 标识的日志格式
|
||||
log_format = _get_log_format_with_worker()
|
||||
|
||||
# 添加控制台输出 handler
|
||||
loguru_logger.add(
|
||||
sys.stdout,
|
||||
format=log_format,
|
||||
level=log_level,
|
||||
colorize=True,
|
||||
backtrace=True,
|
||||
diagnose=True,
|
||||
)
|
||||
|
||||
|
||||
# 为了向后兼容，提供 get_logger 函数
|
||||
# 但实际上直接使用 loguru 的 logger 即可
|
||||
def get_logger(name: str = "app"):
|
||||
"""
|
||||
获取日志器实例（向后兼容）
|
||||
|
||||
注意：建议直接使用 `from loguru import logger`
|
||||
|
||||
Args:
|
||||
name: 日志器名称（loguru 中用于标识，可通过 bind 方法绑定）
|
||||
|
||||
Returns:
|
||||
loguru Logger 实例
|
||||
"""
|
||||
return loguru_logger.bind(name=name)
|
||||
|
||||
|
||||
# issue #12: % 风格占位符检测（%s/%d/%r/%f/%x 等；%% 为转义，单独匹配以便排除）
|
||||
_PERCENT_PLACEHOLDER_PATTERN = re.compile(
|
||||
r"%(?:%|[-+ #0]*(?:\d+|\*)?(?:\.(?:\d+|\*))?[hlL]?[diouxXeEfFgGcrsa])"
|
||||
)
|
||||
|
||||
# issue #12: {} 风格占位符检测（完整的 {...} 对；{{ }} 转义在检测前剔除）
|
||||
_BRACE_PLACEHOLDER_PATTERN = re.compile(r"\{[^{}]*\}")
|
||||
|
||||
|
||||
# 为了向后兼容，提供 Logger 类
|
||||
class Logger:
|
||||
"""
|
||||
日志器类（向后兼容）
|
||||
|
||||
注意：建议直接使用 `from loguru import logger`
|
||||
|
||||
issue #12: 同时支持 logging 风格的 % 占位符（"hello %s"）和
|
||||
loguru 原生的 {} 占位符（"hello {}"）。日志调用绝不向业务代码
|
||||
抛出格式化异常，格式化失败时降级输出消息本体。
|
||||
"""
|
||||
|
||||
def __init__(self, name: str = "app"):
|
||||
"""
|
||||
初始化日志器
|
||||
|
||||
Args:
|
||||
name: 日志器名称
|
||||
"""
|
||||
self.name = name
|
||||
self._logger = loguru_logger.bind(name=name)
|
||||
|
||||
@staticmethod
|
||||
def _contains_percent_placeholder(message: str) -> bool:
|
||||
"""消息是否含真正的 % 占位符（%% 转义不算）"""
|
||||
return any(
|
||||
match.group() != "%%"
|
||||
for match in _PERCENT_PLACEHOLDER_PATTERN.finditer(message)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _contains_brace_placeholder(message: str) -> bool:
|
||||
"""消息是否含 {} 风格占位符（{{ }} 转义不算）"""
|
||||
unescaped = message.replace("{{", "").replace("}}", "")
|
||||
return _BRACE_PLACEHOLDER_PATTERN.search(unescaped) is not None
|
||||
|
||||
def _format_message(self, message, args):
|
||||
"""
|
||||
issue #12: 统一占位符预处理
|
||||
|
||||
规则：
|
||||
1. 含 % 占位符且有位置参数且不含 {} 占位符 → 先做 % 预格式化，
|
||||
args 置空后交给 loguru
|
||||
2. 含 {} 占位符 → 原样交给 loguru（{} 优先于 %，保持 loguru 原生语义）
|
||||
3. 预格式化异常（参数不匹配等）→ 回退为原样传递，不抛错
|
||||
|
||||
Returns:
|
||||
(message, args) 元组，供 loguru 调用使用
|
||||
"""
|
||||
if not args or not isinstance(message, str):
|
||||
return message, args
|
||||
if self._contains_brace_placeholder(message):
|
||||
# {} 占位符优先，交给 loguru 内部的 str.format
|
||||
return message, args
|
||||
if self._contains_percent_placeholder(message):
|
||||
try:
|
||||
return message % args, ()
|
||||
except Exception:
|
||||
# 参数不匹配等预格式化失败：回退为原样传递
|
||||
return message, args
|
||||
return message, args
|
||||
|
||||
def _safe_log(self, log_method, message, args, kwargs) -> None:
|
||||
"""
|
||||
issue #12: 安全调用 loguru 日志方法
|
||||
|
||||
loguru 内部的 str.format 路径若抛错（如消息含孤立 {），
|
||||
捕获后降级为无参输出消息本体，绝不向业务代码抛格式化异常。
|
||||
kwargs（命名 {name} 占位符 / extra）保持透传。
|
||||
"""
|
||||
formatted, remaining_args = self._format_message(message, args)
|
||||
try:
|
||||
log_method(formatted, *remaining_args, **kwargs)
|
||||
except Exception:
|
||||
try:
|
||||
log_method("{}", formatted)
|
||||
except Exception:
|
||||
# 日志降级输出也失败时静默放弃，保证业务调用不受影响
|
||||
pass
|
||||
|
||||
def debug(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录调试日志"""
|
||||
self._safe_log(self._logger.debug, message, args, kwargs)
|
||||
|
||||
def info(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录信息日志"""
|
||||
self._safe_log(self._logger.info, message, args, kwargs)
|
||||
|
||||
def warning(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录警告日志"""
|
||||
self._safe_log(self._logger.warning, message, args, kwargs)
|
||||
|
||||
def error(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录错误日志"""
|
||||
self._safe_log(self._logger.error, message, args, kwargs)
|
||||
|
||||
def critical(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录严重错误日志"""
|
||||
self._safe_log(self._logger.critical, message, args, kwargs)
|
||||
|
||||
def exception(self, message: str, *args, **kwargs) -> None:
|
||||
"""记录异常日志"""
|
||||
self._safe_log(self._logger.exception, message, args, kwargs)
|
||||
@@ -0,0 +1,627 @@
|
||||
"""
|
||||
任务调度器模块
|
||||
|
||||
提供定时任务调度功能，基于 APScheduler 实现
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Optional, Callable, Any, TYPE_CHECKING, Union
|
||||
|
||||
from loguru import logger
|
||||
from dynaconf import Dynaconf
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
from apscheduler.triggers.date import DateTrigger
|
||||
from apscheduler.executors.pool import ThreadPoolExecutor
|
||||
from apscheduler.jobstores.memory import MemoryJobStore
|
||||
|
||||
from .config import get_config, get_settings
|
||||
from ..exceptions import SchedulerError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..jobs.scheduled_job import ScheduledJob
|
||||
|
||||
|
||||
class Scheduler:
|
||||
"""应用任务调度器（基于 APScheduler）"""
|
||||
|
||||
def __init__(self, config: Optional[Union[str, Dynaconf]] = None, enabled: Optional[bool] = None):
|
||||
"""
|
||||
初始化调度器
|
||||
|
||||
Args:
|
||||
config: 配置文件路径或配置对象（Dynaconf），如果为 None 则使用默认配置
|
||||
enabled: 是否启用调度器，如果为 None 则从配置文件读取
|
||||
多 workers 模式下，默认只在 primary worker 启用
|
||||
"""
|
||||
self._logger = logger.bind(name="scheduler")
|
||||
self._scheduled_jobs = {} # 存储 ScheduledJob 对象
|
||||
|
||||
# 获取配置对象
|
||||
if isinstance(config, Dynaconf):
|
||||
# 如果传入的是配置对象，直接使用
|
||||
self._config = config
|
||||
else:
|
||||
# 如果传入的是配置文件路径或 None，则获取配置对象
|
||||
self._config = get_settings(config)
|
||||
|
||||
# 从配置对象读取调度器配置，enabled 参数可覆盖配置
|
||||
config_enabled = self._config.get('scheduler.enabled', True)
|
||||
self._enabled = enabled if enabled is not None else config_enabled
|
||||
timezone_str = self._config.get('scheduler.timezone', 'UTC')
|
||||
self._timezone = self._parse_timezone(timezone_str)
|
||||
max_workers = self._config.get('scheduler.max_workers', 10)
|
||||
|
||||
# 配置 APScheduler
|
||||
jobstores = {
|
||||
'default': MemoryJobStore()
|
||||
}
|
||||
executors = {
|
||||
'default': ThreadPoolExecutor(max_workers=max_workers)
|
||||
}
|
||||
job_defaults = {
|
||||
'coalesce': False, # 不合并错过的任务
|
||||
'max_instances': 3, # 最大并发实例数
|
||||
'misfire_grace_time': 30 # 错过执行的宽限时间（秒）
|
||||
}
|
||||
|
||||
# 创建 APScheduler 实例
|
||||
self._scheduler = BackgroundScheduler(
|
||||
jobstores=jobstores,
|
||||
executors=executors,
|
||||
job_defaults=job_defaults,
|
||||
timezone=self._timezone
|
||||
)
|
||||
|
||||
self._logger.debug(f"调度器初始化完成 - enabled: {self._enabled}, timezone: {timezone_str}, max_workers: {max_workers}")
|
||||
|
||||
def _parse_timezone(self, timezone_str: str):
|
||||
"""
|
||||
解析时区字符串
|
||||
|
||||
APScheduler 会自动使用此时区处理所有任务，无需手动转换
|
||||
"""
|
||||
try:
|
||||
import pytz
|
||||
return pytz.timezone(timezone_str)
|
||||
except ImportError:
|
||||
self._logger.warning("未安装 pytz，使用系统时区")
|
||||
return None
|
||||
except Exception as e:
|
||||
self._logger.warning(f"解析时区失败: {e}，使用系统时区")
|
||||
return None
|
||||
|
||||
def _resolve_job_id(self, job_id: Optional[str], prefix: str, func: Callable) -> str:
|
||||
"""解析并校验任务 ID
|
||||
|
||||
默认 ID 为 ``{prefix}_{模块名}.{限定名}``（限定名含类名），
|
||||
不同类中的同名方法不会冲突（issue #14）。
|
||||
|
||||
注册前显式查重——APScheduler 未启动时不校验 pending 队列中的
|
||||
ID 冲突，问题会延迟到 ``start()`` 落库时才爆发，这里提前拦截：
|
||||
- 显式传入的 ID 已存在：抛 ``SchedulerError``
|
||||
- 自动生成的 ID 已存在（同一函数注册多次）：追加短 uid 消歧并告警
|
||||
"""
|
||||
existing_ids = set(self.list_jobs())
|
||||
|
||||
if job_id is not None:
|
||||
if job_id in existing_ids:
|
||||
raise SchedulerError(
|
||||
f"任务 ID 已存在: {job_id}，请使用唯一的 job_id",
|
||||
scheduler="scheduler",
|
||||
)
|
||||
return job_id
|
||||
|
||||
module = getattr(func, '__module__', 'unknown')
|
||||
qualname = getattr(func, '__qualname__', getattr(func, '__name__', str(func)))
|
||||
auto_id = f"{prefix}_{module}.{qualname}"
|
||||
if auto_id in existing_ids:
|
||||
suffix = uuid.uuid4().hex[:8]
|
||||
self._logger.warning(
|
||||
f"自动生成的任务 ID 冲突: {auto_id}（同一函数注册多次），"
|
||||
f"追加后缀消歧: {auto_id}_{suffix}"
|
||||
)
|
||||
auto_id = f"{auto_id}_{suffix}"
|
||||
return auto_id
|
||||
|
||||
def add_cron_job(
|
||||
self,
|
||||
func: Callable,
|
||||
cron: str,
|
||||
job_id: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> str:
|
||||
"""
|
||||
添加 Cron 任务
|
||||
|
||||
Args:
|
||||
func: 要执行的函数
|
||||
cron: Cron 表达式（标准5位格式：分 时 日 月 周，或使用 CronTrigger 支持的格式）
|
||||
job_id: 任务ID，如果为 None 则自动生成
|
||||
**kwargs: 其他任务参数
|
||||
|
||||
Returns:
|
||||
str: 任务ID
|
||||
"""
|
||||
job_id = self._resolve_job_id(job_id, "cron", func)
|
||||
|
||||
try:
|
||||
# 解析 cron 表达式
|
||||
trigger = self._parse_cron(cron)
|
||||
|
||||
# 添加到 APScheduler
|
||||
self._scheduler.add_job(
|
||||
func=func,
|
||||
trigger=trigger,
|
||||
id=job_id,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
self._logger.info(f"已添加 Cron 任务: {job_id} - {cron}")
|
||||
return job_id
|
||||
|
||||
except Exception as e:
|
||||
self._logger.error(f"添加 Cron 任务失败: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
def _parse_cron(self, cron_expr: str) -> CronTrigger:
|
||||
"""
|
||||
解析 Cron 表达式
|
||||
|
||||
Args:
|
||||
cron_expr: Cron 表达式字符串
|
||||
|
||||
Returns:
|
||||
CronTrigger 对象（使用调度器配置的时区）
|
||||
"""
|
||||
# 尝试使用 CronTrigger.from_crontab 解析标准格式
|
||||
try:
|
||||
return CronTrigger.from_crontab(cron_expr, timezone=self._timezone)
|
||||
except (ValueError, TypeError):
|
||||
# 如果不是标准格式，尝试手动解析
|
||||
parts = cron_expr.split()
|
||||
if len(parts) == 5:
|
||||
# 标准5位格式：分 时 日 月 周
|
||||
minute, hour, day, month, day_of_week = parts
|
||||
return CronTrigger(
|
||||
minute=minute,
|
||||
hour=hour,
|
||||
day=day,
|
||||
month=month,
|
||||
day_of_week=day_of_week,
|
||||
timezone=self._timezone
|
||||
)
|
||||
elif len(parts) == 6:
|
||||
# 6位格式：秒 分 时 日 月 周（兼容旧格式）
|
||||
second, minute, hour, day, month, day_of_week = parts
|
||||
return CronTrigger(
|
||||
second=second,
|
||||
minute=minute,
|
||||
hour=hour,
|
||||
day=day,
|
||||
month=month,
|
||||
day_of_week=day_of_week,
|
||||
timezone=self._timezone
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"无效的 Cron 表达式格式: {cron_expr}，应为5位或6位")
|
||||
|
||||
def add_interval_job(
|
||||
self,
|
||||
func: Callable,
|
||||
interval: int,
|
||||
job_id: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> str:
|
||||
"""
|
||||
添加间隔任务
|
||||
|
||||
Args:
|
||||
func: 要执行的函数
|
||||
interval: 间隔时间（秒）
|
||||
job_id: 任务ID，如果为 None 则自动生成
|
||||
**kwargs: 其他任务参数
|
||||
|
||||
Returns:
|
||||
str: 任务ID
|
||||
"""
|
||||
job_id = self._resolve_job_id(job_id, "interval", func)
|
||||
|
||||
try:
|
||||
# 创建间隔触发器（使用调度器配置的时区）
|
||||
trigger = IntervalTrigger(seconds=interval, timezone=self._timezone)
|
||||
|
||||
# 添加到 APScheduler
|
||||
self._scheduler.add_job(
|
||||
func=func,
|
||||
trigger=trigger,
|
||||
id=job_id,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
self._logger.info(f"已添加间隔任务: {job_id} - {interval}秒")
|
||||
return job_id
|
||||
|
||||
except Exception as e:
|
||||
self._logger.error(f"添加间隔任务失败: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
def add_date_job(
|
||||
self,
|
||||
func: Callable,
|
||||
run_date: str,
|
||||
job_id: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> str:
|
||||
"""
|
||||
添加一次性任务
|
||||
|
||||
Args:
|
||||
func: 要执行的函数
|
||||
run_date: 运行日期时间字符串，格式: 'YYYY-MM-DD HH:MM:SS' 或 'YYYY-MM-DD HH:MM'
|
||||
job_id: 任务ID，如果为 None 则自动生成
|
||||
**kwargs: 其他任务参数
|
||||
|
||||
Returns:
|
||||
str: 任务ID
|
||||
"""
|
||||
job_id = self._resolve_job_id(job_id, "date", func)
|
||||
|
||||
try:
|
||||
# 解析日期时间（返回 naive datetime）
|
||||
run_datetime = self._parse_run_date(run_date)
|
||||
|
||||
# 创建日期触发器（使用调度器配置的时区）
|
||||
trigger = DateTrigger(run_date=run_datetime, timezone=self._timezone)
|
||||
|
||||
# 添加到 APScheduler
|
||||
self._scheduler.add_job(
|
||||
func=func,
|
||||
trigger=trigger,
|
||||
id=job_id,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
self._logger.info(f"已添加一次性任务: {job_id} - {run_date}")
|
||||
return job_id
|
||||
|
||||
except Exception as e:
|
||||
self._logger.error(f"添加一次性任务失败: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
def _parse_run_date(self, run_date: str) -> datetime:
|
||||
"""
|
||||
解析运行日期时间字符串
|
||||
|
||||
Args:
|
||||
run_date: 日期时间字符串
|
||||
|
||||
Returns:
|
||||
naive datetime 对象（APScheduler 会自动应用调度器的全局时区）
|
||||
"""
|
||||
# 尝试解析不同的日期格式
|
||||
formats = [
|
||||
'%Y-%m-%d %H:%M:%S',
|
||||
'%Y-%m-%d %H:%M',
|
||||
'%Y-%m-%d',
|
||||
]
|
||||
|
||||
for fmt in formats:
|
||||
try:
|
||||
# 直接返回 naive datetime，APScheduler 会自动应用全局时区
|
||||
return datetime.strptime(run_date, fmt)
|
||||
except ValueError:
|
||||
continue
|
||||
|
||||
raise ValueError(f"无效的日期格式: {run_date}，支持的格式: 'YYYY-MM-DD HH:MM:SS', 'YYYY-MM-DD HH:MM', 'YYYY-MM-DD'")
|
||||
|
||||
def remove_job(self, job_id: str) -> bool:
|
||||
"""
|
||||
移除任务
|
||||
|
||||
Args:
|
||||
job_id: 任务ID
|
||||
|
||||
Returns:
|
||||
bool: 是否成功移除
|
||||
"""
|
||||
try:
|
||||
if self._scheduler.get_job(job_id):
|
||||
self._scheduler.remove_job(job_id)
|
||||
self._logger.info(f"已移除任务: {job_id}")
|
||||
return True
|
||||
return False
|
||||
except Exception as e:
|
||||
self._logger.error(f"移除任务失败: {e}", exc_info=True)
|
||||
return False
|
||||
|
||||
def get_job(self, job_id: str) -> Optional[Any]:
|
||||
"""
|
||||
获取任务
|
||||
|
||||
Args:
|
||||
job_id: 任务ID
|
||||
|
||||
Returns:
|
||||
APScheduler Job 对象，如果不存在则返回 None
|
||||
"""
|
||||
return self._scheduler.get_job(job_id)
|
||||
|
||||
def list_jobs(self) -> list:
|
||||
"""
|
||||
列出所有任务ID
|
||||
|
||||
Returns:
|
||||
list: 任务ID列表
|
||||
"""
|
||||
return [job.id for job in self._scheduler.get_jobs()]
|
||||
|
||||
def start(self) -> None:
|
||||
"""启动调度器"""
|
||||
# 检查是否启用
|
||||
if not self._enabled:
|
||||
self._logger.info("调度器已禁用，不启动")
|
||||
return
|
||||
|
||||
if not self._scheduler.running:
|
||||
try:
|
||||
self._scheduler.start()
|
||||
self._logger.info(f"任务调度器已启动 (时区: {self._timezone}, 任务数: {len(self._scheduler.get_jobs())})")
|
||||
except Exception as e:
|
||||
self._logger.error(f"启动调度器失败: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
def stop(self) -> None:
|
||||
"""停止调度器"""
|
||||
if self._scheduler.running:
|
||||
try:
|
||||
self._scheduler.shutdown(wait=True)
|
||||
self._logger.info("任务调度器已停止")
|
||||
except Exception as e:
|
||||
self._logger.error(f"停止调度器失败: {e}", exc_info=True)
|
||||
|
||||
def is_running(self) -> bool:
|
||||
"""检查调度器是否运行中"""
|
||||
return self._scheduler.running
|
||||
|
||||
def has_jobs(self) -> bool:
|
||||
"""检查是否有任务"""
|
||||
return len(self._scheduler.get_jobs()) > 0
|
||||
|
||||
def get_job_info(self, job_id: str) -> Optional[dict]:
|
||||
"""
|
||||
获取任务信息
|
||||
|
||||
Args:
|
||||
job_id: 任务ID
|
||||
|
||||
Returns:
|
||||
dict: 任务信息字典
|
||||
"""
|
||||
job = self._scheduler.get_job(job_id)
|
||||
if not job:
|
||||
return None
|
||||
|
||||
# 未启动调度器时 pending Job 对象没有 next_run_time 属性，用 getattr 防御
|
||||
next_run_time = getattr(job, 'next_run_time', None)
|
||||
info = {
|
||||
'job_id': job_id,
|
||||
'func_name': job.func.__name__ if hasattr(job.func, '__name__') else str(job.func),
|
||||
'next_run_time': next_run_time.isoformat() if next_run_time else None,
|
||||
}
|
||||
|
||||
# 根据触发器类型添加特定信息
|
||||
trigger = job.trigger
|
||||
if isinstance(trigger, CronTrigger):
|
||||
info['type'] = 'cron'
|
||||
info['cron'] = str(trigger)
|
||||
elif isinstance(trigger, IntervalTrigger):
|
||||
info['type'] = 'interval'
|
||||
info['interval'] = trigger.interval.total_seconds()
|
||||
elif isinstance(trigger, DateTrigger):
|
||||
info['type'] = 'date'
|
||||
info['run_date'] = trigger.run_date.isoformat() if trigger.run_date else None
|
||||
|
||||
return info
|
||||
|
||||
def list_all_jobs(self) -> list:
|
||||
"""
|
||||
列出所有任务的详细信息
|
||||
|
||||
Returns:
|
||||
list: 任务信息列表
|
||||
"""
|
||||
return [self.get_job_info(job.id) for job in self._scheduler.get_jobs()]
|
||||
|
||||
def enable(self) -> None:
|
||||
"""启用调度器
|
||||
|
||||
任务级 all_workers 支持：非 primary worker 默认调度器禁用，
|
||||
当注册门控放行了 all_workers=True 的任务时调用本方法启用调度器，
|
||||
使其能在 lifespan 中真正启动。
|
||||
"""
|
||||
if not self._enabled:
|
||||
self._enabled = True
|
||||
self._logger.debug("调度器已启用（all_workers 任务触发）")
|
||||
|
||||
def is_enabled(self) -> bool:
|
||||
"""检查调度器是否启用"""
|
||||
return self._enabled
|
||||
|
||||
def get_config(self) -> dict:
|
||||
"""获取调度器配置"""
|
||||
return {
|
||||
'enabled': self._enabled,
|
||||
'timezone': str(self._timezone) if self._timezone else 'system',
|
||||
'running': self._scheduler.running,
|
||||
'job_count': len(self._scheduler.get_jobs()),
|
||||
'scheduled_job_count': len(self._scheduled_jobs)
|
||||
}
|
||||
|
||||
def add_scheduled_job(
|
||||
self,
|
||||
job: 'ScheduledJob',
|
||||
job_id: Optional[str] = None
|
||||
) -> str:
|
||||
"""
|
||||
添加 ScheduledJob 对象到调度器
|
||||
|
||||
Args:
|
||||
job: ScheduledJob 对象
|
||||
job_id: 任务ID，如果为 None 则自动生成
|
||||
|
||||
Returns:
|
||||
str: 任务ID
|
||||
"""
|
||||
from ..jobs.scheduled_job import ScheduledJob as ScheduledJobClass
|
||||
|
||||
if not isinstance(job, ScheduledJobClass):
|
||||
raise TypeError(f"job 必须是 ScheduledJob 的实例，当前类型: {type(job)}")
|
||||
|
||||
if job.trigger is None:
|
||||
raise ValueError("ScheduledJob 必须设置 trigger 属性")
|
||||
|
||||
# 生成任务ID
|
||||
if job_id is None:
|
||||
job_id = f"scheduled_{job.name}_{id(job)}"
|
||||
|
||||
# 保存 ScheduledJob 对象
|
||||
self._scheduled_jobs[job_id] = job
|
||||
job.job_id = job_id
|
||||
|
||||
# 转换触发器格式并添加到调度器
|
||||
trigger = job.trigger
|
||||
|
||||
if isinstance(trigger, str):
|
||||
# 字符串视为 cron 表达式
|
||||
self.add_cron_job(
|
||||
func=job.execute,
|
||||
cron=trigger,
|
||||
job_id=job_id,
|
||||
name=job.name
|
||||
)
|
||||
elif isinstance(trigger, dict):
|
||||
trigger_type = trigger.get('type')
|
||||
if trigger_type == 'cron':
|
||||
self.add_cron_job(
|
||||
func=job.execute,
|
||||
cron=trigger['cron'],
|
||||
job_id=job_id,
|
||||
name=job.name
|
||||
)
|
||||
elif trigger_type == 'interval':
|
||||
interval = trigger.get('seconds', 0) or 0
|
||||
interval += (trigger.get('minutes', 0) or 0) * 60
|
||||
interval += (trigger.get('hours', 0) or 0) * 3600
|
||||
interval += (trigger.get('days', 0) or 0) * 86400
|
||||
if interval <= 0:
|
||||
raise ValueError("间隔时间必须大于 0")
|
||||
self.add_interval_job(
|
||||
func=job.execute,
|
||||
interval=interval,
|
||||
job_id=job_id,
|
||||
name=job.name
|
||||
)
|
||||
elif trigger_type == 'date':
|
||||
self.add_date_job(
|
||||
func=job.execute,
|
||||
run_date=trigger['run_date'],
|
||||
job_id=job_id,
|
||||
name=job.name
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"不支持的触发器类型: {trigger_type}")
|
||||
else:
|
||||
raise ValueError(f"不支持的触发器类型: {type(trigger)}")
|
||||
|
||||
self._logger.info(f"已添加 ScheduledJob: {job.name} (ID: {job_id})")
|
||||
return job_id
|
||||
|
||||
def get_scheduled_job(self, job_id: str) -> Optional['ScheduledJob']:
|
||||
"""
|
||||
获取 ScheduledJob 对象
|
||||
|
||||
Args:
|
||||
job_id: 任务ID
|
||||
|
||||
Returns:
|
||||
ScheduledJob 对象，如果不存在则返回 None
|
||||
"""
|
||||
return self._scheduled_jobs.get(job_id)
|
||||
|
||||
def get_all_scheduled_jobs(self) -> list:
|
||||
"""
|
||||
获取所有 ScheduledJob 对象
|
||||
|
||||
Returns:
|
||||
ScheduledJob 对象列表
|
||||
"""
|
||||
return list(self._scheduled_jobs.values())
|
||||
|
||||
def add_job_object(
|
||||
self,
|
||||
job: 'ScheduledJob',
|
||||
job_id: Optional[str] = None
|
||||
) -> str:
|
||||
"""
|
||||
添加 ScheduledJob 对象到调度器（不设置触发器，用于非定时任务）
|
||||
|
||||
此方法用于添加不需要定时执行的任务，仅用于状态跟踪和管理。
|
||||
如果需要定时执行，请使用 add_scheduled_job() 并设置 trigger。
|
||||
|
||||
Args:
|
||||
job: ScheduledJob 对象
|
||||
job_id: 任务ID，如果为 None 则自动生成
|
||||
|
||||
Returns:
|
||||
str: 任务ID
|
||||
"""
|
||||
from ..jobs.scheduled_job import ScheduledJob as ScheduledJobClass
|
||||
|
||||
if not isinstance(job, ScheduledJobClass):
|
||||
raise TypeError(f"job 必须是 ScheduledJob 的实例，当前类型: {type(job)}")
|
||||
|
||||
# 生成任务ID
|
||||
if job_id is None:
|
||||
job_id = f"job_{job.name}_{id(job)}"
|
||||
|
||||
# 保存 ScheduledJob 对象
|
||||
self._scheduled_jobs[job_id] = job
|
||||
job.job_id = job_id
|
||||
|
||||
self._logger.info(f"已添加任务对象: {job.name} (ID: {job_id})")
|
||||
return job_id
|
||||
|
||||
def remove_scheduled_job(self, job_id: str) -> bool:
|
||||
"""
|
||||
移除 ScheduledJob
|
||||
|
||||
Args:
|
||||
job_id: 任务ID
|
||||
|
||||
Returns:
|
||||
是否成功移除
|
||||
"""
|
||||
if job_id in self._scheduled_jobs:
|
||||
# 同时从调度器中移除（如果存在）
|
||||
self.remove_job(job_id)
|
||||
del self._scheduled_jobs[job_id]
|
||||
self._logger.info(f"已移除 ScheduledJob: {job_id}")
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# 全局调度器实例
|
||||
_scheduler: Optional[Scheduler] = None
|
||||
|
||||
|
||||
def get_scheduler() -> Scheduler:
|
||||
"""获取调度器实例"""
|
||||
global _scheduler
|
||||
|
||||
if _scheduler is None:
|
||||
_scheduler = Scheduler()
|
||||
|
||||
return _scheduler
|
||||
@@ -0,0 +1,451 @@
|
||||
"""
|
||||
服务器管理器
|
||||
|
||||
默认使用 Hypercorn 作为 ASGI 服务器，支持多 workers 模式
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import multiprocessing
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
import sys
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from ..utils import get_local_ip
|
||||
|
||||
# 别名映射：调用方传入的名称 → Hypercorn Config 属性名
|
||||
# application.py 白名单已处理大部分别名，此处仅保留通用兜底映射
|
||||
# 参考：https://hypercorn.readthedocs.io/en/latest/how_to_guides/configuring.html
|
||||
HYPERCORN_CONFIG_ALIASES: Dict[str, str] = {
|
||||
"reload": "use_reloader",
|
||||
"max_incomplete_request_size": "h11_max_incomplete_size",
|
||||
}
|
||||
|
||||
# 序列化 Config 到子进程时跳过：只读或由其他字段推导，子进程 asyncio.serve() 不需/不应重设
|
||||
_HYPERCORN_SERIALIZE_SKIP = frozenset({
|
||||
"ssl_enabled", # Hypercorn：只读，由 certfile/keyfile 推导
|
||||
"workers", # asyncio.serve() 不用此字段；多进程由外层 fork
|
||||
})
|
||||
|
||||
# myboot 内部消耗、不传给 Hypercorn 的键
|
||||
_MYBOOT_INTERNAL = {"host", "port", "app_path"}
|
||||
|
||||
|
||||
def _create_socket_with_reuseport(host: str, port: int, backlog: int = 100) -> socket.socket:
|
||||
"""
|
||||
创建启用 SO_REUSEPORT 的 socket（仅 Linux/macOS）
|
||||
|
||||
Args:
|
||||
host: 绑定的主机地址
|
||||
port: 绑定的端口号
|
||||
|
||||
Returns:
|
||||
已绑定并监听的 socket 对象
|
||||
"""
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
|
||||
# Linux/macOS 支持 SO_REUSEPORT
|
||||
if hasattr(socket, 'SO_REUSEPORT'):
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
|
||||
|
||||
sock.setblocking(False)
|
||||
sock.bind((host, port))
|
||||
sock.listen(backlog)
|
||||
return sock
|
||||
|
||||
|
||||
def _resolve_app_from_path(app_path: str):
|
||||
"""
|
||||
从模块路径解析 ASGI 应用
|
||||
|
||||
支持的格式:
|
||||
- "module.path:app" -> 获取 module.path 模块的 app 属性
|
||||
- "module.path:app.get_fastapi_app()" -> 获取 app 对象后调用 get_fastapi_app()
|
||||
- "module.path:create_app()" -> 调用 module.path 模块的 create_app() 函数
|
||||
|
||||
Args:
|
||||
app_path: 应用模块路径
|
||||
|
||||
Returns:
|
||||
ASGI 应用实例
|
||||
"""
|
||||
import importlib
|
||||
import re
|
||||
|
||||
# 解析模块路径和应用名
|
||||
if ":" in app_path:
|
||||
module_path, app_expr = app_path.rsplit(":", 1)
|
||||
else:
|
||||
module_path, app_expr = app_path, "app"
|
||||
|
||||
# 导入模块
|
||||
# 如果请求的是 main 模块且 __main__ 已存在，直接使用 __main__
|
||||
if module_path == "main" and "__main__" in sys.modules:
|
||||
module = sys.modules["__main__"]
|
||||
else:
|
||||
module = importlib.import_module(module_path)
|
||||
|
||||
# 解析表达式，支持链式调用如 "app.get_fastapi_app()"
|
||||
# 使用简单的解析器处理 attr 和 method() 调用
|
||||
parts = re.split(r'\.(?![^(]*\))', app_expr) # 按 . 分割，但不分割括号内的点
|
||||
|
||||
result = module
|
||||
for part in parts:
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
|
||||
# 检查是否是方法调用 (以 () 结尾)
|
||||
if part.endswith("()"):
|
||||
method_name = part[:-2]
|
||||
result = getattr(result, method_name)()
|
||||
else:
|
||||
result = getattr(result, part)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _worker_serve(
|
||||
app_path: str,
|
||||
config_dict: dict,
|
||||
worker_id: int,
|
||||
total_workers: int,
|
||||
socket_fd: Optional[int] = None
|
||||
):
|
||||
"""
|
||||
Worker 进程入口函数
|
||||
|
||||
Args:
|
||||
app_path: 应用模块路径，格式为 "module.path:app_name"
|
||||
config_dict: Hypercorn 配置字典
|
||||
worker_id: Worker 进程 ID (从 1 开始)
|
||||
total_workers: 总 worker 数量
|
||||
socket_fd: 预创建的 socket 文件描述符（Linux/macOS 多 worker 模式）
|
||||
"""
|
||||
# 设置环境变量，供 Application 读取
|
||||
os.environ["MYBOOT_WORKER_ID"] = str(worker_id)
|
||||
os.environ["MYBOOT_WORKER_COUNT"] = str(total_workers)
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "1" if worker_id == 1 else "0"
|
||||
|
||||
# 重新初始化 worker 日志（设置环境变量后才能正确检测多 worker 模式）
|
||||
from .logger import setup_worker_logging
|
||||
setup_worker_logging(worker_id, total_workers)
|
||||
|
||||
try:
|
||||
import hypercorn.asyncio
|
||||
from hypercorn.config import Config
|
||||
except ImportError:
|
||||
raise ImportError("Hypercorn 未安装，请运行: pip install hypercorn")
|
||||
|
||||
# 从模块路径加载 app
|
||||
# 注意：必须先解析——导入用户模块才会构造 Application 并设置 _current_app
|
||||
app = _resolve_app_from_path(app_path)
|
||||
|
||||
# myboot 应用检测：若用户模块构造了 Application，则在 worker 进程内完成引导
|
||||
# （重读 worker 环境变量、重建调度器、实例化 client/service/controller）。
|
||||
# spawn 与 fork 两种模式在 bootstrap_worker 中收敛，修复：
|
||||
# - fork 模式所有 worker 共享父进程预创建实例（issue #11）
|
||||
# - spawn 模式（Windows）worker 无用户路由
|
||||
# - 所有 worker 都自认 primary 导致调度器多跑
|
||||
# 普通 ASGI 应用（非 myboot）路径不受影响。
|
||||
import myboot.core.application as _myboot_application
|
||||
if _myboot_application._current_app is not None:
|
||||
app = _myboot_application._current_app.bootstrap_worker()
|
||||
|
||||
# 重建配置
|
||||
config = Config()
|
||||
for key, value in config_dict.items():
|
||||
if hasattr(config, key):
|
||||
setattr(config, key, value)
|
||||
|
||||
# 如果提供了 socket_fd，使用 fd:// 绑定方式
|
||||
if socket_fd is not None:
|
||||
config.bind = [f"fd://{socket_fd}"]
|
||||
|
||||
logger.info(f"Worker-{worker_id}/{total_workers} 启动中... (primary={worker_id == 1})")
|
||||
|
||||
# 运行服务器
|
||||
asyncio.run(hypercorn.asyncio.serve(app, config))
|
||||
|
||||
|
||||
class HypercornServer:
|
||||
"""Hypercorn 服务器，支持多 workers 模式"""
|
||||
|
||||
def __init__(self, app, host: str = "0.0.0.0", port: int = 8000, **kwargs):
|
||||
self.app = app
|
||||
self.host = host
|
||||
self.port = port
|
||||
self.kwargs = kwargs
|
||||
self._running = False
|
||||
self._workers: List[multiprocessing.Process] = []
|
||||
self._import_hypercorn()
|
||||
|
||||
def _import_hypercorn(self):
|
||||
"""动态导入 Hypercorn"""
|
||||
try:
|
||||
import hypercorn.asyncio
|
||||
from hypercorn.config import Config
|
||||
self.hypercorn = hypercorn
|
||||
self._config_class = Config
|
||||
except ImportError:
|
||||
raise ImportError("Hypercorn 未安装，请运行: pip install hypercorn")
|
||||
|
||||
def _build_config(self) -> Any:
|
||||
"""构建 Hypercorn Config：将 kwargs 通过别名映射写入 Config 属性。"""
|
||||
config = self._config_class()
|
||||
config.bind = [f"{self.host}:{self.port}"]
|
||||
for key, value in self.kwargs.items():
|
||||
if key in _MYBOOT_INTERNAL:
|
||||
continue
|
||||
hypercorn_key = HYPERCORN_CONFIG_ALIASES.get(key, key)
|
||||
if hasattr(config, hypercorn_key):
|
||||
setattr(config, hypercorn_key, value)
|
||||
return config
|
||||
|
||||
def _config_to_dict(self, config) -> dict:
|
||||
"""将 Hypercorn Config 序列化为字典，供多 worker 子进程重建使用。"""
|
||||
out = {}
|
||||
for attr in dir(config):
|
||||
if attr.startswith("_"):
|
||||
continue
|
||||
if attr in _HYPERCORN_SERIALIZE_SKIP:
|
||||
continue
|
||||
val = getattr(config, attr, None)
|
||||
if callable(val):
|
||||
continue
|
||||
if isinstance(val, (str, int, float, bool, list, dict, type(None))):
|
||||
out[attr] = val
|
||||
return out
|
||||
|
||||
def start(self, app_path: Optional[str] = None) -> None:
|
||||
"""
|
||||
启动 Hypercorn 服务器
|
||||
|
||||
Args:
|
||||
app_path: 应用模块路径（多 workers 模式必需），格式为 "module.path:app_name"
|
||||
例如: "myapp.main:app" 或 "myapp.main:create_app()"
|
||||
"""
|
||||
try:
|
||||
config = self._build_config()
|
||||
workers = self.kwargs.get('workers', 1)
|
||||
|
||||
self._running = True
|
||||
|
||||
if workers > 1:
|
||||
# 多 workers 模式
|
||||
if not app_path:
|
||||
logger.warning(
|
||||
"多 workers 模式需要提供 app_path 参数（如 'myapp.main:app'），"
|
||||
"当前回退到单进程模式"
|
||||
)
|
||||
asyncio.run(self.hypercorn.asyncio.serve(self.app, config))
|
||||
else:
|
||||
self._start_multiple_workers(app_path, config, workers)
|
||||
else:
|
||||
# 单 worker 模式：直接运行
|
||||
asyncio.run(self.hypercorn.asyncio.serve(self.app, config))
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Hypercorn 服务器启动失败: {e}")
|
||||
raise
|
||||
finally:
|
||||
self._running = False
|
||||
self._cleanup_workers()
|
||||
|
||||
def _start_multiple_workers(self, app_path: str, config, workers: int) -> None:
|
||||
"""
|
||||
启动多个 worker 进程
|
||||
|
||||
Args:
|
||||
app_path: 应用模块路径
|
||||
config: Hypercorn 配置
|
||||
workers: worker 数量
|
||||
"""
|
||||
# Windows 需要使用 spawn
|
||||
if sys.platform == 'win32':
|
||||
multiprocessing.set_start_method('spawn', force=True)
|
||||
|
||||
config_dict = self._config_to_dict(config)
|
||||
socket_fd: Optional[int] = None
|
||||
shared_socket: Optional[socket.socket] = None
|
||||
|
||||
# Linux/macOS: 使用 SO_REUSEPORT 预创建 socket，允许多个进程绑定同一端口
|
||||
backlog = config_dict.get("backlog", 100)
|
||||
if sys.platform != 'win32':
|
||||
try:
|
||||
shared_socket = _create_socket_with_reuseport(self.host, self.port, backlog=backlog)
|
||||
socket_fd = shared_socket.fileno()
|
||||
logger.info(f"已创建共享 socket (fd={socket_fd}), 启用 SO_REUSEPORT")
|
||||
except Exception as e:
|
||||
logger.warning(f"创建共享 socket 失败: {e}，回退到普通模式")
|
||||
shared_socket = None
|
||||
socket_fd = None
|
||||
|
||||
logger.info(f"启动 {workers} 个 worker 进程...")
|
||||
|
||||
# 创建并启动 worker 进程
|
||||
for i in range(workers):
|
||||
process = multiprocessing.Process(
|
||||
target=_worker_serve,
|
||||
args=(app_path, config_dict, i + 1, workers, socket_fd),
|
||||
name=f"hypercorn-worker-{i + 1}"
|
||||
)
|
||||
process.start()
|
||||
self._workers.append(process)
|
||||
logger.info(f"Worker-{i + 1}/{workers} 已启动 (PID: {process.pid})")
|
||||
|
||||
# 主进程等待所有 worker
|
||||
try:
|
||||
for process in self._workers:
|
||||
process.join()
|
||||
except KeyboardInterrupt:
|
||||
logger.info("收到中断信号，正在关闭所有 workers...")
|
||||
self._cleanup_workers()
|
||||
finally:
|
||||
# 关闭主进程中的共享 socket
|
||||
if shared_socket:
|
||||
try:
|
||||
shared_socket.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _cleanup_workers(self) -> None:
|
||||
"""清理所有 worker 进程"""
|
||||
for process in self._workers:
|
||||
if process.is_alive():
|
||||
logger.info(f"终止 Worker (PID: {process.pid})...")
|
||||
process.terminate()
|
||||
process.join(timeout=5)
|
||||
if process.is_alive():
|
||||
process.kill()
|
||||
self._workers.clear()
|
||||
|
||||
def stop(self) -> None:
|
||||
"""停止 Hypercorn 服务器"""
|
||||
self._running = False
|
||||
|
||||
@property
|
||||
def is_running(self) -> bool:
|
||||
"""检查服务器是否运行中"""
|
||||
return self._running
|
||||
|
||||
def get_url(self) -> str:
|
||||
"""获取服务器 URL"""
|
||||
# 使用真实 IP 地址显示（如果 host 是 0.0.0.0）
|
||||
display_host = get_local_ip() if self.host == "0.0.0.0" else self.host
|
||||
return f"http://{display_host}:{self.port}"
|
||||
|
||||
|
||||
class ServerManager:
|
||||
"""简化的服务器管理器"""
|
||||
|
||||
def __init__(self):
|
||||
self._current_server: Optional[HypercornServer] = None
|
||||
|
||||
def start_server(
|
||||
self,
|
||||
app,
|
||||
host: str = "0.0.0.0",
|
||||
port: int = 8000,
|
||||
app_path: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> None:
|
||||
"""
|
||||
启动 Hypercorn 服务器
|
||||
|
||||
Args:
|
||||
app: ASGI 应用
|
||||
host: 主机地址
|
||||
port: 端口号
|
||||
app_path: 应用模块路径（多 workers 模式必需），格式为 "module.path:app_name"
|
||||
例如: "myapp.main:app"
|
||||
**kwargs: 其他配置参数，包括:
|
||||
- workers: worker 进程数量，默认 1
|
||||
- reload: 是否启用热重载
|
||||
- keep_alive_timeout: keep-alive 超时时间
|
||||
- graceful_timeout: 优雅关闭超时时间
|
||||
"""
|
||||
if self._current_server and self._current_server.is_running:
|
||||
logger.warning("服务器已在运行中，请先停止当前服务器")
|
||||
return
|
||||
|
||||
self._current_server = HypercornServer(app, host, port, **kwargs)
|
||||
|
||||
# 注册信号处理器
|
||||
self._register_signal_handlers()
|
||||
|
||||
try:
|
||||
self._current_server.start(app_path=app_path)
|
||||
except KeyboardInterrupt:
|
||||
logger.info("收到中断信号，正在关闭服务器...")
|
||||
finally:
|
||||
self.stop_server()
|
||||
|
||||
def stop_server(self) -> None:
|
||||
"""停止当前服务器"""
|
||||
if self._current_server:
|
||||
self._current_server.stop()
|
||||
self._current_server = None
|
||||
|
||||
def _register_signal_handlers(self) -> None:
|
||||
"""注册信号处理器"""
|
||||
def signal_handler(signum, frame):
|
||||
logger.info(f"收到信号 {signum}，正在关闭服务器...")
|
||||
self.stop_server()
|
||||
sys.exit(0)
|
||||
|
||||
signal.signal(signal.SIGINT, signal_handler)
|
||||
signal.signal(signal.SIGTERM, signal_handler)
|
||||
|
||||
def get_server_info(self) -> Dict[str, Any]:
|
||||
"""获取服务器信息"""
|
||||
if not self._current_server:
|
||||
return {"status": "not_running"}
|
||||
|
||||
return {
|
||||
"status": "running" if self._current_server.is_running else "stopped",
|
||||
"url": self._current_server.get_url(),
|
||||
"host": self._current_server.host,
|
||||
"port": self._current_server.port,
|
||||
}
|
||||
|
||||
|
||||
# 全局服务器管理器实例
|
||||
server_manager = ServerManager()
|
||||
|
||||
|
||||
def start_server(
|
||||
app,
|
||||
host: str = "0.0.0.0",
|
||||
port: int = 8000,
|
||||
app_path: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> None:
|
||||
"""
|
||||
启动服务器的便捷函数
|
||||
|
||||
Args:
|
||||
app: ASGI 应用
|
||||
host: 主机地址
|
||||
port: 端口号
|
||||
app_path: 应用模块路径（多 workers 模式必需），格式为 "module.path:app_name"
|
||||
**kwargs: 其他配置参数
|
||||
|
||||
Example:
|
||||
# 单 worker 模式
|
||||
start_server(app, host="0.0.0.0", port=8000)
|
||||
|
||||
# 多 workers 模式（4个进程）
|
||||
start_server(
|
||||
app,
|
||||
host="0.0.0.0",
|
||||
port=8000,
|
||||
app_path="myapp.main:app",
|
||||
workers=4
|
||||
)
|
||||
"""
|
||||
server_manager.start_server(app, host, port, app_path=app_path, **kwargs)
|
||||
@@ -0,0 +1,154 @@
|
||||
"""
|
||||
MyBoot 异常模块
|
||||
|
||||
提供框架相关的异常类
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
|
||||
class MyBootException(Exception):
|
||||
"""MyBoot 框架异常基类"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "MyBoot 框架错误",
|
||||
code: str = "MYBOOT_ERROR",
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.message = message
|
||||
self.code = code
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class ConfigurationError(MyBootException):
|
||||
"""配置错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "配置错误",
|
||||
config_key: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.config_key = config_key
|
||||
super().__init__(message, "CONFIGURATION_ERROR", details)
|
||||
|
||||
|
||||
class ValidationError(MyBootException):
|
||||
"""验证错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "验证失败",
|
||||
field: Optional[str] = None,
|
||||
value: Any = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.field = field
|
||||
self.value = value
|
||||
super().__init__(message, "VALIDATION_ERROR", details)
|
||||
|
||||
|
||||
class InitializationError(MyBootException):
|
||||
"""初始化错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "初始化失败",
|
||||
component: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.component = component
|
||||
super().__init__(message, "INITIALIZATION_ERROR", details)
|
||||
|
||||
|
||||
class DependencyError(MyBootException):
|
||||
"""依赖错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "依赖错误",
|
||||
dependency: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.dependency = dependency
|
||||
super().__init__(message, "DEPENDENCY_ERROR", details)
|
||||
|
||||
|
||||
class ServiceError(MyBootException):
|
||||
"""服务错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "服务错误",
|
||||
service: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.service = service
|
||||
super().__init__(message, "SERVICE_ERROR", details)
|
||||
|
||||
|
||||
class JobError(MyBootException):
|
||||
"""任务错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "任务错误",
|
||||
job_name: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.job_name = job_name
|
||||
super().__init__(message, "JOB_ERROR", details)
|
||||
|
||||
|
||||
class SchedulerError(MyBootException):
|
||||
"""调度器错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "调度器错误",
|
||||
scheduler: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.scheduler = scheduler
|
||||
super().__init__(message, "SCHEDULER_ERROR", details)
|
||||
|
||||
|
||||
class MiddlewareError(MyBootException):
|
||||
"""中间件错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "中间件错误",
|
||||
middleware: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.middleware = middleware
|
||||
super().__init__(message, "MIDDLEWARE_ERROR", details)
|
||||
|
||||
|
||||
class RouteError(MyBootException):
|
||||
"""路由错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "路由错误",
|
||||
route: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.route = route
|
||||
super().__init__(message, "ROUTE_ERROR", details)
|
||||
|
||||
|
||||
class LifecycleError(MyBootException):
|
||||
"""生命周期错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "生命周期错误",
|
||||
phase: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.phase = phase
|
||||
super().__init__(message, "LIFECYCLE_ERROR", details)
|
||||
@@ -0,0 +1,11 @@
|
||||
"""
|
||||
定时任务模块
|
||||
|
||||
包含定时任务和后台任务
|
||||
"""
|
||||
|
||||
from .scheduled_job import ScheduledJob
|
||||
|
||||
__all__ = [
|
||||
'ScheduledJob',
|
||||
]
|
||||
@@ -0,0 +1,203 @@
|
||||
"""
|
||||
定时任务基类模块
|
||||
|
||||
提供 ScheduledJob 基类，用户可以继承此类创建自定义的定时任务
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, Optional, Union
|
||||
|
||||
from loguru import logger
|
||||
|
||||
|
||||
class ScheduledJob(ABC):
|
||||
"""
|
||||
定时任务基类
|
||||
|
||||
用户可以继承此类创建自定义的定时任务
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
name: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
trigger: Union[str, Dict[str, Any]] = None,
|
||||
max_retries: int = 3,
|
||||
retry_delay: float = 1.0,
|
||||
timeout: Optional[float] = None
|
||||
):
|
||||
"""
|
||||
初始化定时任务
|
||||
|
||||
Args:
|
||||
name: 任务名称
|
||||
description: 任务描述
|
||||
trigger: 触发器，可以是：
|
||||
- 字符串：cron 表达式（如 "0 0 * * * *"）
|
||||
- 字典：包含 type 和配置的字典
|
||||
- {'type': 'cron', 'cron': '0 0 * * * *'}
|
||||
- {'type': 'interval', 'seconds': 60}
|
||||
- {'type': 'date', 'run_date': '2024-01-01 12:00:00'}
|
||||
max_retries: 最大重试次数
|
||||
retry_delay: 重试延迟（秒）
|
||||
timeout: 超时时间（秒）
|
||||
"""
|
||||
self.name = name or self.__class__.__name__
|
||||
self.description = description or self.__doc__ or ""
|
||||
self.trigger = trigger
|
||||
self.max_retries = max_retries
|
||||
self.retry_delay = retry_delay
|
||||
self.timeout = timeout
|
||||
|
||||
# 任务状态
|
||||
self.status = "pending" # pending, running, completed, failed, cancelled
|
||||
self.created_at = datetime.now()
|
||||
self.started_at: Optional[datetime] = None
|
||||
self.completed_at: Optional[datetime] = None
|
||||
self.last_run: Optional[datetime] = None
|
||||
self.run_count = 0
|
||||
self.failure_count = 0
|
||||
self.last_error: Optional[Exception] = None
|
||||
self.job_id: Optional[str] = None
|
||||
|
||||
# 日志
|
||||
self.logger = logger.bind(name=f"scheduled_job.{self.name}")
|
||||
|
||||
@abstractmethod
|
||||
def run(self, *args, **kwargs) -> Any:
|
||||
"""
|
||||
执行任务的具体逻辑（子类必须实现）
|
||||
|
||||
Args:
|
||||
*args: 位置参数
|
||||
**kwargs: 关键字参数
|
||||
|
||||
Returns:
|
||||
Any: 任务执行结果
|
||||
"""
|
||||
pass
|
||||
|
||||
def execute(self, *args, **kwargs) -> Any:
|
||||
"""
|
||||
执行任务的主入口（包含状态管理和重试逻辑）
|
||||
|
||||
Args:
|
||||
*args: 位置参数
|
||||
**kwargs: 关键字参数
|
||||
|
||||
Returns:
|
||||
Any: 任务执行结果
|
||||
"""
|
||||
self.status = "running"
|
||||
self.started_at = datetime.now()
|
||||
self.run_count += 1
|
||||
|
||||
self.logger.info(f"开始执行任务: {self.name}")
|
||||
|
||||
try:
|
||||
# 检查超时
|
||||
if self.timeout:
|
||||
result = self._execute_with_timeout(*args, **kwargs)
|
||||
else:
|
||||
result = self._execute_task(*args, **kwargs)
|
||||
|
||||
self.status = "completed"
|
||||
self.completed_at = datetime.now()
|
||||
self.last_run = self.started_at
|
||||
|
||||
duration = (self.completed_at - self.started_at).total_seconds()
|
||||
self.logger.info(f"任务执行完成: {self.name}, 耗时: {duration:.2f}秒")
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
self.status = "failed"
|
||||
self.failure_count += 1
|
||||
self.last_error = e
|
||||
self.completed_at = datetime.now()
|
||||
self.last_run = self.started_at
|
||||
|
||||
self.logger.error(f"任务执行失败: {self.name}, 错误: {e}", exc_info=True)
|
||||
|
||||
# 重试逻辑
|
||||
if self.failure_count <= self.max_retries:
|
||||
self.logger.info(f"任务将在 {self.retry_delay} 秒后重试: {self.name}")
|
||||
time.sleep(self.retry_delay)
|
||||
return self.execute(*args, **kwargs)
|
||||
else:
|
||||
self.logger.error(f"任务重试次数已达上限: {self.name}")
|
||||
raise
|
||||
|
||||
def _execute_with_timeout(self, *args, **kwargs) -> Any:
|
||||
"""带超时的任务执行"""
|
||||
if asyncio.iscoroutinefunction(self.run):
|
||||
# 异步任务
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
return loop.run_until_complete(
|
||||
asyncio.wait_for(self.run(*args, **kwargs), timeout=self.timeout)
|
||||
)
|
||||
finally:
|
||||
loop.close()
|
||||
else:
|
||||
# 同步任务 - 使用 ThreadPoolExecutor 实现跨平台超时
|
||||
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FutureTimeoutError
|
||||
|
||||
def run_task():
|
||||
return self.run(*args, **kwargs)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=1) as executor:
|
||||
future = executor.submit(run_task)
|
||||
try:
|
||||
return future.result(timeout=self.timeout)
|
||||
except FutureTimeoutError:
|
||||
future.cancel()
|
||||
raise TimeoutError(f"任务执行超时: {self.name}")
|
||||
|
||||
def _execute_task(self, *args, **kwargs) -> Any:
|
||||
"""执行任务"""
|
||||
if asyncio.iscoroutinefunction(self.run):
|
||||
# 异步任务
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
return loop.run_until_complete(self.run(*args, **kwargs))
|
||||
finally:
|
||||
loop.close()
|
||||
else:
|
||||
# 同步任务
|
||||
return self.run(*args, **kwargs)
|
||||
|
||||
def get_info(self) -> Dict[str, Any]:
|
||||
"""获取任务信息"""
|
||||
return {
|
||||
"name": self.name,
|
||||
"description": self.description,
|
||||
"status": self.status,
|
||||
"trigger": str(self.trigger) if self.trigger else None,
|
||||
"created_at": self.created_at.isoformat(),
|
||||
"started_at": self.started_at.isoformat() if self.started_at else None,
|
||||
"completed_at": self.completed_at.isoformat() if self.completed_at else None,
|
||||
"last_run": self.last_run.isoformat() if self.last_run else None,
|
||||
"run_count": self.run_count,
|
||||
"failure_count": self.failure_count,
|
||||
"last_error": str(self.last_error) if self.last_error else None,
|
||||
"max_retries": self.max_retries,
|
||||
"retry_delay": self.retry_delay,
|
||||
"timeout": self.timeout,
|
||||
"job_id": self.job_id
|
||||
}
|
||||
|
||||
def reset(self) -> None:
|
||||
"""重置任务状态"""
|
||||
self.status = "pending"
|
||||
self.started_at = None
|
||||
self.completed_at = None
|
||||
self.failure_count = 0
|
||||
self.last_error = None
|
||||
self.logger.info(f"任务状态已重置: {self.name}")
|
||||
|
||||
@@ -0,0 +1,344 @@
|
||||
"""
|
||||
内置 Prometheus 指标模块（F5）
|
||||
|
||||
特性：
|
||||
- 可选依赖：未安装 prometheus-client 时全部 API 退化为 no-op，不报错
|
||||
- 多 worker 模式（Linux/macOS）自动配置 PROMETHEUS_MULTIPROC_DIR 实现指标聚合
|
||||
- 懒初始化：本模块 import 时绝不 import prometheus_client，
|
||||
保证 setup_multiproc_env 的环境变量先于 prometheus_client 初始化生效
|
||||
- 内置 HTTP 指标中间件与 /metrics 端点（由 Application 在 metrics.enabled 时接入）
|
||||
|
||||
配置（conf/config.yaml）::
|
||||
|
||||
metrics:
|
||||
enabled: true # 默认 false
|
||||
path: /metrics # 指标暴露路径
|
||||
http_metrics: true # 是否启用 HTTP 请求指标中间件
|
||||
multiproc_dir: null # multiproc 目录（默认系统临时目录下自动生成）
|
||||
|
||||
安装: pip install myboot[metrics]
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterator, Optional, Sequence, Tuple
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from .exceptions import MyBootException
|
||||
|
||||
# 默认耗时直方图分桶（秒）
|
||||
DEFAULT_DURATION_BUCKETS = (
|
||||
0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0,
|
||||
)
|
||||
|
||||
STAGE_HISTOGRAM_NAME = "myboot_stage_duration_seconds"
|
||||
|
||||
|
||||
class MetricsNotAvailableError(MyBootException):
|
||||
"""prometheus-client 未安装"""
|
||||
|
||||
def __init__(self, message: Optional[str] = None):
|
||||
super().__init__(
|
||||
message
|
||||
or "prometheus-client 未安装，指标功能不可用。"
|
||||
"安装方式: pip install myboot[metrics]",
|
||||
"METRICS_NOT_AVAILABLE",
|
||||
)
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""prometheus-client 是否已安装（不触发实际 import）"""
|
||||
return importlib.util.find_spec("prometheus_client") is not None
|
||||
|
||||
|
||||
def _coerce_bool(value: Any, default: bool = False) -> bool:
|
||||
"""布尔/字符串宽容转换（与 get_config_bool 语义一致）"""
|
||||
if value is None:
|
||||
return default
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.strip().lower() in ("true", "1", "yes", "on")
|
||||
return bool(value)
|
||||
|
||||
|
||||
def is_enabled(config) -> bool:
|
||||
"""读取 metrics.enabled 配置（默认 False）"""
|
||||
return _coerce_bool(config.get("metrics.enabled", False))
|
||||
|
||||
|
||||
def _slugify(name: str) -> str:
|
||||
slug = re.sub(r"[^a-zA-Z0-9_-]+", "_", str(name)).strip("_").lower()
|
||||
return slug or "app"
|
||||
|
||||
|
||||
def setup_multiproc_env(config, app_name: str) -> None:
|
||||
"""配置 Prometheus 多进程聚合环境变量
|
||||
|
||||
须在 prometheus_client 被 import 之前调用（Application.__init__ 早期）。
|
||||
仅当满足以下全部条件时生效，否则 no-op：
|
||||
|
||||
- metrics.enabled 为真
|
||||
- prometheus-client 已安装
|
||||
- server.workers > 1
|
||||
- 非 Windows（win32 多 worker 为 spawn 模式，multiproc 文件聚合不受支持）
|
||||
|
||||
本函数自身绝不 import prometheus_client。
|
||||
"""
|
||||
if not is_enabled(config):
|
||||
return
|
||||
if not is_available():
|
||||
return
|
||||
|
||||
try:
|
||||
workers = int(config.get("server.workers", 1) or 1)
|
||||
except (TypeError, ValueError):
|
||||
workers = 1
|
||||
if workers <= 1:
|
||||
return
|
||||
if sys.platform == "win32":
|
||||
logger.warning(
|
||||
"Windows 多 worker 模式不支持 Prometheus multiproc 聚合，"
|
||||
"各 worker 将仅暴露本进程指标"
|
||||
)
|
||||
return
|
||||
|
||||
# 用户已自行设置 → 尊重，不覆盖、不清理
|
||||
if os.environ.get("PROMETHEUS_MULTIPROC_DIR"):
|
||||
Path(os.environ["PROMETHEUS_MULTIPROC_DIR"]).mkdir(parents=True, exist_ok=True)
|
||||
return
|
||||
|
||||
if "prometheus_client" in sys.modules:
|
||||
logger.warning(
|
||||
"prometheus_client 已在 PROMETHEUS_MULTIPROC_DIR 设置之前被 import，"
|
||||
"多进程指标聚合可能失效（请避免在应用创建前 import prometheus_client）"
|
||||
)
|
||||
|
||||
multiproc_dir = config.get("metrics.multiproc_dir", None)
|
||||
if multiproc_dir:
|
||||
target = Path(str(multiproc_dir))
|
||||
else:
|
||||
target = Path(tempfile.gettempdir()) / f"myboot_prometheus_{_slugify(app_name)}"
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 仅父进程（MYBOOT_WORKER_ID 尚未设置时）清理陈旧 db 文件
|
||||
if "MYBOOT_WORKER_ID" not in os.environ:
|
||||
for stale in target.glob("*.db"):
|
||||
try:
|
||||
stale.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
os.environ["PROMETHEUS_MULTIPROC_DIR"] = str(target)
|
||||
logger.debug(f"Prometheus multiproc 目录: {target}")
|
||||
|
||||
|
||||
def _use_multiproc() -> bool:
|
||||
return bool(os.environ.get("PROMETHEUS_MULTIPROC_DIR"))
|
||||
|
||||
|
||||
def make_metrics_asgi_app():
|
||||
"""构建 /metrics ASGI 应用（懒初始化包装器）
|
||||
|
||||
首次请求时才 import prometheus_client 并构建真实 app：
|
||||
- multiproc 模式（环境变量已设置）→ MultiProcessCollector 聚合所有 worker
|
||||
- 否则使用默认全局 REGISTRY
|
||||
"""
|
||||
state: Dict[str, Any] = {"app": None}
|
||||
|
||||
async def metrics_app(scope, receive, send):
|
||||
if state["app"] is None:
|
||||
if not is_available():
|
||||
raise MetricsNotAvailableError()
|
||||
from prometheus_client import CollectorRegistry, REGISTRY, make_asgi_app
|
||||
|
||||
if _use_multiproc():
|
||||
from prometheus_client import multiprocess
|
||||
|
||||
registry = CollectorRegistry()
|
||||
multiprocess.MultiProcessCollector(registry)
|
||||
else:
|
||||
registry = REGISTRY
|
||||
state["app"] = make_asgi_app(registry=registry)
|
||||
await state["app"](scope, receive, send)
|
||||
|
||||
return metrics_app
|
||||
|
||||
|
||||
def mark_current_process_dead() -> None:
|
||||
"""multiproc 模式下标记本进程退出（清理 gauge 残留文件），其余情况 no-op"""
|
||||
try:
|
||||
if not _use_multiproc() or not is_available():
|
||||
return
|
||||
from prometheus_client import multiprocess
|
||||
|
||||
multiprocess.mark_process_dead(os.getpid())
|
||||
except Exception as e: # 退出路径绝不抛错
|
||||
logger.debug(f"mark_current_process_dead 失败（已忽略）: {e}")
|
||||
|
||||
|
||||
# ==================== 指标工厂（带 no-op 退化） ====================
|
||||
|
||||
|
||||
class _NoopMetric:
|
||||
"""prometheus-client 未安装时的 no-op 桩对象，支持链式调用"""
|
||||
|
||||
def labels(self, *args, **kwargs) -> "_NoopMetric":
|
||||
return self
|
||||
|
||||
def inc(self, *args, **kwargs) -> None:
|
||||
pass
|
||||
|
||||
def observe(self, *args, **kwargs) -> None:
|
||||
pass
|
||||
|
||||
def set(self, *args, **kwargs) -> None:
|
||||
pass
|
||||
|
||||
|
||||
_NOOP_METRIC = _NoopMetric()
|
||||
|
||||
# 进程内指标缓存，防止同名重复注册
|
||||
_metrics_cache: Dict[str, Any] = {}
|
||||
|
||||
|
||||
def get_counter(name: str, documentation: str, labelnames: Sequence[str] = ()):
|
||||
"""获取（或创建）Counter；未安装 prometheus-client 时返回 no-op 桩"""
|
||||
if name in _metrics_cache:
|
||||
return _metrics_cache[name]
|
||||
if not is_available():
|
||||
return _NOOP_METRIC
|
||||
from prometheus_client import Counter
|
||||
|
||||
metric = Counter(name, documentation, list(labelnames))
|
||||
_metrics_cache[name] = metric
|
||||
return metric
|
||||
|
||||
|
||||
def get_histogram(
|
||||
name: str,
|
||||
documentation: str,
|
||||
labelnames: Sequence[str] = (),
|
||||
buckets: Optional[Tuple[float, ...]] = None,
|
||||
):
|
||||
"""获取（或创建）Histogram；未安装 prometheus-client 时返回 no-op 桩"""
|
||||
if name in _metrics_cache:
|
||||
return _metrics_cache[name]
|
||||
if not is_available():
|
||||
return _NOOP_METRIC
|
||||
from prometheus_client import Histogram
|
||||
|
||||
metric = Histogram(
|
||||
name,
|
||||
documentation,
|
||||
list(labelnames),
|
||||
buckets=buckets or DEFAULT_DURATION_BUCKETS,
|
||||
)
|
||||
_metrics_cache[name] = metric
|
||||
return metric
|
||||
|
||||
|
||||
def observe_stage(stage: str, seconds: float, **labels) -> None:
|
||||
"""记录某个处理阶段耗时到内置 myboot_stage_duration_seconds 直方图
|
||||
|
||||
Args:
|
||||
stage: 阶段名（如 "recall"、"rank"）
|
||||
seconds: 耗时（秒），负值忽略
|
||||
**labels: 预留扩展，当前版本忽略额外标签
|
||||
"""
|
||||
if seconds < 0:
|
||||
return
|
||||
histogram = get_histogram(
|
||||
STAGE_HISTOGRAM_NAME,
|
||||
"MyBoot 处理阶段耗时（秒）",
|
||||
labelnames=("stage",),
|
||||
)
|
||||
histogram.labels(stage=stage).observe(seconds)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def time_stage(stage: str) -> Iterator[None]:
|
||||
"""上下文管理器：自动计时 with 体并调用 observe_stage"""
|
||||
t0 = time.perf_counter()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
observe_stage(stage, time.perf_counter() - t0)
|
||||
|
||||
|
||||
# ==================== HTTP 指标中间件 ====================
|
||||
|
||||
try: # starlette 是 fastapi 的必备依赖，正常总是可用
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
except ImportError: # pragma: no cover
|
||||
BaseHTTPMiddleware = object # type: ignore
|
||||
|
||||
|
||||
class HttpMetricsMiddleware(BaseHTTPMiddleware):
|
||||
"""HTTP 请求指标中间件
|
||||
|
||||
- myboot_http_requests_total{method, path, status} 请求计数
|
||||
- myboot_http_request_duration_seconds{method, path} 请求耗时直方图
|
||||
|
||||
path 标签使用路由模板（如 /items/{id}）避免高基数；未匹配路由归入
|
||||
"unmatched"；metrics 自身路径不统计。指标对象懒创建——middleware
|
||||
import / 构造时不触碰 prometheus_client。
|
||||
"""
|
||||
|
||||
def __init__(self, app, metrics_path: str = "/metrics"):
|
||||
super().__init__(app)
|
||||
self.metrics_path = metrics_path
|
||||
self._requests_total = None
|
||||
self._request_duration = None
|
||||
|
||||
def _ensure_metrics(self) -> None:
|
||||
if self._requests_total is None:
|
||||
self._requests_total = get_counter(
|
||||
"myboot_http_requests_total",
|
||||
"HTTP 请求总数",
|
||||
labelnames=("method", "path", "status"),
|
||||
)
|
||||
self._request_duration = get_histogram(
|
||||
"myboot_http_request_duration_seconds",
|
||||
"HTTP 请求耗时（秒）",
|
||||
labelnames=("method", "path"),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _route_path(scope) -> str:
|
||||
route = scope.get("route")
|
||||
if route is None:
|
||||
return "unmatched"
|
||||
# FastAPI APIRoute → path_format（模板，如 /items/{id}）；Mount → path
|
||||
path = getattr(route, "path_format", None) or getattr(route, "path", None)
|
||||
return path or "unmatched"
|
||||
|
||||
async def dispatch(self, request, call_next):
|
||||
raw_path = request.url.path
|
||||
if raw_path == self.metrics_path or raw_path.startswith(self.metrics_path + "/"):
|
||||
return await call_next(request)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
response = await call_next(request)
|
||||
elapsed = time.perf_counter() - t0
|
||||
|
||||
try:
|
||||
path = self._route_path(request.scope)
|
||||
self._ensure_metrics()
|
||||
self._requests_total.labels(
|
||||
method=request.method, path=path, status=str(response.status_code)
|
||||
).inc()
|
||||
self._request_duration.labels(
|
||||
method=request.method, path=path
|
||||
).observe(elapsed)
|
||||
except Exception as e: # 指标采集绝不影响业务请求
|
||||
logger.debug(f"HTTP 指标采集失败（已忽略）: {e}")
|
||||
|
||||
return response
|
||||
@@ -0,0 +1,30 @@
|
||||
"""
|
||||
工具函数模块
|
||||
|
||||
包含工具函数和公共模块
|
||||
"""
|
||||
|
||||
from .common import (
|
||||
generate_id,
|
||||
format_datetime,
|
||||
validate_email,
|
||||
hash_password,
|
||||
verify_password,
|
||||
generate_token,
|
||||
parse_datetime,
|
||||
get_local_ip
|
||||
)
|
||||
from .worker_sync import run_primary_first, clear_markers
|
||||
|
||||
__all__ = [
|
||||
"generate_id",
|
||||
"format_datetime",
|
||||
"validate_email",
|
||||
"hash_password",
|
||||
"verify_password",
|
||||
"generate_token",
|
||||
"parse_datetime",
|
||||
"get_local_ip",
|
||||
"run_primary_first",
|
||||
"clear_markers",
|
||||
]
|
||||
@@ -0,0 +1,187 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8
|
||||
"""
|
||||
异步执行工具模块
|
||||
提供可复用的异步执行方法
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
|
||||
from typing import Callable
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
logger = loguru_logger.bind(name='async_utils')
|
||||
|
||||
|
||||
class AsyncExecutor:
|
||||
"""异步执行器，提供可复用的异步执行方法"""
|
||||
|
||||
def __init__(self, max_workers: int = 4, executor_type: str = 'thread'):
|
||||
"""
|
||||
初始化异步执行器
|
||||
|
||||
Args:
|
||||
max_workers: 最大工作线程/进程数
|
||||
executor_type: 执行器类型，'thread' 或 'process'
|
||||
"""
|
||||
self.max_workers = max_workers
|
||||
self.executor_type = executor_type
|
||||
self._executor = None
|
||||
self._loop = None
|
||||
|
||||
@property
|
||||
def executor(self):
|
||||
"""获取执行器实例"""
|
||||
if self._executor is None:
|
||||
if self.executor_type == 'thread':
|
||||
self._executor = ThreadPoolExecutor(max_workers=self.max_workers)
|
||||
elif self.executor_type == 'process':
|
||||
self._executor = ProcessPoolExecutor(max_workers=self.max_workers)
|
||||
else:
|
||||
raise ValueError(f"不支持的执行器类型: {self.executor_type}")
|
||||
return self._executor
|
||||
|
||||
@property
|
||||
def loop(self):
|
||||
"""获取事件循环"""
|
||||
if self._loop is None:
|
||||
try:
|
||||
self._loop = asyncio.get_event_loop()
|
||||
except RuntimeError:
|
||||
self._loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(self._loop)
|
||||
return self._loop
|
||||
|
||||
async def run_in_background(self, func: Callable, *args, **kwargs) -> asyncio.Task:
|
||||
"""
|
||||
在后台异步执行函数
|
||||
|
||||
Args:
|
||||
func: 要执行的函数
|
||||
*args: 函数参数
|
||||
**kwargs: 函数关键字参数
|
||||
|
||||
Returns:
|
||||
asyncio.Task: 异步任务对象
|
||||
"""
|
||||
task = asyncio.create_task(self._execute_func(func, *args, **kwargs))
|
||||
return task
|
||||
|
||||
async def _execute_func(self, func: Callable, *args, task_name: str = None, **kwargs):
|
||||
"""执行函数的内部方法"""
|
||||
try:
|
||||
# 使用提供的任务名称或函数名
|
||||
display_name = task_name if task_name else func.__name__
|
||||
# logger.info(f"开始执行后台任务: {display_name}")
|
||||
result = await self.loop.run_in_executor(
|
||||
self.executor,
|
||||
functools.partial(func, *args, **kwargs)
|
||||
)
|
||||
# logger.info(f"后台任务执行完成: {display_name}")
|
||||
return result
|
||||
except Exception as e:
|
||||
display_name = task_name if task_name else func.__name__
|
||||
# logger.error(f"后台任务执行失败: {display_name}, 错误: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
def close(self):
|
||||
"""关闭执行器，释放资源"""
|
||||
if self._executor:
|
||||
self._executor.shutdown(wait=True)
|
||||
self._executor = None
|
||||
# logger.info("异步执行器已关闭")
|
||||
|
||||
|
||||
# 全局异步执行器实例
|
||||
_global_executor = None
|
||||
|
||||
|
||||
def get_async_executor(max_workers: int = 4, executor_type: str = 'thread') -> AsyncExecutor:
|
||||
"""
|
||||
获取全局异步执行器实例
|
||||
|
||||
Args:
|
||||
max_workers: 最大工作线程/进程数
|
||||
executor_type: 执行器类型
|
||||
|
||||
Returns:
|
||||
AsyncExecutor: 异步执行器实例
|
||||
"""
|
||||
global _global_executor
|
||||
if _global_executor is None:
|
||||
_global_executor = AsyncExecutor(max_workers, executor_type)
|
||||
return _global_executor
|
||||
|
||||
|
||||
def async_run(func: Callable, *args, **kwargs) -> asyncio.Task:
|
||||
"""
|
||||
快速启动后台任务的便捷函数
|
||||
|
||||
Args:
|
||||
func: 要执行的函数
|
||||
*args: 函数参数
|
||||
**kwargs: 函数关键字参数，支持 task_name 用于日志显示
|
||||
|
||||
Returns:
|
||||
asyncio.Task: 异步任务对象
|
||||
"""
|
||||
executor = get_async_executor()
|
||||
# 从kwargs中提取task_name，如果没有则使用函数名
|
||||
task_name = kwargs.pop('task_name', None)
|
||||
display_name = task_name if task_name else func.__name__
|
||||
|
||||
# 创建异步任务，确保在后台运行
|
||||
task = asyncio.create_task(executor._execute_func(func, *args, task_name=display_name, **kwargs))
|
||||
return task
|
||||
|
||||
|
||||
# 装饰器：将同步函数转换为异步函数
|
||||
def async_task(func: Callable) -> Callable:
|
||||
"""
|
||||
装饰器：将同步函数包装为异步函数
|
||||
|
||||
Usage:
|
||||
@async_task
|
||||
def my_sync_function():
|
||||
# 同步代码
|
||||
pass
|
||||
|
||||
# 现在可以异步调用
|
||||
await my_sync_function()
|
||||
"""
|
||||
|
||||
@functools.wraps(func)
|
||||
async def wrapper(*args, **kwargs):
|
||||
executor = get_async_executor()
|
||||
return await executor._execute_func(func, *args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
# 上下文管理器：自动管理执行器生命周期
|
||||
class AsyncExecutorContext:
|
||||
"""异步执行器上下文管理器"""
|
||||
|
||||
def __init__(self, max_workers: int = 4, executor_type: str = 'thread'):
|
||||
self.max_workers = max_workers
|
||||
self.executor_type = executor_type
|
||||
self.executor = None
|
||||
|
||||
async def __aenter__(self):
|
||||
self.executor = AsyncExecutor(self.max_workers, self.executor_type)
|
||||
return self.executor
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
if self.executor:
|
||||
self.executor.close()
|
||||
|
||||
|
||||
# 清理函数
|
||||
def cleanup_async_executor():
|
||||
"""清理全局异步执行器"""
|
||||
global _global_executor
|
||||
if _global_executor:
|
||||
_global_executor.close()
|
||||
_global_executor = None
|
||||
@@ -0,0 +1,340 @@
|
||||
"""
|
||||
公共工具函数
|
||||
|
||||
包含常用的工具函数和辅助方法
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import secrets
|
||||
import socket
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional, Union
|
||||
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
logger = loguru_logger.bind(name="utils")
|
||||
|
||||
|
||||
def generate_id() -> str:
|
||||
"""生成唯一ID"""
|
||||
return str(uuid.uuid4())
|
||||
|
||||
|
||||
def get_local_ip() -> str:
|
||||
"""获取本机真实 IP 地址"""
|
||||
try:
|
||||
# 连接到一个外部地址来获取本地 IP
|
||||
# 不实际发送数据，只是用于获取本地 IP
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
try:
|
||||
# 连接到公共 DNS（不需要实际连接成功）
|
||||
s.connect(('8.8.8.8', 80))
|
||||
ip = s.getsockname()[0]
|
||||
finally:
|
||||
s.close()
|
||||
return ip
|
||||
except Exception:
|
||||
# 如果失败，尝试使用主机名
|
||||
try:
|
||||
hostname = socket.gethostname()
|
||||
ip = socket.gethostbyname(hostname)
|
||||
# 如果不是有效的 IP（可能是 127.0.0.1），返回 localhost
|
||||
if ip.startswith('127.'):
|
||||
return 'localhost'
|
||||
return ip
|
||||
except Exception:
|
||||
return 'localhost'
|
||||
|
||||
|
||||
def generate_short_id(length: int = 8) -> str:
|
||||
"""生成短ID"""
|
||||
return secrets.token_urlsafe(length)
|
||||
|
||||
|
||||
def format_datetime(dt: datetime, format_str: str = "%Y-%m-%d %H:%M:%S") -> str:
|
||||
"""
|
||||
格式化日期时间
|
||||
|
||||
Args:
|
||||
dt: 日期时间对象
|
||||
format_str: 格式字符串
|
||||
|
||||
Returns:
|
||||
str: 格式化后的字符串
|
||||
"""
|
||||
if dt is None:
|
||||
return ""
|
||||
|
||||
return dt.strftime(format_str)
|
||||
|
||||
|
||||
def parse_datetime(date_str: str, format_str: str = "%Y-%m-%d %H:%M:%S") -> Optional[datetime]:
|
||||
"""
|
||||
解析日期时间字符串
|
||||
|
||||
Args:
|
||||
date_str: 日期时间字符串
|
||||
format_str: 格式字符串
|
||||
|
||||
Returns:
|
||||
Optional[datetime]: 解析后的日期时间对象
|
||||
"""
|
||||
try:
|
||||
return datetime.strptime(date_str, format_str)
|
||||
except ValueError:
|
||||
logger.warning(f"无法解析日期时间: {date_str}")
|
||||
return None
|
||||
|
||||
|
||||
def validate_email(email: str) -> bool:
|
||||
"""
|
||||
验证邮箱格式
|
||||
|
||||
Args:
|
||||
email: 邮箱地址
|
||||
|
||||
Returns:
|
||||
bool: 是否有效
|
||||
"""
|
||||
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
|
||||
return bool(re.match(pattern, email))
|
||||
|
||||
|
||||
def validate_phone(phone: str) -> bool:
|
||||
"""
|
||||
验证手机号格式
|
||||
|
||||
Args:
|
||||
phone: 手机号
|
||||
|
||||
Returns:
|
||||
bool: 是否有效
|
||||
"""
|
||||
pattern = r'^1[3-9]\d{9}$'
|
||||
return bool(re.match(pattern, phone))
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
"""
|
||||
哈希密码
|
||||
|
||||
Args:
|
||||
password: 原始密码
|
||||
|
||||
Returns:
|
||||
str: 哈希后的密码
|
||||
"""
|
||||
salt = secrets.token_hex(16)
|
||||
password_hash = hashlib.pbkdf2_hmac(
|
||||
'sha256',
|
||||
password.encode('utf-8'),
|
||||
salt.encode('utf-8'),
|
||||
100000
|
||||
)
|
||||
return f"{salt}:{password_hash.hex()}"
|
||||
|
||||
|
||||
def verify_password(password: str, password_hash: str) -> bool:
|
||||
"""
|
||||
验证密码
|
||||
|
||||
Args:
|
||||
password: 原始密码
|
||||
password_hash: 哈希后的密码
|
||||
|
||||
Returns:
|
||||
bool: 是否匹配
|
||||
"""
|
||||
try:
|
||||
salt, hash_value = password_hash.split(':')
|
||||
password_hash_check = hashlib.pbkdf2_hmac(
|
||||
'sha256',
|
||||
password.encode('utf-8'),
|
||||
salt.encode('utf-8'),
|
||||
100000
|
||||
)
|
||||
return password_hash_check.hex() == hash_value
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
def generate_token(length: int = 32) -> str:
|
||||
"""
|
||||
生成随机令牌
|
||||
|
||||
Args:
|
||||
length: 令牌长度
|
||||
|
||||
Returns:
|
||||
str: 随机令牌
|
||||
"""
|
||||
return secrets.token_urlsafe(length)
|
||||
|
||||
|
||||
def generate_api_key() -> str:
|
||||
"""生成API密钥"""
|
||||
return f"pk_{secrets.token_urlsafe(32)}"
|
||||
|
||||
|
||||
def mask_sensitive_data(data: str, mask_char: str = "*", visible_chars: int = 4) -> str:
|
||||
"""
|
||||
遮蔽敏感数据
|
||||
|
||||
Args:
|
||||
data: 原始数据
|
||||
mask_char: 遮蔽字符
|
||||
visible_chars: 可见字符数
|
||||
|
||||
Returns:
|
||||
str: 遮蔽后的数据
|
||||
"""
|
||||
if len(data) <= visible_chars:
|
||||
return mask_char * len(data)
|
||||
|
||||
return data[:visible_chars] + mask_char * (len(data) - visible_chars)
|
||||
|
||||
|
||||
def safe_get(data: dict, key: str, default: any = None) -> any:
|
||||
"""
|
||||
安全获取字典值
|
||||
|
||||
Args:
|
||||
data: 字典数据
|
||||
key: 键，支持点号分隔的嵌套键
|
||||
default: 默认值
|
||||
|
||||
Returns:
|
||||
any: 值或默认值
|
||||
"""
|
||||
keys = key.split('.')
|
||||
value = data
|
||||
|
||||
try:
|
||||
for k in keys:
|
||||
value = value[k]
|
||||
return value
|
||||
except (KeyError, TypeError):
|
||||
return default
|
||||
|
||||
|
||||
def chunk_list(lst: list, chunk_size: int) -> list:
|
||||
"""
|
||||
将列表分块
|
||||
|
||||
Args:
|
||||
lst: 原始列表
|
||||
chunk_size: 块大小
|
||||
|
||||
Returns:
|
||||
list: 分块后的列表
|
||||
"""
|
||||
return [lst[i:i + chunk_size] for i in range(0, len(lst), chunk_size)]
|
||||
|
||||
|
||||
def remove_none_values(data: dict) -> dict:
|
||||
"""
|
||||
移除字典中的None值
|
||||
|
||||
Args:
|
||||
data: 原始字典
|
||||
|
||||
Returns:
|
||||
dict: 清理后的字典
|
||||
"""
|
||||
return {k: v for k, v in data.items() if v is not None}
|
||||
|
||||
|
||||
def deep_merge_dict(dict1: dict, dict2: dict) -> dict:
|
||||
"""
|
||||
深度合并字典
|
||||
|
||||
Args:
|
||||
dict1: 第一个字典
|
||||
dict2: 第二个字典
|
||||
|
||||
Returns:
|
||||
dict: 合并后的字典
|
||||
"""
|
||||
result = dict1.copy()
|
||||
|
||||
for key, value in dict2.items():
|
||||
if key in result and isinstance(result[key], dict) and isinstance(value, dict):
|
||||
result[key] = deep_merge_dict(result[key], value)
|
||||
else:
|
||||
result[key] = value
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_current_timestamp() -> int:
|
||||
"""获取当前时间戳（毫秒）"""
|
||||
return int(datetime.now(timezone.utc).timestamp() * 1000)
|
||||
|
||||
|
||||
def format_file_size(size_bytes: int) -> str:
|
||||
"""
|
||||
格式化文件大小
|
||||
|
||||
Args:
|
||||
size_bytes: 字节数
|
||||
|
||||
Returns:
|
||||
str: 格式化后的大小
|
||||
"""
|
||||
if size_bytes == 0:
|
||||
return "0 B"
|
||||
|
||||
size_names = ["B", "KB", "MB", "GB", "TB"]
|
||||
i = 0
|
||||
while size_bytes >= 1024 and i < len(size_names) - 1:
|
||||
size_bytes /= 1024.0
|
||||
i += 1
|
||||
|
||||
return f"{size_bytes:.1f} {size_names[i]}"
|
||||
|
||||
|
||||
def is_valid_url(url: str) -> bool:
|
||||
"""
|
||||
验证URL格式
|
||||
|
||||
Args:
|
||||
url: URL字符串
|
||||
|
||||
Returns:
|
||||
bool: 是否有效
|
||||
"""
|
||||
pattern = r'^https?://(?:[-\w.])+(?:\:[0-9]+)?(?:/(?:[\w/_.])*(?:\?(?:[\w&=%.])*)?(?:\#(?:[\w.])*)?)?$'
|
||||
return bool(re.match(pattern, url))
|
||||
|
||||
|
||||
class RetryHelper:
|
||||
"""重试辅助类"""
|
||||
|
||||
def __init__(self, max_retries: int = 3, delay: float = 1.0, backoff_factor: float = 2.0):
|
||||
self.max_retries = max_retries
|
||||
self.delay = delay
|
||||
self.backoff_factor = backoff_factor
|
||||
|
||||
def execute(self, func, *args, **kwargs):
|
||||
"""执行带重试的函数"""
|
||||
import time
|
||||
|
||||
last_exception = None
|
||||
current_delay = self.delay
|
||||
|
||||
for attempt in range(self.max_retries + 1):
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except Exception as e:
|
||||
last_exception = e
|
||||
|
||||
if attempt < self.max_retries:
|
||||
logger.warning(f"执行失败，{current_delay}秒后重试 (尝试 {attempt + 1}/{self.max_retries + 1}): {e}")
|
||||
time.sleep(current_delay)
|
||||
current_delay *= self.backoff_factor
|
||||
else:
|
||||
logger.error(f"执行失败，已达到最大重试次数: {e}")
|
||||
|
||||
raise last_exception
|
||||
@@ -0,0 +1,29 @@
|
||||
"""
|
||||
命名转换工具
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
|
||||
def camel_to_snake(name: str) -> str:
|
||||
"""
|
||||
将驼峰命名转换为下划线分隔的小写形式
|
||||
|
||||
Args:
|
||||
name: 类名（驼峰命名）
|
||||
|
||||
Returns:
|
||||
下划线分隔的小写字符串
|
||||
|
||||
Examples:
|
||||
UserService -> user_service
|
||||
EmailService -> email_service
|
||||
DatabaseClient -> database_client
|
||||
HTTPClient -> http_client
|
||||
"""
|
||||
# 在大写字母前插入下划线（除了第一个字符）
|
||||
s1 = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
|
||||
# 处理连续大写字母的情况（如 HTTPClient）
|
||||
s2 = re.sub('([a-z0-9])([A-Z])', r'\1_\2', s1)
|
||||
# 转换为小写
|
||||
return s2.lower()
|
||||
@@ -0,0 +1,88 @@
|
||||
"""
|
||||
路径工具模块
|
||||
|
||||
提供项目根目录检测和路径解析功能。
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def get_project_root() -> str:
|
||||
"""
|
||||
获取项目根目录
|
||||
|
||||
从当前文件所在目录开始向上查找，直到找到包含pyproject.toml的目录。
|
||||
如果没找到，则使用当前工作目录。
|
||||
|
||||
Returns:
|
||||
str: 项目根目录的绝对路径
|
||||
"""
|
||||
current_dir = Path(__file__).parent.absolute()
|
||||
|
||||
while current_dir.parent != current_dir:
|
||||
if (current_dir / 'pyproject.toml').exists():
|
||||
return str(current_dir)
|
||||
current_dir = current_dir.parent
|
||||
|
||||
return os.getcwd()
|
||||
|
||||
|
||||
def resolve_path(path: str) -> str:
|
||||
"""
|
||||
解析路径，如果是相对路径则基于项目根目录
|
||||
|
||||
Args:
|
||||
path (str): 要解析的路径，可以是相对路径或绝对路径
|
||||
|
||||
Returns:
|
||||
str: 解析后的绝对路径
|
||||
|
||||
Examples:
|
||||
>>> resolve_path('./data/file.csv')
|
||||
'/path/to/project/data/file.csv'
|
||||
|
||||
>>> resolve_path('/absolute/path/file.csv')
|
||||
'/absolute/path/file.csv'
|
||||
"""
|
||||
if os.path.isabs(path):
|
||||
return path
|
||||
|
||||
# 清理路径，移除多余的./前缀
|
||||
clean_path = path.lstrip('./')
|
||||
return os.path.join(get_project_root(), clean_path)
|
||||
|
||||
|
||||
def get_data_dir() -> str:
|
||||
"""
|
||||
获取数据目录的绝对路径
|
||||
|
||||
Returns:
|
||||
str: 数据目录的绝对路径
|
||||
"""
|
||||
return resolve_path('data')
|
||||
|
||||
|
||||
def get_conf_dir() -> str:
|
||||
"""
|
||||
获取配置目录的绝对路径
|
||||
|
||||
Returns:
|
||||
str: 配置目录的绝对路径
|
||||
"""
|
||||
return resolve_path('conf')
|
||||
|
||||
|
||||
def ensure_dir_exists(path: str) -> str:
|
||||
"""
|
||||
确保目录存在，如果不存在则创建
|
||||
|
||||
Args:
|
||||
path (str): 目录路径
|
||||
|
||||
Returns:
|
||||
str: 目录的绝对路径
|
||||
"""
|
||||
abs_path = resolve_path(path)
|
||||
os.makedirs(abs_path, exist_ok=True)
|
||||
return abs_path
|
||||
@@ -0,0 +1,187 @@
|
||||
"""
|
||||
Worker 同步工具：primary-first 初始化协调
|
||||
|
||||
多 worker 部署下常见需求：primary worker 负责重型初始化（如下载/加载模型），
|
||||
其余 worker 等待 primary 完成后再做轻量加载。本模块通过文件标记实现跨进程协调，
|
||||
仅依赖标准库，兼容 Windows（不使用 fcntl），spawn / fork 两种启动模式均可用。
|
||||
|
||||
典型用法::
|
||||
|
||||
from myboot.utils import run_primary_first
|
||||
|
||||
model = run_primary_first(
|
||||
"sas_rec_model",
|
||||
primary_fn=service.initialize_model, # primary: 下载并初始化
|
||||
secondary_fn=service.load_model_from_disk # 其他 worker: 轻量加载
|
||||
)
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import hashlib
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional, TypeVar
|
||||
|
||||
from loguru import logger
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
_logger = logger.bind(name="worker_sync")
|
||||
|
||||
# 模块加载时间，用于陈旧标记判定（见 run_primary_first docstring）。
|
||||
# spawn 模式下每个 worker 进程独立导入本模块，该时间即 worker 启动时间附近；
|
||||
# fork 模式下可能继承父进程导入时刻，比 worker 启动更早——仍然早于本轮
|
||||
# primary 写入标记的时刻，因此不会误拒本轮标记。
|
||||
_MODULE_LOAD_TIME = time.time()
|
||||
|
||||
# 陈旧标记判定容差（秒）：mtime 早于 (_MODULE_LOAD_TIME - 容差) 的 .done 视为陈旧
|
||||
_STALE_TOLERANCE = 2.0
|
||||
|
||||
|
||||
def _slug() -> str:
|
||||
"""标记目录 slug：优先 MYBOOT_APP_NAME，否则取当前工作目录路径的 hash。
|
||||
|
||||
两者在 spawn / fork 模式下父子进程均一致：环境变量被子进程继承（spawn 下
|
||||
multiprocessing 会复制父进程环境），工作目录同样被继承。
|
||||
"""
|
||||
app_name = os.environ.get("MYBOOT_APP_NAME")
|
||||
if app_name:
|
||||
safe = "".join(c if c.isalnum() or c in "-_" else "_" for c in app_name)
|
||||
return safe[:64]
|
||||
return hashlib.md5(os.getcwd().encode("utf-8")).hexdigest()[:16]
|
||||
|
||||
|
||||
def _marker_dir() -> Path:
|
||||
d = Path(tempfile.gettempdir()) / f"myboot_worker_sync_{_slug()}"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def clear_markers(name: Optional[str] = None) -> None:
|
||||
"""删除标记文件。
|
||||
|
||||
Args:
|
||||
name: 指定时只删除该 name 的 .done / .failed；为 None 时清空整个标记目录。
|
||||
"""
|
||||
d = _marker_dir()
|
||||
if name is not None:
|
||||
patterns = [f"{name}.done", f"{name}.failed"]
|
||||
files = [d / p for p in patterns]
|
||||
else:
|
||||
files = list(d.glob("*.done")) + list(d.glob("*.failed"))
|
||||
for f in files:
|
||||
try:
|
||||
f.unlink()
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
except OSError as e: # pragma: no cover - 极端权限/占用情况
|
||||
_logger.warning(f"无法删除标记文件 {f}: {e!r}")
|
||||
|
||||
|
||||
def _is_fresh(path: Path) -> bool:
|
||||
"""判断标记文件是否属于本轮启动（基于 mtime）。"""
|
||||
try:
|
||||
return path.stat().st_mtime >= _MODULE_LOAD_TIME - _STALE_TOLERANCE
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def run_primary_first(
|
||||
name: str,
|
||||
primary_fn: Callable[[], T],
|
||||
secondary_fn: Optional[Callable[[], T]] = None,
|
||||
*,
|
||||
timeout: float = 300.0,
|
||||
poll_interval: float = 0.5,
|
||||
) -> T:
|
||||
"""primary-first 初始化协调：primary worker 执行重型初始化，其余 worker 等待。
|
||||
|
||||
判定规则：``os.environ.get("MYBOOT_IS_PRIMARY_WORKER", "1") == "1"`` 视为
|
||||
primary（单进程模式无此变量 → 默认 primary，工具退化为直接调用 primary_fn）。
|
||||
|
||||
- primary：先清理本 name 的旧标记，执行 primary_fn()；成功写
|
||||
``<标记目录>/<name>.done``（内容为 ISO 时间戳）并返回结果；
|
||||
异常时写 ``<name>.failed``（内容为异常 repr）后重新抛出。
|
||||
- 非 primary：以 poll_interval 间隔轮询。发现新鲜的 .done → 执行
|
||||
secondary_fn()（为 None 时执行 primary_fn）并返回其结果；发现新鲜的
|
||||
.failed → 抛 RuntimeError（含 primary 的异常信息）；超过 timeout →
|
||||
抛 TimeoutError。
|
||||
|
||||
陈旧标记的保证与限制（双重防护）：
|
||||
|
||||
1. primary 在执行 primary_fn 之前删除本 name 的旧标记（clear_markers）。
|
||||
2. secondary 仅认可 mtime 不早于「本模块加载时间 - 2 秒容差」的标记，
|
||||
上一轮运行遗留的 .done/.failed 因 mtime 过旧会被忽略并继续等待。
|
||||
|
||||
保证：只要新一轮启动距上一轮结束超过容差（2 秒），secondary 不会被上一轮
|
||||
遗留的标记误导；即使 secondary 比 primary 先启动并先于 primary 清理动作
|
||||
看到旧标记，mtime 检查也会将其判为陈旧。
|
||||
|
||||
限制：
|
||||
- fork 模式下模块加载时间可能取自父进程导入时刻（早于 worker fork），
|
||||
若上一轮标记写入晚于父进程导入（如 2 秒内重启），理论上可能误认旧标记；
|
||||
实际部署中重启间隔远大于容差，且 primary 启动后会立即删除旧标记，
|
||||
该窗口极小。
|
||||
- 依赖文件系统 mtime 精度与各 worker 共享同一台机器的 temp 目录；
|
||||
不适用于跨机器协调。
|
||||
|
||||
Args:
|
||||
name: 协调任务名，同一应用内唯一（用作标记文件名）。
|
||||
primary_fn: primary worker 执行的初始化函数。
|
||||
secondary_fn: 非 primary worker 在 primary 完成后执行的函数；
|
||||
为 None 时执行 primary_fn。
|
||||
timeout: 非 primary 等待 .done 的最长秒数。
|
||||
poll_interval: 轮询间隔秒数。
|
||||
|
||||
Returns:
|
||||
primary_fn 或 secondary_fn 的返回值。
|
||||
|
||||
Raises:
|
||||
TimeoutError: 非 primary 等待超时。
|
||||
RuntimeError: 非 primary 检测到 primary 初始化失败。
|
||||
Exception: primary 侧 primary_fn 抛出的原始异常。
|
||||
"""
|
||||
is_primary = os.environ.get("MYBOOT_IS_PRIMARY_WORKER", "1") == "1"
|
||||
d = _marker_dir()
|
||||
done_file = d / f"{name}.done"
|
||||
failed_file = d / f"{name}.failed"
|
||||
|
||||
if is_primary:
|
||||
clear_markers(name)
|
||||
_logger.info(f"[{name}] primary worker 开始初始化")
|
||||
try:
|
||||
result = primary_fn()
|
||||
except BaseException as e:
|
||||
failed_file.write_text(repr(e), encoding="utf-8")
|
||||
_logger.error(f"[{name}] primary 初始化失败: {e!r}")
|
||||
raise
|
||||
done_file.write_text(
|
||||
datetime.datetime.now().isoformat(), encoding="utf-8"
|
||||
)
|
||||
_logger.info(f"[{name}] primary 初始化完成，已写入标记 {done_file}")
|
||||
return result
|
||||
|
||||
# 非 primary：轮询等待
|
||||
_logger.info(f"[{name}] secondary worker 等待 primary 完成 (timeout={timeout}s)")
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
if done_file.exists() and _is_fresh(done_file):
|
||||
_logger.info(f"[{name}] 检测到 primary 完成标记，开始 secondary 加载")
|
||||
fn = secondary_fn if secondary_fn is not None else primary_fn
|
||||
return fn()
|
||||
if failed_file.exists() and _is_fresh(failed_file):
|
||||
detail = ""
|
||||
try:
|
||||
detail = failed_file.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
pass
|
||||
raise RuntimeError(
|
||||
f"primary worker 初始化 '{name}' 失败: {detail}"
|
||||
)
|
||||
if time.monotonic() >= deadline:
|
||||
raise TimeoutError(
|
||||
f"等待 primary worker 完成 '{name}' 超时 ({timeout}s)"
|
||||
)
|
||||
time.sleep(poll_interval)
|
||||
@@ -0,0 +1,17 @@
|
||||
"""
|
||||
Web 模块
|
||||
|
||||
提供 Web 相关的功能，包括中间件、响应格式等
|
||||
"""
|
||||
|
||||
from .response import ResponseWrapper, ApiResponse, response
|
||||
from .middleware import Middleware, FunctionMiddleware, ResponseFormatterMiddleware
|
||||
|
||||
__all__ = [
|
||||
"ResponseWrapper",
|
||||
"ApiResponse",
|
||||
"response",
|
||||
"Middleware",
|
||||
"FunctionMiddleware",
|
||||
"ResponseFormatterMiddleware"
|
||||
]
|
||||
@@ -0,0 +1,286 @@
|
||||
"""
|
||||
Web 路由装饰器
|
||||
|
||||
提供类似 Spring Boot 的注解式路由功能
|
||||
"""
|
||||
|
||||
from functools import wraps
|
||||
from typing import Any, Callable, List, Optional
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
def route(
|
||||
path: str,
|
||||
methods: Optional[List[str]] = None,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 200,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""
|
||||
通用路由装饰器
|
||||
|
||||
Args:
|
||||
path: 路由路径
|
||||
methods: HTTP 方法列表
|
||||
response_model: 响应模型
|
||||
status_code: 状态码
|
||||
tags: 标签
|
||||
summary: 摘要
|
||||
description: 描述
|
||||
**kwargs: 其他 FastAPI 参数
|
||||
"""
|
||||
if methods is None:
|
||||
methods = ["GET"]
|
||||
|
||||
def decorator(func: Callable) -> Callable:
|
||||
# 添加路由元数据
|
||||
func._route_metadata = {
|
||||
'path': path,
|
||||
'methods': methods,
|
||||
'response_model': response_model,
|
||||
'status_code': status_code,
|
||||
'tags': tags or [],
|
||||
'summary': summary or func.__doc__ or "",
|
||||
'description': description or func.__doc__ or "",
|
||||
**kwargs
|
||||
}
|
||||
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def get(
|
||||
path: str,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 200,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""GET 路由装饰器"""
|
||||
return route(
|
||||
path=path,
|
||||
methods=["GET"],
|
||||
response_model=response_model,
|
||||
status_code=status_code,
|
||||
tags=tags,
|
||||
summary=summary,
|
||||
description=description,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
|
||||
def post(
|
||||
path: str,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 201,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""POST 路由装饰器"""
|
||||
return route(
|
||||
path=path,
|
||||
methods=["POST"],
|
||||
response_model=response_model,
|
||||
status_code=status_code,
|
||||
tags=tags,
|
||||
summary=summary,
|
||||
description=description,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
|
||||
def put(
|
||||
path: str,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 200,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""PUT 路由装饰器"""
|
||||
return route(
|
||||
path=path,
|
||||
methods=["PUT"],
|
||||
response_model=response_model,
|
||||
status_code=status_code,
|
||||
tags=tags,
|
||||
summary=summary,
|
||||
description=description,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
|
||||
def delete(
|
||||
path: str,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 204,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""DELETE 路由装饰器"""
|
||||
return route(
|
||||
path=path,
|
||||
methods=["DELETE"],
|
||||
response_model=response_model,
|
||||
status_code=status_code,
|
||||
tags=tags,
|
||||
summary=summary,
|
||||
description=description,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
|
||||
def patch(
|
||||
path: str,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 200,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""PATCH 路由装饰器"""
|
||||
return route(
|
||||
path=path,
|
||||
methods=["PATCH"],
|
||||
response_model=response_model,
|
||||
status_code=status_code,
|
||||
tags=tags,
|
||||
summary=summary,
|
||||
description=description,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
|
||||
def validate_request(model: BaseModel) -> Callable:
|
||||
"""
|
||||
请求验证装饰器
|
||||
|
||||
Args:
|
||||
model: Pydantic 模型类
|
||||
"""
|
||||
def decorator(func: Callable) -> Callable:
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
# 这里可以添加请求验证逻辑
|
||||
# 在实际应用中，FastAPI 会自动处理 Pydantic 模型验证
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def require_auth(roles: Optional[List[str]] = None) -> Callable:
|
||||
"""
|
||||
身份验证装饰器
|
||||
|
||||
Args:
|
||||
roles: 需要的角色列表
|
||||
"""
|
||||
def decorator(func: Callable) -> Callable:
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
# 这里可以添加身份验证逻辑
|
||||
# 在实际应用中，可以集成 JWT、OAuth 等认证方式
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def rate_limit(requests: int = 100, window: int = 60) -> Callable:
|
||||
"""
|
||||
限流装饰器
|
||||
|
||||
Args:
|
||||
requests: 时间窗口内允许的请求数
|
||||
window: 时间窗口（秒）
|
||||
"""
|
||||
def decorator(func: Callable) -> Callable:
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
# 这里可以添加限流逻辑
|
||||
# 在实际应用中，可以集成 Redis 等存储来实现分布式限流
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def cache(ttl: int = 300) -> Callable:
|
||||
"""
|
||||
缓存装饰器
|
||||
|
||||
Args:
|
||||
ttl: 缓存生存时间（秒）
|
||||
"""
|
||||
def decorator(func: Callable) -> Callable:
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
# 这里可以添加缓存逻辑
|
||||
# 在实际应用中，可以集成 Redis、Memcached 等缓存系统
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def async_route(
|
||||
path: str,
|
||||
methods: Optional[List[str]] = None,
|
||||
response_model: Optional[Any] = None,
|
||||
status_code: int = 200,
|
||||
tags: Optional[List[str]] = None,
|
||||
summary: Optional[str] = None,
|
||||
description: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> Callable:
|
||||
"""
|
||||
异步路由装饰器
|
||||
|
||||
用于标记异步处理函数
|
||||
"""
|
||||
if methods is None:
|
||||
methods = ["GET"]
|
||||
|
||||
def decorator(func: Callable) -> Callable:
|
||||
# 添加路由元数据
|
||||
func._route_metadata = {
|
||||
'path': path,
|
||||
'methods': methods,
|
||||
'response_model': response_model,
|
||||
'status_code': status_code,
|
||||
'tags': tags or [],
|
||||
'summary': summary or func.__doc__ or "",
|
||||
'description': description or func.__doc__ or "",
|
||||
'async': True,
|
||||
**kwargs
|
||||
}
|
||||
|
||||
@wraps(func)
|
||||
async def wrapper(*args, **kwargs):
|
||||
return await func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
@@ -0,0 +1,231 @@
|
||||
"""
|
||||
Web 异常模块
|
||||
|
||||
提供 Web 相关的异常类
|
||||
|
||||
ValidationError / ConfigurationError 自 0.2.0 起与 myboot.exceptions
|
||||
中的定义收敛为同一个类（此前两处各自定义、互不兼容），从本模块 import
|
||||
仍然有效。
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from ..exceptions import ConfigurationError, ValidationError # noqa: F401
|
||||
|
||||
|
||||
class HTTPException(Exception):
|
||||
"""HTTP 异常基类"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
status_code: int,
|
||||
message: str = "HTTP Error",
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.status_code = status_code
|
||||
self.message = message
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class BadRequestError(HTTPException):
|
||||
"""400 错误请求异常"""
|
||||
|
||||
def __init__(self, message: str = "错误请求", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(400, message, details)
|
||||
|
||||
|
||||
class UnauthorizedError(HTTPException):
|
||||
"""401 未授权异常"""
|
||||
|
||||
def __init__(self, message: str = "未授权", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(401, message, details)
|
||||
|
||||
|
||||
class ForbiddenError(HTTPException):
|
||||
"""403 禁止访问异常"""
|
||||
|
||||
def __init__(self, message: str = "禁止访问", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(403, message, details)
|
||||
|
||||
|
||||
class NotFoundError(HTTPException):
|
||||
"""404 未找到异常"""
|
||||
|
||||
def __init__(self, message: str = "未找到", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(404, message, details)
|
||||
|
||||
|
||||
class MethodNotAllowedError(HTTPException):
|
||||
"""405 方法不允许异常"""
|
||||
|
||||
def __init__(self, message: str = "方法不允许", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(405, message, details)
|
||||
|
||||
|
||||
class ConflictError(HTTPException):
|
||||
"""409 冲突异常"""
|
||||
|
||||
def __init__(self, message: str = "资源冲突", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(409, message, details)
|
||||
|
||||
|
||||
class UnprocessableEntityError(HTTPException):
|
||||
"""422 无法处理的实体异常"""
|
||||
|
||||
def __init__(self, message: str = "无法处理的实体", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(422, message, details)
|
||||
|
||||
|
||||
class TooManyRequestsError(HTTPException):
|
||||
"""429 请求过多异常"""
|
||||
|
||||
def __init__(self, message: str = "请求过多", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(429, message, details)
|
||||
|
||||
|
||||
class InternalServerError(HTTPException):
|
||||
"""500 内部服务器错误异常"""
|
||||
|
||||
def __init__(self, message: str = "内部服务器错误", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(500, message, details)
|
||||
|
||||
|
||||
class ServiceUnavailableError(HTTPException):
|
||||
"""503 服务不可用异常"""
|
||||
|
||||
def __init__(self, message: str = "服务不可用", details: Optional[Dict[str, Any]] = None):
|
||||
super().__init__(503, message, details)
|
||||
|
||||
|
||||
class AuthenticationError(Exception):
|
||||
"""认证错误异常"""
|
||||
|
||||
def __init__(self, message: str = "认证失败"):
|
||||
self.message = message
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class AuthorizationError(Exception):
|
||||
"""授权错误异常"""
|
||||
|
||||
def __init__(self, message: str = "授权失败"):
|
||||
self.message = message
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class RateLimitError(Exception):
|
||||
"""限流错误异常"""
|
||||
|
||||
def __init__(self, message: str = "请求频率过高", retry_after: Optional[int] = None):
|
||||
self.message = message
|
||||
self.retry_after = retry_after
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class BusinessError(Exception):
|
||||
"""业务错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "业务处理失败",
|
||||
code: str = "BUSINESS_ERROR",
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.message = message
|
||||
self.code = code
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class ExternalServiceError(Exception):
|
||||
"""外部服务错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
service_name: str,
|
||||
message: str = "外部服务调用失败",
|
||||
status_code: Optional[int] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.service_name = service_name
|
||||
self.message = message
|
||||
self.status_code = status_code
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class DatabaseError(Exception):
|
||||
"""数据库错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "数据库操作失败",
|
||||
operation: Optional[str] = None,
|
||||
table: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.message = message
|
||||
self.operation = operation
|
||||
self.table = table
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class CacheError(Exception):
|
||||
"""缓存错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "缓存操作失败",
|
||||
operation: Optional[str] = None,
|
||||
key: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
self.message = message
|
||||
self.operation = operation
|
||||
self.key = key
|
||||
self.details = details or {}
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
class TimeoutError(Exception):
|
||||
"""超时错误异常"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "操作超时",
|
||||
timeout: Optional[float] = None,
|
||||
operation: Optional[str] = None
|
||||
):
|
||||
self.message = message
|
||||
self.timeout = timeout
|
||||
self.operation = operation
|
||||
super().__init__(self.message)
|
||||
|
||||
|
||||
# 异常映射
|
||||
HTTP_STATUS_EXCEPTIONS = {
|
||||
400: BadRequestError,
|
||||
401: UnauthorizedError,
|
||||
403: ForbiddenError,
|
||||
404: NotFoundError,
|
||||
405: MethodNotAllowedError,
|
||||
409: ConflictError,
|
||||
422: UnprocessableEntityError,
|
||||
429: TooManyRequestsError,
|
||||
500: InternalServerError,
|
||||
503: ServiceUnavailableError,
|
||||
}
|
||||
|
||||
|
||||
def create_http_exception(status_code: int, message: str, details: Optional[Dict[str, Any]] = None) -> HTTPException:
|
||||
"""根据状态码创建 HTTP 异常
|
||||
|
||||
已知状态码映射到对应子类（子类构造签名为 ``(message, details)``，
|
||||
status_code 已内置）；未知状态码回退到基类 ``HTTPException``。
|
||||
"""
|
||||
exception_class = HTTP_STATUS_EXCEPTIONS.get(status_code)
|
||||
if exception_class is not None:
|
||||
return exception_class(message, details)
|
||||
return HTTPException(status_code, message, details)
|
||||
@@ -0,0 +1,327 @@
|
||||
"""
|
||||
Web 中间件模块
|
||||
|
||||
提供类似 Spring Boot 的中间件功能
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
from typing import Callable, List, Optional, Union, Pattern
|
||||
import re
|
||||
|
||||
from fastapi import Request, Response
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
|
||||
from starlette.responses import StreamingResponse
|
||||
|
||||
|
||||
class Middleware:
|
||||
"""中间件基类"""
|
||||
|
||||
def __init__(self, middleware_class: type, **kwargs):
|
||||
"""
|
||||
初始化中间件
|
||||
|
||||
Args:
|
||||
middleware_class: 中间件类
|
||||
**kwargs: 中间件参数
|
||||
"""
|
||||
self.middleware_class = middleware_class
|
||||
self.kwargs = kwargs
|
||||
|
||||
|
||||
class FunctionMiddleware(BaseHTTPMiddleware):
|
||||
"""函数中间件包装器
|
||||
|
||||
将函数式中间件转换为 FastAPI 的 BaseHTTPMiddleware 类
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
app,
|
||||
middleware_func: Callable,
|
||||
path_filter: Optional[Union[str, Pattern, List[str]]] = None,
|
||||
methods: Optional[List[str]] = None,
|
||||
condition: Optional[Callable] = None,
|
||||
order: int = 0,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
初始化函数中间件
|
||||
|
||||
Args:
|
||||
app: FastAPI 应用实例
|
||||
middleware_func: 中间件函数，接收 (request, next_handler) 参数
|
||||
path_filter: 路径过滤，支持字符串、正则表达式或字符串列表
|
||||
methods: HTTP 方法过滤，如 ['GET', 'POST']
|
||||
condition: 条件函数，接收 request，返回 bool 决定是否执行中间件
|
||||
order: 执行顺序，数字越小越先执行
|
||||
**kwargs: 其他参数
|
||||
"""
|
||||
super().__init__(app)
|
||||
self.middleware_func = middleware_func
|
||||
self.path_filter = path_filter
|
||||
self.methods = [m.upper() for m in methods] if methods else None
|
||||
self.condition = condition
|
||||
self.order = order
|
||||
self.kwargs = kwargs
|
||||
|
||||
# 编译路径过滤正则表达式
|
||||
if path_filter:
|
||||
if isinstance(path_filter, str):
|
||||
# 将通配符转换为正则表达式
|
||||
pattern = path_filter.replace('*', '.*').replace('?', '.')
|
||||
self.path_pattern = re.compile(f'^{pattern}$')
|
||||
elif isinstance(path_filter, list):
|
||||
patterns = [p.replace('*', '.*').replace('?', '.') for p in path_filter]
|
||||
self.path_pattern = re.compile(f"^({'|'.join(patterns)})$")
|
||||
elif isinstance(path_filter, Pattern):
|
||||
self.path_pattern = path_filter
|
||||
else:
|
||||
self.path_pattern = None
|
||||
else:
|
||||
self.path_pattern = None
|
||||
|
||||
def _should_process(self, request: Request) -> bool:
|
||||
"""判断是否应该处理该请求"""
|
||||
# 检查路径过滤
|
||||
if self.path_pattern:
|
||||
if not self.path_pattern.match(request.url.path):
|
||||
return False
|
||||
|
||||
# 检查 HTTP 方法过滤
|
||||
if self.methods:
|
||||
if request.method not in self.methods:
|
||||
return False
|
||||
|
||||
# 检查条件函数
|
||||
if self.condition:
|
||||
try:
|
||||
if not self.condition(request):
|
||||
return False
|
||||
except Exception:
|
||||
# 条件函数执行失败时跳过该中间件
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
|
||||
"""执行中间件逻辑"""
|
||||
# 如果不符合过滤条件，直接跳过
|
||||
if not self._should_process(request):
|
||||
return await call_next(request)
|
||||
|
||||
# 创建异步 next_handler 包装器
|
||||
async def async_next_handler(req: Request) -> Response:
|
||||
return await call_next(req)
|
||||
|
||||
# 检查中间件函数是否是异步函数
|
||||
if asyncio.iscoroutinefunction(self.middleware_func):
|
||||
# 异步中间件（推荐方式）
|
||||
response = await self.middleware_func(request, async_next_handler)
|
||||
else:
|
||||
# 同步中间件支持（为了向后兼容）
|
||||
# 注意：同步中间件接收的 next_handler 会返回协程，需要手动处理
|
||||
# 建议使用异步中间件以获得更好的性能
|
||||
def sync_wrapper(req: Request):
|
||||
"""同步包装器：返回协程对象，需要调用者手动 await"""
|
||||
return call_next(req)
|
||||
|
||||
# 调用同步中间件函数
|
||||
result = self.middleware_func(request, sync_wrapper)
|
||||
|
||||
# 处理返回值
|
||||
if asyncio.iscoroutine(result):
|
||||
# 如果返回协程，等待它
|
||||
response = await result
|
||||
elif isinstance(result, (Response, StreamingResponse)):
|
||||
# 如果已经是 Response，直接使用
|
||||
response = result
|
||||
else:
|
||||
# 其他情况，尝试转换
|
||||
from fastapi.responses import JSONResponse
|
||||
response = JSONResponse(content=result)
|
||||
|
||||
# 确保返回 Response 对象
|
||||
if not isinstance(response, (Response, StreamingResponse)):
|
||||
# 如果返回的不是 Response，尝试转换为 Response
|
||||
from fastapi.responses import JSONResponse
|
||||
response = JSONResponse(content=response)
|
||||
|
||||
return response
|
||||
|
||||
|
||||
class ResponseFormatterMiddleware(BaseHTTPMiddleware):
|
||||
"""响应格式化中间件
|
||||
|
||||
自动将路由返回的数据包装为统一的 REST API 格式：
|
||||
{
|
||||
"success": true,
|
||||
"code": 200,
|
||||
"message": "操作成功",
|
||||
"data": {...}
|
||||
}
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
app,
|
||||
exclude_paths: Optional[List[str]] = None,
|
||||
auto_wrap: bool = True
|
||||
):
|
||||
"""
|
||||
初始化响应格式化中间件
|
||||
|
||||
Args:
|
||||
app: FastAPI 应用实例
|
||||
exclude_paths: 排除的路径列表（这些路径不进行格式化）
|
||||
auto_wrap: 是否自动包装响应
|
||||
"""
|
||||
super().__init__(app)
|
||||
self.exclude_paths = exclude_paths or []
|
||||
self.auto_wrap = auto_wrap
|
||||
|
||||
# 默认排除的路径（系统路径和文档路径）
|
||||
default_excludes = [
|
||||
"/docs",
|
||||
"/openapi.json",
|
||||
"/redoc",
|
||||
"/health",
|
||||
"/health/ready",
|
||||
"/health/live"
|
||||
]
|
||||
|
||||
# 合并排除路径
|
||||
self.exclude_paths = list(set(self.exclude_paths + default_excludes))
|
||||
|
||||
def _should_format(self, request: Request) -> bool:
|
||||
"""判断是否应该格式化响应"""
|
||||
if not self.auto_wrap:
|
||||
return False
|
||||
|
||||
path = request.url.path
|
||||
|
||||
# 检查是否在排除列表中
|
||||
for exclude_path in self.exclude_paths:
|
||||
if path == exclude_path or path.startswith(exclude_path + "/"):
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def _is_formatted(self, content: dict) -> bool:
|
||||
"""检查响应是否已经是统一格式"""
|
||||
if not isinstance(content, dict):
|
||||
return False
|
||||
|
||||
# 检查是否包含统一格式的必须字段（message 和 data 可选）
|
||||
required_fields = {"success", "code"}
|
||||
return all(field in content for field in required_fields)
|
||||
|
||||
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
|
||||
"""处理请求并格式化响应"""
|
||||
response = await call_next(request)
|
||||
|
||||
# 如果不需要格式化，直接返回
|
||||
if not self._should_format(request):
|
||||
return response
|
||||
|
||||
# 检查响应类型
|
||||
# FastAPI 可能返回 JSONResponse、StreamingResponse 或 _StreamingResponse
|
||||
# 我们需要处理所有可能包含 JSON 内容的响应
|
||||
|
||||
# 检查是否是 JSONResponse
|
||||
is_json_response = isinstance(response, JSONResponse)
|
||||
|
||||
# 如果不是 JSONResponse，检查是否可能是 JSON 响应
|
||||
if not is_json_response:
|
||||
# 检查 media_type 是否为 JSON
|
||||
if hasattr(response, 'media_type') and response.media_type:
|
||||
media_type_lower = response.media_type.lower()
|
||||
if 'json' in media_type_lower:
|
||||
is_json_response = True
|
||||
else:
|
||||
return response
|
||||
elif isinstance(response, (StreamingResponse, Response)):
|
||||
# StreamingResponse 或 Response 如果没有设置 media_type，尝试读取并判断
|
||||
# 对于未知类型的响应，我们尝试读取内容并判断是否为 JSON
|
||||
# 允许继续处理，尝试读取响应体
|
||||
pass
|
||||
else:
|
||||
# 如果既不是 JSONResponse 也没有 media_type，跳过
|
||||
return response
|
||||
|
||||
# 检查响应是否有 body_iterator（所有响应都应该有）
|
||||
if not hasattr(response, 'body_iterator'):
|
||||
return response
|
||||
|
||||
# 获取响应内容
|
||||
try:
|
||||
# 读取响应体 - 使用正确的方式处理流式响应
|
||||
body = b""
|
||||
async for chunk in response.body_iterator:
|
||||
body += chunk
|
||||
|
||||
# 如果没有响应体，直接返回空响应
|
||||
if not body:
|
||||
return response
|
||||
|
||||
# 解析 JSON
|
||||
try:
|
||||
content = json.loads(body.decode('utf-8'))
|
||||
except json.JSONDecodeError:
|
||||
# 如果不是有效的 JSON，返回原始响应
|
||||
return response
|
||||
|
||||
# 如果已经是统一格式，重新构建响应并返回
|
||||
if self._is_formatted(content):
|
||||
# 移除 Content-Length，让服务器重新计算
|
||||
headers = dict(response.headers)
|
||||
headers.pop('content-length', None)
|
||||
return JSONResponse(
|
||||
content=content,
|
||||
status_code=response.status_code,
|
||||
headers=headers
|
||||
)
|
||||
|
||||
# 根据状态码判断成功或失败
|
||||
status_code = response.status_code
|
||||
is_success = 200 <= status_code < 400
|
||||
|
||||
# 自动包装响应
|
||||
from myboot.web.response import ResponseWrapper
|
||||
|
||||
if is_success:
|
||||
# 成功响应
|
||||
formatted_content = ResponseWrapper.success(
|
||||
data=content,
|
||||
message="操作成功",
|
||||
code=status_code
|
||||
)
|
||||
else:
|
||||
# 错误响应（通常由异常处理器处理，但以防万一）
|
||||
message = content.get("message", "操作失败") if isinstance(content, dict) else "操作失败"
|
||||
formatted_content = ResponseWrapper.error(
|
||||
message=message,
|
||||
code=status_code,
|
||||
data=content if isinstance(content, dict) else {"error": str(content)}
|
||||
)
|
||||
|
||||
# 移除 Content-Length，让服务器重新计算新的响应长度
|
||||
headers = dict(response.headers)
|
||||
headers.pop('content-length', None)
|
||||
|
||||
return JSONResponse(
|
||||
content=formatted_content,
|
||||
status_code=status_code,
|
||||
headers=headers
|
||||
)
|
||||
|
||||
except (AttributeError, UnicodeDecodeError, StopAsyncIteration) as e:
|
||||
# 如果无法解析，返回原始响应
|
||||
# 这里可以记录日志，但为了不影响正常响应，静默处理
|
||||
return response
|
||||
except Exception as e:
|
||||
# 捕获所有其他异常，避免中间件崩溃
|
||||
# 在生产环境中可以记录日志
|
||||
return response
|
||||
@@ -0,0 +1,204 @@
|
||||
"""
|
||||
Web 数据模型
|
||||
|
||||
提供请求和响应的数据模型
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
|
||||
class BaseResponse(BaseModel):
|
||||
"""基础响应模型"""
|
||||
|
||||
success: bool = Field(default=True, description="是否成功")
|
||||
message: str = Field(default="操作成功", description="响应消息")
|
||||
data: Optional[Any] = Field(default=None, description="响应数据")
|
||||
timestamp: datetime = Field(default_factory=datetime.now, description="时间戳")
|
||||
|
||||
class ErrorResponse(BaseResponse):
|
||||
"""错误响应模型"""
|
||||
|
||||
success: bool = Field(default=False, description="是否成功")
|
||||
message: str = Field(default="操作失败", description="错误消息")
|
||||
error_code: Optional[str] = Field(default=None, description="错误代码")
|
||||
details: Optional[Dict[str, Any]] = Field(default=None, description="错误详情")
|
||||
|
||||
|
||||
class PaginationRequest(BaseModel):
|
||||
"""分页请求模型"""
|
||||
|
||||
page: int = Field(default=1, ge=1, description="页码")
|
||||
size: int = Field(default=10, ge=1, le=100, description="每页大小")
|
||||
sort: Optional[str] = Field(default=None, description="排序字段")
|
||||
order: str = Field(default="asc", pattern="^(asc|desc)$", description="排序方向")
|
||||
|
||||
@field_validator('page')
|
||||
def validate_page(cls, v):
|
||||
if v < 1:
|
||||
raise ValueError('页码必须大于 0')
|
||||
return v
|
||||
|
||||
@field_validator('size')
|
||||
def validate_size(cls, v):
|
||||
if v < 1 or v > 100:
|
||||
raise ValueError('每页大小必须在 1-100 之间')
|
||||
return v
|
||||
|
||||
|
||||
class PaginationResponse(BaseResponse):
|
||||
"""分页响应模型"""
|
||||
|
||||
data: List[Any] = Field(default=[], description="数据列表")
|
||||
pagination: Dict[str, Any] = Field(description="分页信息")
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
data: List[Any],
|
||||
total: int,
|
||||
page: int,
|
||||
size: int,
|
||||
message: str = "查询成功"
|
||||
) -> "PaginationResponse":
|
||||
"""创建分页响应"""
|
||||
total_pages = (total + size - 1) // size
|
||||
|
||||
pagination = {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"size": size,
|
||||
"total_pages": total_pages,
|
||||
"has_next": page < total_pages,
|
||||
"has_prev": page > 1
|
||||
}
|
||||
|
||||
return cls(
|
||||
success=True,
|
||||
message=message,
|
||||
data=data,
|
||||
pagination=pagination
|
||||
)
|
||||
|
||||
|
||||
class HealthCheckResponse(BaseModel):
|
||||
"""健康检查响应模型"""
|
||||
|
||||
status: str = Field(description="服务状态")
|
||||
app: str = Field(description="应用名称")
|
||||
version: str = Field(description="应用版本")
|
||||
uptime: str = Field(description="运行时间")
|
||||
timestamp: datetime = Field(default_factory=datetime.now, description="检查时间")
|
||||
|
||||
class RequestInfo(BaseModel):
|
||||
"""请求信息模型"""
|
||||
|
||||
method: str = Field(description="HTTP 方法")
|
||||
url: str = Field(description="请求 URL")
|
||||
headers: Dict[str, str] = Field(description="请求头")
|
||||
query_params: Dict[str, Any] = Field(default={}, description="查询参数")
|
||||
path_params: Dict[str, Any] = Field(default={}, description="路径参数")
|
||||
body: Optional[Any] = Field(default=None, description="请求体")
|
||||
client_ip: Optional[str] = Field(default=None, description="客户端 IP")
|
||||
user_agent: Optional[str] = Field(default=None, description="用户代理")
|
||||
timestamp: datetime = Field(default_factory=datetime.now, description="请求时间")
|
||||
|
||||
class ResponseInfo(BaseModel):
|
||||
"""响应信息模型"""
|
||||
|
||||
status_code: int = Field(description="状态码")
|
||||
headers: Dict[str, str] = Field(description="响应头")
|
||||
body: Optional[Any] = Field(default=None, description="响应体")
|
||||
process_time: float = Field(description="处理时间（秒）")
|
||||
timestamp: datetime = Field(default_factory=datetime.now, description="响应时间")
|
||||
|
||||
class ValidationErrorDetail(BaseModel):
|
||||
"""验证错误详情模型"""
|
||||
|
||||
field: str = Field(description="字段名")
|
||||
message: str = Field(description="错误消息")
|
||||
value: Any = Field(description="字段值")
|
||||
type: str = Field(description="错误类型")
|
||||
|
||||
|
||||
class ValidationErrorResponse(ErrorResponse):
|
||||
"""验证错误响应模型"""
|
||||
|
||||
message: str = Field(default="请求参数验证失败", description="错误消息")
|
||||
error_code: str = Field(default="VALIDATION_ERROR", description="错误代码")
|
||||
details: List[ValidationErrorDetail] = Field(description="验证错误详情")
|
||||
|
||||
|
||||
class APIError(BaseModel):
|
||||
"""API 错误模型"""
|
||||
|
||||
code: str = Field(description="错误代码")
|
||||
message: str = Field(description="错误消息")
|
||||
details: Optional[Dict[str, Any]] = Field(default=None, description="错误详情")
|
||||
timestamp: datetime = Field(default_factory=datetime.now, description="错误时间")
|
||||
|
||||
class SuccessResponse(BaseResponse):
|
||||
"""成功响应模型"""
|
||||
|
||||
success: bool = Field(default=True, description="是否成功")
|
||||
message: str = Field(default="操作成功", description="成功消息")
|
||||
data: Optional[Any] = Field(default=None, description="响应数据")
|
||||
|
||||
|
||||
class CreatedResponse(SuccessResponse):
|
||||
"""创建成功响应模型"""
|
||||
|
||||
message: str = Field(default="创建成功", description="成功消息")
|
||||
status_code: int = Field(default=201, description="状态码")
|
||||
|
||||
|
||||
class UpdatedResponse(SuccessResponse):
|
||||
"""更新成功响应模型"""
|
||||
|
||||
message: str = Field(default="更新成功", description="成功消息")
|
||||
status_code: int = Field(default=200, description="状态码")
|
||||
|
||||
|
||||
class DeletedResponse(SuccessResponse):
|
||||
"""删除成功响应模型"""
|
||||
|
||||
message: str = Field(default="删除成功", description="成功消息")
|
||||
status_code: int = Field(default=204, description="状态码")
|
||||
data: Optional[Any] = Field(default=None, description="响应数据")
|
||||
|
||||
|
||||
# 便捷函数
|
||||
def success_response(data: Any = None, message: str = "操作成功") -> SuccessResponse:
|
||||
"""创建成功响应"""
|
||||
return SuccessResponse(data=data, message=message)
|
||||
|
||||
|
||||
def error_response(
|
||||
message: str = "操作失败",
|
||||
error_code: str = "UNKNOWN_ERROR",
|
||||
details: Optional[Dict[str, Any]] = None
|
||||
) -> ErrorResponse:
|
||||
"""创建错误响应"""
|
||||
return ErrorResponse(
|
||||
message=message,
|
||||
error_code=error_code,
|
||||
details=details
|
||||
)
|
||||
|
||||
|
||||
def validation_error_response(errors: List[ValidationErrorDetail]) -> ValidationErrorResponse:
|
||||
"""创建验证错误响应"""
|
||||
return ValidationErrorResponse(details=errors)
|
||||
|
||||
|
||||
def pagination_response(
|
||||
data: List[Any],
|
||||
total: int,
|
||||
page: int,
|
||||
size: int,
|
||||
message: str = "查询成功"
|
||||
) -> PaginationResponse:
|
||||
"""创建分页响应"""
|
||||
return PaginationResponse.create(data, total, page, size, message)
|
||||
@@ -0,0 +1,173 @@
|
||||
"""
|
||||
REST API 响应格式封装
|
||||
|
||||
提供统一的 REST API 响应格式，确保所有 API 返回一致的格式：
|
||||
{
|
||||
"success": true/false,
|
||||
"code": 200,
|
||||
"message": "操作成功",
|
||||
"data": {...}
|
||||
}
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, Optional, Union
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
|
||||
class ApiResponse(BaseModel):
|
||||
"""统一的 REST API 响应格式"""
|
||||
|
||||
model_config = ConfigDict(exclude_none=True)
|
||||
|
||||
success: bool = Field(description="是否成功")
|
||||
code: int = Field(description="HTTP 状态码")
|
||||
message: Optional[str] = Field(default=None, description="响应消息")
|
||||
data: Optional[Any] = Field(default=None, description="响应数据")
|
||||
|
||||
|
||||
class ResponseWrapper:
|
||||
"""响应包装器
|
||||
|
||||
提供便捷方法创建统一格式的响应
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def success(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None,
|
||||
code: int = 200
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
创建成功响应
|
||||
|
||||
Args:
|
||||
data: 响应数据
|
||||
message: 响应消息
|
||||
code: HTTP 状态码
|
||||
|
||||
Returns:
|
||||
统一格式的响应字典
|
||||
"""
|
||||
result = {
|
||||
"success": True,
|
||||
"code": code,
|
||||
}
|
||||
if message is not None:
|
||||
result["message"] = message
|
||||
if data is not None:
|
||||
result["data"] = data
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def error(
|
||||
message: Optional[str] = None,
|
||||
code: int = 500,
|
||||
data: Optional[Any] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
创建错误响应
|
||||
|
||||
Args:
|
||||
message: 错误消息
|
||||
code: HTTP 状态码
|
||||
data: 错误详情数据
|
||||
|
||||
Returns:
|
||||
统一格式的响应字典
|
||||
"""
|
||||
result = {
|
||||
"success": False,
|
||||
"code": code,
|
||||
}
|
||||
if message is not None:
|
||||
result["message"] = message
|
||||
if data is not None:
|
||||
result["data"] = data
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def created(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""创建成功响应（201）"""
|
||||
return ResponseWrapper.success(data=data, message=message, code=201)
|
||||
|
||||
@staticmethod
|
||||
def updated(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""更新成功响应（200）"""
|
||||
return ResponseWrapper.success(data=data, message=message, code=200)
|
||||
|
||||
@staticmethod
|
||||
def deleted(
|
||||
message: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""删除成功响应（200）"""
|
||||
return ResponseWrapper.success(data=None, message=message, code=200)
|
||||
|
||||
@staticmethod
|
||||
def no_content() -> Dict[str, Any]:
|
||||
"""无内容响应（204）"""
|
||||
return ResponseWrapper.success(data=None, message=None, code=204)
|
||||
|
||||
@staticmethod
|
||||
def pagination(
|
||||
data: list,
|
||||
total: int,
|
||||
page: int,
|
||||
size: int,
|
||||
message: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
创建分页响应
|
||||
|
||||
Args:
|
||||
data: 数据列表
|
||||
total: 总记录数
|
||||
page: 当前页码
|
||||
size: 每页大小
|
||||
message: 响应消息
|
||||
|
||||
Returns:
|
||||
统一格式的分页响应
|
||||
"""
|
||||
total_pages = (total + size - 1) // size if size > 0 else 0
|
||||
pagination_data = {
|
||||
"list": data,
|
||||
"pagination": {
|
||||
"total": total,
|
||||
"page": page,
|
||||
"size": size,
|
||||
"totalPages": total_pages,
|
||||
"hasNext": page < total_pages,
|
||||
"hasPrev": page > 1
|
||||
}
|
||||
}
|
||||
return ResponseWrapper.success(data=pagination_data, message=message, code=200)
|
||||
|
||||
@staticmethod
|
||||
def wrap(data: Any, message: Optional[str] = None, code: int = 200) -> Dict[str, Any]:
|
||||
"""
|
||||
包装任意数据为统一格式
|
||||
|
||||
Args:
|
||||
data: 要包装的数据
|
||||
message: 响应消息（如果为 None，返回中不包含该字段）
|
||||
code: HTTP 状态码
|
||||
|
||||
Returns:
|
||||
统一格式的响应字典
|
||||
"""
|
||||
# 如果已经是统一格式，直接返回
|
||||
if isinstance(data, dict) and all(key in data for key in ["success", "code"]):
|
||||
return data
|
||||
|
||||
return ResponseWrapper.success(data=data, message=message, code=code)
|
||||
|
||||
|
||||
# 全局响应包装器实例
|
||||
response = ResponseWrapper()
|
||||
|
||||
+170
@@ -0,0 +1,170 @@
|
||||
[project]
|
||||
name = "myboot"
|
||||
version = "0.2.0"
|
||||
description = "类似 Spring Boot 的 Python 快速开发框架"
|
||||
authors = [
|
||||
{name = "TrumanDu", email = "truman.p.du@qq.com"}
|
||||
]
|
||||
readme = "README.md"
|
||||
license = {text = "MIT"}
|
||||
requires-python = ">=3.9"
|
||||
keywords = ["framework", "web", "api", "scheduler", "logging", "config"]
|
||||
classifiers = [
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Topic :: Software Development :: Libraries :: Application Frameworks",
|
||||
"Topic :: Internet :: WWW/HTTP :: HTTP Servers",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"fastapi>=0.104.0",
|
||||
"hypercorn>=0.14.0",
|
||||
"pydantic>=2.0.0",
|
||||
"pyyaml>=6.0",
|
||||
"APScheduler>=3.10.0",
|
||||
"python-multipart>=0.0.6",
|
||||
"jinja2>=3.1.0",
|
||||
"python-dotenv>=1.0.0",
|
||||
"loguru>=0.7.0",
|
||||
"colorama>=0.4.6",
|
||||
"click>=8.0.0",
|
||||
"dynaconf>=3.2.0",
|
||||
"requests>=2.28.0",
|
||||
"dependency-injector>=4.41.0",
|
||||
"pytz>=2024.1",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=7.0",
|
||||
"pytest-cov>=4.0",
|
||||
"pytest-asyncio>=0.21.0",
|
||||
"black>=23.0",
|
||||
"isort>=5.12",
|
||||
"flake8>=6.0",
|
||||
"mypy>=1.0",
|
||||
"pre-commit>=3.0",
|
||||
]
|
||||
|
||||
test = [
|
||||
"pytest>=7.0",
|
||||
"pytest-cov>=4.0",
|
||||
"pytest-asyncio>=0.21.0",
|
||||
"httpx>=0.24.0",
|
||||
"pytest-mock>=3.10",
|
||||
"prometheus-client>=0.19.0",
|
||||
]
|
||||
|
||||
metrics = [
|
||||
"prometheus-client>=0.19.0",
|
||||
]
|
||||
|
||||
docs = [
|
||||
"sphinx>=7.0",
|
||||
"sphinx-rtd-theme>=1.3",
|
||||
"myst-parser>=2.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
myboot = "myboot.cli:cli"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["myboot"]
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
include = [
|
||||
"/myboot",
|
||||
"/tests",
|
||||
"/pyproject.toml",
|
||||
]
|
||||
|
||||
[tool.black]
|
||||
line-length = 88
|
||||
target-version = ['py39', 'py310', 'py311', 'py312', 'py313']
|
||||
include = '\.pyi?$'
|
||||
extend-exclude = '''
|
||||
/(
|
||||
# directories
|
||||
\.eggs
|
||||
| \.git
|
||||
| \.hg
|
||||
| \.mypy_cache
|
||||
| \.tox
|
||||
| \.venv
|
||||
| build
|
||||
| dist
|
||||
)/
|
||||
'''
|
||||
|
||||
[tool.isort]
|
||||
profile = "black"
|
||||
multi_line_output = 3
|
||||
line_length = 88
|
||||
known_first_party = ["myboot"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.9"
|
||||
warn_return_any = true
|
||||
warn_unused_configs = true
|
||||
disallow_untyped_defs = true
|
||||
disallow_incomplete_defs = true
|
||||
check_untyped_defs = true
|
||||
disallow_untyped_decorators = true
|
||||
no_implicit_optional = true
|
||||
warn_redundant_casts = true
|
||||
warn_unused_ignores = true
|
||||
warn_no_return = true
|
||||
warn_unreachable = true
|
||||
strict_equality = true
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
python_files = ["test_*.py", "*_test.py"]
|
||||
python_classes = ["Test*"]
|
||||
python_functions = ["test_*"]
|
||||
addopts = [
|
||||
"--strict-markers",
|
||||
"--strict-config",
|
||||
"--verbose",
|
||||
"--tb=short",
|
||||
]
|
||||
markers = [
|
||||
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
|
||||
"integration: marks tests as integration tests",
|
||||
"unit: marks tests as unit tests",
|
||||
]
|
||||
|
||||
[tool.coverage.run]
|
||||
source = ["myboot"]
|
||||
omit = [
|
||||
"*/tests/*",
|
||||
"*/test_*",
|
||||
"*/__pycache__/*",
|
||||
"*/venv/*",
|
||||
"*/.venv/*",
|
||||
]
|
||||
|
||||
[tool.coverage.report]
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"def __repr__",
|
||||
"if self.debug:",
|
||||
"if settings.DEBUG",
|
||||
"raise AssertionError",
|
||||
"raise NotImplementedError",
|
||||
"if 0:",
|
||||
"if __name__ == .__main__.:",
|
||||
"class .*\\bProtocol\\):",
|
||||
"@(abc\\.)?abstractmethod",
|
||||
]
|
||||
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
测试模块
|
||||
"""
|
||||
@@ -0,0 +1,41 @@
|
||||
"""
|
||||
pytest 公共 fixture
|
||||
|
||||
提供临时配置文件、干净环境变量等测试基础设施。
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import textwrap
|
||||
|
||||
import pytest
|
||||
|
||||
# 确保从源码导入 myboot（而非已安装版本）
|
||||
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
if PROJECT_ROOT not in sys.path:
|
||||
sys.path.insert(0, PROJECT_ROOT)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_config_file(tmp_path):
|
||||
"""创建临时 YAML 配置文件，返回写入函数
|
||||
|
||||
用法:
|
||||
path = tmp_config_file("app:\\n name: demo")
|
||||
"""
|
||||
|
||||
def _write(content: str, filename: str = "config.yaml") -> str:
|
||||
config_path = tmp_path / filename
|
||||
config_path.write_text(textwrap.dedent(content), encoding="utf-8")
|
||||
return str(config_path)
|
||||
|
||||
return _write
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clean_myboot_env(monkeypatch):
|
||||
"""清除 myboot 相关环境变量，避免测试间相互污染"""
|
||||
for key in list(os.environ.keys()):
|
||||
if key.startswith("MYBOOT_") or key == "CONFIG_FILE":
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
return monkeypatch
|
||||
@@ -0,0 +1,57 @@
|
||||
"""多 worker 集成测试 fixture 应用入口
|
||||
|
||||
通过 subprocess 启动：python main.py
|
||||
环境变量：
|
||||
MW_PORT 监听端口（必需）
|
||||
MW_WORKERS worker 数量（默认 2）
|
||||
MW_HOOK_DIR worker 钩子记录目录
|
||||
MW_JOB_DIR 调度任务记录目录
|
||||
|
||||
注意：本模块在 Windows spawn 模式下会被 multiprocessing 以 __mp_main__
|
||||
重新导入，模块级代码（路径注入、配置覆盖）会在每个 worker 中重新执行，
|
||||
这是预期行为；app.run() 受 __main__ 守卫保护只在父进程执行。
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
FIXTURE_DIR = Path(__file__).parent.absolute()
|
||||
REPO_ROOT = FIXTURE_DIR.parent.parent.parent.parent
|
||||
|
||||
# 确保能导入 mwapp 包与源码版 myboot
|
||||
sys.path.insert(0, str(FIXTURE_DIR))
|
||||
if str(REPO_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from myboot.core import auto_configuration # noqa: E402
|
||||
from myboot.core.application import Application # noqa: E402
|
||||
|
||||
app = Application(
|
||||
name="multiworker-fixture",
|
||||
auto_configuration=True,
|
||||
auto_discover_package="mwapp",
|
||||
)
|
||||
|
||||
# 自动配置管理器默认把 myboot 源码仓库根当作 app_root，
|
||||
# 这里指向 fixture 目录，使 "mwapp" 包能被发现；
|
||||
# 禁用缓存避免在仓库中残留 .myboot_cache_*.json
|
||||
auto_configuration._auto_configuration_manager.app_root = str(FIXTURE_DIR)
|
||||
auto_configuration._auto_configuration_manager.use_cache = False
|
||||
|
||||
PORT = int(os.environ.get("MW_PORT", "8765"))
|
||||
WORKERS = int(os.environ.get("MW_WORKERS", "2"))
|
||||
|
||||
# 显式覆盖配置——仓库 conf/config.yaml（server.workers=1 等）的优先级
|
||||
# 高于 run() 入参，必须在配置层覆盖
|
||||
app.config.set("server.host", "127.0.0.1")
|
||||
app.config.set("server.port", PORT)
|
||||
app.config.set("server.workers", WORKERS)
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run(
|
||||
host="127.0.0.1",
|
||||
port=PORT,
|
||||
workers=WORKERS,
|
||||
app_path="main:app.get_fastapi_app()",
|
||||
)
|
||||
@@ -0,0 +1 @@
|
||||
"""多 worker 集成测试 fixture 应用包"""
|
||||
@@ -0,0 +1,27 @@
|
||||
"""GET /api/whoami：返回当前 worker 与注入的 client 实例信息"""
|
||||
|
||||
import os
|
||||
|
||||
from myboot.core.application import app as current_app
|
||||
from myboot.core.decorators import rest_controller, get
|
||||
|
||||
from mwapp.clients import InstanceClient
|
||||
|
||||
|
||||
@rest_controller('/api')
|
||||
class WhoamiController:
|
||||
|
||||
def __init__(self, instance_client: InstanceClient):
|
||||
self.instance_client = instance_client
|
||||
|
||||
@get('/whoami')
|
||||
def whoami(self):
|
||||
a = current_app()
|
||||
return {
|
||||
"worker_id": a.worker_id,
|
||||
"worker_count": a.worker_count,
|
||||
"is_primary": a.is_primary_worker,
|
||||
"pid": os.getpid(),
|
||||
"client_instance_id": self.instance_client.instance_id,
|
||||
"client_created_pid": self.instance_client.created_pid,
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
"""测试客户端：记录实例 id 与创建进程 pid，用于断言每个 worker 独立实例化"""
|
||||
|
||||
import os
|
||||
import uuid
|
||||
|
||||
from myboot.core.decorators import client
|
||||
|
||||
|
||||
@client()
|
||||
class InstanceClient:
|
||||
"""自动注册为 'instance_client'"""
|
||||
|
||||
def __init__(self):
|
||||
self.instance_id = uuid.uuid4().hex
|
||||
self.created_pid = os.getpid()
|
||||
@@ -0,0 +1,33 @@
|
||||
"""worker 生命周期钩子：每个 worker 各写一个文件，供测试断言触发次数/进程"""
|
||||
|
||||
import os
|
||||
|
||||
from myboot.core.application import app as current_app
|
||||
from myboot.core.decorators import on_worker_start, on_worker_stop
|
||||
|
||||
|
||||
@on_worker_start
|
||||
def record_worker_start():
|
||||
hook_dir = os.environ.get("MW_HOOK_DIR")
|
||||
if not hook_dir:
|
||||
return
|
||||
a = current_app()
|
||||
instance_client = a.get_client("instance_client")
|
||||
instance_id = instance_client.instance_id if instance_client else ""
|
||||
path = os.path.join(hook_dir, f"start_{os.getpid()}.txt")
|
||||
# 追加写：若同一 worker 误触发两次，文件会出现两行，测试可检测到
|
||||
with open(path, "a", encoding="utf-8") as f:
|
||||
f.write(f"{os.getpid()},{a.worker_id},{instance_id}\n")
|
||||
|
||||
|
||||
@on_worker_stop
|
||||
def record_worker_stop():
|
||||
# 注意：Windows 多 worker 模式下父进程 terminate() 硬终止 worker，
|
||||
# 此钩子可能不触发（测试在 Windows 上跳过 stop 断言）
|
||||
hook_dir = os.environ.get("MW_HOOK_DIR")
|
||||
if not hook_dir:
|
||||
return
|
||||
a = current_app()
|
||||
path = os.path.join(hook_dir, f"stop_{os.getpid()}.txt")
|
||||
with open(path, "a", encoding="utf-8") as f:
|
||||
f.write(f"{os.getpid()},{a.worker_id}\n")
|
||||
@@ -0,0 +1,30 @@
|
||||
"""定时任务组件：每秒记录执行进程 pid
|
||||
|
||||
- record_pid: 默认任务，用于断言只在 primary worker 运行
|
||||
- record_pid_all_workers: all_workers=True 任务，用于断言在每个 worker 都运行
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from myboot.core.decorators import component, interval
|
||||
|
||||
|
||||
def _write_tick(prefix: str) -> None:
|
||||
job_dir = os.environ.get("MW_JOB_DIR")
|
||||
if not job_dir:
|
||||
return
|
||||
path = os.path.join(job_dir, f"{prefix}_{os.getpid()}.txt")
|
||||
with open(path, "a", encoding="utf-8") as f:
|
||||
f.write("tick\n")
|
||||
|
||||
|
||||
@component()
|
||||
class MwJobs:
|
||||
|
||||
@interval(seconds=1)
|
||||
def record_pid(self):
|
||||
_write_tick("job")
|
||||
|
||||
@interval(seconds=1, all_workers=True)
|
||||
def record_pid_all_workers(self):
|
||||
_write_tick("awjob")
|
||||
@@ -0,0 +1,93 @@
|
||||
"""
|
||||
Prometheus multiproc 多进程聚合集成测试（F5）
|
||||
|
||||
父进程设置 PROMETHEUS_MULTIPROC_DIR 后 spawn 两个子进程，各自
|
||||
get_counter().inc() 一次后退出；父进程通过 make_metrics_asgi_app()
|
||||
（MultiProcessCollector）scrape，断言聚合值为 2 且目录下存在多个
|
||||
pid 的 db 文件。
|
||||
|
||||
注意：Windows 多 worker 不支持 multiproc 聚合，本文件在 win32 跳过，
|
||||
由 CI（Linux）执行。
|
||||
|
||||
运行方式: uv run pytest tests/integration -m integration
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import multiprocessing
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = [
|
||||
pytest.mark.integration,
|
||||
pytest.mark.skipif(
|
||||
sys.platform == "win32",
|
||||
reason="Windows 不支持 Prometheus multiproc 文件聚合",
|
||||
),
|
||||
]
|
||||
|
||||
COUNTER_NAME = "myboot_multiproc_itest_total"
|
||||
|
||||
|
||||
def _child_inc(multiproc_dir: str) -> None:
|
||||
"""子进程入口：multiproc 模式下计数一次
|
||||
|
||||
必须为模块级函数以兼容 spawn 启动方式。env 在 import
|
||||
prometheus_client 之前设置，确保选用 mmap 文件后端。
|
||||
"""
|
||||
os.environ["PROMETHEUS_MULTIPROC_DIR"] = multiproc_dir
|
||||
|
||||
from myboot import metrics
|
||||
|
||||
counter = metrics.get_counter(COUNTER_NAME, "multiproc integration test")
|
||||
counter.inc()
|
||||
metrics.mark_current_process_dead()
|
||||
|
||||
|
||||
def test_multiproc_counter_aggregation(tmp_path, monkeypatch):
|
||||
multiproc_dir = str(tmp_path)
|
||||
monkeypatch.setenv("PROMETHEUS_MULTIPROC_DIR", multiproc_dir)
|
||||
|
||||
# 用 spawn 确保子进程在设置 env 后全新 import prometheus_client
|
||||
# （fork 会继承父进程可能已按非 multiproc 模式初始化的模块状态）
|
||||
ctx = multiprocessing.get_context("spawn")
|
||||
procs = [
|
||||
ctx.Process(target=_child_inc, args=(multiproc_dir,)) for _ in range(2)
|
||||
]
|
||||
for p in procs:
|
||||
p.start()
|
||||
for p in procs:
|
||||
p.join(timeout=60)
|
||||
assert p.exitcode == 0
|
||||
|
||||
# 目录下应有来自多个 pid 的 counter db 文件
|
||||
db_files = list(tmp_path.glob("counter_*.db"))
|
||||
assert len(db_files) >= 2, f"期望至少 2 个 counter db 文件，实际: {db_files}"
|
||||
|
||||
# 父进程 scrape：MultiProcessCollector 聚合两个子进程的计数
|
||||
import httpx
|
||||
|
||||
from myboot.metrics import make_metrics_asgi_app
|
||||
|
||||
asgi_app = make_metrics_asgi_app()
|
||||
|
||||
async def _scrape() -> str:
|
||||
transport = httpx.ASGITransport(app=asgi_app)
|
||||
async with httpx.AsyncClient(
|
||||
transport=transport, base_url="http://testserver"
|
||||
) as client:
|
||||
resp = await client.get("/")
|
||||
assert resp.status_code == 200
|
||||
return resp.text
|
||||
|
||||
text = asyncio.run(_scrape())
|
||||
|
||||
lines = [
|
||||
line
|
||||
for line in text.splitlines()
|
||||
if line.startswith(COUNTER_NAME) and not line.startswith("#")
|
||||
]
|
||||
assert lines, f"scrape 输出中未找到 {COUNTER_NAME}: {text[:500]}"
|
||||
value = float(lines[0].rsplit(" ", 1)[-1])
|
||||
assert value == 2.0
|
||||
@@ -0,0 +1,240 @@
|
||||
"""
|
||||
多 worker 真实进程集成测试（issue #11）
|
||||
|
||||
启动 tests/integration/fixtures/multiworker 下的 fixture 应用（workers=2），
|
||||
通过 HTTP 与临时文件断言：
|
||||
|
||||
1. 每个 worker 进程独立实例化 client（实例 id 与创建 pid 互不相同）
|
||||
2. @on_worker_start 钩子在每个 worker 各触发一次
|
||||
3. 调度任务只在 primary worker 执行
|
||||
4. （POSIX）@on_worker_stop 钩子在优雅关闭时每个 worker 各触发一次；
|
||||
Windows 下父进程 terminate() 硬终止 worker，跳过 stop 断言
|
||||
|
||||
运行方式: uv run pytest tests/integration -m integration
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
FIXTURE_DIR = Path(__file__).parent / "fixtures" / "multiworker"
|
||||
WORKERS = 2
|
||||
STARTUP_TIMEOUT = 120 # spawn 模式启动较慢，宽松等待（轮询，就绪即继续）
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
return sock.getsockname()[1]
|
||||
|
||||
|
||||
def _http_get_json(url: str, timeout: float = 5.0):
|
||||
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||
payload = json.loads(resp.read().decode("utf-8"))
|
||||
# ResponseFormatterMiddleware 会包一层 {success, code, message, data}
|
||||
if isinstance(payload, dict) and "data" in payload and "success" in payload:
|
||||
return payload["data"]
|
||||
return payload
|
||||
|
||||
|
||||
def _kill_process_tree(proc: subprocess.Popen) -> None:
|
||||
"""终止 fixture 父进程及其全部 worker 子进程"""
|
||||
if proc.poll() is not None:
|
||||
return
|
||||
try:
|
||||
if sys.platform == "win32":
|
||||
subprocess.run(
|
||||
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
else:
|
||||
import signal
|
||||
|
||||
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
|
||||
except Exception:
|
||||
proc.kill()
|
||||
try:
|
||||
proc.wait(timeout=15)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
|
||||
|
||||
def _tail(path: Path, lines: int = 60) -> str:
|
||||
try:
|
||||
content = path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
return "\n".join(content[-lines:])
|
||||
except OSError:
|
||||
return "<no log>"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fixture_app(tmp_path):
|
||||
"""启动 workers=2 的 fixture 应用，返回 (port, hook_dir, job_dir, proc, log)"""
|
||||
port = _free_port()
|
||||
hook_dir = tmp_path / "hooks"
|
||||
job_dir = tmp_path / "jobs"
|
||||
hook_dir.mkdir()
|
||||
job_dir.mkdir()
|
||||
log_path = tmp_path / "fixture_app.log"
|
||||
|
||||
env = {
|
||||
**os.environ,
|
||||
"MW_PORT": str(port),
|
||||
"MW_WORKERS": str(WORKERS),
|
||||
"MW_HOOK_DIR": str(hook_dir),
|
||||
"MW_JOB_DIR": str(job_dir),
|
||||
}
|
||||
popen_kwargs = {}
|
||||
if sys.platform != "win32":
|
||||
popen_kwargs["start_new_session"] = True
|
||||
|
||||
log_file = open(log_path, "w", encoding="utf-8")
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "main.py"],
|
||||
cwd=str(FIXTURE_DIR),
|
||||
env=env,
|
||||
stdout=log_file,
|
||||
stderr=subprocess.STDOUT,
|
||||
**popen_kwargs,
|
||||
)
|
||||
try:
|
||||
yield port, hook_dir, job_dir, proc, log_path
|
||||
finally:
|
||||
_kill_process_tree(proc)
|
||||
log_file.close()
|
||||
|
||||
|
||||
def _wait_for_startup(port, hook_dir, proc, log_path) -> None:
|
||||
"""轮询等待：两个 worker 的 start 钩子文件出现且 /health 可访问"""
|
||||
deadline = time.time() + STARTUP_TIMEOUT
|
||||
health_ok = False
|
||||
while time.time() < deadline:
|
||||
if proc.poll() is not None:
|
||||
pytest.fail(
|
||||
f"fixture 应用提前退出 (returncode={proc.returncode})，"
|
||||
f"日志尾部:\n{_tail(log_path)}"
|
||||
)
|
||||
start_files = list(hook_dir.glob("start_*.txt"))
|
||||
if not health_ok:
|
||||
try:
|
||||
_http_get_json(f"http://127.0.0.1:{port}/health", timeout=2)
|
||||
health_ok = True
|
||||
except (urllib.error.URLError, OSError, ValueError):
|
||||
pass
|
||||
if health_ok and len(start_files) >= WORKERS:
|
||||
return
|
||||
time.sleep(0.5)
|
||||
pytest.fail(
|
||||
f"启动超时: health_ok={health_ok}, "
|
||||
f"start_hooks={len(list(hook_dir.glob('start_*.txt')))}/{WORKERS}，"
|
||||
f"日志尾部:\n{_tail(log_path)}"
|
||||
)
|
||||
|
||||
|
||||
def _read_start_records(hook_dir: Path):
|
||||
"""解析 start 钩子记录: {pid: (worker_id, client_instance_id)}"""
|
||||
records = {}
|
||||
duplicate_lines = []
|
||||
for path in hook_dir.glob("start_*.txt"):
|
||||
lines = [
|
||||
line for line in
|
||||
path.read_text(encoding="utf-8").splitlines() if line.strip()
|
||||
]
|
||||
if len(lines) != 1:
|
||||
duplicate_lines.append((path.name, lines))
|
||||
pid_str, worker_id_str, instance_id = lines[0].split(",")
|
||||
records[int(pid_str)] = (int(worker_id_str), instance_id)
|
||||
assert not duplicate_lines, f"start 钩子在同一 worker 内触发多次: {duplicate_lines}"
|
||||
return records
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_multiworker_per_worker_instances_hooks_and_scheduler(fixture_app):
|
||||
port, hook_dir, job_dir, proc, log_path = fixture_app
|
||||
_wait_for_startup(port, hook_dir, proc, log_path)
|
||||
|
||||
# ---- 1. start 钩子：每个 worker 各触发一次，worker_id 覆盖 1..N ----
|
||||
start_records = _read_start_records(hook_dir)
|
||||
assert len(start_records) == WORKERS, f"期望 {WORKERS} 个 worker 触发 start 钩子: {start_records}"
|
||||
worker_ids = sorted(wid for wid, _ in start_records.values())
|
||||
assert worker_ids == list(range(1, WORKERS + 1)), start_records
|
||||
|
||||
# ---- 2. client 实例独立：不同 worker 的实例 id 互不相同 ----
|
||||
instance_ids = [iid for _, iid in start_records.values()]
|
||||
assert all(instance_ids), f"worker 内未解析到 client 实例: {start_records}"
|
||||
assert len(set(instance_ids)) == WORKERS, (
|
||||
f"不同 worker 共享了同一 client 实例（issue #11 回归）: {start_records}"
|
||||
)
|
||||
|
||||
# ---- 3. HTTP 轮询 /whoami：pid -> 实例 id 是单射，且实例创建于本进程 ----
|
||||
observed = {} # pid -> set(instance_id)
|
||||
for _ in range(50):
|
||||
data = _http_get_json(f"http://127.0.0.1:{port}/api/whoami")
|
||||
pid = data["pid"]
|
||||
observed.setdefault(pid, set()).add(data["client_instance_id"])
|
||||
# client 实例必须创建于响应请求的同一进程（而非 fork 前的父进程）
|
||||
assert data["client_created_pid"] == pid, data
|
||||
# 响应进程必须是已知的 worker 进程，且实例 id 与钩子记录一致
|
||||
assert pid in start_records, (pid, start_records)
|
||||
assert data["client_instance_id"] == start_records[pid][1], data
|
||||
assert data["worker_id"] == start_records[pid][0], data
|
||||
for pid, ids in observed.items():
|
||||
assert len(ids) == 1, f"同一 worker 返回了多个 client 实例 id: {pid} -> {ids}"
|
||||
# 注：Windows 下多 socket 绑定同端口（SO_REUSEADDR）不保证连接均匀分发，
|
||||
# HTTP 观测到的 pid 数量不做强制断言；跨 worker 独立性已由钩子记录覆盖
|
||||
|
||||
# ---- 4. 调度任务仅 primary worker 执行 ----
|
||||
deadline = time.time() + 30
|
||||
while time.time() < deadline and not list(job_dir.glob("job_*.txt")):
|
||||
time.sleep(0.5)
|
||||
job_files = list(job_dir.glob("job_*.txt"))
|
||||
assert job_files, f"调度任务未执行，日志尾部:\n{_tail(log_path)}"
|
||||
# 再观察几秒，确认没有其他 worker 也在跑任务
|
||||
time.sleep(3)
|
||||
job_pids = sorted(
|
||||
int(path.stem.split("_")[1]) for path in job_dir.glob("job_*.txt")
|
||||
)
|
||||
primary_pid = next(
|
||||
pid for pid, (wid, _) in start_records.items() if wid == 1
|
||||
)
|
||||
assert job_pids == [primary_pid], (
|
||||
f"调度任务应只在 primary worker (pid={primary_pid}) 执行，"
|
||||
f"实际执行进程: {job_pids}"
|
||||
)
|
||||
|
||||
# ---- 4b. all_workers=True 任务在每个 worker 都执行 ----
|
||||
deadline = time.time() + 30
|
||||
while time.time() < deadline and len(list(job_dir.glob("awjob_*.txt"))) < WORKERS:
|
||||
time.sleep(0.5)
|
||||
awjob_pids = sorted(
|
||||
int(path.stem.split("_")[1]) for path in job_dir.glob("awjob_*.txt")
|
||||
)
|
||||
assert awjob_pids == sorted(start_records.keys()), (
|
||||
f"all_workers 任务应在全部 worker 执行 (pids={sorted(start_records.keys())})，"
|
||||
f"实际执行进程: {awjob_pids}，日志尾部:\n{_tail(log_path)}"
|
||||
)
|
||||
|
||||
# ---- 5. （POSIX）优雅关闭后 stop 钩子每个 worker 各触发一次 ----
|
||||
if sys.platform != "win32":
|
||||
proc.terminate() # SIGTERM -> 优雅关闭 -> lifespan 关闭阶段
|
||||
try:
|
||||
proc.wait(timeout=30)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
deadline = time.time() + 15
|
||||
while time.time() < deadline and len(list(hook_dir.glob("stop_*.txt"))) < WORKERS:
|
||||
time.sleep(0.5)
|
||||
stop_files = list(hook_dir.glob("stop_*.txt"))
|
||||
assert len(stop_files) == WORKERS, (
|
||||
f"期望 {WORKERS} 个 worker 触发 stop 钩子，实际 {len(stop_files)}"
|
||||
)
|
||||
# Windows: 父进程 terminate() 硬终止 worker，stop 钩子不保证触发，跳过断言
|
||||
@@ -0,0 +1,173 @@
|
||||
"""
|
||||
定时任务任务级 all_workers 支持（F4）单元测试
|
||||
|
||||
覆盖：
|
||||
1. @cron/@interval/@once 的 all_workers 元数据写入（顶层键，默认 False）
|
||||
2. 注册门控：非 primary worker 只注册 all_workers=True 的任务
|
||||
3. scheduler.on_all_workers=true 全局配置：非 primary 注册全部任务
|
||||
4. scheduler.enabled=false：任何 worker 都不注册任何任务
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.auto_configuration import AutoConfigurationManager
|
||||
from myboot.core.decorators import cron, interval, once
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 公共 fixture（与 test_multiworker.py 保持一致）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_worker_env():
|
||||
"""每个测试前后清理 MYBOOT_* 环境变量"""
|
||||
saved = {k: v for k, v in os.environ.items() if k.startswith("MYBOOT_")}
|
||||
for key in saved:
|
||||
del os.environ[key]
|
||||
yield
|
||||
for key in [k for k in os.environ if k.startswith("MYBOOT_")]:
|
||||
del os.environ[key]
|
||||
os.environ.update(saved)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def set_config():
|
||||
"""临时修改全局配置，测试结束后恢复原值"""
|
||||
from myboot.core.config import get_settings
|
||||
|
||||
settings = get_settings()
|
||||
changed = {}
|
||||
|
||||
def _set(key, value):
|
||||
if key not in changed:
|
||||
changed[key] = settings.get(key)
|
||||
settings.set(key, value)
|
||||
|
||||
yield _set
|
||||
|
||||
for key, old in changed.items():
|
||||
settings.set(key, old)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 测试用组件：一个 all_workers 任务 + 一个默认任务
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class JobsComponent:
|
||||
@interval(seconds=60, all_workers=True)
|
||||
def refresh_local_cache(self):
|
||||
pass
|
||||
|
||||
@interval(seconds=60)
|
||||
def primary_only_job(self):
|
||||
pass
|
||||
|
||||
|
||||
def _register(app):
|
||||
manager = AutoConfigurationManager()
|
||||
instance = JobsComponent()
|
||||
manager._register_component_jobs(app, instance, JobsComponent, "tests.jobs")
|
||||
return app.scheduler.list_jobs()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. 装饰器元数据
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAllWorkersMetadata:
|
||||
def test_cron_all_workers_top_level(self):
|
||||
@cron("0 0 * * *", all_workers=True)
|
||||
def job():
|
||||
pass
|
||||
|
||||
meta = job.__myboot_job__
|
||||
assert meta["all_workers"] is True
|
||||
assert "all_workers" not in meta["kwargs"]
|
||||
|
||||
def test_interval_all_workers_top_level(self):
|
||||
@interval(seconds=30, all_workers=True)
|
||||
def job():
|
||||
pass
|
||||
|
||||
meta = job.__myboot_job__
|
||||
assert meta["all_workers"] is True
|
||||
assert "all_workers" not in meta["kwargs"]
|
||||
|
||||
def test_once_all_workers_top_level(self):
|
||||
@once("2099-01-01 00:00:00", all_workers=True)
|
||||
def job():
|
||||
pass
|
||||
|
||||
meta = job.__myboot_job__
|
||||
assert meta["all_workers"] is True
|
||||
assert "all_workers" not in meta["kwargs"]
|
||||
|
||||
def test_all_workers_defaults_to_false(self):
|
||||
@cron("0 0 * * *")
|
||||
def cron_job():
|
||||
pass
|
||||
|
||||
@interval(seconds=1)
|
||||
def interval_job():
|
||||
pass
|
||||
|
||||
@once("2099-01-01 00:00:00")
|
||||
def once_job():
|
||||
pass
|
||||
|
||||
assert cron_job.__myboot_job__["all_workers"] is False
|
||||
assert interval_job.__myboot_job__["all_workers"] is False
|
||||
assert once_job.__myboot_job__["all_workers"] is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. 注册门控
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRegistrationGate:
|
||||
def test_non_primary_registers_only_all_workers_jobs(self):
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "0"
|
||||
app = Application(name="aw-nonprimary", auto_configuration=False)
|
||||
assert app.is_primary_worker is False
|
||||
|
||||
job_ids = _register(app)
|
||||
assert len(job_ids) == 1
|
||||
assert "refresh_local_cache" in job_ids[0]
|
||||
# 非 primary worker 因 all_workers 任务而启用调度器实例
|
||||
assert app.scheduler.is_enabled() is True
|
||||
|
||||
def test_primary_registers_all_jobs(self):
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "1"
|
||||
app = Application(name="aw-primary", auto_configuration=False)
|
||||
assert app.is_primary_worker is True
|
||||
|
||||
job_ids = _register(app)
|
||||
assert len(job_ids) == 2
|
||||
assert any("refresh_local_cache" in j for j in job_ids)
|
||||
assert any("primary_only_job" in j for j in job_ids)
|
||||
|
||||
def test_on_all_workers_config_registers_all_on_non_primary(self, set_config):
|
||||
set_config("scheduler.on_all_workers", True)
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "0"
|
||||
app = Application(name="aw-globalcfg", auto_configuration=False)
|
||||
|
||||
job_ids = _register(app)
|
||||
assert len(job_ids) == 2
|
||||
assert app.scheduler.is_enabled() is True
|
||||
|
||||
def test_scheduler_disabled_registers_nothing(self, set_config):
|
||||
set_config("scheduler.enabled", False)
|
||||
|
||||
# primary worker 也不注册
|
||||
app = Application(name="aw-disabled-primary", auto_configuration=False)
|
||||
assert _register(app) == []
|
||||
assert app.scheduler.is_enabled() is False
|
||||
|
||||
# 非 primary worker（含 all_workers 任务）同样不注册
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "0"
|
||||
app2 = Application(name="aw-disabled-nonprimary", auto_configuration=False)
|
||||
assert _register(app2) == []
|
||||
assert app2.scheduler.is_enabled() is False
|
||||
@@ -0,0 +1,110 @@
|
||||
"""
|
||||
0.2.x F2：lifespan 关闭阶段自动调用 client.close()
|
||||
|
||||
约定：app.clients 中具有可调用 close() 方法的实例，在 worker_stop_hooks
|
||||
之后、shutdown_hooks 之前由框架兜底关闭；close 异常降为 warning 不阻断。
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from myboot.core.application import Application
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make_app(tmp_path, monkeypatch):
|
||||
"""构造禁用自动配置的最小 Application"""
|
||||
import myboot.core.config as config_module
|
||||
|
||||
monkeypatch.setattr(config_module, "_find_project_root", lambda: str(tmp_path))
|
||||
config_module.reload_config()
|
||||
|
||||
def _make():
|
||||
return Application(name="auto-close-test", auto_configuration=False)
|
||||
|
||||
yield _make
|
||||
config_module.reload_config()
|
||||
|
||||
|
||||
class _ClosableClient:
|
||||
def __init__(self):
|
||||
self.closed = 0
|
||||
|
||||
def close(self):
|
||||
self.closed += 1
|
||||
|
||||
|
||||
class _NoCloseClient:
|
||||
pass
|
||||
|
||||
|
||||
class _FailingClient:
|
||||
def close(self):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
|
||||
def _run_lifespan(app: Application) -> None:
|
||||
"""走一遍完整 lifespan（TestClient 上下文进入/退出即 startup/shutdown）"""
|
||||
with TestClient(app.get_fastapi_app()):
|
||||
pass
|
||||
|
||||
|
||||
def test_client_with_close_is_auto_closed(make_app):
|
||||
app = make_app()
|
||||
client = _ClosableClient()
|
||||
app.clients["closable"] = client
|
||||
|
||||
_run_lifespan(app)
|
||||
|
||||
assert client.closed == 1
|
||||
|
||||
|
||||
def test_client_without_close_is_skipped(make_app):
|
||||
app = make_app()
|
||||
app.clients["no_close"] = _NoCloseClient()
|
||||
|
||||
_run_lifespan(app) # 不抛错即通过
|
||||
|
||||
|
||||
def test_failing_close_does_not_block_others(make_app):
|
||||
app = make_app()
|
||||
good = _ClosableClient()
|
||||
# dict 保序：失败的排在前面，验证不阻断后续
|
||||
app.clients["bad"] = _FailingClient()
|
||||
app.clients["good"] = good
|
||||
|
||||
_run_lifespan(app)
|
||||
|
||||
assert good.closed == 1
|
||||
|
||||
|
||||
def test_close_runs_after_worker_stop_hooks_before_shutdown_hooks(make_app):
|
||||
app = make_app()
|
||||
order = []
|
||||
|
||||
class _OrderClient:
|
||||
def close(self):
|
||||
order.append("client_close")
|
||||
|
||||
app.clients["order"] = _OrderClient()
|
||||
app.add_worker_stop_hook(lambda: order.append("worker_stop"))
|
||||
app.add_shutdown_hook(lambda: order.append("shutdown"))
|
||||
|
||||
_run_lifespan(app)
|
||||
|
||||
assert order == ["worker_stop", "client_close", "shutdown"]
|
||||
|
||||
|
||||
def test_async_close_supported(make_app):
|
||||
app = make_app()
|
||||
closed = []
|
||||
|
||||
class _AsyncClient:
|
||||
async def close(self):
|
||||
closed.append(True)
|
||||
|
||||
app.clients["async"] = _AsyncClient()
|
||||
|
||||
_run_lifespan(app)
|
||||
|
||||
assert closed == [True]
|
||||
@@ -0,0 +1,598 @@
|
||||
"""
|
||||
配置系统特征测试（characterization tests）
|
||||
|
||||
固化 myboot.core.config 的「当前实际行为」，作为后续重构的兼容性闸门。
|
||||
所有断言以代码现状为准（已用探针脚本逐项验证），而非文档或「理想行为」。
|
||||
|
||||
已知可疑现状（详见各测试注释，重构时务必先确认再改）：
|
||||
|
||||
1. ``create_settings()`` 把 ``default_settings={...}`` 作为普通 kwarg 传给
|
||||
Dynaconf。Dynaconf 没有这个选项，于是它变成了一个名为
|
||||
``DEFAULT_SETTINGS`` 的普通配置项——内置默认值并 **不会** 出现在
|
||||
``app.name`` / ``server.port`` 等路径上（与 docs/configuration.md 2.3 节相悖）。
|
||||
2. 由 1 推论：在没有任何 YAML 声明对应键时，``SERVER__PORT`` 等环境变量
|
||||
会被 ``ignore_unknown_envvars=True`` 静默忽略（文档 5.6 声称内置默认键
|
||||
可直接用 .env 覆盖，实际不行）。
|
||||
3. 同目录同时存在 ``config.yaml`` 与 ``config.yml`` 时，``.yml`` 后加载，
|
||||
**覆盖** ``.yaml`` 的同名键。
|
||||
4. ``merge_enabled=True`` 下，多文件中同路径的 list 会拼接（可能重复），
|
||||
嵌套 dict 深合并；``dynaconf_merge: false`` 则整块替换（issue #13）。
|
||||
5. ``env_parse_values=True`` 下，``"1"`` 解析为 int 1 而非 bool True。
|
||||
6. 环境变量提供的 JSON 列表会 **整体替换** YAML 中的列表（不拼接）。
|
||||
|
||||
注意：``_find_project_root()`` 以 config.py 所在包向上找 pyproject.toml，
|
||||
与进程工作目录无关（仓库内运行时恒为仓库根）。因此仅靠
|
||||
``monkeypatch.chdir(tmp_path)`` 无法隔离根 config.yaml 查找，必须同时
|
||||
monkeypatch ``_find_project_root``。
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import tempfile
|
||||
import textwrap
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
|
||||
import myboot.core.config as config_module
|
||||
from myboot.core.config import (
|
||||
_get_config_files,
|
||||
_is_url,
|
||||
create_settings,
|
||||
get_config,
|
||||
get_config_bool,
|
||||
get_config_int,
|
||||
get_config_str,
|
||||
get_settings,
|
||||
reload_config,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolate_config(tmp_path, monkeypatch):
|
||||
"""每个测试独立的「项目根」+ 干净的全局单例 / CONFIG_FILE。
|
||||
|
||||
- monkeypatch _find_project_root：见模块 docstring，仅 chdir 不够。
|
||||
- chdir 仍然保留，防止任何依赖 cwd 的退路逻辑读到真实仓库。
|
||||
- 前后都清空模块级单例 _settings，避免测试间污染。
|
||||
"""
|
||||
monkeypatch.setattr(config_module, "_find_project_root", lambda: str(tmp_path))
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.delenv("CONFIG_FILE", raising=False)
|
||||
reload_config()
|
||||
yield
|
||||
reload_config()
|
||||
|
||||
|
||||
def write_yaml(path, content):
|
||||
"""写入 YAML 文件（自动 dedent / 建父目录），返回 str 路径"""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(textwrap.dedent(content), encoding="utf-8")
|
||||
return str(path)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. 多源配置文件优先级
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestConfigFilePriority:
|
||||
def test_root_config_yaml_loaded(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
app:
|
||||
name: root-app
|
||||
""")
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "root-app"
|
||||
|
||||
def test_conf_dir_overrides_root(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
app:
|
||||
name: root-app
|
||||
server:
|
||||
port: 1111
|
||||
only_root: 1
|
||||
""")
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
app:
|
||||
name: conf-app
|
||||
server:
|
||||
port: 2222
|
||||
""")
|
||||
settings = create_settings()
|
||||
# conf/config.yaml 后加载，覆盖根目录同名标量
|
||||
assert settings.get("app.name") == "conf-app"
|
||||
assert settings.get("server.port") == 2222
|
||||
# 仅根目录出现的键仍然保留（merge_enabled=True）
|
||||
assert settings.get("only_root") == 1
|
||||
|
||||
def test_param_config_file_overrides_conf(self, tmp_path, tmp_config_file):
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
app:
|
||||
name: conf-app
|
||||
only_conf: 2
|
||||
""")
|
||||
param = tmp_config_file(
|
||||
"""\
|
||||
app:
|
||||
name: param-app
|
||||
""",
|
||||
filename="param.yaml",
|
||||
)
|
||||
settings = create_settings(param)
|
||||
assert settings.get("app.name") == "param-app"
|
||||
assert settings.get("only_conf") == 2
|
||||
|
||||
def test_config_file_env_var_overrides_param(
|
||||
self, tmp_path, tmp_config_file, monkeypatch
|
||||
):
|
||||
param = tmp_config_file(
|
||||
"""\
|
||||
app:
|
||||
name: param-app
|
||||
only_param: 3
|
||||
""",
|
||||
filename="param.yaml",
|
||||
)
|
||||
env_file = tmp_config_file(
|
||||
"""\
|
||||
app:
|
||||
name: envfile-app
|
||||
""",
|
||||
filename="envfile.yaml",
|
||||
)
|
||||
monkeypatch.setenv("CONFIG_FILE", env_file)
|
||||
settings = create_settings(param)
|
||||
# CONFIG_FILE 指定的文件最后加载，优先级最高（文件来源中）
|
||||
assert settings.get("app.name") == "envfile-app"
|
||||
assert settings.get("only_param") == 3
|
||||
|
||||
def test_get_config_files_order(self, tmp_path, monkeypatch):
|
||||
"""固化 _get_config_files 返回的加载顺序：根 → conf → 参数 → CONFIG_FILE"""
|
||||
root = write_yaml(tmp_path / "config.yaml", "a: 1\n")
|
||||
conf = write_yaml(tmp_path / "conf" / "config.yaml", "a: 2\n")
|
||||
param = write_yaml(tmp_path / "param.yaml", "a: 3\n")
|
||||
envf = write_yaml(tmp_path / "envfile.yaml", "a: 4\n")
|
||||
monkeypatch.setenv("CONFIG_FILE", envf)
|
||||
assert _get_config_files(param) == [root, conf, param, envf]
|
||||
|
||||
def test_nonexistent_param_file_silently_dropped(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", "app:\n name: root-app\n")
|
||||
# 不存在的文件路径不报错，静默跳过
|
||||
settings = create_settings(str(tmp_path / "no-such-file.yaml"))
|
||||
assert settings.get("app.name") == "root-app"
|
||||
|
||||
def test_yml_overrides_yaml_in_same_dir(self, tmp_path):
|
||||
"""可疑现状 #3：config.yml 排在 config.yaml 之后加载，.yml 覆盖 .yaml。
|
||||
|
||||
直觉上 .yaml 是「主」扩展名，但实际 .yml 同名键胜出。
|
||||
"""
|
||||
write_yaml(tmp_path / "config.yaml", "app:\n name: from-yaml\n")
|
||||
write_yaml(tmp_path / "config.yml", "app:\n name: from-yml\n")
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "from-yml"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. 内置默认值（0.2.0 修复：默认值以大写 kwargs 传入 Dynaconf）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDefaultSettings:
|
||||
def test_builtin_defaults_exposed_at_documented_paths(self):
|
||||
"""0.2.0 修复：无任何 YAML 时内置默认值在文档路径上可取到。
|
||||
|
||||
旧版把 default_settings={...} 当 kwarg 传给 Dynaconf（无此参数），
|
||||
默认值整体失效；现以大写 kwargs（APP=/SERVER=...）注册为默认配置项。
|
||||
"""
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "MyBoot App"
|
||||
assert settings.get("server.port") == 8000
|
||||
assert settings.get("logging.level") == "INFO"
|
||||
assert settings.get("scheduler.enabled") is True
|
||||
assert settings.get("scheduler.max_workers") == 10
|
||||
|
||||
def test_no_stray_default_settings_key(self):
|
||||
"""修复后不再产生名为 DEFAULT_SETTINGS 的杂散配置项"""
|
||||
settings = create_settings()
|
||||
assert "DEFAULT_SETTINGS" not in settings
|
||||
|
||||
def test_yaml_partial_override_deep_merges_with_defaults(self, tmp_path):
|
||||
"""YAML 只覆盖 server.port 时，server 其余默认键保留（深合并）"""
|
||||
write_yaml(tmp_path / "config.yaml", "server:\n port: 9000\n")
|
||||
settings = create_settings()
|
||||
assert settings.get("server.port") == 9000
|
||||
assert settings.get("server.host") == "0.0.0.0"
|
||||
assert settings.get("server.workers") == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. 环境变量 `__` 分隔符覆盖
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestEnvVarOverride:
|
||||
def test_double_underscore_overrides_declared_key(self, tmp_path, monkeypatch):
|
||||
# envvar_prefix=False：变量名即配置路径大写形式，无前缀
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
app:
|
||||
name: yaml-app
|
||||
""")
|
||||
monkeypatch.setenv("APP__NAME", "env-app")
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "env-app"
|
||||
|
||||
def test_env_var_beats_config_file_env_source(
|
||||
self, tmp_path, tmp_config_file, monkeypatch
|
||||
):
|
||||
env_file = tmp_config_file(
|
||||
"""\
|
||||
app:
|
||||
name: from-config-file
|
||||
""",
|
||||
filename="cf.yaml",
|
||||
)
|
||||
monkeypatch.setenv("CONFIG_FILE", env_file)
|
||||
monkeypatch.setenv("APP__NAME", "from-envvar")
|
||||
settings = create_settings()
|
||||
# 环境变量键 > CONFIG_FILE 文件
|
||||
assert settings.get("app.name") == "from-envvar"
|
||||
|
||||
def test_undeclared_env_key_silently_ignored(self, tmp_path, monkeypatch):
|
||||
# ignore_unknown_envvars=True：YAML 中没有 database 段时被静默忽略
|
||||
write_yaml(tmp_path / "config.yaml", "app:\n name: yaml-app\n")
|
||||
monkeypatch.setenv("DATABASE__URL", "postgresql://ignored")
|
||||
settings = create_settings()
|
||||
assert settings.get("database.url") is None
|
||||
|
||||
def test_undeclared_leaf_under_declared_parent_also_ignored(
|
||||
self, tmp_path, monkeypatch
|
||||
):
|
||||
# server 段存在但 server.custom_flag 既未在 YAML 也未在内置默认值中
|
||||
# 声明 → SERVER__CUSTOM_FLAG 被 ignore_unknown_envvars 忽略
|
||||
# （注意 server.reload 已是内置默认键，不能再用作"未声明"示例）
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
server:
|
||||
port: 8000
|
||||
""")
|
||||
monkeypatch.setenv("SERVER__CUSTOM_FLAG", "true")
|
||||
settings = create_settings()
|
||||
assert settings.get("server.custom_flag") is None
|
||||
|
||||
def test_env_var_overrides_builtin_defaults_without_yaml(self, monkeypatch):
|
||||
"""0.2.0 修复：内置默认键生效后，无 YAML 时也可用环境变量覆盖
|
||||
（与 docs/configuration.md 5.6 节一致）"""
|
||||
monkeypatch.setenv("SERVER__PORT", "7777")
|
||||
monkeypatch.setenv("APP__NAME", "env-only")
|
||||
settings = create_settings()
|
||||
assert settings.get("server.port") == 7777
|
||||
assert settings.get("app.name") == "env-only"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. 类型自动转换（env_parse_values=True）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestEnvVarTypeConversion:
|
||||
@pytest.fixture(autouse=True)
|
||||
def base_yaml(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
app:
|
||||
debug: false
|
||||
server:
|
||||
port: 8000
|
||||
reload: false
|
||||
cors:
|
||||
allow_origins: ["*"]
|
||||
""")
|
||||
|
||||
def test_numeric_string_to_int(self, monkeypatch):
|
||||
monkeypatch.setenv("SERVER__PORT", "9090")
|
||||
settings = create_settings()
|
||||
value = settings.get("server.port")
|
||||
assert value == 9090
|
||||
assert isinstance(value, int)
|
||||
|
||||
def test_true_false_strings_to_bool(self, monkeypatch):
|
||||
monkeypatch.setenv("SERVER__RELOAD", "true")
|
||||
settings = create_settings()
|
||||
assert settings.get("server.reload") is True
|
||||
|
||||
monkeypatch.setenv("SERVER__RELOAD", "false")
|
||||
settings = create_settings()
|
||||
assert settings.get("server.reload") is False
|
||||
|
||||
def test_string_one_parses_as_int_not_bool(self, monkeypatch):
|
||||
"""可疑现状 #5："1" 被解析为 int 1，不是 bool True。
|
||||
|
||||
依赖 `settings.get("app.debug") is True` 的调用方会受影响；
|
||||
get_config_bool 仍能正确转换（见 TestHelperFunctions）。
|
||||
"""
|
||||
monkeypatch.setenv("APP__DEBUG", "1")
|
||||
settings = create_settings()
|
||||
value = settings.get("app.debug")
|
||||
assert value == 1
|
||||
assert not isinstance(value, bool)
|
||||
assert isinstance(value, int)
|
||||
|
||||
def test_json_list_string_parsed_and_replaces_yaml_list(self, monkeypatch):
|
||||
"""可疑现状 #6：env 提供的 JSON 列表整体替换 YAML 列表，
|
||||
不与原列表 ["*"] 拼接（与多文件加载时的列表拼接行为不同）。
|
||||
"""
|
||||
monkeypatch.setenv(
|
||||
"SERVER__CORS__ALLOW_ORIGINS", '["http://a", "http://b"]'
|
||||
)
|
||||
settings = create_settings()
|
||||
assert list(settings.get("server.cors.allow_origins")) == [
|
||||
"http://a",
|
||||
"http://b",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. 合并行为与 dynaconf_merge: false（issue #13 回归）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMergeBehavior:
|
||||
def _write_base_cors(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["*"]
|
||||
allow_methods: ["*"]
|
||||
allow_headers: ["*"]
|
||||
""")
|
||||
|
||||
def test_nested_dict_deep_merge_keeps_sibling_keys(self, tmp_path):
|
||||
# merge_enabled=True：后加载文件只写部分子键时，旧子键保留
|
||||
self._write_base_cors(tmp_path)
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://x"]
|
||||
""")
|
||||
settings = create_settings()
|
||||
cors = settings.get("server.cors")
|
||||
assert list(cors["allow_methods"]) == ["*"]
|
||||
assert list(cors["allow_headers"]) == ["*"]
|
||||
|
||||
def test_list_values_concatenate_across_files(self, tmp_path):
|
||||
"""可疑现状 #4：同路径列表跨文件「拼接」而非替换，可能产生意外元素。
|
||||
|
||||
基底 ["*"] + 覆盖文件 ["http://x"] → ["*", "http://x"]，
|
||||
通配符 "*" 仍然留在 CORS origins 里。
|
||||
"""
|
||||
self._write_base_cors(tmp_path)
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://x"]
|
||||
""")
|
||||
settings = create_settings()
|
||||
assert list(settings.get("server.cors.allow_origins")) == ["*", "http://x"]
|
||||
|
||||
def test_dynaconf_merge_false_replaces_whole_dict(self, tmp_path):
|
||||
"""issue #13 回归：dynaconf_merge: false 时嵌套 dict 整块替换，
|
||||
先前的 allow_methods / allow_headers 不再保留。
|
||||
"""
|
||||
self._write_base_cors(tmp_path)
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://x"]
|
||||
dynaconf_merge: false
|
||||
""")
|
||||
settings = create_settings()
|
||||
cors = settings.get("server.cors")
|
||||
assert cors.to_dict() == {"allow_origins": ["http://x"]}
|
||||
|
||||
def test_dynaconf_merge_marker_not_exposed_as_config_key(self, tmp_path):
|
||||
self._write_base_cors(tmp_path)
|
||||
write_yaml(tmp_path / "conf" / "config.yaml", """\
|
||||
server:
|
||||
cors:
|
||||
allow_origins: ["http://x"]
|
||||
dynaconf_merge: false
|
||||
""")
|
||||
settings = create_settings()
|
||||
# 合并控制元数据不会作为业务键出现
|
||||
assert settings.get("server.cors.dynaconf_merge") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. 缺失键默认值 / 便捷函数 / 大小写
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHelperFunctions:
|
||||
@pytest.fixture(autouse=True)
|
||||
def base_yaml(self, tmp_path):
|
||||
write_yaml(tmp_path / "config.yaml", """\
|
||||
app:
|
||||
name: helper-app
|
||||
server:
|
||||
port: 8000
|
||||
flags:
|
||||
str_yes: "yes"
|
||||
str_other: "definitely"
|
||||
items:
|
||||
- a
|
||||
- b
|
||||
""")
|
||||
reload_config() # 便捷函数走全局单例，确保读到本测试的 YAML
|
||||
|
||||
def test_get_missing_key_returns_default(self):
|
||||
assert get_config("no.such.key", "fallback") == "fallback"
|
||||
|
||||
def test_get_missing_key_default_none(self):
|
||||
assert get_config("no.such.key") is None
|
||||
|
||||
def test_get_config_str_converts(self):
|
||||
assert get_config_str("server.port") == "8000"
|
||||
assert get_config_str("no.such.key", "dft") == "dft"
|
||||
|
||||
def test_get_config_int_converts_and_falls_back(self):
|
||||
assert get_config_int("server.port") == 8000
|
||||
# 列表无法转 int → 返回 default（当前实现吞掉 TypeError）
|
||||
assert get_config_int("items", 42) == 42
|
||||
assert get_config_int("no.such.key", 7) == 7
|
||||
|
||||
def test_get_config_bool_string_semantics(self):
|
||||
# 字符串仅 'true'/'1'/'yes'/'on'（不区分大小写）为真
|
||||
assert get_config_bool("flags.str_yes") is True
|
||||
assert get_config_bool("flags.str_other") is False
|
||||
assert get_config_bool("no.such.key", True) is True
|
||||
|
||||
def test_get_is_case_insensitive(self):
|
||||
settings = get_settings()
|
||||
assert settings.get("APP.NAME") == "helper-app"
|
||||
assert settings.get("app.name") == "helper-app"
|
||||
|
||||
|
||||
class TestSingletonBehavior:
|
||||
def test_get_settings_ignores_config_file_after_first_call(
|
||||
self, tmp_path, tmp_config_file
|
||||
):
|
||||
write_yaml(tmp_path / "config.yaml", "app:\n name: first\n")
|
||||
other = tmp_config_file("app:\n name: second\n", filename="other.yaml")
|
||||
|
||||
first = get_settings()
|
||||
assert first.get("app.name") == "first"
|
||||
# 单例已固定，后续传入不同 config_file 不生效
|
||||
second = get_settings(other)
|
||||
assert second is first
|
||||
assert second.get("app.name") == "first"
|
||||
|
||||
def test_reload_config_resets_singleton(self, tmp_path, tmp_config_file):
|
||||
write_yaml(tmp_path / "config.yaml", "app:\n name: first\n")
|
||||
other = tmp_config_file("app:\n name: second\n", filename="other.yaml")
|
||||
|
||||
first = get_settings()
|
||||
reload_config()
|
||||
second = get_settings(other)
|
||||
assert second is not first
|
||||
assert second.get("app.name") == "second"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. 远程配置（HTTP URL，mock requests）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _cache_path_for(url):
|
||||
cache_dir = os.path.join(tempfile.gettempdir(), "myboot_config_cache")
|
||||
return os.path.join(cache_dir, hashlib.md5(url.encode()).hexdigest() + ".yaml")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def remote_url():
|
||||
"""每个测试唯一的 URL，避免共享缓存目录（系统临时目录）相互污染"""
|
||||
url = f"https://config.example.invalid/{uuid.uuid4().hex}.yaml"
|
||||
yield url
|
||||
cache = _cache_path_for(url)
|
||||
if os.path.exists(cache):
|
||||
os.remove(cache)
|
||||
|
||||
|
||||
class TestDotenvAutoLoad:
|
||||
"""0.2.x F1：项目根 .env 自动加载（Dynaconf load_dotenv）"""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolate_environ(self):
|
||||
# load_dotenv 会写 os.environ；快照-恢复整个 environ 避免污染后续测试
|
||||
# （不能用 monkeypatch.setattr 替换 os.environ —— dotenv/dynaconf
|
||||
# 内部持有原始 _Environ 对象引用，替换属性不生效）
|
||||
saved = dict(os.environ)
|
||||
yield
|
||||
os.environ.clear()
|
||||
os.environ.update(saved)
|
||||
|
||||
def test_dotenv_loaded_from_project_root(self, tmp_path):
|
||||
(tmp_path / ".env").write_text("APP__NAME=dotenv-app\n", encoding="utf-8")
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "dotenv-app"
|
||||
|
||||
def test_real_env_var_beats_dotenv(self, tmp_path, monkeypatch):
|
||||
# dotenv_override=False：真实环境变量优先于 .env
|
||||
(tmp_path / ".env").write_text("APP__NAME=dotenv-app\n", encoding="utf-8")
|
||||
monkeypatch.setenv("APP__NAME", "real-env")
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "real-env"
|
||||
|
||||
def test_no_dotenv_file_is_fine(self, tmp_path):
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "MyBoot App"
|
||||
|
||||
|
||||
class TestRemoteConfig:
|
||||
def test_is_url_detection(self):
|
||||
assert _is_url("http://example.com/c.yaml")
|
||||
assert _is_url("https://example.com/c.yaml")
|
||||
assert not _is_url("/etc/myboot/config.yaml")
|
||||
assert not _is_url("ftp://example.com/c.yaml")
|
||||
# 空字符串走 `path and ...` 短路，返回原值（falsy），不是 False
|
||||
assert not _is_url("")
|
||||
|
||||
def test_remote_config_downloaded_cached_and_loaded(
|
||||
self, monkeypatch, mocker, remote_url
|
||||
):
|
||||
response = mocker.Mock()
|
||||
response.text = "app:\n name: remote-app\nremote_only: 42\n"
|
||||
response.raise_for_status = mocker.Mock()
|
||||
mock_get = mocker.patch.object(
|
||||
config_module.requests, "get", return_value=response
|
||||
)
|
||||
|
||||
monkeypatch.setenv("CONFIG_FILE", remote_url)
|
||||
settings = create_settings()
|
||||
|
||||
mock_get.assert_called_once_with(remote_url, timeout=30)
|
||||
# 下载内容写入系统临时目录缓存（md5(url).yaml）
|
||||
cache = _cache_path_for(remote_url)
|
||||
assert os.path.exists(cache)
|
||||
assert "remote-app" in open(cache, encoding="utf-8").read()
|
||||
# 远程配置作为 CONFIG_FILE 来源参与合并
|
||||
assert settings.get("app.name") == "remote-app"
|
||||
assert settings.get("remote_only") == 42
|
||||
|
||||
def test_download_failure_falls_back_to_existing_cache(
|
||||
self, monkeypatch, mocker, remote_url
|
||||
):
|
||||
cache = _cache_path_for(remote_url)
|
||||
os.makedirs(os.path.dirname(cache), exist_ok=True)
|
||||
with open(cache, "w", encoding="utf-8") as f:
|
||||
f.write("app:\n name: cached-app\n")
|
||||
|
||||
mocker.patch.object(
|
||||
config_module.requests,
|
||||
"get",
|
||||
side_effect=ConnectionError("network down"),
|
||||
)
|
||||
monkeypatch.setenv("CONFIG_FILE", remote_url)
|
||||
settings = create_settings()
|
||||
assert settings.get("app.name") == "cached-app"
|
||||
|
||||
def test_download_failure_without_cache_raises(
|
||||
self, monkeypatch, mocker, remote_url
|
||||
):
|
||||
# 无缓存且下载失败 → 原始异常向上抛出（create_settings 直接失败）
|
||||
mocker.patch.object(
|
||||
config_module.requests,
|
||||
"get",
|
||||
side_effect=ValueError("network down"),
|
||||
)
|
||||
monkeypatch.setenv("CONFIG_FILE", remote_url)
|
||||
with pytest.raises(ValueError, match="network down"):
|
||||
create_settings()
|
||||
|
||||
def test_remote_param_config_file_also_supported(self, mocker, remote_url):
|
||||
# config_file 参数同样支持 URL（不只 CONFIG_FILE 环境变量）
|
||||
response = mocker.Mock()
|
||||
response.text = "app:\n name: remote-param\n"
|
||||
response.raise_for_status = mocker.Mock()
|
||||
mocker.patch.object(config_module.requests, "get", return_value=response)
|
||||
|
||||
settings = create_settings(remote_url)
|
||||
assert settings.get("app.name") == "remote-param"
|
||||
@@ -0,0 +1,498 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
依赖注入子系统特征测试（characterization tests）
|
||||
|
||||
固化 myboot.core.di（container/registry/providers/decorators）与
|
||||
myboot.core.decorators 中 @service/@client 的「当前实际行为」，
|
||||
作为后续重构的兼容性闸门。
|
||||
|
||||
绝对规则：本文件只断言现状，不修改任何源码。
|
||||
可疑现状行为均在对应用例中以注释标注（搜索 "可疑现状"）。
|
||||
"""
|
||||
|
||||
import inspect
|
||||
from typing import Optional
|
||||
|
||||
import pytest
|
||||
|
||||
from myboot.core.decorators import _camel_to_snake, client, service
|
||||
from myboot.core.di.container import DependencyContainer
|
||||
from myboot.core.di.decorators import Provide, get_injectable_params, inject
|
||||
from myboot.core.di.providers import ServiceProvider
|
||||
from myboot.core.di.registry import ServiceRegistry
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 辅助类（前缀避开 Test，防止被 pytest 收集）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class AlphaService:
|
||||
"""无依赖的服务"""
|
||||
|
||||
def __init__(self):
|
||||
self.tag = "alpha"
|
||||
|
||||
|
||||
class BetaService:
|
||||
"""通过类型注解依赖 AlphaService"""
|
||||
|
||||
def __init__(self, alpha_service: AlphaService):
|
||||
self.alpha_service = alpha_service
|
||||
|
||||
|
||||
class GammaService:
|
||||
"""通过字符串形式的 Provide 注解依赖 alpha_service"""
|
||||
|
||||
def __init__(self, dep: "Provide['alpha_service']"):
|
||||
self.dep = dep
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
# 说明：DI 子系统没有模块级单例——DependencyContainer / ServiceRegistry 都是
|
||||
# 普通类，每个测试构造新实例即可天然隔离。container fixture 在 teardown 时
|
||||
# 仍调用 clear() 以释放 dependency_injector 的 DynamicContainer 状态。
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def registry():
|
||||
return ServiceRegistry()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def container():
|
||||
c = DependencyContainer()
|
||||
yield c
|
||||
c.clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. 服务注册：registry 中 dependencies / dependents 数据结构
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestServiceRegistration:
|
||||
def test_register_service_records_class_and_config(self, registry):
|
||||
registry.register_service(AlphaService, "alpha_service", {"k": "v"})
|
||||
|
||||
assert registry.services == {"alpha_service": AlphaService}
|
||||
assert registry.service_configs == {"alpha_service": {"k": "v"}}
|
||||
assert registry.has_service("alpha_service") is True
|
||||
assert registry.get_service_class("alpha_service") is AlphaService
|
||||
assert registry.get_service_config("alpha_service") == {"k": "v"}
|
||||
|
||||
def test_register_service_default_config_is_empty_dict(self, registry):
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
assert registry.get_service_config("alpha_service") == {}
|
||||
|
||||
def test_no_dependency_service_has_empty_sets(self, registry):
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
|
||||
assert registry.dependencies == {"alpha_service": set()}
|
||||
# register_service 通过 setdefault 为自身建立空 dependents 条目
|
||||
assert registry.dependents == {"alpha_service": set()}
|
||||
|
||||
def test_dependency_recorded_in_both_directions(self, registry):
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
|
||||
assert registry.dependencies["beta_service"] == {"alpha_service"}
|
||||
assert registry.dependencies["alpha_service"] == set()
|
||||
assert registry.dependents["alpha_service"] == {"beta_service"}
|
||||
assert registry.dependents["beta_service"] == set()
|
||||
|
||||
assert registry.get_dependencies("beta_service") == {"alpha_service"}
|
||||
assert registry.get_dependents("alpha_service") == {"beta_service"}
|
||||
|
||||
def test_dependent_registered_first_creates_dependents_entry_for_unregistered_dep(
|
||||
self, registry
|
||||
):
|
||||
"""先注册依赖方时，被依赖方（尚未注册）的 dependents 条目已被预先写入"""
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
|
||||
assert "alpha_service" not in registry.services
|
||||
assert registry.dependents["alpha_service"] == {"beta_service"}
|
||||
|
||||
def test_get_dependencies_of_unknown_service_returns_empty_set(self, registry):
|
||||
assert registry.get_dependencies("nonexistent") == set()
|
||||
assert registry.get_dependents("nonexistent") == set()
|
||||
|
||||
def test_reregister_resets_then_reanalyzes_dependencies(self, registry):
|
||||
"""可疑现状：register_service 无条件重置 dependencies[service_name]，
|
||||
随后重新分析；对 dependents 中旧条目不做清理（只增不减）。"""
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
|
||||
assert registry.dependencies["beta_service"] == {"alpha_service"}
|
||||
assert registry.dependents["alpha_service"] == {"beta_service"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. issue #9 回归测试：注册不得清空已记录的反向依赖（dependents）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestIssue9DependentsNotOverwritten:
|
||||
"""issue #9：register_service 曾无条件 `self.dependents[name] = set()`，
|
||||
后注册的服务会清掉先前分析出的 dependents。
|
||||
当前代码（registry.py:40）已改为 setdefault —— bug 已修复，以下断言修复行为。
|
||||
"""
|
||||
|
||||
def test_registering_dependency_after_dependent_preserves_dependents(
|
||||
self, registry
|
||||
):
|
||||
# 先注册 B（依赖 A）：此时分析出 A 的 dependents 含 B
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
assert registry.dependents["alpha_service"] == {"beta_service"}
|
||||
|
||||
# 再注册 A：A 的 dependents 中 B 必须仍然存在（修复后行为）
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
assert "beta_service" in registry.dependents["alpha_service"], (
|
||||
"issue #9 回归：注册 alpha_service 清空了已记录的 dependents"
|
||||
)
|
||||
assert registry.dependents["alpha_service"] == {"beta_service"}
|
||||
|
||||
def test_initialization_order_correct_when_dependent_registered_first(
|
||||
self, registry
|
||||
):
|
||||
"""dependents 保留后，拓扑排序能把 A 排在 B 之前"""
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
|
||||
order = registry.get_initialization_order()
|
||||
assert order == ["alpha_service", "beta_service"]
|
||||
|
||||
def test_injection_works_when_dependent_registered_first(self, container):
|
||||
"""端到端：先注册 B 再注册 A，构建容器后 B 仍能注入 A"""
|
||||
container.register_service(BetaService, "beta_service")
|
||||
container.register_service(AlphaService, "alpha_service")
|
||||
container.build_container()
|
||||
|
||||
beta = container.get_service("beta_service")
|
||||
alpha = container.get_service("alpha_service")
|
||||
assert isinstance(beta.alpha_service, AlphaService)
|
||||
assert beta.alpha_service is alpha
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. 循环依赖检测
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class CycleServiceA:
|
||||
def __init__(self, dep: "Provide['cycle_b']"):
|
||||
self.dep = dep
|
||||
|
||||
|
||||
class CycleServiceB:
|
||||
def __init__(self, dep: "Provide['cycle_a']"):
|
||||
self.dep = dep
|
||||
|
||||
|
||||
class TestCircularDependencyDetection:
|
||||
def _register_cycle(self, registry):
|
||||
registry.register_service(CycleServiceA, "cycle_a")
|
||||
registry.register_service(CycleServiceB, "cycle_b")
|
||||
|
||||
def test_detect_circular_dependencies_returns_cycle_chain(self, registry):
|
||||
self._register_cycle(registry)
|
||||
cycles = registry.detect_circular_dependencies()
|
||||
|
||||
# DFS 按注册顺序从 cycle_a 出发，环以起点收尾
|
||||
assert cycles == [["cycle_a", "cycle_b", "cycle_a"]]
|
||||
|
||||
def test_detect_returns_empty_list_when_no_cycle(self, registry):
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
assert registry.detect_circular_dependencies() == []
|
||||
|
||||
def test_register_alone_does_not_raise_on_cycle(self, registry):
|
||||
"""注册阶段不检测循环——只有取初始化顺序/构建容器时才报"""
|
||||
self._register_cycle(registry) # 不抛异常即通过
|
||||
|
||||
def test_get_initialization_order_raises_value_error_with_cycle_chain(
|
||||
self, registry
|
||||
):
|
||||
self._register_cycle(registry)
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
registry.get_initialization_order()
|
||||
|
||||
msg = str(exc_info.value)
|
||||
assert "检测到循环依赖" in msg
|
||||
assert "cycle_a -> cycle_b -> cycle_a" in msg
|
||||
|
||||
def test_build_container_propagates_cycle_error(self):
|
||||
container = DependencyContainer()
|
||||
try:
|
||||
container.register_service(CycleServiceA, "cycle_a")
|
||||
container.register_service(CycleServiceB, "cycle_b")
|
||||
with pytest.raises(ValueError, match="检测到循环依赖"):
|
||||
container.build_container()
|
||||
finally:
|
||||
container.clear()
|
||||
|
||||
def test_dependency_graph_is_cached_and_goes_stale(self, registry):
|
||||
"""可疑现状：build_dependency_graph 结果被缓存，且之后的
|
||||
register_service 不会使缓存失效——后注册的服务不会出现在图中，
|
||||
只有 clear() 才重置缓存。"""
|
||||
registry.register_service(AlphaService, "alpha_service")
|
||||
graph1 = registry.build_dependency_graph()
|
||||
assert graph1 == {"alpha_service": set()}
|
||||
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
graph2 = registry.build_dependency_graph()
|
||||
assert "beta_service" not in graph2 # 缓存陈旧（现状如此）
|
||||
assert graph2 is graph1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. 单例语义
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestSingletonSemantics:
|
||||
def test_singleton_get_twice_returns_same_instance(self, container):
|
||||
container.register_service(AlphaService, "alpha_service")
|
||||
container.build_container()
|
||||
|
||||
first = container.get_service("alpha_service")
|
||||
second = container.get_service("alpha_service")
|
||||
assert first is second
|
||||
# 单例实例被缓存在 service_instances
|
||||
assert container.service_instances["alpha_service"] is first
|
||||
|
||||
def test_default_scope_is_singleton(self, container):
|
||||
container.register_service(AlphaService, "alpha_service")
|
||||
assert (
|
||||
container.service_providers["alpha_service"].scope
|
||||
== ServiceProvider.SINGLETON
|
||||
)
|
||||
|
||||
def test_factory_scope_returns_new_instance_each_time(self, container):
|
||||
container.register_service(
|
||||
AlphaService, "alpha_service", scope=ServiceProvider.FACTORY
|
||||
)
|
||||
container.build_container()
|
||||
|
||||
first = container.get_service("alpha_service")
|
||||
second = container.get_service("alpha_service")
|
||||
assert first is not second
|
||||
# 工厂模式不写入 service_instances 缓存
|
||||
assert "alpha_service" not in container.service_instances
|
||||
|
||||
def test_singleton_dependency_shared_across_consumers(self, container):
|
||||
container.register_service(AlphaService, "alpha_service")
|
||||
container.register_service(BetaService, "beta_service")
|
||||
container.register_service(GammaService, "gamma_service")
|
||||
container.build_container()
|
||||
|
||||
beta = container.get_service("beta_service")
|
||||
gamma = container.get_service("gamma_service")
|
||||
alpha = container.get_service("alpha_service")
|
||||
assert beta.alpha_service is alpha
|
||||
assert gamma.dep is alpha
|
||||
|
||||
def test_get_unregistered_service_raises_key_error(self, container):
|
||||
with pytest.raises(KeyError, match="未注册"):
|
||||
container.get_service("nope")
|
||||
|
||||
def test_get_registered_but_unbuilt_singleton_raises_runtime_error(
|
||||
self, container
|
||||
):
|
||||
"""可疑现状：注册后未 build_container 就 get，报 RuntimeError
|
||||
（提供者未配置），而不是更友好的提示。"""
|
||||
container.register_service(AlphaService, "alpha_service")
|
||||
with pytest.raises(RuntimeError, match="提供者未正确配置"):
|
||||
container.get_service("alpha_service")
|
||||
|
||||
def test_register_instance_returns_exact_object(self, container):
|
||||
sentinel = AlphaService()
|
||||
container.register_instance("external_alpha", sentinel)
|
||||
|
||||
assert container.get_service("external_alpha") is sentinel
|
||||
assert container.has_service("external_alpha") is True
|
||||
assert "external_alpha" in container.registry.known_instances
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. @service / @client 装饰器元数据
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestServiceClientDecorators:
|
||||
# 注意：issue #11 多 worker 改造为 @service/@client 新增了顶层 scope 键
|
||||
# （默认 'singleton'），以下精确断言随之更新
|
||||
def test_service_decorator_sets_metadata_with_snake_case_name(self):
|
||||
@service()
|
||||
class OrderService:
|
||||
pass
|
||||
|
||||
assert OrderService.__myboot_service__ == {
|
||||
"name": "order_service",
|
||||
"scope": "singleton",
|
||||
"kwargs": {},
|
||||
}
|
||||
|
||||
def test_service_decorator_explicit_name_and_kwargs(self):
|
||||
@service("custom_name", lazy=True)
|
||||
class OrderService:
|
||||
pass
|
||||
|
||||
assert OrderService.__myboot_service__ == {
|
||||
"name": "custom_name",
|
||||
"scope": "singleton",
|
||||
"kwargs": {"lazy": True},
|
||||
}
|
||||
|
||||
def test_service_decorator_returns_same_class(self):
|
||||
class PlainService:
|
||||
pass
|
||||
|
||||
decorated = service()(PlainService)
|
||||
assert decorated is PlainService
|
||||
|
||||
def test_client_decorator_sets_metadata(self):
|
||||
@client()
|
||||
class RedisClient:
|
||||
pass
|
||||
|
||||
assert RedisClient.__myboot_client__ == {
|
||||
"name": "redis_client",
|
||||
"scope": "singleton",
|
||||
"kwargs": {},
|
||||
}
|
||||
|
||||
def test_client_decorator_explicit_name(self):
|
||||
@client("db", pool_size=10)
|
||||
class DatabaseClient:
|
||||
pass
|
||||
|
||||
assert DatabaseClient.__myboot_client__ == {
|
||||
"name": "db",
|
||||
"scope": "singleton",
|
||||
"kwargs": {"pool_size": 10},
|
||||
}
|
||||
|
||||
def test_camel_to_snake_conversion(self):
|
||||
assert _camel_to_snake("UserService") == "user_service"
|
||||
assert _camel_to_snake("EmailService") == "email_service"
|
||||
assert _camel_to_snake("HTTPClient") == "http_client"
|
||||
assert _camel_to_snake("DatabaseClient") == "database_client"
|
||||
|
||||
def test_inject_decorator_marks_function(self):
|
||||
@inject
|
||||
def init(self, dep):
|
||||
return dep
|
||||
|
||||
# @wraps 把原函数上的 __myboot_inject__ 拷贝到 wrapper
|
||||
assert init.__myboot_inject__ is True
|
||||
assert init(None, "x") == "x"
|
||||
|
||||
def test_provide_class_getitem_returns_plain_string(self):
|
||||
"""Provide['name'] 求值结果就是普通字符串 'name'"""
|
||||
assert Provide["user_service"] == "user_service"
|
||||
assert isinstance(Provide["user_service"], str)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. 依赖分析：get_injectable_params 对构造参数/注解的解析
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestDependencyAnalysis:
|
||||
def test_self_is_skipped(self):
|
||||
params = get_injectable_params(AlphaService.__init__)
|
||||
assert "self" not in params
|
||||
assert params == {}
|
||||
|
||||
def test_class_type_annotation_maps_to_snake_case_service_name(self):
|
||||
params = get_injectable_params(BetaService.__init__)
|
||||
info = params["alpha_service"]
|
||||
assert info["type"] is AlphaService
|
||||
assert info["service_name"] == "alpha_service"
|
||||
assert info["is_optional"] is False
|
||||
assert info["default"] is None
|
||||
|
||||
def test_optional_annotation_unwraps_type_and_marks_optional(self):
|
||||
class Consumer:
|
||||
def __init__(self, dep: Optional[AlphaService] = None):
|
||||
self.dep = dep
|
||||
|
||||
info = get_injectable_params(Consumer.__init__)["dep"]
|
||||
assert info["type"] is AlphaService
|
||||
assert info["service_name"] == "alpha_service"
|
||||
assert info["is_optional"] is True
|
||||
assert info["default"] is None
|
||||
|
||||
def test_string_provide_annotation_extracts_service_name(self):
|
||||
"""注解写成字符串 "Provide['x']" 时按字面解析出服务名"""
|
||||
info = get_injectable_params(GammaService.__init__)["dep"]
|
||||
assert info["service_name"] == "alpha_service"
|
||||
assert info["type"] is None
|
||||
assert info["is_optional"] is False
|
||||
|
||||
def test_actual_provide_subscript_does_not_resolve_service_name(self):
|
||||
"""可疑现状：直接写 Provide['alpha_service']（不加引号，如
|
||||
di/decorators.py 文档示例所写）时，注解在类定义时即被求值为
|
||||
普通字符串 'alpha_service'，不匹配 "Provide['..." 前缀，
|
||||
最终 service_name 为 None（即文档示例写法实际不生效）。"""
|
||||
|
||||
class Consumer:
|
||||
def __init__(self, dep: Provide["alpha_service"]):
|
||||
self.dep = dep
|
||||
|
||||
info = get_injectable_params(Consumer.__init__)["dep"]
|
||||
assert info["service_name"] is None
|
||||
assert info["type"] == "alpha_service" # 残留为字符串
|
||||
|
||||
def test_forward_reference_string_annotation_not_resolved(self):
|
||||
"""可疑现状：普通字符串前向引用注解（'AlphaService'）不会被解析
|
||||
为服务名——字符串既不匹配 Provide 前缀也没有 __name__。"""
|
||||
|
||||
class Consumer:
|
||||
def __init__(self, dep: "AlphaService"):
|
||||
self.dep = dep
|
||||
|
||||
info = get_injectable_params(Consumer.__init__)["dep"]
|
||||
assert info["service_name"] is None
|
||||
assert info["type"] == "AlphaService"
|
||||
|
||||
def test_unannotated_param_yields_no_service_name(self):
|
||||
class Consumer:
|
||||
def __init__(self, plain):
|
||||
self.plain = plain
|
||||
|
||||
info = get_injectable_params(Consumer.__init__)["plain"]
|
||||
assert info["service_name"] is None
|
||||
assert info["type"] is inspect.Parameter.empty
|
||||
assert info["is_optional"] is False
|
||||
assert info["default"] is None
|
||||
|
||||
def test_default_value_marks_optional_and_keeps_default(self):
|
||||
class Consumer:
|
||||
def __init__(self, count=5):
|
||||
self.count = count
|
||||
|
||||
info = get_injectable_params(Consumer.__init__)["count"]
|
||||
assert info["is_optional"] is True
|
||||
assert info["default"] == 5
|
||||
|
||||
def test_registry_analysis_uses_type_annotations(self, registry):
|
||||
"""registry 的依赖分析与 get_injectable_params 一致：
|
||||
类型注解 -> snake_case 服务名"""
|
||||
|
||||
class ReportService:
|
||||
def __init__(self, alpha_service: AlphaService, untyped=None):
|
||||
pass
|
||||
|
||||
registry.register_service(ReportService, "report_service")
|
||||
# 只有可解析出服务名的参数才进入依赖集合
|
||||
assert registry.dependencies["report_service"] == {"alpha_service"}
|
||||
|
||||
def test_registry_analysis_with_string_provide(self, registry):
|
||||
registry.register_service(GammaService, "gamma_service")
|
||||
assert registry.dependencies["gamma_service"] == {"alpha_service"}
|
||||
|
||||
def test_unregistered_dependency_only_warns_not_raises(self, registry):
|
||||
"""依赖的服务从未注册时，build/排序只告警不报错，缺失服务被
|
||||
当作图外节点（现状行为）"""
|
||||
registry.register_service(BetaService, "beta_service")
|
||||
order = registry.get_initialization_order()
|
||||
assert order == ["beta_service"]
|
||||
@@ -0,0 +1,649 @@
|
||||
"""
|
||||
日志系统特征测试（characterization tests）
|
||||
|
||||
固化 myboot/core/logger.py 的「当前实际行为」，作为后续重构的兼容性闸门。
|
||||
失败时优先修改测试以匹配源码行为（除非源码行为本身是回归）。
|
||||
|
||||
注意:
|
||||
- loguru 是全局单例，每个测试添加的 sink 必须移除（见 _loguru_isolation fixture）。
|
||||
- setup_logging / setup_worker_logging 会调用 loguru_logger.remove() 清掉所有
|
||||
handler（包括测试 sink），因此捕获 sink 必须在 setup 调用之后添加，
|
||||
或者通过 capsys 捕获 stdout。
|
||||
- 始终显式传入 Dynaconf 配置对象，避免 get_settings() 全局单例读取
|
||||
项目根目录的 conf/config.yaml 造成测试间/环境间耦合。
|
||||
"""
|
||||
|
||||
import inspect
|
||||
import json
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
from dynaconf import Dynaconf
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
from myboot.core.logger import (
|
||||
Logger,
|
||||
_configure_third_party_loggers,
|
||||
_convert_logging_format_to_loguru,
|
||||
_get_worker_info,
|
||||
_parse_json_config,
|
||||
configure_worker_logger,
|
||||
get_logger,
|
||||
setup_logging,
|
||||
setup_worker_logging,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _loguru_isolation():
|
||||
"""保证测试顺序无关性。
|
||||
|
||||
loguru 是全局单例；setup_logging/setup_worker_logging 会 remove() 所有
|
||||
handler 并 configure() 全局 extra。每个测试结束后清掉所有 handler 并
|
||||
重置 extra，使下一个测试从确定状态开始（无 handler、空 extra）。
|
||||
"""
|
||||
yield
|
||||
loguru_logger.remove()
|
||||
loguru_logger.configure(extra={})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def capture_records():
|
||||
"""添加内存 sink 捕获 loguru record，返回 (records, add) 工具。
|
||||
|
||||
add() 在调用时刻挂 sink（便于在 setup_* 调用之后再挂，避免被
|
||||
loguru_logger.remove() 清掉），finally 由 _loguru_isolation 兜底移除。
|
||||
"""
|
||||
records = []
|
||||
handler_ids = []
|
||||
|
||||
def _add(level="DEBUG"):
|
||||
hid = loguru_logger.add(
|
||||
lambda message: records.append(message),
|
||||
format="{message}",
|
||||
level=level,
|
||||
)
|
||||
handler_ids.append(hid)
|
||||
return hid
|
||||
|
||||
try:
|
||||
yield records, _add
|
||||
finally:
|
||||
for hid in handler_ids:
|
||||
try:
|
||||
loguru_logger.remove(hid)
|
||||
except ValueError:
|
||||
# 已被 setup_* 的 remove() 清掉
|
||||
pass
|
||||
|
||||
|
||||
def _make_config(tmp_path, yaml_content: str) -> Dynaconf:
|
||||
"""从 YAML 字符串创建独立的 Dynaconf 配置对象（不经过全局 get_settings）"""
|
||||
config_path = tmp_path / "logger_test_config.yaml"
|
||||
config_path.write_text(yaml_content, encoding="utf-8")
|
||||
return Dynaconf(settings_files=[str(config_path)])
|
||||
|
||||
|
||||
def _messages(records):
|
||||
"""提取捕获到的纯文本消息列表（去掉末尾换行）"""
|
||||
return [str(m).rstrip("\n") for m in records]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. issue #5 回归：setup_worker_logging 应遵循配置的日志级别（已修复）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSetupWorkerLoggingLevel:
|
||||
def test_signature_accepts_config(self):
|
||||
"""修复后的签名：setup_worker_logging(worker_id, total_workers, config=None)"""
|
||||
params = inspect.signature(setup_worker_logging).parameters
|
||||
assert list(params) == ["worker_id", "total_workers", "config"]
|
||||
assert params["config"].default is None
|
||||
|
||||
def test_respects_configured_warning_level(self, tmp_path, capsys):
|
||||
"""配置 WARNING 级别时，DEBUG/INFO 被抑制（不再硬编码 DEBUG）"""
|
||||
cfg = _make_config(tmp_path, "logging:\n level: WARNING\n")
|
||||
setup_worker_logging(worker_id=1, total_workers=2, config=cfg)
|
||||
|
||||
loguru_logger.debug("debug-should-not-appear-xyz")
|
||||
loguru_logger.info("info-should-not-appear-xyz")
|
||||
loguru_logger.warning("warning-should-appear-xyz")
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert "debug-should-not-appear-xyz" not in out
|
||||
assert "info-should-not-appear-xyz" not in out
|
||||
assert "warning-should-appear-xyz" in out
|
||||
|
||||
def test_respects_configured_debug_level(self, tmp_path, capsys):
|
||||
"""配置 DEBUG 级别时，DEBUG 消息正常输出"""
|
||||
cfg = _make_config(tmp_path, "logging:\n level: DEBUG\n")
|
||||
setup_worker_logging(worker_id=1, total_workers=2, config=cfg)
|
||||
|
||||
loguru_logger.debug("debug-should-appear-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "debug-should-appear-xyz" in out
|
||||
|
||||
def test_default_level_is_info_when_config_missing_key(self, tmp_path, capsys):
|
||||
"""配置中无 logging.level 时，当前默认级别为 INFO"""
|
||||
cfg = _make_config(tmp_path, "app:\n name: demo\n")
|
||||
setup_worker_logging(worker_id=1, total_workers=2, config=cfg)
|
||||
|
||||
loguru_logger.debug("debug-hidden-by-default-xyz")
|
||||
loguru_logger.info("info-shown-by-default-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "debug-hidden-by-default-xyz" not in out
|
||||
assert "info-shown-by-default-xyz" in out
|
||||
|
||||
def test_output_contains_worker_tag(self, tmp_path, capsys):
|
||||
"""worker 日志格式包含 Worker-{id}/{total} 标识"""
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n")
|
||||
setup_worker_logging(worker_id=3, total_workers=8, config=cfg)
|
||||
|
||||
loguru_logger.info("hello-from-worker-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "Worker-3/8" in out
|
||||
assert "hello-from-worker-xyz" in out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Logger 兼容类
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestLoggerCompatClass:
|
||||
def test_instantiation_stores_name(self):
|
||||
log = Logger("svc")
|
||||
assert log.name == "svc"
|
||||
|
||||
def test_default_name_is_app(self):
|
||||
assert Logger().name == "app"
|
||||
|
||||
def test_all_level_methods_callable_without_error(self, capture_records):
|
||||
records, add = capture_records
|
||||
add(level="DEBUG")
|
||||
log = Logger("compat")
|
||||
log.debug("d-msg")
|
||||
log.info("i-msg")
|
||||
log.warning("w-msg")
|
||||
log.error("e-msg")
|
||||
log.critical("c-msg")
|
||||
# exception() 在无活动异常时也不抛错（loguru 行为）
|
||||
log.exception("x-msg")
|
||||
msgs = _messages(records)
|
||||
assert msgs[:5] == ["d-msg", "i-msg", "w-msg", "e-msg", "c-msg"]
|
||||
# 可疑现状：无活动异常时 exception() 仍输出异常段，
|
||||
# 格式化文本末尾附加 "NoneType: None"
|
||||
assert msgs[5] == "x-msg\nNoneType: None"
|
||||
|
||||
def test_bind_name_in_extra(self):
|
||||
"""Logger 内部 _logger 是 bind(name=...) 后的 loguru logger，
|
||||
name 进入 record["extra"]["name"]（不影响 record["name"] 即模块名）"""
|
||||
records = []
|
||||
hid = loguru_logger.add(
|
||||
lambda m: records.append(m.record), format="{message}", level="DEBUG"
|
||||
)
|
||||
try:
|
||||
Logger("my-component").info("bound message")
|
||||
finally:
|
||||
loguru_logger.remove(hid)
|
||||
|
||||
assert len(records) == 1
|
||||
assert records[0]["extra"]["name"] == "my-component"
|
||||
# 可疑现状：record["name"]（loguru 内置，模块路径）不受 bind 影响，
|
||||
# 仍为调用方模块名，而非传入的 name —— 文本格式 {name} 显示的是模块名
|
||||
assert records[0]["name"] != "my-component"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. get_logger
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestGetLogger:
|
||||
def test_returns_logger_bound_with_name(self):
|
||||
records = []
|
||||
hid = loguru_logger.add(
|
||||
lambda m: records.append(m.record), format="{message}", level="DEBUG"
|
||||
)
|
||||
try:
|
||||
log = get_logger("api")
|
||||
log.info("from get_logger")
|
||||
finally:
|
||||
loguru_logger.remove(hid)
|
||||
|
||||
assert len(records) == 1
|
||||
assert records[0]["extra"]["name"] == "api"
|
||||
|
||||
def test_default_name_is_app(self):
|
||||
records = []
|
||||
hid = loguru_logger.add(
|
||||
lambda m: records.append(m.record), format="{message}", level="DEBUG"
|
||||
)
|
||||
try:
|
||||
get_logger().info("default name")
|
||||
finally:
|
||||
loguru_logger.remove(hid)
|
||||
|
||||
assert records[0]["extra"]["name"] == "app"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. loguru {} 占位符格式化
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestBracePlaceholderFormatting:
|
||||
def test_positional_brace_formatting(self, capture_records):
|
||||
records, add = capture_records
|
||||
add()
|
||||
loguru_logger.info("hello {}", "world")
|
||||
assert _messages(records) == ["hello world"]
|
||||
|
||||
def test_named_brace_formatting_via_kwargs(self, capture_records):
|
||||
records, add = capture_records
|
||||
add()
|
||||
loguru_logger.info("hello {who}", who="myboot")
|
||||
assert _messages(records) == ["hello myboot"]
|
||||
|
||||
def test_logger_compat_class_supports_brace_args(self, capture_records):
|
||||
"""Logger 兼容类透传 *args，因此 {} 占位符同样可用"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("count={}", 42)
|
||||
assert _messages(records) == ["count=42"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. %s 风格调用（issue #12：Logger 兼容类同时支持 % 与 {} 占位符）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPercentStyleSupport:
|
||||
def test_percent_s_is_formatted(self, capture_records):
|
||||
"""issue #12：原行为是 %s 字面量输出、参数静默丢弃；
|
||||
修复后 Logger 兼容类对 % 占位符做预格式化"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("hello %s", "world")
|
||||
assert _messages(records) == ["hello world"]
|
||||
|
||||
def test_percent_multiple_args_and_types(self, capture_records):
|
||||
"""issue #12：%s/%d 多参数预格式化"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("user=%s count=%d", "alice", 3)
|
||||
assert _messages(records) == ["user=alice count=3"]
|
||||
|
||||
def test_percent_works_on_all_level_methods(self, capture_records):
|
||||
"""issue #12：六个级别方法统一支持 % 占位符"""
|
||||
records, add = capture_records
|
||||
add(level="DEBUG")
|
||||
log = Logger("x")
|
||||
log.debug("d=%s", 1)
|
||||
log.info("i=%s", 2)
|
||||
log.warning("w=%s", 3)
|
||||
log.error("e=%s", 4)
|
||||
log.critical("c=%s", 5)
|
||||
log.exception("x=%s", 6)
|
||||
msgs = _messages(records)
|
||||
assert msgs[:5] == ["d=1", "i=2", "w=3", "e=4", "c=5"]
|
||||
# exception() 无活动异常时附加 "NoneType: None"（既有 loguru 行为）
|
||||
assert msgs[5] == "x=6\nNoneType: None"
|
||||
|
||||
def test_double_percent_escape(self, capture_records):
|
||||
"""issue #12：%% 是转义不是占位符，% 预格式化后输出单个 %"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("progress 100%% by %s", "worker")
|
||||
assert _messages(records) == ["progress 100% by worker"]
|
||||
|
||||
def test_double_percent_only_without_real_placeholder(self, capture_records):
|
||||
"""issue #12：消息只含 %%（无真正 % 占位符）且带参数时，
|
||||
不做 % 预格式化，参数交给 loguru（无 {} 则被忽略），%% 原样保留"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("progress 100%%", "ignored")
|
||||
assert _messages(records) == ["progress 100%%"]
|
||||
|
||||
def test_mixed_percent_and_brace_prefers_brace(self, capture_records):
|
||||
"""issue #12 设计决策：消息同时含 % 和 {} 占位符时，{} 优先，
|
||||
原样交给 loguru 的 str.format（保持 loguru 原生语义），% 保留字面量"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("pct=%s val={}", 42)
|
||||
assert _messages(records) == ["pct=%s val=42"]
|
||||
|
||||
def test_percent_args_mismatch_falls_back_without_raising(self, capture_records):
|
||||
"""issue #12：% 预格式化参数不匹配时回退为原样输出，不向调用方抛错"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("a=%s b=%s", "only-one")
|
||||
assert _messages(records) == ["a=%s b=%s"]
|
||||
|
||||
def test_lone_brace_with_args_does_not_raise(self, capture_records):
|
||||
"""issue #12：原行为是孤立 { 带参数时 loguru 的 str.format 异常
|
||||
直接传播；修复后走 % 预格式化路径（无完整 {} 占位符），不抛错"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("ratio %s {bad", "x")
|
||||
assert _messages(records) == ["ratio x {bad"]
|
||||
|
||||
def test_lone_brace_without_percent_degrades_safely(self, capture_records):
|
||||
"""issue #12：孤立 { 且无 % 占位符但带参数时，loguru 内部
|
||||
str.format 抛错被捕获，降级为无参输出消息本体，不抛错"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("oops {bad", "x")
|
||||
assert _messages(records) == ["oops {bad"]
|
||||
|
||||
def test_brace_format_failure_degrades_safely(self, capture_records):
|
||||
"""issue #12：{} 路径下 str.format 仍可能失败（如孤立 { 与 {} 共存），
|
||||
捕获后降级输出消息本体，不抛错"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("{bad and {}", "x")
|
||||
assert _messages(records) == ["{bad and {}"]
|
||||
|
||||
def test_percent_s_on_raw_loguru_logger_unchanged(self, capture_records):
|
||||
"""裸 loguru logger 不受 issue #12 影响：仍原样输出 %s，参数被丢弃"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
loguru_logger.info("value=%s", 123)
|
||||
assert _messages(records) == ["value=%s"]
|
||||
|
||||
def test_raw_loguru_brace_chars_still_raise(self, capture_records):
|
||||
"""裸 loguru logger 不受 issue #12 影响：含未转义 { } 且带参数时
|
||||
str.format() 异常仍直接传播"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
with pytest.raises((ValueError, KeyError, IndexError)):
|
||||
loguru_logger.info("ratio %s {bad", "x")
|
||||
|
||||
def test_message_without_args_not_formatted(self, capture_records):
|
||||
"""无参数时不做任何格式化，含 % 或 {} 的消息原样输出（Logger 类与裸 loguru 一致）"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
loguru_logger.info("progress 100% {not-a-placeholder}")
|
||||
Logger("x").info("progress 100% {not-a-placeholder}")
|
||||
assert _messages(records) == [
|
||||
"progress 100% {not-a-placeholder}",
|
||||
"progress 100% {not-a-placeholder}",
|
||||
]
|
||||
|
||||
|
||||
class TestBracePathNoRegression:
|
||||
"""issue #12 回归保护：{} 路径与命名 kwargs 用法保持不变"""
|
||||
|
||||
def test_logger_brace_positional_not_regressed(self, capture_records):
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("count={} name={}", 42, "svc")
|
||||
assert _messages(records) == ["count=42 name=svc"]
|
||||
|
||||
def test_logger_named_kwargs_not_regressed(self, capture_records):
|
||||
"""命名 {name} 占位符通过 kwargs 透传给 loguru"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("hello {who}", who="myboot")
|
||||
assert _messages(records) == ["hello myboot"]
|
||||
|
||||
def test_logger_named_kwargs_with_positional_args(self, capture_records):
|
||||
"""位置 {} 与命名 {name} 混用时 args/kwargs 均透传"""
|
||||
records, add = capture_records
|
||||
add()
|
||||
Logger("x").info("{} meets {who}", "alice", who="bob")
|
||||
assert _messages(records) == ["alice meets bob"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. JSON 日志格式
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestJsonLogging:
|
||||
def test_setup_logging_json_outputs_valid_json(self, tmp_path, capsys, clean_myboot_env):
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n json: true\n")
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("json hello")
|
||||
out = capsys.readouterr().out.strip()
|
||||
|
||||
parsed = json.loads(out)
|
||||
assert parsed["record"]["message"] == "json hello"
|
||||
assert parsed["record"]["level"]["name"] == "INFO"
|
||||
# setup_logging 配置了全局 extra worker（单进程为 "Main"）
|
||||
assert parsed["record"]["extra"]["worker"] == "Main"
|
||||
# serialize=True 时同时包含人类可读的 text 字段
|
||||
assert "json hello" in parsed["text"]
|
||||
|
||||
def test_json_config_accepts_string_true(self, tmp_path, capsys, clean_myboot_env):
|
||||
"""logging.json 为字符串 "true" 时同样启用 JSON（_parse_json_config）"""
|
||||
cfg = _make_config(tmp_path, 'logging:\n level: INFO\n json: "true"\n')
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("string-true json")
|
||||
out = capsys.readouterr().out.strip()
|
||||
assert json.loads(out)["record"]["message"] == "string-true json"
|
||||
|
||||
def test_parse_json_config_truth_table(self):
|
||||
"""固化 _parse_json_config 的当前真值表"""
|
||||
assert _parse_json_config(True) is True
|
||||
assert _parse_json_config(False) is False
|
||||
assert _parse_json_config("true") is True
|
||||
assert _parse_json_config("TRUE") is True
|
||||
assert _parse_json_config("1") is True
|
||||
assert _parse_json_config("yes") is True
|
||||
assert _parse_json_config("on") is True
|
||||
assert _parse_json_config("false") is False
|
||||
assert _parse_json_config("off") is False
|
||||
# 可疑现状：任意其他字符串（如 "enabled"）一律为 False
|
||||
assert _parse_json_config("enabled") is False
|
||||
# 可疑现状：非 bool/str 走 bool() 转换，整数 1 为 True
|
||||
assert _parse_json_config(1) is True
|
||||
assert _parse_json_config(0) is False
|
||||
assert _parse_json_config(None) is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. 第三方库日志级别控制
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestThirdPartyLoggerControl:
|
||||
def test_configure_third_party_loggers_sets_std_logging_level(self):
|
||||
name = "myboot_test_3rdparty_a"
|
||||
try:
|
||||
_configure_third_party_loggers({name: "WARNING"})
|
||||
assert logging.getLogger(name).level == logging.WARNING
|
||||
|
||||
_configure_third_party_loggers({name: "error"}) # 大小写不敏感
|
||||
assert logging.getLogger(name).level == logging.ERROR
|
||||
finally:
|
||||
logging.getLogger(name).setLevel(logging.NOTSET)
|
||||
|
||||
def test_unknown_level_falls_back_to_info(self):
|
||||
"""可疑现状：无效级别名静默回退为 INFO（getattr 默认值），不报错"""
|
||||
name = "myboot_test_3rdparty_b"
|
||||
try:
|
||||
_configure_third_party_loggers({name: "NOT_A_LEVEL"})
|
||||
assert logging.getLogger(name).level == logging.INFO
|
||||
finally:
|
||||
logging.getLogger(name).setLevel(logging.NOTSET)
|
||||
|
||||
def test_non_dict_and_non_str_values_ignored(self):
|
||||
name = "myboot_test_3rdparty_c"
|
||||
try:
|
||||
# 非 dict 输入：直接 return，不抛错
|
||||
_configure_third_party_loggers("not-a-dict")
|
||||
_configure_third_party_loggers(None)
|
||||
# 可疑现状：级别值非字符串（如 int 30）被静默忽略
|
||||
_configure_third_party_loggers({name: 30})
|
||||
assert logging.getLogger(name).level == logging.NOTSET
|
||||
finally:
|
||||
logging.getLogger(name).setLevel(logging.NOTSET)
|
||||
|
||||
def test_setup_logging_applies_third_party_config(self, tmp_path, capsys, clean_myboot_env):
|
||||
name = "myboot_test_3rdparty_d"
|
||||
cfg = _make_config(
|
||||
tmp_path,
|
||||
f"logging:\n level: INFO\n third_party:\n {name}: ERROR\n",
|
||||
)
|
||||
try:
|
||||
setup_logging(config=cfg)
|
||||
assert logging.getLogger(name).level == logging.ERROR
|
||||
finally:
|
||||
logging.getLogger(name).setLevel(logging.NOTSET)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# setup_logging 整体行为
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSetupLogging:
|
||||
def test_respects_configured_level(self, tmp_path, capsys, clean_myboot_env):
|
||||
cfg = _make_config(tmp_path, "logging:\n level: ERROR\n")
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("info-suppressed-xyz")
|
||||
loguru_logger.error("error-shown-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "info-suppressed-xyz" not in out
|
||||
assert "error-shown-xyz" in out
|
||||
|
||||
def test_removes_preexisting_handlers(self, tmp_path, capsys, clean_myboot_env):
|
||||
"""setup_logging 调用 loguru_logger.remove() 清掉之前的所有 sink"""
|
||||
before = []
|
||||
loguru_logger.add(lambda m: before.append(m), format="{message}")
|
||||
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n")
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("after-setup-xyz")
|
||||
assert before == [] # 之前的 sink 已被移除，收不到任何消息
|
||||
assert "after-setup-xyz" in capsys.readouterr().out
|
||||
|
||||
def test_single_worker_uses_simple_format_without_worker_tag(
|
||||
self, tmp_path, capsys, clean_myboot_env
|
||||
):
|
||||
"""无 MYBOOT_WORKER_COUNT 环境变量时使用简单格式，不含 worker 标识"""
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n")
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("simple-format-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "simple-format-xyz" in out
|
||||
assert "Main" not in out
|
||||
assert "Worker-" not in out
|
||||
|
||||
def test_multi_worker_env_enables_worker_tag(self, tmp_path, capsys, clean_myboot_env):
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_COUNT", "4")
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_ID", "2")
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n")
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("tagged-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "Worker-2/4" in out
|
||||
assert "tagged-xyz" in out
|
||||
|
||||
def test_enable_worker_info_false_suppresses_worker_tag(
|
||||
self, tmp_path, capsys, clean_myboot_env
|
||||
):
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_COUNT", "4")
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_ID", "2")
|
||||
cfg = _make_config(tmp_path, "logging:\n level: INFO\n")
|
||||
setup_logging(config=cfg, enable_worker_info=False)
|
||||
|
||||
loguru_logger.info("untagged-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "Worker-2/4" not in out
|
||||
assert "untagged-xyz" in out
|
||||
|
||||
def test_custom_logging_format_converted_to_loguru(self, tmp_path, capsys, clean_myboot_env):
|
||||
"""logging.format 支持标准 logging 风格 %(...)s 占位符并被转换"""
|
||||
cfg = _make_config(
|
||||
tmp_path,
|
||||
'logging:\n level: INFO\n format: "%(levelname)s | %(message)s"\n',
|
||||
)
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("converted-format-xyz")
|
||||
out = capsys.readouterr().out
|
||||
assert "INFO" in out
|
||||
assert "converted-format-xyz" in out
|
||||
|
||||
def test_file_handler_writes_to_configured_path(self, tmp_path, capsys, clean_myboot_env):
|
||||
log_file = tmp_path / "logs" / "app.log"
|
||||
cfg = _make_config(
|
||||
tmp_path,
|
||||
f"logging:\n level: INFO\n file: {json.dumps(str(log_file))}\n",
|
||||
)
|
||||
setup_logging(config=cfg)
|
||||
|
||||
loguru_logger.info("written-to-file-xyz")
|
||||
# 移除 handler 以确保文件被 flush/关闭后再读取（Windows）
|
||||
loguru_logger.remove()
|
||||
|
||||
assert log_file.exists() # 父目录由 _add_file_handler 自动创建
|
||||
content = log_file.read_text(encoding="utf-8")
|
||||
assert "written-to-file-xyz" in content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# configure_worker_logger 与辅助函数
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWorkerContextHelpers:
|
||||
def test_configure_worker_logger_sets_extra_worker(self):
|
||||
configure_worker_logger(worker_id=3, total_workers=8)
|
||||
|
||||
records = []
|
||||
hid = loguru_logger.add(
|
||||
lambda m: records.append(m.record), format="{message}", level="DEBUG"
|
||||
)
|
||||
try:
|
||||
loguru_logger.info("ctx check")
|
||||
finally:
|
||||
loguru_logger.remove(hid)
|
||||
|
||||
assert records[0]["extra"]["worker"] == "Worker-3/8"
|
||||
|
||||
def test_get_worker_info_main_without_env(self, clean_myboot_env):
|
||||
assert _get_worker_info() == "Main"
|
||||
|
||||
def test_get_worker_info_with_env(self, clean_myboot_env):
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_ID", "2")
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_COUNT", "4")
|
||||
assert _get_worker_info() == "Worker-2/4"
|
||||
|
||||
def test_get_worker_info_requires_both_env_vars(self, clean_myboot_env):
|
||||
"""可疑现状：只设置 WORKER_ID 而无 WORKER_COUNT 时返回 "Main" """
|
||||
clean_myboot_env.setenv("MYBOOT_WORKER_ID", "2")
|
||||
assert _get_worker_info() == "Main"
|
||||
|
||||
|
||||
class TestFormatConversion:
|
||||
def test_standard_logging_placeholders_converted(self):
|
||||
converted = _convert_logging_format_to_loguru(
|
||||
"%(asctime)s %(name)s %(levelname)s %(message)s "
|
||||
"%(filename)s %(funcName)s %(lineno)d"
|
||||
)
|
||||
assert converted == (
|
||||
"{time:YYYY-MM-DD HH:mm:ss} {name} {level: <8} {message} "
|
||||
"{file.name} {function} {line}"
|
||||
)
|
||||
|
||||
def test_unknown_placeholders_left_untouched(self):
|
||||
"""可疑现状：映射表之外的占位符（如 %(thread)d）原样保留，
|
||||
进入 loguru format 后会原样输出在每条日志中"""
|
||||
assert _convert_logging_format_to_loguru("%(thread)d %(message)s") == (
|
||||
"%(thread)d {message}"
|
||||
)
|
||||
@@ -0,0 +1,285 @@
|
||||
"""
|
||||
F5 内置 Prometheus 指标模块单元测试
|
||||
|
||||
覆盖：
|
||||
- prometheus-client 未安装时的优雅退化（no-op 桩、Application 只告警不挂）
|
||||
- setup_multiproc_env 的各种生效/不生效分支
|
||||
- get_counter / get_histogram 幂等缓存
|
||||
- HttpMetricsMiddleware 路由模板标签、unmatched 归类与 metrics 自身路径排除
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import myboot.metrics as metrics
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.config import get_settings
|
||||
|
||||
|
||||
def _uniq(prefix: str) -> str:
|
||||
return f"{prefix}_{uuid.uuid4().hex}"
|
||||
|
||||
|
||||
class FakeConfig:
|
||||
"""支持点号 key 的最小配置桩"""
|
||||
|
||||
def __init__(self, data=None):
|
||||
self.data = data or {}
|
||||
|
||||
def get(self, key, default=None):
|
||||
return self.data.get(key, default)
|
||||
|
||||
|
||||
# ==================== 可用性与 no-op 退化 ====================
|
||||
|
||||
|
||||
class TestAvailability:
|
||||
def test_is_available_true_when_installed(self):
|
||||
# prometheus-client 已随 test extra 安装
|
||||
assert metrics.is_available() is True
|
||||
|
||||
def test_is_available_false_when_not_installed(self, monkeypatch):
|
||||
monkeypatch.setattr(importlib.util, "find_spec", lambda name: None)
|
||||
assert metrics.is_available() is False
|
||||
|
||||
def test_get_counter_noop_when_unavailable(self, monkeypatch):
|
||||
monkeypatch.setattr(metrics, "is_available", lambda: False)
|
||||
counter = metrics.get_counter(_uniq("noop_counter"), "doc", ("a", "b"))
|
||||
# 链式调用不报错
|
||||
counter.labels(a="1", b="2").inc()
|
||||
counter.inc(5)
|
||||
assert isinstance(counter, metrics._NoopMetric)
|
||||
|
||||
def test_get_histogram_noop_when_unavailable(self, monkeypatch):
|
||||
monkeypatch.setattr(metrics, "is_available", lambda: False)
|
||||
hist = metrics.get_histogram(_uniq("noop_hist"), "doc", ("stage",))
|
||||
hist.labels(stage="x").observe(0.1)
|
||||
assert isinstance(hist, metrics._NoopMetric)
|
||||
|
||||
def test_observe_stage_and_time_stage_noop_when_unavailable(self, monkeypatch):
|
||||
monkeypatch.setattr(metrics, "is_available", lambda: False)
|
||||
metrics.observe_stage("recall", 0.05)
|
||||
with metrics.time_stage("rank"):
|
||||
pass
|
||||
|
||||
def test_application_only_warns_when_enabled_but_not_installed(self, monkeypatch):
|
||||
monkeypatch.setattr(metrics, "is_available", lambda: False)
|
||||
settings = get_settings()
|
||||
settings.set("metrics.enabled", True)
|
||||
try:
|
||||
app = Application(name="metrics-warn-test", auto_configuration=False)
|
||||
fastapi_app = app.get_fastapi_app()
|
||||
# 未安装 → 不挂载 /metrics、不加中间件，应用可正常服务
|
||||
client = TestClient(fastapi_app)
|
||||
resp = client.get("/health")
|
||||
assert resp.status_code == 200
|
||||
resp = client.get("/metrics")
|
||||
assert resp.status_code == 404
|
||||
finally:
|
||||
settings.set("metrics.enabled", False)
|
||||
|
||||
|
||||
# ==================== is_enabled ====================
|
||||
|
||||
|
||||
class TestIsEnabled:
|
||||
@pytest.mark.parametrize(
|
||||
"value,expected",
|
||||
[
|
||||
(True, True),
|
||||
(False, False),
|
||||
("true", True),
|
||||
("True", True),
|
||||
("1", True),
|
||||
("yes", True),
|
||||
("on", True),
|
||||
("false", False),
|
||||
("0", False),
|
||||
(None, False),
|
||||
],
|
||||
)
|
||||
def test_bool_and_string_values(self, value, expected):
|
||||
cfg = FakeConfig({"metrics.enabled": value})
|
||||
assert metrics.is_enabled(cfg) is expected
|
||||
|
||||
def test_default_false(self):
|
||||
assert metrics.is_enabled(FakeConfig()) is False
|
||||
|
||||
|
||||
# ==================== setup_multiproc_env ====================
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clean_prom_env(monkeypatch):
|
||||
monkeypatch.delenv("PROMETHEUS_MULTIPROC_DIR", raising=False)
|
||||
monkeypatch.delenv("MYBOOT_WORKER_ID", raising=False)
|
||||
return monkeypatch
|
||||
|
||||
|
||||
class TestSetupMultiprocEnv:
|
||||
def _cfg(self, workers=4, enabled=True, multiproc_dir=None):
|
||||
return FakeConfig(
|
||||
{
|
||||
"metrics.enabled": enabled,
|
||||
"server.workers": workers,
|
||||
"metrics.multiproc_dir": multiproc_dir,
|
||||
}
|
||||
)
|
||||
|
||||
def test_single_worker_noop(self, clean_prom_env):
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
metrics.setup_multiproc_env(self._cfg(workers=1), "demo")
|
||||
assert "PROMETHEUS_MULTIPROC_DIR" not in os.environ
|
||||
|
||||
def test_disabled_noop(self, clean_prom_env):
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
metrics.setup_multiproc_env(self._cfg(workers=4, enabled=False), "demo")
|
||||
assert "PROMETHEUS_MULTIPROC_DIR" not in os.environ
|
||||
|
||||
def test_win32_noop(self, clean_prom_env):
|
||||
clean_prom_env.setattr(sys, "platform", "win32")
|
||||
metrics.setup_multiproc_env(self._cfg(workers=4), "demo")
|
||||
assert "PROMETHEUS_MULTIPROC_DIR" not in os.environ
|
||||
|
||||
def test_multiworker_sets_env_and_cleans_stale_db(self, clean_prom_env, tmp_path):
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
target = tmp_path / "prom_dir"
|
||||
target.mkdir()
|
||||
stale = target / "counter_12345.db"
|
||||
stale.write_bytes(b"stale")
|
||||
keep = target / "note.txt"
|
||||
keep.write_text("keep")
|
||||
|
||||
metrics.setup_multiproc_env(
|
||||
self._cfg(workers=4, multiproc_dir=str(target)), "demo"
|
||||
)
|
||||
|
||||
assert os.environ["PROMETHEUS_MULTIPROC_DIR"] == str(target)
|
||||
assert target.is_dir()
|
||||
assert not stale.exists() # 陈旧 .db 被父进程清理
|
||||
assert keep.exists() # 非 .db 文件保留
|
||||
|
||||
def test_worker_process_does_not_clean(self, clean_prom_env, tmp_path):
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
clean_prom_env.setenv("MYBOOT_WORKER_ID", "2")
|
||||
target = tmp_path / "prom_dir"
|
||||
target.mkdir()
|
||||
stale = target / "counter_12345.db"
|
||||
stale.write_bytes(b"stale")
|
||||
|
||||
metrics.setup_multiproc_env(
|
||||
self._cfg(workers=4, multiproc_dir=str(target)), "demo"
|
||||
)
|
||||
assert stale.exists() # 非父进程不清理
|
||||
|
||||
def test_user_preset_env_respected(self, clean_prom_env, tmp_path):
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
user_dir = tmp_path / "user_dir"
|
||||
clean_prom_env.setenv("PROMETHEUS_MULTIPROC_DIR", str(user_dir))
|
||||
metrics.setup_multiproc_env(
|
||||
self._cfg(workers=4, multiproc_dir=str(tmp_path / "other")), "demo"
|
||||
)
|
||||
assert os.environ["PROMETHEUS_MULTIPROC_DIR"] == str(user_dir)
|
||||
assert user_dir.is_dir()
|
||||
|
||||
def test_default_dir_under_tempdir_with_app_slug(self, clean_prom_env):
|
||||
import tempfile
|
||||
|
||||
clean_prom_env.setattr(sys, "platform", "linux")
|
||||
metrics.setup_multiproc_env(self._cfg(workers=2), "My Demo App!")
|
||||
value = os.environ["PROMETHEUS_MULTIPROC_DIR"]
|
||||
assert value.startswith(tempfile.gettempdir())
|
||||
assert "myboot_prometheus_my_demo_app" in value
|
||||
|
||||
|
||||
# ==================== 指标工厂幂等性 ====================
|
||||
|
||||
|
||||
class TestMetricFactories:
|
||||
def test_get_counter_idempotent(self):
|
||||
name = _uniq("idem_counter")
|
||||
c1 = metrics.get_counter(name, "doc", ("x",))
|
||||
c2 = metrics.get_counter(name, "doc", ("x",))
|
||||
assert c1 is c2
|
||||
c1.labels(x="a").inc()
|
||||
|
||||
def test_get_histogram_idempotent(self):
|
||||
name = _uniq("idem_hist")
|
||||
h1 = metrics.get_histogram(name, "doc", ("x",))
|
||||
h2 = metrics.get_histogram(name, "doc", ("x",))
|
||||
assert h1 is h2
|
||||
h1.labels(x="a").observe(0.5)
|
||||
|
||||
def test_observe_stage_negative_ignored(self):
|
||||
metrics.observe_stage("noop", -1.0) # 不报错
|
||||
|
||||
def test_time_stage_records(self):
|
||||
with metrics.time_stage(_uniq("stage")):
|
||||
pass
|
||||
|
||||
|
||||
# ==================== HttpMetricsMiddleware 端到端 ====================
|
||||
|
||||
|
||||
class TestHttpMetricsMiddleware:
|
||||
@pytest.fixture
|
||||
def metrics_app(self):
|
||||
settings = get_settings()
|
||||
settings.set("metrics.enabled", True)
|
||||
try:
|
||||
app = Application(name="metrics-mw-test", auto_configuration=False)
|
||||
|
||||
async def get_item(item_id: str):
|
||||
return {"id": item_id}
|
||||
|
||||
app.add_route("/items/{item_id}", get_item, ["GET"])
|
||||
yield app
|
||||
finally:
|
||||
settings.set("metrics.enabled", False)
|
||||
|
||||
def test_http_metrics_collected(self, metrics_app):
|
||||
client = TestClient(metrics_app.get_fastapi_app())
|
||||
|
||||
assert client.get("/items/1").status_code == 200
|
||||
assert client.get("/items/2").status_code == 200
|
||||
assert client.get("/no/such/route").status_code == 404
|
||||
|
||||
scrape = client.get("/metrics")
|
||||
assert scrape.status_code == 200
|
||||
text = scrape.text
|
||||
|
||||
# path 标签是路由模板，两个不同 id 归并到同一序列
|
||||
template_lines = [
|
||||
line
|
||||
for line in text.splitlines()
|
||||
if line.startswith("myboot_http_requests_total")
|
||||
and 'path="/items/{item_id}"' in line
|
||||
]
|
||||
assert len(template_lines) == 1
|
||||
assert 'method="GET"' in template_lines[0]
|
||||
assert 'status="200"' in template_lines[0]
|
||||
assert template_lines[0].rstrip().endswith("2.0")
|
||||
# 具体 id 不出现为独立标签（避免高基数）
|
||||
assert 'path="/items/1"' not in text
|
||||
assert 'path="/items/2"' not in text
|
||||
|
||||
# 404 归入 unmatched
|
||||
assert 'path="unmatched"' in text
|
||||
|
||||
# metrics 自身路径不被统计
|
||||
assert 'path="/metrics"' not in text
|
||||
|
||||
# 耗时直方图存在
|
||||
assert "myboot_http_request_duration_seconds" in text
|
||||
|
||||
def test_metrics_endpoint_not_wrapped_by_response_formatter(self, metrics_app):
|
||||
client = TestClient(metrics_app.get_fastapi_app())
|
||||
resp = client.get("/metrics")
|
||||
assert resp.status_code == 200
|
||||
# Prometheus 文本格式而非统一 JSON 包装
|
||||
assert "text/plain" in resp.headers.get("content-type", "")
|
||||
@@ -0,0 +1,631 @@
|
||||
"""
|
||||
多 worker 服务独立实例化（issue #11）单元测试
|
||||
|
||||
覆盖：
|
||||
1. @service/@client 的 scope 元数据透传（顶层键，装饰期校验）
|
||||
2. ServiceProvider 的 scope -> Provider 映射（request -> ContextLocalSingleton）
|
||||
3. Application.run() 多 worker 模式下延迟引导（父进程跳过 apply_auto_configuration）
|
||||
4. Application.bootstrap_worker() 环境变量重读 / 调度器门控 / fork 残留重置
|
||||
5. server._worker_serve 与 bootstrap_worker 的桥接
|
||||
6. lifespan 中 worker 启动/停止钩子的触发顺序
|
||||
7. auto_discover 幂等守卫与 @on_worker_start/@on_worker_stop 的 AST 发现
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import textwrap
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from dependency_injector import providers as di_providers
|
||||
|
||||
import myboot.core.application as application_module
|
||||
import myboot.core.server as server_module
|
||||
from myboot.core.application import Application
|
||||
from myboot.core.auto_configuration import AutoConfigurationManager
|
||||
from myboot.core.decorators import (
|
||||
service,
|
||||
client,
|
||||
on_worker_start,
|
||||
on_worker_stop,
|
||||
)
|
||||
from myboot.core.di import DependencyContainer
|
||||
from myboot.core.di.providers import ServiceProvider
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 公共 fixture
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_worker_env():
|
||||
"""每个测试前后清理 MYBOOT_* 环境变量
|
||||
|
||||
_worker_serve / 测试本身会写入 MYBOOT_WORKER_ID 等变量，
|
||||
若不清理会污染后续 Application 构造。
|
||||
"""
|
||||
saved = {k: v for k, v in os.environ.items() if k.startswith("MYBOOT_")}
|
||||
for key in saved:
|
||||
del os.environ[key]
|
||||
yield
|
||||
for key in [k for k in os.environ if k.startswith("MYBOOT_")]:
|
||||
del os.environ[key]
|
||||
os.environ.update(saved)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def set_config():
|
||||
"""临时修改全局配置，测试结束后恢复原值"""
|
||||
from myboot.core.config import get_settings
|
||||
|
||||
settings = get_settings()
|
||||
changed = {}
|
||||
|
||||
def _set(key, value):
|
||||
if key not in changed:
|
||||
changed[key] = settings.get(key)
|
||||
settings.set(key, value)
|
||||
|
||||
yield _set
|
||||
|
||||
for key, old in changed.items():
|
||||
settings.set(key, old)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. scope 元数据透传
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestScopeDecorators:
|
||||
def test_service_scope_defaults_to_singleton(self):
|
||||
@service()
|
||||
class AlphaService:
|
||||
pass
|
||||
|
||||
assert AlphaService.__myboot_service__["scope"] == "singleton"
|
||||
|
||||
def test_service_scope_stored_as_top_level_key(self):
|
||||
"""scope 必须是顶层键（而非落入 kwargs），注册时才能读到"""
|
||||
|
||||
@service(scope="request")
|
||||
class BetaService:
|
||||
pass
|
||||
|
||||
meta = BetaService.__myboot_service__
|
||||
assert meta["scope"] == "request"
|
||||
assert "scope" not in meta["kwargs"]
|
||||
|
||||
def test_service_invalid_scope_raises_at_decoration(self):
|
||||
with pytest.raises(ValueError, match="scope"):
|
||||
@service(scope="sigleton") # 故意拼错
|
||||
class TypoService:
|
||||
pass
|
||||
|
||||
def test_client_scope_defaults_to_singleton(self):
|
||||
@client()
|
||||
class AlphaClient:
|
||||
pass
|
||||
|
||||
assert AlphaClient.__myboot_client__["scope"] == "singleton"
|
||||
|
||||
def test_client_scope_stored_as_top_level_key(self):
|
||||
@client(scope="factory")
|
||||
class BetaClient:
|
||||
pass
|
||||
|
||||
meta = BetaClient.__myboot_client__
|
||||
assert meta["scope"] == "factory"
|
||||
assert "scope" not in meta["kwargs"]
|
||||
|
||||
def test_client_invalid_scope_raises_at_decoration(self):
|
||||
with pytest.raises(ValueError, match="scope"):
|
||||
@client(scope="prototype")
|
||||
class TypoClient:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Provider 映射
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestProviderScopes:
|
||||
class _Dummy:
|
||||
pass
|
||||
|
||||
def test_singleton_scope_creates_singleton_provider(self):
|
||||
sp = ServiceProvider(self._Dummy, "dummy", scope="singleton")
|
||||
assert isinstance(sp.create_provider(), di_providers.Singleton)
|
||||
|
||||
def test_request_scope_creates_context_local_singleton(self):
|
||||
sp = ServiceProvider(self._Dummy, "dummy", scope="request")
|
||||
assert isinstance(sp.create_provider(), di_providers.ContextLocalSingleton)
|
||||
|
||||
def test_factory_scope_creates_factory_provider(self):
|
||||
sp = ServiceProvider(self._Dummy, "dummy", scope="factory")
|
||||
provider = sp.create_provider()
|
||||
assert isinstance(provider, di_providers.Factory)
|
||||
assert not isinstance(provider, di_providers.Singleton)
|
||||
|
||||
def test_request_scope_instances_per_asyncio_task(self):
|
||||
"""request 作用域：跨 asyncio 任务不同实例，同一任务内同一实例"""
|
||||
|
||||
class ReqService:
|
||||
pass
|
||||
|
||||
container = DependencyContainer()
|
||||
container.register_service(ReqService, "req_service", scope="request")
|
||||
container.build_container()
|
||||
|
||||
async def resolve_twice():
|
||||
first = container.get_service("req_service")
|
||||
second = container.get_service("req_service")
|
||||
return first, second
|
||||
|
||||
async def main():
|
||||
(a1, a2), (b1, b2) = await asyncio.gather(
|
||||
resolve_twice(), resolve_twice()
|
||||
)
|
||||
return a1, a2, b1, b2
|
||||
|
||||
a1, a2, b1, b2 = asyncio.run(main())
|
||||
assert a1 is a2 # 同一任务（请求）内单例
|
||||
assert b1 is b2
|
||||
assert a1 is not b1 # 跨任务（请求）不同实例
|
||||
# request 作用域不进入单例缓存
|
||||
assert "req_service" not in container.service_instances
|
||||
|
||||
def test_register_service_accepts_decorator_metadata_with_scope(self):
|
||||
"""回归：装饰器元数据（含顶层 scope 键）作为 config 传入时
|
||||
不得与 register_service 的显式 scope 参数冲突"""
|
||||
|
||||
@service(scope="request")
|
||||
class MetaService:
|
||||
pass
|
||||
|
||||
meta = MetaService.__myboot_service__
|
||||
container = DependencyContainer()
|
||||
container.register_service(
|
||||
MetaService, meta["name"], scope=meta["scope"], config=meta
|
||||
)
|
||||
container.build_container()
|
||||
|
||||
provider = container.service_providers[meta["name"]]
|
||||
assert provider.scope == "request"
|
||||
|
||||
def test_factory_scope_not_cached_by_container(self):
|
||||
class FactService:
|
||||
pass
|
||||
|
||||
container = DependencyContainer()
|
||||
container.register_service(FactService, "fact_service", scope="factory")
|
||||
container.build_container()
|
||||
|
||||
first = container.get_service("fact_service")
|
||||
second = container.get_service("fact_service")
|
||||
assert first is not second
|
||||
assert "fact_service" not in container.service_instances
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. run() 延迟引导
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRunDefersBootstrap:
|
||||
def _run(self, monkeypatch, set_config, workers, app_path):
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
application_module, "auto_discover",
|
||||
lambda pkg: calls.append(("discover", pkg)),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
application_module, "apply_auto_configuration",
|
||||
lambda app: calls.append(("apply", app)),
|
||||
)
|
||||
|
||||
app = Application(name="run-defer-test")
|
||||
monkeypatch.setattr(
|
||||
app.server_manager, "start_server",
|
||||
lambda **kwargs: calls.append(("serve", kwargs)),
|
||||
)
|
||||
# conf/config.yaml 中 server.workers=1 会覆盖 run() 入参，显式设置
|
||||
set_config("server.workers", workers)
|
||||
app.run(workers=workers, app_path=app_path)
|
||||
return app, calls
|
||||
|
||||
def test_multi_worker_with_app_path_skips_apply(self, monkeypatch, set_config):
|
||||
app, calls = self._run(monkeypatch, set_config, workers=2, app_path="main:app")
|
||||
events = [name for name, _ in calls]
|
||||
assert "discover" in events # 父进程仍做 AST 发现（预热缓存）
|
||||
assert "apply" not in events # 实例化延迟到 worker 内
|
||||
assert "serve" in events
|
||||
serve_kwargs = dict(calls)["serve"]
|
||||
assert serve_kwargs["workers"] == 2
|
||||
assert serve_kwargs["app_path"] == "main:app"
|
||||
|
||||
def test_single_worker_applies_in_parent(self, monkeypatch, set_config):
|
||||
app, calls = self._run(monkeypatch, set_config, workers=1, app_path=None)
|
||||
events = [name for name, _ in calls]
|
||||
assert "discover" in events
|
||||
assert "apply" in events
|
||||
|
||||
def test_multi_worker_without_app_path_falls_back_to_parent_apply(
|
||||
self, monkeypatch, set_config
|
||||
):
|
||||
"""无 app_path 的回退路径（父进程自己 serve）保持原有引导行为"""
|
||||
app, calls = self._run(monkeypatch, set_config, workers=2, app_path=None)
|
||||
events = [name for name, _ in calls]
|
||||
assert "apply" in events
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. bootstrap_worker()
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestBootstrapWorker:
|
||||
def test_reads_worker_env_and_disables_scheduler_for_non_primary(self):
|
||||
app = Application(name="bw-env-test", auto_configuration=False)
|
||||
# 构造时无环境变量：默认自认 primary（这正是被修复的 bug 场景）
|
||||
assert app.is_primary_worker is True
|
||||
|
||||
os.environ["MYBOOT_WORKER_ID"] = "2"
|
||||
os.environ["MYBOOT_WORKER_COUNT"] = "3"
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "0"
|
||||
|
||||
old_scheduler = app.scheduler
|
||||
result = app.bootstrap_worker()
|
||||
|
||||
assert app.worker_id == 2
|
||||
assert app.worker_count == 3
|
||||
assert app.is_primary_worker is False
|
||||
# 调度器被重建且按 primary 门控禁用
|
||||
assert app.scheduler is not old_scheduler
|
||||
assert app.scheduler.is_enabled() is False
|
||||
assert result is app.get_fastapi_app()
|
||||
|
||||
def test_primary_worker_keeps_scheduler_enabled(self):
|
||||
app = Application(name="bw-primary-test", auto_configuration=False)
|
||||
os.environ["MYBOOT_WORKER_ID"] = "1"
|
||||
os.environ["MYBOOT_WORKER_COUNT"] = "3"
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "1"
|
||||
|
||||
app.bootstrap_worker()
|
||||
assert app.is_primary_worker is True
|
||||
assert app.scheduler.is_enabled() is True
|
||||
|
||||
def test_scheduler_on_all_workers_overrides_primary_gate(self, set_config):
|
||||
set_config("scheduler.on_all_workers", True)
|
||||
app = Application(name="bw-allworkers-test", auto_configuration=False)
|
||||
os.environ["MYBOOT_WORKER_ID"] = "2"
|
||||
os.environ["MYBOOT_WORKER_COUNT"] = "2"
|
||||
os.environ["MYBOOT_IS_PRIMARY_WORKER"] = "0"
|
||||
|
||||
app.bootstrap_worker()
|
||||
assert app.is_primary_worker is False
|
||||
assert app.scheduler.is_enabled() is True
|
||||
|
||||
def test_defensive_reset_on_fork_inherited_state(self):
|
||||
"""fork 子进程继承父进程已引导状态时：清空注册表并重建 FastAPI 应用"""
|
||||
app = Application(name="bw-reset-test", auto_configuration=False)
|
||||
|
||||
class InheritedService:
|
||||
pass
|
||||
|
||||
app.di_container = DependencyContainer()
|
||||
app.di_container.register_service(InheritedService, "inherited_service")
|
||||
app.services["inherited_service"] = InheritedService()
|
||||
app.clients["inherited_client"] = object()
|
||||
app.components["inherited_component"] = object()
|
||||
app._client_type_map = {InheritedService: "inherited_client"}
|
||||
app._component_type_map = {InheritedService: "inherited_component"}
|
||||
app.worker_start_hooks.append(lambda: None)
|
||||
app.worker_stop_hooks.append(lambda: None)
|
||||
old_fastapi_app = app._fastapi_app
|
||||
|
||||
result = app.bootstrap_worker()
|
||||
|
||||
assert app.services == {}
|
||||
assert app.clients == {}
|
||||
assert app.components == {}
|
||||
assert app._client_type_map == {}
|
||||
assert app._component_type_map == {}
|
||||
assert app.worker_start_hooks == []
|
||||
assert app.worker_stop_hooks == []
|
||||
assert not app.di_container.service_providers
|
||||
# FastAPI 应用被重建（丢弃绑定到 fork 前实例的路由）
|
||||
assert app._fastapi_app is not old_fastapi_app
|
||||
assert result is app._fastapi_app
|
||||
|
||||
def test_clean_state_keeps_pristine_fastapi_app(self):
|
||||
"""正常延迟引导路径：注册表为空时不触发重置，沿用原 FastAPI 实例"""
|
||||
app = Application(name="bw-clean-test", auto_configuration=False)
|
||||
old_fastapi_app = app._fastapi_app
|
||||
|
||||
result = app.bootstrap_worker()
|
||||
assert app._fastapi_app is old_fastapi_app
|
||||
assert result is old_fastapi_app
|
||||
|
||||
def test_runs_auto_configuration_inside_worker(self, monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
application_module, "auto_discover",
|
||||
lambda pkg: calls.append(("discover", pkg)),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
application_module, "apply_auto_configuration",
|
||||
lambda app: calls.append(("apply", app)),
|
||||
)
|
||||
app = Application(name="bw-autoconf-test")
|
||||
app.bootstrap_worker()
|
||||
|
||||
events = [name for name, _ in calls]
|
||||
assert "discover" in events
|
||||
assert "apply" in events
|
||||
|
||||
def test_auto_configuration_disabled_skips_bootstrap_apply(self, monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
application_module, "apply_auto_configuration",
|
||||
lambda app: calls.append("apply"),
|
||||
)
|
||||
app = Application(name="bw-noautoconf-test", auto_configuration=False)
|
||||
app.bootstrap_worker()
|
||||
assert calls == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. _worker_serve 桥接
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestWorkerServe:
|
||||
@pytest.fixture
|
||||
def serve_recorder(self, monkeypatch):
|
||||
"""mock hypercorn.asyncio.serve 与 setup_worker_logging"""
|
||||
import importlib
|
||||
|
||||
import hypercorn.asyncio
|
||||
|
||||
# 注意：myboot.core 包的 `from .logger import logger` 会用 loguru
|
||||
# 实例遮蔽包属性 myboot.core.logger，必须经 importlib 拿到真实模块
|
||||
logger_module = importlib.import_module("myboot.core.logger")
|
||||
|
||||
served = {}
|
||||
|
||||
async def fake_serve(app, config):
|
||||
served["app"] = app
|
||||
served["config"] = config
|
||||
|
||||
monkeypatch.setattr(hypercorn.asyncio, "serve", fake_serve)
|
||||
monkeypatch.setattr(
|
||||
logger_module, "setup_worker_logging", lambda *a, **k: None
|
||||
)
|
||||
return served
|
||||
|
||||
def test_myboot_app_served_via_bootstrap_worker(
|
||||
self, monkeypatch, serve_recorder
|
||||
):
|
||||
resolved = object()
|
||||
monkeypatch.setattr(
|
||||
server_module, "_resolve_app_from_path", lambda path: resolved
|
||||
)
|
||||
|
||||
bootstrapped_app = object()
|
||||
bootstrap_calls = []
|
||||
|
||||
class FakeMybootApp:
|
||||
def bootstrap_worker(self):
|
||||
bootstrap_calls.append(True)
|
||||
return bootstrapped_app
|
||||
|
||||
monkeypatch.setattr(application_module, "_current_app", FakeMybootApp())
|
||||
|
||||
server_module._worker_serve("main:app", {}, 2, 3)
|
||||
|
||||
# 桥接：serve 的是 bootstrap_worker() 的返回值，而非解析出的原对象
|
||||
assert bootstrap_calls == [True]
|
||||
assert serve_recorder["app"] is bootstrapped_app
|
||||
# 环境变量在引导前已设置
|
||||
assert os.environ["MYBOOT_WORKER_ID"] == "2"
|
||||
assert os.environ["MYBOOT_WORKER_COUNT"] == "3"
|
||||
assert os.environ["MYBOOT_IS_PRIMARY_WORKER"] == "0"
|
||||
|
||||
def test_primary_worker_env_flag(self, monkeypatch, serve_recorder):
|
||||
monkeypatch.setattr(
|
||||
server_module, "_resolve_app_from_path", lambda path: object()
|
||||
)
|
||||
monkeypatch.setattr(application_module, "_current_app", None)
|
||||
|
||||
server_module._worker_serve("main:app", {}, 1, 2)
|
||||
assert os.environ["MYBOOT_IS_PRIMARY_WORKER"] == "1"
|
||||
|
||||
def test_plain_asgi_app_path_untouched(self, monkeypatch, serve_recorder):
|
||||
"""非 myboot 应用（_current_app 为 None）：直接 serve 解析出的对象"""
|
||||
resolved = object()
|
||||
monkeypatch.setattr(
|
||||
server_module, "_resolve_app_from_path", lambda path: resolved
|
||||
)
|
||||
monkeypatch.setattr(application_module, "_current_app", None)
|
||||
|
||||
server_module._worker_serve("asgi_module:app", {}, 2, 2)
|
||||
assert serve_recorder["app"] is resolved
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. worker 生命周期钩子
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestWorkerHookDecorators:
|
||||
def test_bare_decorator_marks_function(self):
|
||||
@on_worker_start
|
||||
def hook():
|
||||
pass
|
||||
|
||||
assert hook.__myboot_worker_hook__ == {"event": "start", "order": 0}
|
||||
|
||||
def test_decorator_with_order(self):
|
||||
@on_worker_stop(order=5)
|
||||
def hook():
|
||||
pass
|
||||
|
||||
assert hook.__myboot_worker_hook__ == {"event": "stop", "order": 5}
|
||||
|
||||
def test_auto_register_worker_hooks_sorted_by_order(self):
|
||||
class FakeApp:
|
||||
def __init__(self):
|
||||
self.worker_start_hooks = []
|
||||
self.worker_stop_hooks = []
|
||||
|
||||
def add_worker_start_hook(self, hook):
|
||||
self.worker_start_hooks.append(hook)
|
||||
|
||||
def add_worker_stop_hook(self, hook):
|
||||
self.worker_stop_hooks.append(hook)
|
||||
|
||||
@on_worker_start(order=2)
|
||||
def start_late():
|
||||
pass
|
||||
|
||||
@on_worker_start(order=1)
|
||||
def start_early():
|
||||
pass
|
||||
|
||||
@on_worker_stop
|
||||
def stop_hook():
|
||||
pass
|
||||
|
||||
manager = AutoConfigurationManager(use_cache=False)
|
||||
manager.discovered_components["worker_hooks"] = [
|
||||
{"function": start_late, "module": "m", "type": "function_on_worker_start"},
|
||||
{"function": start_early, "module": "m", "type": "function_on_worker_start"},
|
||||
{"function": stop_hook, "module": "m", "type": "function_on_worker_stop"},
|
||||
]
|
||||
|
||||
fake_app = FakeApp()
|
||||
manager._auto_register_worker_hooks(fake_app)
|
||||
|
||||
assert fake_app.worker_start_hooks == [start_early, start_late]
|
||||
assert fake_app.worker_stop_hooks == [stop_hook]
|
||||
|
||||
|
||||
class TestWorkerHooksInLifespan:
|
||||
def test_hook_firing_order_in_lifespan(self):
|
||||
"""startup -> worker_start(按 order) -> scheduler.start ...
|
||||
scheduler.stop -> worker_stop -> shutdown"""
|
||||
events = []
|
||||
|
||||
class StubScheduler:
|
||||
def has_jobs(self):
|
||||
return True
|
||||
|
||||
def start(self):
|
||||
events.append("scheduler_start")
|
||||
|
||||
def is_running(self):
|
||||
return True
|
||||
|
||||
def stop(self):
|
||||
events.append("scheduler_stop")
|
||||
|
||||
app = Application(name="hooks-lifespan-test", auto_configuration=False)
|
||||
app.scheduler = StubScheduler()
|
||||
|
||||
def startup_hook():
|
||||
events.append("startup")
|
||||
|
||||
def worker_start_sync():
|
||||
events.append("worker_start_sync")
|
||||
|
||||
async def worker_start_async():
|
||||
events.append("worker_start_async")
|
||||
|
||||
async def worker_stop_async():
|
||||
events.append("worker_stop_async")
|
||||
|
||||
def shutdown_hook():
|
||||
events.append("shutdown")
|
||||
|
||||
app.add_startup_hook(startup_hook)
|
||||
app.add_worker_start_hook(worker_start_sync)
|
||||
app.add_worker_start_hook(worker_start_async)
|
||||
app.add_worker_stop_hook(worker_stop_async)
|
||||
app.add_shutdown_hook(shutdown_hook)
|
||||
|
||||
with TestClient(app.get_fastapi_app()):
|
||||
assert events == [
|
||||
"startup",
|
||||
"worker_start_sync",
|
||||
"worker_start_async",
|
||||
"scheduler_start",
|
||||
]
|
||||
|
||||
assert events == [
|
||||
"startup",
|
||||
"worker_start_sync",
|
||||
"worker_start_async",
|
||||
"scheduler_start",
|
||||
"scheduler_stop",
|
||||
"worker_stop_async",
|
||||
"shutdown",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. auto_discover 幂等 + worker 钩子 AST 发现
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAutoDiscover:
|
||||
def _make_package(self, tmp_path):
|
||||
pkg = tmp_path / "demo_pkg"
|
||||
pkg.mkdir()
|
||||
(pkg / "__init__.py").write_text("", encoding="utf-8")
|
||||
(pkg / "stuff.py").write_text(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
from myboot.core.decorators import (
|
||||
service, on_worker_start, on_worker_stop
|
||||
)
|
||||
|
||||
|
||||
@service()
|
||||
class DemoService:
|
||||
pass
|
||||
|
||||
|
||||
@on_worker_start
|
||||
def start_hook():
|
||||
pass
|
||||
|
||||
|
||||
@on_worker_stop(order=1)
|
||||
def stop_hook():
|
||||
pass
|
||||
"""
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return pkg
|
||||
|
||||
def test_auto_discover_is_idempotent(self, tmp_path):
|
||||
self._make_package(tmp_path)
|
||||
manager = AutoConfigurationManager(app_root=str(tmp_path), use_cache=False)
|
||||
|
||||
manager.auto_discover("demo_pkg")
|
||||
assert len(manager._component_metadata["services"]) == 1
|
||||
assert len(manager._component_metadata["worker_hooks"]) == 2
|
||||
|
||||
# 重复调用（fork 子进程场景）：不追加重复条目
|
||||
manager.auto_discover("demo_pkg")
|
||||
assert len(manager._component_metadata["services"]) == 1
|
||||
assert len(manager._component_metadata["worker_hooks"]) == 2
|
||||
|
||||
def test_worker_hooks_discovered_via_ast(self, tmp_path):
|
||||
self._make_package(tmp_path)
|
||||
manager = AutoConfigurationManager(app_root=str(tmp_path), use_cache=False)
|
||||
manager.auto_discover("demo_pkg")
|
||||
|
||||
hook_types = sorted(
|
||||
item["type"] for item in manager._component_metadata["worker_hooks"]
|
||||
)
|
||||
assert hook_types == [
|
||||
"function_on_worker_start",
|
||||
"function_on_worker_stop",
|
||||
]
|
||||
@@ -0,0 +1,336 @@
|
||||
"""
|
||||
调度器特征测试（characterization tests）
|
||||
|
||||
固化 myboot.core.scheduler.Scheduler 的「当前实际行为」，作为后续重构的兼容性闸门。
|
||||
这些测试只断言现状，不代表理想行为；行为有意变更时应同步更新本文件。
|
||||
|
||||
注意：
|
||||
- 全程不调用 scheduler.start()，只测注册层（BackgroundScheduler 未启动时
|
||||
任务进入 pending 队列，get_job/get_jobs/remove_job 均可操作 pending 任务）。
|
||||
- 通过直接构造 Dynaconf 对象传入 Scheduler，避免污染全局配置单例。
|
||||
|
||||
环境基线：APScheduler 3.11.x + pytz。
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from dynaconf import Dynaconf
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from apscheduler.triggers.date import DateTrigger
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
|
||||
from myboot.core.scheduler import Scheduler
|
||||
from myboot.exceptions import SchedulerError
|
||||
|
||||
|
||||
def make_scheduler(scheduler_config: dict = None, enabled=None) -> Scheduler:
|
||||
"""构造一个使用独立 Dynaconf 配置的 Scheduler（不依赖全局配置单例）"""
|
||||
if scheduler_config is None:
|
||||
config = Dynaconf()
|
||||
else:
|
||||
config = Dynaconf(scheduler=scheduler_config)
|
||||
return Scheduler(config=config, enabled=enabled)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scheduler():
|
||||
return make_scheduler()
|
||||
|
||||
|
||||
def sample_task():
|
||||
"""用于注册的示例任务函数"""
|
||||
pass
|
||||
|
||||
|
||||
# issue #14：默认 job_id 为 {prefix}_{模块名}.{限定名}
|
||||
SAMPLE_TASK_QUALNAME = f"{sample_task.__module__}.{sample_task.__qualname__}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. 构造与配置读取
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSchedulerConstruction:
|
||||
def test_default_enabled_is_true(self):
|
||||
s = make_scheduler()
|
||||
assert s.is_enabled() is True
|
||||
|
||||
def test_enabled_read_from_config(self):
|
||||
s = make_scheduler({"enabled": False})
|
||||
assert s.is_enabled() is False
|
||||
|
||||
def test_enabled_param_overrides_config(self):
|
||||
# application.py 即以 Scheduler(config=..., enabled=...) 方式覆盖
|
||||
s = make_scheduler({"enabled": False}, enabled=True)
|
||||
assert s.is_enabled() is True
|
||||
|
||||
def test_default_timezone_is_utc(self):
|
||||
s = make_scheduler()
|
||||
assert str(s._timezone) == "UTC"
|
||||
|
||||
def test_not_running_before_start(self, scheduler):
|
||||
assert scheduler.is_running() is False
|
||||
|
||||
def test_get_config_shape(self):
|
||||
s = make_scheduler({"timezone": "Asia/Shanghai", "enabled": False})
|
||||
cfg = s.get_config()
|
||||
assert cfg == {
|
||||
"enabled": False,
|
||||
"timezone": "Asia/Shanghai",
|
||||
"running": False,
|
||||
"job_count": 0,
|
||||
"scheduled_job_count": 0,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. 时区解析 _parse_timezone
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestParseTimezone:
|
||||
def test_valid_timezone_string(self, scheduler):
|
||||
tz = scheduler._parse_timezone("Asia/Shanghai")
|
||||
assert str(tz) == "Asia/Shanghai"
|
||||
|
||||
def test_invalid_timezone_falls_back_to_none(self, scheduler):
|
||||
# 当前行为：解析失败仅记 warning，返回 None（即使用系统时区）
|
||||
assert scheduler._parse_timezone("Not/AZone") is None
|
||||
|
||||
def test_invalid_timezone_in_config_reports_system(self):
|
||||
s = make_scheduler({"timezone": "Not/AZone"})
|
||||
assert s._timezone is None
|
||||
assert s.get_config()["timezone"] == "system"
|
||||
|
||||
def test_valid_timezone_in_config_applied(self):
|
||||
s = make_scheduler({"timezone": "Asia/Shanghai"})
|
||||
assert str(s._timezone) == "Asia/Shanghai"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Cron 表达式解析 _parse_cron
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestParseCron:
|
||||
def test_standard_5_field(self, scheduler):
|
||||
trigger = scheduler._parse_cron("0 2 * * *")
|
||||
assert isinstance(trigger, CronTrigger)
|
||||
text = str(trigger)
|
||||
assert "hour='2'" in text
|
||||
assert "minute='0'" in text
|
||||
# 5 位格式不含秒字段
|
||||
assert "second" not in text
|
||||
|
||||
def test_5_field_with_step(self, scheduler):
|
||||
trigger = scheduler._parse_cron("*/15 * * * *")
|
||||
assert isinstance(trigger, CronTrigger)
|
||||
assert "minute='*/15'" in str(trigger)
|
||||
|
||||
def test_6_field_with_seconds(self, scheduler):
|
||||
# 6 位格式：秒 分 时 日 月 周（兼容旧格式，走手动解析分支）
|
||||
trigger = scheduler._parse_cron("30 0 2 * * *")
|
||||
assert isinstance(trigger, CronTrigger)
|
||||
text = str(trigger)
|
||||
assert "second='30'" in text
|
||||
assert "minute='0'" in text
|
||||
assert "hour='2'" in text
|
||||
|
||||
def test_trigger_uses_scheduler_timezone(self):
|
||||
s = make_scheduler({"timezone": "Asia/Shanghai"})
|
||||
trigger = s._parse_cron("0 2 * * *")
|
||||
assert str(trigger.timezone) == "Asia/Shanghai"
|
||||
|
||||
def test_too_few_fields_raises_value_error(self, scheduler):
|
||||
with pytest.raises(ValueError):
|
||||
scheduler._parse_cron("* * *")
|
||||
|
||||
def test_too_many_fields_raises_value_error(self, scheduler):
|
||||
with pytest.raises(ValueError):
|
||||
scheduler._parse_cron("* * * * * * *")
|
||||
|
||||
def test_out_of_range_field_value_raises_value_error(self, scheduler):
|
||||
# from_crontab 与手动构造均拒绝越界值
|
||||
with pytest.raises(ValueError):
|
||||
scheduler._parse_cron("99 * * * *")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. add_cron_job / add_interval_job / add_date_job
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAddCronJob:
|
||||
def test_default_job_id_format(self, scheduler):
|
||||
job_id = scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
# issue #14：0.2.0 起默认 job_id 为 cron_{模块名}.{限定名}（含类名）
|
||||
assert job_id == f"cron_{SAMPLE_TASK_QUALNAME}"
|
||||
|
||||
def test_explicit_job_id_used_verbatim(self, scheduler):
|
||||
job_id = scheduler.add_cron_job(sample_task, "0 2 * * *", job_id="my_job")
|
||||
assert job_id == "my_job"
|
||||
assert scheduler.get_job("my_job") is not None
|
||||
# 默认 ID 没有被注册
|
||||
assert scheduler.get_job(f"cron_{SAMPLE_TASK_QUALNAME}") is None
|
||||
|
||||
def test_registered_job_retrievable(self, scheduler):
|
||||
job_id = scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
job = scheduler.get_job(job_id)
|
||||
assert job is not None
|
||||
assert job.id == job_id
|
||||
assert isinstance(job.trigger, CronTrigger)
|
||||
|
||||
def test_invalid_cron_raises_value_error(self, scheduler):
|
||||
with pytest.raises(ValueError):
|
||||
scheduler.add_cron_job(sample_task, "not a cron")
|
||||
# 失败的注册不会留下任务
|
||||
assert scheduler.list_jobs() == []
|
||||
|
||||
|
||||
class TestAddIntervalJob:
|
||||
def test_default_job_id_format(self, scheduler):
|
||||
job_id = scheduler.add_interval_job(sample_task, 60)
|
||||
# issue #14：0.2.0 起默认 job_id 为 interval_{模块名}.{限定名}
|
||||
assert job_id == f"interval_{SAMPLE_TASK_QUALNAME}"
|
||||
|
||||
def test_explicit_job_id_used_verbatim(self, scheduler):
|
||||
job_id = scheduler.add_interval_job(sample_task, 60, job_id="tick")
|
||||
assert job_id == "tick"
|
||||
|
||||
def test_interval_seconds_applied(self, scheduler):
|
||||
job_id = scheduler.add_interval_job(sample_task, 60)
|
||||
job = scheduler.get_job(job_id)
|
||||
assert isinstance(job.trigger, IntervalTrigger)
|
||||
assert job.trigger.interval.total_seconds() == 60.0
|
||||
|
||||
|
||||
class TestAddDateJob:
|
||||
def test_default_job_id_format(self, scheduler):
|
||||
job_id = scheduler.add_date_job(sample_task, "2099-12-31 23:59:59")
|
||||
# issue #14：0.2.0 起默认 job_id 为 date_{模块名}.{限定名}
|
||||
assert job_id == f"date_{SAMPLE_TASK_QUALNAME}"
|
||||
|
||||
def test_explicit_job_id_used_verbatim(self, scheduler):
|
||||
job_id = scheduler.add_date_job(
|
||||
sample_task, "2099-12-31 23:59:59", job_id="once"
|
||||
)
|
||||
assert job_id == "once"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"run_date",
|
||||
["2099-12-31 23:59:59", "2099-12-31 23:59", "2099-12-31"],
|
||||
)
|
||||
def test_supported_date_formats(self, scheduler, run_date):
|
||||
job_id = scheduler.add_date_job(sample_task, run_date)
|
||||
job = scheduler.get_job(job_id)
|
||||
assert isinstance(job.trigger, DateTrigger)
|
||||
|
||||
def test_invalid_date_raises_value_error(self, scheduler):
|
||||
with pytest.raises(ValueError):
|
||||
scheduler.add_date_job(sample_task, "2099-13-01")
|
||||
with pytest.raises(ValueError):
|
||||
scheduler.add_date_job(sample_task, "31/12/2099")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. 同名函数重复注册（默认 ID 冲突）的当前行为
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDuplicateRegistration:
|
||||
def test_duplicate_auto_id_disambiguated_with_uid_suffix(self, scheduler):
|
||||
"""0.2.0（issue #14 配套）：注册时显式查重。同一函数注册两次时，
|
||||
第二个自动生成的 ID 追加 8 位 uid 后缀消歧（并打 warning），
|
||||
不再出现重复 ID 进入 pending 队列、延迟到 start() 才爆发的问题。
|
||||
"""
|
||||
id1 = scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
id2 = scheduler.add_cron_job(sample_task, "0 3 * * *")
|
||||
assert id1 == f"cron_{SAMPLE_TASK_QUALNAME}"
|
||||
assert id2.startswith(f"cron_{SAMPLE_TASK_QUALNAME}_")
|
||||
assert len(id2) == len(f"cron_{SAMPLE_TASK_QUALNAME}_") + 8
|
||||
assert sorted(scheduler.list_jobs()) == sorted([id1, id2])
|
||||
|
||||
def test_duplicate_explicit_id_raises_scheduler_error(self, scheduler):
|
||||
"""0.2.0（issue #14 配套）：显式传入的 job_id 已存在时抛 SchedulerError"""
|
||||
scheduler.add_cron_job(sample_task, "0 2 * * *", job_id="my_job")
|
||||
with pytest.raises(SchedulerError, match="my_job"):
|
||||
scheduler.add_cron_job(sample_task, "0 3 * * *", job_id="my_job")
|
||||
# 失败的注册不留残留
|
||||
assert scheduler.list_jobs() == ["my_job"]
|
||||
|
||||
def test_same_method_name_in_different_classes_no_conflict(self, scheduler):
|
||||
"""issue #14 核心场景：两个类中的同名方法注册互不冲突"""
|
||||
|
||||
class AlphaJobs:
|
||||
def job(self):
|
||||
pass
|
||||
|
||||
class BetaJobs:
|
||||
def job(self):
|
||||
pass
|
||||
|
||||
id1 = scheduler.add_cron_job(AlphaJobs().job, "0 2 * * *")
|
||||
id2 = scheduler.add_cron_job(BetaJobs().job, "0 3 * * *")
|
||||
assert id1 != id2
|
||||
assert "AlphaJobs.job" in id1
|
||||
assert "BetaJobs.job" in id2
|
||||
|
||||
def test_remove_job_after_disambiguation(self, scheduler):
|
||||
"""消歧后两个任务可分别独立移除"""
|
||||
id1 = scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
id2 = scheduler.add_cron_job(sample_task, "0 3 * * *")
|
||||
assert scheduler.remove_job(id1) is True
|
||||
assert scheduler.list_jobs() == [id2]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. remove_job / get_job / list_jobs / has_jobs / get_job_info
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestJobManagement:
|
||||
def test_remove_existing_job_returns_true(self, scheduler):
|
||||
job_id = scheduler.add_interval_job(sample_task, 60)
|
||||
assert scheduler.remove_job(job_id) is True
|
||||
assert scheduler.get_job(job_id) is None
|
||||
assert scheduler.list_jobs() == []
|
||||
|
||||
def test_remove_missing_job_returns_false(self, scheduler):
|
||||
assert scheduler.remove_job("does_not_exist") is False
|
||||
|
||||
def test_get_missing_job_returns_none(self, scheduler):
|
||||
assert scheduler.get_job("does_not_exist") is None
|
||||
|
||||
def test_list_jobs_and_has_jobs(self, scheduler):
|
||||
assert scheduler.has_jobs() is False
|
||||
scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
scheduler.add_interval_job(sample_task, 60)
|
||||
assert scheduler.has_jobs() is True
|
||||
assert sorted(scheduler.list_jobs()) == sorted([
|
||||
f"cron_{SAMPLE_TASK_QUALNAME}",
|
||||
f"interval_{SAMPLE_TASK_QUALNAME}",
|
||||
])
|
||||
|
||||
def test_get_job_info_on_pending_job_returns_dict(self, scheduler):
|
||||
"""0.2.0 修复：调度器未启动时 pending Job 没有 next_run_time 属性，
|
||||
get_job_info 现在返回信息字典（next_run_time 为 None）而不是抛
|
||||
AttributeError。
|
||||
"""
|
||||
job_id = scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
info = scheduler.get_job_info(job_id)
|
||||
assert info is not None
|
||||
assert info["job_id"] == job_id
|
||||
assert info["func_name"] == "sample_task"
|
||||
assert info["next_run_time"] is None
|
||||
assert info["type"] == "cron"
|
||||
|
||||
def test_list_all_jobs_on_pending_jobs(self, scheduler):
|
||||
"""list_all_jobs 同样不再因 pending 任务崩溃"""
|
||||
scheduler.add_cron_job(sample_task, "0 2 * * *")
|
||||
scheduler.add_interval_job(sample_task, 60)
|
||||
infos = scheduler.list_all_jobs()
|
||||
assert len(infos) == 2
|
||||
assert all(i is not None for i in infos)
|
||||
|
||||
def test_get_job_info_missing_job_returns_none(self, scheduler):
|
||||
assert scheduler.get_job_info("does_not_exist") is None
|
||||
@@ -0,0 +1,698 @@
|
||||
"""
|
||||
Web 层特征测试（characterization tests）
|
||||
|
||||
固化 myboot web 层的「当前实际行为」，作为后续重构的兼容性闸门。
|
||||
这些测试不评判行为是否正确——可疑之处照实断言并以注释标记，
|
||||
重构时若行为有意变更，应同步更新对应测试。
|
||||
|
||||
覆盖范围：
|
||||
- myboot/web/decorators.py 路由装饰器元数据
|
||||
- myboot/core/decorators.py 路由装饰器与 @rest_controller（auto_configuration 实际消费的那套）
|
||||
- myboot/core/auto_configuration.py 的 AST 扫描 route_type 前缀（issue #8 回归）与路由注册分支
|
||||
- myboot/web/models.py BaseResponse 系列模型
|
||||
- myboot/web/response.py ResponseWrapper / ApiResponse
|
||||
- myboot/web/exceptions.py HTTP 异常体系
|
||||
- myboot/exceptions.py 与 myboot/web/exceptions.py 的同名异常类二义性
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import textwrap
|
||||
from datetime import datetime
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from myboot.core.auto_configuration import (
|
||||
AutoConfigurationError,
|
||||
AutoConfigurationManager,
|
||||
)
|
||||
from myboot.core.decorators import delete as core_delete
|
||||
from myboot.core.decorators import get as core_get
|
||||
from myboot.core.decorators import patch as core_patch
|
||||
from myboot.core.decorators import post as core_post
|
||||
from myboot.core.decorators import put as core_put
|
||||
from myboot.core.decorators import rest_controller
|
||||
from myboot.core.decorators import route as core_route
|
||||
from myboot.web.decorators import (
|
||||
async_route,
|
||||
delete,
|
||||
get,
|
||||
patch,
|
||||
post,
|
||||
put,
|
||||
route,
|
||||
)
|
||||
from myboot.web.exceptions import (
|
||||
HTTP_STATUS_EXCEPTIONS,
|
||||
BadRequestError,
|
||||
ConflictError,
|
||||
ForbiddenError,
|
||||
HTTPException,
|
||||
InternalServerError,
|
||||
MethodNotAllowedError,
|
||||
NotFoundError,
|
||||
ServiceUnavailableError,
|
||||
TooManyRequestsError,
|
||||
UnauthorizedError,
|
||||
UnprocessableEntityError,
|
||||
create_http_exception,
|
||||
)
|
||||
from myboot.web.models import (
|
||||
BaseResponse,
|
||||
CreatedResponse,
|
||||
DeletedResponse,
|
||||
ErrorResponse,
|
||||
SuccessResponse,
|
||||
UpdatedResponse,
|
||||
error_response,
|
||||
success_response,
|
||||
)
|
||||
from myboot.web.response import ApiResponse, ResponseWrapper
|
||||
|
||||
|
||||
def _make_manager() -> AutoConfigurationManager:
|
||||
"""每个测试创建独立的 manager，避免共享 discovered_components 状态。
|
||||
|
||||
AutoConfigurationManager 的组件元数据是实例级状态（非模块级全局
|
||||
registry），use_cache=False 确保不读写磁盘缓存。
|
||||
"""
|
||||
return AutoConfigurationManager(app_root=".", use_cache=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. myboot/web/decorators.py 路由装饰器元数据
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWebRouteDecorators:
|
||||
"""myboot.web.decorators 装饰器把元数据写入 func._route_metadata"""
|
||||
|
||||
def test_get_decorator_metadata(self):
|
||||
@get("/users", tags=["user"])
|
||||
def handler():
|
||||
"""获取用户列表"""
|
||||
return []
|
||||
|
||||
meta = handler._route_metadata
|
||||
assert meta["path"] == "/users"
|
||||
assert meta["methods"] == ["GET"]
|
||||
assert meta["status_code"] == 200
|
||||
assert meta["tags"] == ["user"]
|
||||
# summary/description 未显式提供时回退到 __doc__
|
||||
assert meta["summary"] == "获取用户列表"
|
||||
assert meta["description"] == "获取用户列表"
|
||||
assert meta["response_model"] is None
|
||||
|
||||
def test_default_status_codes_per_method(self):
|
||||
# 当前行为：post 默认 201，delete 默认 204，get/put/patch 默认 200
|
||||
cases = [
|
||||
(get, ["GET"], 200),
|
||||
(post, ["POST"], 201),
|
||||
(put, ["PUT"], 200),
|
||||
(delete, ["DELETE"], 204),
|
||||
(patch, ["PATCH"], 200),
|
||||
]
|
||||
for decorator, methods, status_code in cases:
|
||||
@decorator("/x")
|
||||
def handler():
|
||||
return None
|
||||
|
||||
meta = handler._route_metadata
|
||||
assert meta["methods"] == methods
|
||||
assert meta["status_code"] == status_code
|
||||
|
||||
def test_route_decorator_defaults_and_extra_kwargs(self):
|
||||
@route("/multi", methods=["GET", "POST"], deprecated=True)
|
||||
def handler():
|
||||
return None
|
||||
|
||||
meta = handler._route_metadata
|
||||
assert meta["methods"] == ["GET", "POST"]
|
||||
# 额外 kwargs 被直接展开合并进元数据字典
|
||||
assert meta["deprecated"] is True
|
||||
|
||||
@route("/default")
|
||||
def handler2():
|
||||
return None
|
||||
|
||||
# methods 缺省为 ["GET"]
|
||||
assert handler2._route_metadata["methods"] == ["GET"]
|
||||
|
||||
def test_summary_falls_back_to_empty_string_without_docstring(self):
|
||||
@get("/nodoc")
|
||||
def handler():
|
||||
return None
|
||||
|
||||
meta = handler._route_metadata
|
||||
assert meta["summary"] == ""
|
||||
assert meta["description"] == ""
|
||||
assert meta["tags"] == []
|
||||
|
||||
def test_decorator_returns_wrapper_that_still_works(self):
|
||||
# 当前行为：装饰器返回 @wraps 包装函数（非原函数），
|
||||
# functools.wraps 复制 __dict__，因此包装函数上也带 _route_metadata
|
||||
@get("/calc")
|
||||
def add(a, b):
|
||||
return a + b
|
||||
|
||||
assert add(1, 2) == 3
|
||||
assert add.__name__ == "add"
|
||||
assert add._route_metadata["path"] == "/calc"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_route_metadata_and_await(self):
|
||||
@async_route("/async", methods=["POST"])
|
||||
async def handler(value):
|
||||
return value * 2
|
||||
|
||||
meta = handler._route_metadata
|
||||
assert meta["path"] == "/async"
|
||||
assert meta["methods"] == ["POST"]
|
||||
# async_route 额外写入 'async': True 标记
|
||||
assert meta["async"] is True
|
||||
assert await handler(21) == 42
|
||||
assert asyncio.iscoroutinefunction(handler)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1b. myboot/core/decorators.py 路由装饰器（auto_configuration 消费的元数据）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCoreRouteDecorators:
|
||||
"""myboot.core.decorators 装饰器把元数据写入 func.__myboot_route__"""
|
||||
|
||||
def test_core_get_sets_myboot_route_and_returns_original(self):
|
||||
def raw(a):
|
||||
return a
|
||||
|
||||
decorated = core_get("/ping", tags=["t"])(raw)
|
||||
# 当前行为：core 装饰器不包装，直接返回原函数对象
|
||||
assert decorated is raw
|
||||
assert raw.__myboot_route__ == {
|
||||
"path": "/ping",
|
||||
"methods": ["GET"],
|
||||
"kwargs": {"tags": ["t"]},
|
||||
}
|
||||
|
||||
def test_core_http_method_decorators(self):
|
||||
cases = [
|
||||
(core_get, ["GET"]),
|
||||
(core_post, ["POST"]),
|
||||
(core_put, ["PUT"]),
|
||||
(core_delete, ["DELETE"]),
|
||||
(core_patch, ["PATCH"]),
|
||||
]
|
||||
for decorator, methods in cases:
|
||||
@decorator("/x")
|
||||
def handler():
|
||||
return None
|
||||
|
||||
assert handler.__myboot_route__["methods"] == methods
|
||||
|
||||
def test_core_route_default_methods(self):
|
||||
@core_route("/r")
|
||||
def handler():
|
||||
return None
|
||||
|
||||
assert handler.__myboot_route__["methods"] == ["GET"]
|
||||
assert handler.__myboot_route__["kwargs"] == {}
|
||||
|
||||
def test_rest_controller_sets_class_attribute(self):
|
||||
@rest_controller("/api/items/", version=1)
|
||||
class ItemController:
|
||||
pass
|
||||
|
||||
config = ItemController.__myboot_rest_controller__
|
||||
# base_path 尾部 / 被 rstrip 去掉
|
||||
assert config["base_path"] == "/api/items"
|
||||
assert config["kwargs"] == {"version": 1}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. route_type 前缀格式（issue #8 回归）与路由注册分支
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRouteTypeDiscovery:
|
||||
"""AST 扫描产生的 type 字段格式：f'function_{dec}' / f'class_{dec}'"""
|
||||
|
||||
def test_ast_scan_route_type_prefixes(self, tmp_path):
|
||||
# issue #8 回归：发现阶段的 type 必须以 function_ / class_ 开头
|
||||
source = textwrap.dedent(
|
||||
"""
|
||||
from myboot.core.decorators import rest_controller, get, route
|
||||
|
||||
@rest_controller('/api/x')
|
||||
class XController:
|
||||
pass
|
||||
|
||||
@get('/ping')
|
||||
def ping():
|
||||
return 'pong'
|
||||
|
||||
@route('/r')
|
||||
def plain_route():
|
||||
return 'r'
|
||||
"""
|
||||
)
|
||||
mod_file = tmp_path / "scanned_mod.py"
|
||||
mod_file.write_text(source, encoding="utf-8")
|
||||
|
||||
manager = _make_manager()
|
||||
manager._scan_file_ast(mod_file, "scanned_mod")
|
||||
|
||||
controllers = manager._component_metadata["rest_controllers"]
|
||||
assert controllers == [
|
||||
{
|
||||
"module": "scanned_mod",
|
||||
"class_name": "XController",
|
||||
"type": "class_rest_controller",
|
||||
}
|
||||
]
|
||||
|
||||
routes = manager._component_metadata["routes"]
|
||||
types = {item["func_name"]: item["type"] for item in routes}
|
||||
assert types == {
|
||||
"ping": "function_get",
|
||||
"plain_route": "function_route",
|
||||
}
|
||||
for item in routes:
|
||||
assert item["type"].startswith("function_") or item["type"].startswith(
|
||||
"class_"
|
||||
)
|
||||
|
||||
def test_function_route_type_is_registered(self):
|
||||
@core_route("/fr", methods=["GET"])
|
||||
def fr():
|
||||
return "x"
|
||||
|
||||
calls = []
|
||||
|
||||
class FakeApp:
|
||||
def add_route(self, **kwargs):
|
||||
calls.append(kwargs)
|
||||
|
||||
manager = _make_manager()
|
||||
manager.discovered_components["routes"] = [
|
||||
{"type": "function_route", "function": fr, "module": "m"}
|
||||
]
|
||||
manager._auto_register_routes(FakeApp())
|
||||
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["path"] == "/fr"
|
||||
assert calls[0]["handler"] is fr
|
||||
assert calls[0]["methods"] == ["GET"]
|
||||
|
||||
def test_function_get_route_type_registers(self):
|
||||
# 0.2.0 修复（issue #8 残留）：_auto_register_routes 现以前缀匹配
|
||||
# route_type，AST 扫描产生的 'function_get'/'function_post' 等
|
||||
# 模块级函数路由不再被静默跳过。
|
||||
@core_get("/fn")
|
||||
def fn():
|
||||
return "x"
|
||||
|
||||
calls = []
|
||||
|
||||
class FakeApp:
|
||||
def add_route(self, **kwargs):
|
||||
calls.append(kwargs)
|
||||
|
||||
manager = _make_manager()
|
||||
manager.discovered_components["routes"] = [
|
||||
{"type": "function_get", "function": fn, "module": "m"}
|
||||
]
|
||||
manager._auto_register_routes(FakeApp())
|
||||
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["path"] == "/fn"
|
||||
assert calls[0]["methods"] == ["GET"]
|
||||
|
||||
def test_class_route_type_registers_methods_without_error(self):
|
||||
# 0.2.0 修复：class 类型条目没有 'function' 键，旧版收尾 debug 日志
|
||||
# 访问 route_info['function'] 抛 KeyError 导致注册必然失败；
|
||||
# 现在方法路由注册成功且不再抛 AutoConfigurationError。
|
||||
@core_route("/cls")
|
||||
class ClassController:
|
||||
@core_get("/m")
|
||||
def m(self):
|
||||
return 1
|
||||
|
||||
calls = []
|
||||
|
||||
class FakeApp:
|
||||
def add_route(self, **kwargs):
|
||||
calls.append(kwargs)
|
||||
|
||||
manager = _make_manager()
|
||||
manager.discovered_components["routes"] = [
|
||||
{"type": "class_route", "class": ClassController, "module": "m"}
|
||||
]
|
||||
manager._auto_register_routes(FakeApp())
|
||||
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["path"] == "/m"
|
||||
|
||||
|
||||
class TestRestControllerEndToEnd:
|
||||
"""用最小 FastAPI app + TestClient 验证 @rest_controller 注册后端点可访问
|
||||
|
||||
myboot 的注册逻辑只依赖 app.add_route 和 app.di_container，
|
||||
用桩对象把 add_route 转接到 FastAPI 即可隔离全局 Application 状态。
|
||||
"""
|
||||
|
||||
def _build_client(self):
|
||||
@rest_controller("/api/items")
|
||||
class ItemController:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@core_get("/list")
|
||||
def list_items(self):
|
||||
return {"items": [1, 2]}
|
||||
|
||||
@core_get("//absolute")
|
||||
def absolute(self):
|
||||
return {"abs": True}
|
||||
|
||||
@core_get("relative")
|
||||
def rel(self):
|
||||
return {"rel": True}
|
||||
|
||||
fastapi_app = FastAPI()
|
||||
|
||||
class FakeApp:
|
||||
def __init__(self):
|
||||
self.di_container = SimpleNamespace(has_service=lambda name: False)
|
||||
self.components = {}
|
||||
self.clients = {}
|
||||
|
||||
def add_route(self, path, handler, methods, **kwargs):
|
||||
fastapi_app.add_api_route(path, handler, methods=methods)
|
||||
|
||||
manager = _make_manager()
|
||||
manager.discovered_components["rest_controllers"] = [
|
||||
{
|
||||
"class": ItemController,
|
||||
"module": "tests.unit.test_web",
|
||||
"type": "class_rest_controller",
|
||||
}
|
||||
]
|
||||
manager._auto_register_rest_controllers(FakeApp())
|
||||
return TestClient(fastapi_app)
|
||||
|
||||
def test_method_path_appended_to_base_path(self):
|
||||
client = self._build_client()
|
||||
resp = client.get("/api/items/list")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"items": [1, 2]}
|
||||
|
||||
def test_double_slash_prefix_means_absolute_path(self):
|
||||
# 路径合并规则：方法路径以 // 开头时作为绝对路径（去掉一个 /），
|
||||
# 不拼接 base_path
|
||||
client = self._build_client()
|
||||
resp = client.get("/absolute")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"abs": True}
|
||||
# base_path 下不存在该路由
|
||||
assert client.get("/api/items/absolute").status_code == 404
|
||||
|
||||
def test_relative_path_appended_with_slash(self):
|
||||
client = self._build_client()
|
||||
resp = client.get("/api/items/relative")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"rel": True}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. BaseResponse 响应结构
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestBaseResponse:
|
||||
def test_defaults(self):
|
||||
r = BaseResponse()
|
||||
assert r.success is True
|
||||
assert r.message == "操作成功"
|
||||
assert r.data is None
|
||||
assert isinstance(r.timestamp, datetime)
|
||||
|
||||
def test_model_dump_field_names(self):
|
||||
dumped = BaseResponse().model_dump()
|
||||
# 字段名固定为这 4 个
|
||||
assert sorted(dumped.keys()) == ["data", "message", "success", "timestamp"]
|
||||
assert dumped["success"] is True
|
||||
assert dumped["message"] == "操作成功"
|
||||
# 当前行为：data=None 也会出现在 model_dump 结果中
|
||||
assert dumped["data"] is None
|
||||
assert isinstance(dumped["timestamp"], datetime)
|
||||
|
||||
def test_error_response_defaults(self):
|
||||
r = ErrorResponse()
|
||||
assert r.success is False
|
||||
assert r.message == "操作失败"
|
||||
assert r.error_code is None
|
||||
assert r.details is None
|
||||
dumped = r.model_dump()
|
||||
assert sorted(dumped.keys()) == [
|
||||
"data",
|
||||
"details",
|
||||
"error_code",
|
||||
"message",
|
||||
"success",
|
||||
"timestamp",
|
||||
]
|
||||
|
||||
def test_crud_response_defaults(self):
|
||||
assert SuccessResponse().message == "操作成功"
|
||||
|
||||
created = CreatedResponse()
|
||||
assert created.message == "创建成功"
|
||||
# 当前行为：status_code 是模型字段，会出现在序列化结果里
|
||||
assert created.status_code == 201
|
||||
assert created.model_dump()["status_code"] == 201
|
||||
|
||||
assert UpdatedResponse().status_code == 200
|
||||
assert UpdatedResponse().message == "更新成功"
|
||||
|
||||
deleted = DeletedResponse()
|
||||
assert deleted.status_code == 204
|
||||
assert deleted.message == "删除成功"
|
||||
|
||||
def test_helper_functions(self):
|
||||
ok = success_response(data={"id": 1}, message="done")
|
||||
assert isinstance(ok, SuccessResponse)
|
||||
assert ok.data == {"id": 1}
|
||||
assert ok.message == "done"
|
||||
|
||||
err = error_response(message="bad", error_code="E1", details={"k": "v"})
|
||||
assert isinstance(err, ErrorResponse)
|
||||
assert err.success is False
|
||||
assert err.error_code == "E1"
|
||||
assert err.details == {"k": "v"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. ResponseWrapper 包装行为
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestResponseWrapper:
|
||||
def test_success_minimal_omits_none_fields(self):
|
||||
# message/data 为 None 时根本不出现在返回字典中
|
||||
assert ResponseWrapper.success() == {"success": True, "code": 200}
|
||||
|
||||
def test_success_with_data_and_message(self):
|
||||
result = ResponseWrapper.success(data=[1], message="ok", code=200)
|
||||
assert result == {"success": True, "code": 200, "message": "ok", "data": [1]}
|
||||
|
||||
def test_error_defaults_to_500(self):
|
||||
assert ResponseWrapper.error() == {"success": False, "code": 500}
|
||||
result = ResponseWrapper.error(message="boom", code=400, data={"f": 1})
|
||||
assert result == {
|
||||
"success": False,
|
||||
"code": 400,
|
||||
"message": "boom",
|
||||
"data": {"f": 1},
|
||||
}
|
||||
|
||||
def test_created_updated_deleted_no_content_codes(self):
|
||||
assert ResponseWrapper.created(data={"id": 1}) == {
|
||||
"success": True,
|
||||
"code": 201,
|
||||
"data": {"id": 1},
|
||||
}
|
||||
assert ResponseWrapper.updated() == {"success": True, "code": 200}
|
||||
# 可疑现状：deleted() 的 code 是 200（文档注释也写明 200），
|
||||
# 而 DeletedResponse 模型的 status_code 默认 204，两处不一致
|
||||
assert ResponseWrapper.deleted() == {"success": True, "code": 200}
|
||||
# no_content() 标记 204 但仍返回非空响应体字典
|
||||
assert ResponseWrapper.no_content() == {"success": True, "code": 204}
|
||||
|
||||
def test_pagination_structure(self):
|
||||
result = ResponseWrapper.pagination(data=[1, 2], total=5, page=1, size=2)
|
||||
assert result["success"] is True
|
||||
assert result["code"] == 200
|
||||
# 分页字段为 camelCase（totalPages/hasNext/hasPrev），
|
||||
# 与 PaginationResponse 模型的 snake_case（total_pages 等）不一致
|
||||
assert result["data"] == {
|
||||
"list": [1, 2],
|
||||
"pagination": {
|
||||
"total": 5,
|
||||
"page": 1,
|
||||
"size": 2,
|
||||
"totalPages": 3,
|
||||
"hasNext": True,
|
||||
"hasPrev": False,
|
||||
},
|
||||
}
|
||||
|
||||
def test_pagination_size_zero_total_pages_zero(self):
|
||||
result = ResponseWrapper.pagination(data=[], total=0, page=1, size=0)
|
||||
assert result["data"]["pagination"]["totalPages"] == 0
|
||||
|
||||
def test_wrap_plain_data(self):
|
||||
assert ResponseWrapper.wrap({"id": 1}) == {
|
||||
"success": True,
|
||||
"code": 200,
|
||||
"data": {"id": 1},
|
||||
}
|
||||
assert ResponseWrapper.wrap("text", message="m", code=201) == {
|
||||
"success": True,
|
||||
"code": 201,
|
||||
"message": "m",
|
||||
"data": "text",
|
||||
}
|
||||
|
||||
def test_wrap_passthrough_when_keys_present(self):
|
||||
# 当前行为：只要 dict 同时含 "success" 和 "code" 键就原样透传，
|
||||
# 即使是手工构造、值类型任意的字典也不会被再包装
|
||||
already = {"success": False, "code": 500, "message": "x"}
|
||||
assert ResponseWrapper.wrap(already) is already
|
||||
fake = {"success": "yes", "code": "200", "extra": 1}
|
||||
assert ResponseWrapper.wrap(fake) is fake
|
||||
|
||||
def test_api_response_model_dump_includes_none(self):
|
||||
# ConfigDict(exclude_none=True) 在 FastAPI JSON 响应中生效；
|
||||
# model_dump() 默认仍包含 None 字段，需显式 exclude_none=True 才会排除
|
||||
r = ApiResponse(success=True, code=200)
|
||||
assert r.model_dump() == {
|
||||
"success": True,
|
||||
"code": 200,
|
||||
"message": None,
|
||||
"data": None,
|
||||
}
|
||||
assert r.model_dump(exclude_none=True) == {"success": True, "code": 200}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. HTTP 异常体系
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHTTPExceptions:
|
||||
def test_http_status_exceptions_mapping_complete(self):
|
||||
# 10 个状态码齐全，映射到对应异常类
|
||||
assert HTTP_STATUS_EXCEPTIONS == {
|
||||
400: BadRequestError,
|
||||
401: UnauthorizedError,
|
||||
403: ForbiddenError,
|
||||
404: NotFoundError,
|
||||
405: MethodNotAllowedError,
|
||||
409: ConflictError,
|
||||
422: UnprocessableEntityError,
|
||||
429: TooManyRequestsError,
|
||||
500: InternalServerError,
|
||||
503: ServiceUnavailableError,
|
||||
}
|
||||
assert len(HTTP_STATUS_EXCEPTIONS) == 10
|
||||
|
||||
def test_each_exception_default_status_code_and_message(self):
|
||||
expected = {
|
||||
BadRequestError: (400, "错误请求"),
|
||||
UnauthorizedError: (401, "未授权"),
|
||||
ForbiddenError: (403, "禁止访问"),
|
||||
NotFoundError: (404, "未找到"),
|
||||
MethodNotAllowedError: (405, "方法不允许"),
|
||||
ConflictError: (409, "资源冲突"),
|
||||
UnprocessableEntityError: (422, "无法处理的实体"),
|
||||
TooManyRequestsError: (429, "请求过多"),
|
||||
InternalServerError: (500, "内部服务器错误"),
|
||||
ServiceUnavailableError: (503, "服务不可用"),
|
||||
}
|
||||
for exc_class, (status_code, message) in expected.items():
|
||||
exc = exc_class()
|
||||
assert exc.status_code == status_code
|
||||
assert exc.message == message
|
||||
# details 缺省为 {}（非 None）
|
||||
assert exc.details == {}
|
||||
assert isinstance(exc, HTTPException)
|
||||
assert str(exc) == message
|
||||
|
||||
def test_exception_with_custom_message_and_details(self):
|
||||
exc = NotFoundError(message="用户不存在", details={"user_id": 1})
|
||||
assert exc.status_code == 404
|
||||
assert exc.message == "用户不存在"
|
||||
assert exc.details == {"user_id": 1}
|
||||
|
||||
def test_create_http_exception_unknown_status_returns_base(self):
|
||||
exc = create_http_exception(418, "teapot", {"hint": "rfc2324"})
|
||||
assert type(exc) is HTTPException
|
||||
assert exc.status_code == 418
|
||||
assert exc.message == "teapot"
|
||||
assert exc.details == {"hint": "rfc2324"}
|
||||
|
||||
def test_create_http_exception_known_status_returns_mapped_subclass(self):
|
||||
# 0.2.0 修复：旧版对已知状态码用三个位置参数调用 (message, details)
|
||||
# 签名的子类，对全部 10 个已知状态码都抛 TypeError；
|
||||
# 现在已知状态码返回对应子类实例
|
||||
for status_code, exc_class in HTTP_STATUS_EXCEPTIONS.items():
|
||||
exc = create_http_exception(status_code, "msg", {"k": "v"})
|
||||
assert isinstance(exc, exc_class)
|
||||
assert exc.status_code == status_code
|
||||
assert exc.message == "msg"
|
||||
assert exc.details == {"k": "v"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. myboot/exceptions.py 与 myboot/web/exceptions.py 同名类二义性
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestExceptionClassUnification:
|
||||
"""0.2.0 起两套同名异常收敛为同一个类（以 myboot.exceptions 为准），
|
||||
web 路径保留 re-export，import 兼容但身份唯一"""
|
||||
|
||||
def test_validation_error_is_same_class(self):
|
||||
from myboot.exceptions import MyBootException
|
||||
from myboot.exceptions import ValidationError as CoreValidationError
|
||||
from myboot.web.exceptions import ValidationError as WebValidationError
|
||||
|
||||
# 0.2.0：同一个类对象，except 任一路径都能捕获
|
||||
assert CoreValidationError is WebValidationError
|
||||
|
||||
exc = WebValidationError(field="name", value="x")
|
||||
assert isinstance(exc, MyBootException)
|
||||
assert exc.code == "VALIDATION_ERROR"
|
||||
assert exc.message == "验证失败"
|
||||
assert exc.field == "name"
|
||||
assert exc.value == "x"
|
||||
# 旧 web 版独有的 error_type 参数已移除（以核心版签名为准）
|
||||
assert not hasattr(exc, "error_type")
|
||||
|
||||
def test_configuration_error_is_same_class(self):
|
||||
from myboot.exceptions import ConfigurationError as CoreConfigurationError
|
||||
from myboot.exceptions import MyBootException
|
||||
from myboot.web.exceptions import (
|
||||
ConfigurationError as WebConfigurationError,
|
||||
)
|
||||
|
||||
assert CoreConfigurationError is WebConfigurationError
|
||||
|
||||
exc = WebConfigurationError(config_key="app.name")
|
||||
assert isinstance(exc, MyBootException)
|
||||
assert exc.code == "CONFIGURATION_ERROR"
|
||||
assert exc.config_key == "app.name"
|
||||
assert exc.details == {}
|
||||
@@ -0,0 +1,135 @@
|
||||
"""run_primary_first / clear_markers 单元测试"""
|
||||
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from myboot.utils import worker_sync
|
||||
from myboot.utils.worker_sync import run_primary_first, clear_markers
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolated_marker_dir(tmp_path, monkeypatch):
|
||||
"""每个测试使用独立的标记目录，避免相互污染。"""
|
||||
import tempfile
|
||||
monkeypatch.setattr(tempfile, "gettempdir", lambda: str(tmp_path))
|
||||
monkeypatch.setenv("MYBOOT_APP_NAME", "testapp")
|
||||
monkeypatch.delenv("MYBOOT_IS_PRIMARY_WORKER", raising=False)
|
||||
yield tmp_path
|
||||
|
||||
|
||||
def _marker_dir(tmp_path):
|
||||
return tmp_path / "myboot_worker_sync_testapp"
|
||||
|
||||
|
||||
def test_single_process_passthrough():
|
||||
"""无环境变量时默认 primary，直通执行并返回值。"""
|
||||
assert run_primary_first("m1", lambda: 42) == 42
|
||||
|
||||
|
||||
def test_primary_writes_done(monkeypatch, isolated_marker_dir):
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
assert run_primary_first("model", lambda: "ok") == "ok"
|
||||
done = _marker_dir(isolated_marker_dir) / "model.done"
|
||||
assert done.exists()
|
||||
assert done.read_text(encoding="utf-8") # ISO 时间戳非空
|
||||
|
||||
|
||||
def test_secondary_waits_then_runs_secondary_fn(monkeypatch, isolated_marker_dir):
|
||||
"""secondary 在另一线程先等待，primary 后完成 → secondary 执行 secondary_fn。"""
|
||||
results = {}
|
||||
|
||||
def secondary_thread():
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
results["secondary"] = run_primary_first(
|
||||
"model",
|
||||
primary_fn=lambda: "heavy",
|
||||
secondary_fn=lambda: "light",
|
||||
timeout=5.0,
|
||||
poll_interval=0.05,
|
||||
)
|
||||
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
t = threading.Thread(target=secondary_thread)
|
||||
t.start()
|
||||
time.sleep(0.2)
|
||||
# primary 完成
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
assert run_primary_first("model", lambda: "heavy") == "heavy"
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
t.join(timeout=5)
|
||||
assert not t.is_alive()
|
||||
assert results["secondary"] == "light"
|
||||
|
||||
|
||||
def test_secondary_defaults_to_primary_fn(monkeypatch, isolated_marker_dir):
|
||||
"""secondary_fn 为 None 时 secondary 执行 primary_fn。"""
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
run_primary_first("m2", lambda: 1)
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
assert run_primary_first("m2", lambda: 1, timeout=1.0, poll_interval=0.05) == 1
|
||||
|
||||
|
||||
def test_primary_failure_writes_failed_and_raises(monkeypatch, isolated_marker_dir):
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
|
||||
def boom():
|
||||
raise ValueError("download failed")
|
||||
|
||||
with pytest.raises(ValueError, match="download failed"):
|
||||
run_primary_first("model", boom)
|
||||
|
||||
failed = _marker_dir(isolated_marker_dir) / "model.failed"
|
||||
assert failed.exists()
|
||||
assert "download failed" in failed.read_text(encoding="utf-8")
|
||||
|
||||
# secondary 检测到 .failed → RuntimeError 含异常信息
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
with pytest.raises(RuntimeError, match="download failed"):
|
||||
run_primary_first("model", lambda: 1, timeout=2.0, poll_interval=0.05)
|
||||
|
||||
|
||||
def test_secondary_timeout(monkeypatch):
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
with pytest.raises(TimeoutError):
|
||||
run_primary_first("never", lambda: 1, timeout=0.3, poll_interval=0.05)
|
||||
|
||||
|
||||
def test_stale_done_marker_ignored(monkeypatch, isolated_marker_dir):
|
||||
"""上一轮遗留的旧 .done（mtime 过旧）不会让 secondary 跳过等待。"""
|
||||
d = _marker_dir(isolated_marker_dir)
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
stale = d / "model.done"
|
||||
stale.write_text("old-run", encoding="utf-8")
|
||||
old = worker_sync._MODULE_LOAD_TIME - 100
|
||||
os.utime(stale, (old, old))
|
||||
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "0")
|
||||
with pytest.raises(TimeoutError):
|
||||
run_primary_first("model", lambda: 1, timeout=0.3, poll_interval=0.05)
|
||||
|
||||
|
||||
def test_primary_clears_stale_markers(monkeypatch, isolated_marker_dir):
|
||||
"""primary 执行前清理旧标记，成功后 .failed 不残留。"""
|
||||
d = _marker_dir(isolated_marker_dir)
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
(d / "model.failed").write_text("old failure", encoding="utf-8")
|
||||
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
assert run_primary_first("model", lambda: "ok") == "ok"
|
||||
assert not (d / "model.failed").exists()
|
||||
assert (d / "model.done").exists()
|
||||
|
||||
|
||||
def test_clear_markers(monkeypatch, isolated_marker_dir):
|
||||
monkeypatch.setenv("MYBOOT_IS_PRIMARY_WORKER", "1")
|
||||
run_primary_first("a", lambda: 1)
|
||||
run_primary_first("b", lambda: 1)
|
||||
d = _marker_dir(isolated_marker_dir)
|
||||
clear_markers("a")
|
||||
assert not (d / "a.done").exists()
|
||||
assert (d / "b.done").exists()
|
||||
clear_markers()
|
||||
assert not (d / "b.done").exists()
|
||||
Reference in New Issue
Block a user