Add single-node deployment manifests

This commit is contained in:
Qingshan Chen
2026-06-25 14:16:28 +08:00
parent 8f477f8fa9
commit 49da74e87f
21 changed files with 1743 additions and 584 deletions
+1 -1
View File
@@ -115,7 +115,7 @@ jobs:
- name: Deploy manifest to cluster
shell: bash
run: |
cp deployments/k8s/clawmanager.yaml /tmp/clawmanager-ci.yaml
cp deployments/k8s/cluster/clawmanager.yaml /tmp/clawmanager-ci.yaml
python - <<'PY'
from pathlib import Path
+4
View File
@@ -56,6 +56,10 @@ Thumbs.db
*.tmp
*.temp
/test_*
/deploy/
/test.tar
/fix-install-plan.md
/docs/deployment-storage-remediation-plan.zh-CN.md
.codex-logs/
+7 -3
View File
@@ -125,13 +125,17 @@ Runtime authors can follow the [Hermes Runtime Guide](./docs/hermes-runtime-agen
## Get Started
ClawManager now has clearer entry points for both full Kubernetes deployments and lightweight cluster setups. If you want to evaluate the product quickly, start with the guide that matches your environment and then follow the first-use walkthrough.
ClawManager now separates the Kubernetes distribution from the storage profile. Choose `k3s` or `k8s` first, then choose the storage profile that matches the cluster shape:
- Standard Kubernetes deployment: [deployments/k8s/clawmanager.yaml](./deployments/k8s/clawmanager.yaml)
- K3s or lightweight deployment: [deployments/k3s/clawmanager.yaml](./deployments/k3s/clawmanager.yaml)
- k3s single-node HostPath: [deployments/k3s/single-node/clawmanager.yaml](./deployments/k3s/single-node/clawmanager.yaml)
- k3s cluster CSI/RWX example: [deployments/k3s/cluster/clawmanager.yaml](./deployments/k3s/cluster/clawmanager.yaml)
- Kubernetes single-node HostPath: [deployments/k8s/single-node/clawmanager.yaml](./deployments/k8s/single-node/clawmanager.yaml)
- Kubernetes cluster CSI/RWX example: [deployments/k8s/cluster/clawmanager.yaml](./deployments/k8s/cluster/clawmanager.yaml)
- Operations-oriented quick start and first login flow: [User Guide](./docs/use_guide_en.md)
- Deployment notes and architecture-level context: [Deployment Guide](./docs/deployment.md)
The cluster profile is validated with Longhorn (`longhorn` for RWO data and `longhorn-rwx` for RWX workspaces), but these StorageClass names are examples. You can replace them with any CSI classes that provide the same access modes.
## Three Control Planes
### AI Gateway
+7 -3
View File
@@ -126,13 +126,17 @@ Runtime 开发方可以参考 [Hermes Runtime Guide](./docs/hermes-runtime-agent
<a id="get-started"></a>
## 快速开始
ClawManager 现在同时提供标准 Kubernetes 与轻量级集群的清晰入口。如果你想快速评估产品,可以先从匹配你环境的部署路径开始,再进入首次登录与上手流程。
ClawManager 现在 Kubernetes 发行版与存储 profile 拆开。先选择 `k3s``k8s`,再选择匹配集群形态的存储 profile:
- 标准 Kubernetes 部署: [deployments/k8s/clawmanager.yaml](./deployments/k8s/clawmanager.yaml)
- K3s / 轻量集群部署: [deployments/k3s/clawmanager.yaml](./deployments/k3s/clawmanager.yaml)
- k3s 单节点 HostPath: [deployments/k3s/single-node/clawmanager.yaml](./deployments/k3s/single-node/clawmanager.yaml)
- k3s 多节点 CSI/RWX 示例: [deployments/k3s/cluster/clawmanager.yaml](./deployments/k3s/cluster/clawmanager.yaml)
- Kubernetes 单节点 HostPath: [deployments/k8s/single-node/clawmanager.yaml](./deployments/k8s/single-node/clawmanager.yaml)
- Kubernetes 多节点 CSI/RWX 示例: [deployments/k8s/cluster/clawmanager.yaml](./deployments/k8s/cluster/clawmanager.yaml)
- 首次登录与操作流程: [用户指南](./docs/use_guide_cn.md)
- 部署说明与架构背景: [Deployment Guide(英文)](./docs/deployment.md)
多节点 cluster profile 使用 Longhorn 作为官方验证示例,其中 `longhorn` 用于 RWO 数据卷,`longhorn-rwx` 用于 RWX workspace。项目不绑定 Longhorn,用户可以替换为具备相同访问模式能力的 CSI StorageClass。
## 三大控制平面
<a id="ai-gateway"></a>
+187 -36
View File
@@ -16,11 +16,19 @@ type Config struct {
Database DatabaseConfig `yaml:"database"`
JWT JWTConfig `yaml:"jwt"`
Kubernetes KubernetesConfig `yaml:"kubernetes"`
Storage StorageConfig `yaml:"storage"`
Runtime RuntimePoolConfig `yaml:"runtime"`
ObjectStorage ObjectStorageConfig `yaml:"objectStorage"`
SkillScanner SkillScannerConfig `yaml:"skillScanner"`
}
const (
StorageProfileCluster = "cluster"
StorageProfileSingle = "single-node"
StorageProfileLegacyNFS = "legacy-nfs"
DefaultPVCBindTimeout = 2 * time.Minute
)
// ServerConfig holds server-related configuration
type ServerConfig struct {
Address string `yaml:"address"`
@@ -119,24 +127,36 @@ type RuntimePVCConfig struct {
HostPathPrefix string `yaml:"hostPathPrefix"`
}
// StorageConfig controls the validated installation storage profile.
type StorageConfig struct {
Profile string `yaml:"profile"`
HostPathFallbackEnabled bool `yaml:"hostPathFallbackEnabled"`
PVCBindTimeout time.Duration `yaml:"pvcBindTimeout"`
ControlPlaneStorageClass string `yaml:"controlPlaneStorageClass"`
InstanceStorageClass string `yaml:"instanceStorageClass"`
WorkspaceStorageClass string `yaml:"workspaceStorageClass"`
WorkspaceAccessMode string `yaml:"workspaceAccessMode"`
}
// RuntimePoolConfig holds shared V2 runtime pool configuration.
type RuntimePoolConfig struct {
Namespace string `yaml:"namespace"`
WorkspaceRoot string `yaml:"workspaceRoot"`
WorkspaceNFSServer string `yaml:"workspaceNfsServer"`
WorkspaceNFSPath string `yaml:"workspaceNfsPath"`
AgentControlToken string `yaml:"agentControlToken"`
AgentReportToken string `yaml:"agentReportToken"`
BackendReplicaID string `yaml:"backendReplicaId"`
RedisURL string `yaml:"redisUrl"`
SchedulerEnabled bool `yaml:"schedulerEnabled"`
HeartbeatTimeout time.Duration `yaml:"heartbeatTimeout"`
SchedulerTick time.Duration `yaml:"schedulerTick"`
OpenClawImage string `yaml:"openClawImage"`
HermesImage string `yaml:"hermesImage"`
MaxGatewaysPerPod int `yaml:"maxGatewaysPerPod"`
GatewayPortStart int `yaml:"gatewayPortStart"`
GatewayPortEnd int `yaml:"gatewayPortEnd"`
Namespace string `yaml:"namespace"`
WorkspaceRoot string `yaml:"workspaceRoot"`
WorkspacePVCClaimName string `yaml:"workspacePvcClaimName"`
WorkspaceNFSServer string `yaml:"workspaceNfsServer"`
WorkspaceNFSPath string `yaml:"workspaceNfsPath"`
AgentControlToken string `yaml:"agentControlToken"`
AgentReportToken string `yaml:"agentReportToken"`
BackendReplicaID string `yaml:"backendReplicaId"`
RedisURL string `yaml:"redisUrl"`
SchedulerEnabled bool `yaml:"schedulerEnabled"`
HeartbeatTimeout time.Duration `yaml:"heartbeatTimeout"`
SchedulerTick time.Duration `yaml:"schedulerTick"`
OpenClawImage string `yaml:"openClawImage"`
HermesImage string `yaml:"hermesImage"`
MaxGatewaysPerPod int `yaml:"maxGatewaysPerPod"`
GatewayPortStart int `yaml:"gatewayPortStart"`
GatewayPortEnd int `yaml:"gatewayPortEnd"`
}
// LoggingConfig holds logging configuration
@@ -167,6 +187,7 @@ type SkillScannerConfig struct {
// Load loads configuration from file and environment variables
func Load() (*Config, error) {
runtimeNamespace := getEnv("RUNTIME_NAMESPACE", getEnv("K8S_NAMESPACE", "clawmanager-system"))
defaultStorageClass := getEnv("K8S_STORAGE_CLASS", "standard")
config := &Config{
Server: ServerConfig{
Address: ":9001",
@@ -196,7 +217,7 @@ func Load() (*Config, error) {
},
Common: CommonKubernetesConfig{
Namespace: getEnv("K8S_NAMESPACE", "clawreef"),
StorageClass: getEnv("K8S_STORAGE_CLASS", "standard"),
StorageClass: defaultStorageClass,
Timeout: 30,
RetryCount: 3,
AutoCreateNamespace: true,
@@ -223,23 +244,33 @@ func Load() (*Config, error) {
LogAPICalls: false,
},
},
Storage: StorageConfig{
Profile: getEnv("CLAWMANAGER_STORAGE_PROFILE", StorageProfileCluster),
HostPathFallbackEnabled: getEnvBool("K8S_HOSTPATH_FALLBACK_ENABLED", false),
PVCBindTimeout: getEnvDuration("K8S_PVC_BIND_TIMEOUT", DefaultPVCBindTimeout),
ControlPlaneStorageClass: getEnv("K8S_CONTROL_PLANE_STORAGE_CLASS", defaultStorageClass),
InstanceStorageClass: getEnv("K8S_INSTANCE_STORAGE_CLASS", defaultStorageClass),
WorkspaceStorageClass: getEnv("K8S_WORKSPACE_STORAGE_CLASS", defaultStorageClass),
WorkspaceAccessMode: getEnv("K8S_WORKSPACE_ACCESS_MODE", "ReadWriteMany"),
},
Runtime: RuntimePoolConfig{
Namespace: runtimeNamespace,
WorkspaceRoot: getEnv("RUNTIME_WORKSPACE_ROOT", "/workspaces"),
WorkspaceNFSServer: getEnv("RUNTIME_WORKSPACE_NFS_SERVER", defaultWorkspaceNFSServer(runtimeNamespace)),
WorkspaceNFSPath: getEnv("RUNTIME_WORKSPACE_NFS_PATH", "/"),
AgentControlToken: getEnv("RUNTIME_AGENT_CONTROL_TOKEN", ""),
AgentReportToken: getEnv("RUNTIME_AGENT_REPORT_TOKEN", ""),
BackendReplicaID: getEnv("HOSTNAME", "clawmanager-backend-local"),
RedisURL: getEnv("PLATFORM_REDIS_URL", getEnv("TEAM_REDIS_URL", "")),
SchedulerEnabled: getEnvBool("RUNTIME_SCHEDULER_ENABLED", true),
HeartbeatTimeout: getEnvDuration("RUNTIME_HEARTBEAT_TIMEOUT", 10*time.Second),
SchedulerTick: getEnvDuration("RUNTIME_SCHEDULER_TICK", 2*time.Second),
OpenClawImage: getEnv("OPENCLAW_RUNTIME_IMAGE", "ghcr.io/yuan-lab-llm/agentsruntime/openclaw-lite:latest"),
HermesImage: getEnv("HERMES_RUNTIME_IMAGE", "ghcr.io/yuan-lab-llm/agentsruntime/hermes-lite:latest"),
MaxGatewaysPerPod: getEnvInt("RUNTIME_MAX_GATEWAYS_PER_POD", 100),
GatewayPortStart: getEnvInt("RUNTIME_GATEWAY_PORT_START", 20000),
GatewayPortEnd: getEnvInt("RUNTIME_GATEWAY_PORT_END", 20099),
Namespace: runtimeNamespace,
WorkspaceRoot: getEnv("RUNTIME_WORKSPACE_ROOT", "/workspaces"),
WorkspacePVCClaimName: getEnv("RUNTIME_WORKSPACE_PVC_CLAIM", ""),
WorkspaceNFSServer: getEnv("RUNTIME_WORKSPACE_NFS_SERVER", ""),
WorkspaceNFSPath: getEnv("RUNTIME_WORKSPACE_NFS_PATH", "/"),
AgentControlToken: getEnv("RUNTIME_AGENT_CONTROL_TOKEN", ""),
AgentReportToken: getEnv("RUNTIME_AGENT_REPORT_TOKEN", ""),
BackendReplicaID: getEnv("HOSTNAME", "clawmanager-backend-local"),
RedisURL: getEnv("PLATFORM_REDIS_URL", getEnv("TEAM_REDIS_URL", "")),
SchedulerEnabled: getEnvBool("RUNTIME_SCHEDULER_ENABLED", true),
HeartbeatTimeout: getEnvDuration("RUNTIME_HEARTBEAT_TIMEOUT", 10*time.Second),
SchedulerTick: getEnvDuration("RUNTIME_SCHEDULER_TICK", 2*time.Second),
OpenClawImage: getEnv("OPENCLAW_RUNTIME_IMAGE", "ghcr.io/yuan-lab-llm/agentsruntime/openclaw-lite:latest"),
HermesImage: getEnv("HERMES_RUNTIME_IMAGE", "ghcr.io/yuan-lab-llm/agentsruntime/hermes-lite:latest"),
MaxGatewaysPerPod: getEnvInt("RUNTIME_MAX_GATEWAYS_PER_POD", 100),
GatewayPortStart: getEnvInt("RUNTIME_GATEWAY_PORT_START", 20000),
GatewayPortEnd: getEnvInt("RUNTIME_GATEWAY_PORT_END", 20099),
},
ObjectStorage: ObjectStorageConfig{
Endpoint: getEnv("OBJECT_STORAGE_ENDPOINT", ""),
@@ -294,6 +325,7 @@ func Load() (*Config, error) {
// Override with environment variables
applyEnvOverrides(config)
normalizeStorageConfig(config)
return config, nil
}
@@ -349,14 +381,35 @@ func applyEnvOverrides(config *Config) {
if hostPathPrefix := os.Getenv("K8S_PV_HOST_PATH_PREFIX"); hostPathPrefix != "" {
config.Kubernetes.Runtime.PVC.HostPathPrefix = hostPathPrefix
}
if profile := os.Getenv("CLAWMANAGER_STORAGE_PROFILE"); profile != "" {
config.Storage.Profile = profile
}
if fallback := os.Getenv("K8S_HOSTPATH_FALLBACK_ENABLED"); fallback != "" {
config.Storage.HostPathFallbackEnabled = strings.EqualFold(fallback, "true")
}
if timeout := os.Getenv("K8S_PVC_BIND_TIMEOUT"); timeout != "" {
if parsed, err := time.ParseDuration(timeout); err == nil {
config.Storage.PVCBindTimeout = parsed
}
}
if storageClass := os.Getenv("K8S_CONTROL_PLANE_STORAGE_CLASS"); storageClass != "" {
config.Storage.ControlPlaneStorageClass = storageClass
}
if storageClass := os.Getenv("K8S_INSTANCE_STORAGE_CLASS"); storageClass != "" {
config.Storage.InstanceStorageClass = storageClass
}
if storageClass := os.Getenv("K8S_WORKSPACE_STORAGE_CLASS"); storageClass != "" {
config.Storage.WorkspaceStorageClass = storageClass
}
if accessMode := os.Getenv("K8S_WORKSPACE_ACCESS_MODE"); accessMode != "" {
config.Storage.WorkspaceAccessMode = accessMode
}
config.Runtime.Namespace = getEnv("RUNTIME_NAMESPACE", getEnv("K8S_NAMESPACE", config.Runtime.Namespace))
config.Runtime.WorkspaceRoot = getEnv("RUNTIME_WORKSPACE_ROOT", config.Runtime.WorkspaceRoot)
config.Runtime.WorkspacePVCClaimName = getEnv("RUNTIME_WORKSPACE_PVC_CLAIM", config.Runtime.WorkspacePVCClaimName)
config.Runtime.WorkspaceNFSServer = getEnv("RUNTIME_WORKSPACE_NFS_SERVER", config.Runtime.WorkspaceNFSServer)
config.Runtime.WorkspaceNFSPath = getEnv("RUNTIME_WORKSPACE_NFS_PATH", config.Runtime.WorkspaceNFSPath)
if strings.TrimSpace(config.Runtime.WorkspaceNFSServer) == "" {
config.Runtime.WorkspaceNFSServer = defaultWorkspaceNFSServer(config.Runtime.Namespace)
}
if strings.TrimSpace(config.Runtime.WorkspaceNFSPath) == "" {
config.Runtime.WorkspaceNFSPath = "/"
}
@@ -414,6 +467,50 @@ func applyEnvOverrides(config *Config) {
}
}
func normalizeStorageConfig(config *Config) {
if config == nil {
return
}
config.Storage.Profile = normalizeStorageProfile(config.Storage.Profile)
if config.Storage.PVCBindTimeout <= 0 {
config.Storage.PVCBindTimeout = DefaultPVCBindTimeout
}
if strings.TrimSpace(config.Kubernetes.Common.StorageClass) == "" {
config.Kubernetes.Common.StorageClass = "standard"
}
if strings.TrimSpace(config.Storage.ControlPlaneStorageClass) == "" {
config.Storage.ControlPlaneStorageClass = config.Kubernetes.Common.StorageClass
}
if strings.TrimSpace(config.Storage.InstanceStorageClass) == "" {
config.Storage.InstanceStorageClass = config.Kubernetes.Common.StorageClass
}
if strings.TrimSpace(config.Storage.WorkspaceStorageClass) == "" {
config.Storage.WorkspaceStorageClass = config.Kubernetes.Common.StorageClass
}
if strings.TrimSpace(config.Storage.WorkspaceAccessMode) == "" {
config.Storage.WorkspaceAccessMode = "ReadWriteMany"
}
if config.Storage.Profile == StorageProfileLegacyNFS && strings.TrimSpace(config.Runtime.WorkspaceNFSServer) == "" {
config.Runtime.WorkspaceNFSServer = defaultWorkspaceNFSServer(config.Runtime.Namespace)
}
if strings.TrimSpace(config.Runtime.WorkspaceNFSPath) == "" {
config.Runtime.WorkspaceNFSPath = "/"
}
}
func normalizeStorageProfile(value string) string {
switch strings.ToLower(strings.TrimSpace(value)) {
case "", "cluster", "cluster-csi", "csi":
return StorageProfileCluster
case "single", "single-node", "single_node", "hostpath":
return StorageProfileSingle
case "legacy", "legacy-nfs", "legacy_nfs", "nfs":
return StorageProfileLegacyNFS
default:
return strings.ToLower(strings.TrimSpace(value))
}
}
func getEnv(key, defaultValue string) string {
if value := os.Getenv(key); value != "" {
return value
@@ -480,6 +577,60 @@ func (c *Config) GetStorageClass() string {
return c.Kubernetes.Common.StorageClass
}
func (c *Config) GetStorageProfile() string {
if c == nil {
return StorageProfileCluster
}
return normalizeStorageProfile(c.Storage.Profile)
}
func (c *Config) GetPVCBindTimeout() time.Duration {
if c == nil || c.Storage.PVCBindTimeout <= 0 {
return DefaultPVCBindTimeout
}
return c.Storage.PVCBindTimeout
}
func (c *Config) GetControlPlaneStorageClass() string {
if c == nil {
return "standard"
}
if value := strings.TrimSpace(c.Storage.ControlPlaneStorageClass); value != "" {
return value
}
return c.GetStorageClass()
}
func (c *Config) GetInstanceStorageClass() string {
if c == nil {
return "standard"
}
if value := strings.TrimSpace(c.Storage.InstanceStorageClass); value != "" {
return value
}
return c.GetStorageClass()
}
func (c *Config) GetWorkspaceStorageClass() string {
if c == nil {
return "standard"
}
if value := strings.TrimSpace(c.Storage.WorkspaceStorageClass); value != "" {
return value
}
return c.GetStorageClass()
}
func (c *Config) GetWorkspaceAccessMode() string {
if c == nil {
return "ReadWriteMany"
}
if value := strings.TrimSpace(c.Storage.WorkspaceAccessMode); value != "" {
return value
}
return "ReadWriteMany"
}
// GetHostPathPrefix returns the host path prefix for PV creation
func (c *Config) GetHostPathPrefix() string {
if c.Kubernetes.Runtime.PVC.HostPathPrefix != "" {
+92 -1
View File
@@ -1,6 +1,9 @@
package config
import "testing"
import (
"testing"
"time"
)
func TestLoadRuntimeDefaults(t *testing.T) {
for _, key := range []string{
@@ -11,6 +14,7 @@ func TestLoadRuntimeDefaults(t *testing.T) {
"HOSTNAME",
"PLATFORM_REDIS_URL",
"TEAM_REDIS_URL",
"RUNTIME_WORKSPACE_NFS_SERVER",
} {
t.Setenv(key, "")
}
@@ -27,3 +31,90 @@ func TestLoadRuntimeDefaults(t *testing.T) {
t.Fatalf("expected OpenClaw default image %q, got %q", want, got)
}
}
func TestLoadStorageProfileDefaultsDisableHostPathFallback(t *testing.T) {
for _, key := range []string{
"CLAWMANAGER_STORAGE_PROFILE",
"K8S_HOSTPATH_FALLBACK_ENABLED",
"K8S_PVC_BIND_TIMEOUT",
"K8S_CONTROL_PLANE_STORAGE_CLASS",
"K8S_INSTANCE_STORAGE_CLASS",
"K8S_WORKSPACE_STORAGE_CLASS",
"K8S_WORKSPACE_ACCESS_MODE",
"K8S_STORAGE_CLASS",
"RUNTIME_WORKSPACE_NFS_SERVER",
} {
t.Setenv(key, "")
}
cfg, err := Load()
if err != nil {
t.Fatalf("Load returned error: %v", err)
}
if got, want := cfg.Storage.Profile, "cluster"; got != want {
t.Fatalf("storage profile = %q, want %q", got, want)
}
if cfg.Storage.HostPathFallbackEnabled {
t.Fatalf("hostPath fallback must default to disabled for cluster installs")
}
if got, want := cfg.GetPVCBindTimeout(), 2*time.Minute; got != want {
t.Fatalf("PVC bind timeout = %s, want %s", got, want)
}
if got, want := cfg.GetControlPlaneStorageClass(), "standard"; got != want {
t.Fatalf("control-plane storage class = %q, want %q", got, want)
}
if got, want := cfg.GetInstanceStorageClass(), "standard"; got != want {
t.Fatalf("instance storage class = %q, want %q", got, want)
}
if got, want := cfg.GetWorkspaceStorageClass(), "standard"; got != want {
t.Fatalf("workspace storage class = %q, want %q", got, want)
}
if got, want := cfg.GetWorkspaceAccessMode(), "ReadWriteMany"; got != want {
t.Fatalf("workspace access mode = %q, want %q", got, want)
}
if got := cfg.Runtime.WorkspaceNFSServer; got != "" {
t.Fatalf("workspace NFS server must not default to in-cluster service DNS, got %q", got)
}
}
func TestLoadStorageProfileEnvOverrides(t *testing.T) {
t.Setenv("CLAWMANAGER_STORAGE_PROFILE", "single-node")
t.Setenv("K8S_HOSTPATH_FALLBACK_ENABLED", "true")
t.Setenv("K8S_PVC_BIND_TIMEOUT", "45s")
t.Setenv("K8S_CONTROL_PLANE_STORAGE_CLASS", "manual")
t.Setenv("K8S_INSTANCE_STORAGE_CLASS", "manual")
t.Setenv("K8S_WORKSPACE_STORAGE_CLASS", "manual-workspace")
t.Setenv("K8S_WORKSPACE_ACCESS_MODE", "ReadWriteMany")
t.Setenv("RUNTIME_WORKSPACE_PVC_CLAIM", "clawmanager-workspaces")
cfg, err := Load()
if err != nil {
t.Fatalf("Load returned error: %v", err)
}
if got, want := cfg.Storage.Profile, "single-node"; got != want {
t.Fatalf("storage profile = %q, want %q", got, want)
}
if !cfg.Storage.HostPathFallbackEnabled {
t.Fatalf("hostPath fallback should be explicitly enabled")
}
if got, want := cfg.GetPVCBindTimeout(), 45*time.Second; got != want {
t.Fatalf("PVC bind timeout = %s, want %s", got, want)
}
if got, want := cfg.GetControlPlaneStorageClass(), "manual"; got != want {
t.Fatalf("control-plane storage class = %q, want %q", got, want)
}
if got, want := cfg.GetInstanceStorageClass(), "manual"; got != want {
t.Fatalf("instance storage class = %q, want %q", got, want)
}
if got, want := cfg.GetWorkspaceStorageClass(), "manual-workspace"; got != want {
t.Fatalf("workspace storage class = %q, want %q", got, want)
}
if got, want := cfg.GetWorkspaceAccessMode(), "ReadWriteMany"; got != want {
t.Fatalf("workspace access mode = %q, want %q", got, want)
}
if got, want := cfg.Runtime.WorkspacePVCClaimName, "clawmanager-workspaces"; got != want {
t.Fatalf("runtime workspace PVC claim = %q, want %q", got, want)
}
}
+34 -18
View File
@@ -29,15 +29,23 @@ const (
// Client wraps the Kubernetes client
type Client struct {
Clientset kubernetes.Interface
Config *rest.Config
Namespace string
StorageClass string
HostPathPrefix string
WorkspaceRoot string
WorkspaceNFSServer string
WorkspaceNFSPath string
Mode ConnectionMode
Clientset kubernetes.Interface
Config *rest.Config
Namespace string
StorageClass string
StorageProfile string
HostPathFallbackEnabled bool
PVCBindTimeout time.Duration
ControlPlaneStorageClass string
InstanceStorageClass string
WorkspaceStorageClass string
WorkspaceAccessMode string
HostPathPrefix string
WorkspaceRoot string
WorkspacePVCClaimName string
WorkspaceNFSServer string
WorkspaceNFSPath string
Mode ConnectionMode
}
var (
@@ -100,15 +108,23 @@ func Initialize(cfg *config.Config) error {
}
globalClient = &Client{
Clientset: clientset,
Config: restConfig,
Namespace: cfg.GetNamespace(),
StorageClass: cfg.GetStorageClass(),
HostPathPrefix: cfg.GetHostPathPrefix(),
WorkspaceRoot: cfg.Runtime.WorkspaceRoot,
WorkspaceNFSServer: cfg.Runtime.WorkspaceNFSServer,
WorkspaceNFSPath: cfg.Runtime.WorkspaceNFSPath,
Mode: detectedMode,
Clientset: clientset,
Config: restConfig,
Namespace: cfg.GetNamespace(),
StorageClass: cfg.GetStorageClass(),
StorageProfile: cfg.GetStorageProfile(),
HostPathFallbackEnabled: cfg.Storage.HostPathFallbackEnabled,
PVCBindTimeout: cfg.GetPVCBindTimeout(),
ControlPlaneStorageClass: cfg.GetControlPlaneStorageClass(),
InstanceStorageClass: cfg.GetInstanceStorageClass(),
WorkspaceStorageClass: cfg.GetWorkspaceStorageClass(),
WorkspaceAccessMode: cfg.GetWorkspaceAccessMode(),
HostPathPrefix: cfg.GetHostPathPrefix(),
WorkspaceRoot: cfg.Runtime.WorkspaceRoot,
WorkspacePVCClaimName: cfg.Runtime.WorkspacePVCClaimName,
WorkspaceNFSServer: cfg.Runtime.WorkspaceNFSServer,
WorkspaceNFSPath: cfg.Runtime.WorkspaceNFSPath,
Mode: detectedMode,
}
return nil
+126 -54
View File
@@ -24,8 +24,12 @@ type PVCService struct {
}
const (
nodeHostnameLabel = "kubernetes.io/hostname"
noProvisionerName = "kubernetes.io/no-provisioner"
nodeHostnameLabel = "kubernetes.io/hostname"
noProvisionerName = "kubernetes.io/no-provisioner"
storageProfileLabel = "clawmanager.io/storage-profile"
storageProfileSingle = "single-node"
storageProfileLegacyNFS = "legacy-nfs"
defaultPVCBindTimeout = 2 * time.Minute
)
// NewPVCService creates a new PVC service
@@ -57,7 +61,7 @@ func (s *PVCService) CreatePVC(ctx context.Context, userID, instanceID int, stor
// Use default storage class if not specified
if storageClass == "" {
storageClass = s.client.StorageClass
storageClass = firstNonEmpty(s.client.InstanceStorageClass, s.client.StorageClass)
}
fmt.Printf("Creating PVC %s in namespace %s with storageClass: %s\n", pvcName, namespace, storageClass)
@@ -121,7 +125,7 @@ func (s *PVCService) CreatePVC(ctx context.Context, userID, instanceID int, stor
// Wait for PVC to be bound, if not bound within timeout, create PV manually
fmt.Printf("PVC %s created, scheduling async binding monitor...\n", pvcName)
go s.monitorPVCBinding(context.Background(), namespace, pvcName, userID, instanceID, storageSizeGB, storageClass, 15*time.Second)
go s.monitorPVCBinding(context.Background(), namespace, pvcName, userID, instanceID, storageSizeGB, storageClass, s.pvcBindTimeout())
return createdPVC, nil
}
@@ -142,9 +146,10 @@ func (s *PVCService) CreateTeamSharedPVC(ctx context.Context, userID, teamID, st
pvcName := s.client.GetTeamSharedPVCName(teamID)
namespace := s.client.GetNamespace(userID)
if storageClass == "" {
storageClass = s.client.StorageClass
storageClass = firstNonEmpty(s.client.WorkspaceStorageClass, s.client.StorageClass)
}
storageSize := resource.MustParse(fmt.Sprintf("%dGi", storageSizeGB))
workspaceAccessMode := workspacePVCAccessMode(s.client.WorkspaceAccessMode)
useWorkspaceNFS := strings.TrimSpace(s.client.WorkspaceNFSServer) != ""
if useWorkspaceNFS {
if err := s.ensureTeamSharedWorkspaceDirectory(userID, teamID); err != nil {
@@ -164,7 +169,7 @@ func (s *PVCService) CreateTeamSharedPVC(ctx context.Context, userID, teamID, st
},
Spec: corev1.PersistentVolumeClaimSpec{
AccessModes: []corev1.PersistentVolumeAccessMode{
corev1.ReadWriteMany,
workspaceAccessMode,
},
Resources: corev1.VolumeResourceRequirements{
Requests: corev1.ResourceList{
@@ -186,7 +191,7 @@ func (s *PVCService) CreateTeamSharedPVC(ctx context.Context, userID, teamID, st
}
}
if existingPVC.Status.Phase != corev1.ClaimBound {
go s.monitorTeamSharedPVCBinding(context.Background(), namespace, pvcName, userID, teamID, storageSizeGB, storageClass, 15*time.Second)
go s.monitorTeamSharedPVCBinding(context.Background(), namespace, pvcName, userID, teamID, storageSizeGB, storageClass, s.pvcBindTimeout())
}
return existingPVC, nil
}
@@ -199,7 +204,7 @@ func (s *PVCService) CreateTeamSharedPVC(ctx context.Context, userID, teamID, st
return nil, err
}
}
go s.monitorTeamSharedPVCBinding(context.Background(), namespace, pvcName, userID, teamID, storageSizeGB, storageClass, 15*time.Second)
go s.monitorTeamSharedPVCBinding(context.Background(), namespace, pvcName, userID, teamID, storageSizeGB, storageClass, s.pvcBindTimeout())
return createdPVC, nil
}
@@ -225,7 +230,10 @@ func (s *PVCService) waitForTeamSharedPVCBinding(ctx context.Context, namespace,
for {
select {
case <-timeoutChan:
fmt.Printf("Team shared PVC %s binding timeout, creating hostPath RWX PV manually\n", pvcName)
if !s.hostPathFallbackEnabled() {
return nil, fmt.Errorf("Team shared PVC %s was not bound after %s and hostPath fallback is disabled for storage profile %q", pvcName, timeout, s.storageProfile())
}
fmt.Printf("Team shared PVC %s binding timeout, creating hostPath PV manually\n", pvcName)
return s.createPVForTeamSharedPVC(ctx, namespace, pvcName, userID, teamID, storageSizeGB, storageClass)
case <-ticker.C:
pvc, err := s.client.Clientset.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, pvcName, metav1.GetOptions{})
@@ -284,10 +292,11 @@ func (s *PVCService) createPVForTeamSharedPVC(ctx context.Context, namespace, pv
ObjectMeta: metav1.ObjectMeta{
Name: pvName,
Labels: map[string]string{
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"team-id": fmt.Sprintf("%d", teamID),
"managed-by": "clawreef",
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"team-id": fmt.Sprintf("%d", teamID),
"managed-by": "clawreef",
storageProfileLabel: storageProfileSingle,
},
},
Spec: corev1.PersistentVolumeSpec{
@@ -295,7 +304,7 @@ func (s *PVCService) createPVForTeamSharedPVC(ctx context.Context, namespace, pv
corev1.ResourceStorage: storageSize,
},
AccessModes: []corev1.PersistentVolumeAccessMode{
corev1.ReadWriteMany,
workspacePVCAccessMode(s.client.WorkspaceAccessMode),
},
PersistentVolumeReclaimPolicy: corev1.PersistentVolumeReclaimRetain,
StorageClassName: storageClass,
@@ -367,10 +376,11 @@ func (s *PVCService) ensureWorkspaceNFSPVForTeamSharedPVC(ctx context.Context, n
ObjectMeta: metav1.ObjectMeta{
Name: pvName,
Labels: map[string]string{
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"team-id": fmt.Sprintf("%d", teamID),
"managed-by": "clawreef",
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"team-id": fmt.Sprintf("%d", teamID),
"managed-by": "clawreef",
storageProfileLabel: storageProfileLegacyNFS,
},
},
Spec: corev1.PersistentVolumeSpec{
@@ -629,7 +639,9 @@ func (s *PVCService) waitForPVCBinding(ctx context.Context, namespace, pvcName s
for {
select {
case <-timeoutChan:
// Timeout, try to create PV manually
if !s.hostPathFallbackEnabled() {
return nil, fmt.Errorf("PVC %s was not bound after %s and hostPath fallback is disabled for storage profile %q", pvcName, timeout, s.storageProfile())
}
fmt.Printf("PVC %s binding timeout, creating PV manually\n", pvcName)
return s.createPVForPVC(ctx, namespace, pvcName, userID, instanceID, storageSizeGB, storageClass)
case <-ticker.C:
@@ -711,10 +723,11 @@ func (s *PVCService) createPVForPVC(ctx context.Context, namespace, pvcName stri
ObjectMeta: metav1.ObjectMeta{
Name: pvName,
Labels: map[string]string{
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"instance-id": fmt.Sprintf("%d", instanceID),
"managed-by": "clawreef",
"app": "clawreef",
"user-id": fmt.Sprintf("%d", userID),
"instance-id": fmt.Sprintf("%d", instanceID),
"managed-by": "clawreef",
storageProfileLabel: storageProfileSingle,
},
},
Spec: corev1.PersistentVolumeSpec{
@@ -808,7 +821,7 @@ func (s *PVCService) NodeSelectorForPVC(ctx context.Context, userID, instanceID
return nil, fmt.Errorf("failed to get PVC %s: %w", pvcName, err)
}
storageClass = pvcStorageClassName(pvc, storageClass)
storageClass = pvcStorageClassName(pvc, firstNonEmpty(storageClass, s.client.InstanceStorageClass, s.client.StorageClass))
if strings.TrimSpace(pvc.Spec.VolumeName) != "" {
pv, err := s.client.Clientset.CoreV1().PersistentVolumes().Get(ctx, pvc.Spec.VolumeName, metav1.GetOptions{})
@@ -828,7 +841,7 @@ func (s *PVCService) nodeSelectorForStorageClass(ctx context.Context, storageCla
storageClass = strings.TrimSpace(storageClass)
if storageClass == "" {
storageClass = strings.TrimSpace(s.client.StorageClass)
storageClass = firstNonEmpty(s.client.InstanceStorageClass, s.client.StorageClass)
}
if storageClass == "" {
return nil, nil
@@ -855,6 +868,63 @@ func pvcStorageClassName(pvc *corev1.PersistentVolumeClaim, fallback string) str
return storageClass
}
func firstNonEmpty(values ...string) string {
for _, value := range values {
if trimmed := strings.TrimSpace(value); trimmed != "" {
return trimmed
}
}
return ""
}
func workspacePVCAccessMode(value string) corev1.PersistentVolumeAccessMode {
switch strings.ToLower(strings.TrimSpace(value)) {
case "readwriteonce", "rwo":
return corev1.ReadWriteOnce
case "readonlymany", "rox":
return corev1.ReadOnlyMany
case "readwritemany", "rwx", "":
return corev1.ReadWriteMany
default:
return corev1.ReadWriteMany
}
}
func (s *PVCService) pvcBindTimeout() time.Duration {
if s != nil && s.client != nil && s.client.PVCBindTimeout > 0 {
return s.client.PVCBindTimeout
}
return defaultPVCBindTimeout
}
func (s *PVCService) storageProfile() string {
if s == nil || s.client == nil {
return ""
}
return strings.ToLower(strings.TrimSpace(s.client.StorageProfile))
}
func (s *PVCService) hostPathFallbackEnabled() bool {
if s == nil || s.client == nil || !s.client.HostPathFallbackEnabled {
return false
}
profile := s.storageProfile()
return profile == "" || profile == storageProfileSingle
}
func isClawManagerHostPathPV(pv *corev1.PersistentVolume) bool {
if pv == nil || pv.Spec.HostPath == nil {
return false
}
if pv.Labels["managed-by"] != "clawreef" {
return false
}
if strings.EqualFold(pv.Labels[storageProfileLabel], storageProfileSingle) {
return true
}
return strings.HasPrefix(pv.Name, "clawreef-pv-user-")
}
func nodeSelectorForHostname(hostname string) map[string]string {
hostname = strings.TrimSpace(hostname)
if hostname == "" {
@@ -897,10 +967,7 @@ func (s *PVCService) DeletePVC(ctx context.Context, userID, instanceID int) erro
if errors.IsNotFound(err) {
// PVC doesn't exist, still try to delete the PV directly
fmt.Printf("PVC %s not found, trying to delete PV %s directly\n", pvcName, pvName)
deleteErr := s.client.Clientset.CoreV1().PersistentVolumes().Delete(ctx, pvName, metav1.DeleteOptions{})
if deleteErr != nil && !errors.IsNotFound(deleteErr) {
fmt.Printf("Warning: failed to delete PV %s: %v\n", pvName, deleteErr)
}
s.deleteManagedHostPathPV(ctx, pvName)
return nil
}
return fmt.Errorf("failed to get PVC %s: %w", pvcName, err)
@@ -924,34 +991,43 @@ func (s *PVCService) DeletePVC(ctx context.Context, userID, instanceID int) erro
// Delete the manually created PV
// First try the predictable name
err = s.client.Clientset.CoreV1().PersistentVolumes().Delete(ctx, pvName, metav1.DeleteOptions{})
if err != nil {
if errors.IsNotFound(err) {
fmt.Printf("PV %s (predictable name) not found\n", pvName)
} else {
fmt.Printf("Warning: failed to delete PV %s: %v\n", pvName, err)
}
} else {
fmt.Printf("PV %s deleted successfully\n", pvName)
}
s.deleteManagedHostPathPV(ctx, pvName)
// Also try to delete the bound PV if it exists and is different
if boundPVName != "" && boundPVName != pvName {
err = s.client.Clientset.CoreV1().PersistentVolumes().Delete(ctx, boundPVName, metav1.DeleteOptions{})
if err != nil {
if errors.IsNotFound(err) {
fmt.Printf("PV %s (bound) not found or already deleted\n", boundPVName)
} else {
fmt.Printf("Warning: failed to delete bound PV %s: %v\n", boundPVName, err)
}
} else {
fmt.Printf("Bound PV %s deleted successfully\n", boundPVName)
}
s.deleteManagedHostPathPV(ctx, boundPVName)
}
return nil
}
func (s *PVCService) deleteManagedHostPathPV(ctx context.Context, pvName string) {
pvName = strings.TrimSpace(pvName)
if pvName == "" || s == nil || s.client == nil || s.client.Clientset == nil {
return
}
pv, err := s.client.Clientset.CoreV1().PersistentVolumes().Get(ctx, pvName, metav1.GetOptions{})
if err != nil {
if errors.IsNotFound(err) {
fmt.Printf("PV %s not found or already deleted\n", pvName)
} else {
fmt.Printf("Warning: failed to get PV %s before delete: %v\n", pvName, err)
}
return
}
if !isClawManagerHostPathPV(pv) {
fmt.Printf("Skipping PV %s delete because it is not a ClawManager-managed hostPath PV\n", pvName)
return
}
if err := s.client.Clientset.CoreV1().PersistentVolumes().Delete(ctx, pvName, metav1.DeleteOptions{}); err != nil {
if !errors.IsNotFound(err) {
fmt.Printf("Warning: failed to delete PV %s: %v\n", pvName, err)
}
return
}
fmt.Printf("PV %s deleted successfully\n", pvName)
}
// DeleteTeamSharedPVC deletes a Team shared PVC and the predictable hostPath PV
// used by the single-node manual storage fallback. It does not delete an
// arbitrary bound PV because that may belong to a real RWX provisioner.
@@ -969,11 +1045,7 @@ func (s *PVCService) DeleteTeamSharedPVC(ctx context.Context, userID, teamID int
return fmt.Errorf("failed to delete Team shared PVC %s/%s: %w", namespace, pvcName, err)
}
}
if err := s.client.Clientset.CoreV1().PersistentVolumes().Delete(ctx, pvName, metav1.DeleteOptions{}); err != nil {
if !errors.IsNotFound(err) {
return fmt.Errorf("failed to delete Team shared PV %s: %w", pvName, err)
}
}
s.deleteManagedHostPathPV(ctx, pvName)
return nil
}
@@ -6,9 +6,11 @@ import (
"path/filepath"
"strings"
"testing"
"time"
corev1 "k8s.io/api/core/v1"
storagev1 "k8s.io/api/storage/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes/fake"
@@ -326,6 +328,134 @@ func TestCreateTeamSharedPVCRejectsExistingNonWorkspaceNFSPV(t *testing.T) {
}
}
func TestCreatePVCUsesInstanceStorageClassDefault(t *testing.T) {
ctx := context.Background()
client := &Client{
Clientset: fake.NewSimpleClientset(),
Namespace: "clawmanager",
StorageClass: "standard",
InstanceStorageClass: "longhorn",
}
service := &PVCService{
client: client,
namespaceService: &NamespaceService{client: client},
}
pvc, err := service.CreatePVC(ctx, 1, 371, 10, "")
if err != nil {
t.Fatalf("CreatePVC returned error: %v", err)
}
if pvc.Spec.StorageClassName == nil || *pvc.Spec.StorageClassName != "longhorn" {
t.Fatalf("PVC storage class = %#v, want longhorn", pvc.Spec.StorageClassName)
}
}
func TestCreateTeamSharedPVCUsesWorkspaceStorageDefaults(t *testing.T) {
ctx := context.Background()
client := &Client{
Clientset: fake.NewSimpleClientset(),
Namespace: "clawmanager",
StorageClass: "standard",
WorkspaceStorageClass: "longhorn-rwx",
WorkspaceAccessMode: "ReadWriteMany",
}
service := &PVCService{
client: client,
namespaceService: &NamespaceService{client: client},
}
pvc, err := service.CreateTeamSharedPVC(ctx, 1, 28, 10, "")
if err != nil {
t.Fatalf("CreateTeamSharedPVC returned error: %v", err)
}
if pvc.Spec.StorageClassName == nil || *pvc.Spec.StorageClassName != "longhorn-rwx" {
t.Fatalf("PVC storage class = %#v, want longhorn-rwx", pvc.Spec.StorageClassName)
}
if len(pvc.Spec.AccessModes) != 1 || pvc.Spec.AccessModes[0] != corev1.ReadWriteMany {
t.Fatalf("PVC access modes = %#v, want ReadWriteMany", pvc.Spec.AccessModes)
}
}
func TestWaitForPVCBindingDoesNotCreateHostPathPVWhenFallbackDisabled(t *testing.T) {
ctx := context.Background()
namespace := "clawmanager-user-162"
pvc := &corev1.PersistentVolumeClaim{
ObjectMeta: metav1.ObjectMeta{
Name: "clawreef-371-pvc",
Namespace: namespace,
},
Status: corev1.PersistentVolumeClaimStatus{Phase: corev1.ClaimPending},
}
clientset := fake.NewSimpleClientset(pvc)
service := &PVCService{
client: &Client{
Clientset: clientset,
Namespace: "clawmanager",
StorageProfile: "cluster",
HostPathFallbackEnabled: false,
},
}
_, err := service.waitForPVCBinding(ctx, namespace, pvc.Name, 162, 371, 10, "longhorn", time.Millisecond)
if err == nil || !strings.Contains(err.Error(), "hostPath fallback is disabled") {
t.Fatalf("expected disabled fallback error, got %v", err)
}
pvs, listErr := clientset.CoreV1().PersistentVolumes().List(ctx, metav1.ListOptions{})
if listErr != nil {
t.Fatalf("failed to list PVs: %v", listErr)
}
if len(pvs.Items) != 0 {
t.Fatalf("expected no hostPath PV to be created, got %#v", pvs.Items)
}
}
func TestDeletePVCSkipsDynamicProvisionerPV(t *testing.T) {
ctx := context.Background()
namespace := "clawmanager-user-162"
pvc := &corev1.PersistentVolumeClaim{
ObjectMeta: metav1.ObjectMeta{
Name: "clawreef-371-pvc",
Namespace: namespace,
Labels: map[string]string{"instance-id": "371"},
},
Spec: corev1.PersistentVolumeClaimSpec{
VolumeName: "pvc-dynamic-123",
},
Status: corev1.PersistentVolumeClaimStatus{Phase: corev1.ClaimBound},
}
pv := &corev1.PersistentVolume{
ObjectMeta: metav1.ObjectMeta{
Name: "pvc-dynamic-123",
Labels: map[string]string{
"managed-by": "external-provisioner",
},
},
Spec: corev1.PersistentVolumeSpec{
PersistentVolumeSource: corev1.PersistentVolumeSource{
CSI: &corev1.CSIPersistentVolumeSource{Driver: "driver.longhorn.io"},
},
},
}
client := &Client{
Clientset: fake.NewSimpleClientset(pvc, pv),
Namespace: "clawmanager",
}
service := &PVCService{
client: client,
namespaceService: &NamespaceService{client: client},
}
if err := service.DeletePVC(ctx, 162, 371); err != nil {
t.Fatalf("DeletePVC returned error: %v", err)
}
if _, err := client.Clientset.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, pvc.Name, metav1.GetOptions{}); !errors.IsNotFound(err) {
t.Fatalf("expected PVC to be deleted, got err=%v", err)
}
if _, err := client.Clientset.CoreV1().PersistentVolumes().Get(ctx, pv.Name, metav1.GetOptions{}); err != nil {
t.Fatalf("dynamic provisioner PV must not be deleted, got %v", err)
}
}
func TestHostPathPVNodeAffinitySkipsHardTaintedNodes(t *testing.T) {
service := &PVCService{
client: &Client{
@@ -30,20 +30,21 @@ const (
)
type RuntimeDeploymentSpec struct {
Name string
Namespace string
RuntimeType string
Image string
Replicas int32
WorkspaceNFSServer string
WorkspaceNFSPath string
WorkspaceMountPath string
GatewayPortStart int
GatewayPortEnd int
AgentControlToken string
AgentReportToken string
BackendURL string
TrustedProxyCIDRs string
Name string
Namespace string
RuntimeType string
Image string
Replicas int32
WorkspacePVCClaimName string
WorkspaceNFSServer string
WorkspaceNFSPath string
WorkspaceMountPath string
GatewayPortStart int
GatewayPortEnd int
AgentControlToken string
AgentReportToken string
BackendURL string
TrustedProxyCIDRs string
}
type RuntimeDeploymentPod struct {
@@ -126,13 +127,8 @@ func BuildRuntimeDeployment(spec RuntimeDeploymentSpec) *appsv1.Deployment {
},
Volumes: []corev1.Volume{
{
Name: runtimeWorkspaceVolume,
VolumeSource: corev1.VolumeSource{
NFS: &corev1.NFSVolumeSource{
Server: spec.WorkspaceNFSServer,
Path: spec.WorkspaceNFSPath,
},
},
Name: runtimeWorkspaceVolume,
VolumeSource: runtimeWorkspaceVolumeSource(spec),
},
},
},
@@ -141,6 +137,29 @@ func BuildRuntimeDeployment(spec RuntimeDeploymentSpec) *appsv1.Deployment {
}
}
func runtimeWorkspaceVolumeSource(spec RuntimeDeploymentSpec) corev1.VolumeSource {
if claimName := strings.TrimSpace(spec.WorkspacePVCClaimName); claimName != "" {
return corev1.VolumeSource{
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
ClaimName: claimName,
},
}
}
if server := strings.TrimSpace(spec.WorkspaceNFSServer); server != "" {
nfsPath := strings.TrimSpace(spec.WorkspaceNFSPath)
if nfsPath == "" {
nfsPath = "/"
}
return corev1.VolumeSource{
NFS: &corev1.NFSVolumeSource{
Server: server,
Path: nfsPath,
},
}
}
return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}
}
func buildRuntimeAgentEnv(spec RuntimeDeploymentSpec, workspaceRoot string, gatewayPortStart, gatewayPortEnd int) []corev1.EnvVar {
env := []corev1.EnvVar{
{Name: "CLAWMANAGER_RUNTIME_TYPE", Value: spec.RuntimeType},
@@ -74,6 +74,33 @@ func TestBuildRuntimeDeployment(t *testing.T) {
}
}
func TestBuildRuntimeDeploymentUsesWorkspacePVCWhenClaimConfigured(t *testing.T) {
deployment := BuildRuntimeDeployment(RuntimeDeploymentSpec{
Name: "runtime-openclaw",
Namespace: "runtime-system",
RuntimeType: "openclaw",
Image: "registry/openclaw:latest",
Replicas: 2,
WorkspacePVCClaimName: "clawmanager-workspaces",
WorkspaceNFSServer: "workspace-store.clawmanager-system.svc.cluster.local",
WorkspaceNFSPath: "/exports/workspaces",
})
if len(deployment.Spec.Template.Spec.Volumes) != 1 {
t.Fatalf("expected one volume, got %#v", deployment.Spec.Template.Spec.Volumes)
}
volume := deployment.Spec.Template.Spec.Volumes[0]
if volume.Name != "workspaces" || volume.PersistentVolumeClaim == nil {
t.Fatalf("expected workspaces PVC volume, got %#v", volume)
}
if got, want := volume.PersistentVolumeClaim.ClaimName, "clawmanager-workspaces"; got != want {
t.Fatalf("PVC claim name = %q, want %q", got, want)
}
if volume.NFS != nil {
t.Fatalf("PVC workspace volume must not also include NFS source: %#v", volume.NFS)
}
}
func TestBuildRuntimeDeploymentKeepsCapacityLimitOutOfRuntimeEnv(t *testing.T) {
deployment := BuildRuntimeDeployment(RuntimeDeploymentSpec{
Name: "runtime-openclaw",
+201
View File
@@ -0,0 +1,201 @@
# ClawManager K3s Single-Node Manifest
This directory contains the all-in-one K3s manifest for a single-node
ClawManager deployment:
- `clawmanager.yaml`: ClawManager workloads plus static HostPath PVs.
- No Longhorn dependency.
- No dynamic StorageClass dependency.
Use this profile only when ClawManager workloads and user runtimes can run on
one storage node. For multi-node clusters, use `deployments/k3s/cluster` instead.
## Storage Model
The manifest uses static PV/PVC binding:
- StorageClass name: `manual`
- System HostPath root: `/data/clawmanager/system`
- Runtime HostPath root: `/data/clawreef`
- Storage node label: `clawmanager.io/storage-node=true`
The `manual` value is only a matching name between bundled PVs and PVCs. It does
not require a Kubernetes `StorageClass` object or dynamic provisioner.
The bundled PVs use `persistentVolumeReclaimPolicy: Retain`, so deleting the
manifest does not erase HostPath data.
## Install
### 1. Enter This Directory
```sh
cd deployments/k3s/single-node
```
Check that `kubectl` points to the target cluster:
```sh
kubectl config current-context
kubectl get nodes -o wide
```
Expected result: the target node is `Ready`.
### 2. Choose And Label One Storage Node
Label exactly one node before installing:
```sh
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl get nodes -l clawmanager.io/storage-node=true
```
Expected result: only one node is listed.
If more than one node has this label, remove the label from the extra nodes:
```sh
kubectl label node <node> clawmanager.io/storage-node-
```
### 3. Check For Old Installation State
```sh
kubectl get ns clawmanager-system clawmanager-user-1 --ignore-not-found
kubectl get pv | grep clawmanager || true
kubectl get pvc -A | grep clawmanager || true
```
Expected result for a fresh install: no old ClawManager namespace, PV, or PVC
remains.
If old resources exist, run the uninstall flow in this README before installing
again.
### 4. Check HostPath Directories
The kubelet can create the HostPath directories automatically, but checking the
target path first makes permission and disk issues easier to diagnose:
```sh
df -h /data || true
ls -ld /data || true
```
If `/data` is not the right disk for this node, edit these paths in
`clawmanager.yaml` before applying it:
- `/data/clawmanager/system/mysql`
- `/data/clawmanager/system/minio`
- `/data/clawmanager/system/redis`
- `/data/clawmanager/system/workspaces`
- `K8S_PV_HOST_PATH_PREFIX=/data/clawreef`
### 5. Check Image Availability
The manifest uses the images listed in `clawmanager.yaml`. Make sure the node
can pull from the referenced registry before installing:
```sh
grep -n 'image:' clawmanager.yaml | head -n 40
```
If this is an offline or private-registry deployment, load or push the images to
the registry used by the manifest before continuing.
### 6. Apply The Manifest
```sh
kubectl apply -f clawmanager.yaml
```
### 7. Check ClawManager
```sh
kubectl -n clawmanager-system get pvc
kubectl -n clawmanager-system rollout status deployment/clawmanager-app --timeout=15m
kubectl -n clawmanager-system rollout status deployment/openclaw-runtime --timeout=15m
kubectl -n clawmanager-system rollout status deployment/hermes-runtime --timeout=15m
kubectl -n clawmanager-system get deploy,pod,pvc -o wide
```
Expected result:
- `mysql-data`, `redis-data`, `minio-data`, and `clawmanager-workspaces` are `Bound`.
- `clawmanager-app`, `openclaw-runtime`, and `hermes-runtime` are available.
- MySQL, Redis, MinIO, and skill scanner pods are running.
If PVCs stay `Pending`, check the static PV binding:
```sh
kubectl get pv | grep clawmanager
kubectl -n clawmanager-system describe pvc <pvc-name>
```
If pods stay `Pending` with node affinity errors, check the storage node label:
```sh
kubectl get nodes -l clawmanager.io/storage-node=true
kubectl describe pod <pod-name> -n clawmanager-system
```
If pods stay `ContainerCreating`, check recent events:
```sh
kubectl -n clawmanager-system get events --sort-by=.lastTimestamp | tail -n 80
```
## Uninstall
Use this order for a disposable single-node test deployment.
### 1. Delete ClawManager Resources
```sh
kubectl delete -f clawmanager.yaml --ignore-not-found
kubectl delete namespace clawmanager-user-1 --ignore-not-found
kubectl delete namespace clawmanager-system --ignore-not-found
```
Check:
```sh
kubectl get ns clawmanager-system clawmanager-user-1 --ignore-not-found
kubectl get pv | grep clawmanager || true
kubectl get pvc -A | grep clawmanager || true
```
The PVs may remain because they use `Retain`. This is expected.
### 2. Remove Retained PVs When Data Is No Longer Needed
Only run this after confirming the data can be discarded:
```sh
kubectl delete pv \
clawmanager-mysql-pv \
clawmanager-minio-pv \
clawmanager-redis-pv \
clawmanager-workspaces-pv \
--ignore-not-found
```
### 3. Remove HostPath Data When Data Is No Longer Needed
Run this on the labeled storage node only after confirming the data can be
discarded:
```sh
rm -rf /data/clawmanager/system /data/clawreef
```
## Common Issues
| Symptom | Likely Cause | Check Or Fix |
| :--- | :--- | :--- |
| PVC stays `Pending` | Static PV and PVC did not bind | `kubectl get pv`, then `kubectl describe pvc -n clawmanager-system <pvc-name>` |
| Pod stays `Pending` | Missing or wrong `clawmanager.io/storage-node=true` label | `kubectl get nodes -l clawmanager.io/storage-node=true` |
| Pod is `ImagePullBackOff` | Node cannot pull an image | `kubectl describe pod <pod-name> -n <namespace>` |
| Pod is `ContainerCreating` | HostPath mount or permission issue | `kubectl -n clawmanager-system get events --sort-by=.lastTimestamp` |
| Runtime creation fails | Runtime HostPath root is unavailable | Check `K8S_PV_HOST_PATH_PREFIX` and disk space on the labeled node |
@@ -1,3 +1,8 @@
# ClawManager single-node install profile.
# Official validation target: HostPath on one labeled storage node.
# Before installing, label exactly one node:
# kubectl label node <node> clawmanager.io/storage-node=true --overwrite
# Replace paths or StorageClass names only if you understand the single-node constraints.
apiVersion: v1
kind: Namespace
metadata:
@@ -716,6 +721,11 @@ apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-mysql-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 5Gi
@@ -724,6 +734,14 @@ spec:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/mysql
type: DirectoryOrCreate
@@ -746,6 +764,11 @@ apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-minio-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 10Gi
@@ -754,6 +777,14 @@ spec:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/minio
type: DirectoryOrCreate
@@ -772,6 +803,92 @@ spec:
storageClassName: manual
volumeName: clawmanager-minio-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-redis-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/redis
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-data
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: manual
volumeName: clawmanager-redis-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-workspaces-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 50Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/workspaces
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: clawmanager-workspaces
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
storageClassName: manual
volumeName: clawmanager-workspaces-pv
---
apiVersion: apps/v1
kind: Deployment
metadata:
@@ -791,6 +908,8 @@ spec:
- name: mysql
image: mysql:8.4.8
imagePullPolicy: IfNotPresent
args:
- "--innodb-use-native-aio=0"
ports:
- containerPort: 3306
env:
@@ -944,7 +1063,8 @@ spec:
mountPath: /data
volumes:
- name: redis-data
emptyDir: {}
persistentVolumeClaim:
claimName: redis-data
---
apiVersion: v1
kind: Service
@@ -973,84 +1093,6 @@ spec:
targetPort: redis
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-workspaces-pv
spec:
capacity:
storage: 50Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
hostPath:
path: /data/clawmanager/system/workspaces
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: clawmanager-workspaces
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
storageClassName: manual
volumeName: clawmanager-workspaces-pv
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: workspace-store
namespace: clawmanager-system
spec:
replicas: 1
selector:
matchLabels:
app: workspace-store
template:
metadata:
labels:
app: workspace-store
spec:
containers:
- name: nfs-server
image: itsthenetwork/nfs-server-alpine:12
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
env:
- name: SHARED_DIRECTORY
value: /exports/workspaces
ports:
- name: nfs
containerPort: 2049
volumeMounts:
- name: workspaces
mountPath: /exports/workspaces
volumes:
- name: workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: v1
kind: Service
metadata:
name: workspace-store
namespace: clawmanager-system
spec:
selector:
app: workspace-store
ports:
- name: nfs
port: 2049
targetPort: nfs
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: clawmanager-app
@@ -1177,7 +1219,7 @@ metadata:
name: clawmanager-app
namespace: clawmanager-system
spec:
replicas: 3
replicas: 1
selector:
matchLabels:
app: clawmanager-app
@@ -1247,6 +1289,20 @@ spec:
value: "clawmanager"
- name: K8S_STORAGE_CLASS
value: "manual"
- name: CLAWMANAGER_STORAGE_PROFILE
value: "single-node"
- name: K8S_HOSTPATH_FALLBACK_ENABLED
value: "true"
- name: K8S_PVC_BIND_TIMEOUT
value: "2m"
- name: K8S_CONTROL_PLANE_STORAGE_CLASS
value: "manual"
- name: K8S_INSTANCE_STORAGE_CLASS
value: "manual"
- name: K8S_WORKSPACE_STORAGE_CLASS
value: "manual"
- name: K8S_WORKSPACE_ACCESS_MODE
value: "ReadWriteMany"
- name: K8S_PV_HOST_PATH_PREFIX
value: "/data/clawreef"
- name: CLAWMANAGER_WORKSPACE_ARCHIVE_MAX_MIB
@@ -1263,10 +1319,8 @@ spec:
value: "clawmanager-system"
- name: RUNTIME_WORKSPACE_ROOT
value: "/workspaces"
- name: RUNTIME_WORKSPACE_NFS_SERVER
value: "workspace-store.clawmanager-system.svc.cluster.local"
- name: RUNTIME_WORKSPACE_NFS_PATH
value: "/"
- name: RUNTIME_WORKSPACE_PVC_CLAIM
value: "clawmanager-workspaces"
- name: RUNTIME_MAX_GATEWAYS_PER_POD
value: "100"
- name: RUNTIME_GATEWAY_PORT_START
@@ -1328,9 +1382,8 @@ spec:
secretName: clawmanager-tls
optional: true
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: apps/v1
kind: Deployment
@@ -1416,9 +1469,8 @@ spec:
mountPath: /workspaces
volumes:
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: apps/v1
kind: Deployment
@@ -1515,9 +1567,8 @@ spec:
mountPath: /workspaces
volumes:
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: v1
kind: Service
@@ -1532,7 +1583,7 @@ spec:
- name: https
port: 443
targetPort: https
nodePort: 39443
nodePort: 30443
---
apiVersion: v1
kind: Service
+201
View File
@@ -0,0 +1,201 @@
# ClawManager Kubernetes Single-Node Manifest
This directory contains the all-in-one Kubernetes manifest for a single-node
ClawManager deployment:
- `clawmanager.yaml`: ClawManager workloads plus static HostPath PVs.
- No Longhorn dependency.
- No dynamic StorageClass dependency.
Use this profile only when ClawManager workloads and user runtimes can run on
one storage node. For multi-node clusters, use `deployments/k8s/cluster` instead.
## Storage Model
The manifest uses static PV/PVC binding:
- StorageClass name: `manual`
- System HostPath root: `/data/clawmanager/system`
- Runtime HostPath root: `/data/clawreef`
- Storage node label: `clawmanager.io/storage-node=true`
The `manual` value is only a matching name between bundled PVs and PVCs. It does
not require a Kubernetes `StorageClass` object or dynamic provisioner.
The bundled PVs use `persistentVolumeReclaimPolicy: Retain`, so deleting the
manifest does not erase HostPath data.
## Install
### 1. Enter This Directory
```sh
cd deployments/k8s/single-node
```
Check that `kubectl` points to the target cluster:
```sh
kubectl config current-context
kubectl get nodes -o wide
```
Expected result: the target node is `Ready`.
### 2. Choose And Label One Storage Node
Label exactly one node before installing:
```sh
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl get nodes -l clawmanager.io/storage-node=true
```
Expected result: only one node is listed.
If more than one node has this label, remove the label from the extra nodes:
```sh
kubectl label node <node> clawmanager.io/storage-node-
```
### 3. Check For Old Installation State
```sh
kubectl get ns clawmanager-system clawmanager-user-1 --ignore-not-found
kubectl get pv | grep clawmanager || true
kubectl get pvc -A | grep clawmanager || true
```
Expected result for a fresh install: no old ClawManager namespace, PV, or PVC
remains.
If old resources exist, run the uninstall flow in this README before installing
again.
### 4. Check HostPath Directories
The kubelet can create the HostPath directories automatically, but checking the
target path first makes permission and disk issues easier to diagnose:
```sh
df -h /data || true
ls -ld /data || true
```
If `/data` is not the right disk for this node, edit these paths in
`clawmanager.yaml` before applying it:
- `/data/clawmanager/system/mysql`
- `/data/clawmanager/system/minio`
- `/data/clawmanager/system/redis`
- `/data/clawmanager/system/workspaces`
- `K8S_PV_HOST_PATH_PREFIX=/data/clawreef`
### 5. Check Image Availability
The manifest uses the images listed in `clawmanager.yaml`. Make sure the node
can pull from the referenced registry before installing:
```sh
grep -n 'image:' clawmanager.yaml | head -n 40
```
If this is an offline or private-registry deployment, load or push the images to
the registry used by the manifest before continuing.
### 6. Apply The Manifest
```sh
kubectl apply -f clawmanager.yaml
```
### 7. Check ClawManager
```sh
kubectl -n clawmanager-system get pvc
kubectl -n clawmanager-system rollout status deployment/clawmanager-app --timeout=15m
kubectl -n clawmanager-system rollout status deployment/openclaw-runtime --timeout=15m
kubectl -n clawmanager-system rollout status deployment/hermes-runtime --timeout=15m
kubectl -n clawmanager-system get deploy,pod,pvc -o wide
```
Expected result:
- `mysql-data`, `redis-data`, `minio-data`, and `clawmanager-workspaces` are `Bound`.
- `clawmanager-app`, `openclaw-runtime`, and `hermes-runtime` are available.
- MySQL, Redis, MinIO, and skill scanner pods are running.
If PVCs stay `Pending`, check the static PV binding:
```sh
kubectl get pv | grep clawmanager
kubectl -n clawmanager-system describe pvc <pvc-name>
```
If pods stay `Pending` with node affinity errors, check the storage node label:
```sh
kubectl get nodes -l clawmanager.io/storage-node=true
kubectl describe pod <pod-name> -n clawmanager-system
```
If pods stay `ContainerCreating`, check recent events:
```sh
kubectl -n clawmanager-system get events --sort-by=.lastTimestamp | tail -n 80
```
## Uninstall
Use this order for a disposable single-node test deployment.
### 1. Delete ClawManager Resources
```sh
kubectl delete -f clawmanager.yaml --ignore-not-found
kubectl delete namespace clawmanager-user-1 --ignore-not-found
kubectl delete namespace clawmanager-system --ignore-not-found
```
Check:
```sh
kubectl get ns clawmanager-system clawmanager-user-1 --ignore-not-found
kubectl get pv | grep clawmanager || true
kubectl get pvc -A | grep clawmanager || true
```
The PVs may remain because they use `Retain`. This is expected.
### 2. Remove Retained PVs When Data Is No Longer Needed
Only run this after confirming the data can be discarded:
```sh
kubectl delete pv \
clawmanager-mysql-pv \
clawmanager-minio-pv \
clawmanager-redis-pv \
clawmanager-workspaces-pv \
--ignore-not-found
```
### 3. Remove HostPath Data When Data Is No Longer Needed
Run this on the labeled storage node only after confirming the data can be
discarded:
```sh
rm -rf /data/clawmanager/system /data/clawreef
```
## Common Issues
| Symptom | Likely Cause | Check Or Fix |
| :--- | :--- | :--- |
| PVC stays `Pending` | Static PV and PVC did not bind | `kubectl get pv`, then `kubectl describe pvc -n clawmanager-system <pvc-name>` |
| Pod stays `Pending` | Missing or wrong `clawmanager.io/storage-node=true` label | `kubectl get nodes -l clawmanager.io/storage-node=true` |
| Pod is `ImagePullBackOff` | Node cannot pull an image | `kubectl describe pod <pod-name> -n <namespace>` |
| Pod is `ContainerCreating` | HostPath mount or permission issue | `kubectl -n clawmanager-system get events --sort-by=.lastTimestamp` |
| Runtime creation fails | Runtime HostPath root is unavailable | Check `K8S_PV_HOST_PATH_PREFIX` and disk space on the labeled node |
@@ -1,3 +1,8 @@
# ClawManager single-node install profile.
# Official validation target: HostPath on one labeled storage node.
# Before installing, label exactly one node:
# kubectl label node <node> clawmanager.io/storage-node=true --overwrite
# Replace paths or StorageClass names only if you understand the single-node constraints.
apiVersion: v1
kind: Namespace
metadata:
@@ -16,6 +21,10 @@ stringData:
runtime-agent-control-token: change-me-runtime-control-token
runtime-agent-report-token: change-me-runtime-report-token
openclaw-gateway-token: change-me-openclaw-gateway-token
minio-root-user: minioadmin
minio-root-password: minioadmin123
minio-access-key: minioadmin
minio-secret-key: minioadmin123
---
apiVersion: v1
kind: ConfigMap
@@ -709,6 +718,35 @@ data:
);
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-mysql-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/mysql
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-data
@@ -719,7 +757,137 @@ spec:
resources:
requests:
storage: 5Gi
storageClassName: local-path
storageClassName: manual
volumeName: clawmanager-mysql-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-minio-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 10Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/minio
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: minio-data
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: manual
volumeName: clawmanager-minio-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-redis-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/redis
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-data
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: manual
volumeName: clawmanager-redis-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: clawmanager-workspaces-pv
labels:
app: clawmanager
managed-by: clawmanager
clawmanager.io/storage-profile: single-node
clawmanager.io/storage-node: "true"
spec:
capacity:
storage: 50Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: clawmanager.io/storage-node
operator: In
values:
- "true"
hostPath:
path: /data/clawmanager/system/workspaces
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: clawmanager-workspaces
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
storageClassName: manual
volumeName: clawmanager-workspaces-pv
---
apiVersion: apps/v1
kind: Deployment
@@ -740,6 +908,8 @@ spec:
- name: mysql
image: mysql:8.4.8
imagePullPolicy: IfNotPresent
args:
- "--innodb-use-native-aio=0"
ports:
- containerPort: 3306
env:
@@ -790,6 +960,75 @@ spec:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: minio
namespace: clawmanager-system
spec:
replicas: 1
selector:
matchLabels:
app: minio
template:
metadata:
labels:
app: minio
spec:
containers:
- name: minio
image: minio/minio:latest
imagePullPolicy: IfNotPresent
args:
- server
- /data
- --console-address
- ":9001"
env:
- name: MINIO_ROOT_USER
valueFrom:
secretKeyRef:
name: clawmanager-secrets
key: minio-root-user
- name: MINIO_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: clawmanager-secrets
key: minio-root-password
ports:
- name: api
containerPort: 9000
- name: console
containerPort: 9001
volumeMounts:
- name: minio-data
mountPath: /data
readinessProbe:
tcpSocket:
port: api
initialDelaySeconds: 10
periodSeconds: 10
volumes:
- name: minio-data
persistentVolumeClaim:
claimName: minio-data
---
apiVersion: v1
kind: Service
metadata:
name: minio
namespace: clawmanager-system
spec:
selector:
app: minio
ports:
- name: api
port: 9000
targetPort: api
- name: console
port: 9001
targetPort: console
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: clawmanager-team-redis
namespace: clawmanager-system
@@ -824,7 +1063,8 @@ spec:
mountPath: /data
volumes:
- name: redis-data
emptyDir: {}
persistentVolumeClaim:
claimName: redis-data
---
apiVersion: v1
kind: Service
@@ -853,67 +1093,6 @@ spec:
targetPort: redis
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: clawmanager-workspaces
namespace: clawmanager-system
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
storageClassName: local-path
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: workspace-store
namespace: clawmanager-system
spec:
replicas: 1
selector:
matchLabels:
app: workspace-store
template:
metadata:
labels:
app: workspace-store
spec:
containers:
- name: nfs-server
image: itsthenetwork/nfs-server-alpine:12
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
env:
- name: SHARED_DIRECTORY
value: /exports/workspaces
ports:
- name: nfs
containerPort: 2049
volumeMounts:
- name: workspaces
mountPath: /exports/workspaces
volumes:
- name: workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: v1
kind: Service
metadata:
name: workspace-store
namespace: clawmanager-system
spec:
selector:
app: workspace-store
ports:
- name: nfs
port: 2049
targetPort: nfs
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: clawmanager-app
@@ -981,11 +1160,66 @@ subjects:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: skill-scanner
namespace: clawmanager-system
spec:
replicas: 1
selector:
matchLabels:
app: skill-scanner
template:
metadata:
labels:
app: skill-scanner
spec:
containers:
- name: skill-scanner
image: ghcr.io/yuan-lab-llm/skill-scanner:latest
imagePullPolicy: IfNotPresent
command:
- /opt/skill-scanner-venv/bin/skill-scanner-api
- --host
- 0.0.0.0
- --port
- "8000"
env:
- name: SKILL_SCANNER_LLM_API_KEY
value: ""
- name: SKILL_SCANNER_LLM_MODEL
value: ""
- name: SKILL_SCANNER_LLM_BASE_URL
value: ""
- name: SKILL_SCANNER_META_LLM_API_KEY
value: ""
- name: SKILL_SCANNER_META_LLM_MODEL
value: ""
- name: SKILL_SCANNER_META_LLM_BASE_URL
value: ""
ports:
- name: http
containerPort: 8000
---
apiVersion: v1
kind: Service
metadata:
name: skill-scanner
namespace: clawmanager-system
spec:
selector:
app: skill-scanner
ports:
- name: http
port: 8000
targetPort: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: clawmanager-app
namespace: clawmanager-system
spec:
replicas: 3
replicas: 1
selector:
matchLabels:
app: clawmanager-app
@@ -1027,12 +1261,50 @@ spec:
secretKeyRef:
name: clawmanager-secrets
key: jwt-secret
- name: OBJECT_STORAGE_ENDPOINT
value: "minio.clawmanager-system.svc.cluster.local:9000"
- name: OBJECT_STORAGE_REGION
value: ""
- name: OBJECT_STORAGE_ACCESS_KEY
valueFrom:
secretKeyRef:
name: clawmanager-secrets
key: minio-access-key
- name: OBJECT_STORAGE_SECRET_KEY
valueFrom:
secretKeyRef:
name: clawmanager-secrets
key: minio-secret-key
- name: OBJECT_STORAGE_BUCKET
value: "clawmanager-skills"
- name: OBJECT_STORAGE_USE_SSL
value: "false"
- name: OBJECT_STORAGE_BASE_PATH
value: "skills"
- name: OBJECT_STORAGE_FORCE_PATH_STYLE
value: "true"
- name: K8S_MODE
value: "incluster"
- name: K8S_NAMESPACE
value: "clawmanager"
- name: K8S_STORAGE_CLASS
value: "local-path"
value: "manual"
- name: CLAWMANAGER_STORAGE_PROFILE
value: "single-node"
- name: K8S_HOSTPATH_FALLBACK_ENABLED
value: "true"
- name: K8S_PVC_BIND_TIMEOUT
value: "2m"
- name: K8S_CONTROL_PLANE_STORAGE_CLASS
value: "manual"
- name: K8S_INSTANCE_STORAGE_CLASS
value: "manual"
- name: K8S_WORKSPACE_STORAGE_CLASS
value: "manual"
- name: K8S_WORKSPACE_ACCESS_MODE
value: "ReadWriteMany"
- name: K8S_PV_HOST_PATH_PREFIX
value: "/data/clawreef"
- name: CLAWMANAGER_WORKSPACE_ARCHIVE_MAX_MIB
value: "500"
- name: PLATFORM_REDIS_URL
@@ -1047,10 +1319,8 @@ spec:
value: "clawmanager-system"
- name: RUNTIME_WORKSPACE_ROOT
value: "/workspaces"
- name: RUNTIME_WORKSPACE_NFS_SERVER
value: "workspace-store.clawmanager-system.svc.cluster.local"
- name: RUNTIME_WORKSPACE_NFS_PATH
value: "/"
- name: RUNTIME_WORKSPACE_PVC_CLAIM
value: "clawmanager-workspaces"
- name: RUNTIME_MAX_GATEWAYS_PER_POD
value: "100"
- name: RUNTIME_GATEWAY_PORT_START
@@ -1086,43 +1356,34 @@ spec:
value: "clawmanager-system"
- name: SKILL_SCANNER_DEPLOYMENT
value: "skill-scanner"
- name: OBJECT_STORAGE_LOCAL_FALLBACK
value: "/data/object-storage"
volumeMounts:
- name: tls-cert
mountPath: /var/run/clawreef-tls
readOnly: true
- name: object-storage
mountPath: /data/object-storage
- name: workspaces
mountPath: /workspaces
readinessProbe:
httpGet:
scheme: HTTPS
path: /healthz
port: https
scheme: HTTPS
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
periodSeconds: 10
livenessProbe:
httpGet:
scheme: HTTPS
path: /healthz
port: https
scheme: HTTPS
initialDelaySeconds: 30
periodSeconds: 20
timeoutSeconds: 5
volumes:
- name: tls-cert
secret:
secretName: clawmanager-tls
optional: true
- name: object-storage
emptyDir: {}
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: apps/v1
kind: Deployment
@@ -1208,9 +1469,8 @@ spec:
mountPath: /workspaces
volumes:
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: apps/v1
kind: Deployment
@@ -1307,9 +1567,8 @@ spec:
mountPath: /workspaces
volumes:
- name: workspaces
nfs:
server: workspace-store.clawmanager-system.svc.cluster.local
path: /exports/workspaces
persistentVolumeClaim:
claimName: clawmanager-workspaces
---
apiVersion: v1
kind: Service
@@ -1324,7 +1583,7 @@ spec:
- name: https
port: 443
targetPort: https
nodePort: 39443
nodePort: 30443
---
apiVersion: v1
kind: Service
@@ -1353,58 +1612,3 @@ spec:
- name: proxy
port: 3128
targetPort: 9001
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: skill-scanner
namespace: clawmanager-system
spec:
replicas: 1
selector:
matchLabels:
app: skill-scanner
template:
metadata:
labels:
app: skill-scanner
spec:
containers:
- name: skill-scanner
image: ghcr.io/yuan-lab-llm/skill-scanner:latest
imagePullPolicy: IfNotPresent
command:
- /opt/skill-scanner-venv/bin/skill-scanner-api
- --host
- 0.0.0.0
- --port
- "8000"
env:
- name: SKILL_SCANNER_LLM_API_KEY
value: ""
- name: SKILL_SCANNER_LLM_MODEL
value: ""
- name: SKILL_SCANNER_LLM_BASE_URL
value: ""
- name: SKILL_SCANNER_META_LLM_API_KEY
value: ""
- name: SKILL_SCANNER_META_LLM_MODEL
value: ""
- name: SKILL_SCANNER_META_LLM_BASE_URL
value: ""
ports:
- name: http
containerPort: 8000
---
apiVersion: v1
kind: Service
metadata:
name: skill-scanner
namespace: clawmanager-system
spec:
selector:
app: skill-scanner
ports:
- name: http
port: 8000
targetPort: http
+68 -10
View File
@@ -6,10 +6,14 @@ ClawManager is packaged as a Kubernetes-first platform. This guide is the operat
Choose the deployment path that matches your environment:
- Standard Kubernetes: [`deployments/k8s/clawmanager.yaml`](../deployments/k8s/clawmanager.yaml)
- K3s or lightweight clusters: [`deployments/k3s/clawmanager.yaml`](../deployments/k3s/clawmanager.yaml)
- K3s single-node HostPath: [`deployments/k3s/single-node/clawmanager.yaml`](../deployments/k3s/single-node/clawmanager.yaml)
- K3s multi-node CSI/RWX: [`deployments/k3s/cluster/clawmanager.yaml`](../deployments/k3s/cluster/clawmanager.yaml)
- Kubernetes single-node HostPath: [`deployments/k8s/single-node/clawmanager.yaml`](../deployments/k8s/single-node/clawmanager.yaml)
- Kubernetes multi-node CSI/RWX: [`deployments/k8s/cluster/clawmanager.yaml`](../deployments/k8s/cluster/clawmanager.yaml)
- End-to-end first-use walkthrough: [User Guide](./use_guide_en.md)
The cluster profile is validated with Longhorn as the example CSI implementation. It uses `longhorn` for RWO control-plane and instance volumes, and `longhorn-rwx` for the RWX workspace volume. These names are not a project requirement; replace them with compatible StorageClasses for your storage provider.
## What Gets Deployed
- ClawManager frontend and backend
@@ -20,25 +24,79 @@ Choose the deployment path that matches your environment:
## Repository Entry Points
- Kubernetes manifest: [`deployments/k8s/clawmanager.yaml`](../deployments/k8s/clawmanager.yaml)
- K3s manifest: [`deployments/k3s/clawmanager.yaml`](../deployments/k3s/clawmanager.yaml)
- Kubernetes single-node manifest: [`deployments/k8s/single-node/clawmanager.yaml`](../deployments/k8s/single-node/clawmanager.yaml)
- Kubernetes cluster manifest: [`deployments/k8s/cluster/clawmanager.yaml`](../deployments/k8s/cluster/clawmanager.yaml)
- K3s single-node manifest: [`deployments/k3s/single-node/clawmanager.yaml`](../deployments/k3s/single-node/clawmanager.yaml)
- K3s cluster manifest: [`deployments/k3s/cluster/clawmanager.yaml`](../deployments/k3s/cluster/clawmanager.yaml)
- Container startup script: [`deployments/container/start.sh`](../deployments/container/start.sh)
- Nginx config: [`deployments/nginx/nginx.conf`](../deployments/nginx/nginx.conf)
## Deployment Workflow
1. Choose the deployment path: standard Kubernetes or K3s/lightweight.
2. Prepare the cluster, storage strategy, and image source strategy for that environment.
3. Review the bundled manifest and adjust secrets, images, storage classes, and ingress exposure for your environment.
4. Deploy the platform components into the cluster.
5. Wait for the core services to become ready.
6. Validate frontend access, AI Gateway management pages, Security Center connectivity, and runtime creation flows.
1. Choose the Kubernetes distribution: `k3s` or `k8s`.
2. Choose the storage profile: `single-node` or `cluster`.
3. Check the storage prerequisites for that profile.
4. Review the bundled manifest and adjust secrets, images, StorageClass names, and ingress exposure for your environment.
5. Deploy the platform components into the cluster.
6. Wait for the core services to become ready.
7. Validate frontend access, AI Gateway management pages, Security Center connectivity, and runtime creation flows.
Single-node example:
```bash
kubectl get nodes
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl apply -f deployments/k8s/single-node/clawmanager.yaml
kubectl get pvc -n clawmanager-system
kubectl get pods -n clawmanager-system
```
Cluster example:
```bash
kubectl get storageclass longhorn longhorn-rwx
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
kubectl get pvc -n clawmanager-system
kubectl get pods -n clawmanager-system
```
## Storage Profiles
### Single-Node
The `single-node` profile is the official HostPath validation path. Label exactly one node with `clawmanager.io/storage-node=true` before installation. The manifest pins HostPath PVs through node affinity and runs `clawmanager-app` as a single replica. MySQL, Redis, MinIO, workspace, and runtime data are all backed by persistent volumes; durable data must not use `emptyDir`.
### Cluster
The `cluster` profile is the official multi-node CSI/RWX validation path. The bundled manifest uses Longhorn as an example only:
- `longhorn`: RWO MySQL, Redis, MinIO, and instance volumes
- `longhorn-rwx`: RWX workspace volume shared by ClawManager and runtime Pods
Set these environment variables in the ClawManager app deployment when replacing the storage provider:
- `CLAWMANAGER_STORAGE_PROFILE=cluster`
- `K8S_HOSTPATH_FALLBACK_ENABLED=false`
- `K8S_PVC_BIND_TIMEOUT=2m`
- `K8S_CONTROL_PLANE_STORAGE_CLASS=<rwo-storage-class>`
- `K8S_INSTANCE_STORAGE_CLASS=<rwo-storage-class>`
- `K8S_WORKSPACE_STORAGE_CLASS=<rwx-storage-class>`
- `K8S_WORKSPACE_ACCESS_MODE=ReadWriteMany`
Unsupported combinations:
- multi-node HostPath as a production or shared workspace strategy
- `local-path` or other node-local storage pretending to provide RWX across nodes
- cluster-internal Service DNS such as `workspace-store.clawmanager-system.svc.cluster.local` as an NFS server for kubelet-mounted PVs
- durable MySQL, Redis, MinIO, workspace, or object data on `emptyDir`
- cluster profile with implicit HostPath fallback
## Operational Notes
- ClawManager is designed around in-cluster services and platform-mediated access rather than direct pod exposure.
- Resource Management features depend on object storage and `skill-scanner` being available.
- Runtime workspace `.openclaw` and `.hermes` archive import/export size is controlled by `CLAWMANAGER_WORKSPACE_ARCHIVE_MAX_MIB`. The default is `500` MiB; set the env var on the ClawManager app deployment when a larger or smaller limit is needed.
- For install issues, collect `kubectl get storageclass`, `kubectl get pvc -n clawmanager-system`, `kubectl get pods -n clawmanager-system`, `kubectl get events -n clawmanager-system --sort-by=.lastTimestamp`, and `kubectl describe pvc -n clawmanager-system <pvc-name>` output before filing an issue.
- Production environments should review images, credentials, TLS, persistence, and networking policies before rollout.
## Related Guides
+59 -96
View File
@@ -29,18 +29,22 @@
<a id="sec-02"></a>
## 二、部署方式总览
你可以按以下两种方式之一部署
先选择 Kubernetes 发行版,再选择存储 profile
### 方案 Ak3s 部署
适合单机、测试环境或轻量生产环境
- `k3s/single-node`:单节点 HostPath 官方验证路径,需要标记一个存储节点。
- `k3s/cluster`:多节点 CSI/RWX 路径,默认使用 Longhorn 示例
- `k8s/single-node`:标准 Kubernetes 单节点 HostPath 官方验证路径。
- `k8s/cluster`:标准 Kubernetes 多节点 CSI/RWX 路径,默认使用 Longhorn 示例。
### 方案 B:标准 Kubernetes 部署
适合已经具备标准 Kubernetes 集群的服务器环境。
Longhorn 是官方验证示例,不是强绑定依赖。只要你的存储方案能提供相同的 RWO/RWX 能力,就可以替换 `longhorn``longhorn-rwx`
无论使用哪种方式,最终都会执行同一套 ClawManager 清单
推荐入口
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
deployments/k3s/single-node/clawmanager.yaml
deployments/k3s/cluster/clawmanager.yaml
deployments/k8s/single-node/clawmanager.yaml
deployments/k8s/cluster/clawmanager.yaml
```
---
@@ -105,19 +109,19 @@ kubectl get ns
正常应看到至少 1 个 `Ready` 节点。
### 4.2 检查默认 StorageClass
ClawManager 中的 MySQL、MinIO 需要持久化存储。建议先检查集群是否有默认 `StorageClass`
### 4.2 检查 StorageClass 能力
ClawManager 中的 MySQL、Redis、MinIO、workspace 和实例 PVC 都需要持久化存储。cluster profile 需要一个 RWO StorageClass 和一个 RWX workspace StorageClass
```bash
kubectl get storageclass
```
如果集群中已有默认存储类,可直接继续部署
默认 cluster 清单使用 `longhorn` 作为 RWO 示例,使用 `longhorn-rwx` 作为 RWX workspace 示例。如果你使用其他 CSI 存储,请替换这些 StorageClass 名称。多节点场景不要使用 `local-path` 这类节点本地存储来冒充 RWX workspace
如果 **没有默认 StorageClass**,建议提前准备好可用的 PV / PVC 或使用本地路径存储方案,否则后续可能出现
部署前先确认两个 StorageClass 都存在
```text
pod has unbound immediate PersistentVolumeClaims
```bash
kubectl get storageclass longhorn longhorn-rwx
```
---
@@ -168,10 +172,26 @@ cd ClawManager
```
### 6.2 应用部署清单
在仓库根目录执行:
在仓库根目录按场景选择一个入口执行:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
# k3s 单节点
kubectl get nodes
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl apply -f deployments/k3s/single-node/clawmanager.yaml
# k3s 多节点
kubectl get storageclass longhorn longhorn-rwx
kubectl apply -f deployments/k3s/cluster/clawmanager.yaml
# 标准 Kubernetes 单节点
kubectl get nodes
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl apply -f deployments/k8s/single-node/clawmanager.yaml
# 标准 Kubernetes 多节点
kubectl get storageclass longhorn longhorn-rwx
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
### 6.3 查看基础资源
@@ -193,9 +213,14 @@ kubectl get svc -n clawmanager-system
0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims
```
说明集群存储 MySQL / MinIO 因 PVC 未绑定而无法启动,请直接跳转文末
说明当前选择的存储 profile 未准备好。先收集事件,再检查 PVC 和 StorageClass
- [十一.1 存储问题专项处理(PV/PVC)](#sec-14-storage)
```bash
kubectl get pvc -n clawmanager-system
kubectl get pods -n clawmanager-system
kubectl get events -n clawmanager-system --sort-by=.lastTimestamp
kubectl describe pvc -n clawmanager-system clawmanager-workspaces
```
---
@@ -672,92 +697,30 @@ AI 审计页面用于查看最近的受管模型调用记录,帮助管理员
0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims
```
说明集群存储未自动绑定。此时可以按单机 x86 服务器方式,手动创建本地 `hostPath` PV/PVC。
说明当前选择的存储 profile 未准备好。不要在多节点集群里临时补 HostPath PV;请回到两条官方验证路径之一:
> 这一方案适合单节点服务器测试或轻量环境;生产环境建议改为 NFS、Ceph、云盘等正式存储
- 单节点:给唯一存储节点打 `clawmanager.io/storage-node=true` 标签,再应用 `deployments/<k3s|k8s>/single-node/clawmanager.yaml`
- 多节点:确认 RWO 和 RWX StorageClass 存在,再应用 `deployments/<k3s|k8s>/cluster/clawmanager.yaml`
不支持的组合:
- 多节点 HostPath
- 多节点场景使用 `local-path` 作为 RWX workspace
- 使用 `workspace-store.clawmanager-system.svc.cluster.local` 这类集群内 Service DNS 作为 kubelet 挂载的 NFS server
- MySQL、Redis、MinIO、workspace 或 object 这类持久数据落到 `emptyDir`
#### 11.1.1 收集诊断信息
#### 11.1.1 创建 PV
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysql-pv-local
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
hostPath:
path: /tmp/mysql-data
EOF
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
name: minio-pv-local
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
hostPath:
path: /tmp/minio-data
EOF
```
#### 11.1.2 创建 PVC
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-data
namespace: clawmanager-system
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
volumeName: mysql-pv-local
EOF
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: minio-data
namespace: clawmanager-system
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
volumeName: minio-pv-local
EOF
```
#### 11.1.3 重建 Pod
```bash
kubectl delete pod --all -n clawmanager-system
```
#### 11.1.4 重新观察状态
```bash
kubectl get storageclass
kubectl get pvc -n clawmanager-system
kubectl get events -n clawmanager-system --sort-by=.lastTimestamp
kubectl get pods -n clawmanager-system -w
```
预期应看到:
- `mysql-data` / `minio-data``Bound`
- `mysql` / `minio` / `skill-scanner` / `clawmanager-app` 最终为 `Running`
- `mysql-data` / `redis-data` / `minio-data` / `clawmanager-workspaces``Bound`
- `mysql` / `clawmanager-team-redis` / `minio` / `skill-scanner` / `clawmanager-app` 最终为 `Running`
---
@@ -765,7 +728,7 @@ kubectl get pods -n clawmanager-system -w
| :--- | :--- | :--- |
| `kubectl` 连接 `localhost:8080` 被拒绝 | kubeconfig 未配置 | 设置 `KUBECONFIG` 或复制到 `~/.kube/config` |
| Pod 拉镜像超时 | 网络到 Docker Hub / GHCR 不稳定 | 配置镜像加速或代理 |
| MySQL / MinIO 一直 `Pending` | PVC 未绑定 | 检查 `StorageClass` 或手动创建 PV/PVC |
| MySQL / Redis / MinIO 一直 `Pending` | PVC 未绑定 | 检查 StorageClass、PVC 状态和 PVC 事件 |
| 浏览器打不开页面 | NodePort 未放通 / `port-forward` 进程未保持 | 放行端口或保持转发终端运行 |
| 页面能打开但无法创建 OpenClaw 实例 | 未配置安全模型 | 先在 **AI 网关 → 模型** 中配置并启用安全模型 |
| 实例长时间“创建中” | 首次拉镜像耗时长 / 存储或网络问题 | 耐心等待,必要时检查 Pod 和事件 |
+2 -2
View File
@@ -40,7 +40,7 @@ Geeignet für Serverumgebungen, die bereits über einen Standard-Kubernetes-Clus
Unabhängig davon, welche Methode Sie verwenden, wird am Ende dasselbe ClawManager-Manifest angewendet:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
---
@@ -170,7 +170,7 @@ cd ClawManager
Führen Sie im Wurzelverzeichnis des Repositorys aus:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
### 6.3 Basisressourcen prüfen
+59 -96
View File
@@ -29,18 +29,22 @@
<a id="sec-02"></a>
## II. Deployment Options Overview
You can deploy using either of the following methods:
Choose a Kubernetes distribution and a storage profile:
### Option A: k3s deployment
Suitable for single-node, test, or lightweight production environments.
- `k3s/single-node`: HostPath validation path for one labeled node.
- `k3s/cluster`: multi-node CSI/RWX path, validated with Longhorn examples.
- `k8s/single-node`: HostPath validation path for one labeled node.
- `k8s/cluster`: multi-node CSI/RWX path, validated with Longhorn examples.
### Option B: standard Kubernetes deployment
Suitable for server environments that already have a standard Kubernetes cluster.
Longhorn is the official cluster validation example, not a hard dependency. You may replace `longhorn` and `longhorn-rwx` with StorageClasses from another CSI provider if they provide the same RWO/RWX behavior.
No matter which method you use, you will ultimately apply the same ClawManager manifest:
Recommended manifests:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
deployments/k3s/single-node/clawmanager.yaml
deployments/k3s/cluster/clawmanager.yaml
deployments/k8s/single-node/clawmanager.yaml
deployments/k8s/cluster/clawmanager.yaml
```
---
@@ -104,19 +108,19 @@ kubectl get ns
Normally, you should see at least one `Ready` node.
### 4.2 Check the default StorageClass
MySQL and MinIO in ClawManager require persistent storage. It is recommended to first check whether the cluster has a default `StorageClass`:
### 4.2 Check StorageClass capabilities
MySQL, Redis, MinIO, workspaces, and runtime PVCs require persistent storage. For the cluster profile, prepare one RWO StorageClass and one RWX workspace StorageClass:
```bash
kubectl get storageclass
```
If the cluster already has a default storage class, you can continue with deployment directly.
The bundled cluster manifests use `longhorn` for RWO volumes and `longhorn-rwx` for RWX workspaces. Replace those names if your cluster uses another CSI provider. Do not use node-local storage such as `local-path` as the RWX workspace class in a multi-node cluster.
If there is **no default StorageClass**, it is recommended to prepare available PV / PVC resources or use a local path storage solution in advance; otherwise, you may later encounter:
Before applying the cluster manifest, confirm that both StorageClasses exist:
```text
pod has unbound immediate PersistentVolumeClaims
```bash
kubectl get storageclass longhorn longhorn-rwx
```
---
@@ -167,10 +171,26 @@ cd ClawManager
```
### 6.2 Apply the deployment manifest
Run in the repository root directory:
Run one of the following commands from the repository root:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
# k3s single node
kubectl get nodes
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl apply -f deployments/k3s/single-node/clawmanager.yaml
# k3s multi-node cluster
kubectl get storageclass longhorn longhorn-rwx
kubectl apply -f deployments/k3s/cluster/clawmanager.yaml
# standard Kubernetes single node
kubectl get nodes
kubectl label node <node> clawmanager.io/storage-node=true --overwrite
kubectl apply -f deployments/k8s/single-node/clawmanager.yaml
# standard Kubernetes multi-node cluster
kubectl get storageclass longhorn longhorn-rwx
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
### 6.3 Check base resources
@@ -192,9 +212,14 @@ If you see the following error:
0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims
```
it means MySQL / MinIO in cluster storage cannot start because the PVC is not bound. Please jump directly to the end of this document:
it means persistent storage cannot bind. Collect events and verify the selected storage profile:
- [XI.1 Dedicated Handling for Storage Issues (PV/PVC)](#sec-14-storage)
```bash
kubectl get pvc -n clawmanager-system
kubectl get pods -n clawmanager-system
kubectl get events -n clawmanager-system --sort-by=.lastTimestamp
kubectl describe pvc -n clawmanager-system clawmanager-workspaces
```
---
@@ -671,92 +696,30 @@ If you see the following error:
0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims
```
it means the cluster storage was not bound automatically. In this case, you can manually create local `hostPath` PV/PVC in the x86 single-node server style.
it means the selected storage profile is not ready. Do not patch a multi-node cluster with ad hoc HostPath PVs. Use one of the validated paths:
> This solution is suitable for single-node server testing or lightweight environments. For production environments, it is recommended to use formal storage such as NFS, Ceph, or cloud disks instead.
- Single-node: label one node with `clawmanager.io/storage-node=true`, then apply `deployments/<k3s|k8s>/single-node/clawmanager.yaml`.
- Cluster: ensure the RWO and RWX StorageClasses exist, then apply `deployments/<k3s|k8s>/cluster/clawmanager.yaml`.
Unsupported combinations:
- multi-node HostPath
- `local-path` RWX workspace in a multi-node cluster
- cluster-internal Service DNS such as `workspace-store.clawmanager-system.svc.cluster.local` as an NFS server
- durable MySQL, Redis, MinIO, workspace, or object data on `emptyDir`
#### 11.1.1 Collect diagnostics
#### 11.1.1 Create PV
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysql-pv-local
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
hostPath:
path: /tmp/mysql-data
EOF
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
name: minio-pv-local
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
hostPath:
path: /tmp/minio-data
EOF
```
#### 11.1.2 Create PVC
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-data
namespace: clawmanager-system
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
volumeName: mysql-pv-local
EOF
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: minio-data
namespace: clawmanager-system
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
volumeName: minio-pv-local
EOF
```
#### 11.1.3 Recreate Pod
```bash
kubectl delete pod --all -n clawmanager-system
```
#### 11.1.4 Observe status again
```bash
kubectl get storageclass
kubectl get pvc -n clawmanager-system
kubectl get events -n clawmanager-system --sort-by=.lastTimestamp
kubectl get pods -n clawmanager-system -w
```
Expected results:
- `mysql-data` / `minio-data` are `Bound`
- `mysql` / `minio` / `skill-scanner` / `clawmanager-app` are finally `Running`
- `mysql-data` / `redis-data` / `minio-data` / `clawmanager-workspaces` are `Bound`
- `mysql` / `clawmanager-team-redis` / `minio` / `skill-scanner` / `clawmanager-app` are finally `Running`
---
@@ -764,7 +727,7 @@ Expected results:
| :--- | :--- | :--- |
| `kubectl` connection to `localhost:8080` is refused | kubeconfig is not configured | Set `KUBECONFIG` or copy it to `~/.kube/config` |
| Pod image pull timeout | Network to Docker Hub / GHCR is unstable | Configure image acceleration or a proxy |
| MySQL / MinIO remain `Pending` | PVC not bound | Check the `StorageClass` or manually create PV/PVC |
| MySQL / Redis / MinIO remain `Pending` | PVC not bound | Inspect StorageClass, PVC status, and PVC events |
| The browser cannot open the page | NodePort is not open / the `port-forward` process was not kept running | Open the port or keep the forwarding terminal running |
| The page opens but an OpenClaw instance cannot be created | Secure model is not configured | First configure and enable the secure model under **AI Gateway → Models** |
| The instance remains “Creating” for a long time | The first image pull takes a long time / storage or network issues | Wait patiently, and if necessary check Pods and events |
+2 -2
View File
@@ -40,7 +40,7 @@
どの方式を使用しても、最終的には同じ ClawManager マニフェストを適用します:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
---
@@ -170,7 +170,7 @@ cd ClawManager
リポジトリのルートディレクトリで実行します:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
### 6.3 基本リソースの確認
+2 -2
View File
@@ -40,7 +40,7 @@
어떤 방식을 사용하든 최종적으로 동일한 ClawManager 매니페스트를 적용합니다:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
---
@@ -170,7 +170,7 @@ cd ClawManager
저장소 루트 디렉터리에서 실행합니다:
```bash
kubectl apply -f deployments/k8s/clawmanager.yaml
kubectl apply -f deployments/k8s/cluster/clawmanager.yaml
```
### 6.3 기본 리소스 확인