修复 Native 观测性兼容并补齐验证流程 #2
@@ -40,3 +40,5 @@ out/
|
||||
.env.*
|
||||
!.env.example
|
||||
*.log
|
||||
|
||||
__pycache__/
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# Agent Notes
|
||||
|
||||
- 后续修改必须新建分支并提交 PR;未经维护者明确指示,不直接推 main、不自行合并。
|
||||
- 默认中文维护项目文档、commit、issue 和 PR;代码与上游 API 名称保留英文。
|
||||
- 项目使用 Java 与 Spring;Native 是交付约束,不是可选优化,不引入 Kotlin。
|
||||
- 变更需通过适用的 JVM 测试和 Native 集成测试;未执行的验证明确报告。
|
||||
|
||||
@@ -46,7 +46,7 @@ Native 构建与原生二进制上的认证测试是交付要求;JVM 测试通
|
||||
[项目初始化](docs/bootstrap.md)。使用 JDK 25 执行:
|
||||
|
||||
```sh
|
||||
./gradlew test
|
||||
./gradlew test testAot
|
||||
./gradlew bootRun
|
||||
```
|
||||
|
||||
@@ -59,4 +59,6 @@ Native 构建与原生二进制上的认证测试是交付要求;JVM 测试通
|
||||
./gradlew nativeCompile
|
||||
```
|
||||
|
||||
这些是构建入口,不代表完整认证链路已通过 Native 验收。
|
||||
Docker 开发使用 `scripts/gradle-in-docker`,默认持久挂载 Gradle 缓存。
|
||||
原生应用可用 `python3 scripts/native-smoke.py` 检查启动、默认访问控制和 HTTP 指标。
|
||||
这些检查不代表完整认证链路已通过 Native 验收。
|
||||
|
||||
@@ -51,3 +51,23 @@ dependencies {
|
||||
tasks.named('test') {
|
||||
useJUnitPlatform()
|
||||
}
|
||||
|
||||
// Run generated AOT test context on the JVM before the more expensive native compile.
|
||||
tasks.register('testAot', Test) {
|
||||
dependsOn tasks.named('aotTestClasses')
|
||||
testClassesDirs = sourceSets.test.output.classesDirs
|
||||
classpath = sourceSets.test.runtimeClasspath + sourceSets.aotTest.output
|
||||
systemProperty 'spring.aot.enabled', 'true'
|
||||
useJUnitPlatform()
|
||||
shouldRunAfter tasks.named('test')
|
||||
}
|
||||
|
||||
// Native tests check behavior, not throughput. Keep the application binary optimized.
|
||||
graalvmNative {
|
||||
binaries {
|
||||
test {
|
||||
quickBuild = true
|
||||
runtimeArgs.add(providers.systemProperty('user.home').map { "-Duser.home=${it}" })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+57
-20
@@ -32,12 +32,10 @@ Actuator 与 Prometheus 依赖存在不等于监控端点已按生产策略开
|
||||
构建环境为 `ghcr.io/graalvm/native-image-community:25`,固定 digest:
|
||||
`sha256:0d936f32bb8acb5bc60c41b33e05f064d7a6aaf36b726538296c54949bd4a3c0`。
|
||||
|
||||
2026-09-25 在上述 GraalVM 容器(Java 25.0.2)中执行 `./gradlew --no-daemon --max-workers=4 test`
|
||||
通过,包含测试 AOT 处理和 1 个 JVM 上下文测试。Grafana LGTM Testcontainer 实际启动成功。
|
||||
当前 Gradle 骨架的 `nativeCompile` 与 `nativeTest` 尚未执行,不声明 Native 已通过。
|
||||
|
||||
生成的上下文测试通过 Testcontainers 启动 Grafana LGTM,需访问 Docker。
|
||||
它不连接真实 AD 或 Hydra,也不验证 MFA。Native 骨架测试不代替完整认证链路验收。
|
||||
验证分为 `test`(JVM)、`testAot`(JVM 上的 AOT 上下文)、`nativeTest`(原生测试)
|
||||
和应用二进制 HTTP 检查。JVM/AOT/Native 测试均使用真实 LGTM Testcontainer,
|
||||
显式初始化指标、追踪导出器,并验证容器连接与 CPU 时间读取。结果随 PR 记录。
|
||||
这些检查不连接真实 AD 或 Hydra,也不验证 MFA。
|
||||
|
||||
## Docker 开发与 Gradle 缓存
|
||||
|
||||
@@ -45,23 +43,62 @@ Linux 主机示例:将缓存保留在宿主机用户缓存目录,避免每
|
||||
插件和依赖。项目工作目录也需要挂载,以保留 `build/` 和项目级 `.gradle/`。
|
||||
|
||||
```sh
|
||||
IAM_GRADLE_CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/iam-login/gradle"
|
||||
mkdir -p "$IAM_GRADLE_CACHE"
|
||||
docker run --rm --network host \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--group-add "$(stat -c %g /var/run/docker.sock)" \
|
||||
-e LANG=C.UTF-8 \
|
||||
-e GRADLE_USER_HOME=/gradle \
|
||||
-e TESTCONTAINERS_HOST_OVERRIDE=127.0.0.1 \
|
||||
-v "$IAM_GRADLE_CACHE:/gradle" \
|
||||
-v "$PWD:/workspace" \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-w /workspace --entrypoint /bin/bash \
|
||||
ghcr.io/graalvm/native-image-community@sha256:0d936f32bb8acb5bc60c41b33e05f064d7a6aaf36b726538296c54949bd4a3c0 \
|
||||
-c './gradlew --no-daemon --max-workers=4 test'
|
||||
scripts/gradle-in-docker test testAot
|
||||
scripts/gradle-in-docker nativeTest nativeCompile
|
||||
python3 scripts/native-smoke.py
|
||||
```
|
||||
|
||||
`scripts/gradle-in-docker` 默认挂载 `${XDG_CACHE_HOME:-$HOME/.cache}/iam-login/gradle` 到
|
||||
容器 `/gradle`,并设置 Gradle 缓存与构建用户目录到该目录,避免无 passwd 条目的宿主 UID
|
||||
导致 Testcontainers 向项目内的 `?/` 写配置。原生测试同时显式传入 `user.home`。可通过 `IAM_GRADLE_CACHE` 覆盖宿主路径;
|
||||
默认限制 4 CPU、8 GiB 内存,可通过 `IAM_BUILD_CPUS`、`IAM_BUILD_MEMORY` 调整。
|
||||
需要 sudo 才能访问 Docker 时设置 `IAM_DOCKER_USE_SUDO=1`;只提升 Docker 命令权限,
|
||||
容器内仍使用当前用户 UID/GID。
|
||||
|
||||
宿主机 Docker socket 供 Testcontainers 使用,host network 让测试能访问它启动的动态端口。
|
||||
Docker Desktop 的网络方式需按平台调整。首次本地验证使用 `/tmp/iam-login-gradle` 作为
|
||||
上述缓存目录,未写入仓库;长期开发使用持久缓存目录。构建容器使用宿主 UID/GID,
|
||||
避免产物和缓存变为 root 所有。
|
||||
|
||||
## 原生启动检查
|
||||
|
||||
`native-smoke.py` 在 Linux 上直接启动 `build/native/nativeCompile/iam-login` ELF 文件,
|
||||
不调用 Java。需要 Python 3 标准库及编译产物依赖的系统库;监听随机 loopback 端口,
|
||||
结束时清理进程,并检查包括关闭阶段在内的 Native 注册错误;结果保存在 `build/reports/native-smoke/`。
|
||||
|
||||
检查匿名 liveness 端点、应用与指标端点拒绝匿名访问、使用临时测试凭据读取 Prometheus,
|
||||
以及健康请求的 HTTP 计数器增长。记录二进制大小、可响应耗时和请求后 RSS;这些单次
|
||||
数据不是负载基准或资源预算。脚本不请求依赖目录的聚合健康端点,
|
||||
不验证目录就绪、MFA、Hydra 或追踪导出。
|
||||
|
||||
应用在构建期声明 `health`、`prometheus` 暴露范围与健康探针。默认安全配置继续保护
|
||||
指标端点;liveness 使用应用可用性状态,不把尚未配置 AD 的聚合健康状态描述为正常。
|
||||
|
||||
## AOT 测试与容器连接
|
||||
|
||||
初始生成的 `@Bean + @ServiceConnection` 测试配置在 Boot 4.1.1 的 Native 测试中
|
||||
出现过缺少 `OtlpLoggingConnectionDetails` 的启动失败。测试改用类静态字段上的
|
||||
`@Container + @ServiceConnection`,由测试上下文在运行时重建连接信息,并断言日志
|
||||
导出 URL 指向本次运行的 LGTM 容器。JVM 开发入口仍使用原有配置类。
|
||||
|
||||
`testAot` 使用生成的 AOT 测试上下文在 JVM 中运行,可先发现 AOT 装配错误;它不替代
|
||||
`nativeTest`。Native 测试二进制启用 `quickBuild` 降低编译成本,应用 `nativeCompile`
|
||||
保持默认优化;启动和内存数据必须来自应用二进制,不能取测试二进制的数据。
|
||||
|
||||
参考 [Spring Boot Testcontainers](https://docs.spring.io/spring-boot/reference/testing/testcontainers.html)
|
||||
与 [Native Build Tools Gradle 配置](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html)。
|
||||
|
||||
## Protobuf Native 元数据
|
||||
|
||||
Micrometer OTLP 使用的 Protobuf 4.35.1 通过反射调用 `ExtensionRegistry` 的
|
||||
`getEmptyRegistry()` 与 `newInstance()`。实际原生应用启动曾因前一个方法缺少注册而失败,
|
||||
项目的 `reachability-metadata.json` 只补充这两个工厂方法。升级依赖时应重新验证能否移除。
|
||||
|
||||
上下文测试显式启用 `@AutoConfigureMetrics` 与 `@AutoConfigureTracing`,初始化真实导出器,
|
||||
避免 Spring 测试默认关闭导出导致 Native 问题漏测。它验证初始化及容器连接信息,完整的
|
||||
OTLP 数据接收、持久化和查询验收仍属后续观测性集成测试。
|
||||
|
||||
Micrometer 1.17.1 自带的反射声明已覆盖 CPU 使用率和文件描述符读取,但遗漏了
|
||||
`OperatingSystemMXBean.getProcessCpuTime()`。项目只补这一项;回归测试直接读取
|
||||
Prometheus 注册器中的 CPU 时间计数器并断言数值有效,避免后台导出异常被误报为测试通过。
|
||||
其他 JVM 指标在 Native 中的语义仍需分别验证,不能保证 HotSpot 仪表盘完全适用。
|
||||
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
repo_root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
cache_dir=${IAM_GRADLE_CACHE:-${XDG_CACHE_HOME:-$HOME/.cache}/iam-login/gradle}
|
||||
mkdir -p -- "$cache_dir"
|
||||
cache_dir=$(cd -- "$cache_dir" && pwd)
|
||||
docker_cmd=(docker)
|
||||
if [[ ${IAM_DOCKER_USE_SUDO:-0} == 1 ]]; then
|
||||
docker_cmd=(sudo -n docker)
|
||||
fi
|
||||
if (($# == 0)); then
|
||||
set -- test
|
||||
fi
|
||||
|
||||
# Linux host networking is needed for Testcontainers' published ports.
|
||||
exec "${docker_cmd[@]}" run --rm --network host \
|
||||
--cpus "${IAM_BUILD_CPUS:-4}" --memory "${IAM_BUILD_MEMORY:-8g}" \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--group-add "$(stat -c %g /var/run/docker.sock)" \
|
||||
-e LANG=C.UTF-8 \
|
||||
-e HOME=/gradle \
|
||||
-e JAVA_TOOL_OPTIONS=-Duser.home=/gradle \
|
||||
-e GRADLE_USER_HOME=/gradle \
|
||||
-e TESTCONTAINERS_HOST_OVERRIDE=127.0.0.1 \
|
||||
-v "$cache_dir:/gradle" \
|
||||
-v "$repo_root:/workspace" \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-w /workspace --entrypoint /workspace/gradlew \
|
||||
ghcr.io/graalvm/native-image-community@sha256:0d936f32bb8acb5bc60c41b33e05f064d7a6aaf36b726538296c54949bd4a3c0 \
|
||||
--no-daemon --max-workers="${IAM_BUILD_CPUS:-4}" "$@"
|
||||
Executable
+123
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Exercise the real native executable without a JVM or production services."""
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import secrets
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
def health_requests(metrics):
|
||||
total = 0.0
|
||||
for line in metrics.splitlines():
|
||||
match = re.match(r'^http_server_requests_seconds_count\{([^}]*)\}\s+([\d.eE+-]+)', line)
|
||||
if match and re.search(r'uri="/actuator/health[^"]*"', match[1]):
|
||||
total += float(match[2])
|
||||
return total
|
||||
|
||||
|
||||
def main():
|
||||
binary = Path(sys.argv[1] if len(sys.argv) > 1 else 'build/native/nativeCompile/iam-login').resolve()
|
||||
with binary.open('rb') as source:
|
||||
if source.read(4) != b'\x7fELF':
|
||||
raise RuntimeError('Expected a Linux native ELF executable')
|
||||
report_dir = Path('build/reports/native-smoke')
|
||||
report_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_path = report_dir / 'application.log'
|
||||
result_path = report_dir / 'result.json'
|
||||
result_path.unlink(missing_ok=True)
|
||||
password = secrets.token_urlsafe(32)
|
||||
env = os.environ.copy()
|
||||
env.update(SPRING_SECURITY_USER_NAME='native-smoke', SPRING_SECURITY_USER_PASSWORD=password)
|
||||
started = time.monotonic()
|
||||
process = None
|
||||
try:
|
||||
with log_path.open('w') as log:
|
||||
process = subprocess.Popen([
|
||||
str(binary), '--server.address=127.0.0.1', '--server.port=0',
|
||||
], env=env, stdout=log, stderr=subprocess.STDOUT)
|
||||
deadline = started + 60
|
||||
while True:
|
||||
if process.poll() is not None:
|
||||
raise RuntimeError(f'Native process exited before startup; see {log_path}')
|
||||
match = re.search(r'Tomcat started on port (\d+)', log_path.read_text(errors='replace'))
|
||||
if match:
|
||||
break
|
||||
if time.monotonic() > deadline:
|
||||
raise RuntimeError(f'Native startup timed out; see {log_path}')
|
||||
time.sleep(0.1)
|
||||
base_url = f'http://127.0.0.1:{match[1]}'
|
||||
basic = base64.b64encode(f'native-smoke:{password}'.encode()).decode()
|
||||
# Keep loopback traffic independent of developer proxy settings.
|
||||
client = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
|
||||
def request(path, authenticated=False):
|
||||
headers = {'Accept': 'text/plain' if authenticated and path == '/actuator/prometheus' else 'application/json'}
|
||||
if authenticated:
|
||||
headers['Authorization'] = f'Basic {basic}'
|
||||
req = urllib.request.Request(base_url + path, headers=headers)
|
||||
try:
|
||||
with client.open(req, timeout=10) as response:
|
||||
return response.status, response.read().decode()
|
||||
except urllib.error.HTTPError as error:
|
||||
return error.code, error.read().decode()
|
||||
|
||||
while True:
|
||||
status, body = request('/actuator/health/liveness')
|
||||
if status == 200 and json.loads(body)['status'] == 'UP':
|
||||
break
|
||||
if time.monotonic() > deadline:
|
||||
raise RuntimeError('Liveness endpoint did not become UP')
|
||||
time.sleep(0.1)
|
||||
ready_seconds = time.monotonic() - started
|
||||
assert request('/actuator/prometheus')[0] == 401, 'Anonymous metrics must be rejected'
|
||||
assert request('/')[0] == 401, 'Anonymous application access must be rejected'
|
||||
status, before = request('/actuator/prometheus', authenticated=True)
|
||||
assert status == 200, 'Authenticated Prometheus scrape failed'
|
||||
for _ in range(3):
|
||||
assert request('/actuator/health/liveness')[0] == 200
|
||||
metric_deadline = time.monotonic() + 5
|
||||
while True:
|
||||
status, after = request('/actuator/prometheus', authenticated=True)
|
||||
assert status == 200
|
||||
delta = health_requests(after) - health_requests(before)
|
||||
if delta >= 3:
|
||||
break
|
||||
if time.monotonic() > metric_deadline:
|
||||
raise RuntimeError(f'HTTP metric did not count health requests: {delta}')
|
||||
time.sleep(0.1)
|
||||
memory = Path(f'/proc/{process.pid}/status').read_text()
|
||||
rss = re.search(r'^VmRSS:\s+(\d+) kB', memory, re.MULTILINE)
|
||||
report = {
|
||||
'binary_bytes': binary.stat().st_size,
|
||||
'ready_seconds': round(ready_seconds, 3),
|
||||
'rss_after_requests_kib': int(rss[1]) if rss else None,
|
||||
'health_request_count_delta': delta,
|
||||
'liveness': 'UP',
|
||||
'anonymous_application_and_metrics': 401,
|
||||
'authenticated_metrics': 200,
|
||||
'scope': 'native bootstrap only; no directory readiness, AD/MFA/Hydra or trace-export validation',
|
||||
}
|
||||
finally:
|
||||
if process is not None and process.poll() is None:
|
||||
process.terminate()
|
||||
try:
|
||||
process.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
runtime_log = log_path.read_text(errors='replace')
|
||||
if re.search(r'MissingReflectionRegistrationError|MissingResourceRegistrationError|UnsupportedFeatureError', runtime_log):
|
||||
raise RuntimeError(f'Native runtime registration error, including during shutdown; see {log_path}')
|
||||
result_path.write_text(json.dumps(report, indent=2) + '\n')
|
||||
print(json.dumps(report, indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"reflection": [
|
||||
{
|
||||
"type": "com.google.protobuf.ExtensionRegistry",
|
||||
"methods": [
|
||||
{
|
||||
"name": "getEmptyRegistry",
|
||||
"parameterTypes": []
|
||||
},
|
||||
{
|
||||
"name": "newInstance",
|
||||
"parameterTypes": []
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "com.sun.management.OperatingSystemMXBean",
|
||||
"methods": [
|
||||
{
|
||||
"name": "getProcessCpuTime",
|
||||
"parameterTypes": []
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,3 +1,12 @@
|
||||
spring:
|
||||
application:
|
||||
name: iam-login
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: health,prometheus
|
||||
endpoint:
|
||||
health:
|
||||
probes:
|
||||
enabled: true
|
||||
|
||||
@@ -1,15 +1,51 @@
|
||||
package top.ddupan.iam.login;
|
||||
|
||||
import io.micrometer.core.instrument.MeterRegistry;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.beans.factory.annotation.Qualifier;
|
||||
import org.springframework.boot.micrometer.metrics.test.autoconfigure.AutoConfigureMetrics;
|
||||
import org.springframework.boot.micrometer.tracing.test.autoconfigure.AutoConfigureTracing;
|
||||
import org.springframework.boot.opentelemetry.autoconfigure.logging.otlp.OtlpLoggingConnectionDetails;
|
||||
import org.springframework.boot.opentelemetry.autoconfigure.logging.otlp.Transport;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
|
||||
import org.testcontainers.grafana.LgtmStackContainer;
|
||||
import org.testcontainers.junit.jupiter.Container;
|
||||
import org.testcontainers.junit.jupiter.Testcontainers;
|
||||
import org.testcontainers.utility.DockerImageName;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
@Import(TestcontainersConfiguration.class)
|
||||
@SpringBootTest
|
||||
@AutoConfigureMetrics
|
||||
@AutoConfigureTracing
|
||||
@Testcontainers
|
||||
class IamLoginApplicationTests {
|
||||
|
||||
// Field-based service connections are recreated by the test context in AOT mode.
|
||||
@Container
|
||||
@ServiceConnection
|
||||
static final LgtmStackContainer grafanaLgtm = new LgtmStackContainer(
|
||||
DockerImageName.parse(TestcontainersConfiguration.LGTM_IMAGE));
|
||||
|
||||
@Autowired
|
||||
OtlpLoggingConnectionDetails loggingConnectionDetails;
|
||||
|
||||
@Autowired
|
||||
@Qualifier("prometheusMeterRegistry")
|
||||
MeterRegistry prometheus;
|
||||
|
||||
@Test
|
||||
void contextLoads() {
|
||||
void processCpuTimeCanBeRead() {
|
||||
assertThat(prometheus.get("process.cpu.time").functionCounter().count()).isFinite().isNotNegative();
|
||||
}
|
||||
|
||||
@Test
|
||||
void loggingConnectionUsesRunningContainer() {
|
||||
assertThat(grafanaLgtm.isRunning()).isTrue();
|
||||
assertThat(loggingConnectionDetails.getUrl(Transport.HTTP))
|
||||
.isEqualTo(grafanaLgtm.getOtlpHttpUrl() + "/v1/logs");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -9,10 +9,12 @@ import org.testcontainers.utility.DockerImageName;
|
||||
@TestConfiguration(proxyBeanMethods = false)
|
||||
class TestcontainersConfiguration {
|
||||
|
||||
static final String LGTM_IMAGE = "grafana/otel-lgtm@sha256:b966ea107831d526d9eb8fe4d2d86c9e5731392fad9dce8296bcf2072031f07c";
|
||||
|
||||
@Bean
|
||||
@ServiceConnection
|
||||
LgtmStackContainer grafanaLgtmContainer() {
|
||||
return new LgtmStackContainer(DockerImageName.parse("grafana/otel-lgtm:latest"));
|
||||
return new LgtmStackContainer(DockerImageName.parse(LGTM_IMAGE));
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user