Troubleshooting

简体中文

Start with the monitor’s own endpoints before changing CPA or auth files.

For a first pass that does not start the dashboard, run:

codex-quota-monitor print-config
codex-quota-monitor doctor

doctor reports which config file was loaded, whether the listen port is bindable, and whether the SQLite state path is writable. Use codex-quota-monitor init --mode user to regenerate a clean starter config when the local service wiring is unclear. After a Python install, cqm can replace codex-quota-monitor in these interactive commands.

1. Is The Monitor Running?

curl -fsS http://127.0.0.1:4515/healthz

Expected output:

{"ok":true}

If this fails, check the process or service manager first:

• Foreground Python or Nix: look at the terminal where codex-quota-monitor is running.
• systemd: systemctl status codex-quota-monitor --no-pager and journalctl -u codex-quota-monitor -n 100 --no-pager.
• Docker: docker ps and docker logs <container>.

/healthz only means the monitor process is alive. It does not prove that CPA, the management key, auth files, SQLite history, or benchmark imports are healthy.

2. Where Are Logs?

The monitor follows a Unix service model:

• stdout is for command output such as print-config, doctor --json, and generated init artifacts.
• Runtime logs are written to stderr; foreground runs show them in the terminal, systemd stores them in the journal, and Docker exposes them through docker logs.
• --log-format text is the default. Use --log-format json when a collector expects one JSON object per line.
• HTTP access logs use the codex-quota-monitor.http logger and are enabled by default. Disable them with --access-log off or CODEX_QUOTA_MONITOR_ACCESS_LOG=off when request-level logs are too noisy.
• SQLite Audit history records operational snapshot changes. It is not a user access audit log.

Do not scrape stderr as a stable API. Use /api/v1/status, /api/v1/diagnostics, /api/v1/alerts, /api/v1/recommendations, /metrics, or the configured SQLite history database for machine-readable state. codex-quota-benchmark is the exception that writes experiment artifacts under its output directory, including temporary gateway stdout/stderr when requested.

3. Is CPA Reachable?

curl -fsS http://127.0.0.1:4515/api/v1/diagnostics | jq '.'

The Diagnostics tab and /api/v1/diagnostics separate:

• gateway /healthz
• CPA management APIs
• direct quota sampling
• SQLite history
• benchmark summary import

If the Docker container cannot reach host CPA, use host.docker.internal as the CPA host and, on Linux Docker, add:

--add-host=host.docker.internal:host-gateway

4. Does The Management Key Work?

CPA management endpoints normally require a management key, including localhost requests. Pass it with --management-key-file /path/to/management-key, or with CODEX_QUOTA_MONITOR_MANAGEMENT_KEY_FILE for service and Docker deployments. For a bare CPA endpoint, the file must contain the same plaintext value as CPA remote-management.secret-key.

• 401 usually means the key is missing or does not match CPA remote-management.secret-key.
• 403 remote management disabled usually means CPA sees the request as non-localhost; use a loopback URL or enable remote management intentionally.
• 404 on /v0/management/* usually means recent CPA disabled the management API because no remote-management.secret-key is configured.

5. Why Are Quotas Unknown?

Unknown quota values usually mean direct Codex quota sampling is unavailable:

• --auth-dir is missing, points at the wrong path, or is not mounted into the container.
• The service user cannot read the auth files.
• The auth file is disabled, unavailable, expired, or in a CPA cooldown state.
• The direct quota API returned deactivated_workspace; the dashboard treats that account/workspace as unavailable and excludes it from pool capacity.
• Direct Codex quota response shape changed.

Check:

curl -fsS http://127.0.0.1:4515/api/v1/status | jq '.tabs.diagnostics'
curl -fsS http://127.0.0.1:4515/api/v1/alerts | jq '.'

If Diagnostics says the direct quota response shape changed, share only a redacted diagnostics/alerts payload and the package version. Do not share raw auth files or direct provider responses that may contain credential material.

6. Why Is The Pool Capacity Lower Than Raw 5h?

By default, total 5h capacity is capped by weekly remaining * 6.0. This avoids overstating short-window capacity when weekly quota is nearly exhausted.

Override it with:

codex-quota-monitor --weekly-to-five-hour-multiplier 8.0

Disable it with:

codex-quota-monitor --weekly-to-five-hour-multiplier off

7. Why Are Trends Or Audit Empty?

Trends and Audit require a writable SQLite state database.

• Direct nix run and ad hoc foreground usage need --state-db /path/to/history.sqlite3.
• The Docker image defaults to /data/history.sqlite3; mount /data if you want persistence.
• The NixOS module defaults to /var/lib/codex-quota-monitor/history.sqlite3.

Check the Diagnostics tab for SQLite write errors.

State lifecycle basics:

# Back up
cp /var/lib/codex-quota-monitor/history.sqlite3 /var/lib/codex-quota-monitor/history.sqlite3.bak

# Clear history by stopping the service and removing the DB
systemctl stop codex-quota-monitor
rm /var/lib/codex-quota-monitor/history.sqlite3
systemctl start codex-quota-monitor

Adjust the path for rootless services (~/.local/state/codex-quota-monitor/history.sqlite3) or Docker (/data/history.sqlite3 inside the container volume). The monitor recreates the DB on the next successful write.

8. Why Does The Dashboard Periodically Use CPU?

The browser polls /api/v1/status. Cached responses are cheap, but when refreshSeconds expires the monitor refreshes CPA management data, optional direct quota samples, the dashboard snapshot, and SQLite-backed Trends/Stats summaries. Keep refreshSeconds long enough for your host; 15s is responsive but can be noisy on thermally constrained laptops when the CPA pool is large.

SQLite history is retained by historyRetentionDays, and deleted pages can stay inside the database file as SQLite freelist pages. That is file-size pressure, not active history rows. The service does not run VACUUM in the request path because it can block the monitor. If the file has grown too large, back it up, stop the service, and run maintenance manually:

systemctl stop codex-quota-monitor
cp /var/lib/codex-quota-monitor/history.sqlite3 /var/lib/codex-quota-monitor/history.sqlite3.bak
sqlite3 /var/lib/codex-quota-monitor/history.sqlite3 'PRAGMA freelist_count; VACUUM;'
systemctl start codex-quota-monitor

If you do not need Trends, ETA, Audit, Stats history, or native history backup, run without --state-db to disable SQLite history. That reduces local work but also removes the historical dashboard features.

9. What Is Safe To Share?

Do not share auth files, CPA admin keys, API keys, private gateway URLs, or full logs that contain them. It is usually safe to share:

• package version
• install method
• redacted command line
• /healthz result
• redacted /api/v1/diagnostics
• redacted /api/v1/alerts
• service manager error text

Use codex-quota-monitor redact to create copies for sharing:

codex-quota-monitor redact /tmp/cqm-diagnostics.json --output /tmp/cqm-diagnostics-redacted.json
codex-quota-monitor redact /path/to/history.sqlite3 --output /tmp/history-redacted.sqlite3

10. Is It Safe To Bind 0.0.0.0?

Only if you also control the firewall or put the dashboard behind an authenticated reverse proxy. The UI and /api/v1/status are designed not to expose auth secret contents, but they can reveal account labels, quota state, model activity, and gateway health.

Direct non-loopback access also requires --allowed-host <hostname-or-ip> for the browser-facing authority; unexpected Host and Origin values are rejected. Keep the application on 127.0.0.1 and expose it through a reverse proxy when another device needs access. For example, protect the proxy with basic auth or an external access layer before forwarding https://quota.example/ to http://127.0.0.1:4515/.

The supported reverse-proxy topology is a dedicated hostname at the root path. Path-prefix deployments such as https://example/tools/quota/ are not supported in v1 because the UI assets and browser API requests use root-relative paths.

11. How Can I Notice Failures Without Prometheus?

Prometheus is optional. For a simple local check:

curl -fsS http://127.0.0.1:4515/api/v1/alerts | jq '.ok'

For systemd services, Restart=on-failure is included in the examples. Use systemctl status codex-quota-monitor --no-pager for system services and systemctl --user status codex-quota-monitor --no-pager for rootless user services.