Administration
Troubleshooting
Common problems, the messages they produce, and how to fix them. Start with the logs (Operations): the agent log on the node and the controller log say what went wrong in plain words.
Installing the controller
| Problem / message | Fix |
|---|---|
| The setup page asks for a token | You opened it from another computer. Get the token: Windows C:\ProgramData\NGCP\Controller\setup-token.txt; Linux sudo ngcp-controller setup-url; Docker docker compose exec controller cat /etc/ngcp/setup-token.txt. |
public_url must be like https://panel.example.com ... |
Enter only scheme, host and optional port; no path, no trailing slash. |
use https:// when the panel serves TLS itself, http:// otherwise |
TLS mode and URL scheme must match: files / self-signed need https://; none needs http://, or a reverse proxy’s https:// URL. |
tls_cert_file file not found on the server |
The path is read on the controller machine, not your PC. On Linux the files must also be readable by the user ngcp. |
| Built-in database fails on Windows | Check the setup’s message and C:\ProgramData\NGCP\Controller\postgres\. The setup can be repeated; it reuses what it already created. |
| Linux: no local database offered | PostgreSQL must be installed on this server; the package depends on it. Reinstall the package so its script prepares the ngcp role, or use Existing database. |
secrets: no master key configured |
The controller cannot find its master key. Check NGCP_MASTER_KEY_FILE in controller.env and that the file exists and is readable. Restore it from your backup; without it the sealed secrets cannot be recovered. |
database schema is at version N, this binary needs M |
Start with NGCP_AUTO_MIGRATE=true (the setup sets it) or run ngcpctl migrate. |
database schema N is newer than this binary |
You started an older controller on a newer database. Install the newer version again. |
listen on ...: address already in use |
Another program uses 8080 or 8443. Change NGCP_LISTEN / NGCP_GRPC_LISTEN or stop the other program. |
| Browser shows a certificate warning | Expected with a self-signed certificate. Use certificate files from a real CA or a reverse proxy. |
| Sign-in works but the page reloads to sign-in again over HTTP | NGCP_COOKIE_SECURE=always sends the cookie only over HTTPS. Use HTTPS, or auto. |
Adding nodes
| Agent log / output | Fix |
|---|---|
the controller's CA (...) does not match --ca-hash (...) ... the token was not used |
The URL reaches a different panel, or the hash has a typo. Copy the command from the panel again. |
the token must start with ngcpe_ / --ca-hash must be sha256:<64 hex characters> |
The command was cut when copying. |
the controller refused the enrollment (invalid_token) |
The token expired, was used, or was cancelled. Create a new one. |
contact the controller at ... |
The node cannot reach the public URL. Check DNS, firewall and NGCP_PUBLIC_URL. |
link to controller lost; retrying ... dial tcp |
Port 8443 blocked, or the agent address (NGCP_GRPC_ADVERTISE) is not reachable from the node. |
certificate is valid for ..., not ... |
The node dials a name that is not in the AgentLink certificate. Add it with NGCP_GRPC_HOSTNAMES and restart the controller. |
the controller rejected this node |
The node was removed. Enroll again with a new token (and --force). |
this machine is already enrolled (...); use --force to replace it |
Add --force. |
this node is not enrolled yet |
Run ngcp-agent enroll ... with the command from the panel. |
| Windows node takes ~20 s to come online the first time | Known; later connections are fast. |
run this as root (sudo bash) / script fails on Windows |
Run the installer as root / in an elevated PowerShell. |
neither dpkg nor rpm found |
Use the bare agent binary and the manual steps in Adding a node. |
Creating and running services
| Message | Fix |
|---|---|
the node does not accept new services (draining) / (disabled) |
Set the node to Active, or pick another node. |
<game> does not run on windows / linux |
The game definition does not support that OS. |
the node's agent has not reported its services directory yet; start the agent first |
The node never connected. Check the agent. |
no free ports left in the node's range 27000-29999 |
Widen the node’s port range in its settings, or use another node. |
the owner already has a service with this name |
Pick another name. |
the service's node is offline |
The agent is not connected; actions wait for it. Game servers keep their last state. |
cannot start a service that is running (and similar) |
The action is not allowed in the current status; see the action table. |
the game is not installed yet; use reinstall |
Update needs an installed game; run Reinstall. |
| Service keeps going to crashed | Read the console’s last lines. Common causes: a wrong variable, a missing EULA acceptance (Minecraft), not enough memory (a memory limit or the node), a port used by another program on the node. After the crash-policy limit it stays crashed until started. |
| Minecraft does not start | Tick I accept the Minecraft EULA in the service’s settings and restart. Choose a Java version new enough for the Minecraft version (25 runs all current ones). |
| ArmA 3 on Windows: console shows only log lines | Normal: ArmA writes its console to its own window, so the panel tails the .rpt log. |
| ArmA 3 RCON / BattlEye does not answer | Restart the service so bound settings are written; BattlEye reads its config only from the place the definition writes it. Services installed with definition version 1 need a reinstall. |
| Server not visible in the server browser | Check the node’s firewall and your router for the service’s ports (the overview lists them). The status query in the panel runs on the node and does not prove outside reachability. |
| Game ports clash with other software on the node | NGCP does not see ports used by programs outside NGCP. Keep the node’s port range free of other software. |
Installs and SteamCMD
| Message | Fix |
|---|---|
SteamCMD cannot run: this Linux node needs the 32-bit libraries |
sudo dpkg --add-architecture i386 && sudo apt update && sudo apt install lib32gcc-s1 lib32stdc++6, then reinstall. |
this game needs a Steam account: add one in Settings > Steam accounts |
Add an account and mark it default (Steam). |
Steam rejected the login (check the account's password in Settings > Steam accounts) |
Fix the password in the vault. |
| Install waits with “SteamCMD is waiting for a Steam Guard code” | Type the code into the service’s console. |
no Steam Guard code was entered in time |
Start the install again and enter the code sooner. |
items ... were not downloaded / SteamCMD said "..." |
Workshop items need an account that owns the game (ArmA 3); hidden or removed items cannot be downloaded. Check the item ids. |
checksum mismatch for ... |
The download was corrupted or changed; try again later. |
git is not installed on this node |
Install git on the node. |
unknown Minecraft version "..." / no stable Paper build for "..." |
Use latest or an existing version number. |
| An update seems to do nothing for mods | Saving mods only marks them; run Update to download. |
Files and SFTP
| Problem / message | Fix |
|---|---|
access to this path is denied |
A deny rule hides the path (from the game definition or the service). Administrators are not affected. |
the service's disk quota would be exceeded |
Free space or ask an administrator to raise the quota. |
the file changed since it was read |
Someone (often the game) changed the file after you opened it. Reload and redo your edit. |
the file is too large for this operation |
The editor opens files up to 2 MiB; download larger ones. |
| Upload stopped after a controller restart | Upload sessions live in the controller’s memory; start the upload again. |
| SFTP: “permission denied” for the right password | Check the user name <panel user>.<service id>, the port (2022), and that you have the files permission. Accounts with two-step verification need an SSH key unless the SuperAdmin allows passwords. |
SFTP: too many failed logins |
20 failures in 10 minutes from your address; wait 10 minutes. |
| SFTP: still works after disabling an account | Logins are cached for up to 60 seconds. |
| SFTP: connection refused | sftp_listen is off in agent.yaml, or port 2022 is blocked by a firewall. |
RCON and player counts
| Message | Fix |
|---|---|
this game has no remote console |
The definition has no rcon section. |
the game rejected the RCON password |
The game’s config differs from the panel. Restart the service so bound settings are written; check for hand edits. |
the game did not answer (is RCON enabled and the server up?) |
Wait until the server finished starting; check the game’s RCON settings. |
| Players always empty | The game has no query section, the query port is wrong, or the game does not answer queries until fully started. |
Scheduled tasks
| Problem | Fix |
|---|---|
| A task did not run at its time | Was the node off or the agent stopped then? Missed runs are not run late (more than five minutes). Check the service’s run history and console. |
| A task ran at the wrong hour | Check its time zone (default UTC). |
time zone "...": unknown time zone |
The name is not an IANA time zone. Use names such as Europe/Stockholm or America/Chicago (not CET abbreviations or Windows names like W. Europe Standard Time). NGCP carries its own time-zone database, so nothing has to be installed. |
Run skipped: N players online |
Only when empty is on and players were online. |
Run skipped: controller unreachable; updates need it |
Updates need the controller (Steam logins live there). |
Run skipped: the service is stopped |
A restart task does not start a service that is meant to be stopped. Use a start task. |
script action refused |
Only SuperAdmins and Admins can create, change or delete script tasks. |
this game has no remote console when saving an rcon task |
The game has no RCON. |
Backups
| Message | Fix |
|---|---|
pick a backup target |
The target does not exist or is not user selectable for you. |
the backup target is not available on this node |
The target was removed or changed and the node has not received the update yet; let the node reconnect, then try again. |
backup: sftp host key SHA256:... does not match the target's SHA256:... |
The SFTP server’s key changed, or you reached another machine. Verify, then update the fingerprint. |
backup: a local target needs an absolute path |
Use a path like /srv/backups or D:\Backups. |
N files could not be read (in use?) and are not in the backup |
The running game holds those files locked (Windows). Stop the server for a complete backup, or exclude them. |
a backup is already running for this service / a restore ... |
Wait for it to finish. |
only a completed backup can be restored / the backup's target was removed |
Pick a completed backup on an existing target. |
S3 errors (Access Denied, NoSuchBucket) |
Check endpoint, bucket, region, keys and the key’s permissions; turn on use_tls for public providers. |
Notifications
| Problem | Fix |
|---|---|
Delivery failed after one attempt with receiver answered 4xx |
The receiver rejected it (wrong URL, bad token). Fix it and send a test. |
| Delivery pending with retries | The receiver is down or slow (5xx, 429, timeouts); retries continue for 8 attempts. |
SMTP is not configured (Settings -> Notifications) |
Set the SMTP server. |
security: a password is never sent without TLS |
Use starttls or tls when the mail server needs a login. |
| Signature does not verify | Compute the HMAC over the raw body bytes with the timestamp header, exactly <timestamp>.<body>; see Notifications. |
No node.offline for a node that was already down when the controller started |
node.offline is sent when a connected node goes away. A node that never reconnects after a controller restart is not reported; check the Nodes page after maintenance. |
Accounts and sign-in
| Message / code | Fix |
|---|---|
invalid_credentials |
Wrong user name or password. |
account_locked |
Too many failures. Wait (15 minutes, doubling) or ask an administrator to unlock. |
rate_limited |
Too many sign-ins from your address; wait a few seconds. |
ip_not_allowed |
The admin IP allow-list does not include your address. Sign in from an allowed address and fix the list. In an emergency, clear it in the database and restart the controller: UPDATE settings SET value = jsonb_set(value, '{admin_ip_allowlist}', '[]') WHERE key = 'security'; |
mfa_enrollment_required |
Two-step verification is required for administrators; turn it on first. |
| Lost authenticator | Use a recovery code, or ask an administrator to turn off two-step verification for you. |
last_superadmin |
The last active SuperAdmin cannot be removed or demoted; create another one first. |