Commands, policy and retention
Sending commands from Central to a device, changing its policy, and how data buffered while offline gets synced
Required permission: OperateOPERATOR · ADMIN
Commands
Central cannot open a connection to a device behind a factory NAT, so the device fetches commands by long polling, waiting up to commands_poll_s (default 25 seconds). The device always reports (acks) the result, so on screen you can watch a command go Queued (대기 중) → Delivered (전달됨) → Done (완료) / Failed (실패). Sending commands needs the OperateOPERATOR · ADMIN permission.
- Open the Commands (명령) tab on the device detail page. ① Pick a command type and ② press Send command (명령 보내기).
Commands tab — ① command type ② Send command ③ a failed command keeps the error the device reported - The device picks it up on its next poll and handles it, and the outcome stays in the Result (결과) column. ③ A failed command shows the error exactly as the device reported it (here, a 404 from trying to pull a model version with no artifacts).
| Command | Screen name | What the device does | Example result |
|---|---|---|---|
ping | Ping (핑) | Answers without changing anything | {"pong": true, "device_id": "edge-bench-01"} |
restart | Restart (재시작) | Ends the agent with exit code 3 — the supervisor starts it again | {"restarting": true} |
resync | Resync (재동기화) | Puts queue items folded into failed back into the queue and sends right away | {"requeued": 0} |
pull_model | Pull model (모델 내려받기) | Pulls and activates the model given by the arguments {name, version, activate} | {"name": …, "activated": true} |
set_policy | (automatic on policy save) | Applies the new policy | {"applied": true} |
A command that is not picked up within 24 hours (the default) becomes Expired (만료). While the agent is running, a command delivered again is still executed only once.
Restart means exit code 3
The agent does not re-execute itself. A process that has been flagged as misbehaving is not trusted to repair itself. On restart it wraps up the transfer in progress and ends with exit code 3, and systemd (Restart=always) or the container runtime (restart: unless-stopped) starts a clean new process. The Restart (재시작) button at the top right of the device detail page sends the same command.
Verified result:
WARNING geo_mlops_sdk.edge.runtime: restart requested: command 1a1d2559-…
$ echo $?
3| Exit code | Meaning |
|---|---|
0 | Stopped on request (SIGTERM, Ctrl-C) |
3 | Restart requested — the supervisor must start it again |
4 | Could not open the local API (port in use and so on) — a config problem that restarting will not fix |
Offline block
The Offline (Offline) button on the device detail page is a server-side switch, not a command. While it is on, every call made with that device's token is rejected with 503 (Retry-After: 60). From the device it looks as if Central has briefly disappeared: it keeps what it could not send in the queue and waits, and when the block is lifted it uploads the backlog. Unlike revoking the token, you can undo it right from the screen.
Policy
A policy sets, per device and from Central, the heartbeat interval, the command poll interval, retention limits and upload throttling (retention, sync). Changing it needs the SettingsADMIN permission.
- Open the Policy (정책) tab on the device detail page. ① The current revision is shown. Edit the values and ② press Save policy (정책 저장).
Policy tab — ① current revision ② Save policy (the revision goes up and the device picks it up on its next heartbeat) - Saving raises the revision by 1 and also queues a
set_policycommand so the change applies right away. The device also fetches the new policy when it sees thatpolicy_revisionin the heartbeat response has changed.
| Screen field | Setting key |
|---|---|
| Heartbeat interval (s) (하트비트 주기 (초)) | heartbeat_interval_s |
| Retention period (days) (보존 기간 (일)) | retention.max_age_days |
| Retention size (GiB) (보존 용량 (GiB)) | retention.max_bytes |
| Upload rate limit (B/s) (전송 속도 상한 (B/s)) | sync.max_bytes_per_s |
| CPU pause threshold (%) (CPU 일시정지 임계 (%)) | sync.cpu_pause_percent |
| Allowed upload windows (전송 허용 시간대) | sync.windows |
The heartbeat interval must be well below the server's offline threshold (180 seconds by default). If it is too long, a healthy device flickers to Unhealthy (비정상).
Retention — nothing is thrown away when the link drops
For an edge device, pulled cables, power cuts and days offline are normal. The agent works on that assumption.
- Collection never stops. Collectors write to the local queue (SQLite + file spool) regardless of link state.
- It buffers within limits. It keeps to whichever of the three
retentionlimits is hit first (max_bytes50 GiB,max_age_days30 days,free_disk_min_bytes5 GiB), and when they overflow it drops the lowest-priority, oldest items first. The number dropped goes up in the heartbeat'sevicted_24hand shows as a red−Nin the Backlog (대기) column of the fleet list — a sign that data is piling up faster than it is sent. - When the link comes back, it sends right away. After two connection checks in a row (
link.online_after_ok) it registers → sends a heartbeat immediately → drains the queue. It sends the highest priority first. - Nothing is stored twice. The record id is an idempotency key, so if a response is lost and the same batch is sent again, Central counts them as
duplicatesand stores them once. Files resume upload chunk by chunk.
On a device with a busy CPU we saw uploads stop because of cpu_pause_percent (default 85), with 7 records piling up, and all of them uploaded right after the limit was lifted.
"sync": { "state": "paused", "last_error": "paused: cpu above threshold" },
"backlog": { "count": 7, "bytes": 1150, "by_kind": { "blob": 1, "http": 3, "robot": 3 } }The values that throttle uploads are in the sync table of the Configuration reference. Check items folded into failure with geo-mlops-edge queue --state failed, and send them again with the Resync (재동기화) command or POST /api/v1/sync:retry-failed.