Run as a service

arc run is the whole robot side. Run it under systemd so that it starts when the robot computer boots and comes back after a failure. It reconnects to the relay by itself after a network drop.

Run it once by hand first, as in the Quickstart, and set it up as a service only when that works.

Before you startCopy Anchor Link

• arc doctor robot.toml has no FAIL line.
• You know the user that ran arc enroll. The service must run as that user, because the robot's identity is in that user's home directory.
• You know three paths. The examples use user robot, the Python environment /home/robot/arc_env, and the robot's files in /home/robot/robot.

The unitCopy Anchor Link

Create /etc/systemd/system/arc.service with your user and paths:

[Unit]
Description=ARC robot client
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=robot
WorkingDirectory=/home/robot/robot
Environment=PYTHONUNBUFFERED=1
ExecStart=/home/robot/arc_env/bin/arc run /home/robot/robot/robot.toml
Restart=on-failure
RestartSec=10
# Stop signal goes to the main process only. It stops each arm in order.
KillMode=mixed
SyslogIdentifier=arc

[Install]
WantedBy=multi-user.target

What matters in it:

• User is the user that enrolled. Do not run the service as root.
• ExecStart uses the full path of arc inside the Python environment, so the environment does not need to be activated. Give robot.toml as a full path too.
• KillMode=mixed lets the client stop the arms in order when the service stops. With the default mode the arm processes are signalled at the same time as the client.
• Restart=on-failure starts the client again if it exits with an error. It does not restart it after systemctl stop.

The user must be able to write to the directory that holds robot.toml. The session logs go there, and a settings change from the console is written into robot.toml there.

CAN links at bootCopy Anchor Link

For YAM and MakerMods Metal arms, both CAN links must be up, at 1 Mbit/s, before the arms can open. Bring them up at boot the way you do for the rest of your system. If the client starts before a link is up, it warns, prints the command that brings the link up, and keeps trying that arm until it opens.

Start at bootCopy Anchor Link

sudo systemctl daemon-reload
sudo systemctl enable --now arc

enable makes the service start at every boot. --now also starts it now. Then check it:

systemctl status arc

The robot shows as Online on the Robots page in the console once the client has connected.

Only one client can run per robot computer. Stop the service before you run arc run by hand or arc doctor --identify.

LogsCopy Anchor Link

The client's output goes to the journal:

journalctl -u arc -f

-f follows the output. journalctl -u arc -b shows everything since the last boot.

A good start has these lines, in this order, with other lines between them:

1. [arc] Session log: and the path of the log file.
2. [yam-relay] config: and the table of every setting with where its value came from.
3. [yam-relay] config sync: and what the settings sync did, on an enrolled robot.
4. [yam-relay] checked in as and the robot id.
5. [yam-relay] Connected to relay and the relay address.
6. [yam-relay] Streaming. Ctrl+C to stop.

Every start also writes a session log file to logs/ next to robot.toml, named arc-<date>-<time>-<pid>.log. It holds the same output, is readable only by the service's user, and is complete after a crash. This is the file to send when you ask for help.

Old session logs are not deleted. Remove old files from logs/ from time to time, or add --no-log to ExecStart and rely on the journal alone.

Settings changes and restartsCopy Anchor Link

The client reads robot.toml once, when it starts.

• After you edit robot.toml, restart the service.
• A change pushed from the console is applied at the next start. Restart the service to apply it now. The config sync: line in the journal says what was applied.

See how settings sync.

Network watchdogCopy Anchor Link

A robot computer on Wi-Fi can drop off the network and stay off: after a few failed attempts NetworkManager stops trying until someone steps in. On a computer with Wi-Fi managed by NetworkManager, the installer adds a watchdog that brings the connection back. It is a systemd timer, arc-net-watchdog.timer, separate from the arc service.

What it does:

• Every 30 seconds it pings the default gateway and a public address. The network is up when either answers.
• After 3 failed checks in a row, and again after every 3 more, it brings the Wi-Fi connection up with nmcli connection up.
• After 10 minutes of failure it reloads the Wi-Fi driver, at most once every 10 minutes.
• It never reboots the computer and never touches the CAN links.

The connection it brings up is the Wi-Fi connection that was active during the install, or the one named with --wifi-connection NAME. Install lists the installer options for it.

The settings file is read on every check, so an edit applies without a restart:

Wi-Fi settings that keep a robot offlineCopy Anchor Link

arc doctor checks the active Wi-Fi connection and prints a WARN line, with the nmcli command that fixes it, for each of these:

The fixes, with the name of your connection in place of NAME:

sudo nmcli connection modify NAME 802-11-wireless.bssid ''
sudo nmcli connection modify NAME connection.autoconnect-retries 0
sudo nmcli connection modify NAME 802-11-wireless.powersave 2

For a second Wi-Fi connection that should not be used: sudo nmcli connection modify OTHER connection.autoconnect no.

Network notesCopy Anchor Link

• The robot opens every connection itself: TCP port 443 and UDP ports 4433 to 4440, outbound. Nothing has to be opened inbound. How it works has the details for your network team.
• A wired connection is better than Wi-Fi. When the link is slow or drops, the arms hold their position, and the operator resumes when it is back.
• After a drop the client reconnects by itself. It checks in again before every connection, so nothing needs a restart.

UpgradeCopy Anchor Link

1. Run the install command again, with the same options as the first time. It downloads the new release into its own directory and upgrades the packages in the existing Python environment. See Install.
2. Check the version: /home/robot/arc_env/bin/arc --version.
3. Fold or support the arms, then restart the service: sudo systemctl restart arc.

The running client keeps using the old version until the restart. robot.toml, the enrollment and the unit file are not touched by an upgrade. The Changelog lists what each release changed.

NextCopy Anchor Link

If the service does not come up, or the robot stays Offline, see Troubleshooting.