User guideWayseer 0.28.3Contents

External modules

A module you write can run as a program of its own. The app runs one with kind: external. It starts the program, talks to it over a private local connection, and starts it again, with back-off, if it fails. The program's own options go under the inner options.

Only a signed, installed package runs: a program the config names directly never does. You sign your own module with wayseer dev sign, and install the package with >modules.install.

SQL databases, Kubernetes and files are built into Wayseer, so they need no program and use their own kinds.

The license

External modules need a license key that unlocks modules, shown as external modules in >license, and that covers the release you run. Without one, the program never starts: the instance shows as locked in the sources panel and the status strip, and the status line says widgets is locked: external modules need a license key. Add a key while Wayseer runs and the module starts, with no restart.

A package you signed yourself runs only under the license your developer certificate is for.

A config that runs an old first-party program, such as wayseer-sql, runs the built-in module instead, and needs only the key that module needs (below).

Installing a package

A package is one file, such as acme-widget-1.4.0-linux-amd64.wsmod.tar.gz (.wsmod.zip for Windows). Install it with:

>modules.install file=~/Downloads/acme-widget-1.4.0-linux-amd64.wsmod.tar.gz

The app checks the package before it writes anything: that a license key unlocks external modules, that its signature verifies, that it was signed for your license, that its program is the one its manifest names, and that no revocation list covers it. Then it unpacks it into the state directory, beside license.key:

modules/acme/widget/1.4.0/
    module            the program (module.exe on Windows)
    manifest.yaml     what the module is and may do
    signature
    acme-widget-1.4.0-linux-amd64.wsmod.tar.gz    the package, kept as it came

The status line says installed acme/widget 1.4.0, Widget; a config names it with module: acme/widget, and the sources panel opens with the package at its foot, under Installed, with what it declares. The panel forgets it when it closes. Installing the same package again changes nothing. A different package with the same id and version is refused: remove the old version's directory first. To remove a module, delete its directory; the app never deletes one itself.

Nothing is downloaded, and nothing runs while a package installs.

What a module declares

The sources panel (S) lists, beneath each external instance, who signed its package and what its signed manifest says it may do, a plain row each:

ext                                                   on
external · live · 12 entities
signed by developer cert D-7Q2…
version 1.4.0
action restart: Restarts the rack's controller
kinds acme/rack, acme/pdu
network api.acme.example:443
reads the keyring

• Signed by is the marketplace, or developer cert and the certificate's ID. It never shows the certificate's display name or its license ID. Until the instance first starts, the row says signer checked when it starts, since each start checks the signature again.
• Action rows are the actions the module may offer, each with its one line on what it changes; no actions if none. Each still waits for you to confirm it (actions).
• Kinds, network and keyring are the kinds it may make, the endpoints it says it reaches, and whether it reads secrets from the keyring.

A long manifest is cut to fit: at most nine rows, with four rows for actions and the rest counted, four kinds and four endpoints named and the rest counted, and a long row ends in ….

Before you install a newer version of a marketplace module, >updates.show lists what its manifest adds and drops (newer modules). >modules.update with the module's id downloads it, shows the same rows, and installs it only when you press Y (updating a module).

Configuration

modules:
  - kind: external
    name: widgets
    options:
      module: acme/widget        # the package's id
      version: 1.4.0             # optional: the newest installed if none
      args: [--verbose]          # optional: the program's arguments
      env: [WIDGET_TOKEN, LANG=C]
      options:
        url: http://localhost:8080

Without version, the newest installed version runs, chosen when the config is read: a version installed later runs after the next reload. A module that isn't installed is a config error that says so.

command runs only the first-party programs below. Any other program is a config error: command runs an unsigned program; sign it with wayseer dev sign and name the package with module: instead (Upgrading).

Environment. The program sees only what env lists. NAME passes on the app's own value, if it has one, and NAME=value sets one. List what the module needs:

• HOME, for a path that starts with ~;
• the variable a secret_env names;
• on Linux, DBUS_SESSION_BUS_ADDRESS for a secret_keyring, so the module can reach the keyring;
• anything else the module's page lists, such as PATH for Kubernetes credential plugins.

Actions. An external module's actions are offered only when the entry's actions lists them, as for any module (Actions). The entry may list only actions the package's signed manifest declares; another is a config error naming the action and the module, such as line 6: module widgets: unknown action "purge" (its manifest declares restart). The app learns which actions the program offers when it starts it, so an action it does not offer shows as the module's error then.

An action the program offers but its manifest doesn't declare, or offers on a kind its manifest doesn't name for it, is left out: it isn't offered, the language model never sees it, and a request for it never reaches the program. The status line says so once, such as widgets offers actions its manifest doesn't declare, left out: purge, restart on acme/rack.

Errors. The inner options are checked by the module when it starts. An error names the line in the config file, as for any module.

Why a module is refused

Every time an external module starts, and again after a restart, the app checks its installed package before it runs anything, in this order. The program is hashed each time, so a change to it after install is caught. A refusal locks the instance, as no key does: the program never starts, the sources panel shows it locked with the reason, and the status line says why, once.

A new key checks a refused module again, with no restart, as turning it off and on does. For a changed program, remove its version's directory and install the package again.

When a module is revoked

Wayseer can revoke a module: one version, a range of versions, everything under one developer certificate or license, or everything signed by one key. It does so through a revocation list that Wayseer signs. The app holds one list, the newest it can verify: the list built into this release, or a newer one it fetched or you added. A newer release brings a newer built-in list.

A revoked module is deactivated at once, whatever the reason:

• At start, it never runs. The status line says widgets is revoked (security): Withdrawn after a vulnerability., with the list's reason and its words, and the sources panel shows it as revoked (security).
• While running, it is stopped as soon as a list covering it is added, with no restart. The status line says the same, and the log and standard error say stopped widgets: revoked (security) by list 42.
• Installing a revoked package is refused, and nothing is written: acme/widget 1.4.0 is revoked (security): Withdrawn after a vulnerability.

The reasons are security, unmaintained, unstable, publisher (its publisher asked) and other. A revocation by license or certificate shows only its reason and words, never which license or certificate it names.

To add a list Wayseer sent you, such as on a machine that is never online:

>modules.revocations.add file=~/Downloads/index

The status line says revocation list 43 added, and then what that list changed. A list is refused, and the one held kept, if it isn't a revocation list, if its signature doesn't verify, or if it is older than the list held: revocation list refused: it is older than the list held (43). A list can't be rolled back.

Each revocation is recorded in revocations/revoked.yaml in the state directory, beside the list held, revocations/index. The record outlives the list: deleting index doesn't bring a revoked module back. Only a newer list that no longer covers a module lifts its revocation, and then it starts again with no restart: widgets is no longer revoked.

Checking for newer lists

While it runs, Wayseer fetches the newest list in the background: first 1 to 10 minutes after it starts, then once a day, each time a few random minutes later. A newer list that verifies is held as if you had added it, and stops what it revokes at once. Wayseer never waits for the fetch: the window, start-up and every module carry on while it runs.

The fetch is one plain HTTPS GET of https://wayseer.app/v1/index, the same file for everyone. It sends User-Agent: Wayseer and nothing else: no license, no machine or user identifier, no version, no cookie. It follows a redirect only to https on the same host, and gives up after 30 seconds or 2 MiB. It honours the system's HTTPS_PROXY.

A fetch that fails changes nothing: the list held stays, every module keeps running, and nothing is locked. One line in the in-app log names the kind of failure, such as revocation check failed: timed out, and the next fetch waits for the next interval. A list refused for its signature, or for being older than the one held, fails the same way.

When the list held is over 30 days old, for any reason, the sources panel shows one muted line beneath the sources: revocation list is 34 days old. It never asks anything or stops anything.

The fetch is set in the config (revocations):

revocations:
  check: true                        # false: hold no list fetched
  interval: 24h                      # at least 1h
  url: https://wayseer.app/v1/index  # https only; a mirror's URL here
updates:
  check: true                        # false: no notice of newer releases
  download: false                    # true: download newer marketplace modules, never install

The fetch runs while either check is on. On an air-gapped host, set both to false, so Wayseer never makes a request, and add lists by hand with >modules.revocations.add, as above. A mirror inside your network serves the same signed file; Wayseer verifies it whoever serves it. updates.check is for the notice of newer releases, and updates.download for downloading newer modules.

When it fails

The program's standard error goes to the app's log, the first 64 KiB each time it starts. If the program exits or stops answering, its module shows an error, and its entities go stale, as for any failing source. The app keeps running and starts the program again. The internal module shows each module's state.

A module built for another major version of the module contract is refused, with an error saying which version each side speaks. Rebuild it against the SDK version the app requires. A module built for an older minor version still runs; it just lacks what came later, such as ranking a query's top entities (contract 1.2), offering actions (contract 1.3), sending traffic on edges (contract 1.4) or sending places (contract 1.5).

Configs written for the SQL, Kubernetes and file programs

Earlier versions ran SQL, Kubernetes and files as programs: wayseer-sql, wayseer-kubernetes and wayseer-file, or mindseye-sql, mindseye-kubernetes and mindseye-file. An external entry whose command names one of these, with or without a path, now runs the built-in module instead. Its inner options and its actions are used unchanged, and its command and env are ignored.

The app never changes your config. It says which entries it ran built in, on the status line when it starts and in its log, as db runs built in: set kind: sql. To stop the message, write the entry the new way: its kind, with the inner options moved up a level.

# Before
  - kind: external
    name: db
    options:
      command: [/usr/lib/wayseer/wayseer-sql]
      env: [SHOP_DSN]
      options:
        driver: postgres
        secret_env: SHOP_DSN

# After
  - kind: sql
    name: db
    options:
      driver: postgres
      secret_env: SHOP_DSN

A built-in module reads the environment Wayseer was started with, so a variable that only env set, as NAME=value, must be set before Wayseer starts instead. A change to only the command's path is not a change on reload: the instance keeps running.