Proc Spec

This spec defines the ins and outs of the Proc API.

Version 1.0-beta3, published October 13, 2021.

Note that this spec is subject to change while Proc is in public beta, though we expect changes to be minimally disruptive for our users. We expect to finalize this spec in a few short weeks.

Proc defines functions mounted at unique endpoints that can be called through an http api. Each proc is called with input and arguments, returning an output. Here's a simple example using the type.array.flatten proc:

POST [0, [1]] >> proc.run/type/array/flatten?level=1 >> [0, 1]

The example above demonstrates proc's ability to handle simple values, but proc also handles complex values that might include nested proc calls and/or full compositions. This allows behavior to be defined elsewhere, passed to proc, and evaluated within the platform. Proc is designed to be expressive enough to build complex programs but simple enough to manually construct payloads and dispatch them with basic tooling like cURL.

Calling Procs

Proc calls must be POST requests to a namespaced path.

POST proc.run/type/string/reverse

Authorization

Proc calls must include an authorization. The preferred approach is to pass the secret or token through the authorization header with type bearer:

authorization: bearer {authorization}

Proc also supports basic authentication, allowing the authorization to be expressed through the url (supporting clients will add the correct authorization header):

https://{authorization}@proc.dev

Input Serialization

Proc accepts and responds with plaintext payloads by default:

POST foo >> proc.run/core/echo >> foo

Proc supports json payloads by passing the content-type header with a value of application/json or application/vnd.proc+json (see Complex Payloads below).

Proc supports msgpack payloads by passing the content-type header with a value of application/msgpack or application/vnd.proc+msgpack (see Complex Payloads below).

The response format can be specified through the accept header:

accept: text/plain
content-type: application/json

Unless specified, the default response format will match that of the request format.

Query String Arguments

Proc arguments can be passed through the query string.

POST hello >> proc.run/type/string/truncate?length=1 >> h

Note that this approach only works for simple values—complex values such as json arrays and objects can only be passed through a complex payload, explained below.

Complex Payloads

Proc supports complex payloads that can define input, arguments, and even variables, calls, or compositions. This payload structure makes proc fully programmable—entire programs can be represented in a payload and evaluated at call time by the proc platform. These programs, which we call compositions, can also be stored and called just like other procs, making the proc platform fully extensible.

Complex payloads must be serialized in a multi-dimensional array structure, defined below. Supported serialization formats include json and msgpack, indicated by one of these two supported content-type values:

• application/vnd.proc+json
• application/vnd.proc+msgpack

The payload must be an array of tuples, each tuple beginning with one of the following values to define its type:

• >>: Input
• $$: Argument
• @@: Variable
• %%: Literal
• (): Call
• {}: Composition

Each tuple type expects one or more values following its type definition.

Inputs

Inputs are defined with >> followed by the value:

POST [[">>", "foo"]] >> proc.run/core/echo >> "foo"

Arguments

Arguments are defined with $$ followed by the argument name and value:

POST [[">>", "foo"], ["$$", "length", 1]] >> proc.run/type/string/truncate >> "f"

Variables

Variables are defined with @@ followed by the variable name and an optional options object. Values for variables are passed as normal arguments.

Here's a complete example that uses a variable for input:

POST [[">>", ["@@", "input"]], ["$$", "input", "foo"]] >> proc.run/core/echo >> "foo"

Option: default

Sets a default value for the variable if a value is not passed as an argument:

POST [[">>", ["@@", "input", {"default": "bar"}]]] >> proc.run/core/echo >> "bar"

Option: type

Typecasts the value to the defined type:

POST [[">>", ["@@", "input", {"type": "integer"}]], ["$$", "input", "1"]] >> proc.run/core/echo >> 1

Supported types:

• boolean
• integer
• number
• string
• object

Literals

Literals are defined with %% followed by the value:

POST [[">>", ["%%", "foo"]]] >> proc.run/core/echo >> "foo"

Literals can be passed for any value, including as input (as above) or an argument value. Use literals to guarantee a value is not interpreted as another value type.

Calls

Calls are defined with () followed by the proc name, input, and/or arguments:

POST [[">>", ["()", "echo", [">>", "foo"]]]] >> proc.run/type/string/reverse >> "oof"

Input and arguments are scoped to the call and not available to other calls. If a call does not define input then input from the calling scope is used. Same for arguments—if a call does not define a needed argument, a value is obtained from the calling scope.

Compositions

Compositions are defined with {} followed by one or more calls. Input and arguments can be defined and made available for calls within the composition. When evaluated, each call is evaluated in order, the result of a call passed as input to the next:

POST [["$$", "proc", ["{}", [">>", "foo"], ["()", "type.string.reverse"], ["()", "type.string.capitalize"]]]]] >> proc.run/proc/exec >> "Oof"

Like calls, input and arguments are scoped to the composition. If a composition does not define input then input from the calling scope is used. Same for arguments—if a composition does not define an argument, a value is obtained from the calling scope.

Responses

Responses to requests with a complex payload use the same format, providing output under the << value type:

[["<<", "output"]]

Status Codes

Proc responds with one of the following status codes:

• 200: The proc call succeeded.
• 400: The given input or arguments are invalid.
• 401: The authorization is invalid.
• 403: The account lacks the ability.
• 404: The proc is undefined.
• 408: The proc call took too long to execute.
• 413: The request was too large.
• 429: The account is being rate limited.
• 500: Proc failed internally when handling the call.
• 508: Proc exceeded the maximum stack size.

Error Responses

Proc includes error messages in the response body. For plaintext responses, the error message will be returned as the full value of the response body. For json responses, the error will be returned as an error object:

{
  "error": {
    "message": "something went wrong"
  }
}

For proc json responses, the error object will be returned under the !! tuple value:

[["!!", {"message": "something went wrong"}]]

Cursors

Enumerable procs like keyv.scan support paging and return a cursor in the x-cursor response header. The returned values will be the first page of results, limited to count with a maximum of 250 values. The next page of results can be fetched by replaying the original request and passing the cursor through the cursor argument.

Hard Limits

• Proc payloads have a defined max size of 131,072 bytes.
• Proc calls have a defined execution limit of 5 seconds.