Proc Spec
This spec defines the ins and outs of the Proc API.
Version 1.0-beta3, published October 13, 2021.
Note that this spec is subject to change while Proc is in public beta, though we expect changes to be minimally disruptive for our users. We expect to finalize this spec in a few short weeks.
Proc defines functions mounted at unique endpoints that can be called through an http api. Each proc is called with
input and arguments, returning an output. Here's a simple example using the type.array.flatten
proc:
POST [0, [1]] >> proc.run/type/array/flatten?level=1 >> [0, 1]
The example above demonstrates proc's ability to handle simple values, but proc also handles complex values that might include nested proc calls and/or full compositions. This allows behavior to be defined elsewhere, passed to proc, and evaluated within the platform. Proc is designed to be expressive enough to build complex programs but simple enough to manually construct payloads and dispatch them with basic tooling like cURL.
Calling Procs
Proc calls must be POST
requests to a namespaced path.
POST proc.run/type/string/reverse
Authorization
Proc calls must include an authorization. The preferred approach is to pass the secret or token through the
authorization
header with type bearer
:
authorization: bearer {authorization}
Proc also supports basic authentication, allowing the authorization to be expressed through the url (supporting clients
will add the correct authorization
header):
https://{authorization}@proc.dev
Input Serialization
Proc accepts and responds with plaintext payloads by default:
POST foo >> proc.run/core/echo >> foo
Proc supports json payloads by passing the content-type
header with a value of application/json
or
application/vnd.proc+json
(see Complex Payloads below).
Proc supports msgpack payloads by passing the content-type
header with a value of application/msgpack
or
application/vnd.proc+msgpack
(see Complex Payloads below).
The response format can be specified through the accept
header:
accept: text/plain
content-type: application/json
Unless specified, the default response format will match that of the request format.
Query String Arguments
Proc arguments can be passed through the query string.
POST hello >> proc.run/type/string/truncate?length=1 >> h
Note that this approach only works for simple values—complex values such as json arrays and objects can only be passed through a complex payload, explained below.
Complex Payloads
Proc supports complex payloads that can define input, arguments, and even variables, calls, or compositions. This payload structure makes proc fully programmable—entire programs can be represented in a payload and evaluated at call time by the proc platform. These programs, which we call compositions, can also be stored and called just like other procs, making the proc platform fully extensible.
Complex payloads must be serialized in a multi-dimensional array structure, defined below. Supported serialization
formats include json
and msgpack
, indicated by one of these two supported content-type
values:
application/vnd.proc+json
application/vnd.proc+msgpack
The payload must be an array of tuples, each tuple beginning with one of the following values to define its type:
>>
: Input$$
: Argument@@
: Variable%%
: Literal()
: Call{}
: Composition
Each tuple type expects one or more values following its type definition.
Inputs
Inputs are defined with >>
followed by the value:
POST [[">>", "foo"]] >> proc.run/core/echo >> "foo"
Arguments
Arguments are defined with $$
followed by the argument name and value:
POST [[">>", "foo"], ["$$", "length", 1]] >> proc.run/type/string/truncate >> "f"
Variables
Variables are defined with @@
followed by the variable name and an optional options object. Values for variables are
passed as normal arguments.
Here's a complete example that uses a variable for input:
POST [[">>", ["@@", "input"]], ["$$", "input", "foo"]] >> proc.run/core/echo >> "foo"
Option: default
Sets a default value for the variable if a value is not passed as an argument:
POST [[">>", ["@@", "input", {"default": "bar"}]]] >> proc.run/core/echo >> "bar"
Option: type
Typecasts the value to the defined type:
POST [[">>", ["@@", "input", {"type": "integer"}]], ["$$", "input", "1"]] >> proc.run/core/echo >> 1
Supported types:
boolean
integer
number
string
object
Literals
Literals are defined with %%
followed by the value:
POST [[">>", ["%%", "foo"]]] >> proc.run/core/echo >> "foo"
Literals can be passed for any value, including as input (as above) or an argument value. Use literals to guarantee a value is not interpreted as another value type.
Calls
Calls are defined with ()
followed by the proc name, input, and/or arguments:
POST [[">>", ["()", "echo", [">>", "foo"]]]] >> proc.run/type/string/reverse >> "oof"
Input and arguments are scoped to the call and not available to other calls. If a call does not define input then input from the calling scope is used. Same for arguments—if a call does not define a needed argument, a value is obtained from the calling scope.
Compositions
Compositions are defined with {}
followed by one or more calls. Input and arguments can be defined and made available
for calls within the composition. When evaluated, each call is evaluated in order, the result of a call passed as input
to the next:
POST [["$$", "proc", ["{}", [">>", "foo"], ["()", "type.string.reverse"], ["()", "type.string.capitalize"]]]]] >> proc.run/proc/exec >> "Oof"
Like calls, input and arguments are scoped to the composition. If a composition does not define input then input from the calling scope is used. Same for arguments—if a composition does not define an argument, a value is obtained from the calling scope.
Responses
Responses to requests with a complex payload use the same format, providing output under the <<
value type:
[["<<", "output"]]
Status Codes
Proc responds with one of the following status codes:
200
: The proc call succeeded.400
: The given input or arguments are invalid.401
: The authorization is invalid.403
: The account lacks the ability.404
: The proc is undefined.408
: The proc call took too long to execute.413
: The request was too large.429
: The account is being rate limited.500
: Proc failed internally when handling the call.508
: Proc exceeded the maximum stack size.
Error Responses
Proc includes error messages in the response body. For plaintext responses, the error message will be returned as the full value of the response body. For json responses, the error will be returned as an error object:
{
"error": {
"message": "something went wrong"
}
}
For proc json responses, the error object will be returned under the !!
tuple value:
[["!!", {"message": "something went wrong"}]]
Cursors
Enumerable procs like keyv.scan
support paging and return a cursor in the x-cursor
response header. The returned
values will be the first page of results, limited to count
with a maximum of 250 values. The next page of results can
be fetched by replaying the original request and passing the cursor through the cursor
argument.
Hard Limits
- Proc payloads have a defined max size of 131,072 bytes.
- Proc calls have a defined execution limit of 5 seconds.
Stuck? Want to chat about an idea? Join the community on Discord.