Getting Started with Goblins

What is Goblins?

Goblins is a library for building distributed, peer-to-peer applications in Racket and Guile. It is a part of the Spritely ecosystem of ambitious projects attempting to fix the centralized internet.

Goblins adopts the principle of CapTP (Capability Transport Protocol) and OCAP (Object Capability). This guide aims to be a first practical introductory programming guide in Goblins for Racket programmers. Some experience programming in a Lisp dialect is useful, otherwise the general sections on OCAP and CapTP are still a good read.

What is Object Capability (OCAP)?

OCAP has been around for decades, but it hasn't been easy to find introductory resources on the internet. The most accessible one being Mark Miller's paper Capability Myths Demolished.

According to this Pony tutorial:

A capability is an unforgeable token that (a) designates an object and (b) gives the program the authority to perform a specific set of actions on that object.

The unforgeable token is, at the language level, just a reference to an object. In Object Capabilty's speak, "if you don't have it, you can't do anything with it".

The core idea of it is that programs are built with Principle of Least Authority (POLA), instead of starting out with the complete opposite (maximum authority). But before we get ahead of ourselves, let's step back to talk about how programs behave today.

When a user runs a program on the computer, that program is likely being run with that user's privilege level. That means aside from doing what it's supposed to do (like displaying a game window and interacting with I/O device for a game app), it can potentially do much more--whether it's reading and writing to local files, connecting to the network, and etc.--All under the radar of the unsuspecting user.

This generic authority to do many things implicitly is known as ambient authority, and it comes with today's operating systems' access control list (ACL).

In object capability, every entity that attempts to create some side effect to interact with the world needs access to specific objects in order to do so. To give an example in a fictional programming language; To manipulate a window, a scaleWindow(window: Window) procedure needs to be passed a Window object. To access a file on a file system, a readFile(file: File) procedure needs a File object handed to it. This way, one can be certain that a program won't be capable of doing anything other than what it is requested to do.

What is Capability Transport Protocol (CapTP)?

Similar to OCAP, it is quite difficult to find a gentle introduction to CapTP out there. CapTP takes the cue from OCAP and applies it to security of distributed, possibly networked systems. In CapTP, objects known as actors live in event loops known as vats. There can be one or more vats running on a machine's OS process, vats running on a machine's different processes or different machines entirely.

Actors that live in the same vat are said to be near one another, while those that live in different vats are said to be far from one another. Nearby actors can communicate with one another by synchronously call each other (or immediate call), while faraway actors have to communicate asynchronously with one another through message-sending (or eventual send) and promise-resolving, as you will see in the next examples. In common scenarios, the network boundaries are abstracted away so that the programmers won't have to worry about them. For example, objects living in two different vats on the same machine and in two different vats on two different networked machines should appear analogous to the programmer.

Quick Setup

Before anything else, make sure you have DrRacket installed and the IDE opened. You will need to install Goblins by going to File > Package Manager and type "goblins" in the Package Source box and hit enter to install the library.

Immediately Calling an Object in a Vat

In this first example, we will be creating a car object that knows its location with a 'where method and is able to move to different ones with a 'move-to method. The object will be bootstrapped to live in a vat using an a-run procedure. This way, it is simple to experiment on DrRacket's interactive prompt without having to run full vat programs.

Paste the following code in the DrRacket editor, then hit run:

#lang racket
(require goblins)
(require goblins/actor-lib/bootstrap)

;; Define a vat constructor.
(define a-vat 
  (make-vat))

;; Define a vat runner.
(define-vat-run a-run
  a-vat)

;; Define a car object constructor.
(define (^car bcom [x 0] [y 0])
  (lambda (msg [next-x 0] [next-y 0])
    (cond
      [(eq? msg 'where) (cons x y)]
      [(eq? msg 'move-to) (bcom (^car bcom next-x next-y))])))

Let's take a closer look at the ^car factory procedure (The ^ is called the hard hat symbol; A naming convention for object factories). Without fuzzing too much over bcom ("become"), let's just think of it as a wrapper that creates the next version of the object. As you'll see, when creating an object with ^car, the caller won't have to supply bcom with a value.

The outer ^car procedure is called to spawn or create an object (Note the optionalx and y arguments for the car's location which both default to 0). The procedure returns an anonymous procedure, which takes a symbol message and two optional arguments next-x and next-y. The message acts as a method or instruction within the cond block. Currently, it accepts two messages 'where and 'move-to. The 'where message returns a pair of current x and y coordinate of the car, and 'move-to, along with next-x and next-y arguments, transition the car's location to a new state.

In the interactive prompt, try creating a car object and calling it with messages like this:

;; Create a new car object and assign to `my-car`.
> (define my-car 
    (a-run (spawn ^car))

;; Call the object with 'where symbol.
;; `$` stands for immediate call.
> (a-run ($ my-car 'where))
;; Returns a pair '(0 . 0)

;; Call with 'move-to symbol and a new coordinate.
> (a-run ($ my-car 'move-to 20 30))

;; Calling with 'where again to check the updated location.
> (a-run ($ my-car 'where))
;; '(20 . 30)

;; Move to a different coordinate.
> (a-run ($ my-car 'move-to 3 5))

;; Check the new location.
> (a-run ($ my-car 'where))
;; '(3 . 5)

Note the a-run procedure that was required to "simulate" a running vat in the prompt.

Another way to define the car factory is through a method macro, which is a syntactic sugar to make it more familiar to OOP programmers. The objects created are analogous to the previous ones and can be called in a similar fashion.

;; Import the methods macro.
(require goblins/actor-lib/methods)


(define (^car bcom [x 0] [y 0])
  (methods
    [(where)
      (cons x y)]
    [(move-to next-x next-y)
      (bcom (^car bcom next-x next-y))]))

Eventual Send and Handling Promises in Many Vats

In the case that two object lives in different vats or even on different machines, we invoke them using the eventual send <- procedure instead of immediate call $ procedure.

When we eventually send a message, the program returns a promise immediately without blocking. Then, we call the on procedure with the returned promise and a callback. This shouldn't be a surprise if you have programmed in JavaScript before.

#lang racket
(require goblins)
(require goblins/actor-lib/bootstrap)

;; Define a vat constructor.
(define a-vat 
  (make-vat))

;; Define a vat runner.
(define-vat-run a-run
  a-vat)

;; Define another vat runner.
(define b-vat
  (make-vat))

(define-vat-run b-run
  b-vat)

;; Define a receiver constructor.
;; A receiver either receive a 'ping or 'pong message 
;; from a sender and respond with a contrary message.
(define (^receiver bcom name)
  (lambda (msg)
    (cond
      [(eq? msg 'ping) (format "pong from ~a" name)]
      [(eq? msg 'pong) (format "ping from ~a" name)]
      [else (format "~a doesn't understand you" name)])))

;; Define a sender constructor.
;; A sender take a receiver and message as arguments, then 
;; just immediately call the receiver object with that message.
(define (^sender bcom name)
  (lambda (recipient msg)
    ($ recipient msg)))

Now let's run the new code and return to the interpreter to experiment.

;; Define alice in a-vat.
> (define alice
    (a-run (spawn ^receiver "Alice")))

;; Define bob in b-vat.
> (define bob
    (b-run (spawn ^sender "Bob")))

;; Now Bob wants to send a 'ping message to Alice.
;; Bob being in the b-vat, we call `b-run` to bootstrap the vat.
> (b-run ($ bob alice 'ping))

;; not-callable: Not in the same vat: #<local-object ^receiver>

We got a not-callable: Not in the same vat error because inside the definition of ^sender it immediately calls the recipient with the $ but the recipient is an object that lives in a different vat (a-vat). Recall that objects living in different vats cannot immediately call one another. To prove the point, let's try to immediately call alice from b-vat:

> (b-run ($ alice 'ping))

;; not-callable: Not in the same vat: #<local-object ^receiver>

To fix this, we have to change to eventual send with <-. This call is non-blocking and returns a promise to the caller. Let's try invoking alice directly from b-vat again, but this time with <-:

> (b-run (<- alice 'ping))

;; #<local-promise>

You get a promise in return! From our b-vat land, we can handle a promise with the on procedure, which takes a promise and a callback as arguments. Here is a callback that only prints out the respond from alice:

> (define my-promise
    (b-run (<- alice 'ping)))

> (b-run (on my-promise
             (lambda (respond)
               (displayln respond))))

;; Prints "pong from Alice"

So to fix this just replace the sender definition with the following so a sender in one vat can "remotely" call (send) a receiver in another vat:

(define (^sender bcom name)
  (lambda (recipient msg)
    (<- recipient msg)))

Conclusion

We have learned what OCAP and CapTP are and get our hands dirty coding with Goblins and learning immediate object calls in a single vat and sending messages between objects in different vats. Somehow, we have baredly scratched the surface.

Currently Goblins is in rapid development and there will be changes along the way. Check out the repo for project updates.