Developer Guide

tabset-1
json JSON
tabset-2
none Text
tabset-3
python Python
php PHP
ruby Ruby
tabset-4
bash Shell
tabset-5
http HTTP
tabset-6
html HTML
tabset-7
javascript JavaScript
tabset-8
xml XML
tabset-9
python Python
ruby Ruby
php PHP
javascript Node
tabset-10
python Python
tabset-11
python Python
ruby Ruby
php PHP
javascript Node
go Go
py Django
tabset-12
json Message

Introduction

Fanout is a publish-subscribe service used for pushing data to browsers, client applications, and servers over the Internet.

There are two primary ways to use Fanout:

Data Messaging: Send JSON objects to recipients using protocols such as Bayeux and XMPP. Recipients may use any compatible receiving libraries. WebSockets are supported automatically when possible. Data Messaging is great for projects where you control both the sending and receiving codebases, and want to get realtime push working in just a few lines of code.

Custom API: Send arbitrary data to recipients via a variety of low-level network transports. Fanout supports five transports: HTTP long-polling, HTTP streaming, WebSockets, Webhooks (outbound HTTP), and XMPP. You use your own domain name and have complete control over the protocol frames exchanged with recipients. Custom API is ideal if you’re creating an API or want to migrate an existing API to Fanout. The Custom API feature is also referred to as GRIP.

QuickStart

If you don’t have a Fanout account yet, go ahead and sign up for free. Once your account is created, sign in to access the Fanout Control Panel. You should see your Realm ID and Realm Key displayed, which you’ll need to use in any sending/receiving code.

Let’s start out by using Fanout Data Messaging to send JSON objects to the browser. First, include the client library in your HTML. Replace {realm-id} with your Realm ID.

<script src="http://{realm-id}.fanoutcdn.com/bayeux/static/faye-browser-min.js"></script>

Then use the following JavaScript to listen for data on a channel called “test”.

var client = new Faye.Client('http://{realm-id}.fanoutcdn.com/bayeux');
client.subscribe('/test', function (data) {
    alert('got data: ' + data);
});

Note that the channel name is prefixed with a slash ‘/’ character. This is required when using Bayeux for connectivity, as we are doing here.

For sending data, use one of Fanout’s publishing libraries. Replace {realm-id} with your Realm ID and {realm-key} with your Realm Key.

# pip install fanout
import fanout

fanout.realm = '{realm-id}'
fanout.key = '{realm-key}'

fanout.publish('test', 'Test publish!')
# gem install fanout
require 'fanout'

fanout = Fanout.new('{realm-id}', '{realm-key}')
fanout.publish_async('test', 'Test publish!')
<?php
// composer require fanout/fanout

$fanout = new Fanout('{realm-id}', '{realm-key}');
$fanout->publish('test', 'Test publish!');

?>
// npm install --save fanoutpub
var fanout = require('fanoutpub');

var fanout = new fanout.Fanout('{realm-id}', '{realm-key}');
fanout.publish('test', 'Test publish!');
// go get github.com/fanout/go-fanout

import "github.com/fanout/go-fanout"
import "encoding/base64"

func publish() {
    // Decode the base64 encoded key:
    decodedKey, err := base64.StdEncoding.DecodeString("{realm-key}")
    if err != nil {
        panic("Failed to base64 decode the key")
    }

    // Pass true as the 3rd parameter for SSL or false for non-SSL:
    fanout := fanout.NewFanout("{realm-id}", decodedKey, true)

    // Publish
    err = fanout.Publish("test", "Test publish!", "", "")
    if err != nil {
        panic("Publish failed with: " + err.Error())
    }
}
# settings.py:

FANOUT_REALM = '{realm-id}'
FANOUT_KEY = '{realm-key}'

# views.py or elsewhere:

# pip install django-fanout
import django_fanout as fanout

fanout.publish('test', 'Test publish!')

You can also test receiving and sending directly from the Fanout Control Panel. Simply sign in and then click on the Push Test Page button. Once loaded, you can use the form on the page to send strings to the “test” channel.

Of course, you can always use curl to publish data, too. At the bottom of the Push Test Page, you’ll see a curl command with an included authentication token, ready to copy-and-paste into a terminal. It’ll look something like this:

curl -H "Authorization: Bearer [...]" \
     -H "Content-Type: application/json" \
     -d '{"items":[{"channel":"test","json-object":"hello world"}]}' \
     https://api.fanout.io/realm/{realm-id}/publish/

Authentication tokens are JSON Web Tokens sent in an Authorization header using the Bearer type. See the Authentication section for details.

Congratulations, you’ve reached the end of the quickstart!

Now what? We suggest:

Or, just read on to learn more about Fanout.

Network Architecture

_images/network.png

In a nutshell, the origin server (your server) publishes messages to Fanout, and Fanout multicasts those messages to one or more subscribed clients. It’s important to note that Fanout communication to clients is unidirectional. If a client wishes to interact with your application or publish data to other clients, it needs to do so by contacting the origin server. The origin server may then publish messages to Fanout on the client’s behalf.

When Fanout Custom API is used with client-initiated transports such as HTTP long-polling, HTTP streaming, and WebSockets, then Fanout operates as a reverse proxy in front of the origin server. In this case, clients may send data through Fanout in order to reach the origin server.

Realms, Channels, Domains

Fanout is organized by realms and channels. A realm is like a channel namespace. Different realms may use the same channel names without conflict. Realms may contain any number of channels. Messages are published to channels, and messages are relayed to the subscribers of each channel for that realm. When you create a Fanout account, one realm will be automatically created for you. You may wish to make additional realms, for example to have a realm for each of your deployment environments. Realms may be optionally labeled with a friendly name in the Fanout Control Panel.

Channels exist on demand. There is no explicit channel creation step. You may subscribe to any channel or publish to any channel at any time. Data delivery happens if any subscribers exist at the time of a publish.

If you are using Fanout Custom API with a client-initiated transport (HTTP long-polling, HTTP streaming, WebSockets) or XMPP, then one or more domain names must be associated with the realm. Each domain name must be set up as a CNAME record pointing to the routing domain specified in the Fanout Control Panel (e.g. {realm-id}.fanoutcdn.com). See the Custom API section for details.

Please note that Data Messaging (used for pushing to browsers) as well as the Custom API Webhook transport do not require setting up any domains.

Data Messaging

Fanout Data Messaging lets you send JSON objects to recipients. There are two ways recipients can listen for data:

  1. Bayeux
  2. XMPP Publish-Subscribe (XEP-0060)

This section will focus on Bayeux, which is the approach we recommend unless you have special requirements. For details on the other delivery mechanism, see JSON over XMPP Publish-Subscribe.

Listening for data in the browser is easy. First, include the client library in your HTML. Replace {realm-id} with your Realm ID.

<script src="http://{realm-id}.fanoutcdn.com/bayeux/static/faye-browser-min.js"></script>

Fanout hosts a recent, unmodified version of the Faye JavaScript client on its servers for your convenience. Of course, you can use your own copy of Faye instead of ours, or any other Bayeux-compatible client library.

Then use the following JavaScript to listen for data. Replace {channel} with the channel you wish to listen on.

var client = new Faye.Client('http://{realm-id}.fanoutcdn.com/bayeux');
client.subscribe('/{channel}', function (data) {
    console.log('got data: ' + data);
});

That’s it! The client will connect in the background, and automatically reconnect as necessary. The Faye client will choose the best transport to use (either WebSocket or HTTP long-polling) based on the browser version and networking conditions.

If you want to know when a subscription is established, the subscribe() call returns a promise that you can use to find out:

var client = new Faye.Client('http://{realm-id}.fanoutcdn.com/bayeux');
client.subscribe('/{channel}', function (data) {
    console.log('got data: ' + data);
}).then(function() {
    console.log('subscribed!');
});

To unsubscribe, keep the return value of subscribe() handy and call cancel() on it:

var client = new Faye.Client('http://{realm-id}.fanoutcdn.com/bayeux');
var sub = client.subscribe('/{channel}', function (data) {
    console.log('got data: ' + data);
});

...

sub.cancel();

For sending data, use one of Fanout’s publishing libraries. Replace {realm-id} with your Realm ID and {realm-key} with your Realm Key.

# pip install fanout
import fanout

fanout.realm = '{realm-id}'
fanout.key = '{realm-key}'

fanout.publish('{channel}', 'Test publish!')
# gem install fanout
require 'fanout'

fanout = Fanout.new('{realm-id}', '{realm-key}')
fanout.publish_async('{channel}', 'Test publish!')
<?php
// composer require fanout/fanout

$fanout = new Fanout('{realm-id}', '{realm-key}');
$fanout->publish('{channel}', 'Test publish!');

?>
// npm install --save fanoutpub
var fanout = require('fanoutpub');

var fanout = new fanout.Fanout('{realm-id}', '{realm-key}');
fanout.publish('{channel}', 'Test publish!');
// go get github.com/fanout/go-fanout

import "github.com/fanout/go-fanout"
import "encoding/base64"

func publish() {
    // Decode the base64 encoded key:
    decodedKey, err := base64.StdEncoding.DecodeString("{realm-key}")
    if err != nil {
        panic("Failed to base64 decode the key")
    }

    // Pass true as the 3rd parameter for SSL or false for non-SSL:
    fanout := fanout.NewFanout("{realm-id}", decodedKey, true)

    // Publish
    err = fanout.Publish("{channel}", "Test publish!", "", "")
    if err != nil {
        panic("Publish failed with: " + err.Error())
    }
}
# settings.py:

FANOUT_REALM = '{realm-id}'
FANOUT_KEY = '{realm-key}'

# views.py or elsewhere:

# pip install django-fanout
import django_fanout as fanout

fanout.publish('{channel}', 'Test publish!')

You also don’t have to use a client library. If you’re using a programming language we don’t support, publishing is just one HTTP POST:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "{channel}",
      "json-object": "Test publish!"
    }
  ]
}

Note: Publish HTTP requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

Custom API

With Fanout Custom API, you can make the Fanout service speak your own API to recipients. This allows you to delegate your realtime infrastructure operations to us without having to compromise on your external interface. You can point your own domain name at Fanout’s servers and control the exact bytes of HTTP and WebSocket packets. Fanout is effectively invisible to your API consumers.

Fanout also makes implementing custom realtime APIs incredibly easy, much easier than writing all the connection management code yourself. The service uses a novel approach that we call the Generic Realtime Intermediary Protocol, and you’ll see the term “GRIP” appear in various places when integrating.

Fanout supports sending data over five low-level transports:

  • HTTP long-polling. You can cause certain incoming HTTP requests to be held open, and publish HTTP responses to those held requests. See the Custom HTTP API section.
  • HTTP streaming. You can cause certain incoming HTTP requests to be held open indefinitely, and publish HTTP response body payloads to those held requests, without the response completing. See the Custom HTTP API section, and also Server-Sent Events.
  • WebSockets. Publish messages to be injected into active WebSocket connections. You can even drive a WebSocket API without having to run a WebSocket server. See the Custom WebSocket API section.
  • WebHooks. Publish HTTP requests, to URLs subscribed to channels. See the Persistent Subscriptions and Webhooks sections.
  • XMPP. Publish XMPP stanzas, to XMPP addresses subscribed to channels. See the Persistent Subscriptions and Pushing XMPP Stanzas sections.

Each transport works a little bit differently and has its own requirements. The HTTP long-polling, HTTP streaming, and WebSocket transports require Fanout to operate as a reverse proxy service in front of your own backend server. These transports as well as the XMPP transport require configuring a domain. As a convenience, Fanout hosts a built-in domain for each realm ({realm-id}.fanoutcdn.com) that can be used for this, or you can add your own custom domains. Notably, the Webhook transport does not require configuring a domain.

Custom HTTP API

To create a Custom API for HTTP, you must configure a domain to route to an origin server that you control. You can use the built-in HTTP domain (e.g. {realm-id}.fanoutcdn.com), or you can add your own custom domain (e.g. realtime.yourcompany.com). Edit the domain and enter an origin server destination (e.g. origin.yourcompany.com:80). An origin server is needed so that you can handle incoming requests from Fanout. If you are using a custom domain, you must then create a CNAME record in your domain’s DNS settings that points to the routing domain displayed in the control panel. Domain names are globally unique, and only one realm at a time may be associated with a particular domain name.

Once you’ve set up a domain, the Fanout service operates as a reverse proxy in front of your origin server:

_images/fanouthttp.png

When a client performs an HTTP request against your domain, the request first goes to Fanout which forwards it as-is to the origin server that you specified. The origin server then tells Fanout either to respond immediately with a normal HTTP response or to hold the HTTP connection open instead. If a connection is held open, then it can be used later on for data pushing. When data is to be pushed over a held connection, the complete HTTP response is specified, headers and all.

The approach is simple but powerful. Fanout handles the operational complexity of running a push service, by holding open potentially thousands of connections on your behalf. Yet, at the same time it gives you the freedom to present your HTTP interface precisely the way you choose.

You can service incoming HTTP requests however you want. Normal responses sent to Fanout will be relayed to clients as-is. To cause a request to be held open and subscribed to channels you need to include special instructions via headers. Include a Grip-Hold header to indicate that a request should be held open. Its value should be either response (for long-polling) or stream (for streaming).

For example, here’s how the origin server should respond to hold a request open as a long-poll, subscribed to “mychannel”:

HTTP/1.1 200 OK
Content-Type: application/json
Grip-Hold: response
Grip-Channel: mychannel

{}

When the Fanout service receives this response from your origin server, it will process it as an instruction rather than relaying it to the client. At this point, the HTTP request/response interaction between Fanout and the origin server is complete, but the HTTP request/response interaction between the client and Fanout remains.

After 55 seconds pass (you can override this with Grip-Timeout), the request will time out and Fanout will send the above response to the client with the Grip-* headers stripped. Here we are basically saying that if there is no data to publish on this channel before the request times out, then the client should receive an empty JSON object.

For example, if the request times out, then the client would see something like this:

HTTP/1.1 200 OK
Content-Type: application/json

{}

Of course, you may specify any content as the timeout response. You can use any headers and the body doesn’t have to be JSON.

Constructing a hold response is easy in your favorite language:

if request.method == 'GET':
    response = HttpResponse('{}\n', content_type='application/json')
    response['Grip-Hold'] = 'response'
    response['Grip-Channel'] = 'mychannel'
    return response
def do_GET(request, response):
    response.status = 200
    response.body = '{}\n'
    response['Content-Type'] = 'application/json'
    response['Grip-Hold'] = 'response'
    response['Grip-Channel'] = 'mychannel'
<?php

header('Content-Type: application/json');
header('Grip-Hold: response');
header('Grip-Channel: mychannel');

?>
{}
http.createServer(function (req, res) {
    res.writeHead(200, {
        'Content-Type': 'application/json',
        'Grip-Hold': 'response',
        'Grip-Channel': 'mychannel'
    });
    res.end('{}\n');
}).listen(80, '0.0.0.0');
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-gripcontrol"
import "net/http"

func HandleRequest(writer http.ResponseWriter, request *http.Request) {
    channel := gripcontrol.CreateGripChannelHeader([]*gripcontrol.Channel {
        &gripcontrol.Channel{Name: "mychannel"}})

    writer.Header().Set("Grip-Hold", "response")
    writer.Header().Set("Grip-Channel", channel)
}

func Listen() {
    http.HandleFunc("/", HandleRequest)
    http.ListenAndServe(":80", nil)
}
# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py:

# pip install django-grip
from django_grip import set_hold_longpoll

if request.method == 'GET':
    set_hold_longpoll(request, 'mychannel')
    return HttpResponse('{}\n', content_type='application/json')

Whenever you want to send data to listening clients, publish an “HTTP response” payload containing the content:

# pip install gripcontrol
from gripcontrol import GripPubControl

pub = GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': b64decode('{realm-key}')})

pub.publish_http_response('mychannel', '{"hello": "world"}\n')
# gem install gripcontrol
require 'gripcontrol'

pub = GripPubControl.new({
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => Base64.decode64('{realm-key}')})

pub.publish_http_response_async('mychannel', '{"hello": "world"}\n')
<?php
// composer require fanout/gripcontrol

$pub = new GripPubControl(array(
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => base64_decode('{realm-key}')));

$pub->publish_http_response('mychannel', '{"hello": "world"}\n');

?>
// npm install --save grip
var grip = require('grip');

var pub = new grip.GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': new Buffer('{realm-key}', 'base64')});

pub.publishHttpResponse('mychannel', '{"hello": "world"}\n');
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-gripcontrol"
import "encoding/base64"

func publish() {
    // Decode the base64 encoded key:
    decodedKey, err := base64.StdEncoding.DecodeString("{realm-key}")
    if err != nil {
        panic("Failed to base64 decode the key")
    }

    pub := gripcontrol.NewGripPubControl([]map[string]interface{} {
        map[string]interface{} {
        "control_uri": "https://api.fanout.io/realm/{realm-id}",
        "control_iss": "{realm-id}",
        "key": decodedKey}})

    // Publish
    err = pub.PublishHttpResponse("mychannel", "{\"hello\": \"world\"}\n", "", "")
    if err != nil {
        panic("Publish failed with: " + err.Error())
    }
}
# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py or elsewhere:

# pip install django-grip
from gripcontrol import HttpResponseFormat
from django_grip import publish

publish('mychannel', HttpResponseFormat('{"hello": "world"}\n'))

If you’re using a programming language we don’t support, publishing is just one HTTP POST:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "mychannel",
      "http-response": {
        "body": "{\"hello\": \"world\"}\n"
      }
    }
  ]
}

Note: Publish HTTP requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

In the above publishing examples, we are specifying the HTTP response body to send to listening clients. The headers in the original instructional response will be merged with this content, minus any Grip-* headers. After publishing data, clients with held requests would receive a response looking like this:

HTTP/1.1 200 OK
Content-Type: application/json

{"hello": "world"}

What’s notable about Fanout’s Custom HTTP API approach is that the client doesn’t directly participate in the publish-subscribe process. For all the client knows, it is simply making an HTTP request and receiving a potentially delayed response. The channels to subscribe to are selected by your origin server, not the client, and the client never actually sees the channels you’re using.

Fanout also supports HTTP streaming. It works similarly to the HTTP long-polling mechanism. The difference is that you provide initial response content at subscription time, and you publish partial HTTP body payloads rather than whole HTTP responses.

Here’s how to hold a request open in stream mode:

if request.method == 'GET':
    response = HttpResponse('Stream opened, prepare yourself!\n',
        content_type='text/plain')
    response['Grip-Hold'] = 'stream'
    response['Grip-Channel'] = 'mychannel'
    return response
def do_GET(request, response):
    response.status = 200
    response['Content-Type'] = 'text/plain'
    response['Grip-Hold'] = 'stream'
    response['Grip-Channel'] = 'mychannel'
    response.body = 'Stream opened, prepare yourself!\n'
<?php

header('Content-Type: text/plain');
header('Grip-Hold: stream');
header('Grip-Channel: mychannel');

?>
Stream opened, prepare yourself!
http.createServer(function (req, res) {
    res.writeHead(200, {
        'Content-Type': 'text/plain',
        'Grip-Hold': 'stream',
        'Grip-Channel': 'mychannel'
    });
    res.end('Stream opened, prepare yourself!\n');
}).listen(80, '0.0.0.0');
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-gripcontrol"
import "net/http"
import "io"

func HandleRequest(writer http.ResponseWriter, request *http.Request) {
    channel := gripcontrol.CreateGripChannelHeader([]*gripcontrol.Channel {
        &gripcontrol.Channel{Name: "mychannel"}})

    writer.Header().Set("Grip-Hold", "stream")
    writer.Header().Set("Grip-Channel", channel)
    io.WriteString(writer, "Stream opened, prepare yourself!\n")
}

func Listen() {
    http.HandleFunc("/", HandleRequest)
    http.ListenAndServe(":80", nil)
}
# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py:

# pip install django-grip
from django_grip import set_hold_stream

if request.method == 'GET':
    set_hold_stream(request, 'mychannel')
    return HttpResponse('Stream opened, prepare yourself!\n',
        content_type='application/json')

When a client makes a request through Fanout and your origin server responds with a hold instruction in stream mode as shown above, then the client will immediately receive an initial response:

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked

Stream opened, prepare yourself!

Note that the Grip-* headers were stripped. Also, the request won’t have Content-Length set, so that it can stay open indefinitely as you publish additional body content.

Whenever you want to send data to listening clients, publish an “HTTP stream” payload containing the content:

# pip install gripcontrol
from gripcontrol import GripPubControl

pub = GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': b64decode('{realm-key}')})

pub.publish_http_stream('mychannel', 'hello there\n')
# gem install gripcontrol
require 'gripcontrol'

pub = GripPubControl.new({
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => Base64.decode64('{realm-key}')})

pub.publish_http_stream_async('mychannel', 'hello there\n')
<?php
// composer require fanout/gripcontrol

$pub = new GripPubControl(array(
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => base64_decode('{realm-key}')));

$pub->publish_http_stream('mychannel', 'hello there\n');

?>
// npm install --save grip
var grip = require('grip');

var pub = new grip.GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': new Buffer('{realm-key}', 'base64')});

pub.publishHttpStream('mychannel', 'hello there\n');
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-gripcontrol"
import "encoding/base64"

func publish() {
    // Decode the base64 encoded key:
    decodedKey, err := base64.StdEncoding.DecodeString("{realm-key}")
    if err != nil {
        panic("Failed to base64 decode the key")
    }

    pub := gripcontrol.NewGripPubControl([]map[string]interface{} {
        map[string]interface{} {
        "control_uri": "https://api.fanout.io/realm/{realm-id}",
        "control_iss": "{realm-id}",
        "key": decodedKey}})

    // Publish
    err = pub.PublishHttpStream("mychannel", "hello there\n", "", "")
    if err != nil {
        panic("Publish failed with: " + err.Error())
    }
}
# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py or elsewhere:

# pip install django-grip
from gripcontrol import HttpStreamFormat
from django_grip import publish

publish('mychannel', HttpStreamFormat('hello there\n'))

If you’re using a programming language we don’t support, publishing is just one HTTP POST:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "mychannel",
      "http-stream": {
        "content": "hello there\n"
      }
    }
  ]
}

Note: Publish HTTP requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

Unlike response holds, stream holds remain open and can be repeatedly pushed to. And as before, the content you send may be of any type. In the above examples, we are publishing plain text lines instead of JSON.

You can also stream data using Server-Sent Events.

Custom WebSocket API

To create a Custom API for WebSockets, you must configure a domain to route to an origin server that you control. You can use the built-in HTTP domain (e.g. {realm-id}.fanoutcdn.com), or you can add your own custom domain (e.g. realtime.yourcompany.com). Edit the domain and enter an origin server destination (e.g. origin.yourcompany.com:80). An origin server is needed so that you can handle incoming requests from Fanout. If you are using a custom domain, you must then create a CNAME record in your domain’s DNS settings that points to the routing domain displayed in the control panel. Domain names are globally unique, and only one realm at a time may be associated with a particular domain name.

Tip: You may use the same domain entry to handle WebSocket connections as well as normal HTTP requests.

Once you’ve set up a domain, the Fanout service operates as a reverse proxy in front of your origin server:

_images/fanoutws.png

When a client opens a WebSocket connection to your domain, the connection request first goes to Fanout. Fanout then opens a WebSocket connection (or uses an emulation over HTTP, see below) to the origin server that you specified. Fanout does not accept the incoming WebSocket connection from the client until its outbound WebSocket connection has been accepted by the origin. Once the connections are established, messages can flow end-to-end in either direction. Using special control messages, you can subscribe connections to channels and then publish WebSocket messages to be injected into those connections.

Fanout communicates with the origin server in one of two modes:

  1. Normal WebSockets: For every WebSocket connection between a client and Fanout, there will be a corresponding WebSocket connection between Fanout and the origin server.
  2. WebSocket-over-HTTP protocol: Instead of using a WebSocket connection to the origin, Fanout encodes WebSocket events into HTTP requests/responses. Events include OPEN, TEXT, PING, PONG, and CLOSE, and are encoded in a format similar to HTTP chunked encoding. You don’t really have to think about the encoding, though, as our client libraries take care of it for you. For details, see the WebSocket-over-HTTP specification.

We recommend using the WebSocket-over-HTTP protocol unless you have a good reason not to use it. The approach is great for stateless application protocols, and the origin server doesn’t have to maintain long-lived connections nor even support WebSockets. The one gotcha is that the origin server can only send spontaneous messages by publishing, but in a stateless protocol this should be expected. The WebSocket-over-HTTP mode is enabled by checking the “Origin Server Uses WebSocket-Over-HTTP Protocol” checkbox in the Fanout control panel.

Having Fanout proxy your WebSocket connections alone is not very interesting. In order to use publish-subscribe, the grip WebSocket extension must be negotiated. Fanout will include a Sec-WebSocket-Extensions request header with this extension, which must be acknowledged in the origin response. You need to do this regardless of whether Fanout is communicating with your server using a normal WebSocket or with the WebSocket-over-HTTP protocol.

For example, a WebSocket connection request might look like this:

GET /path HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ=
Sec-WebSocket-Extensions: grip

In which case the origin server should respond as such:

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Extensions: grip

If Fanout is communicating to the origin server using the WebSocket-over-HTTP protocol, then a connection request might look like the example below. Note that the characters “\r\n” represent a two-byte sequence of carriage return and newline.

POST /path HTTP/1.1
Sec-WebSocket-Extensions: grip
Content-Type: application/websocket-events
Accept: application/websocket-events

OPEN\r\n

The origin server should then respond accordingly. Note that even though this is not a WebSocket connection, the Sec-WebSocket-Extensions header is still used to negotiate GRIP:

HTTP/1.1 200 OK
Sec-WebSocket-Extensions: grip
Content-Type: application/websocket-events

OPEN\r\n

At this point, the proxied WebSocket session is established with GRIP activated. If Fanout has a normal WebSocket connection with the origin server, then Fanout and the origin server may exchange WebSocket messages normally. If Fanout has a WebSocket-over-HTTP session with the origin server, then Fanout and the origin server may exchange WebSocket “events” over HTTP.

When a WebSocket session has GRIP mode activated, messages sent from the origin server to Fanout must be prefixed with either m: (regular message) or c: (control message). A message without a prefix will be ignored by Fanout. The prefixing is needed to disambiguate regular messages from control messages. When Fanout receives a regular message in this way, the prefix is stripped before relaying the message to the client.

It is possible to override the regular message prefix by specifying the message-prefix parameter in the GRIP extension negotiation response. You can even set it to a blank string, allowing you to send normal messages without a prefix, if you’re sure that none of your regular messages will ever begin with c:. For example:

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Extensions: grip; message-prefix=""

Control messages are formatted as a JSON object following the c: prefix. The object has a type field that indicates the type of control message. All other fields of the object depend on the type.

There are three possible control messages:

  • subscribe: Subscribe connection to the channel specified by the channel field.
  • unsubscribe: Unsubscribe connection from the channel specified by the channel field.
  • detach: Terminate the session between Fanout and the origin server, but retain the connection between the client and Fanout. Any messages Fanout receives from the client will be dropped.

Here’s how the origin server would subscribe a connection to a channel called “mychannel”:

c:{"type": "subscribe", "channel": "mychannel"}

If you are using WebSocket-over-HTTP communication, then the message must be wrapped in a TEXT event and sent in an HTTP response:

HTTP/1.1 200 OK
Content-Type: application/websocket-events

TEXT 2F\r\n
c:{"type": "subscribe", "channel": "mychannel"}\r\n

Accepting a WebSocket-over-HTTP connection and subscribing it to a channel is easy in your favorite language. Our client libraries take care of encoding events for you:

# pip install gripcontrol
from gripcontrol import WebSocketEvent, decode_websocket_events, \
    encode_websocket_events, websocket_control_message

if request.method == 'POST':
    in_events = decode_websocket_events(request.body)
    out_events = []
    if in_events[0].type == 'OPEN':
        out_events.append(WebSocketEvent('OPEN'))
        out_events.append(WebSocketEvent('TEXT', 'c:' +
            websocket_control_message('subscribe', {'channel': 'mychannel'})))
    resp = HttpResponse(encode_websocket_events(out_events),
        content_type='application/websocket-events')
    resp['Sec-WebSocket-Extensions'] = 'grip'
    return resp
# gem install gripcontrol
require 'gripcontrol'

def do_POST(request, response):
    response.status = 200
    response['Sec-WebSocket-Extensions'] = 'grip'
    response['Content-Type'] = 'application/websocket-events'

    in_events = GripControl.decode_websocket_events(request.body)
    out_events = []
    if in_events[0].type == 'OPEN'
        out_events.push(WebSocketEvent.new('OPEN'))
        out_events.push(WebSocketEvent.new('TEXT', 'c:' +
            GripControl.websocket_control_message('subscribe',
            {'channel' => 'mychannel'})))
    response.body = GripControl.encode_websocket_events(out_events)
<?php
// composer require fanout/gripcontrol

$in_events = GripControl::decode_websocket_events(
    file_get_contents("php://input"));
$out_events = array();
if ($in_events[0]->type == 'OPEN')
{
    $out_events[] = new WebSocketEvent('OPEN');
    $out_events[] = new WebSocketEvent('TEXT', 'c:' .
        GripControl::websocket_control_message('subscribe',
        array('channel' => 'mychannel')));
}

header('Content-Type: application/websocket-events');
header('Sec-WebSocket-Extensions: grip');
http_response_code(200);

echo GripControl::encode_websocket_events($out_events);

?>
// npm install --save grip
var grip = require('grip');

http.createServer(function (req, res) {
    res.writeHead(200, {
        'Sec-WebSocket-Extensions': 'grip',
        'Content-Type': 'application/websocket-events'
    });

    var body = '';
    req.on('data', function (chunk) {
        body += chunk;
    });

    req.on('end', function() {
        var inEvents = grip.decodeWebSocketEvents(body);
        var outEvents = [];
        if (inEvents[0].getType() == 'OPEN') {
            outEvents.push(new grip.WebSocketEvent('OPEN'));
            outEvents.push(new grip.WebSocketEvent('TEXT', 'c:' +
                grip.webSocketControlMessage('subscribe',
                {'channel': 'mychannel'})));
        }

        res.end(grip.encodeWebSocketEvents(outEvents));
    });
}).listen(80, '0.0.0.0');
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-gripcontrol"
import "io"
import "io/ioutil"
import "net/http"

func HandleRequest(writer http.ResponseWriter, request *http.Request) {
    writer.Header().Set("Sec-WebSocket-Extensions", "grip")
    writer.Header().Set("Content-Type", "application/websocket-events")

    body, _ := ioutil.ReadAll(request.Body)
    inEvents, err := gripcontrol.DecodeWebSocketEvents(string(body))
    if err != nil {
        panic("Failed to decode WebSocket events: " + err.Error())
    }

    if inEvents[0].Type == "OPEN" {
        // Create the WebSocket control message:
        wsControlMessage, err := gripcontrol.WebSocketControlMessage("subscribe",
            map[string]interface{} { "channel": "mychannel" })
        if err != nil {
            panic("Unable to create control message: " + err.Error())
        }

        // Open the WebSocket and subscribe it to a channel:
        outEvents := []*gripcontrol.WebSocketEvent {
            &gripcontrol.WebSocketEvent { Type: "OPEN" },
            &gripcontrol.WebSocketEvent { Type: "TEXT",
                Content: "c:" + wsControlMessage }}
        io.WriteString(writer, gripcontrol.EncodeWebSocketEvents(outEvents))
    }
}

func Listen() {
    http.HandleFunc("/", HandleRequest)
    http.ListenAndServe(":80", nil)
}
# pip install django-grip

# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py:

def endpoint(request):
    # middleware provides a pseudo-socket object
    ws = request.wscontext

    # if this is a new connection, accept it and subscribe it to a channel
    if ws.is_opening():
        ws.accept()
        ws.subscribe('mychannel')

    # here we loop over any messages
    while ws.can_recv():
        message = ws.recv()

        # if return value is None, then the connection is closed
        if message is None:
            # ack the close
            ws.close()
            break

    # return an empty response, which middleware will fill with events
    return HttpResponse()

Whenever you want to send data to listening clients, publish a “WebSocket message” payload containing the content:

# pip install gripcontrol
from pubcontrol import Item
from gripcontrol import GripPubControl, WebSocketMessageFormat

pub = GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': b64decode('{realm-key}')})

item = Item(WebSocketMessageFormat('{"hello": "world"}'))
pub.publish('mychannel', item)
# gem install gripcontrol
require 'pubcontrol'
require 'gripcontrol'

pub = GripPubControl.new({
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => Base64.decode64('{realm-key}')})

item = Item.new(WebSocketMessageFormat.new('{"hello": "world"}'))
pub.publish_async('mychannel', item)
<?php
// composer require fanout/gripcontrol

$pub = new GripPubControl(array(
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => base64_decode('{realm-key}')));

$item = new Item(new WebSocketMessageFormat('{"hello": "world"}'));
$pub->publish('mychannel', $item);

?>
// npm install --save grip
var pubcontrol = require('pubcontrol');
var grip = require('grip');

var pub = new grip.GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': new Buffer('{realm-key}', 'base64')});

var item = new pubcontrol.Item(new grip.WebSocketMessageFormat('{"hello": "world"}'));
pub.publish('mychannel', item);
// go get github.com/fanout/go-gripcontrol

import "github.com/fanout/go-pubcontrol"
import "github.com/fanout/go-gripcontrol"
import "encoding/base64"

func publish() {
    // Decode the base64 encoded key:
    decodedKey, err := base64.StdEncoding.DecodeString("{realm-key}")
    if err != nil {
        panic("Failed to base64 decode the key")
    }

    pub := gripcontrol.NewGripPubControl([]map[string]interface{} {
        map[string]interface{} {
        "control_uri": "https://api.fanout.io/realm/{realm-id}",
        "control_iss": "{realm-id}",
        "key": decodedKey}})

    // Publish
    format := &gripcontrol.WebSocketMessageFormat {
        Content: []byte("{\"hello\": \"world\"}") }
    item := pubcontrol.NewItem([]pubcontrol.Formatter{format}, "", "")
    err = pub.Publish("mychannel", item)
    if err != nil {
        panic("Publish failed with: " + err.Error())
    }
}
# settings.py:

GRIP_PROXIES = [
    {
        'control_uri': 'https://api.fanout.io/realm/{realm-id}',
        'control_iss': '{realm-id}',
        'key': b64decode('{realm-key}')
    }
]

MIDDLEWARE_CLASSES = (
    'django_grip.GripMiddleware',
    ...
)

# views.py or elsewhere:

# pip install django-grip
from gripcontrol import WebSocketMessageFormat
from django_grip import publish

publish('mychannel', WebSocketMessageFormat('{"hello": "world"}'))

If you’re using a programming language we don’t support, publishing is just one HTTP POST:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "mychannel",
      "ws-message": {
        "content": "{\"hello\": \"world\"}\n"
      }
    }
  ]
}

Note: Publish HTTP requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

Authentication

HTTP API calls are authenticated using a JSON Web Token. The token must be signed using the HMAC SHA-256 algorithm using the realm’s key that was provided to you. Its claims object must contain an iss value set to the Realm ID and an exp value indicating token expiration.

Below is an example JWT structure and sample resulting token.

JWT Header:

{"alg": "HS256", "typ": "JWT"}

JWT Claim:

{"exp": 1300819380, "iss": "{realm-id}"}

Token: (linebreaks added for readability)

eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.IntcImV4cFwiOiAxMzAwOD
E5MzgwXCIsIFwiaXNzXCI6IFwieW91cmNvbXBhbnlcIn0i.7JAj_xYUrVdAymWY
YJLhClHZvjiAtdorLLlqf-mpWDg

Example code to build JSON Web Token:

import time
from base64 import b64decode
import jwt

key = b64decode('{realm-key}')
claim = {'iss': '{realm-id}', 'exp': int(time.time()) + 3600}
token = jwt.encode(claim, key)
include 'JWT.php';

$key = base64_decode('{realm-key}');
$claim = array('iss' => '{realm-id}', 'exp' => time() + 3600);
$token = JWT::encode($claim, $key);
require 'jwt'

key = Base64.decode64('{realm-key}')
claim = {'iss' => '{realm-id}', 'exp' => Time.now.to_i + 3600}
token = JWT.encode(claim, key)

Python example uses pyjwt. PHP example uses php-jwt. Ruby example uses ruby-jwt.

You can also download this handy python program: create_fanout_token.py. It requires pyjwt, which you can install with pip:

sudo pip install pyjwt

You can then make tokens like this:

python create_fanout_token.py {realm-id} {realm-key}

The generated token is used in the Authorization HTTP Header. For example:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.IntcImV4cFw...
Content-Type: application/json

{
  "...": "etc"
}

(header trimmed for brevity)

With the above create_fanout_token.py script, you can even make calls via curl:

export TOKEN=`python create_fanout_token.py {realm-id} {realm-key}`
curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json"
  -d '... content ...' https://api.fanout.io/realm/{realm-id}/publish/

(text wrapped for readability)

Persistent Subscriptions

The incoming connection oriented HTTP and WebSocket transports maintain temporary subscriptions to channels that last only as long as their associated connections. However, other transports such as Webhooks and XMPP require persistent subscriptions to be made in advance. You must use API calls to subscribe URLs or XMPP addresses to channels as necessary.

In the examples below, replace {realm-id} with your Realm ID and {channel} with the channel of interest.

Subscribing a URL:

POST /realm/{realm-id}/channel/{channel}/hrq-subscriptions/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/x-www-form-urlencoded

url=http:%2F%2Fexample.com%2Fpath%2F

Unsubscribing a URL:

DELETE /realm/{realm-id}/channel/{channel}/hrq-subscription/http:%2F%2Fexample.com%2Fpath%2F/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

Subscribing an XMPP address:

POST /realm/{realm-id}/channel/{channel}/xs-subscriptions/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/x-www-form-urlencoded

jid=user%40example.com

Unsubscribing an XMPP address:

DELETE /realm/{realm-id}/channel/{channel}/xs-subscription/user%40example.com/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

Server-Sent Events

Fanout Custom API makes it easy to implement Server-Sent Events. Set up a domain and origin server as described in the Custom HTTP API section, and then use the following guidelines for holding and publishing.

On the origin server, respond with hold instructions using stream mode and content type text/event-stream:

if request.method == 'GET':
    response = HttpResponse(content_type='text/event-stream')
    response['Grip-Hold'] = 'stream'
    response['Grip-Channel'] = 'mychannel'
    return response
def do_GET(request, response):
    response.status = 200
    response['Content-Type'] = 'text/event-stream'
    response['Grip-Hold'] = 'stream'
    response['Grip-Channel'] = 'mychannel'
<?php

header('Content-Type: text/event-stream');
header('Grip-Hold: stream');
header('Grip-Channel: mychannel');

?>
http.createServer(function (req, res) {
    res.writeHead(200, {
        'Content-Type': 'text/event-stream',
        'Grip-Hold': 'stream',
        'Grip-Channel': 'mychannel'
    });
    res.end();
}).listen(80, '0.0.0.0');

Then, whenever you want to send data, publish an HTTP stream payload containing SSE formatting:

# pip install gripcontrol
from gripcontrol import GripPubControl

pub = GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': b64decode('{realm-key}')})

pub.publish_http_stream('mychannel', 'event: update\ndata: hello world\n\n')
# gem install gripcontrol
require 'gripcontrol'

pub = GripPubControl.new({
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => Base64.decode64('{realm-key}')})

pub.publish_http_stream_async('mychannel', 'event: update\ndata: hello world\n\n')
<?php
// composer require fanout/gripcontrol

$pub = new GripPubControl(array(
    'control_uri' => 'https://api.fanout.io/realm/{realm-id}',
    'control_iss' => '{realm-id}',
    'key' => base64_decode('{realm-key}')));

$pub->publish_http_stream('mychannel', 'event: update\ndata: hello world\n\n');

?>
// npm install --save grip
var pub = new grip.GripPubControl({
    'control_uri': 'https://api.fanout.io/realm/{realm-id}',
    'control_iss': '{realm-id}',
    'key': new Buffer('{realm-key}', 'base64')});

pub.publishHttpStream('mychannel', 'event: update\ndata: hello world\n\n');

Pushing XMPP Stanzas

The XMPP transport requires setting up a domain. This is needed in order for XMPP dialback authentication to work. You can optionally configure an origin server as well, if you wish to be able to receive incoming traffic on this domain. If no origin server is specified, then Fanout will respond to any stanzas sent to the domain with an item-not-found error. The XMPP addresses to push to must be set up in advance. See Persistent Subscriptions.

To push a stanza to all subscribed XMPP addresses of a channel:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "{channel},
      "xmpp-stanza": {
        "content": "<message xmlns=\"jabber:client\" from=\"user@yourcompany.com\">
                    <body>hello</body></message>"
      }
    }
  ]
}

(text wrapped for readability)

The stanza must be valid XML and use the jabber:client namespace. It is not necessary to provide a to address, since it will be filled in differently for each address that the stanza is relayed to. If supplied, it will be overwritten.

Note: Publish requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

Webhooks

This is also known as the HTTP request transport. It does not require setting up any kind of domain, since connections are only ever made outbound. The URLs to push to must be set up in advance. See Persistent Subscriptions.

To push an HTTP request to all subscribed URLs of a channel:

POST /realm/{realm-id}/publish/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/json

{
  "items": [
    {
      "channel": "{channel}",
      "http-request": {
        "method": "POST",
        "body": "{ \"foo\": \"bar\" }"
      }
    }
  ]
}

The domain or path of the request is not supplied since these values come from the list of subscribed URLs for the channel. HTTP responses are ignored by Fanout.

Note: Publish requests must be authenticated. See Authentication to learn how to provide an authentication token using your Realm Key.

Accessing the API via XMPP

The entire Fanout API is available via both HTTP and XMPP. In the case of XMPP there are two interfaces available: ad-hoc commands and chat bot. In order to use the XMPP API, you must first whitelist at least one XMPP address that is allowed to access the service. The XMPP address set in your Fanout account settings profile is automatically whitelisted. You can also add any number of additional XMPP addresses via the Fanout API.

Shown below is how to add the XMPP address using the HTTP API. Once this is done then you can access the XMPP API from that address.

Adding an admin address:

POST /realm/{realm-id}/admins/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]
Content-Type: application/x-www-form-urlencoded

jid=user%40example.com

Once your XMPP address is whitelisted, you can add the chat bot to your contact list. The chat bot’s address uses the form admin-{realm}@api.fanout.io where {realm-id} should be replaced with your Realm ID. To check out the ad-hoc commands interface, use service discovery to browse api.fanout.io.

You can also revoke admin rights of the address:

DELETE /realm/{realm-id}/admin/user%40example.com/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

JSON over XMPP Publish-Subscribe

Fanout Data Messaging, which is primarily used to push JSON to browsers and other clients, is also capable of publishing to arbitrary XMPP addresses. This can be ideal for pushing messages to native XMPP applications, or if you desire a more powerful access method from within the browser. An XMPP PubSub service is offered at the address pubsub.fanout.io with nodes of the form /{realm-id}/{channel}. Any XMPP address from any server may subscribe to Fanout’s PubSub service.

For example, suppose your XMPP address is alice@example.com and you want to subscribe to the messages of channel test, for Realm ID 12345678. First, send presence to the service, as only temporary subscriptions are supported at this time:

<presence to="pubsub.fanout.io"/>

Now subscribe:

<iq to="pubsub.fanout.io" type="set" id="sub1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <subscribe node="/12345678/test" jid="alice@example.com"/>
  </pubsub>
</iq>

Success response:

<iq from="pubsub.fanout.io" to="alice@example.com/1" type="result" id="sub1">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <subscription
        node="/12345678/test"
        subscription="subscribed"
        subid="6196c125-8976-40ab-8fd8-5fd8dc9c0377"
        jid="alice@example.com"/>
  </pubsub>
</iq>

Now, whenever a JSON object is published via Data Messaging, a PubSub event will be sent to any XMPP subscribers:

<message from="pubsub.fanout.io" to="alice@example.com" type="headline">
  <event xmlns="http://jabber.org/protocol/pubsub#event">
    <items node="/12345678/test">
      <item id="3ebc205b-ce77-46c3-843a-49e203e7945b">
        <json xmlns="urn:xmpp:json:0">
          {"foo": "bar"}
        </json>
      </item>
    </items>
  </event>
</message>

As temporary subscriptions are used, subscriptions are cleared whenever a subscriber address goes offline.

Data Messaging Custom Domains

By default, Bayeux service is available at base URL http://{realm-id}.fanoutcdn.com/bayeux. However, this service can also be used with your own custom domain names.

To use your own domain, add a domain in the Domains section of the Fanout Control Panel, then edit the domain and set the Bayeux Mount Point to a path that you want the service to be accessible from. For example, you could add the domain realtime.yourcompany.com with Bayeux Mount Point /bayeux, and the Bayeux service would be accessible via base URL http://realtime.yourcompany.com/bayeux.

If a domain also has an Origin Server specified, then the Bayeux Mount Point will take precedence. Only requests made to paths outside of the mount point will be proxied for GRIP usage.

You can even specify / as the Bayeux Mount Point, so that the Bayeux base URL simply becomes the domain itself (e.g. http://realtime.yourcompany.com/). Note that if you do this, it won’t be possible to use the domain for GRIP.

Message Size

By default, messages are limited to 65,535 bytes for the “content” portion of the format being published. For Data Messaging, content size is the JSON object serialized to UTF-8 (for example, {"foo":"bar"} would be counted as 13 bytes when receiving and delivering, even if additional framing is added when delivering; also please allow for variance that may occur during JSON serialization within Fanout Cloud). For Custom API, content size is the number of HTTP body bytes or WebSocket message bytes (TEXT frames converted to UTF-8).

The message limit may be increased by contacting info@fanout.io. Note that messages are billed for every 65,536 byte block in the message (including a partial block). If your limit is increased and you publish a 150,000 byte message, it will be billed as three messages received and then three messages for each subscriber it is delivered to.

If you need to send a large payload to a client, we recommend publishing a reference to the data rather than using large messages. For example, you could publish a message containing a URL that the client can then retrieve separately.

Subscription Feeds

It is possible to discover which channels have subscribers via the subscription feed for the realm. You can query for the current list of subscriptions as well as listen for updates in realtime. Please note that this feature does not work yet for recipients listening via XMPP or Webhooks, but it should work for all other connectivity mechanisms.

Here’s how to query for the list of subscriptions:

GET /realm/{realm-id}/subscriptions/items/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

The server will return items, if any:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "items": [
    {
      "state": "unsubscribed",
      "channel": "news-1"
    },
    {
      "state": "subscribed",
      "channel": "news-2"
    }
  ],
  "last_cursor": "MTM5MDUwNjQyN19hMTZfMTM5MDQ5NjI1Nl8wXzkyNTA1MDU3OA=="
}

Each item consists of a channel name and a state value. The state is either “subscribed” or “unsubscribed”. Items representing unsubscribed channels are included to enable synchronization.

The /items/ endpoint will return a maximum of 50 items. The last_cursor value can be used in a subsequent response to receive the next page of items. Pass it as a value to the since query parameter, with prefix “cursor:”.

Querying for the next page:

GET /realm/{realm-id}/subscriptions/items/?since=cursor%3AMTM5MD[...]U3OA%3D%3D HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

Server response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "items": [
    {
      "state": "subscribed",
      "channel": "news-3"
    }
  ],
  "last_cursor": "MTM5MDUwNjQyN19hMTZfMTM5MDQ5NjI1Nl8wXzkyNTA1MDU3OA=="
}

You may continue to page through the entire result set by taking the last_cursor of the latest response and passing it into the next request.

To detect for changes to the subscription feed in realtime, make a request to the /items/ endpoint with the wait query parameter set to “true”:

GET /realm/{realm-id}/subscriptions/items/?since=cursor%3AMTM5MD[...]U3OA%3D%3D&wait=true HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

If this request would normally return an empty list of items, the request will instead hang open until an item becomes available.

It is also possible to receive changes to the feed in realtime via an HTTP streaming endpoint:

GET /realm/{realm-id}/subscriptions/stream/ HTTP/1.1
Host: api.fanout.io
Authorization: Bearer [... auth token ...]

{"cursor": "MTM5MD[...]MyOTg=", "item": {"state": "subscribed", "channel": "news-1"},
"prev_cursor": "MTM5MD[...]U3OA=="}
{"cursor": "MTM5MD[...]MyOTg=", "item": {"state": "unsubscribed", "channel": "news-1"},
"prev_cursor": "MTM5MD[...]MyOTg="}
...

With this endpoint, the request will remain open indefinitely as items are streamed down the response. Each line of the response is a JSON object containing the item as well as cursor information.

Security and SSL

Fanout supports SSL for nearly all methods of connectivity. The built-in realm domains (i.e. {realm-id}.fanoutcdn.com) can be accessed with either http or https schemes. This works for both the <script> includes as well as the URLs passed to clients (e.g. https://{realm-id}.fanoutcdn.com/bayeux).

For custom domains, it is possible to upload your own certificate to be used for SSL. Fanout supports TLS Server Name Indication (SNI) so that a dedicated IP address is not needed for SSL. However, some browsers do not support this (notably old versions of Internet Explorer), so if you really do need a dedicated IP address please contact info@fanout.io and one can be provisioned for your account.

For Custom APIs that route to origin servers, it is possible to specify origin server settings for SSL and non-SSL traffic. If both are provided, then Fanout will use whichever one matches the scheme of an incoming request.

URLs subscribed to receive Webhooks may use either http or https.

XMPP transports use TLS for server-to-server communication, but without certificate validation as it is not a standard practice. Contact us if you require stronger security here.

Support

Please do not hesitate to contact info@fanout.io with any questions.

Happy pushing!