Skip to content

Commit 7146c24

Browse files
committed
I think appearance and text are nearing completion'
1 parent 4ad3425 commit 7146c24

File tree

8 files changed

+188
-203
lines changed

8 files changed

+188
-203
lines changed

example/Makefile

+5-8
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ SHELL := /bin/bash
22
PANDOC:=pandoc --normalize --smart --write=html5+yaml_metadata_block+footnotes --read=markdown --standalone --section-divs -V title:scuttlebutt --template ./template.html
33
BROWSERIFY:=browserify -d js/index.js
44

5-
html/index.html: index.md html/bundle.js html/style.css html/assets/
5+
html/index.html: index.md html/bundle.js html/style.css html/assets/ add_containers.js
66
echo '<script type="text/javascript" src="./bundle.js" async></script>' > html/bundle.html
7-
$(PANDOC) --include-in-header=html/bundle.html -c style.css index.md > html/index.html
7+
$(PANDOC) --include-in-header=html/bundle.html -c style.css index.md | ./add_containers.js > html/index.html
88
rm html/bundle.html
99

1010
html/bundle.js: js/* ../lib/gossip.js
@@ -28,12 +28,9 @@ install:
2828

2929
prod:
3030
$(PANDOC) -V include-after:"<script class=bundle></script>" -V header-includes:"<style class=style></style>" index.md > html/temp.html
31-
node add-assets.js > html/index.html
32-
cp html/index.html ../.git/index.html
33-
cp -r html/assets ../.git/assets.html
31+
cp -r html/ ../.git/html
3432
git checkout gh-pages
35-
cp ../.git/index.html ../index.html
36-
cp -r ../.git/assets.html ../assets/
37-
git add ../index.html ../assets
33+
cp -r ../.git/html/* ..
34+
git add assets/* index.html bundle.js style.css
3835
git commit -a -m "update"
3936

example/add-assets.js

-16
This file was deleted.

example/index.md

+76-67
Original file line numberDiff line numberDiff line change
@@ -2,96 +2,97 @@
22
% **[A Scuttlebutt Demo][scuttlebutt]**
33
% with [d3 force directed layouts](https://github.com/mbostock/d3/wiki/Force-Layout) and node.js
44

5-
<aside>For best viewing, use chrome, opera or safari.</aside>
5+
<aside class="admonition">For best viewing, use chrome, opera or safari.</aside>
66

7-
## What is it? [&sect;](#history-and-compaction)##
7+
##What is it?##
88

9-
The [Scuttlebutt Gossip protocol][scuttlebutt] is a method for propagating
10-
state, which we take to be a mapping from keys to values, across a network. In
11-
general, the network could be any distributed system-- computers distributed in
12-
space, processes in a single computer, or as is the case here, svg polygons
13-
(![inline-ten](./assets/ten.svg)) in the DOM.
9+
This is a demo of *Simple-Scuttle*, a [Javascript implementation][simple-scuttle]
10+
of the Scuttlebutt gossip protocol as it is described in [van Renesse et
11+
al.][scuttlebutt]. *Simple-Scuttle* builds on the
12+
[node.js](http://nodejs.org/) core library, leveraging node streams[^stream] to
13+
manage data over time, meaning it plays well with other elements of node
14+
core, like `http` or `tcp`.
1415

15-
The examples here rely on a [Javascript implementation][simple-scuttle], called
16-
*Simple-Scuttle*, of the gossip protocol. *Simple-Scuttle* defines a prototype
17-
which inherits from node.js streams[^stream]. The brightly colored force directed graphs are visualizations of the protocol in action.
16+
[Scuttlebutt][scuttlebutt] is a protocol for flow control and efficient
17+
reconciliation-- meaning it propagates information across a network, and does
18+
it well. In general, the network could be any distributed system-- computers
19+
distributed in space, processes in a single machine, or as is the case here,
20+
svg polygons (![inline-ten](./assets/ten.svg)) in the DOM.
1821

19-
## Visualization Details ##
20-
21-
The accompanying visualization of the protocol uses polygons,
22-
![pair](./assets/pair.svg), ![ten](./assets/ten.svg), or ![twenty](./assets/twenty.svg),
23-
to for nodes in the network. The network propagates the number of times each
24-
node has been clicked. The mapping from node id to click count is
25-
the state in this instance.
26-
27-
Each polygon keeps track of the number of times *it* has been clicked, but must
28-
learn about the number of times *every other* polygon has been clicked through
29-
gossip. The polygon has a representation as an object in the DOM with a
30-
*Simple-Scuttle* class instance attribute, which has been set through the DOM's
31-
javascript api.
22+
Polygons, ![pair](./assets/pair.svg), ![ten](./assets/ten.svg), or
23+
![twenty](./assets/twenty.svg) represent nodes in the network. In the toy
24+
examples here, each node is responsible for a single value - the number of
25+
times it has been clicked - and each node reports that value to the nodes with
26+
which it shares an edge.
3227

3328
**Click on a node to update its state!**
3429

3530
The state at each node is represented by the polygon's shape- there is one
36-
point per node in the network, so ![pair](./assets/pair.svg) describes a network
37-
with two nodes in it, and ![ten](./assets/ten.svg) a network with ten. When the
38-
user clicks on a node, its corresponding point distends,
31+
point per node in the network, so ![pair](./assets/pair.svg) describes the
32+
initial the state of a network with two nodes in it, and
33+
![ten](./assets/ten.svg) a network with ten. When the user clicks on a node,
34+
its corresponding point distends,
3935

40-
####so after a few clicks, ![full](./assets/pair.svg) might turn into ![full](./assets/distended-pair.svg).####
36+
<aside class=inline>so after a few clicks, ![full](./assets/pair.svg) might turn into ![full](./assets/distended-pair.svg).</aside>
4137

42-
Edges between ![ten](./assets/ten.svg) indicate the *Simple-Scuttle* attatched to
43-
the dom element can share information about their state with one another. There
44-
is no other way one node can learn the state of another.
45-
46-
## The Protocol [&sect;](#history-and-compaction) ##
38+
## The Protocol##
4739

4840
*I recommend reading the [paper][scuttlebutt] for the full story, including a
49-
discussion of it's performance characteristics.*
41+
discussion of Scuttlebutt's performance characteristics.*
5042

5143
There are essentially three data structures to Scuttlebutt-- some sort of store
52-
for the state, a [vector clock](http://npm.im/vector-clock-class) which helps
44+
for the state, a [vector clock](http://npm.im/vector-clock-class), which helps
5345
determine what updates to ask for, and a structure for replaying, compacting,
5446
and holding on to
5547
[history](https://github.com/AWinterman/simple-scuttle/blob/master/lib/history.js).
56-
Additionally, there is an ettiquette for how gossip should happen.
48+
Additionally, there is an etiquette for how gossip should happen.
49+
50+
The vector clock allows sensible application of version numbers to all updates,
51+
and ensures that updates received by a node can be at least partially ordered
52+
by a `precedes` relation, which allows quick exchange about the most recent
53+
information you have seen. See [the next section](#the-vector-clock) for more
54+
detail on vector clocks
5755

58-
The vector clock ensures that updates received by a node can be at least
59-
partially ordered by a `precedes` relation, which allows quick exchange about
60-
the most recent information you have seen. The history structure ensures
61-
that you can tell your peer the news when you have heard something they have
62-
not.
56+
The history data structure keeps track of new updates as they come in, and
57+
allows them to be replayed on request.
6358

6459
Gossip between two peers begins by exchanging vector clocks-- each peer sends
6560
the other a list of highest version numbered update they've seen from everybody
6661
in the network (including themselves).
6762

6863
Suppose ![pair](./assets/pair.svg) sends its vector clock to
69-
![red-pair](./assets/red-pair.svg). In that list there's the pair
70-
(![red-pair](./assets/red-pair.svg), 10). ![red-pair](./assets/red-pair.svg) responds with
71-
all the updates it has seen locally which are greater than 10, ordering them in
72-
chronological order. If there are more updates than fit in bandwidth, then are
73-
simply omitted until the next round of communication, when the two nodes will
74-
begin where they left off.
64+
![red-pair](./assets/red-pair.svg). In that list there's the two-ple
65+
(![red-pair](./assets/red-pair.svg), 10), so ![red-pair](./assets/red-pair.svg)
66+
responds with all the updates it has heard about from itself (e.g. local
67+
updates) with version numbers greater than 10, ordering them in chronological order.
68+
![red-pair](./assets/red-pair.svg) sends them one at a time until it has sent them
69+
all, or exceeded the its bandwidth. The next time
70+
![red-pair](./assets/red-pair.svg) and ![pair](./assets/pair.svg) gossip, they will
71+
again exchange vector clocks, which will ensure that they neither repeate
72+
themselves nor leave anything out. This extends directly to a more complicated
73+
network, since gossip is always pairwise.
7574

7675
Scuttlebutt is cpu and network efficient, and **eventually** consistent.
7776

78-
## Versioning [&sect;](#conflicts)##
77+
## The Vector Clock##
7978

80-
How do peers decided when to apply updates? How do they know an update occuring
81-
elsewhere happened before their own update? Scuttlebutt applies a partial
82-
ordering on updates by means of a [a vector clock][vector clock], described in
83-
full in [Lamport 1978][vector clock]. It works, more or less as follows:
79+
How do we version updates occuring across a distributed network? How does one
80+
node tell an update coming from a peer occured before a local update?
8481

85-
Suppose peers `A`, `B` must exhange updates. They will each maintain a vector
86-
of logical times for each peer. Logical time refers to the number of events the
87-
peer has seen. This vector is called the `clock`, and is updated according to
88-
the following two rules (IR1 and IR2 from [the paper][vector clock]):
82+
Scuttlebutt [partially orders](partial-ordering) updates by means of a [a vector
83+
clock][vector clock], described in full in [Lamport 1978][vector clock]. It
84+
works, more or less as follows:
8985

90-
1. Each peer must update it's own entry in the list between any two updates
91-
2. If A sends an update to B, it must also send along the logical time, `t`, at which the update occured.
92-
Upon receive the update, B updates A's entry in its own clock to `t`, and then ensures its own
93-
entry in its clock is greater than `t`.
86+
Suppose nodes `A`, `B` must exhange updates. Each node maintains a vector,
87+
called the `clock` of logical times[^logical-time] for each node in the
88+
network. The `clock` is updated according to the following two rules (IR1 and
89+
IR2 from [the paper][vector clock]):
9490

91+
1. Each peer must update it's own entry in the vector between any two updates
92+
2. If A sends an update to B, it must also send along the logical time, `t`, at
93+
which the update occured. Upon receive the update, B updates A's entry in
94+
its own clock to `t`, and then ensures its own entry in its clock is greater
95+
than `t`.
9596

9697
In the beginning, they each maintain a vector clock that looks like this:
9798

@@ -107,11 +108,11 @@ A : [1, 0]
107108
B : [0, 0]
108109
```
109110

110-
When A gossips with B, it sends an update along with the version number at
111-
which the update occured, in this case `1`. B applies the
112-
update since it has a version number since `1` is higher than the most recent
113-
time it has seen. It also increments its own version number to be one higher
114-
than A's. Note that no event is marked with time 2.
111+
When `A` gossips with `B`, it sends an update along with the version number at
112+
which the update occured, in this case `1`. `B` updates the entry in the clock
113+
corresponding to `A`. It also increments its own entry in its clock to be one
114+
higher than `A`'s. Note that no individual update is marked with time 2, and
115+
none will be.
115116

116117
```
117118
A : [1, 0]
@@ -133,7 +134,10 @@ A: [4, 3]
133134
B: [1, 3]
134135
```
135136

136-
## Conflict ##
137+
## Conflict
138+
139+
How do peers decided when to apply updates? In general, tehre is no simple
140+
answer here. Different concerns will necessitate different update rules.
137141

138142
What happens if B encounters another local update?
139143

@@ -180,7 +184,7 @@ constructor function called
180184
Conceivably, you could implement your own clock synchronization algorithm, and
181185
plug in its results here.
182186

183-
## Relation to [npm.im/scuttlebutt][] and _[van Renesse et al.][scuttlebutt]_ [&sect;](#relation-to-npm.imscuttlebutt-and-van-renesse-et-al.scuttlebutt) ##
187+
## Relation to [npm.im/scuttlebutt][] and [*van Renesse et al.*][scuttlebutt]
184188

185189
My implementation, and consequently this module, was inspired by [Dominic
186190
Tarr's scuttlebut module][npm.im/scuttlebutt], which, though totally awesome, I
@@ -197,12 +201,16 @@ course to the restrictions imposed by the format and language (javascript
197201
rather than maths), although once I had internalized the concepts this ceased
198202
to be a conscious effort.
199203

200-
201204
[^stream]: [Node Streams][node streams] abstractions built into the node core
202205
library for handling data over time. They present a unix-like api which allows
203206
one to write to sinks, read from sources, and pipe sources to sinks. They are
204207
available in the browser via [browserify][].&nbsp;
205208

209+
[^logical-time]: Logical time, as distinct from physical time, is essentially
210+
the count of events witnessed by the node. Physical time is somewhat
211+
similar&mdash; seconds count the number of times a physical clock's second hand
212+
ticked, or the number of particles emitted from an atom.
213+
206214
[^system-time]: If anyone has contact information for the author, or is able to
207215
grant legitimate free access to the paper, please [contact
208216
me](https://twitter.com/andywinterman).&nbsp;
@@ -218,4 +226,5 @@ me](https://twitter.com/andywinterman).&nbsp;
218226
[system time]: http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=1091674&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D1091674
219227
[node streams]: http://nodejs.org/api/stream.html
220228
[browserify]: http://browserify.org/
229+
[partial-ordering]: http://en.wikipedia.org/wiki/Partially_ordered_set
221230

example/js/Demo.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,11 @@ function translate(d) {
4040
}
4141

4242
Demo.prototype.dim = function() {
43+
console.log(this.el)
4344
var rect = this.el.getBoundingClientRect()
4445
this.width = rect.width
4546
this.height = rect.height
4647

47-
4848
if(!this.height || !this.width) {
4949
return false
5050
}

example/js/index.js

+16-31
Original file line numberDiff line numberDiff line change
@@ -27,40 +27,25 @@ ticker.on('data', function(framerate) {
2727
var sections = sizzle('section')
2828
, article = document.querySelector('article')
2929

30-
for(var i = 0; i < sections.length; ++i) {
31-
sections[i].outerHTML = '<section class="container">' + sections[i].outerHTML + '</section>'
32-
}
30+
sections = sizzle('.container .example')
3331

34-
sections = sizzle('.container')
35-
36-
var pair = create_demo('example pair')
37-
var pair_conflict = create_demo('example pair conflict')
38-
var ring = create_demo('example ring')
39-
var ring_conflict = create_demo('example ring conflict')
40-
var random = create_demo('example random')
41-
var random2 = create_demo('example random')
42-
43-
sections[0].appendChild(pair)
44-
sections[1].appendChild(ring)
45-
sections[2].appendChild(random2)
46-
sections[3].appendChild(random)
47-
sections[4].appendChild(ring_conflict)
48-
sections[5].appendChild(pair_conflict)
32+
var networks = [
33+
new networks.Ring(10)
34+
, new networks.Pair()
35+
, new networks.Random(20)
36+
, new networks.PairConflict()
37+
, new networks.RingConflict(10)
38+
]
4939

5040
setTimeout(function() {
51-
interaction(pair, new networks.Pair())
52-
interaction(pair_conflict, new networks.PairConflict())
53-
interaction(ring, new networks.Ring(10))
54-
interaction(ring_conflict, new networks.RingConflict(10))
55-
interaction(random, new networks.Random(20))
56-
interaction(random2, new networks.Random(10))
41+
for(var i = 0, len = Math.max(networks.length, sections.length); i < len; ++i) {
42+
var method
43+
if(i == 1) {
44+
method = 'sequential'
45+
}
46+
47+
interaction(sections[i], networks[i], method)
48+
}
5749
}, 1)
5850

59-
function create_demo(classname) {
60-
var demo = document.createElement('section')
61-
62-
demo.className = classname
63-
demo.innerHTML = '<svg></svg>'
6451

65-
return demo
66-
}

example/js/interaction.js

+11-6
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ module.exports = interaction
33
var Demo = require('./Demo')
44
, debounce = require('debounce')
55

6-
function interaction(el, network) {
6+
function interaction(el, network, gossip_type) {
77
var demo = new Demo(sizzle('svg', el)[0], network)
88

99
var going = false
@@ -21,21 +21,26 @@ function interaction(el, network) {
2121
window.addEventListener('scroll', debounce(onscroll))
2222
window.addEventListener('resize', debounce(resize))
2323

24-
setInterval(gossip_once, 200)
24+
setInterval(gossip_once.bind(gossip_type), 200)
2525

2626
function resize() {
2727
if(!going) {
28-
demo.force.resume()
2928
demo.dim()
30-
demo.force.stop()
3129
}
3230

3331
demo.dim()
3432
demo.force.resume()
3533
}
3634

37-
function gossip_once() {
38-
var i = Math.floor(Math.random() * demo.n)
35+
function gossip_once(type) {
36+
var i = 0
37+
38+
if(!type || type === 'random') {
39+
i = Math.floor(Math.random() * demo.n)
40+
} else if (type === 'sequential') {
41+
i = (i + 1) % demo.n
42+
}
43+
3944

4045
demo.node[0][i].__data__.gossip.gossip()
4146
}

example/package.json

+3-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@
88
"d3": "~3.3.10",
99
"debounce": "0.0.3",
1010
"fps": "0.0.3",
11-
"simple-scuttle": "~1.0.0"
11+
"simple-scuttle": "~1.0.0",
12+
"trumpet": "~1.6.3",
13+
"concat-stream": "~1.2.1"
1214
},
1315
"devDependencies": {
1416
"hyperstream": "~0.4.0",

0 commit comments

Comments
 (0)