Pipes 2.5.0 release.

Author: okram

CHANGELOG.textile

@@ -5,13 +5,13 @@ h2. Pipes 2.y.z
 
 !https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-2.png!
 
-h3. Version 2.5.0 (NOT OFFICIALLY RELEASED YET)
+h3. Version 2.5.0 (April 14, 2014)
 
 ```xml
 <dependency>
   <groupId>com.tinkerpop</groupId>
   <artifactId>pipes</artifactId>
-  <version>2.5.0-SNAPSHOT</version>
+  <version>2.5.0</version>
 </dependency>
 ```
 

doc/Acknowledgments.textile

@@ -0,0 +1,10 @@
+<img width="100" src="https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-character-2.png"/>
+
+This section provides a list of the people that have contributed in some way to the creation of Pipes.
+
+# "Marko A. Rodriguez":http://markorodriguez.com -- designed, developed, tested, and documented Pipes.
+# "Darrick Weibe":http://github.com/pangloss -- implemented Pipe paths.
+# "Zach Cox":http://theza.ch - helped develop the fluent model.
+# "Ketrina Yim":http://www.ketrinayim.com/ -- designed the Pipes logo.
+
+Please review Pipes' "pom.xml":http://github.com/tinkerpop/pipes/blob/master/pom.xml. Pipes would not be possible without the work done by others to create these useful packages.

doc/Basic-Pipes.textile

@@ -0,0 +1,52 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-plumber.png!
+
+A @Pipe<S,E>@ is a Java interface that extends both the @Iterable<E>@ and @Iterator<E>@ interface. A @Pipe<S,E>@ takes, as input, an iterator or iterable yielding objects of type @S@ and produces/emits objects of type @E@. The character "S" stands for "starts" and the character "E" stands for "ends". 
+
+Here is a simple example demonstrating a single pipe that capitalizes the characters of the strings that come into it.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+capsPipe.setStarts(Arrays.asList("this", "is", "the", "end."));
+while(capsPipe.hasNext()) {
+  System.out.print(capPipe.next() + " ");
+}
+```
+
+This pipe will produce the following output.
+
+bc. THIS IS THE END.
+
+Given that @Pipe<S,E>@ extends @Iterator<E>@ and @Iterable<E>@, its possible to string together pipes to create a processing line.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+Pipe<String,Integer> countPipe = new CountPipe();
+capsPipe.setStarts(Arrays.asList("this", "is", "the", "end."));
+countPipe.setStarts(capsPipe);
+while(countPipe.hasNext()) {
+  System.out.print(countPipe.next() + " ");
+}
+```
+
+If @CountPipe@ takes a @String@ and emits the number of characters in that @String@, then the previous code will yield the following output.
+
+bc. 4 2 3 4
+
+Realize that the output of one pipe must be of the same type as the input of the next pipe. Given that @Pipe<S,E>@ extends @Iterator<E>@, The @E@ of the first pipe becomes the @S@ of the second pipe. In order to make it easier to create chains of pipes, there is a handy @Pipeline<S,E>@ class. This class implements @Pipe<S,E>@ and thus, @Pipeline<S,E>@ objects can be combined like any other pipe (i.e. you can create pipelines of pipelines). Here is an example using a @Pipeline<S,E>@ object.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+Pipe<String,Integer> countPipe = new CountPipe();
+Pipe<Integer,String> wordPipe = new WordPipe();
+Pipeline<String,String> pipeline = new Pipeline<String,String>(capsPipe, countPipe, wordPipe);
+pipeline.setStarts(Arrays.asList("this", "is", "the", "end."));
+while(pipeline.hasNext()) {
+  System.out.print(pipeline.next() + " ");
+}
+```
+
+Assuming that @WordPipe@ emits the word version of an incoming integer, the pipeline will produce the following output.
+
+bc. four two three four
+
+That's Pipes in a nutshell.

doc/Creating-a-Pipe.textile

@@ -0,0 +1,54 @@
+[[https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-3.png]]
+
+The pipes that come prepackaged with Pipes are not sufficient for every use case. In many situations, it will be necessary to create a pipe that will solve a particular mapping from an input to an output. An easy way to accomplish this is via the respective @XXXFunctionPipe@ and @PipeFunction@ implementation. However, it is possible to write a specific pipe without using @XXXFunctionPipe@-based pipes.
+
+h2. Transform-Based Pipe with AbstractPipe
+
+Pipes make it easy to create new pipes with the @AbstractPipe@ class. In @AbstractPipe@, there is a single method that needs to be implemented: @AbstractPipe.processNextStart()@. The example below demonstrates the internals of a simple pipe that maps a string to the number of characters contained in that string.
+
+```java
+public class WordLengthPipe extends AbstractPipe<String, Integer> implements TransformPipe<String,Integer> {
+  public Integer processNextStart() {
+    String start = this.starts.next();
+    return start.length();
+  }
+}
+```
+
+h2. Filter-Based Pipe with AbstractPipe
+
+The general pattern for filter-based pipes is explained with an example.
+
+```java
+public class WordFilterPipe extends AbstractPipe<String,String> implements FilterPipe<String> {
+  public String processNextStarts() {
+    while(true) {
+       String start = this.starts.next();
+       if(start.length() > 4)
+          return start;
+    }
+  }
+}
+```
+
+A @FilterPipe@ will usually make use of a @while(true)@ loop that will continually pull in new objects. If the current object meets some criteria, it will be emitted, else, the @while(true)@ will loop again and pull in a new object. There is no need to worry about handling @NoSuchElementExceptions@. If @this.starts.next()@ throws a @NoSuchElementException@ then this will be handled appropriately by @AbstractPipe@.
+
+h2. SideEffect-Based Pipe with AbstractPipe
+
+Below is an example of a side-effect based pipe using @AbstractPipe@.
+
+```java
+public class WordPrintPipe extends AbstractPipe<String,String> implements SideEffectPipe<String,Object> {
+  public String processNextStarts() {
+     String start = this.starts.next();
+     System.out.println(start);
+     return start; 
+  }
+
+  public Object getSideEffect() {
+     return null;
+  }
+}
+```
+
+Given that the side-effect is simply a @println@ to the @System.out@, there is no @Object@ side-effect that can be later retrieved. As such, the @SideEffectPipe.getSideEffect()@ method simply returns @null@.

doc/Filter-Pipes.textile

@@ -0,0 +1,109 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/duck-plumber.jpg!
+
+A common Pipes pattern is the filtering of objects. A filter-based pipe will consume objects and either emit them or not. If they are not emitted, then they are considered filtered by the pipe. A useful interface to implement that describes this behavior is @FilterPipe<S> extends Pipe<S,S>@.
+
+h2. Generic Filter Pipe
+
+The generic filter pipe is @FilterFunctionPipe@. A @FilterFunctionPipe@ takes a @PipeFunction@ (see [[Pipe Types]]) that computes on @S@ and either emits it or not. An example @PipeFunction@ is provided below:
+
+```java
+public class CharCountPipeFunction implements PipeFunction<String,Boolean> {
+  
+  private final int number;
+  
+  public CharCountPipeFunction(int number) {
+    this.number = number;
+  }
+
+  public Boolean compute(String argument) {
+     return argument.length() == this.number;
+  }
+}
+```
+
+When put in the context of a @FilterFunctionPipe@, the code looks as follows:
+
+```java
+Pipe<String, String> pipe = new FilterFunctionPipe<String>(new CharCountPipeFunction(4));
+pipe.setStarts(Arrays.asList("tell", "me", "your", "name"));
+// the results of the iteration are: "tell", "your", "name"
+```
+
+h2. Basic Filtering
+
+The @RandomFilterPipe@ comes with the Pipes distribution. @RandomFilterPipe@ will only allow a consumed object to be emitted if a biased coin toss lands on "heads." At the extremes, if @bias@ is 0.0 then no incoming objects are emitted and if @bias@ is 1.0, then every incoming object is emitted.
+
+```java
+public class RandomFilterPipe<S> extends AbstractPipe<S, S> implements FilterPipe<S> {
+    private static final Random RANDOM = new Random();
+    private final double bias;
+    public SampleFilterPipe(final double bias) {
+        this.bias = bias;
+    }
+    protected S processNextStart() {
+        while (this.starts.hasNext()) {
+            S s = this.starts.next();
+            if (bias >= RANDOM.nextDouble()) {
+                return s;
+            }
+        }
+        throw new NoSuchElementException();
+    }
+}
+```
+
+The @processNextStart()@ method structure above is a common pattern in filter-based pipes. In short, a @while(this.starts.hasNext())@ is evaluated until an incoming start object meets the criteria and is returned (i.e. emitted). If there are no more starts and the criteria is not met, then a @NoSuchElementException@ is thrown. Beware of using recursion to accomplish a similar behavior. As the amount of data grows that is streaming through, recursion-based methods can easily yield a @StackOverflowError@.
+
+h2. Comparison-Based Filtering
+
+More complicated filters can be created than what was presented above. These filters usually check objects to determine whether to emit them or not. There is an interface called @ComparisonFilterPipe<S,T>@. Implementations of this interface consume objects of type @S@. If a check of that object passes (i.e. return @true@), then the @S@ object is emitted, else it is filtered. @ComparisonFilterPipe@ has the following signature:
+
+```java
+public interface ComparisonFilterPipe<S, T> extends FilterPipe<S> {
+    public enum Filter {
+        EQUAL, NOT_EQUAL, GREATER_THAN, LESS_THAN, GREATER_THAN_EQUAL, LESS_THAN_EQUAL
+    }
+    public boolean compareObjects(T leftObject, T rightObject);
+}
+```
+
+The important method is @compareObjects()@. This method returns @true@ if the left hand object is @EQUAL@, @NOT_EQUAL@, etc. to the right hand object.
+
+Next, there is an abstract class called @AbstractComparisonFilterPipe<S,T>@ that implements @ComparisonFilterPipe@. More specifically, it provides a standard implementation of @ComparisonFilterPipe.compareObjects()@. For most situations, this method is sufficient. However, if not, simply override the method with an implementation that meets the requirements of the designed pipe.
+
+```java
+public boolean compareObjects(final T leftObject, final T rightObject) {
+        switch (this.filter) {
+            case EQUAL:
+                if (null == leftObject)
+                    return rightObject == null;
+                return leftObject.equals(rightObject);
+            case NOT_EQUAL:
+                if (null == leftObject)
+                    return rightObject != null;
+                return !leftObject.equals(rightObject);
+            case GREATER_THAN:
+                if (null == leftObject || rightObject == null)
+                    return false;
+                return ((Comparable) leftObject).compareTo(rightObject) == 1;
+            case LESS_THAN:
+                if (null == leftObject || rightObject == null)
+                    return false;
+                return ((Comparable) leftObject).compareTo(rightObject) == -1;
+            case GREATER_THAN_EQUAL:
+                if (null == leftObject || rightObject == null)
+                    return false;
+                return ((Comparable) leftObject).compareTo(rightObject) >= 0;
+            case LESS_THAN_EQUAL:
+                if (null == leftObject || rightObject == null)
+                    return false;
+                return ((Comparable) leftObject).compareTo(rightObject) <= 0;
+            default:
+                throw new RuntimeException("Invalid state as no valid filter was provided");
+        }
+    }
+```
+
+Note: it many situations, such as @ObjectFilterPipe@, the right hand object to allow or disallow is stored in the pipe and compared with each object passed through it.
+
+!http://github.com/tinkerpop/pipes/raw/master/doc/images/filter-example.png!

doc/Home.textile

@@ -0,0 +1,58 @@
+[[https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-logo.png|height=149px]]
+
+Pipes is a "dataflow":http://en.wikipedia.org/wiki/Dataflow_programming framework using "process graphs":http://en.wikipedia.org/wiki/Kahn_process_networks. A process graph is composed of @Pipe@ vertices connected by communication edges. A @Pipe@ implements a simple computational step that can be composed with other @Pipe@ objects to create a larger computation. Such data flow graphs allow for the splitting, merging, looping, and in general, the transformation of data from input to output. 
+
+There are numerous Pipe classes that come with the main Pipes distribution. Once a good understanding of each Pipe accomplished, then using the framework is straightforward. In general, the best way to learn about all the Pipes provided is through the project JavaDoc. 
+
+Please join the Gremlin users group at "http://groups.google.com/group/gremlin-users":http://groups.google.com/group/gremlin-users for all "TinkerPop":http://tinkerpop.com related discussions.
+
+Pipes JavaDoc: "2.5.0":http://tinkerpop.com/docs/javadocs/pipes/2.5.0/ - "2.4.0":http://tinkerpop.com/docs/javadocs/pipes/2.4.0/ - "2.3.0":http://tinkerpop.com/docs/javadocs/pipes/2.3.0/ - "2.2.0":http://tinkerpop.com/docs/javadocs/pipes/2.2.0/ - "2.1.0":http://tinkerpop.com/docs/javadocs/pipes/2.1.0/ - "2.0.0":http://tinkerpop.com/docs/javadocs/pipes/2.0.0/ - "1.0":http://tinkerpop.com/maven2/com/tinkerpop/pipes/1.0/api/ - "0.9":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.9/api/ -  "0.8":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.8/api/ - "0.7":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.7/api/ - "0.6":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.6/api/ - "0.5":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.5/api/ - "0.4":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.4/api/ - "0.3":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.3/api/ - "0.2":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.2/api/ - "0.1":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.1/api/ 
+Pipes WikiDocs: "2.5.0":http://tinkerpop.com/docs/wikidocs/pipes/2.5.0/Home.html - "2.4.0":http://tinkerpop.com/docs/wikidocs/pipes/2.4.0/Home.html - "2.3.0":http://tinkerpop.com/docs/wikidocs/pipes/2.3.0/Home.html - "2.2.0":http://tinkerpop.com/docs/wikidocs/pipes/2.2.0/Home.html - "2.1.0":http://tinkerpop.com/docs/wikidocs/pipes/2.1.0/Home.html - "2.0.0":http://tinkerpop.com/docs/wikidocs/pipes/2.0.0/Home.html
+
+```xml
+<dependency>
+  <groupId>com.tinkerpop</groupId>
+  <artifactId>pipes</artifactId>
+  <version>2.5.0</version>
+</dependency>
+```
+
+Non-Maven users can get the raw release jars from Apache's "Central Repository":http://maven.org.  Snapshots can be obtained from "Sonatype":https://oss.sonatype.org/content/repositories/snapshots/  (see "Maven Repositories":https://github.com/tinkerpop/homepage/wiki/Maven-Repositories for more information).
+
+```java
+Pipe<String,Integer> pipeline = new Pipeline<String,Integer>(
+  new RemoveCharacterPipe("o"), 
+  new CountCharactersPipe());
+pipeline.setStarts(Arrays.asList("marko","josh","peter"));
+for(Integer number : pipeline) {
+  System.out.println(number);
+}
+
+4
+3
+5
+```
+
+"On the Nature of Pipes":http://markorodriguez.com/2011/08/03/on-the-nature-of-pipes/ (*Graphical Presentation of the Pipe Mechanics*)
+
+==<hr/>==
+
+* Basic Concepts
+  ** [[Basic Pipes]]
+  ** [[Pipe Types]]
+    *** [[Transform Pipes]]
+    *** [[Filter Pipes]]
+    *** [[SideEffect Pipes]]
+  ** Pipe SubTypes
+    *** [[MetaPipes]]
+    *** Branch Pipes
+* Advanced Concepts
+  ** [[Creating a Pipe]]
+  ** [[Transformation Paths]]
+* Conclusion
+  ** [[Acknowledgments]]
+  ** [[Release Notes]]
+
+==<hr/>==
+
+fn1. Pipes documentation is up to date with the current Pipes "codebase":http://github.com/tinkerpop/pipes/tree/master, not with the latest Pipes release.

doc/MetaPipes.textile

@@ -0,0 +1,52 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-3.png!
+
+A metapipe is a pipe that "wraps" another pipe. MetaPipes are useful in that they can make decisions based upon the behavior of the pipe(s) they wrap. There are numerous metapipes and this section will discuss a few of the more interesting cases.
+
+h2. Basic Pattern
+
+In many situations, the basic pattern used when creating a metapipe is to have the constructor of the metapipe take @n@ number of pipes. Then for each incoming @S@ object to the metapipe, it pushes that @S@ object to its @n@ pipes and makes a emit decision based upon how the pipes behave. In many cases, you can think of a metapipe as an "overseer" of its internal pipes. This idea is presented in the typical code pattern found in metapipes.
+
+```java
+public E processNextStart() {
+  S s = this.starts.next();
+  for (Pipe<S, E> pipe : this.pipes) {
+    pipe.setStarts(new SingleIterator<S>(s));
+    E e = pipe.next();
+    // decide about what to do given the output of the internal pipe
+  }
+  // if all pipes behave as desired, perhaps do:
+  return s.getE();
+}
+```
+
+@SingleIterator<S>@ is a handy class that wraps an object for a single legal @next()@. This class is much more efficient than doing @Arrays.asList(s).iterator()@.
+
+h2. Pipelines
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipeline-example.png!
+
+A @Pipeline@ is a commonly used pipe. A @Pipeline<S,E>@ implements @Pipe<S,E>@ and as such, a @Pipeline@ is simply a @Pipe@. A @Pipeline@ takes an ordered list of pipes in its constructor. It connects these pipes, whereby the input of pipe @n@ is connected to the output of pipe @n-1@. Note that the output type of pipe @n-1@ must be the same type as the input to pipe @n@. This idea is elucidated in the diagram above.
+
+The benefit of using a @Pipeline@ is that is greatly reduces the complexity of a process and allows for easy reuse. It is analogous, in many ways, to creating a function to wrap a body of code/instructions. As such, the function serves as a blackbox with input and output.
+
+h2. And/Or Pipes
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/andfilterpipe-example.png!
+
+There are two logical @FilterPipe@ pipes called @AndFilterPipe<S>@ and @OrFilterPipe<S>@. These pipes take an object of type @S@ and emit an object of type @S@. However, they only emit the @S@ object if the collection of @Pipe<S,Boolean>@ pipes that they wrap return @true@. For @AndFilterPipe@ to emit its incoming @S@ object, all of its wrapped pipes must return @true@ that @S@ object. For the @OrFilterPipe@, only one of its wrapped pipes must return @true@. If you want to see if a number is greater than or equal to 1 *and* less than 10, then use the following @AndFilterPipe@.
+
+```java
+Pipe<Integer,Integer> pipeA = new ObjectFilterPipe<Integer>(1, ComparisonFilterPipe.GREATER_THAN_EQUAL);
+Pipe<Integer,Integer> pipeB = new ObjectFilterPipe<Integer>(10, ComparisonFilterPipe.LESS_THAN);
+Pipe<Integer,Integer> pipe1 = new AndFilterPipe<Integer>(new HasNextPipe<Integer>(pipeA), new HasNextPipe<Integer>(pipeB));
+pipe1.setStarts(Arrays.asList(1,22,10,136,7,2,67));
+while(pipe1.hasNext()) {
+  System.out.print(pipe1.next() + "...");
+}
+```
+
+The @System.out@ of the previous code is as follows.
+
+bc. 1...7...2...
+
+Note the use of @HasNextPipe<S>@ which implements @Pipe<S,Boolean>@. If the @ObjectFilterPipe@ pipes that compose the @AndFilterPipe@ filter their incoming @S@ object, then they will return false on a call to @hasNext()@. Thus, @HasNextPipe@ is useful for determining if a filter pipe has filtered an object.