Build system for Hack on Windows

Summary: This PR contains a build-system for Hack on windows. It relies on OCPWin (as described in `hphp/hack/README.win32`). This alternative build system may also be used on Linux and OS X (see `hphp/hack/README.ocp-build`).

*Warnings*: on Windows, the resulting binaries are not working yet, all tests are known to fail.

The main file describing the build process is `hphp/hack/src/build.ocp`. It defines a set of libraries and programs. The library architecture has been sketched with our young knownledge about Hack. If someone with more experience could have a look at it, it would be appreciated.

Some additional descriptions specific to Hack (i.e. not required when build Flow) may be found in the following files:

- `hphp/hack/build.ocp`
- `hphp/hack/src/h2tp/build.ocp`

While in the long term it might require some efforts to maintain two build systems, for now we take care not to modify (and not to conflict with) the current build system.
Closes facebook#5775

Reviewed By: @jwatzman

Differential Revision: D2268695

Pulled By: @gabelevi

hphp/hack/.gitignore

@@ -12,3 +12,5 @@ src/parsing/parser.mli
 *.so
 *.annot
 *.opt
+_obuild/
+.ocp/

hphp/hack/00_config.ocp

@@ -0,0 +1,3 @@
+
+have_lz4 = false
+debug = false

hphp/hack/README.ocpbuild

@@ -0,0 +1,45 @@
+This document describes an alternative build-system for Hack. This is
+currently the only build system available on Windows. Meanwhile, this
+document focuses on using this alternative build system on Linux and
+OS X; for Windows, see `README.win32`.
+
+### ocp-build, a build system for OCaml applications
+
+ocp-build is a build system for OCaml applications, based on simple
+descriptions of packages. ocp-build combines the descriptions of
+packages in different directories, and optimize the parallel incremental
+compilation of files depending on the number of cores and the 
+automatically-infered dependencies between source files.
+
+For more details, you can take a look at
+
+   http://www.typerex.org/ocp-build.html
+
+### Download and install `ocp-build` by using OPAM
+
+OPAM is a source-based package manager for OCaml. It supports multiple
+simultaneous compiler installations, flexible package constraints, and
+a Git-friendly development workflow.
+
+To install OPAM, please check http://opam.ocaml.org/doc/Install.html
+
+Once OPAM is installed correctly with an OCaml compiler, you can setup
+your configuration and installl `ocp-build`:
+
+    $ opam init
+    $ eval $(opam config env)
+    $ opam install ocp-build
+
+Note that you will always need to use `eval $(opam config env)` when restarting
+a shell in a terminal, to get your PATH properly configured.
+
+### Compiling Hack with ocp-build
+
+For Windows, see `README.win32.` Otherwise in `hack/src`, run:
+
+  $ make fast
+
+Optionally, you may run:
+
+  $ make test-ocp
+

hphp/hack/README.win32

@@ -0,0 +1,67 @@
+We are currently porting Hack and Flow to Windows 64 bits. This
+document describes the build system, if you want to test
+the prototype. Be aware that this work will take some time
+and most tests are currently known to fail.
+
+### Dependency: OCPWin
+
+OCPWin is a binary OCaml distribution for Windows. It is
+self-contained and runs natively on Windows, which means you can
+generate OCaml applications with it without installing Cygwin or
+Windows SDK.
+
+To download and install OCPWin, go to http://www.typerex.org/ocpwin.html,
+and choose the "full" version. OCPWin installer is known to have problems
+with PATHs exceeding 1024 characters. Be careful to save your PATH before
+the installation to be able to restore it in case of problem, until the
+problem is fixed.
+
+### Configuration and Compilation
+
+In the Hack root directory, edit the file  `00_config.ocp`
+and adjust the two following variables:
+
+    has_lz4 = false (* true if lz4 is installed on your machine *)
+    debug = false   (* true if you want to activate the debug mode *)
+
+NOTE: lz4 is not required to build Hack. If you want to use it, and lz4 is not
+      installed in the default path, you have to set the
+      `ccopt` variable correctly in the `00_config.ocp` file. For instance:
+
+    ccopt = [ "-I/PATH/TO/LZ4/DIRECTORY" ]
+
+Then, execute the `make.bat` script:
+
+    ./make
+
+This will call `ocp-build` and copy the generated files into the bin/
+sub-directory.
+
+If you want to clean your project, you may use:
+
+    ./make clean
+
+Optionally, to run the test-suite adjust the path to the `python.exe`
+in `make.bat`, and run:
+
+    ./make test
+
+### Installing
+
+Coming soon
+
+### FAQ
+
+Q: While installing OCPWin, I got this message error:
+
+    This version of C:\xxxx is not compatible with the version of
+    Windows that you're running. Check your computer's system information
+    and then contact the software publisher.
+
+   What should I do ?
+
+A: You probably installed the wrong version of OCPWin, check that you
+   got the 32 or 64 bits version according to your system. Note that Hack
+   and Flow currently only work on 64 bits systems.
+
+

hphp/hack/build.ocp

@@ -0,0 +1,14 @@
+(**
+ * Copyright (c) 2015, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under the BSD-style license found in the
+ * LICENSE file in the "hack" directory of this source tree. An additional grant
+ * of patent rights can be found in the PATENTS file in the same directory.
+ *
+ *)
+
+begin library "ROOTPROJECT"
+  files = []
+end
+

hphp/hack/make.bat

@@ -0,0 +1,62 @@
+
+@echo off
+
+if "%1" == "" goto build
+if "%1" == "build" goto build
+if "%1" == "all" goto build
+if "%1" == "test" goto test
+if "%1" == "init" goto init
+if "%1" == "clean" goto clean
+REM Invalid argument
+echo Invalid argument, please check README.win32
+goto end
+
+REM initialize ocp-build and create _obuild directory
+:init
+if not exist "_obuild/" ocp-build init
+goto end
+
+REM 1/ check if ocp-build init has already be done
+REM 2/ generate get_build_id.gen.c
+REM 3/ start build hack with ocp-build
+:build
+if not exist "_obuild/" ocp-build init
+ocaml.exe unix.cma ./src/scripts/gen_build_id.ml ./src/utils/get_build_id.gen.c
+ocp-build
+md bin 2>NUL
+copy _obuild\hh_server\hh_server.asm.exe bin\hh_server.exe
+copy _obuild\hh_client\hh_client.asm.exe bin\hh_client.exe
+copy _obuild\hh_single_type_check\hh_single_type_check.asm.exe bin\hh_single_type_check.exe
+copy _obuild\hh_format\hh_format.asm.exe bin\hh_format.exe
+copy _obuild\h2tp\h2tp.asm.exe bin\h2tp.exe
+
+goto end
+
+REM clean _obuild directory and executables in bin/
+:clean
+ocp-build clean
+del bin\hh_server.exe 2>NUL
+del bin\hh_client.exe 2>NUL
+del bin\hh_single_type_check.exe 2>NUL
+del bin\hh_format.exe 2>NUL
+del bin\h2tp.exe 2>NUL
+goto end
+
+REM execute the Hack testsuite
+:test
+@echo on
+
+set "python3=python.exe"
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/autocomplete
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/color
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/coverage
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/dumpsymbolinfo
+"%python3%" test/verify.py --program bin/hh_format.exe test/format
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/suggest
+"%python3%" test/verify.py --program bin/hh_single_type_check.exe test/typecheck
+"%python3%" test/verify.py --program bin/hh_format.exe test/typecheck ^
+		--disabled-extension .no_format ^
+		--out-extension .format_out ^
+		--expect-extension '' ^
+		--flags --root .
+:end

hphp/hack/src/Makefile

@@ -103,6 +103,7 @@ LINKER_FLAGS=$(NATIVE_OBJECT_FILES) $(NATIVE_LIB_OPTS) $(EXTRA_LIB_OPTS) \
 rwildcard=$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
 
 all: build-hhi-archive build-hack copy-hack-files
+all-ocp: build-hhi-archive build-hack-with-ocp copy-hack-files-ocp
 
 clean:
 	ocamlbuild -clean
@@ -111,6 +112,10 @@ clean:
 
 build-hack: $(TARGETS) build-hhi-archive
 
+build-hack-with-ocp: build-hhi-archive build-hack-stubs-with-ocp
+	[ -d ${ROOT}/../_obuild ] || ( cd ${ROOT}/.. && ocp-build init )
+	ocp-build build
+
 # All targets are built in one time, so no parallelization is necessary
 .NOTPARALLEL: $(TARGETS)
 # As there is no efficient way to calculate the dependencies of
@@ -131,9 +136,15 @@ $(TARGETS): $(ALL_SRC_FILES)
 build-hack-native-deps: build-hack-stubs
 	ocamlbuild -cflags "-g $(EXTRA_INCLUDE_OPTS) $(EXTRA_CC_OPTS)" $(NATIVE_OBJECT_FILES)
 
+# If you modify this rule, do not forget to edit the file
+# `$(ROOT)/scripts/gen_build_id.ml` used on Windows.
 build-hack-stubs:
 	REV="const char* const BuildInfo_kRevision = \"$$(git rev-parse HEAD)\";"; \
-	if [ "$$REV" != "$$(cat utils/get_build_id.gen.c)" ]; then echo "$$REV" > utils/get_build_id.gen.c; fi
+	if [ "$$REV" != "$$(cat scripts/get_build_id.gen.c)" ]; then echo "$$REV" > utils/get_build_id.gen.c; fi
+
+build-hack-stubs-with-ocp:
+	ocaml unix.cma $(ROOT)/scripts/gen_build_id.ml \
+                       $(ROOT)/utils/get_build_id.gen.c
 
 build-hhi-archive:
 	mkdir -p ../bin
@@ -152,7 +163,27 @@ endif
 	cp _build/hh_format.native ../bin/hh_format
 	cp _build/h2tp/h2tp.native ../bin/h2tp
 
+copy-hack-files-ocp: build-hack-with-ocp
+	mkdir -p ../bin
+ifeq ($(OS), Linux)
+	objcopy --add-section hhi=../bin/hhi.tar.gz ../_obuild/hh_server/hh_server.asm ../bin/hh_server
+else
+	cp ../_obuild/hh_server/hh_server.asm ../bin/hh_server
+endif
+	cp ../_obuild/hh_client/hh_client.asm ../bin/hh_client
+	cp ../_obuild/hh_single_type_check/hh_single_type_check.asm ../bin/hh_single_type_check
+	cp ../_obuild/hh_format/hh_format.asm ../bin/hh_format
+	cp ../_obuild/h2tp/h2tp.asm ../bin/h2tp
+
+
+.PHONY: test test-ocp do-test
 test: build-hack copy-hack-files
+	${MAKE} do-test
+
+test-ocp: build-hack-with-ocp copy-hack-files-ocp
+	${MAKE} do-test
+
+do-test:
 	python3 ../test/verify.py --program ../bin/hh_single_type_check ../test/autocomplete
 	python3 ../test/verify.py --program ../bin/hh_single_type_check ../test/color
 	python3 ../test/verify.py --program ../bin/hh_single_type_check ../test/coverage