Skip to content

Latest commit

 

History

History
52 lines (47 loc) · 4.62 KB

README.md

File metadata and controls

52 lines (47 loc) · 4.62 KB

Hydra

Welcome to the Hydra project! Hydra is a unique functional programming language based on the LambdaGraph data model. It explores an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, programs are graphs, and graphs are programs. Hydra has the ability to translate its own kernel into several other languages, including Haskell (Hydra-Haskell), Java (Hydra-Java), and Python (Hydra-Python; in progress). Hydra is used for data modeling, validation, and transforms at Microsoft, while its closed-source predecessor Dragon was used for data integration and graph construction at Uber. The language is now being developed for its own sake, with the intention of becoming an Apache Incubator project, integrating more directly with Apache TinkerPop, and branching out into additional concrete programming languages. Typical use cases include:

  • Graph construction. Hydra supports TinkerPop-style property graphs as well as RDF and SHACL, and has been used in combination with the ISO/IEC GQL standard. Hydra provides DSLs for defining schemas and mappings, as well as tools for validating schemas and data, and moving them seamlessly into and out of the graph formats.

  • Data integration. Hydra includes "coders" (encoders+decoders) for many data and schema languages which you can easily compose together to build data transform pipelines. Some of the currently supported languages and formats include Protobuf, Avro, JSON and YAML, RDF formats including N-Triples, GraphQL, LinkedIn's PDL Schema language, as well as simple tabular data (CSV/TSV). Hydra has been used extensively with (Delta) Parquet, although this support is not currently open source. There is also limited support for C Sharp and Scala (Hydra-Scala).

  • Computational graphs. Unusually among graph data models and query languages, Hydra has deep support for polymorphism, as well as embedding computational elements with a graph (sometimes called computational knowledge graphs). As we mentioned, programs are graphs and vice versa. See the KGC 2024 presentation Graphs, logics, and lambda calculus for examples.

One of the less exciting, but very important aspects of Hydra is its test suite, which guarantees parity across the supported programming languages. This ensures that each Hydra implementation behaves the same, and we think it will be very useful in heterogeneous environments like TinkerPop where we need exactly the same logic -- validation, query steps, user-defined functions, etc. -- to be manifested identically in more than one programming language.

If any of the above sounds interesting, feel free to ask questions or get involved via the LambdaGraph Discord server. Near-term goals of the project include what we call "closing the loop" (completely folding Hydra's kernel into the Hydra DSL), finishing Hydra-Python and updating Hydra-Java, and getting Hydra out into the community. There is a recent demo video you can check out, as well as a couple of earlier presentations here and here, and the original design document here.

Share and enjoy.