Could this adapted to work with srclib?

[lilith]

Perhaps someone here knows enough about both pfff and srclib to help.

Would it be possible to make pff work as a srclib toolchain backend?

[aryx]

Sure. I exchanged a few emails with the people from sourcegraph.com and srclib and I think it's totally doable. One could take the codegraph file generated by pfff and generate the appropriate metadata srclib expects.
The codegraph file format is very simple, see https://github.com/facebook/pfff/blob/master/graph_code/graph_code.mli

[aryx]

One could also to the reverse, use srclib object file to generate the codegraph file so one could use some of the pfff frontend tools (codemap, codegraph, codequery, etc) on the languages supported by srclib (Go, python, ruby).

[lilith]

Awesome!

[aryx]

@sqs who I think was the person who contacted me a few months ago

[sqs]

I'm happy to help with any efforts to share stuff between srclib and pfff. (I'm one of the srclib creators.) Let me know how I could help.

The current srclib schema is defined in Go structs (that serialize to JSON with eponymous fields) at, e.g., https://srclib.org/toolchains/grapher-output/. We'll have a JSON Schema or similar schema soon. As a first version, just getting the names, paths, and file/byte locations would be a great start.

[phooji]

@nathanaeljones Do you have a particular language that you're interested in?

[lilith]

I'm trying to add C# and F# support. I assume that F# support (given its OCaml roots) wouldn't be a long stretch?

[lilith]

My larger goal is to combine generated docs and annotated, literate source browsing.

Literate source browsing, like docco style.

Input data needed would seem to include {filesystem, srclib output, extracted docs, class/package/unit structure, and some optional manual components}.

I'm unclear on whether pfff or srclib can help with doc extraction, but it's certainly an easier parsing task. The structure is another thing I'm still working on analyzing.

The idea of giving a tool a list of method or class names, and letting it generate step-by-step source-code walkthroughs is kind of seductive.

[aryx]

Regarding srclib data generation, I think the problem is language independent. The codegraph file (graph_code.marshall) of pfff is language independent and I think it's the only thing you need to generate srclib data. Once this is done every language supported by pfff will then be available to srclib too.
One needs to write this graph_code.marshall -> GO scrclib JSON thing though ...

[krishnavaidy]

This pull request might help: #129

The pr converts graph_code.marshall to the GraphSON format (json). But could also easily be tweaked to serialize to xml and protocol buffers, from which GO srclib can read...

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.