need complete and documented plugin example included in source/docs

[baallan]

The adiak plugin interface needs a complete documented example included showing:

• what a plugin must and might do (and when, in lifecycle terms).
• what threading contract the plugin must comply with (looks like adiak is by design single-threaded, but I don't find it stated anywhere). e.g. will it be called from inside openmp non-gpu threads? pthreads?

It's not clear to me if tests/testlib.c is a plugin demo or if it is complete wrt both how a plugin is called and what other things in the adiak api a plugin can call.

[daboehme]

Hi @baallan, thanks for pointing this out. I'll consider adding a complete Adiak tool/plugin example and extend the documentation to cover the points you raised. To answer your specific questions:

• A tool can use any of the public API functions to read and/or write Adiak values. Tools can read values with adiak_list_namevals() or adiak_get_nameval() and/or register callbacks with adiak_register_cb() as needed. Adiak self-initializes, therefore tools do not need to call adiak_init(). Tools should not call adiak_clean() because that would erase values that other tools still might want to read. Tools should take into account that many application write values like figures of merit late in their lifecycle, however applications are expected to write all values before calling MPI_Finalize().
• Adiak is indeed single-threaded, in particular users must ensure that write operations do not run concurrently with any other Adiak operations. However it should be fine to run read-only operations like adiak_get_nameval() concurrently. There are no restrictions on which thread(s) can run Adiak operations as long as those accesses are properly synchronized. In practice most programs only call Adiak from their main thread.

Let me know if you have any additional questions.

[baallan]

@daboehme @mplegendre In the adiak documentation and api there's a brief mention of flush, the semantics of which seems to be plugin dependent and perhaps intentionally vague (since things like log files, caliper, ldms, or my new project (currently called ADC) are all different). Please let me know if i've misinterpreted this.

The use case i'm thinking of is an app publishes a several distinct named parameters (say at the beginning of the run) and then wants to signal a plugin that, as a group, they should be shoved out (in the case of ADC, in a 'packaged' way).

For ADC i could approximate this by implementing "flush" in a way that publishes all parameters that have been added/updated since the last "flush". But this is likely not what most adiak users are thinking flush will mean; am i wrong in thinking this?
As an example, at run beginning, call adiak_collect_all, then flush, the as periodic progress occurs a flush call in ADC would export the new data.

We could also do something like defining some structure/usage to the subcategory argument that might give common semantic hints across tools, such as "group:$G,otherattr:$A". Unfortunately, i don't have details on what any other tool actually does with subcategory. In the ADC case, values of $G might just be consecutive integers as groups are created (though this is probably end-user-hostile) or tags like "startup", "progress", "result". Flush or some other operation (flush_group($G)) would signal the tool plugin that values with subcategory $G should all be published. This starts to overlap a little in the the enum category concept, but we can't be extending the enum with a value for each tool implemented (though maybe that's what happened 'performance' for the caliper case?).

Thoughts?
should we have a meeting?