Run

Run it

From the command line execute:

./source/papers.py --config my_config.ini

This assumes my_config.ini is the name of your config file and is present in the config directory.

If there are missing API keys etc the pipeline will halt with a (hopefully) useful error message.

Once the pipeline is running it will output information to the screen giving an indication of what it is doing. It will also output information to a log file in the logs directory. It can be useful to watch this as it is running.

The bulk of the run time is likely to be spent downloading metadata. All of this metadata is cached as it is downloaded, so if the pipeline crashes or has to be stopped for any reason then it can be safely restarted and should pick up from where it got to in the previous run. A slight nuance to this is that if a metadata record is not available for a publication from a source (e.g. if a DOI is given in Zotero but Scopus has no record of this DOI) then even if a complete run has happened it will still do the query for this data the next time the pipeline is run.