Conversation
…ueries` (#126) 1. Support input file (with SPARQL queries and a description for each) in both TSV and YML; refactor the parsing code accordingly 2. Add option to generate a YML result file suitable for processing with our evaluation web app; see #171 3. Add unit tests 4. Rename from `example-queries` to `benchmark-queries` because that is really what this command is doing; the old functionality, which was a special case, is still there using the `--example-queries` option Co-authored-by: Hannah Bast <[email protected]>
hannahbast
left a comment
hannahbast
left a comment
There was a problem hiding this comment.
Can you please resolve the conflict?
ullingerc
left a comment
ullingerc
left a comment
There was a problem hiding this comment.
Thanks a lot for adding the penalty option, however there is still a problem with the geometric mean. Can you please fix it?
| bw_1_to_5 += 1 | ||
| runtimes.append(runtime) | ||
| total_time += runtime | ||
| total_log_time += max(math.log(runtime), 0.001) |
There was a problem hiding this comment.
| total_log_time += max(math.log(runtime), 0.001) | |
| total_log_time += math.log(runtime) |
This line is the problem, why we see different results on the evaluation web app and it is not quite correct. While the inputs to the geometric mean are supposed to be positive (the times in this case), the result of the logarithm needn't be positive. Please change this line or just import the stdlib library statistics and apply query_data["gmeanTime"] = statistics.geometric_mean(runtimes). (The library also has mean and median)
There was a problem hiding this comment.
Yes, you are right. I have updated the code to use the statistics module for mean, geometric_mean and median.
…to output yaml file in `benchmark-queries`
…and add long query to tooltip (but not copied text!)
…ge and fix UI issues
- Add --title and --description arguments to override title and description in --queries_yml - Make output yml structure the same as input yml structure for consistency
- Add --title-overview-page argument - Keep old query and sparql keys intact for query objects to not break the web app
3 participants
No description provided.