Make review configurable #126
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…d review defaults.
Populate the reviewTypes array with `perform`, `analyseScreenAndCreatePilotStep`, `createPrompt` and `extractTaggedOutputs`.
asafkorem
requested changes
Mar 22, 2025
packages/core/src/performers/auto-performer/AutoPerformerPromptCreator.ts
Show resolved
Hide resolved
…e `extractTaggedOutputs` for internal use only.
Also remove redundant condition, and pass `undefined` when needed.
asafkorem
requested changes
Mar 24, 2025
Comment on lines
52
to
64
| export function extractPilotSummaryOutputs(text: string): { | ||
| [K in keyof typeof PILOT_SUMMARY]: string; | ||
| } { | ||
| return extractTaggedOutputs({ text, outputsMapper: PILOT_SUMMARY }); | ||
| } | ||
|
|
||
| export function extractReviewOutputs(text: string): { | ||
| [K in keyof typeof PILOT_REVIEW_SECTION]: string; | ||
| } { | ||
| return extractTaggedOutputs({ text, outputsMapper: PILOT_REVIEW_SECTION }); | ||
| } | ||
|
|
||
| export function extractAutoPilotStepOutputs( |
There was a problem hiding this comment.
YES!
This looks WAY BETTER 🙌🏼 🚀
| "For each applicable review section (`UX`, `Accessibility`, `Internationalization`), create a comprehensive report enclosed within corresponding tags (e.g., `<UX>`, `<ACCESSIBILITY>`, `<INTERNATIONALIZATION>`), including a summary, findings, and a score out of 10.", | ||
| "Ensure each section is clearly labeled and formatted as shown in the examples.", | ||
| "If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.", | ||
| "If the goal is achieved, add a `<SUMMARY>` block inside the `<THOUGHTS>` section, summarizing the overall flow and findings.", |
There was a problem hiding this comment.
Is that how we actually handle the outputs? So this means the thoughts of the last step will always have a <SUMMARY> block inside its contents. Looks like we missed something
I think we should put the summary outside the Thoughts block, and handle it separately..
There was a problem hiding this comment.
Here - https://github.com/wix-incubator/pilot/blob/master/packages/core/src/performers/auto-performer/AutoPerformer.ts#L181-L183
we should extract it from the promptResults.
Let's do this change as part of this PR or create a new small task for that, WDYT?
asafkorem
reviewed
Mar 24, 2025
asafkorem
requested changes
Mar 24, 2025
packages/core/src/performers/auto-performer/AutoPerformerPromptCreator.ts
Outdated
Show resolved
Hide resolved
tzvielwix
approved these changes
Mar 27, 2025
asafkorem
approved these changes
Mar 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.