pushed updates

[ParinAcharyaGit]

updated prompt for LLM-as-a-judge

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
llm-evaluator	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 16, 2025 10:20pm

[pull-request-review-agent]

src/services/groqService.js

Make the judge prompt more specific to improve understanding
Create Issue

    const judgePrompt = \`You are an expert LLM scoring system. Your task is to analyze these responses to a given prompt and return ONLY a JSON object with scores for each response. The scores should range from 0 to 10 based on relevant metrics to test the quality of responses. Do not include any explanation or additional text.

    Prompt: "${prompt}"

    ${modelDisplayNames.llm1}: ${responses.llm1}
    ${modelDisplayNames.llm2}: ${responses.llm2}
    ${modelDisplayNames.llm3}: ${responses.llm3}

    Return JSON object in this exact format (no other text):
    {"llm1":N,"llm2":N,"llm3":N}\`;

Use JSON.parse with a reviver function to validate scores
Create Issue

    const validateScores = (key, value) => {
      if (typeof value !== 'number' || value < 0 || value > 10) {
        throw new Error('Invalid score value');
      }
      return value;
    };

    try {
      const scores = JSON.parse(jsonMatch[0], validateScores);
      return scores;
    } catch (parseError) {
      console.error('No valid JSON found in:', jsonMatch[0]);
      throw new Error('No valid JSON found in response');
    }