Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

increase claim extraction document length, improve claim extraction prompt #203

Merged
merged 1 commit into from
Sep 11, 2024
Merged

increase claim extraction document length, improve claim extraction prompt #203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
export const MAX_POSTS_PER_PAGE = 100

export const MAX_CHARS_PER_POST = 1500
export const MAX_CHARS_PER_DOCUMENT = 3000
13 changes: 10 additions & 3 deletions app/repositories/fact-checking.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { type Transaction, sql } from 'kysely'
import OpenAI from 'openai'
import { zodResponseFormat } from 'openai/helpers/zod'
import { z } from 'zod'
import { MAX_CHARS_PER_POST } from '#app/constants.ts'
import { MAX_CHARS_PER_DOCUMENT } from '#app/constants.ts'
import { createPost, getPost } from '#app/repositories/post.ts'
import { type PollType, type Post } from '#app/types/api-types.ts'
import { type DB } from '#app/types/kysely-types.ts'
Expand Down Expand Up @@ -49,15 +49,22 @@ export const ClaimExtractionSchema = z.object({
export type ClaimList = z.infer<typeof ClaimExtractionSchema>

export async function extractClaims(content: string): Promise<ClaimList> {
invariant(content.length <= MAX_CHARS_PER_POST, 'Post content too long')
invariant(content.length > 0, 'Post content too short')
invariant(
content.length <= MAX_CHARS_PER_DOCUMENT,
'Document for claim extraction is content too long',
)
invariant(
content.length > 0,
'Document for claim extraction is content too short',
)

const openai = new OpenAI()

const systemMessage = `
Extract all the claims made in the provided piece of content.
Claims should be direct: For example, if a speaker makes claim X, extract "X", not "Speaker claims X".
Don't use the passive form.
If a speaker uses the personal pronoun "I", try to infer the person's name and restate the claim in the third person.
`

const completion = await openai.beta.chat.completions.parse({
Expand Down
4 changes: 2 additions & 2 deletions app/routes/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { Markdown } from '#app/components/markdown.tsx'
import PollResult from '#app/components/ui/poll-result.tsx'
import { PostContent } from '#app/components/ui/post-content.tsx'
import { Textarea } from '#app/components/ui/textarea.tsx'
import { MAX_CHARS_PER_POST } from '#app/constants.ts'
import { MAX_CHARS_PER_DOCUMENT } from '#app/constants.ts'
import { db } from '#app/db.ts'
import { type ClaimList } from '#app/repositories/fact-checking.ts'
import { getChronologicalPolls } from '#app/repositories/ranking.ts'
Expand Down Expand Up @@ -75,7 +75,7 @@ Press **Ctrl + Enter** to extract claims.
placeholder="A statement to extract claims from."
name="content"
value={statementValue}
maxLength={MAX_CHARS_PER_POST}
maxLength={MAX_CHARS_PER_DOCUMENT}
onChange={event => setStatementValue(event.target.value)}
className="mb-2 min-h-[150px] w-full"
onKeyDown={(event: React.KeyboardEvent<HTMLTextAreaElement>) => {
Expand Down