Skip to content

Commit 0591729

Browse files
committed
Updated readme, config
1 parent f69adff commit 0591729

File tree

3 files changed

+3
-13
lines changed

3 files changed

+3
-13
lines changed

python-crawl4ai/README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
1-
# Trigger.dev + Python web crawler headless example
1+
# Trigger.dev + Python headless web crawler example
22

33
This demo showcases how to use Trigger.dev with Python to build a web crawler that uses a headless browser to navigate websites and extract content.
44

55
## Features
66

77
- [Trigger.dev](https://trigger.dev) for background task orchestration
8+
- [Trigger.dev Python build extension](https://trigger.dev/docs/config/extensions/pythonExtension) to install the dependencies and run the Python script
89
- [Crawl4AI](https://github.com/unclecode/crawl4ai), an open source LLM friendly web crawler
910
- [Playwright](https://playwright.dev/) to create a headless chromium browser
1011

python-crawl4ai/package-lock.json

+1-2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

python-crawl4ai/trigger.config.ts

-10
Original file line numberDiff line numberDiff line change
@@ -17,16 +17,6 @@ export default defineConfig({
1717
installPlaywrightChromium(),
1818
],
1919
},
20-
retries: {
21-
enabledInDev: true,
22-
default: {
23-
maxAttempts: 1,
24-
minTimeoutInMs: 1_000,
25-
maxTimeoutInMs: 5_000,
26-
factor: 1.6,
27-
randomize: true,
28-
},
29-
},
3020
});
3121

3222
export function installPlaywrightChromium(): BuildExtension {

0 commit comments

Comments
 (0)