Skip to content
This repository has been archived by the owner on Jul 8, 2023. It is now read-only.

Bug: concurrent crawling #48

Open
atersolis opened this issue Jun 18, 2019 · 3 comments
Open

Bug: concurrent crawling #48

atersolis opened this issue Jun 18, 2019 · 3 comments

Comments

@atersolis
Copy link

atersolis commented Jun 18, 2019

I've been playing around with foxr to find out if I can use it as a substitute of puppeteer for firefox-based browsers.
I tried this code :

const foxr = require("foxr").default;

async function main() {
	const browser = await foxr.launch({
		headless: false,
		args: [
			"-marionette",
			"-no-remote",
			"-profile",
			"/tmp/firefox_9a139bbd688bad29dce5b4fd"
		],
		executablePath: "/usr/bin/firefox"
	});

	const urls = ["http://google.com", "http://youtube.com", "http://facebook.com"]
	//await lib.crawl(browser, urls);
	const page1 = await browser.newPage();
	const page2 = await browser.newPage();
	const page3 = await browser.newPage();

	await Promise.all([
		page1.goto(urls[0]),
		page2.goto(urls[1]),
		page3.goto(urls[2])
	]);
}

main();

Expected behavior : google, youtube and facebook open in three different tabs (code works with puppeteer)
Observer behavior: I get two empty tabs, and one tab with facebook.
Context : firefox 67.0.2 on ubuntu, foxr 0.9.0

I also noticed that firefox closes on its own after a few seconds of inactivity. Is that an effect of -marionette ?

@deepsweet
Copy link
Owner

deepsweet commented Jun 18, 2019

code works with puppeteer

unfortunately that's not how Marionette works – there is no way to send command to particular page by internal ID or something, it affects only a "current" page.

you might want to give puppeteer-firefox a try (keep in mind that it downloads a custom-built Firefox)

firefox closes on its own after a few seconds of inactivity

that shouldn't really happen, at least not in a few seconds. any simple way to reproduce it?

@atersolis
Copy link
Author

unfortunately that's not how Marionette works – there is no way to send command to particular page by internal ID or something, it affects only a "current" page.

Too bad, I will have to use puppeteer-firefox. I wanted to avoid it because like you said, it's a custom-built Firefox

that shouldn't really happen, at least not in a few seconds. any simple way to reproduce it?

const foxr = require("foxr").default;

async function main() {
	const browser = await foxr.launch({
		headless: false,
		args: [
			"-marionette",
			"-no-remote",
			"-profile",
			"/tmp/firefox_9a139bbd688bad29dce5b4fd"
		],
		executablePath: "/usr/bin/firefox"
	});
}

main();

With this code, Firefox starts, then closes after about 10 seconds on my PC.

@deepsweet
Copy link
Owner

With this code, Firefox starts, then closes after about 10 seconds on my PC.

I've figured it out – actually it's "by design". Whenever Node.js has nothing more to do it quits, and by default Foxr is trying to cleanup and stop Firefox processes it has launched.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants