Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fallback selectors to ItemLoader #30

Open
ejulio opened this issue May 10, 2019 · 2 comments
Open

Add fallback selectors to ItemLoader #30

ejulio opened this issue May 10, 2019 · 2 comments
Labels
enhancement New feature or request

Comments

@ejulio
Copy link
Collaborator

ejulio commented May 10, 2019

In some cases it is common to have fallback selectors for certain fields.
This way, we end up writing a piece of code like

loader = MyLoader(response=response)
loader.add_css('my_field', 'selector1')
loader.add_css('my_field', 'selector2') # fallback 1
loader.add_css('my_field', 'selector3') # fallback 2

However, a, maybe, better way would be

loader = MyLoader(response=response)
loader.add_css('my_field', [
    'selector1',
    'selector2', # fallback 1
    'selector3', # fallback 2
])

The API above would be the equivalent of the first example.
However, @cathalgarvey also shared a nice idea to stop in the first matching selector.

loader = MyLoader(response=response)
loader.add_css('my_field', [
    'selector1',
    'selector2', # fallback 1
    'selector3', # fallback 2
], selectors_as_preferences=True)

Then, if selector1 yields a result, the other ones are not attempted, otherwise we fallback to selector2 and so on.

The same API should be applied to loader.add_xpath.

@BurnzZ
Copy link
Member

BurnzZ commented May 26, 2019

Hi @ejulio @Gallaecio! I'd like to know your thoughts on scrapy/scrapy#3795 for discussion as it's closely related to this. :)

@stav
Copy link

stav commented May 26, 2019

Then, if selector1 yields a result, the other ones are attempted,...

Then, if selector1 yields a result, the other ones are NOT attempted,...

@Gallaecio Gallaecio transferred this issue from scrapy/scrapy Oct 30, 2020
@Gallaecio Gallaecio added the enhancement New feature or request label Feb 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants