Installation

Requirements installation

First, install Scrapy and redis-py.

pip install scrapy>=2.6.0 redis>=4.2

Second, setup redis server and enable port 6379 for redis server.

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee 
/etc/apt/sources.list.d/redis.list
sudo apt update && sudo apt install -y redis

Or setup redis server by the official tutorial.

Stable release

To install Scrapy-Redis, run this command in your terminal:

pip install scrapy-redis

If you don’t have pip installed, this Python installation guide can guide you through the process.

From sources

The sources for Scrapy-Redis can be downloaded from the Github repo.

You can either clone the public repository:

git clone git://github.com/rolando/scrapy-redis

Or download the tarball:

curl -OL https://github.com/rolando/scrapy-redis/tarball/master

Once you have a copy of the source, you can install it with:

pip install -e .