Skip to content

Allows specification of weight by range#2

Open
israelshirk wants to merge 2 commits intomasterfrom
unknown repository
Open

Allows specification of weight by range#2
israelshirk wants to merge 2 commits intomasterfrom
unknown repository

Conversation

@israelshirk
Copy link

Allows specification of weight by range rather than by integer value.

Useful when you need to scale up on a production server without interruption/cache busting. Example:

Scaling

  1. Start up pool with single server and a 0, 100 range.
    connection_hasher = RangeHash({'redis://1.1.1.1:6379': [0, 100]})
  2. A while later, it's running too hot but you'd rather not freeze production. So you can set up Redis replication, wait until it's synced, and then execute:
    connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]})
  3. Now you're running half of your load to the original server and half to the new one, and you haven't lost anything in terms of consisteny.
  4. Repeat as you continue scaling.

Over time, you can scale up more or less indefinitely without any downtime.

Failover

You can also handle failover gracefully since hashing is independent of hostname - starting the example above, you'd run connection_hasher.remove_nodes(['redis://2.2.2.2:6379]) then connection_hasher.add_nodes({'redis://3.3.3.3:6379': [0, 50]}) to handle failing over from 2.2.2.2 to 3.3.3.3 with no service interruptions, cache stampede, and only minor loss of consistency.

Includes the other commit, but you might want this addition structured differently, or have it in a separate distribution - LMK if that's the case. Thanks!

@yummybian
Copy link
Collaborator

I think this feature is not necessary.As both of the range [0,50] and the integer 50 equivalent to xrange(0,50).

@israelshirk
Copy link
Author

I'm not sure what you mean - could you give an example?

@yummybian
Copy link
Collaborator

I mean that the RangeHash({'redis://1.1.1.1:6379': [0, 100]}) could be instead with the ConsistentHash('redis://1.1.1.1:6379': 100}.Refer to the ‘def _node_keys(self, node)’ routine, they are same.

@yummybian
Copy link
Collaborator

Because the hashing is independent of hostname, so after connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]}), the range [0, 50] will hit 2.2.2.2, it will cause mishit the range [0, 50] of the original mapping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants