Skip to content

counter_incs by incrementRows #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion happybase/table.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
"""

import logging
from itertools import izip
from numbers import Integral
from operator import attrgetter
from struct import Struct

from .hbase.ttypes import TScan
from .hbase.ttypes import TScan, TIncrement
from .util import thrift_type_to_dict, str_increment, OrderedDict
from .batch import Batch

Expand Down Expand Up @@ -568,3 +569,24 @@ def counter_dec(self, row, column, value=1):
:rtype: int
"""
return self.counter_inc(row, column, -value)

def counters_inc(self, row, data):
"""

This method increments (or decrements) the counter columns in the row
specified by `row`. The `data` argument is iteratble data type that
contains tuple of column and value, e.g.
[(`cf:col`, 1), (`cf:col2`, 2)].

Note that unlike `counter_inc`, does not return value after
incrementing.

:param str row: the row key
:param list data: list of tuple for columns and values
"""
if data is not None and not isinstance(data, (list, tuple, izip)):
raise TypeError("'data' must be a iterable data types of tuple")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not simply require a dict here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About type of data? I think that iterable collections are more general and efficiency than dict, especially for bunch of data processing, and also dict can be used with iteritems method. But it seems a little bit verbose...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any numbers to back up this claim? In the end everything will be pulled into memory anyway when constructing the TIncrement instances, and using a dict won't really add much overhead. Other parts of the Happybase API extensively use dict as well, so I'm inclined to require a dict here as well for consistency.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.. yes you're right. At the end the data will be wrapped as '''TIncrement''' anyway. ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though you're right that passing a dict will result in stuff in memory twice, but I think it's not a problem.


self.connection.client.incrementRows(
[TIncrement(table=self.name, row=row, column=column, ammount=value)
for column, value in data])
28 changes: 28 additions & 0 deletions tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,34 @@ def test_atomic_counters():
assert_equal(10, table.counter_dec(row, column, -7))


def test_multiple_counters():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, tests look good.

row = 'row-with-counters'
columns = ['cf1:counter1', 'cf1:counter2', 'cf1:counter3']

table.counters_inc(row, zip(columns, [0, 0, 0]))
assert_equal([0, 0, 0], [table.counter_get(row, c) for c in columns])

table.counters_inc(row, zip(columns, [1, 3, 5]))
assert_equal([1, 3, 5], [table.counter_get(row, c) for c in columns])

table.counters_inc(row, zip(columns, [-1, -3, -5]))
assert_equal([0, 0, 0], [table.counter_get(row, c) for c in columns])

for col, delta in zip(columns, [1, 3, 5]):
table.counters_inc(row, [(col, delta)])
assert_equal(delta, table.counter_get(row, col))
table.counter_dec(row, [(col, -delta)])
assert_equal(0, table.counter_get(row, col))

table.counters_inc(row, zip(columns, [1, 3, 5]))
table.counters_inc(row, zip(columns, [1, 3, 5]))
assert_equal([2, 6, 10], [table.counter_get(row, c) for c in columns])

table.counters_inc(row, zip(columns, [-1, -3, -5]))
table.counters_inc(row, zip(columns, [-1, -3, -5]))
assert_equal([0, 0, 0], [table.counter_get(row, c) for c in columns])


def test_batch():
with assert_raises(TypeError):
table.batch(timestamp='invalid')
Expand Down