From c6629c1b2473f4246e92fa61c4d635aa2c376dea Mon Sep 17 00:00:00 2001 From: Andreas Motl Date: Sun, 15 Sep 2024 01:10:25 +0200 Subject: [PATCH] Chore: Remove DynamoDB backlog --- cratedb_toolkit/io/dynamodb/backlog.md | 35 -------------------------- 1 file changed, 35 deletions(-) delete mode 100644 cratedb_toolkit/io/dynamodb/backlog.md diff --git a/cratedb_toolkit/io/dynamodb/backlog.md b/cratedb_toolkit/io/dynamodb/backlog.md deleted file mode 100644 index 1361357d..00000000 --- a/cratedb_toolkit/io/dynamodb/backlog.md +++ /dev/null @@ -1,35 +0,0 @@ -# DynamoDB Backlog - -## Iteration +1 -- Pagination / Batch Getting. - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/programming-with-python.html#programming-with-python-pagination - -- Use `batch_get_item`. - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/client/batch_get_item.html - -- Scan by query instead of full. - - -## Iteration +2 - -### Resumption on errors? -Another variant to scan the table, probably for resuming on errors? -```python -key = None -while True: - if key is None: - response = table.scan() - else: - response = table.scan(ExclusiveStartKey=key) - key = response.get("LastEvaluatedKey", None) -``` - -### Item transformations? -That's another item transformation idea picked up from an example program. -Please advise if this is sensible in all situations, or if it's just a -special case. - -```python -if 'id' in item and not isinstance(item['id'], str): - item['id'] = str(item['id']) -```