Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added block handler to handle indexing of reserve balance #4

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ services:
- --db-schema=app
- --workers=4
- --batch-size=30
- --unsafe
# - --log-level=debug
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why unsafe? Isn't it unsafe? :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subql uses a restricted version of node to create a controlled env for running the code. sort of like endojs for hardened js

Therefore a lot of functionality (such as making api calls) is restricted. to enable it you need to run it in unsafe mode. you can read more about it here

# - --unfinalized-blocks=true
healthcheck:
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
"dependencies": {
"@subql/types-cosmos": "^3.2.3",
"@types/node": "^17.0.21",
"cross-fetch": "^4.0.0",
"pino": "^7.8.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use axios?

After comparing axios and cross-fetch, I think axios has more features compared to cross-fetch. It's easier to use, has automatic JSON parsing, and more popular :D

cross-fetch does have the advantage of being lightweight at 9.8kB minified, compared to axios's 30.5kB.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

axios unfortunately does not work in the restricted environment properly. i've tried a node-fetch as well which did not work

only cross-fetch seems to work properly

"ts-proto": "^1.112.1",
"tslib": "^2.3.1"
Expand Down
12 changes: 6 additions & 6 deletions project.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,12 @@ const project: CosmosProject = {
mapping: {
file: "./dist/index.js",
handlers: [
// {
// Using block handlers slows your project down as they can be executed with each and every block.
// Only use if you need to
// handler: 'handleEvent',
// kind: CosmosHandlerKind.Block,
// },
{
// Using block handlers slows your project down as they can be executed with each and every block.
// Only use if you need to
handler: 'handleBlock',
kind: CosmosHandlerKind.Block,
},
// {
// handler: "handleEvent",
// kind: CosmosHandlerKind.Event,
Expand Down
8 changes: 8 additions & 0 deletions schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -260,3 +260,11 @@ type ReserveMetrics @entity {
totalFeeMinted: BigInt!
allocations: [ReserveAllocationMetrics] @derivedFrom(field: "reserveMetrics")
}

type ReserveBalance @entity {
id: ID!
blockHeight: BigInt!
address: String!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@toliaqat I am not sure about it. But what if the blockHeight is much larger than what a BigInt type can handle? Should we also use a String type for blockHeight?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigInt can store super long numbers such as: 10 ^(10^8) which i dont think we can physically reach

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BigInt is good enough.

balance: BigInt!
denomination: String!
}
20 changes: 19 additions & 1 deletion src/mappings/mappingHandlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ import {
VaultManagerMetricsDaily,
PsmMetricDaily,
ReserveAllocationMetricsDaily,
ReserveBalance,
} from "../types";
import { CosmosEvent } from "@subql/types-cosmos";
import { CosmosEvent, CosmosBlock } from "@subql/types-cosmos";
import {
b64encode,
b64decode,
Expand All @@ -26,6 +27,7 @@ import {
extractBrand,
resolveBrandNamesAndValues,
dateToDayKey,
fetch,
} from "./utils";

import { EVENT_TYPES, STORE_KEY, VSTORAGE_VALUE, KEY_KEY, VALUE_KEY } from "./constants";
Expand All @@ -40,6 +42,22 @@ BigInt.prototype.toJSON = function () {
return this.toString();
};

const API_ENDPOINT = 'https://main-a.api.agoric.net:443'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a parameter in project.js called network which has an endpoint URL list. Project.js is complied into project.yaml. Is there a was way to read network.endpoint list instead of creating another variable here?


export async function handleBlock(block: CosmosBlock): Promise<void> {
const moduleAccounts = await fetch(`${API_ENDPOINT}/cosmos/auth/v1beta1/module_accounts`);

const reserveAccount = moduleAccounts.accounts.find((account: any) => account.name === 'vbank/reserve');
const reserveAccountAddress = reserveAccount.base_account.address;

const reserveAccountBalances = await fetch(`${API_ENDPOINT}/cosmos/bank/v1beta1/balances/${reserveAccountAddress}`);

const istBalance = reserveAccountBalances.balances.find((balance: any) => balance.denom === "uist")

const record = new ReserveBalance(block.block.id, BigInt(block.header.height), reserveAccountAddress, BigInt(istBalance.amount), 'uist');
await record.save();
}

export async function handleStateChangeEvent(cosmosEvent: CosmosEvent): Promise<void> {
const { event, block } = cosmosEvent;

Expand Down
14 changes: 14 additions & 0 deletions src/mappings/utils.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import crossFetch from 'cross-fetch';

export function extractBrand(str: string): string {
return str.replace("Alleged: ", "").replace(" brand", "");
}
Expand Down Expand Up @@ -95,3 +97,15 @@ export function dateToDayKey(timestamp: any): number {
const day = date.getUTCDate().toString().padStart(2, "0");
return parseInt(`${year}${month}${day}`);
}

export const fetch = async (url: string) => {
const response = await crossFetch(url);

if (response.status >= 400) {
const errorText = await response.text();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of checking the response status? What if it's in the 300 range?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use try/catch? That way you don't need to check for response status on line 104? If there is an error, the catch block can throw an error accordingly.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crossFetch wont automatically throw an error in cases where it receives a proper response. such as when we hit this URL https://main-a.api.agoric.net/cosssmos/auth/v1beta1/module_accountss
We receive a 501 but it doesnt throw an error. in those cases we manually check if we have an error code and throw an error ourselves

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it's in the 300 range?

3XX are for redirect codes which are not errrors

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crossFetch wont automatically throw an error in cases where it receives a proper response. such as when we hit this URL https://main-a.api.agoric.net/cosssmos/auth/v1beta1/module_accountss

Interesting. I reckon this is something wrong with the endpoint we're hitting, right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3XX are for redirect codes which are not errrors

Yeah. However, it does not indicate a successful response. Perhaps we should handle it too.

Copy link
Collaborator Author

@frazarshad frazarshad Apr 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I reckon this is something wrong with the endpoint we're hitting, right?

This is the case with most API responses actually. a lot of libraries dont throw an error implicitly and expect you to do it yourself

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. However, it does not indicate a successful response. Perhaps we should handle it too.

300 are successful responses in the sense that we can actually obtain useful responses from them. compared to 400s and 500s which just represent a failure

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

300 are successful responses in the sense that we can actually obtain useful responses from them. compared to 400s and 500s which just represent a failure

In that regard, 400 and 500 are also useful responses converting us what's wrong :D

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rabi-siddique we look for errors in 400+ range. Here we are just checking for errors in response.

throw new Error(`Status: ${response.status} -- ${errorText}`);
}

const responseJson = await response.json();
return responseJson;
}
Loading