Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added metrics for collectibles #21280

Merged
merged 1 commit into from
Oct 22, 2024
Merged

Added metrics for collectibles #21280

merged 1 commit into from
Oct 22, 2024

Conversation

vkjr
Copy link
Contributor

@vkjr vkjr commented Sep 18, 2024

fixes #21279

Summary

Added tracking for events:

  • user navigated collectibles tab (from wallet home or account)
  • user navigated collectible details
  • user fetched collectibles, we track the number of collectibles fetched according to the comment decided not to implement number tracking in this PR

Review notes

Platforms

  • Android
  • iOS

Areas that maybe impacted

Functional
  • wallet / transactions

status: ready

@@ -45,3 +45,7 @@
#(rf/dispatch [:profile.login/login-with-biometric-if-available
(get-in db [:profile/login :key-uid])]))
:shell? true}]]]})))

;; Events do nothing but they will be intercepted and tracked
(rf/reg-event-fx :centralized-metrics/navigated-to-collectibles-tab (fn []))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a choice - create events under specific namespace, like wallet/navigated-to-collectibles-tab, or under centralized-metrics. I selected the second one for the sake of code readability. Developer will immediately see that event exists for metrics.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, we should not be introducing new events. We can just hook in the :navigate-to event that already exists.

The idea with interceptors is to make the metrics layer invisible. We don't emit metric specific events, we just read other events, and track some to compute metrics.

@status-im-auto
Copy link
Member

status-im-auto commented Sep 18, 2024

Jenkins Builds

Click to see older builds (38)
Commit #️⃣ Finished (UTC) Duration Platform Result
0466ced #1 2024-09-18 08:59:48 ~2 min tests 📄log
b93b5b9 #2 2024-09-18 09:03:02 ~2 min tests 📄log
✔️ b93b5b9 #2 2024-09-18 09:06:36 ~6 min android-e2e 🤖apk 📲
✔️ b93b5b9 #2 2024-09-18 09:07:02 ~6 min android 🤖apk 📲
✔️ b93b5b9 #2 2024-09-18 09:11:40 ~11 min ios 📱ipa 📲
6d14dc6 #3 2024-09-19 17:07:07 ~2 min tests 📄log
✔️ 6d14dc6 #3 2024-09-19 17:12:08 ~7 min android-e2e 🤖apk 📲
✔️ 6d14dc6 #3 2024-09-19 17:12:34 ~7 min android 🤖apk 📲
✔️ 6d14dc6 #3 2024-09-19 17:15:10 ~10 min ios 📱ipa 📲
b177f55 #4 2024-09-20 10:17:11 ~3 min tests 📄log
✔️ 15bd27d #6 2024-09-20 10:25:00 ~3 min tests 📄log
✔️ 15bd27d #6 2024-09-20 10:29:29 ~8 min android-e2e 🤖apk 📲
✔️ 15bd27d #6 2024-09-20 10:29:53 ~8 min android 🤖apk 📲
✔️ 15bd27d #6 2024-09-20 10:34:48 ~13 min ios 📱ipa 📲
✔️ 774a80f #7 2024-09-23 14:52:07 ~4 min tests 📄log
✔️ 774a80f #7 2024-09-23 14:55:21 ~7 min android-e2e 🤖apk 📲
✔️ 774a80f #7 2024-09-23 14:55:48 ~7 min android 🤖apk 📲
✔️ 774a80f #7 2024-09-23 14:57:41 ~9 min ios 📱ipa 📲
7d7f46b #8 2024-09-26 17:06:22 ~2 min tests 📄log
✔️ 7d7f46b #8 2024-09-26 17:11:03 ~7 min android-e2e 🤖apk 📲
✔️ 7d7f46b #8 2024-09-26 17:11:27 ~7 min android 🤖apk 📲
✔️ 7d7f46b #8 2024-09-26 17:13:43 ~9 min ios 📱ipa 📲
8357b42 #9 2024-09-27 15:02:31 ~3 min tests 📄log
✔️ 8357b42 #9 2024-09-27 15:05:33 ~6 min android-e2e 🤖apk 📲
✔️ 8357b42 #9 2024-09-27 15:09:07 ~9 min android 🤖apk 📲
6f21b21 #10 2024-09-27 15:13:44 ~2 min tests 📄log
✔️ 6f21b21 #10 2024-09-27 15:17:04 ~6 min android-e2e 🤖apk 📲
✔️ 6f21b21 #10 2024-09-27 15:19:03 ~8 min android 🤖apk 📲
✔️ 6f21b21 #10 2024-09-27 15:21:36 ~10 min ios 📱ipa 📲
✔️ 1d05775 #12 2024-09-27 15:35:40 ~8 min tests 📄log
✔️ 1d05775 #12 2024-09-27 15:36:43 ~9 min ios 📱ipa 📲
✔️ 1d05775 #12 2024-09-27 15:37:03 ~10 min android-e2e 🤖apk 📲
✔️ 1d05775 #12 2024-09-27 15:38:41 ~11 min android 🤖apk 📲
✔️ fedbc1b #13 2024-10-17 18:20:41 ~4 min tests 📄log
✔️ fedbc1b #13 2024-10-17 18:23:32 ~7 min android-e2e 🤖apk 📲
✔️ fedbc1b #13 2024-10-17 18:23:59 ~7 min android 🤖apk 📲
fedbc1b #13 2024-10-17 18:26:09 ~9 min ios 📄log
✔️ fedbc1b #14 2024-10-18 14:55:37 ~10 min ios 📱ipa 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 84c37e8 #14 2024-10-21 12:51:43 ~4 min tests 📄log
✔️ 84c37e8 #14 2024-10-21 12:53:54 ~6 min android-e2e 🤖apk 📲
✔️ 84c37e8 #14 2024-10-21 12:54:20 ~6 min android 🤖apk 📲
✔️ 84c37e8 #15 2024-10-21 13:01:36 ~14 min ios 📱ipa 📲
✔️ 156bde2 #15 2024-10-22 14:59:49 ~5 min tests 📄log
✔️ 156bde2 #15 2024-10-22 15:01:40 ~7 min android 🤖apk 📲
✔️ 156bde2 #15 2024-10-22 15:03:13 ~8 min android-e2e 🤖apk 📲
✔️ 156bde2 #16 2024-10-22 15:06:33 ~12 min ios 📱ipa 📲

@@ -45,3 +45,7 @@
#(rf/dispatch [:profile.login/login-with-biometric-if-available
(get-in db [:profile/login :key-uid])]))
:shell? true}]]]})))

;; Events do nothing but they will be intercepted and tracked
(rf/reg-event-fx :centralized-metrics/navigated-to-collectibles-tab (fn []))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, we should not be introducing new events. We can just hook in the :navigate-to event that already exists.

The idea with interceptors is to make the metrics layer invisible. We don't emit metric specific events, we just read other events, and track some to compute metrics.

@vkjr
Copy link
Contributor Author

vkjr commented Sep 18, 2024

@shivekkhurana, switching to collectibles tab doesn't produce events because it happens within page and without re-frame usage

@shivekkhurana
Copy link
Contributor

shivekkhurana commented Sep 18, 2024

I fear that if we have explicit metrics capturing events, then the code will become littered with these.

In this particular case, one solution can be to lift the local state to Re-Frame. What do you think about it ?

Update:
We decided on discord to intercept another event that occurs when the NFT tab is opened.

@vkjr vkjr force-pushed the collectible-metrics branch 3 times, most recently from 2af004c to 15bd27d Compare September 20, 2024 10:20
@vkjr
Copy link
Contributor Author

vkjr commented Sep 20, 2024

@shivekkhurana , metrics were rewritten to remove empty metrics-specific events. Navigation to collectible tab now uses re-frame subscription instead of local state to support this.

@@ -35,12 +35,16 @@
:on-press #(rf/dispatch [:show-bottom-sheet {:content new-account}])
:type :add-account})

(def first-tab-id :assets)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you lift state to re-frame ?

Copy link
Contributor Author

@vkjr vkjr Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes
this is an initial state value for the case when there is no value in re-frame db. As soon as user selects any tab re-frame value will be used.
Similar approach used on account page

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vkjr, I thought we had a place for setting up initial state. For the wallet it would be wallet/db.cljs. If we do that, then we can remove the or around the sub :wallet/home-tab.

@shivekkhurana
Copy link
Contributor

I'm okay with this PR, just want someone else to review this too.

I have one more question: Did you check that the event is being saved in mixpanel?
(you can login to MixPanel with your status email to see the dashboard)

@vkjr
Copy link
Contributor Author

vkjr commented Sep 23, 2024

@shivekkhurana, yes I checked, events appear in mixpanel

@vkjr vkjr requested a review from a team September 23, 2024 10:50
Copy link
Contributor

@ulisesmac ulisesmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @vkjr !

Thanks for the PR!

I left some comments.


(defn track-view-id-event
[view-id]
(when (contains? view-ids-to-track view-id)
(navigation-event (name view-id))))

(defn collectilbes-fetched-event
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo: collectilbes

[db]
(let [accounts (get-in db [:wallet :accounts])
amount-on-all-accounts (reduce (fn [collectibles-amount account]
(+ collectibles-amount (:current-collectible-idx account)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:current-collectible-idx is the index for the current collectible request and it varies. It doesn't necessarily reflect the total amount of collectibles for an account.

E.g., if we fetch collectibles by batches of 5, then this value will be 5, then when the user scrolls up to the end of the collectible listing, it'll be 10 and we'll fetch the following 5 collectibles, and so on. So this value will be:

5 -> 10 -> 15 -> 20 -> ...

I just wanted to clarify that this number will be small at the beginning and it changes, just in case this is not the intended purpose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is intended. It doesn't reflect the total amount but reflects our current knowledge about account and I think that should be ok. As far as I know our api doesn't provide total amount of collectibles before we fetched them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess in this case the value of sending the "total amount" metric will be lost, since we won't know whether the users have x amount of collectibles, we'll just get the first fetch amount unless they scroll till the end. Is it even worth sending it? What will we learn from this metric?

Comment on lines 81 to 74
:wallet/select-account-tab
(when (= second-parameter :collectibles)
(navigated-to-collectibles-tab-event :account))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this when a sign that this case should become a cond?

What could second-parameter be besides :collectibles? If possible, it'd be great to rename it, second-parameter provides zero context, it's almost the same as name it parameter 😞

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case here checks the event-name parameter, while second-parameter can be any because it is the second parameter of re-frame event that we are intercepting (agree that new name needed). For different case clauses second-parameter can be different or absent. So I'm not sure cond will work here because condition will look like (and (= event-name :wallet/select-account-tab) (= second-parameter :collectibles). Wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, it'd be great to rename it, second-parameter provides zero context, it's almost the same as name it parameter

@ulisesmac, the name second-parameter was chosen intentionally because it signals that we don't know what it is, it could be anything and that it is only the second (could be a third and so on). We can rename it to something else of course, but it will always be generic because the value varies per event. Please, see my other comment for the function tracked-event where I give an alternative to how we could design the code.

[init-loaded? set-init-loaded] (rn/use-state false)
{:keys [formatted-balance]} (rf/sub [:wallet/aggregated-token-values-and-balance])
theme (quo.theme/use-theme)]
(let [selected-tab (or (rf/sub [:wallet/home-tab]) first-tab-id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expression:

(or (rf/sub[...]) value-if-not-found)

Seems not necessary, since this is a new sub and it'll be cleaner if all the logic is wrapped within the sub.

We have a def for first-tab-id, instead of putting it in the view, we can define it near the subscription, so the sub directly returns.

Wdyt?

src/status_im/subs/wallet/wallet.cljs Show resolved Hide resolved
Comment on lines 69 to 70
(defn tracked-event
[[event-name second-parameter]]
[[event-name second-parameter] db]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is the right way of extending this API, it is becoming more restrictive instead of more flexible.

IMO, we could create a two-arity version instead, since with this change, we call it only once as:

(tracked-event '[...] db)

And multiple times as:

(tracked-event '[...] {})

And... actually, this second-parameter value is also not passed to the function call sometimes 🤔
I think the signature could have been the following since the beginning:

(defn tracked-event
  [event-name & {:keys[...] :as opts}]
  ...)

CC: @ilmotta

Copy link
Contributor

@ilmotta ilmotta Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tagging me @ulisesmac. Below I share a few roughly sketched ideas, not blocking the PR from my PoV. Text is long, hard to explain, but I hope it makes some sense.

I think it's not the best that the tracking code now has access to the entire app-db and knows the inner details of its structure. Better I would say we should give the least amount of data to this layer and keep it plain simple, also because it's not well QAed.

If we need to compute something from the app-db that needs to be tracked, and which is decoupled from the metrics layer we could:

  1. Define a root key in the app-db, for example, :centralized-metrics/event-data, which will store any arbitrary data that's safe to be tracked and which is serializable to JSON.
  2. The event that wants to be tracked can assoc to this root key. In this case, in event :wallet/flush-collectibles-fetched it would assoc the total number of collectibles.
  3. The interceptor centralized-metrics-interceptor will then extract the value of :centralized-metrics/event-data and pass it to tracking/tracked-event.

With this solution, the centralized metrics layer will not know how to compute anything, but will still be able to get computed track data and/or arguments passed to any re-frame event. This would scale to any other event in re-frame and we would still keep this layer simple/dumb.


About the signature of the function tracked-event, an alternative is to implement tracked-event with the case macro, but the event argument (first arg to tracked-event) would be passed to downstream functions, and then we would only destructure the re-frame event in each specific function signature, thus solving the naming problem of second-parameter or how to best destructure the event because each function will destructure and name the arguments knowing what they really are.

In this example, there's no generic name like second-parameter and we don't need to worry about the full signature of the re-frame event:

(defn- metrics-enabled-event
  [[_ enabled?]]
  (key-value-event "events.metrics-enabled" :enabled enabled?))

(defn tracked-event
  [[event-name :as event] event-data]
  (case event-name
    :profile/get-profiles-overview-success
    (user-journey-event app-started-event)

    :centralized-metrics/toggle-centralized-metrics
    (metrics-enabled-event event)

    :set-view-id
    (track-view-id-event event)

    (:wallet/select-account-tab :wallet/select-home-tab)
    (navigated-to-collectibles-tab-event event)

    ;; => NOTE: Here `event-data` was computed outside the centralized metrics layer and was extracted by the global interceptor when calling tracked-event 
    :wallet/flush-collectibles-fetched
    (collectilbes-fetched-event event-data)

    nil))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tagging me @ulisesmac. Below I share a few roughly sketched ideas, not blocking the PR from my PoV. Text is long, hard to explain, but I hope it makes some sense.

I think it's not the best that the tracking code now has access to the entire app-db and knows the inner details of its structure. Better I would say we should give the least amount of data to this layer and keep it plain simple, also because it's not well QAed.

If we need to compute something from the app-db that needs to be tracked, and which is decoupled from the metrics layer we could:

  1. Define a root key in the app-db, for example, :centralized-metrics/event-data, which will store any arbitrary data that's safe to be tracked and which is serializable to JSON.
  2. The event that wants to be tracked can assoc to this root key. In this case, in event :wallet/flush-collectibles-fetched it would assoc the total number of collectibles.
  3. The interceptor centralized-metrics-interceptor will then extract the value of :centralized-metrics/event-data and pass it to tracking/tracked-event.

With this solution, the centralized metrics layer will not know how to compute anything, but will still be able to get computed track data and/or arguments passed to any re-frame event. This would scale to any other event in re-frame and we would still keep this layer simple/dumb.


About the signature of the function tracked-event, an alternative is to implement tracked-event with the case macro, but the event argument (first arg to tracked-event) would be passed to downstream functions, and then we would only destructure the re-frame event in each specific function signature, thus solving the naming problem of second-parameter or how to best destructure the event because each function will destructure and name the arguments knowing what they really are.

In this example, there's no generic name like second-parameter and we don't need to worry about the full signature of the re-frame event:

(defn- metrics-enabled-event
  [[_ enabled?]]
  (key-value-event "events.metrics-enabled" :enabled enabled?))

(defn tracked-event
  [[event-name :as event] event-data]
  (case event-name
    :profile/get-profiles-overview-success
    (user-journey-event app-started-event)

    :centralized-metrics/toggle-centralized-metrics
    (metrics-enabled-event event)

    :set-view-id
    (track-view-id-event event)

    (:wallet/select-account-tab :wallet/select-home-tab)
    (navigated-to-collectibles-tab-event event)

    ;; => NOTE: Here `event-data` was computed outside the centralized metrics layer and was extracted by the global interceptor when calling tracked-event 
    :wallet/flush-collectibles-fetched
    (collectilbes-fetched-event event-data)

    nil))

Thanks for sharing your thoughts @ilmotta

I agree with you about not passing db to this event. I'm still not sure about growing app-db horizontally instead of nested, but that's a conversation for another moment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this solution I proposed to work the app-db key would need to be separate from everything else because that would be the only way to create a single abstraction that works for every re-frame event we might want to track. For instance, if we store the computation for MixPanel events nested in other parts of the app-db, we will be in trouble because the interceptor would need to know how to reach the app-db leaf for various events, then it will complicate the interceptor because it wouldn't know anymore how to get data generically.

Maybe I'm missing something, but this solution wouldn't grow the app-db horizontally, just one extra key for any and all events that need to send computed data to MixPanel.

Copy link
Contributor Author

@vkjr vkjr Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ilmotta, I have one more thought. Personally I would be happy to avoid both - referencing db from metrics namespace and having a separate key in re-frame db, because it should be not only set but also cleaned after every event to make sure that intercepted event didn't get wrong data from previous event. And some events just don't need it.

The way that seems more clean for me - is having dedicated event every time when we need some additional data to be computed from db and intercepting that event.
For example in current case when :wallet/flush-collectibles-fetched event fired it can dispatch one more event - :metrics/track-user-has-collectibles? which receives parameter :has-collectibles? as an argument and this event we can intercept from metrics layer.

So flush-collectibles will become:

(defn flush-collectibles
  [{:keys [db]}]
  (let [collectibles-per-account (get-in db [:wallet :ui :collectibles :fetched])
        accounts               (get-in db [:wallet :accounts])
        has-collectibles? (some #(pos? (:current-collectible-idx %)) (vals accounts))]
    {:db (-> db
             (update-in [:wallet :ui :collectibles] dissoc :pending-requests :fetched)
             (update-in [:wallet :accounts] move-collectibles-to-accounts collectibles-per-account))
     :fx [:dispatch
          [:metrics/track-user-has-collectibles? has-collectibles?]]}))

We discussed with @shivekkhurana that he would prefer not to pollute our events code with metrics-related calculations but if we anyway need some calculations to be done, it better be done not in the metrics namespace as you suggested, and i think it is better to be in a form of separate event. It name would provide a very clear explanation of what is going on here, more high-level than plain assoc-in.
Wdyt?

Copy link
Contributor

@ilmotta ilmotta Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because it should be not only set but also cleaned after every event

Good point @vkjr about clean-up. We can use the interceptor itself to clean up because the context we receive has the :db effect and we can grab the value of :centralized-metrics/event-data, use it to generate the MixPanel event and then return a new context with the :db effect dissoc'ing :centralized-metrics/event-data data key. The result is that nothing will leak to another event and, actually, the app-db won't be modified to persist the computed event data. No subscription recomputations should happen if we do this correctly.


I also agree with @shivekkhurana that we shouldn't pollute normal code with metrics code and that it's highly desirable to keep the metrics layer dumb.

I think the idea of using separate events can work well 👍🏼, but I would personally use that only when necessary because of the overhead and extra verbosity. For example, in the example you provided, a simple assoc gets the job done and it's decent for readability. Also considering the clean-up will be automatic by the interceptor.

Including event data computation in the tracked re-frame event is not bad actually, there's an advantage, the computation can be unit tested with the rest of the handler, which is actually quite important because if we start to track incorrect computations it will really complicate analysis in MixPanel and I doubt QAs will have the time to manually double-check metrics are generated correctly. Actually unfair to ask them to do that.

(defn flush-collectibles
  [{:keys [db]}]
  (let [collectibles-per-account (get-in db [:wallet :ui :collectibles :fetched])
        accounts                 (get-in db [:wallet :accounts])
        has-collectibles?        (some #(pos? (:current-collectible-idx %)) (vals accounts))]
    {:db (-> db
             (update-in [:wallet :ui :collectibles] dissoc :pending-requests :fetched)
             (update-in [:wallet :accounts] move-collectibles-to-accounts collectibles-per-account)
             (assoc :centralized-metrics/event-data has-collectibles?))}))

@vkjr
Copy link
Contributor Author

vkjr commented Sep 23, 2024

@ulisesmac, thanks a lot for the review, all your comments are highly relevant! as always ;)

@@ -35,12 +35,16 @@
:on-press #(rf/dispatch [:show-bottom-sheet {:content new-account}])
:type :add-account})

(def first-tab-id :assets)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vkjr, I thought we had a place for setting up initial state. For the wallet it would be wallet/db.cljs. If we do that, then we can remove the or around the sub :wallet/home-tab.

Comment on lines 81 to 74
:wallet/select-account-tab
(when (= second-parameter :collectibles)
(navigated-to-collectibles-tab-event :account))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, it'd be great to rename it, second-parameter provides zero context, it's almost the same as name it parameter

@ulisesmac, the name second-parameter was chosen intentionally because it signals that we don't know what it is, it could be anything and that it is only the second (could be a third and so on). We can rename it to something else of course, but it will always be generic because the value varies per event. Please, see my other comment for the function tracked-event where I give an alternative to how we could design the code.

(navigated-to-collectibles-tab-event :home))

:wallet/flush-collectibles-fetched
(collectilbes-fetched-event db)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vkjr @shivekkhurana, why do we need to track the total number of collectibles from users? Which business answers we intend to answer from collecting this? I'm asking this just so we consider if we can collect less data that can be attributed to any individual, but which would still be valuable to learn.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking this parameter was solely my initiative and as a developer I felt comfortable to track as much info as possible :) But agree that if we intend to track less data then this is a wrong approach.
But I think it would be valuable to know if user has any collectibles or not. We can pass a boolean parameter to event, like :has-collectibles?. @shivekkhurana, @ilmotta, wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you @vkjr, knowing if users have collectibles is useful and it's a good middle ground in terms of privacy because knowing exactly how many collectibles users have seem not super necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The number of collectibles tell us if people are using the NFT features. If there are no collectibles for the majority of the users, then this screen becomes less important.

Comment on lines 69 to 70
(defn tracked-event
[[event-name second-parameter]]
[[event-name second-parameter] db]
Copy link
Contributor

@ilmotta ilmotta Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tagging me @ulisesmac. Below I share a few roughly sketched ideas, not blocking the PR from my PoV. Text is long, hard to explain, but I hope it makes some sense.

I think it's not the best that the tracking code now has access to the entire app-db and knows the inner details of its structure. Better I would say we should give the least amount of data to this layer and keep it plain simple, also because it's not well QAed.

If we need to compute something from the app-db that needs to be tracked, and which is decoupled from the metrics layer we could:

  1. Define a root key in the app-db, for example, :centralized-metrics/event-data, which will store any arbitrary data that's safe to be tracked and which is serializable to JSON.
  2. The event that wants to be tracked can assoc to this root key. In this case, in event :wallet/flush-collectibles-fetched it would assoc the total number of collectibles.
  3. The interceptor centralized-metrics-interceptor will then extract the value of :centralized-metrics/event-data and pass it to tracking/tracked-event.

With this solution, the centralized metrics layer will not know how to compute anything, but will still be able to get computed track data and/or arguments passed to any re-frame event. This would scale to any other event in re-frame and we would still keep this layer simple/dumb.


About the signature of the function tracked-event, an alternative is to implement tracked-event with the case macro, but the event argument (first arg to tracked-event) would be passed to downstream functions, and then we would only destructure the re-frame event in each specific function signature, thus solving the naming problem of second-parameter or how to best destructure the event because each function will destructure and name the arguments knowing what they really are.

In this example, there's no generic name like second-parameter and we don't need to worry about the full signature of the re-frame event:

(defn- metrics-enabled-event
  [[_ enabled?]]
  (key-value-event "events.metrics-enabled" :enabled enabled?))

(defn tracked-event
  [[event-name :as event] event-data]
  (case event-name
    :profile/get-profiles-overview-success
    (user-journey-event app-started-event)

    :centralized-metrics/toggle-centralized-metrics
    (metrics-enabled-event event)

    :set-view-id
    (track-view-id-event event)

    (:wallet/select-account-tab :wallet/select-home-tab)
    (navigated-to-collectibles-tab-event event)

    ;; => NOTE: Here `event-data` was computed outside the centralized metrics layer and was extracted by the global interceptor when calling tracked-event 
    :wallet/flush-collectibles-fetched
    (collectilbes-fetched-event event-data)

    nil))

@vkjr
Copy link
Contributor Author

vkjr commented Sep 24, 2024

@ilmotta, I like the approach you suggested, thanks!

@vkjr vkjr force-pushed the collectible-metrics branch 2 times, most recently from 7d7f46b to 8357b42 Compare September 27, 2024 14:58
@vkjr
Copy link
Contributor Author

vkjr commented Sep 27, 2024

@ilmotta, @ulisesmac, code updated, please take a look

  • event stores calculated data in dedicated re-frame db key
  • metrics take data from this key and cleanup after (p.s we use db from :effects now, because :coeffects don't have changes inroduced by event)
  • tracking-event renamed to metrics-event for clarity and takes only one arg - map with keys to solve problem when keys aren't always required

@vkjr vkjr requested a review from ilmotta September 27, 2024 15:19
@churik
Copy link
Member

churik commented Oct 14, 2024

@mohsen-ghafouri @vkjr
please, let us know when it should be tested, thanks

@VolodLytvynenko
Copy link
Contributor

Hi @vkjr, did I understand correctly from the QA side that it's expected to check in this PR if no regression issues appear?

@VolodLytvynenko VolodLytvynenko self-assigned this Oct 14, 2024
@status-im-auto
Copy link
Member

86% of end-end tests have passed

Total executed tests: 7
Failed tests: 1
Expected to fail tests: 0
Passed tests: 6
IDs of failed tests: 727229 

Failed tests (1)

Click to expand
  • Rerun failed tests

  • Class TestWalletMultipleDevice:

    1. test_wallet_send_eth, id: 727229

    Device 2: Find `Text` by `xpath`: `//android.view.ViewGroup[@content-desc='container']/android.widget.TextView[@text='Ether']/../android.widget.TextView[3]`
    Device 2: `Text` is `0.08789 ETH`

    critical/test_wallet.py:159: in test_wallet_send_eth
        self.errors.verify_no_errors()
    base_test_case.py:192: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Sender balance is not updated on Etherscan, it is 0.3933 but expected to be 0.3934
    



    Passed tests (6)

    Click to expand

    Class TestCommunityOneDeviceMerged:

    1. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    2. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    Class TestWalletMultipleDevice:

    1. test_wallet_send_asset_from_drawer, id: 727230

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_message_edit, id: 702843
    Device sessions

    Class TestWalletOneDevice:

    1. test_wallet_add_remove_regular_account, id: 727231
    Device sessions

    @VolodLytvynenko
    Copy link
    Contributor

    @churik already explained to me how to handle with question

    @vkjr
    Copy link
    Contributor Author

    vkjr commented Oct 14, 2024

    @VolodLytvynenko, this PR is not intended to be tested yet because it contains implementation that should be redone. So as a first step we expect that PR #21379 will be merged and this one will go after.

    @VolodLytvynenko
    Copy link
    Contributor

    @VolodLytvynenko, this PR is not intended to be tested yet because it contains implementation that should be redone. So as a first step we expect that PR #21379 will be merged and this one will go after.

    Got it. Thank you. I will take the current PR after merging #21379

    @mohsen-ghafouri
    Copy link
    Contributor

    My PR is merged.
    CC @vkjr @VolodLytvynenko

    @shivekkhurana
    Copy link
    Contributor

    After Mohsen's PR, I think this will need to be reworked ?
    I have marked it as low prio for now. But please continue to work on this. We need to be expanding the surface area of analytics.

    @vkjr

    @VolodLytvynenko
    Copy link
    Contributor

    hi @vkjr could you please rebase current PR and resolve existing conflicts? thanx

    @VolodLytvynenko VolodLytvynenko removed their assignment Oct 18, 2024
    @pavloburykh pavloburykh self-assigned this Oct 18, 2024
    @pavloburykh
    Copy link
    Contributor

    Hey @vkjr thanks for the PR. Got one question: according to PR description

    user fetched collectibles, we track the number of collectibles fetched

    Where can I find collectibles number parameter? I do not see it in MixPanel wallet-collectibles-fetched event

    Mixpanel - Product Analytics 2024-10-21 17-08-41

    @pavloburykh
    Copy link
    Contributor

    @vkjr another question related to

    user navigated collectible details

    Do I understand correctly the this event is triggered when user opens collectible from the list?

    I see such event which is named navigation and wallet.collectible. Am I looking at the correct event? Maybe it would be better to specify the name as navigated-to-collectible-details instead of just navigation?

    Mixpanel - Product Analytics 2024-10-21 17-30-03

    @status-im-auto
    Copy link
    Member

    88% of end-end tests have passed

    Total executed tests: 8
    Failed tests: 1
    Expected to fail tests: 0
    Passed tests: 7
    
    IDs of failed tests: 702843 
    

    Failed tests (1)

    Click to expand
  • Rerun failed tests

  • Class TestCommunityMultipleDeviceMerged:

    1. test_community_message_edit, id: 702843

    Device 2: Find `Text` by `xpath`: `//android.view.ViewGroup[@content-desc='chat-item']//android.widget.TextView[contains(@text,'https://status.app/c/')]`
    Device 2: Wait for element `Button` for max 120s and click when it is available

    Test setup failed: critical/chats/test_public_chat_browsing.py:350: in prepare_devices
        self.community_2.join_community()
    ../views/chat_view.py:420: in join_community
        self.join_button.wait_and_click(120)
    ../views/base_element.py:100: in wait_and_click
        self.wait_for_visibility_of_element(sec)
    ../views/base_element.py:147: in wait_for_visibility_of_element
        raise TimeoutException(
     Device 2: Button by accessibility id:`show-request-to-join-screen-button` is not found on the screen after wait_for_visibility_of_element
    



    Device sessions

    Passed tests (7)

    Click to expand

    Class TestCommunityOneDeviceMerged:

    1. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    2. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
    Device sessions

    Class TestWalletOneDevice:

    1. test_wallet_add_remove_regular_account, id: 727231
    2. test_wallet_balance_mainnet, id: 740490

    Class TestWalletMultipleDevice:

    1. test_wallet_send_asset_from_drawer, id: 727230
    2. test_wallet_send_eth, id: 727229

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    @vkjr
    Copy link
    Contributor Author

    vkjr commented Oct 22, 2024

    @pavloburykh, thanks for checking the pr!

    As for amount of collectibles fetched - in this PR discussion we came to an agreement that this is too much info to gather while we want to minimize amount of info gathered, so now we just collect information if user have any collectibles at all or not.

    As for naming navigation event. Yes, you are looking at the correct one. All screen navigation events we intercept in a single place under common name "navigation" while more specific cases covered with additional events. So I though it is more suitable to leave visiting collectible details page under "navigation" name to not introduce discrepancy.

    @pavloburykh
    Copy link
    Contributor

    @vkjr thanks for the PR. It is ready for merge.

    @vkjr vkjr merged commit f8d4f56 into develop Oct 22, 2024
    5 checks passed
    @vkjr vkjr deleted the collectible-metrics branch October 22, 2024 15:10
    ilmotta pushed a commit that referenced this pull request Oct 22, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    Archived in project
    Status: Done
    Development

    Successfully merging this pull request may close these issues.

    Track collectibles-related metrics
    10 participants