Wrong results for LastFM-Collaborative indexing

[mgulcin]

I'm trying to replicate the results from the paper (on LastFm dataset).

Even if I managed to replicate results for 'random' and 'sequential' indexing, I couldn't replicate results for 'collaborative' (OpenP5-T5-C (un/seen)).

Here is an example result I obtained, using same command and model (from checkpoints provided):
OpenP5-T5-C (unseen)

testing filtered LastFM dataset on sequential task
hit@5: 0.0027522935779816515
hit@10: 0.0027522935779816515
ndcg@5: 0.001392461064959632
ndcg@10: 0.001392461064959632

whereas results in the paper:

hit@5: 0.0504 
ndcg@5: 0.0332 
hit@10: 0.0724 
ndcg@10:0.0420

Can you please guide me with right command or right model (checkpoint)?

[xiaxin1998]

same problem here. I used the command in the repo.

[lifemomo]

I ran into the same problem, I used ML100K and the checkpoint provided by the author, and the result was as follows:
hit@5: 0.010604453870625663
hit@10: 0.015906680805938492
ndcg@5: 0.005464306777753837
ndcg@10: 0.007180079919836173
But the document gives the following results:
hit@5: 0.0530
hit@10: 0.1006
ndcg@5: 0.0331
ndcg@10:0.0481

[lifemomo]

If it has anything to do with the collaborative_token_size 100 and collaborative_cluster 10 parameters in the training script, I'll try again