how to reproduce community discovery experiment

[sillysun]

Hi, i have question in reporducing the key alignment experiment both in COPT and GOT. i did this way:

1. use nx.stochastic_block_model to construct a 40-node 4-community graph, where the probs of adding edges within/between communities are 0.9, 0.1 respectively.
2. then i delete 10% edges (about 20) and permute the new graph.
3. use graph.graph_dist(args, plot=False, Ly=Ly, take_ly_exp=False) to calculate COPT distance, and the args here is same as that in utils.parse_args().
4. also use got_stochastic.find_permutaion as in runGraph.perm_mi

However, the nmi is only about 0.5, not nearly 1 like table 2 shows. Now i'm not sure the exact reason, but two guesses:

1. does the probs parameter of nx.stochastic_block_model matters? i noticed that this parameter in utils.create_graph(40, gtype='block', params=params, seed=seed) is [0.97, 0.01, 0.01, 0.01], my case use [0.9, 0.1, 0.1, 0.1].
2. or should we tune some hyper-parameter, while both of COPT and GOT are a little fuzzy in this experiment？like in copt, the args focus on self.optim: lr and hiking.

hope for you reply, thx !

[twistedcubic]

@sillysun the two reasons you listed could be the reason. The construction of graphs, edge removal, as well as permutation are done here.

Note that for some graph structures, removal of edges naturally makes the two graphs before and after dissimilar, therefore the ground truth normalized MI should not be close to 1. In the paper, we use the NMI as a measure to show commensurability between GOT and COPT when the numbers of nodes of the two graphs are equal, and indeed your results show the same.