Asking for the source code of Transformer/Multi. BERT/XLM

[Jiahuan-Pei]

Hi Zihan,

Thanks for contributing your source code of the paper https://arxiv.org/pdf/1911.09273.pdf.

In this source code, I have not find the Transformer/Multi. BERT/XLM models, as they are the state-of-the-art models as reported in the paper.

Could you please share the models or let us know how to reproduce the reported model.

Regards,
Jiahuan

[Jiahuan-Pei]

Could you please also share the settings for reproducing the reported results?

This is what we get using your settings in the README file.

Goal ACC: 0.4840. Joint ACC: 0.1306. Request ACC: 0.8092. Avg ACC: 0.4699

[zliucr]

Hi,

In our experiments, using multilingual word embeddings can achieve comparable or sometimes better results than using Multi. BERT/XLM. If you want to use Multi.BERT/XLM, you can simply replace the word embeddings with the Multi. BERT/XLM embeddings.

As for reproducing the results, we have provided the scripts in the README (in "How to run" section).

Thank you!

[Jiahuan-Pei]

Thanks for your kind reply!

As for reproducing the results, we have provided the scripts in the README (in "How to run" section).
==> Yes, we use the the scripts in the README (in "How to run" section), DST Task, Zero-shot Adaptation for German. And the reproduced results are:
Goal ACC: 0.4840. Joint ACC: 0.1306. Request ACC: 0.8092. Avg ACC: 0.4699
The results reported in the paper are:

As for model, I do not think it's only the problem of embeddings.
(1) In the Figure 3 in the paper (https://arxiv.org/pdf/1911.09273.pdf), it shows you used Transformer encoder. So I assume it's not only need to change Multi. BERT/XLM embeddings.
(2) In the Table 1, for German sub table, in the paper, it shows you used Model namely,
MUSE
XLM (MLM)∗ + Transformer
XLM (MLM+TLM)∗+ Transformer
Multi. BERT∗+ Transformer

Could you please provide more details about the Model and settings for reproducing the models?

[zliucr]

Hi,

The script you run should be able to reproduce the results of MUSE model. The hyper-parameter settings are in config.py file. Can you check whether you use the correct embeddings we have provided in the data folder? Thanks.

As for the code for Multi. BERT/XLM + Transformer, we are sorry that we didn't provide the code, since it is a bit messy in our codebase. If you need, we can try to wrap the corresponding code and upload it in next following days.

Thank you!

[Jiahuan-Pei]

Can you check whether you use the correct embeddings we have provided in the data folder? Thanks.
==> Yes, to make sure the reproducibility, we have NOT changed anything and only use all the codes/data in this repository.

[Jiahuan-Pei]

If you need, we can try to wrap the corresponding code and upload it in next following days.
==>Yes, of course :) It would be much more than appreciated! Thank you!

[bumbumya]

@zliucr
Thanks for your great work.
Can you share the code for Multi. BERT/XLM + Transformer?