evaluate pretrain stage1

[fanfan01fan]

Thank you for your excellent work!

I would like to ask about the stage1 of pre-training for Itemic-Text Alignment。
After the training is completed, how should we evaluate it and monitor the training progress? Is it necessary to construct a new dataset for this purpose? Could you provide some helpful code snippets?

Thanks again for your help! Looking forward to your reply.

[HonghuiBao2000]

@fanfan01fan

Thanks for your interest in our work.

In our experiments, we did not construct a specific evaluation dataset for Stage 1 (Itemic-Text Alignment). Instead, we mainly monitored the training loss during Stage 1 and evaluated the model after completing Stage 2 using downstream benchmarks.

The rationale is that Stage 1 serves primarily as an alignment/warm-up stage for the newly introduced item tokens, while the final performance is better reflected by the benchmark results obtained after the full training pipeline. Therefore, we did not develop a dedicated evaluation protocol or dataset specifically for Stage 1.

We hope this helps.

Honghui