Cant reproduce results

[ernlavr]

Hi,

I am running the codebase with the following commands and getting perplexity 4629, what's up with the reproducibility?

echo "1/2: PERFORMING DATA WHITENING"
python SVDLLM.py --step 1 --ratio 0.2 --model jeffwan/llama-7b-hf --whitening_nsamples 8 --dataset wikitext2 --seed 0 --model_seq_len 2048 --save_path $FINE_TUNE_PATH

echo "2/4: PERFORMING WHITENED EVAL"
python SVDLLM.py --step 4 --model_path $FINE_TUNE_PATH/jeffwan_llama_7b_hf_whitening_only_0.8.pt

echo "3/4: FINE-TUNING U-V MATRICES"
python utils/LoRA.py --prune_model $FINE_TUNE_PATH/jeffwan_llama_7b_hf_whitening_only_0.8.pt --data_path yahma/alpaca-cleaned --output_dir LORA_OUTPUT_PATH  --lora_r 8 --num_epochs 2 --learning_rate 1e-4 --batch_size 64

echo "4/4: FINAL EVAL"
python SVDLLM.py --step 4 --model_path $FINE_TUNE_PATH/jeffwan_llama_7b_hf_whitening_only_0.8.pt --lora output_dir/jeff_llama/lora

PPL after performing whitened eval
PPL after pruning: {'wikitext2': 4629.2990907940175}

PPL after performing eval with LoRA (default training settings)
PPL after pruning: {'wikitext2': 7571.078142011893}