DoublyCal

This repository contains the source code for the paper "Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence" accepted by IJCAI 2026.

1. Environment Setup

conda create -n doublycal python=3.9
conda activate doublycal
pip install -r requirements.txt 

2. Download Dataset

Download the publicly available RoG benchmark dataset:

cd src
bash scripts/download_ori_datasets.sh

The dataset will be saved to datasets/ori/.

3. Preprocessing

3.1 Extract & Calibrate KG Evidence

Extract KG evidence (i.e., constrained relational paths) and perform Bayesian calibration for confidence estimation:

bash scripts/evidence_preprocess.sh

The results are saved to datasets/paths.

3.2 Build Training Datasets

Building Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) datasets for training the Proxy model:

bash scripts/train_preprocess.sh

The results are saved to datasets/betap-[ALPHA].[BETA].

4. Training the Proxy Model

The training process of the Proxy model consists of two stages: Supervised Fine-tuning (SFT) followed by Reinforcement Learning (RL).

Access Token Setup: Remember to set the ACCESS_TOKEN for the base model llama2-7b-chat in src/training/sft_finetuning.py.

4.1 Supervised Fine-Tuning (SFT)

bash scripts/train_proxy_sft.sh

The Proxy model trained after SFT is saved to saved_model/betap0.5.0.5-sft. In our experiments, training was early-stopped at checkpoint-800 based on the validation loss.

4.2 Reinforcement Learning (RL)

bash scripts/train_proxy_rl.sh

The Proxy model trained after RL is saved to saved_model/betap0.5.0.5-rl. In our experiments, training was early-stopped at checkpoint-1200 based on the validation reward.

5. Inference

5.1 Generate Calibrated KG Evidence

Use the trained Proxy model to generate KG evidence with calibrated confidence scores:

bash scripts/gen_evidence_sft.sh

bash scripts/gen_evidence_rl.sh

The results are saved to results/gen_evidence.

5.2 Obtain LLM Predictions with Calibrated Confidence

The well-calibrated KG evidence serves as context to assist black-box LLMs in predicting the final answer and expressing confidence in the prediction:

API Key Setup: Remember to set the API_KEY in the corresponding environment file (e.g., src/env/.env.[MODEL_NAME]) before running the prediction script.

bash scripts/predict_answer.sh

Key script arguments:

• MODEL_LIST: Specifies the black-box LLMs to use (e.g., gpt-3.5-tubo).
• UQ_MODE: Sets the verbalized Uncertainty Quantification (UQ) technique (e.g., vanilla).
• RULE_PATH: Provides the file path to the KG evidence generated by the proxy model.