Bridging Large Language Models and Logic Programming

A novel framework for automated natural language to Prolog code generation and execution.

Research Paper

This repository implements the framework described in the paper:

"Bridging Large Language Models and Logic Programming" Authors: Valerio Crocetti, Aldo Franco Dragoni

🎯 Abstract

This work presents an innovative approach to bridge the gap between natural language processing and formal logic programming. Our framework leverages Large Language Models (LLMs) to automatically translate natural language problem descriptions into executable Prolog code, incorporating novel self-correcting mechanisms and adaptive temperature scaling for enhanced code quality and reliability.

🚀 Key Innovations

🔄 Self-Correcting LLM Pipeline

• Iterative Error Detection: Automatically identifies syntax and semantic errors in generated code
• Feedback-Driven Refinement: Uses error information to guide subsequent code generation attempts
• Multi-Stage Validation: Combines syntactic parsing with execution-based verification

🎯 Dynamic Temperature Scaling

• Adaptive Creativity Control: Progressively increases LLM temperature based on correction attempts
• Exploration vs. Exploitation: Balances deterministic fixes with creative problem-solving approaches
• Convergence Optimization: Ensures systematic exploration of solution space

⚖️ Dual-Mode Logic Processing

• Standard Mode: For computational problems requiring variable bindings and mathematical operations
• Boolean Mode: Specialized handling for true/false logical reasoning tasks
• Context-Aware Selection: User-selectable modes optimized for different problem types

🔬 Technical Approach

Architecture Overview

Natural Language Input → Problem Analysis → LLM Code Generation → 
Prolog Parsing → Syntax Validation → Execution → Result Translation
     ↑                                                           ↓
     ←←←←←←←← Error Feedback & Self-Correction Loop ←←←←←←←←←←←←←←←

Core Components

• Prompt Engineering: Specialized prompts for Prolog code generation
• Code Filtering: Regex-based extraction of valid Prolog clauses from LLM output
• Query Synthesis: Automatic generation of appropriate Prolog queries
• Result Translation: Conversion of technical Prolog output to natural language

⚠️ Platform Support

Fully Tested: Windows 10/11
Experimental: macOS and Linux (manual configuration may be required for SWI-Prolog integration)

🏃 Quick Start

Prerequisites

• Python 3.9+
• SWI-Prolog
• Hugging Face API access

Installation

1. Install SWI-Prolog:

   # Windows: Download from https://www.swi-prolog.org/download/stable
   # macOS: brew install swi-prolog
   # Linux: sudo apt-get install swi-prolog

2. Install Python Dependencies:

   pip install -r requirements.txt

3. Configure API Access: Edit config.json:

   {
       "HF_API_TOKEN": "your_huggingface_api_token_here",
       "SWI_PROLOG_PATH": "C:/Program Files/swipl",
       "MODEL_NAME": "Qwen/Qwen2.5-Coder-32B-Instruct",
       "INFERENCE_PROVIDER": "nebius"
   }

4. Launch Application:

   python run.py

   Navigate to http://localhost:5000

📊 Reproducing Paper Results

To replicate the experimental results presented in our paper, follow these steps:

1. Configure Test API Access: The test suite uses a separate configuration file. Create or edit test_config.json in the root directory and add your Hugging Face API token:

   {
       "HF_API_TOKEN": "your_huggingface_api_token_here"
   }

2. Run the Comprehensive Test Suite: Execute the main test script from the root directory:

   python test/test.py

   The script will run all model configurations against the test dataset and save the results in the Test_Results/ directory.

3. ⚠️ Important Note: The test script automatically checkpoints results after each question, allowing you to resume the process at any time without losing progress.
   Warning: Running the full test suite requires a large number of API tokens due to the extensive number of

🔧 Advanced Configuration

Model Selection

{
    "MODEL_NAME": "Qwen/Qwen2.5-Coder-32B-Instruct",
    "INFERENCE_PROVIDER": "nebius"                     
}

Temperature Control Parameters

• Initial Temperature: 0.0 (deterministic)
• Max Temperature: 1.0 (highly creative)
• Scaling Factor: 0.1 per iteration

📋 Implementation Details

Core Modules

• chatbot.py: Main problem-solving logic and coordination
• prolog_utils.py: LLM integration, code filtering, and execution
• config.py: Configuration management and path resolution
• Web Interface: Flask-based frontend with real-time interaction

Key Algorithms

1. Iterative Code Refinement: Progressive improvement through error feedback
2. Temperature Annealing: Systematic exploration of solution space
3. Dual-Mode Processing: Problem-type-specific optimization strategies

🐛 Troubleshooting

Issue	Solution
Startup Errors	Verify Python 3.9+ and dependency installation
SWI-Prolog Not Found	Check installation path in config.json
API Connection Issues	Validate Hugging Face token and model access
Empty Results	Start with simpler problems, check debug output

📄 License

MIT License. This project utilizes the Qwen model family through Hugging Face.

---

📑 How to Cite

If you use this repository, please cite our paper as follows (currently accepted, to appear in IEEE Xplore):

@INPROCEEDINGS{BLLMLP2025,
  author    = {Crocetti, Valerio and Dragoni, Aldo Franco},
  title     = {Bridging Large Language Models and Logic Programming},
  booktitle = {2025 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)},
  year      = {2025},
  address   = {Ancona, Italy},
  month     = {Oct. 22--24},
  note      = {(to appear)}
}

This framework represents a significant step forward in automated program synthesis, demonstrating the potential of LLMs for formal logic programming tasks.