@inproceedings{raheja-etal-2024-enhancing,
    title = "Enhancing Large Language Models through Transforming Reasoning Problems into Classification Tasks",
    author = "Raheja, Tarun  and
      Sinha, Raunak  and
      Deepak, Advit  and
      Healy, Will  and
      Srinivasa, Jayanth  and
      Lee, Myungjin  and
      Kompella, Ramana",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.532",
    pages = "6007--6016",
    abstract = "In this paper, we introduce a novel approach for enhancing the reasoning capabilities of large language models (LLMs) for constraint satisfaction problems (CSPs), by converting reasoning problems into classification tasks. Our method leverages the LLM{'}s ability to decide when to call a function from a set of logical-linguistic primitives, each of which can interact with a local {``}scratchpad{''} memory and logical inference engine. Invocation of these primitives in the correct order writes the constraints to the scratchpad memory and enables the logical engine to verifiably solve the problem. We additionally propose a formal framework for exploring the {``}linguistic{''} hardness of CSP reasoning-problems for LLMs. Our experimental results demonstrate that under our proposed method, tasks with significant computational hardness can be converted to a form that is easier for LLMs to solve and yields a 40{\%} improvement over baselines. This opens up new avenues for future research into hybrid cognitive models that integrate symbolic and neural approaches.",
}

In this paper, we introduce a novel approach for enhancing the reasoning capabilities of large language models (LLMs) for constraint satisfaction problems (CSPs), by converting reasoning problems into classification tasks. Our method leverages the LLM{'}s ability to decide when to call a function from a set of logical-linguistic primitives, each of which can interact with a local {``}scratchpad{''} memory and logical inference engine. Invocation of these primitives in the correct order writes the constraints to the scratchpad memory and enables the logical engine to verifiably solve the problem. We additionally propose a formal framework for exploring the {``}linguistic{''} hardness of CSP reasoning-problems for LLMs. Our experimental results demonstrate that under our proposed method, tasks with significant computational hardness can be converted to a form that is easier for LLMs to solve and yields a 40{\%} improvement over baselines. This opens up new avenues for future research into hybrid cognitive models that integrate symbolic and neural approaches.
Raheja, Tarun Sinha, Raunak Deepak, Advit Healy, Will Srinivasa, Jayanth Lee, Myungjin Kompella, Ramana
Enhancing Large Language Models through Transforming Reasoning Problems into Classification Tasks