ConceptSearch: Towards Efficient Program Search Using LLMs for Abstraction and Reasoning Corpus (ARC) (Student Abstract)

摘要

The Abstraction and Reasoning Corpus (ARC) poses a significant challenge to artificial intelligence, demanding broad generalization and few-shot learning capabilities that remain elusive for current deep learning methods, including large language models (LLMs) (Chollet 2019). While LLMs excel in program synthesis, their direct application to ARC yields limited success. To address this, we introduce ConceptSearch, a novel function-search algorithm that leverages LLMs for program generation and employs a concept-based scoring method to guide the search efficiently. Experimental results demonstrate that ConceptSearch outperforms direct GPT-4 prompting, with our novel scoring function boosting efficiency by ~30% compared to the baseline Hamming distance scoring. Code at https://github.com/kksinghal/concept-search