Enterprise AI Analysis

Our approach focuses on enhancing Large Language Models (LLMs) through parameter-efficient fine-tuning (PEFT), specifically using QLoRA, to specialize them for CUDA-to-SYCL kernel translation. We selected a diverse set of 19 open-source code models, ranging from 0.5 to 34 billion parameters, including CodeLlama, StarCoderBase, HPC-Coder, Qwen2.5-Coder, GraniteCodeBase, and CodeGemma.

The training dataset comprised 22 pairs of CUDA and corresponding SYCL compute kernels sourced from HeCBench, a comprehensive suite of heterogeneous computing benchmarks. This curated dataset was designed to expose the models to real-world parallel computing constructs. For evaluation, a separate test dataset of 62 CUDA kernels from HeCBench, ensuring no overlap with the training set, was used. A custom verification harness automated the compilation and execution of generated SYCL codes on an H100 GPU using the Intel DPC++ compiler with CUDA Plugin, validating both functional correctness and performance.

Initial evaluations revealed that most base LLMs struggled significantly with CUDA-to-SYCL translation, yielding correctness rates close to zero. However, parameter-efficient fine-tuning dramatically improved their capabilities, adapting them to the intricate semantics of parallel programming models.

The study also revealed a non-linear relationship between model size and correctness, with some smaller models outperforming larger ones in specific scenarios, though larger models generally showed more capacity to learn complex patterns when fine-tuned. Lower generation temperatures (e.g., 0.2) generally yielded more focused and correct outputs compared to higher temperatures, which introduced more variability.

Our future work will extend ChatPORT's capabilities to support other parallel programming models, including OpenMP and OpenACC. We aim to develop a comprehensive porting solution that provides performance portability across a wider range of hardware accelerators. Additionally, we plan to investigate the inference capabilities of ChatPORT for similar programming models without requiring further fine-tuning, exploring its zero-shot or few-shot learning potential.

Further research will also focus on integrating a feedback loop for self-correction, similar to approaches that use compilation errors and refactoring prompts, to iteratively refine the model's output and achieve even higher correctness rates, moving closer to fully autonomous and robust code translation for HPC.