Skip to content

NeurIPS

2024

SEMCODER: Training Code Language Models with Comprehensive Semantics Reasoning

code

PyX: a clean Python corpus synthesized from existing dataset magicoder and WizardCoder. monologue reasoning: try to understand and explain the code to themselves, like rubber-duck debugging

Existing literature on program semantics include: 1. approximate 2. symbolic 3. operational 4. abstract

2023

CROSSCODEEVAL: A Diverse and Multilingual Benchmark for Cross-File Code Completion

Use static analysis to pinpoint cross-file context.

Steps: - Use the StarCoder tokenizer to compute the number of tokens. - Replace import statements with empty classes, and run static analysis to identify undefined names which locate cross-file usages.

Evaluation (3 parts): - Only In-File Context (baseline) - Retrieved Cross-file Context: adopted the retrieve-and-generate (RG) framework by Zhang et al., 2023 for repository-level code completion. - Retrieval with Reference: the reference code completion to retrieve cross-file context

Metrics - Code Match: exact match (EM) and edit similarity (ES) - Identifier Match: API