Does BERT understand code? - An exploratory study on the detection of architectural tactics in codeshort-paperResearch Track
Quality-driven design decisions are often addressed by using architectural tactics that are re-usable solution options for certain quality concerns. However, it is not sufficient to only make good design decisions but also to review the realization of design decisions in code. As manual creation of traceability links for design decisions into code is costly, some approaches perform structural analyses to recover traceability links. However, architectural tactics are high-level solutions described in terms of roles and interactions and there is a wide range of possibilities to implement each. Therefore, structural analyses only yield limited results. Transfer-learning approaches using language models like BERT are a recent trend in the field of natural language processing. These approaches yield state-of-the-art results for tasks like text classification. In this paper, we experiment with BERT and present an approach to detect architectural tactics in code by fine-tuning BERT. A 10-fold cross-validation shows promising results with an average F1-Score of 90%, which is on a par with state-of-the-art approaches. We additionally apply our approach on a case study, where the results of our approach show promising potential but fall behind the state-of-the-art. Therefore, we discuss our approach and look at potential reasons as well as issues and downsides. Moreover, we present ideas for future work to improve such a transfer-learning approach.
Fri 18 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:10 - 16:50 | S13: Smells and Technical Debt (II)Research Papers / Doctoral Symposium at ECSA 2020 Teams Channel Chair(s): Xabier Larrucea Tecnalia, Gabriel A. Moreno Carnegie Mellon University Virtualization support: Claudio Di Sipio | ||
16:10 20m | Does BERT understand code? - An exploratory study on the detection of architectural tactics in codeshort-paperResearch Track Research Papers Jan Keim Karlsruhe Institute of Technology (KIT), Angelika Kaplan Karlsruhe Institute of Technology, Anne Koziolek Karlsruhe Institute of Technology, Mehdi Mirakhorli Rochester Institute of Technology | ||
16:30 20m | A Semiautomatic Approach to Identify Architectural Technical Debt from Heterogeneous ArtifactsDoctoral Symposium Doctoral Symposium Boris Rainiero Perez Gutierrez University of Los Andes, Colombia |