Over the past decades, software systems have been widely used in almost all aspects of human lives, and are making our lives more and more convenient. However, software systems also inevitably suffer from bugs or faults, which can incur significant loss of properties and even lives. During modern software development, developers always reuse Application Programming Interfaces (APIs) provided by third-party libraries and frameworks rather than implementing from scratch to improve work efficiency and code quality. As a result, API related bugs spread widely due to API misuses, reducing software performance or causing software crashes [1], [2]. In Java, for example, the correct way to create a new thread is to call † Corresponding author. API Thread.start(), while Thread.run() is often misused, which does not create new threads. Therefore, analyzing the behaviors of API misuses is essential and can provide practical guidance for software development, especially the detection of API misuses. Such misuse behaviors can be potentially obtained from historical bug fixes.
A large number of historical bug fixes are publicly available on open-source platforms (e.g., GitHub [3] and Source-Forge [4]) and issue tracking systems (e.g., JIRA [5]). Many researchers have conducted various studies to analyze bug fixes and have many findings. For example, Martinez and Monperrus [6] found that most bug fixes are related to more than one source file, and statement-level code changes (e.g., inserting or deleting a statement) are most prevalent by analyzing 89,993 historical bug fixes. Even though these researchers have studied historical bug fixes, they only examined a small dataset, and the findings could be limited. In this paper, we conduct a study on API misuses by analyzing much more comprehensive bug-fixing dataset than any prior work.
Many techniques define API misuses as violations of certain frequent API usage patterns which are mined from source code [7]- [11]. These techniques were evaluated to be effective on some kinds of misuses. However, a recent study on stateof-the-art API-misuse detectors [1] shows that API misuses are prevalent and existing API-misuse detectors suffer from extremely low precision and recall on the widely used APImisuse dataset MUBENCH [12]. MUBENCH includes only 89 API misuses from 33 real-world projects, which not only makes it hard to analyze the distribution of various API misuses but also incurs dataset overfitting issues for evaluating existing API-misuse detection techniques.
Due to the limited amount of dataset in previous studies of bug-fixing and API misuses, in this paper, we conduct a more systematic and extensive empirical study by analyzing a large-scale bug-fixing dataset. We start with mining the historical repositories from GitHub Archive [13] that records the public GitHub timeline dataset. Then, we extract all bugfixing commits of Java projects from 2011 to 2018 according to specific search criteria, resulting in 528,546 bug-fixing commits. Finally, we extract fine-grained edit operations on AST (Abstract Syntax Tree) of buggy and fixed source code via leveraging GumTree [14] to identify API misuses. Following existing studies [1], we classify API misuses into different categories and statistically analyze their distributions. From the statistical results, we observe various practical guidelines regarding the importance of different categories of API misuses. For example, API calls (missing/redundant) are the most frequent (34.05% of all API misuses) and synchronization is the least frequent (0.06% of all API misuses). In addition, we find that about 38.33% API misuses are replacing API components (i.e., arguments, receiver, or name), which have never been systematically explored by any prior studies.
Inspired by the statistical results, we extract a large set of frequent API-misuse patterns for each category, and carry out two studies on API-misuse detection. (1) API-misuse detection in MUBENCH. Following previous studies [1], [15], we analyze how many API-misuse patterns in MUBENCH can also be found in the historical bug fixes and compare the results with existing state-of-the-art techniques. The results show that 12 out of 32 unique patterns in MUBENCH have already occurred before, and 7 of them cannot be detected by any existing techniques, demonstrating the possibility to build more effective misuse detection tools via analyzing large-scale historical bug fixes. (2) API-misuse detection in the wild. Based on our dataset, we perform a user study to manually analyze the usage constraints of 10 representative API-misuse patterns to explore whether the mined patterns can guide the design of future API-misuse detection tools. The results of the experiments on 688 Apache projects show that the misuses extracted from historical records still exist in the latest projects (7,541 potential API misuses in total). We have reported 149 misuses to developers. Up to now, 57 of them have already been confirmed and fixed while 15 are rejected. The promising initial results indicate that the misuse patterns complement existing approaches and can potentially improve the practicability of future API-misuse detection techniques.
In summary, this paper makes the following contributions:
In this section, we first introduce previous studies on historical bug fixes and then discuss existing techniques working on API-misuse detection.
During modern software development, a large number of historical bug fixes get accumulated on open-source platforms (e.g., GitHub and SourceForge) and issue tracking systems (e.g., JIRA). Understanding and analyzing the historical bug fixes can potentially provide practical guidelines for manual or automated bug detection [16]- [19], localization [20]- [25], and repair [26]- [30]. Therefore, various studies have been conducted on historical bug fixes. Zhong and Su [31] built a bug-fixing extraction tool named BugStat and analyzed more than 9,000 real bug fixes from six popular open-source Java projects. They found that most bug fixes only update existing source code files and do not add or delete source files. Additionally, they found that most bug fixes are related to if conditions, which is also confirmed by Soto et al. [32]. Pan et al. [33] also analyzed the distribution of different bugfixing patterns from seven open-source projects. They found that updating method call parameters and if conditions are the most common bug fixes. Recently, the benchmark De-fects4J [34] which includes various real bugs has been widely used in the field of software debugging. Sobreira et al. [35] studied 395 bugs in Defects4J and found that the top-3 most applied bug-fixing actions (77% of the total bugs) are addition of method calls, conditionals, and assignments. Martinez et al. and Madeiral et al. proposed different bug detection tools (Coming [36] and PPD [37] respectively) and evaluated them on Defects4J benchmark with promising performance.
Besides, there were also studies on domain-specific bug patterns or bug pattern distribution across different projects. Wan et al. [38] used the card-sorting approach to analyze the characteristics of bugs in Blockchain systems. Meng et al. [39] conducted an empirical study on StackOverflow posts related to code security, which revealed the huge gap between security theory and coding practices, and informed effective secure coding assistance. Similarly, Hanam et al. [40] studied the bug patterns in JavaScript projects and found that the same bug-fixing patterns exist among different JavaScript projects. Additionally, this finding was further confirmed on Java projects by Yue et al. [41] and Nguyen et al. [42]. Ray et al. [43] found that although source code is highly repetitive and predictable (like natural languages), the buggy code tends to be unnatural. After comparing with different statistical models, they found that "entropy" is a relatively good model to measure the similarity between code fragments, which can be used in search-based bug-fixing approaches.
Although various existing studies have already conducted on general software bugs, the API misuses have not been systematically explored yet. Therefore, we aim to perform an extensive study on the categorization and distribution of API misuses in the wild, which complements existing research. Furthermore, to the best of our knowledge, our study involves 528,546 historical bug-fixing commits from opensource projects and represents the most extensive study on historical bug fixes to date.
API usage is often subject to certain constraints [1]. For example, a resource must be released or closed after it is used. Violations of such usage patterns are regarded as API misuses. A large number of techniques have been proposed to detect API misuses automatically over the past decades.
Most approaches utilize data mining techniques to detect API misuses. Livshits et al. [44] introduced DynaMiner, aiming to mine software revision histories to detect misuses violating method pairs or certain mined state machines. Similarly, Li and Zhou [45] proposed PR-Miner to extract implicit programming rules of APIs by leveraging frequent itemset mining approaches on source code. Acharya and Xie [9] proposed to mine specifications from static program traces. Although such techniques utilize different data sources, they share the same assumption that the more frequent a pattern is, the higher possibility a pattern is correct. Other similar techniques also include DMMC [2], [46], GrouMiner [47], COLIBRI/ML [7], etc. Additionally, Wasylkowski et al. [8] and Nguyen et al. [48] proposed to employ graph theories for mining programming artifacts.
Researchers have also proposed data-mining-based techniques to detect other specific types of API misuses. Williams et al. [19] and Hovemeyer et al. [18] targeted missing NULL pointer checks, while Thummalapenta and Xie focused on exception-handling related misuses [10] and neglectedcondition misuses [11]. More recently, Liang et al. [17] aimed at detecting missing NULL pointer and resource-leaking misuses (e.g., missing API invocations to close resource accesses) via analyzing existing bug fixes of the same projects.
Besides data mining, researchers also employed program analysis and machine learning for API-misuse detection. Ramanathan et al. proposed CHRONICLER [49] and RGJ07 [50], utilizing path-sensitive control-flow or data-flow analysis to infer function precedence protocols or predicates. Wasylkowski and Zeller [51] proposed TIKANGA to combine static analysis with model checking for mining Computation Tree Logic (CTL) formulas. Nguyen et al. [52] leveraged Hidden Markov Model to check anomalies of call sequences. Most recently, Wen et al. [15] applied mutation analysis to discover API misuse patterns to improve the state-of-the-art.
Although various techniques have been proposed for detecting different types of API misuses, it is not even clear how different types of API misuses distribute among all API misuses or projects. Whether the existing techniques have covered all major types of API misuses is also not investigated. Furthermore, the recent widely used API-misuse dataset MUBENCH [12] includes only limited number of API misuses, and is insufficient for evaluating API-misuse detection techniques. Therefore, in this work, we aim to perform a systematic and extensive empirical study to characterize the distribution of various types of API misuses in the wild and construct a much larger dataset for API-misuse detection.
In this section, we introduce how we construct our dataset and conduct our study. We first introduce the collection of the dataset used in our study (Section III-A), and then introduce the categorization of API misuses (Section III-B). Finally, we discuss how we apply source-code differencing to infer API misuses from bug fixes (Section III-C).
We aim to mine API-misuse patterns from all bug-fixing commits of Java projects on GitHub. To collect our dataset, we first download all public GitHub events for all program languages from GitHub Archive [13] between 2011 and 2018. We then focus on Java projects and exclude all test cases since they are not functional parts and cannot reflect API usages. Next, following prior work [53], we identify a commit as a bug fix if its commit message contains the keywords ("fix" or "solve" or "repair") and ("bug" or "failure" or "issue" or "error" or "fault" or "defect" or "flaw" or "glitch"). Since the commit message may not identify bug-fixing commits accurately, we randomly select 100 commit samples, and two authors independently analyze them to check whether they are actual bug fixes. The result is that 94% of the identified bugfixing commits are real bug fixes, which provides us more confidence for the subsequent analysis. We also keep only unique commits by removing duplicates. Next, we download the source files before and after the code change for each bug-fixing commit. To mitigate the impact of irrelevant code changes, we discard commits or files that meet any of the following criteria.
• Commits with changes involving more than five Java files or six lines of source code [53], [54], since such commits may include many changes not related to bug fixes. • Non-Java files as they are irrelevant to Java API misuses.
• Java files that deleted or newly introduced in the commits. As a result, we finally get 528,546 bug-fixing commits (including 220,053 projects and 744,000 pairs of buggy and fixed files) for further API-misuse pattern mining.
Following prior study [1], we define an API misuse as a pair of a violation type and an API-usage element (e.g., API call, iteration, condition, and exception handling) involved in a bug fix. Besides the missing and redundant violation types studied in prior study [1], in this paper, we further investigate the type of replaced, describing that an API is incorrectly invoked and should be replaced with another one. This type of API misuses has never been systematically studied before, but is prevalent in real-world projects (will be shown in the following sections). In total, we classify API misuses into four basic categories, including Condition, Exception, Synchronization and API call, each of which consists of some specific subcategories. In the following, we demonstrate each category of API misuses in detail.
Condition. This category includes missing and redundant guard conditions for certain API invocations. Following the previous study [1], we further categorize it into the following three sub-categories: • Object state. This sub-category indicates the removed or newly introduced if condition relates to some variables that will be used in an API call immediately, e.g., a = list.get(i); => if(i > 0){a = list.get(i);}. Importantly, the three sub-categories of Condition are not orthogonal to each other as one if condition may belong to multiple categories. We will introduce this in detail in Section III-C.
Exception. This category includes missing and redundant exception handlers, following the definitions in the prior work [1]. More specifically, we further divide this category into two types of fine-grained code changes, i.e., inserting or deleting Try or Catch blocks. The reason is that in the studied commits, we find that some fixes are related to a complete try-catch statement, but some others may only involve catch blocks. Therefore, we analyze them separately. Especially, we regard a try or catch as API-related iff there exist API calls in the corresponding code block; otherwise, we consider it as API irrelevant. Besides, like the Condition category introduced above, a code change may involve both try and catch blocks. In such cases, we record these two categories respectively.
• Try. This category subjects to addition or deletion of try blocks, in which some API invocations reside. • Catch. This category subjects to addition or deletion of catch blocks, whose corresponding try blocks contain API invocations.
Synchronization. This category includes missing and redundant synchronizations in multi-threaded environments, following the previous study by Amann et al. [1]. The difference is // replaced arguments ---row=Math.abs(rand.nextInt(seed)%data.length-1); +++ row=Math.abs(rand.nextInt()%(data.length-1)); // replaced name ---nVal=tmp1.substring(0,tmp1.indexOf("\"")); +++ nVal=tmp1.substring(1,tmp1.lastIndexOf("\"")); // replaced name and arguments ---Statement stmt = con.createStatement(); +++ PreparedStatement stmt=con. prepareStatement(sql); // replaced receiver ---return this_path.equals(that_path); +++ return Objects.equals(this_path, that_path); Fig. 1: Examples of replaced bugs that we classify this type of code changes as an independent one rather than a sub-category of Condition. API call. Previous studies have focused on missing [45], [46] and redundant [52] API call misuses. However, more finegrained API changes (such as changing only the arguments, names or receiver objects of API invocations) were not been systematically and extensively studied by existing studies. In this paper, besides missing and redundant API changes, we further investigate the distributions of replaced API misuses, which include four categories in detail. For the missing and redundant API misuses, previous study [1] already introduced them (a.k.a. Method Call). To make the article self-contained, we redundantly explain them briefly.
• Missing & Redundant API call. Missing API call denotes that an API is not called at a certain place, where the API usage constraint requires the API as a must. For example, after opening a file and writing data, the API of File. close() should be called. Otherwise, errors would be incurred. This kind of code changes is usually related to those pairwise APIs that have usage dependency. Similarly, Redundant API call represents that an API is redundantly used at an improper place. For example, we cannot call the API of List.remove() to delete elements in a list that is being iterated over. Otherwise, exceptions would be raised. This kind of code changes is usually caused when the API has side-effects, whose execution may conflict with the followed functionality. • Replaced arguments. This category indicates that developers may pass incorrect arguments or arguments with wrong orders when invoking an API. This type of code changes usually appears in classes with multiple methods with similar functionalities for polymorphism, such as the first example shown in Figure 1, where the desired API is nextInt() without arguments. On the contrary, a wrong API nextInt(int) is used with an argument seed , which will constrain the upper bound of the generated random value. Please note that we consider replaced API misuses iff the types of arguments do not match before and after the change (order matters), while it is not our cases to change the referred object of same types. For example, the code change replacing 10 in nextInt (10) with 100 is not regarded as a replaced API misuse since the argument type is not changed and thus the API is not Before introducing the classification process in detail, we first introduce some preliminary concepts and notations: Definition 1. An abstract syntax tree (AST) is a partial ordered tree whose root node can be represented as a tuple l, v, p, i, C , where
• l: denotes the label of the root node of the subtree. (e.g., StringLiteral.)
• v: saves the value if it is a leaf node, otherwise is ⊥.
(e.g., 4.)
• p: represents its parent node in a super tree if exists, otherwise is ⊥.
• i: is the index of the root node in a super tree p, it is undefined if p = ⊥.
• C: contains a sequence of immediate child nodes in the subtree, it will be ∅ for leaf nodes.
Finally, based on the description of operations in GumTree [14], it will be straightforward to give the operation definitions under the AST definition. Definition 2. A GumTree operation is one of the following AST node changes:
• update(t, t ′ ): replace the subtree rooted t with a subtree rooted t ′ ;
• delete(t): deletes subtree rooted node t.
• insert(t, t ′ , i): adds a new node t as the i th child of node t ′ if t ′ is not ⊥. Otherwise, t is the new root node and the previous root node will be the only child of t.
• move(t, t ′ , i): moves subtree rooted node t to be the th child of node t ′ .
Particularly, we discard all move operations in the mapping process as it mainly changes the code structure but not the API itself, which is hard to be automatically analyzed as misuses. Next, we use the operations shown in Figure 4 as examples to demonstrate the mapping process in detail. According to the explanation for each category of API misuses in Section III-B, the operation delete(t1) will be classified as redundant API call, while the operation update(t2, t2 ′ ) will be classified as Replaced name of API call. In particular, when the name and arguments of a method call are changed together, the operations will be combined as one Replaced name and arguments (e.g., updating createStatement() to prepareStatement( String) in Figure 1). For the operation move(t4, t5, 2), we simply ignore it and in fact it does not misuse any API. Finally, as for the operation of insert(t5, t3, 2), from the figure we can see that a NULL check condition for the variable window is inserted, which is the returned value of API getWindow(). As a consequence, it will be classified as missing both NULL checks and Return value. Additionally, variable window is further used by API setWindowAnimations() that has control-dependency on the condition w.r.t. window. As a result, it will be classified as Object state as well. Thus, one operation may be classified into multiple categories in Condition. Based on this process, we automatically classify GumTree operations into different categories for further analysis.
According to the previous sections, we collect a large number of API misuses in real-world projects. In this section, we conduct various empirical studies and discuss the results.
In this research question, we count the number of edit operations from GumTree for API misuses and non-API misuses, and then analyze the distribution of different categories of API misuses. The result shows that there are 576,515 studied operations involving API misuses, which is about 51.7% of all edit operations (i.e., 48.3% for non-API misuses). This finding shows that developers tend to introduce API misuses frequently in modern software development. One potential reason is that developers are using more and more third-party libraries to save development efforts and improve code quality. To our best knowledge, this is the first study quantitatively demonstrating the importance of API-misuse in modern software systems. Table I presents the distributions of different categories of API misuses described in Section III-B. In the table, the first column represents the categories of misuses, and the last two columns represent the number of operations for missing and redundant misuses, respectively. Particularly, the percentage in the table represents the number of operations over that of all API-related operations (i.e., 576,515). As explained before, different categories may overlap each other (e.g., NULL checks and Return value). Besides, we also separately list the number and percentage of operations related to misuses of Replaced API, which is an important category in the study. In addition, for clarity, we omit the operations that are not related to API misuses as they are not the focus of this paper. From this table, we have following findings.
First, API call and Replaced API misuses are more prevalent. From the table, the percentages of operations about API call and Replaced API are more than 70%. Particularly, 38.33% misuses are about Replaced API, which are more than any other types of API misuses. By analyzing the data, we find that one important reason for such a large portion of Replaced API misuses is that most of APIs share similar signatures when their functionalities are close. Therefore, if developers do not well understand the difference between APIs, they tend to be confused and use a wrong API. For example, when one wants to only get the miliseconds of current time, System. currentTimeMillis() is a preferable API with high efficiency. However, developers tend to misuse the API new Date(). getTime(), which is simply a wrapper of the former. Due to the new Date object, the latter API may cause performance issues, especially when it is intensively used in time-critical programs. Therefore, if possible, it is better to directly use the API System.currentTimeMillis() to speed up the underlying system. The result demonstrates the importance of Replaced API misuses, and more research efforts are informed to be dedicated to detecting such misuses.
Second, missing API is more prevalent than redundant. The results show that developers tend to miss some API calls or handlers (such as Condition, Synchronization, and Exception) rather than writing redundant ones. For example, the percentage of missing API calls is 24.67%, almost three times higher than the opposite. Similarly, the percentage of missing condition is also much higher than that of redundant condition (11.02% vs 1.53%).
Finding 1: (1) API call and Replaced API are the most prevalent API misuses, and Replaced API misuses accounts for the largest portion among all misuses, calling for new detection approaches. ( 2) Developers tend to miss some components to satisfy the constraint of a certain API.
In Section IV-A, we have performed quantitative analysis on API misuses. In this research question, we further qualitatively analyze the misuse patterns mined from the studied dataset. We first extract a ranked list for each misuse category according to the cross-project frequencies. The reason we consider crossproject frequencies is that the mined patterns should be more helpful in detecting unknown misuses if they widely exist in more various projects. We remove patterns related to printing and logging because they are usually for debugging and maintenance purposes; we also remove APIs with "Android" and "Javax" since we target general Java programs. Table II presents the popular misuse patterns for each general category. Column 1 denotes the category names. Column 2 presents the top-5 popular patterns for each category. Column 3 and 4 show the number of projects in which the corresponding pattern appeared and the total number of pattern occurrences, respectively. From this table we have following findings.
First, API misuses related to class java.lang.String account for the vast majority of all misuse patterns. For example, 18 out of 45 misuses in all categories are from the class of String. Also, for both missing API and missing condition misuses, all top-5 API patterns are related to String . Specifically, we have the following observations. (1) For the missing condition, most patterns miss checking if the index of a substring or a character inside a String is valid, such as String.charAt(int). (2) There are various ways to fix bugs related to API misuses. For example, to deal with potential bugs of String.lastIndexOf(String), inserting either condition or exception handling is reasonable in historical bugfixing dataset. The findings show the importance of String and guide developers how to detect and fix misuses related to String by mining bug-fixing dataset.
Second, most Replaced API misuse patterns are related to the names and arguments in the same class, even though the Replaced Receiver is the majority in Table I. For example, developers tend to misunderstand between Integer.valueOf (String) and Integer.parseInt(String), where the former returns an Integer object while the latter returns a primitive int value.
Third, it is possible to design an automated technique based on the mined patterns to detect unknown misuses in other projects. For example, there is a misuse pattern File.mkdir()=>File.mkdirs() in the ranking list. We have detected such misuses existing in Apache projects, and one submitted pull request has been accepted by developers, shown in Figure 7. In fact, there are many valuable patterns in the ranking list and we will introduce how we are inspired to improve misuse detection in the following two sections.
In this section, we present the potential recall of misuse detection on the recently widely used benchmark MUBENCH [1], [12] with the patterns mined from our dataset. We manually analyze the fix patterns in the MUBENCH and then check whether the same patterns exist in the studied dataset. We assume that an ideal detection approach can accurately mine API-misuse patterns from historical fixes if at least one fix instance exists in the dataset. In addition, to explore the complementariness to existing approaches, we also include the results of state-of-the-art misuse detection approaches, including MutAPI [15], DMMC [2], Jadet [8], Tikanga [51] and GrouMiner [47].
Figure 5 presents the overlaps of misuses detected by different approaches, where "This work" denotes the results mined from our dataset. As a result, the 53 The former API will throw JSONException when the JSONObject does not has the query attribute (i.e., the given argument), which may crash the program if it is not handled. Therefore, to detect such misuses, we search the usages of the API where the exception is not properly tackled. Figure 6 shows one accepted misuse in project Apache Unomi. In this example, the queried keys may not exist and exceptions will be thrown and crash the program. As a result, they are immediately confirmed and fixed after reporting to maintainers. getJSONArray(String) and getJSONObject(String) are similar.
File.mkdir()=>File.mkdirs(). Both File.mkdir() and File.mkdirs() are used to create a directory and return a boolean value to indicate whether the creation succeeds or not. The difference is the latter can recursively create the directories when nested paths do not exist, while mkdir cannot. The failure of directory creation may cause file access errors during program running, and is hard to debug. Therefore, when may be NULL so that the NullPointerException would be thrown. We detect this kind of misuses by checking the possibility of causing NullPointerException. That is, there is no guard condition to check the nullness of the object before using.
As discussed above, a lot of submitted misuses have been accepted by maintainers. However, there are 15 misuses are rejected. We investigate these misuses and find some major reasons as follows. (1) There are no performance differences between two APIs in a sample or small project. For example, in project Apache CXF, the project maintainer rejected our submitted misuse by claiming that System.currentTimeMillis() and new Date().getTime() would not make the difference since this case occurs in a sample (small) project under Apache CXF. In Apache NetBeans, the project maintainer doubts that the change File.mkdir()=>File.mkdirs() is just a theoretical problem so they reject our pull request. (2) The submitted cases will change the code style of the entire project. For example, in project Apache NetBeans, the project maintainers claim that the change Date.getTime()=>System .currentTimeMillis() will reduce the readability since other Date cases can not be changed due to the context. Finding 4: Based on the 10 Replaced misuse patterns, we have reported 149 misuses in latest Apache projects; 57 of them have been fixed by project maintainers so far.
The threats to external validity lie in the dataset used. To collect a large set of data for analysis, we mined bug-fixing commits from GitHub repositories. The dataset may be noisy for different reasons (i.e., not real bug fixes). We also define API misuses as code edit operations related to some APIs in a bug fix. In fact, it may also inaccurate results. The reasons are twofold. First, API misuses may occur in regular code changes, while we compute the percentage of API misuses over the operations from bug-fixing commits. Second, edit operations related to APIs may be not real API misuses.
The threats to internal validity relate to our implementation. To reduce errors, we use GumTree to extract AST operations, which is widely used in previous studies [30], [36], [37], [56]. However, we cannot get certain misuse categories, such as missing NULL checks directly from GumTree. Therefore, we revise GumTree by adding detailed program analysis to map operations to our classifications. To mitigate the threats of categorization noise from GumTree, we sample 100 operations for each category and confirm that 76% operations are correctly classified. Furthermore, we carefully review our code and scripts to ensure their correctness as much as we can.
In this paper, we conduct an extensive empirical study on API misuses based on 528,546 bug-fixing commits. We extract fine-grained edit operations on AST of source code and classify them into different categories of API misuses. We also extract various frequent API-misuse patterns based on the categories. The results show that API misuses are prominent in practice and provide a set of guidelines for future research. Finally, based on our dataset, we perform a user study to manually analyze the usage constraints of 10 patterns to explore whether the mined patterns can guide the design of future API-misuse detection tools. The results show that 57 misuses (out of 149 reported misuses) have been fixed, indicating the importance of historical API misuses and the promising future for API-misuse detection. However, the current implementation still depends on our handcrafted detection rules. Effective and automated rule mining techniques should be further explored, such as combining machine learning [52] to characterize more context features, etc.
All experimental data and source code are open-source that can be downloaded at: https://github.com/BID3/BID3.