skip to main content


Title: Successes, challenges, and rethinking – an industrial investigation on crowdsourced mobile application testing
The term crowdsourcing – a compound contraction of crowd and outsourcing – is a new paradigm for utilizing the power of crowds of people to facilitate large-scale tasks that are costly or time consuming with traditional methods. This paradigm offers mobile application companies the possibility to outsource their testing activities to crowdsourced testers (crowdtesters) who have various testing facilities and environments, as well as different levels of skills and expertise. With this so-called Crowdsourced Mobile Application Testing (CMAT), some of the well-recognized issues in testing mobile applications, such as multitude of mobile devices, fragmentation of device models, variety of OS versions, and omnifariousness of testing scenarios, could be mitigated. However, how effective is CMAT in practice? What are the challenges and issues presented by the process of applying CMAT? How can these issues and challenges be overcome and CMAT be improved? Although CMAT has attracted attention from both academia and industry, these questions have not been addressed or researched in depth based on a large-scale and real-life industrial study. Since June 2015, we have worked with Mooctest, Inc., a CMAT intermediary, on testing five real-life Android applications using their CMAT platform – Kikbug. Throughout the process, we have collected 1013 bug reports from 258 crowdtesters and found 247 bugs in total. This paper will present our industrial study thoroughly and give an insightful analysis to investigate the successes and challenges of applying CMAT.  more » « less
Award ID(s):
1822137
NSF-PAR ID:
10097670
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Empirical software engineering
Volume:
24
Issue:
2
ISSN:
1382-3256
Page Range / eLocation ID:
537–561
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Ever-increasing demands for energy, particularly being environmentally friendly have promoted the transition from fossil fuels to renewable energy.1Lithium-ion batteries (LIBs), arguably the most well-studied energy storage system, have dominated the energy market since their advent in the 1990s.2However, challenging issues regarding safety, supply of lithium, and high price of lithium resources limit the further advancement of LIBs for large-scale energy storage applications.3Therefore, attention is being concentrated on an alternative electrochemical energy storage device that features high safety, low cost, and long cycle life. Rechargeable aqueous zinc-ion batteries (ZIBs) is considered one of the most promising alternative energy storage systems due to the high theoretical energy and power densities where the multiple electrons (Zn2+) . In addition, aqueous ZIBs are safer due to non-flammable electrolyte (e.g., typically aqueous solution) and can be manufactured since they can be assembled in ambient air conditions.4As an essential component in aqueous Zn-based batteries, the Zn metal anode generally suffers from the growth of dendrites, which would affect battery performance in several ways. Second, the led by the loose structure of Zn dendrite may reduce the coulombic efficiency and shorten the battery lifespan.5

    Several approaches were suggested to improve the electrochemical stability of ZIBs, such as implementing an interfacial buffer layer that separates the active Zn from the bulk electrolyte.6However, the and thick thickness of the conventional Zn metal foils remain a critical challenge in this field, which may diminish the energy density of the battery drastically. According to a theretical calculation, the thickness of a Zn metal anode with an areal capacity of 1 mAh cm-2is about 1.7 μm. However, existing extrusion-based fabrication technologies are not capable of downscaling the thickness Zn metal foils below 20 μm.

    Herein, we demonstrate a thickness controllable coating approach to fabricate an ultrathin Zn metal anode as well as a thin dielectric oxide separator. First, a 1.7 μm Zn layer was uniformly thermally evaporated onto a Cu foil. Then, Al2O3, the separator was deposited through sputtering on the Zn layer to a thickness of 10 nm. The inert and high hardness Al2O3layer is expected to lower the polarization and restrain the growth of Zn dendrites. Atomic force microscopy was employed to evaluate the roughness of the surface of the deposited Zn and Al2O3/Zn anode structures. Long-term cycling stability was gauged under the symmetrical cells at 0.5 mA cm-2for 1 mAh cm-2. Then the fabricated Zn anode was paired with MnO2as a full cell for further electrochemical performance testing. To investigate the evolution of the interface between the Zn anode and the electrolyte, a home-developed in-situ optical observation battery cage was employed to record and compare the process of Zn deposition on the anodes of the Al2O3/Zn (demonstrated in this study) and the procured thick Zn anode. The surface morphology of the two Zn anodes after circulation was characterized and compared through scanning electron microscopy. The tunable ultrathin Zn metal anode with enhanced anode stability provides a pathway for future high-energy-density Zn-ion batteries.

    Obama, B., The irreversible momentum of clean energy.Science2017,355(6321), 126-129.

    Goodenough, J. B.; Park, K. S., The Li-ion rechargeable battery: a perspective.J Am Chem Soc2013,135(4), 1167-76.

    Li, C.; Xie, X.; Liang, S.; Zhou, J., Issues and Future Perspective on Zinc Metal Anode for Rechargeable Aqueous Zinc‐ion Batteries.Energy & Environmental Materials2020,3(2), 146-159.

    Jia, H.; Wang, Z.; Tawiah, B.; Wang, Y.; Chan, C.-Y.; Fei, B.; Pan, F., Recent advances in zinc anodes for high-performance aqueous Zn-ion batteries.Nano Energy2020,70.

    Yang, J.; Yin, B.; Sun, Y.; Pan, H.; Sun, W.; Jia, B.; Zhang, S.; Ma, T., Zinc Anode for Mild Aqueous Zinc-Ion Batteries: Challenges, Strategies, and Perspectives.Nanomicro Lett2022,14(1), 42.

    Yang, Q.; Li, Q.; Liu, Z.; Wang, D.; Guo, Y.; Li, X.; Tang, Y.; Li, H.; Dong, B.; Zhi, C., Dendrites in Zn-Based Batteries.Adv Mater2020,32(48), e2001854.

    Acknowledgment

    This work was partially supported by the U.S. National Science Foundation (NSF) Award No. ECCS-1931088. S.L. and H.W.S. acknowledge the support from the Improvement of Measurement Standards and Technology for Mechanical Metrology (Grant No. 22011044) by KRISS.

    Figure 1

     

    more » « less
  2. Recent publications have pointed out a number of challenges and challenges for when engineers validate mobile apps using a conventional way inside a testing laboratory. Top two issues include: a) higher test costs due to the diversity of mobile devices and platforms, b) difficulty in conducting large-sale user-oriented performance and usability testing. A new testing approach, known as crowdsourced testing, provides a promising way to address these challenges and issues. This paper provides a comprehensive tutorial on crowdsourced test services, and informative concepts, insights, and detailed discussion about common questions raised by engineers and managers. It presents a clear comparative view between mobile crowdsourced testing with traditional lab-based mobile testing. In addition, it also summarizes and compares different major players, their commercial products, and solutions in mobile crowdsourced test services. Furthermore, it examines the major issues, challenges, and needs. 
    more » « less
  3. Abstract Why the new findings matter

    The process of teaching and learning is complex, multifaceted and dynamic. This paper contributes a seminal resource to highlight the digitisation of the educational sciences by demonstrating how new machine learning methods can be effectively and reliably used in research, education and practical application.

    Implications for educational researchers and policy makers

    The progressing digitisation of societies around the globe and the impact of the SARS‐COV‐2 pandemic have highlighted the vulnerabilities and shortcomings of educational systems. These developments have shown the necessity to provide effective educational processes that can support sometimes overwhelmed teachers to digitally impart knowledge on the plan of many governments and policy makers. Educational scientists, corporate partners and stakeholders can make use of machine learning techniques to develop advanced, scalable educational processes that account for individual needs of learners and that can complement and support existing learning infrastructure. The proper use of machine learning methods can contribute essential applications to the educational sciences, such as (semi‐)automated assessments, algorithmic‐grading, personalised feedback and adaptive learning approaches. However, these promises are strongly tied to an at least basic understanding of the concepts of machine learning and a degree of data literacy, which has to become the standard in education and the educational sciences.

    Demonstrating both the promises and the challenges that are inherent to the collection and the analysis of large educational data with machine learning, this paper covers the essential topics that their application requires and provides easy‐to‐follow resources and code to facilitate the process of adoption.

     
    more » « less
  4. No skill is more important for a student of mechanics than the ability to draw a complete and accurate free-body diagram (FBD). A good FBD facilitates proper accounting of forces when writing the balances that lead to governing equations in statics, solid mechanics, and dynamics. Because this skill is essential, educational approaches that improve the ability of students to draw correct FBDs are critical for maximizing the potential of the next generation of engineers. Traditionally, learning to draw FBDs involves classroom instruction followed by homework practice consisting of problems drawn from a textbook. Homework as practice does not serve all students well, because it does not scaffold the process of drawing FBDs in terms of distinct tasks (e.g., isolating the body, considering support reactions) nor does it offer immediate feedback, which students often need to avoid falling into the same error repeatedly. To address these shortcomings, we embarked on the design, implementation, and testing of a mobile application (app) that offers an alternative venue for FBD practice. The app provides students with asynchronous opportunities for training, varied tasks that target specific FBD issues, and several levels of immediate feedback. We hypothesize that the gamified environment and puzzle-based gameplay will improve student skill and self-efficacy in drawing FBDs, particularly for women, who may feel less confident in their spatial skills. Data collected to describe student experiences may also provide additional insight into how to improve FBD instruction generally. In this paper, we detail the process for designing and implementing the app and provide initial data regarding student impressions and use. The app was piloted in Fall 2022 in a large Introduction to Statics course as a non-graded study activity; all students except one (n=97) participated in an evaluation of its design features and user experiences. Approximately half (54%) of students indicated they had played half or more of the available games. When commenting about how the FBD app did, or did not, help their learning, 49% of respondents appreciated that the app allowed additional opportunities for practice. Students used these opportunities to further develop several skills, such as visualizing the system and setting up accurate diagrams, which strengthened their confidence and reviewed key concepts. While describing the value of practicing through the app, 21% of students called out how the app provided feedback. They specifically mentioned the positive experiences of receiving feedback that is immediate, that explains boundary connections, and that deepens learning after mistakes are made. These and other findings from the pilot study are discussed with corresponding next steps for development. 
    more » « less
  5. No skill is more important for a student of mechanics than the ability to draw a complete and accurate free-body diagram (FBD). A good FBD facilitates proper accounting of forces when writing the balances that lead to governing equations in statics, solid mechanics, and dynamics. Because this skill is essential, educational approaches that improve the ability of students to draw correct FBDs are critical for maximizing the potential of the next generation of engineers. Traditionally, learning to draw FBDs involves classroom instruction followed by homework practice consisting of problems drawn from a textbook. Homework as practice does not serve all students well, because it does not scaffold the process of drawing FBDs in terms of distinct tasks (e.g., isolating the body, considering support reactions) nor does it offer immediate feedback, which students often need to avoid falling into the same error repeatedly. To address these shortcomings, we embarked on the design, implementation, and testing of a mobile application (app) that offers an alternative venue for FBD practice. The app provides students with asynchronous opportunities for training, varied tasks that target specific FBD issues, and several levels of immediate feedback. We hypothesize that the gamified environment and puzzle-based gameplay will improve student skill and self-efficacy in drawing FBDs, particularly for women, who may feel less confident in their spatial skills. Data collected to describe student experiences may also provide additional insight into how to improve FBD instruction generally. In this paper, we detail the process for designing and implementing the app and provide initial data regarding student impressions and use. The app was piloted in Fall 2022 in a large Introduction to Statics course as a non-graded study activity; all students except one (n=97) participated in an evaluation of its design features and user experiences. Approximately half (54%) of students indicated they had played half or more of the available games. When commenting about how the FBD app did, or did not, help their learning, 49% of respondents appreciated that the app allowed additional opportunities for practice. Students used these opportunities to further develop several skills, such as visualizing the system and setting up accurate diagrams, which strengthened their confidence and reviewed key concepts. While describing the value of practicing through the app, 21% of students called out how the app provided feedback. They specifically mentioned the positive experiences of receiving feedback that is immediate, that explains boundary connections, and that deepens learning after mistakes are made. These and other findings from the pilot study are discussed with corresponding next steps for development.   
    more » « less